<figure><img src="image_1.png" /></figure>

HAL
open science

# Business Process Model Similarity as a Proxy for Group Consensus

Peter Rittgen

<figure><img src="image_2.png" /></figure>

## To cite this version:

Peter Rittgen. Business Process Model Similarity as a Proxy for Group Consensus. 4th Practice of Enterprise Modeling (PoEM), Nov 2011, Oslo, Norway. pp.12-24, 10.1007/978-3-642-24849-8_2. hal-01572395

HAL Id: hal-01572395

https://inria.hal.science/hal-01572395

Submitted on 7 Aug 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Business Process Model Similarity as a Proxy for Group Consensus

Peter Rittgen

Vlerick Leuven Gent Management School, Reep1, 9000 Gent, Belgium, and University of
Borås, 501 90 Borås, Sweden
peter.rittgen@vlerick.com

**Abstract.** Consensus is an important measure for the success of any business process modeling effort. Although intensively studied in the general literature on group processes, consensus has hardly been considered in business process modeling and never seriously measured. We define consensus as the level of agreement of group members’ views on the process and introduce business process similarity as a proxy. We validate the measure by comparing it to an existing self-reported measure of consensus.

**Keywords.** Business process modeling, model similarity, group consensus, mental model, view, visualization

## 1 Introduction

The literature on business process modeling is vast and the importance of measuring the success of process modeling projects and sessions has been widely recognized [1-6]. But prevalent success measures for individual modeling sessions primarily involve some form of model quality measure [7-10]. While it is undisputed that the quality of a business process model is relevant to modeling success it is not the only and perhaps not even the most important success factor.

The reason for this is twofold: the process model itself is a social construction, and its purpose is again to support some social process, e.g. a change project or system development project. In other words: the model documents the results of one social process (modeling) and serves as a point of departure for another one.

If the model were to be processed by a computer its quality would be of prime importance to ensure correct interpretation by the machine. But the results that are documented in the model are primarily the mutual knowledge that has been developed in the modeling session, the conflicts that had to be solved on the way, and the consensus that has been achieved among the group members as a result.

It is precisely this consensus that is a prerequisite for people's commitment to the ensuing change project, for example. Often a poor model with high consensus goes further than a good model with little consensus. Hence consensus is a major result that needs to be achieved in business process modeling sessions much like in many other forms of group work.
But while there is considerable research on consensus in other areas [11-13] the topic received little attention in business process modeling with researchers barely mentioning the issue [14-17] and, to the best of our knowledge, not researching it in a systematic way, let alone measuring consensus.

The purpose of this paper is to develop such a measure. To do so we first define the concept of consensus in the next section, Group consensus in process modeling. For this purpose we rely on cognitive theories of modeling.

Based on the cognitive concept of a view and the model as its externalization we can interpret consensus as “view agreement” and hence as “model similarity”. The section Business process model similarity therefore introduces a measure for the latter.

A proper evaluation of a new measure typically establishes validity by comparison to an existing measure of the same concept. The section Other group consensus measures therefore introduces an established measure for group consensus. The actual validation of the new measure was done in field experiments. The set-up of these experiments is described in the section Comparing model similarity and consensus in field experiments.

The section Data analysis reports on the analysis of the data that we collected in the experiments. The results and implications of this analysis are treated in the section Discussion. The paper concludes with a summary of the findings and an outlook on future work.

## 2 Group consensus in process modeling

Group modeling is a cognitive as well as a group process. We therefore define group consensus in business process modeling as the extent to which the group members' views on the process agree with each other. The problem with this definition is that the views that are entertained by the group members are not directly accessible so the measure of consensus needs to be based on some external representation of these views.

For this purpose we need to resort to the cognitive theory on the modeling process. The foundations for our understanding of model cognition were laid by Johnson-Laird [18, 19] who introduced the idea of so-called mental models that the mind constructs when it imagines a situation. A mental model consists of a mental system of relations that has a structure similar to the system that is imagined.

When the mind engages in the process of deduction it performs the following three steps: comprehension, description, and validation. [20] found that the individual part of the modeling process can be described well in terms of Johnson-Laird's deduction. For our purposes the relevant step is that of description, which proceeds as follows:

1. Build *mental model*

2. Extract *view* from mental model

3. Transcribe view to *visualization*

4. Conduct within-model testing

5. If any test fails, go back to step 2 or 3, possibly modifying the mental model
In short: *mental models* are stored in long-term memory but not directly accessible; to work on them the mind needs to create *views* in working memory that correspond to these mental models. If the work on the views becomes too complex, which is often the case for modeling, external representations have to be created in order to extend the internal working memory. These external representations are called *visualizations* and in business process modeling they typically take the form of process diagrams.

We now come back to the original problem: if group consensus in modeling is the mutual agreement of the views (and thereby of the underlying mental models), then consensus can be measured also in terms of the agreement of the visualizations of the views, which correspond to the views.

So comparing the views of the group members can be replaced by comparing the externalizations of these views, i.e. the business process models. If we want to assess the degree to which two people agree on their views on a process we need to assess how similar the business process models are that visualize their views.

## 3 Business process model similarity

Measures for business process model similarity have been investigated thoroughly [21-26]. A comprehensive overview can be found in [24]. We use the measure suggested in [24] because it covers most prevalent modeling languages as well as all aspects of model similarity that have been introduced so far in the literature: node matching, structural and behavioral similarity.

The selected measure works for the Business Process Modeling Notation (BPMN), the language which was used by the organizations where we conducted the study. In the following we describe the 3 aspects of model similarity.

*Node matching* tries to map nodes from the one model to nodes of the other model by comparing the labels, attributes and types of nodes. Types in BPMN are task, start event, end event, parallel gateway, exclusive gateway and so on. Attributes are the swim lanes where the nodes are located, for example, and labels are the names of e.g. activities ("Send invoice").

Node matching can be effected with semantic or syntactic measures. The latter is based on the string-edit distance, i.e. the number of letters that need to be added, replaced or deleted to transform the label of an activity in one model to that of an activity in the other model. It is useful in the case of spelling mistakes as it is able to identify two words where one is spelled wrong (e.g. confirmation and confrimation), whereas semantic measures consider that as a 0% match. But syntactic measures will also (wrongly) identify words with similar spellings but different meanings, such as e.g. plane and plate.

To avoid such misclassifications we have decided to drop syntactic measures in favor of semantic matching. To ensure that wrong spellings do not introduce faulty mismatches we have spell-checked all models manually.

Semantic matching is based on a database of synonyms. We have used Wordnet for this purpose which is freely available online and contains 147,278 unique words. It will identify labels and attributes with similar meanings (e.g. "Send invoice" and
"Send bill"), which is exactly our purpose as participants in modeling sessions often phrase the same activity in different terms.

Based on the node matching the two graphs can be compared now with the help of structural or behavioral similarity. The former uses only structural information on the graph, i.e. the way in which activities are connected with "arrows" but does not look at their meaning in terms of control flow. Two models are considered structurally equivalent if two nodes are always connected in the same way in one model as their matching counterparts in the other.

Behavioral equivalence looks at the actual execution of the processes described by the models, e.g. bisimulation equivalence. Here two models are considered equivalent if, at any time during process execution, an activity that can be performed in one process can also be performed in the other, and vice versa.

One would expect the latter measure to give a more realistic and reliable account of the similarity of process models as it considers actual behavior. But [24] found that the structural measure performs equally well, i.e. it correctly identifies behaviorally similar and dissimilar processes without looking at behavior.

This is perhaps not surprising as the behavior of a process is largely determined by the structure of the control flow graph and the semantics of the gateways whose similarity is checked on the node level already, and both are accounted for in the structural measure, too.

As structural similarity is equally accurate but computationally more advantageous we have used it in our study. It is based on the concept of graph-edit distance, i.e. the number of nodes and edges that need to be added, replaced, or deleted in order to turn one model into the other.

The overall similarity measure is the inverse of the number of edit steps in relation to the graph size where nodes are matched by type and semantic similarity of their labels and attributes. It is a number that ranges from 0 to 1 where 0 refers to two process models that have nothing in common and 1 to a perfect match.

More details on the used algorithms can be found in the section Comparing model similarity and consensus in field experiments.

## 4 Other group consensus measures

Group consensus is considered as an important outcome of a group process and as a consequence there is a large body of literature that provides measurements for it. The majority of these publications are in the area of decision-making where consensus is usually defined as the mutual agreement of decision-makers' preferences concerning a number of decision alternatives. Preferences can be binary, i.e. an alternative is either preferred or not, or linguistic (qualitative), i.e. the preference values are labeled "none", "very low", "low", "medium", "high", "very high", and "perfect". In addition the membership in a category can be strict, i.e. in exactly one category, or fuzzy, i.e. to a certain extent [0...1] in several categories.

It is common to all these approaches that they require both explicit alternatives and preferences. These measures do therefore not easily carry over to modeling where the decision alternatives are rather implicit.
In our search for a suitable and established measure of consensus in business
process modeling we have therefore turned our attention to another stream of research
where group consensus is viewed in terms of solution satisfaction [27-32].

The purpose of business process modeling is to find a solution either to the
problem of accurately representing an existing process, the knowledge about which is
distributed over a number of domain experts (AS-IS modeling), or to the problem of
creating a new process that overcomes problems of the old one or provides new
products or services (TO-BE modeling).

A modeling group exhibits consensus if they are satisfied with the solution that was generated in the modeling session. Solution satisfaction is therefore a suitable measure of group consensus in the context of business process modeling.

We used the measure described in [33] for group decision processes that uses five
items rated on a 5-point Likert scale with the anchors "Not at all", "To a little extent",
"To some extent", "To a great extent", and "To a very great extent". The items run:

1. How satisfied are you with the quality of your group's solution?

2. To what extent does the final solution reflect your inputs?

3. To what extent do you feel committed to the group solution?

4. To what extent are you confident that the group solution is correct?

5. To what extent do you feel personally responsible for the correctness of the group solution?

Using a measure for group decision processes in modeling is appropriate because
the group modeling process can be conceptualized as a process where decisions on the
development of the model are negotiated [34]. Moreover, the measure has been
validated by its authors.

The group consensus measure is also a self-reported measure and therefore suited for comparison with the model similarity measure, which is based on observation. This means that we can compare the agreement as perceived by the participants with the actual agreements of their views as “objectively” measured according to the previous section.

5 Comparing model similarity and consensus in field experiments

We set up field experiments to test the validity of business process model similarity as a proxy for group consensus in process modeling. For this purpose we asked members of process modeling groups to draw up a model of the business process **after** the modeling session.

Participants had to prepare the after models without access to the group model. In
half of the cases the after model was done immediately after the session, in the other
half one week later to control for group model bias (see section Data analysis).

We also varied the order of drawing the after models and applying the consensus measure. In both cases, immediate and one week later, half of the participants were drawing the models after and half of them before the consensus measure was applied. Again a more detailed discussion can be found in the Data analysis section.
Each after model was compared to the group model by computing the business process model similarity measure introduced in section Group consensus in process modeling. The formulae we used are (all from [24]):

* Definition 14 (Graph-edit distance similarity)

* Definition 13 (Graph-edit distance)

* Definition 12 (Node matching similarity)

* Definition 7 (Attribute similarity, *Simattr*)

* Definition 6 (Semantic similarity, *Simsem*)

The similarity of two nodes $n_1$ and $n_2$ is defined as ($\tau$ = type of node):

$$ Sim(n_1, n_2) = \begin{cases} 0, & \tau_1 \neq \tau_2 \\ \frac{1}{2} \cdot Simsem(n_1, n_2) + \frac{1}{2} \cdot Simattr(n_1, n_2), & \tau_1 = \tau_2 \end{cases} $$

We also asked participants to fill in the questionnaire related to group consensus. Participants had to rate the five questions with respect to the final model that was adopted by the group. For this purpose they were shown the group model to facilitate the answering of the questions.

We conducted three half-day modeling workshops at each of the following five organizations: the insurance branch of a large bank, an IT solution provider in the automobile industry, a psychiatric hospital, a bio-engineering laboratory, and a city administration. All workshops revolved around the development of a TO-BE model for a business process or a substantial part of it.

In total we conducted 15 modeling sessions with 6-12 group members and a total of 122 participants.

## 6 Data analysis

In order to assess whether business process model similarity is an acceptable proxy for consensus on the group model we compared the consensus measure to the model similarity measure. We assumed that agreement with the group model is a result of the agreement of an individual's view on the process with the group process model and that the individual view is accurately represented in the externalization of this view as manifested in the visualization created by this individual (the after model).

If this assumption is true then we should see a high correlation between the items C1 to C5 of the self-reported consensus measure and the similarity degree objectively measured between the after model and the group model.

We have first subjected the data to a Kolmogorov-Smirnov test which was significant on the 1% level for all six variables (C1-C5 and Similarity). We can therefore assume a normal distribution of the data and make use of parametric correlation analysis based on bivariate Pearson correlation coefficients.

The results of the correlation analysis are shown in Table 1. All correlations are significant on the 1% level.
<table><caption>**Table 1.** Pearson correlations for consensus items and business process model similarity</caption><thead><tr><th rowspan="2"></th><th colspan="5">Consensus</th></tr><tr><th>C1</th><th>C2</th><th>C3</th><th>C4</th><th>C5</th></tr></thead><tbody><tr><th>Similarity</th><td>0.822</td><td>0.720</td><td>0.734</td><td>0.777</td><td>0.758</td></tr></tbody></table>

On a side note it should be mentioned that the inter-item correlations are also both significant and strong for all item pairs which provides some further validation of the consensus instrument.

The coefficients are reasonably high which confirms that the self-reported level of consensus (individual agreement with the group result) is indeed closely linked to the agreement of the individual view on the process and the group view as measured by the similarity between the individual model and the group model.

A potential threat to validity might be that the knowledge about the group model might have influenced people in drawing the after models. This phenomenon is called “group thinking” and people exhibit this behavior to conform to group pressure either knowingly or unknowingly.

We took three measures to counteract this possible bias. First we told participants explicitly that the model they draw should reflect their own ideas about a good TO-BE model. We warned them not to try and copy the group model, or what they remembered of it.

A second measure was that we asked people in half of the sessions to draw their diagram a week after the session so that the chances of remembering the group model were quite low. Table 2 shows the correlations and Table 3 the averages and standard deviations in both cases.

<table><caption>**Table 2.** Correlations of similarity with consensus items after session and 1 week later</caption><thead><tr><th>Time of drawing</th><th>C1</th><th>C2</th><th>C3</th><th>C4</th><th>C5</th></tr></thead><tbody><tr><th>After session</th><td>0.835</td><td>0.762</td><td>0.715</td><td>0.775</td><td>0.775</td></tr><tr><th>1 week later</th><td>0.793</td><td>0.650</td><td>0.751</td><td>0.753</td><td>0.752</td></tr></tbody></table>

<table><caption>**Table 3.** Averages and standard deviations after session and 1 week later</caption><thead><tr><th rowspan="2">Item</th><th colspan="2">After session</th><th colspan="2">1 week later</th></tr><tr><th>Mean</th><th>Std. deviation</th><th>Mean</th><th>Std. deviation</th></tr></thead><tbody><tr><td>C1</td><td>3.31</td><td>1.05</td><td>2.90</td><td>1.13</td></tr><tr><td>C2</td><td>3.19</td><td>1.03</td><td>2.83</td><td>1.07</td></tr><tr><td>C3</td><td>3.06</td><td>1.09</td><td>2.73</td><td>1.28</td></tr><tr><td>C4</td><td>3.16</td><td>1.00</td><td>2.70</td><td>1.16</td></tr><tr><td>C5</td><td>3.18</td><td>1.14</td><td>2.95</td><td>1.03</td></tr><tr><td>Similarity</td><td>0.55</td><td>0.22</td><td>0.43</td><td>0.34</td></tr></tbody></table>

The correlations are slightly higher for all items except C3 when measurements are made immediately after the sessions. This means that the link between similarity and
consensus is stronger in the beginning and weakens over time. An explanation can be found in the increased standard deviations, especially in the case of similarity, that indicate a greater variety in the data that was measured after one week. This shows that time did indeed have an impact on memory as the drawing used for the similarity measure requires an accurate recall of the view on the business process.

The consensus values have decreased by only 11% on average but the similarity values have gone down by 22%. This means that people can apparently remember the level of agreement that was achieved better than the details of the models, both their own and that of the group.

Another bias could be the fact that the group model is shown to the participants to facilitate the answering of the consensus questions. If the drawing is done after the consensus measure is applied, they might use parts of the recently seen group model in their own drawing.

To see whether this is the case we have let people do the drawing both before and after the consensus measure. This is the third measure we used to control for group thinking bias. Table 4 shows the results.

<table><caption>**Table 4.** Average consensus and similarity depending on order of measure application</caption><thead><tr><th>Consensus measure</th><th>Average consensus</th><th>Average similarity</th></tr></thead><tbody><tr><td>After drawing</td><td>2.8</td><td>0.49</td></tr><tr><td>Before drawing</td><td>3.2</td><td>0.49</td></tr></tbody></table>

Group thinking is clearly not at play here as it would have increased the similarity values in the groups that did the consensus measure before the drawing because they might have used some information from the group model shown during questioning in the drawing.

On the other side, the consensus values are lower in the groups that answered the consensus questions after the drawing. This is probably due to the fact that they had better knowledge of their own view after expressing it explicitly in a model and could therefore remember more distinctions between their own view and the group model.

## 7 Discussion

The fact that there is a strong correlation between the similarity measure and all items of the consensus measure shows that the perceived agreement with the group model is closely related to the degree in which the group model coincides with the individual perception of the process.

Note that all process models were TO-BE models so agreement is much harder to achieve than in AS-IS modeling because different personal interests are at stake. As a consequence the average consensus value was not very high after only one half-day session with an average 3 on the 1-5 Likert scale and 0.49 on model similarity.
But nevertheless the correlation was strong both in the case of little consensus and in the case of good consensus. The maximum consensus was a 4.6 on the Likert scale and 0.89 on similarity, the minimum 1.6 and 0.08, respectively.

As a consequence of our results business process model similarity is a valid proxy for measuring individual agreement with the group result in business modeling of processes. The similarity measure can also be extended to capture the overall group consensus.

For this purpose it is useful to know that model similarity is actually measured as the inverse of model distance, i.e. the effort required to turn one model into the other. If we have n models that reflect the views of n group members we first compute all pairwise distances and then determine the model $M_j$ with the lowest sum of incident distances. The effort to turn all other models into this one is the lowest possible effort to make all models agree. It can be calculated as:

$$ dist(M_1, \dots, M_n) = \frac{\sum_{i, i \neq j} dist(M_i, M_j)}{(n-1) \cdot size(M_j) + \sum_{i, i \neq j} size(M_i)} $$

The similarity is then just the distance subtracted from one. This measure delivers a number between zero and one for any number of models and can be seen as the overall similarity between them, i.e. the inverse of the minimum effort required to make all the models equal.

This figure also tells us how far or close the individual views are from each other and is therefore a measure of the group consensus. Please observe that this measure does not require a group model and can therefore be used at any stage of modeling. This is important as consensus can otherwise only be measured after a group model is available.

To use model similarity as a proxy for consensus also has the advantage that it can be measured in a much more objective manner. Other consensus measures have to rely on self-reported perceptions whose measurement is notoriously subjective and inexact.

Model similarity, on the other hand, is an exact measure that accurately portrays in how far the view of an individual as expressed in the individual model, really agrees with the group model, or in how far the group members' views agree with each other. The only drawback is that the drawing of a model takes more time than the answering of a few questions.

But this disadvantage becomes obsolete when modeling is done in a collaborative way where individual models are produced as a by-product. This form of modeling is increasingly becoming the rule as can be seen in the growing number of publications that advocate collaborative modeling, participatory modeling, or end-user modeling [35-38]. In these scenarios the participants in group modeling take on an active role in model creation.

They draw models themselves, e.g. with the help of a computerized tool and the models, or rather proposals are then viewed and commented by other group members which leads to revisions of these proposals and hence a second round of drawing. This
process is not only a revision process but also a collective learning process [39] where
participants learn to understand others' views and adapt their own views accordingly.

But beyond this modeling is also a negotiation process [34] where individuals have to make concessions to move towards an agreement. In collaborative modeling they can do so by incorporating features that are of interest to others into their proposals thereby making them more attractive and acceptable for others.

Over time initially different proposals will converge and a consensus group model will emerge. In the meantime the similarity between the proposals at each stage of model development will give us a precise understanding of the level of consensus that has already been achieved without the necessity for instruments that require additional inputs from the participants and that rely on perceptions only.

Each modeling session can be evaluated by comparing the similarity of models
before and after the session to see whether consensus has been increased during the
session. Model similarity can also be used as a tool of modeling project management
by specifying a minimum consensus level that is required. A modeling project is only
considered closed after this level has been reached. This makes sense because the
success of follow-up projects often depends on the achievement of a satisfactory level
of agreement in modeling.

If a session cannot increase consensus the facilitator can use conflict resolution techniques to resolve the issues before proceeding with the constructive part of the modeling session. Similarity measures can hence also be used to detect situations of serious conflict and guide session management.

8 Conclusion and Outlook

We have suggested the use of business process model similarity as a proxy for group
consensus in business process modeling. We have shown that this proxy is a reliable
measure of group consensus as compared to an established self-reported consensus
measure. The proxy is also a more objective and accurate measure and has therefore
the potential to outperform self-reported measures because it eliminates the group
thinking bias usually associated with self-reported measures of group performance. It
does also not rely on the existence of a group model and can therefore be used at any
stage in the modeling process.

The scenarios for using this proxy are manifold and the Discussion section has shown some of them: evaluation of modeling session performance, modeling project management, conflict detection, and so on. But the real advantage lies in integrating this measure into new methods for collaborative business process modeling where the progress towards a consensus model can actually be measured and controlled.

This will allow us to organize modeling in such a way that the steps in the method really lead to an improvement in consensus so that the success of modeling sessions can actually be planned. Because of the collaborative nature of these sessions, the individual group members, or perhaps small teams, will generate model proposals anyway so that additional drawing of models (as in our experiments) will no longer be necessary.
A continuous assessment of the status quo can therefore be made at any time in the modeling project as the computation of model similarity can be done automatically once the views of individuals or teams are available as models. An evaluation of the similarity between views on the business process can hence be the driver for the whole modeling effort.

This opens up possibilities for developing a new range of consensus-driven, or consensus-oriented business process modeling methods and supporting tools for the creation, maintenance, review, revision, and integration of model proposals and their convergence to a consensus model for the group.

Beyond this the further evolution of the model after implementation can also be supported in a consensus-oriented and decentralized way.

While all this is still hypothetical the assessment and control of consensus in business process modeling is a relevant issue already today. We believe that the measurement of consensus in an objective way is an important key to solving the consensus-related issues in process modeling.

## References

1. Dennis, A.R., Carte, T.A., Kelly, G.G.: Breaking the rules: success and failure in groupware-supported business process reengineering. Decision Support Systems 36, 31-47 (2003)

2. Lu, R., Sadiq, S.: A Survey of Comparative Business Process Modeling Approaches. In: Abramowicz, W. (ed.) Business Information Systems, vol. 4439, pp. 82-94. Springer Berlin / Heidelberg (2007)

3. Rosemann, M.: Potential pitfalls of process modeling: part A. Business Process Management Journal 12, 249-254 (2006)

4. Sedera, W., Gable, G.G., Rosemann, M., Smyth, R.W.: A success model for business process modeling: findings from a multiple case study. Eighth Pacific Asia Conference on Information Systems. Association for Information System (2004)

5. Luo, W., Tung, Y.A.: A framework for selecting business process modeling methods. Industrial Management & Data Systems 99, 312-319 (1999)

6. Recker, J.C., Rosemann, M., Indulska, M., Green, P.: Business process modeling : a comparative analysis. Journal of the Association for Information Systems 10, 333-363 (2009)

7. Dean, D.L., Orwig, R.E., Lee, J.D., Vogel, D.R.: Modeling with a group modeling tool: Group support, model quality, and validation. Information systems: Collaboration technology, organizational systems and technology, Proceedings of the 27th Hawaii International Conference on System Sciences, vol. 4, pp. 214-223. IEEE Computer Society Press, Los Alamitos, CA (1994)

8. Mendling, J., Recker, J.: Extending the Discussion of Model Quality: Why Clarity and Completeness may not always be enough. In: 19th International Conference on Advanced Information Systems Engineering (CAiSE'07), pp. 109-121. (Year)

9. Moody, D.L., Sindre, G., Brasethvik, T., Sølvberg, A.: Evaluating the quality of information models: empirical analysis of a conceptual model quality framework.
In: Dillon, L., Tichy, W. (eds.) Proceedings of the International Conference on Software Engineering, ICSE 2003, Portland, USA, (2003)

10. Sánchez-González, L., García, F., Mendling, J., Ruiz, F., Piattini, M.: Prediction of Business Process Model Quality Based on Structural Metrics. In: Parsons, J., Saeki, M., Shoval, P., Woo, C., Wand, Y. (eds.) Conceptual Modeling – ER 2010, vol. 6412, pp. 458-463. Springer Berlin / Heidelberg (2010)

11. DeStephen, R., Hirokawa, R.Y.: Small Group Consensus. Small Group Research 19, 227-239 (1988)

12. Priem, R.L.: Top management team group factors, consensus, and firm performance. Strategic Management Journal 11, 469-478 (1990)

13. Yoo, Y., Alavi, M.: Media and Group Cohesion: Relative Influences on Social Presence, Task Participation, and Group Consensus. MIS Quarterly 25, 371-390 (2001)

14. Clegg, B.T.: Building a Holarchy Using Business Process-Oriented Holonic (PrOH) Modeling. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 37, 23-40 (2007)

15. Decker, B., Rech, J., Althoff, K.-D., Klotz, A., Leopold, E., Voss, A.: eParticipative Process Learning--process-oriented experience management and conflict solving. Data & Knowledge Engineering 52, 5-31 (2005)

16. Kumarapeli, P., De Lusignan, S., Ellis, T., Jones, B.: Using Unified Modelling Language (UML) as a process-modelling technique for clinical-research process improvement. Informatics for Health and Social Care 32, 51-64 (2007)

17. Rittgen, P.: Quality and perceived usefulness of process models. Proceedings of the 2010 ACM Symposium on Applied Computing. ACM, Sierre, Switzerland (2010)

18. Johnson-Laird, P.N.: Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Harvard University Press, Cambridge, MA (1983)

19. Johnson-Laird, P.N.: The Computer and the Mind. Harvard University Press, Cambridge, MA (1988)

20. Waisel, L.B., Wallace, W.A., Willemain, T.R.: Visualizing modeling heuristics: an exploratory study. Proceedings of the 20th international conference on Information Systems. Association for Information Systems, Charlotte, North Carolina, United States (1999)

21. Dijkman, R.: Diagnosing Differences between Business Process Models. In: Dumas, M., Reichert, M., Shan, M.-C. (eds.) Business Process Management, vol. 5240, pp. 261-277. Springer, Berlin / Heidelberg (2008)

22. Dijkman, R., Dumas, M., García-Bañuelos, L.: Graph Matching Algorithms for Business Process Model Similarity Search. In: Dayal, U., Eder, J., Koehler, J., Reijers, H. (eds.) Business Process Management, vol. 5701, pp. 48-63. Springer, Berlin / Heidelberg (2009)

23. Dijkman, R., Dumas, M., García-Bañuelos, L., Käärik, R.: Aligning Business Process Models. In: 2009 IEEE International Enterprise Distributed Object Computing Conference (EDOC 2009), pp. 45-53. (Year)

24. Dijkman, R., Dumas, M., van Dongen, B., Käärik, R., Mendling, J.: Similarity of business process models: Metrics and evaluation. Information Systems 36, 498-516 (2010)
25. Ehrig, M., Koschmider, A., Oberweis, A.: Measuring similarity between semantic business process models. Proceedings of the fourth Asia-Pacific conference on conceptual modelling - Volume 67. Australian Computer Society, Inc., Ballarat, Australia (2007)

26. van Dongen, B., Dijkman, R., Mendling, J.: Measuring Similarity between Business Process Models. In: Bellahsène, Z., Léonard, M. (eds.) Advanced Information Systems Engineering, vol. 5074, pp. 450-464. Springer, Berlin / Heidelberg (2008)

27. Dennis, A.R., Aronson, J.E., Heninger, W.G., Walker II, E.D.: Structuring Time and Task in Electronic Brainstorming. MIS Quarterly 23, 95-108 (1999)

28. Dennis, A.R., Hilmer, K.M., Taylor, N.J.: Information Exchange and Use in GSS and Verbal Group Decision-Making: Effects of Minority Influence. Journal of MIS 14, 61-88 (1997)

29. Dennis, A.R., Wixom, B.H.: Investigating the Moderators of the Group Support Systems Use with Meta-Analysis. Journal of MIS 18, 235-258 (2001)

30. Ocker, R., Fjermestad, J., Hiltz, S.R., Johnson, K.: Effects of Four Modes of Group Communication on Outcomes of Software Requirements Determination. Journal of Management Information Systems 15, 99-118 (1998)

31. Reinig, B.A.: Toward an Understanding of Satisfaction with the Process and Outcomes of Teamwork. Journal of Management Information Systems 19, 65-83 (2003)

32. Mennecke, B.E., Valacich, J.S.: Information Is What You Make of It: The Influence of Group History and Computer Support on Information Sharing, Decision Quality, and Member Perceptions. Journal of MIS 15, 173-197 (1998)

33. Green, S.G., Taber, T.D.: The effects of three social decision schemes on decision group process. Organizational Behavior and Human Performance 25, 97-106 (1980)

34. Rittgen, P.: Negotiating Models. In: Krogstie, J., Opdahl, A., Sindre, G. (eds.) Advanced Information Systems Engineering, 19th International Conference, CAiSE 2007, Trondheim, Norway, June 2007, Proceedings, LNCS 4495, pp. 561-573. Springer, Berlin (2007)

35. Dean, D.L., Orwig, R.E., Vogel, D.R.: Facilitation Methods for Collaborative Modeling Tools. Group Decision and Negotiation 9, 109-127 (2000)

36. Gjersvik, R., Krogstie, J., Følstad, A.: Participatory Development of Enterprise Process Models. In: Krogstie, J., Halpin, T.A., Siau, K. (eds.) Information Modeling Methods and Methodologies, pp. 195-215. IGI Global, Hershey, PA (2005)

37. Rittgen, P.: Collaborative Modeling - Roles, Activities and Team Organization. International Journal of Information System Modeling and Design 1, 1-19 (2010)

38. Ssebuggwawo, D., Hoppenbrouwers, S.J.B.A., Proper, H.A.: Assessing Collaborative Modeling Quality Based on Modeling Artifacts. Proceedings of the 3rd IFIP WG8.1 Working Conference on the Practice of Enterprise Modelling (PoEM2010). Springer, Berlin (2010)

39. Gronau, N., Müller, C., Uslar, M.: The KMDL Knowledge Management Approach: Integrating Knowledge Conversions and Business Process Modeling. In: Karagiannis, D., Reimer, U. (eds.) Practical Aspects of Knowledge Management, vol. 3336, pp. 1-10. Springer Berlin / Heidelberg (2004)