# Evaluating and Improving the Visualisation of CHOOSE, an Enterprise Architecture Approach for SMEs

Sarah Boone, Maxime Bernaert, Ben Roelens, Steven Mertens, Geert Poels

► To cite this version:

Sarah Boone, Maxime Bernaert, Ben Roelens, Steven Mertens, Geert Poels. Evaluating and Improving the Visualisation of CHOOSE, an Enterprise Architecture Approach for SMEs. 7th IFIP Working Conference on The Practice of Enterprise Modeling (PoEM), Nov 2014, Manchester, United Kingdom. pp.87-102, 10.1007/978-3-662-45501-2_7 . hal-01281991

HAL Id: hal-01281991

https://inria.hal.science/hal-01281991v1

Submitted on 3 Mar 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Evaluating and Improving the Visualisation of CHOOSE, an Enterprise Architecture Approach for SMEs

Sarah Boone, Maxime Bernaert, Ben Roelens, Steven Mertens and Geert Poels

Department of Management Information Systems and Operations Management
Faculty of Economics and Business Administration,
Ghent University, Tweekekenstraat 2, B-9000 Ghent, Belgium
{SarahM.Boone,Maxime.Bernaert,Ben.Roelens,
Steven.Mertens,Geert.Poels}@UGent.be

**Abstract.** Enterprise architecture (EA) serves as a means to improve business-IT and strategy-operations alignment in an organisation. While it is a fairly mature domain in large enterprises, the need for EA in small and medium-sized enterprises (SMEs) has only been recently addressed. As SMEs have different characteristics and cope with specific problems, a different approach is essential to enable a successful adoption of EA. In order to meet these particular requirements of SMEs, the EA approach CHOOSE has been developed. In previous research, emphasis has been put on refining the method and metamodel of CHOOSE and on the development of supporting software tools. However, the visual notation of CHOOSE has not been investigated yet, while the form of representation has a great impact on the cognitive effectiveness of a diagram. This paper assesses the current visualisation of CHOOSE, describes alternatives and conducts an experimental comparison.

**Key words:** Enterprise architecture, business architecture, small and medium-sized enterprises, CHOOSE, visualisation

## 1 Introduction

Enterprise Architecture (EA) is a structural approach to improve a company's business-IT and strategy-operations alignment [1]. Besides, it is a key instrument in controlling the complexity of an organisation [2]. This is achieved by creating a holistic overview of the organisation through describing and controlling the structure, processes, applications and technology in an integrated way [3]. Although EA is a fairly mature domain in large enterprises, the adoption in small and medium-sized enterprises (SMEs) is lagging behind due to the complexity involved in using the current EA approaches [4]. SMEs often lack the expertise required to implement these approaches and do not have the financial resources to hire consultants [5, 6]. In order to tackle this issue, Bernaert et al. have developed a new approach called CHOOSE, which is adapted to the needs of the target group (section 2.1) [2]. In previous research, the method and metamodel of CHOOSE have been refined and tool support
has been developed [7-11]. These investigations have already put a lot of emphasis on
the comprehensibility of the approach for inexperienced enterprise modellers.
However, up to now the visual notation of CHOOSE has not been evaluated nor
improved, while the form of representation has an important impact on the cognitive
effectiveness of a diagram [12, 13]. This impact is especially crucial in the case of
novice users, which makes it very worthwhile to investigate the visual notation of
CHOOSE [12]. The research in this paper therefore focuses on how CHOOSE should
be visualised in order to allow the users to interpret the diagrams in a cognitively
effective way. Besides, the effect of the form of representation on the perceived ease
of use, perceived usefulness and the intention to use is investigated as well. The result
should enable effective and time efficient communication about the EA within SMEs.

Section 2 provides the theoretical background needed to conduct this research. First, the EA approach CHOOSE is briefly explained [7]. Next, Moody's Physics of Notations [12], a theory for visual notation design, is discussed. Last, related work is shortly summarized. The actual research consists of three major parts: first, the current visualisation is assessed based on the principles of the Physics of Notations (section 3) [12]. Second, alternative representations are developed (section 4). Third, an experiment is conducted to verify which visualisation has the best outcomes in terms of cognitive effectiveness on the one hand and perceived ease of use, perceived usefulness and intention to use on the other hand (section 5 and 6).

## 2 Background

### 2.1 CHOOSE for EA in SMEs

Implementing EA allows SMEs to create an overview of the company. In order to guide them in this process, Bernaert et al. have developed the CHOOSE approach [2]. CHOOSE is an acronym for ‘keep Control, by means of a Holistic Overview, based on Objectives and kept Simple, of your Enterprise’, which refers to the essential requirements for implementing EA in an enterprise [2]. Especially the term ‘Simple’ deserves some additional attention in the context of SMEs, because the word reflects six specific criteria an EA approach must satisfy in order to enable successful adoption in SMEs [7]:

1. The approach should enable SMEs to time efficiently deal with strategic issues.

2. A person with limited IT skills should be able to apply the approach.

3. It should be possible to apply the approach with little assistance of external experts.

4. The approach should enable making descriptions of the processes in the company.

5. The CEO must be involved in the approach.

6. The expected revenues of the approach must exceed the expected costs and risks.

The metamodel of CHOOSE incorporates these criteria, which means it enables SMEs to create simple, yet comprehensive models [7]. These models represent an overview of the business architecture layer, integrating elements of the information systems and technology layers [7, 10]. They consist of four viewpoints: goals (why), actors (who), operations (how) and objects (what) (Fig. 1).
<figure><img src="image_2.png" /><figcaption>Fig. 1. CHOOSE metamodel [7]</figcaption></figure>

An example of a model that has been created with CHOOSE is demonstrated in Fig. 2. As the content is rather straightforward, the reader is encouraged to analyse the diagram making use of the legend (see Appendix). At the same time, the example shows the current visual notation of CHOOSE. As will become clear in section 3, there is still a lot of room for improvement with respect to this visual notation.

## 2.2 Moody's Physics of Notations

Numerous papers cover the evaluation of a notation on the semantic level (e.g. [14, 15]). However, as stated in the introduction, the visual syntax of a notation has a great impact on the cognitive effectiveness of it as well [12, 13]. A couple of theories for evaluating the visual syntax of notations have been developed, such as the Cognitive Dimensions of Notations (CDs) framework [16], the semiotic quality (SEQUAL) framework [17] and Moody's Physics of Notations [12]. Genon et al. argue that the first two frameworks lack theoretical and empirical foundations concerning the visual aspects of notations [18]. Besides, in Moody's evaluation of the CDs framework, several additional shortcomings of that framework can be found [19]. Therefore, Moody's Physics of Notations is used as a basis for this research.

Moody states that a clear design goal needs to be identified before a visual notation can be developed [12]. Common design goals are e.g. simplicity and expressiveness. However, these goals are considered to be vague and subjective. A more objective and scientific goal is cognitive effectiveness, which is the speed, ease and accuracy with which a representation can be processed [13]. To enable designers to create cognitively effective visual notations, Moody has defined nine principles [12]. These are explained in the next paragraphs together with their relevance for this paper.

**Semiotic Clarity.** Each semantic construct should be represented by exactly one graphical symbol, and vice versa. Four kinds of anomalies can occur in a notation:

- Symbol redundancy: a semantic construct is represented by multiple symbols

- Symbol overload: one symbol represents more than one semantic construct

- Symbol excess: a symbol is created that does not represent any semantic construct

- Symbol deficit: there is no symbol provided for a certain semantic construct

This principle is incorporated in this paper with the intention to obtain an unambiguous notation that inherently avoids misconceptions.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Model created with the current visual notation of CHOOSE</figcaption></figure>

**Perceptual Discriminability.** It should be possible to easily and accurately distinguish between different symbols. This is determined by the number of visual variables on which symbols differ, combined with the magnitude of the differences. A greater visual distance between symbols leads to a faster and more accurate recognition. Shape is a detrimental factor in distinguishing between symbols. Therefore, it should be used as the primary visual variable. Perceptual discriminability is very important in the case of CHOOSE, because this notation is used by novices and the requirements for discriminability are higher for novices than for experts.

**Semantic Transparency.** The representation of a construct should suggest its meaning. One way to design semantic transparent symbols is by using icons, which lead to a faster recognition and recall of the constructs. Besides, they especially enhance the comprehensibility of the notation for novice users, which makes it very worthwhile to incorporate this principle in this research.

**Visual Expressiveness.** This is determined by the number of visual variables used in a notation and the extent to which they are used. While perceptual discriminability is a measure for the pairwise discrepancy between symbols, visual expressiveness measures the diversity of the visual vocabulary as a whole. Colour is a strong mechanism for enhancing the visual expressiveness of a notation, as contrast in colour is seen faster than differences in other variables. However, it should only be used in a redundant way, because differences disappear when diagrams are printed in grayscale.

**Complexity Management.** Diagrammatic complexity is measured by the number of elements in a diagram. This type of complexity can be reduced in two ways. First, the diagram can be split into smaller sub diagrams, which is called modularisation.
Furthermore, diagrams can be hierarchically structured to limit the levels of detail. This principle is very important in the case of CHOOSE, because novices have more difficulties dealing with complexity than experts [20].

**Dual Coding.** According to Moody, text can be used as a supplement for graphics. However, it is still important that symbols are distinguishable based on the graphics rather than the text. Labels can be used to distinguish between symbol instances, not between symbol types [12]. Therefore, this principle is somewhat less addressed here.

**Cognitive Integration.** The notation should enable integrating information from different diagrams. Although this principle should not be neglected, it is not incorporated in this research. As CHOOSE targets novices in enterprise modelling, one notation to model everything is preferred. Besides, when SMEs grow and more detail needs to be added to the EA models, it might be useful to map the CHOOSE models on the ArchiMate standard [21]. Bernaert et al. have already conducted a research on this [22], which makes it less relevant to include it in this paper.

**Graphic Economy.** The number of symbol types in a notation should be limited. This principle can be adopted in three ways. First, semantic constructs can be removed. However, the number of constructs in CHOOSE is already limited to the bare minimum. Second, symbol deficit can be introduced, but this harms the semiotic clarity of the notation (see above). Third, visual expressiveness can be used. Manipulating multiple visual variables reduces the need to lower the amount of symbols. In this research, this third action is applied in order to pursue graphic economy. Therefore, the principle by itself will not be individually investigated.

**Cognitive Fit.** Cognitively effective notations for novices might not be cognitively effective for experts, and vice versa. This principle therefore states that different audiences need different notations. CHOOSE targets SMEs, which is a very diverse audience in terms of expertise. However, this principle is not included in this research because in general most users of the target group are novices in enterprise modelling.

## 2.3 Related Work

Several visual notations such as UML [23], i* [24], BPMN [18] and UCM [25] have been evaluated based on the principles of the Physics of Notations. These studies constitute a useful basis for this paper, because they demonstrate a methodology to identify shortcomings in a notation. This methodology is also applied for evaluating the CHOOSE visualisation (section 3). However, the four articles have two limitations in common: the suggested improvements have not been thoroughly elaborated and the findings have not been empirically evaluated.

Gopalakrishnan et al. have compared two notation alternatives for process modelling by conducting a controlled experiment [26]. Although similar goals as in this research are pursued, they do not use the concept of cognitive effectiveness. Furthermore, Huang et al. have conducted an experiment to compare different graph visualisations, based on a cognitive load perspective [27]. Their research does not focus on visual notations, but several aspects of the test design provide useful insights for the experiment described in this paper.
## 3 Analysis of the CHOOSE Visualisation

In this section, the current visual notation of CHOOSE is evaluated based on five principles from Moody’s theory. As mentioned in the previous section dual coding, cognitive integration, graphic economy and cognitive fit are not covered.

**Semiotic Clarity.** Currently, there is no symbol redundancy, excess or deficit. The only anomaly that occurs is symbol overload, which can cause misinterpretation [12]. For CHOOSE, the relationships *association*, *concern* and *control* are represented by the same symbol, which is also the case for *input* and *output* (see Appendix). For these latter two, the problem is not tremendous, since they represent the same content in the opposite direction. For *association*, *concern* and *control*, it is important to resolve this anomaly because the meaning of these relationships cannot be linked.

**Perceptual Discriminability.** Shape is a very important factor in distinguishing between different symbols. However, all ten entities are represented by one shape: a rounded rectangle. Besides, many relationships have equal shapes as well. For the total of 32 semantic constructs, only 12 different shapes are used. This is a crucial shortcoming that will have to be eliminated when designing alternative visualisations.

**Semantic Transparency.** There is clearly a lot of room for improvement regarding this principle. Only four symbols show a certain presence of semantic transparency, which are the symbols of *goal*, *conflict*, *human actor* and *device*. This means 28 symbols do not suggest the meaning of their construct at all.

**Visual Expressiveness.** In total, there are eight visual variables that can be modified: shape, size, colour, brightness, orientation, texture, horizontal and vertical position [12]. Currently, the variables shape, colour, brightness, horizontal and vertical position are used, which is better than most visual notations [24]. However, some of them are more adequately used than others. Constructs belonging to the same viewpoint are e.g. represented by one colour and they are grouped into the same corner. These variables are properly utilised. Brightness on the other hand is categorised as a used variable, because *informed* and *monitor* are represented in a slightly different grey. One could doubt whether the variable is utilised in the right context, because *informed* and *monitor* do not have any meaning in common.

**Complexity Management.** Currently, all information is modelled in one diagram. This means no mechanisms are provided for managing complexity. However, diagrams can quickly become too complex for novices [12]. Hence, integrating this principle would benefit the cognitive effectiveness of the notation. As the metamodel of CHOOSE clearly distinguishes between four viewpoints, it can be useful to apply the mechanism of modularisation and as such split the diagram into sub diagrams.

## 4 Alternative Visualisations Development

The evaluation of the current visual notation served as a basis for the development of three alternatives. During the establishment of the first alternative, special attention was paid to the principles of semiotic clarity, perceptual discriminability, semantic
transparency and visual expressiveness. When, as a little exploratory research, the
resulting diagram was presented to four CEOs of SMEs, the major remark was the
lack of uniformity in style. Although this aspect is not incorporated in the Physics of
Notations, the interview revealed that it should not be neglected. Besides, the research
of Sonderegger and Sauer showed that aesthetics have a positive influence on the
users' performance and the perceived usability [28]. It is therefore worthwhile to
incorporate this in the visualisation. Hence, a second visualisation alternative was
developed with the intention to achieve this uniformity in style. After this, complexity
management was integrated, which resulted in a third visualisation alternative.

In the first alternative, some essential problems of the original notation are handled (Fig. 3). First of all, it is made sure that every semantic construct corresponds with exactly one graphical symbol, and vice versa. Only the relationships *input* and *output* are still represented by the same symbol, for reasons stated in section 3. Second, different constructs within one viewpoint are represented by symbols that have the same shape, while the shapes differ between the viewpoints. The contrast between the viewpoints is further enlarged by using clearly distinguishable colours. Third, icons are used in order to improve the semantic transparency of the symbols. *Operations* are represented by a gear, the relationship *monitor* by an eye, *control* by a steering wheel, etc. Last, visual variables are used in a consistent way. The variable brightness is only used when it can have a meaningful contribution. In the case of the symbols of *RACI*, relationships that involve a higher responsibility are represented by a darker colour.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Model created with the first alternative visual notation</figcaption></figure>

In order to develop the second alternative visualisation, the first alternative is used
as a starting point. This notation does not add any improvements in terms of Moody's
principles. However, as explained above, it is developed in order to obtain uniformity
in style. The result can be seen in Fig. 4.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Model created with the second alternative visual notation</figcaption></figure>

The previous alternatives display all information in one diagram. However, even for a small example as in the images in this paper, relationships between the viewpoints turn the diagram into a complicated maze of information. Therefore, incorporating mechanisms to enable complexity management might improve the comprehensibility of the notation. Several functionalities are hence applied on the previous alternative. First of all, it is made possible to interpret a single viewpoint at a time (Fig. 5). Second, relationships between viewpoints can be analysed in a diagram that only displays the elements of two viewpoints and their interconnections (Fig. 6).

<figure><img src="image_6.png" /><figcaption>Fig. 5. Single viewpoint</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6. Pairwise relationships</figcaption></figure>
These two measures drastically reduce the number of graphical elements displayed, which should lead to an easier and faster understanding of the content. However, if these two representations would be the only ways to access the content, the overview might get lost. This should be avoided because attaining a holistic overview is one of the major advantages of implementing CHOOSE in an organisation. It should therefore still be possible to access the entire diagram. Hence, a third functionality is added. When the entire diagram is displayed, and the user places the cursor on an element in the diagram, that specific element is highlighted together with all adjacent elements (Fig. 7). The combination of these three additional functionalities should lead to better results during the controlled experiment.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Entire view with cursor on the goal ‘Lower variability in production’</figcaption></figure>

# 5 Evaluation

## 5.1 Test Design

In order to determine which representation of CHOOSE is the most comprehensive one, a controlled experiment is conducted. This approach is more appropriate than carrying out case studies because it would be impossible to compare different notations based on a real-life example of an SME without generating learning effects. Yet, it is difficult to execute an experiment of this magnitude within the target group of CHOOSE (i.e. SMEs). Therefore, the test is conducted appealing to a homogeneous group of (on average) 20-year old business engineering students without enterprise modelling experience, as they have many similar characteristics.
Once this is known, the decision needs to be made whether a within-subjects or a between-subjects design is used. A major advantage of a within-subjects design is the need for fewer subjects [29]. However, this design would dramatically increase the duration of the survey, which could lead to a fatigue bias in the results. Therefore, a between-subjects design is applied. This means the students are divided into four groups, and each group receives the same survey but with another visual notation.

The goal of the survey is to examine whether the newly established visualisations result in a better cognitive effectiveness on the one hand and in improved perceived ease of use (PEOU), perceived usefulness (PU) and intention to use (IU) on the other hand. These last variables originate from the Technology Acceptance Model (TAM), which states that improvements in these variables increase the chance of adoption [30]. TAM is used in accordance to the research of Gopalakrishnan et al. [26].

As shortcomings are gradually managed within the developed visualisations, it is expected that each alternative outperforms the previous one. The overall hypotheses are described below. Null hypotheses are not mentioned due to limited space.

H_a: notation i outperforms notation i-1 in terms of cognitive effectiveness (i = 1 to 3)

H_b: notation i outperforms notation i-1 in terms of PEOU, PU and IU (i = 1 to 3)

Cognitive effectiveness (CE) is a variable composed out of three other variables: accuracy, time and mental effort. Accuracy (A) is expressed as the percentage of correct answers in the survey. Time (T) is expressed as the average time used to answer a question, while the subjects are asked to report the mental effort (ME) needed to answer a content question on a 9-point Likert scale [31]. Since these variables are expressed in different units of measurement, the variables are standardised before they are combined into the formula of cognitive effectiveness. Analogous to [32], CE is then calculated as follows:

$$ \text{Cognitive effectiveness} = \frac{Z(A) - Z(T) - Z(ME)}{\sqrt{3}} $$

The survey<fnref n="1" /> consists of three parts. In the first part, general questions are asked to verify the students’ prior knowledge regarding enterprise architecture and conceptual modelling. As a between-subjects design is used, these questions are important to avoid an accidental group selection bias [26]. The second part comprises 12 questions to examine the understanding of the diagram(s), which are all accompanied by a question that inquires for the mental effort needed to answer the content question. The question groups (content + mental effort) are randomised in order to avoid obtaining overall better results for the last questions. The third and last part consists of 14 questions based on [26] that gauge the PEOU, PU and IU. The answers are measured on a 5-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’.

## 5.2 Experiment Results

In total, 120 useful observations can be analysed. Six results are omitted, because there are clear indications that those students have not conscientiously filled in the survey. The four sample sizes are slightly different, ranging from 29 responses to 32. Descriptive statistics for each variable can be found in Table 1.

<footnote n="1">¹ The survey questions can be accessed using the following link: http://www.mis.ugent.be/choose/electronicappendix.pdf</footnote>
<table><caption>**Table 1.** Descriptive statistics per group</caption><thead><tr><th rowspan="2">Variable</th><th colspan="2">Current notation<br>(N=29)</th><th colspan="2">Alternative 1<br>(N=30)</th><th colspan="2">Alternative 2<br>(N=29)</th><th colspan="2">Alternative 3<br>(N=32)</th></tr><tr><th>Mean</th><th>SD</th><th>Mean</th><th>SD</th><th>Mean</th><th>SD</th><th>Mean</th><th>SD</th></tr></thead><tbody><tr><td>CE</td><td>-1.0353</td><td>0.9941</td><td>-0.4365</td><td>1.0696</td><td>-0.3956</td><td>1.0331</td><td>0.7080</td><td>0.8134</td></tr><tr><td>A</td><td>0.8276</td><td>0.1230</td><td>0.8810</td><td>0.1344</td><td>0.8916</td><td>0.1303</td><td>0.9665</td><td>0.0544</td></tr><tr><td>T</td><td>35.6616</td><td>8.6862</td><td>31.6336</td><td>6.3017</td><td>33.4239</td><td>5.7900</td><td>27.7378</td><td>4.6965</td></tr><tr><td>ME</td><td>3.3736</td><td>1.0176</td><td>3.4333</td><td>1.0941</td><td>3.1695</td><td>1.0323</td><td>2.7891</td><td>0.9499</td></tr><tr><td>PEOU</td><td>3.2690</td><td>0.3752</td><td>3.1533</td><td>0.4862</td><td>3.2138</td><td>0.6255</td><td>3.4375</td><td>0.4172</td></tr><tr><td>PU</td><td>3.6621</td><td>0.5017</td><td>3.6533</td><td>0.7482</td><td>3.5448</td><td>0.6277</td><td>4.0250</td><td>0.3619</td></tr><tr><td>IU</td><td>3.0776</td><td>0.7621</td><td>3.0750</td><td>0.7689</td><td>3.3707</td><td>0.5733</td><td>3.5625</td><td>0.5198</td></tr></tbody></table>

The variable CE satisfies all criteria to be analysed by means of an ANOVA. The other variables violate at least one of the assumptions. Therefore, these variables are examined with the Kruskal-Wallis test and the Mann-Whitney U test. These tests assume that the distributions of the different groups have equal shapes. It should be mentioned however that this assumption is not entirely satisfied for the variables ME and PEOU. Hence, these variables should be cautiously analysed. All analyses have been conducted with a significance level of 5%. The results in Table 2 demonstrate that the third alternative has a significantly higher cognitive effectiveness than the other visual notations, while the differences between the other notations are not significant. These results can be explained by analysing the component variables of cognitive effectiveness. All three alternatives have better scores for accuracy than the current notation, but alternative 3 outperforms alternative 1 and 2. Next to this, the average time needed to answer a question is tremendously lower for alternative three than for the other alternatives. And last, only for the third alternative, the mental effort required to interpret the notation is significantly lower than for the current notation. For the variable PEOU, the only significant result that can be observed is the difference between alternative 1 and 3. The boxplots reveal that alternatives 2 and 3 have a higher median than the current notation and the first alternative, yet the differences are not significant. Possibly, the true significance level has shifted due to the unequally shaped distributions [33]. Regarding PU, alternative 3 has significantly better results than the other notations. Finally, the IU is significantly better for alternative 3 than for the current notation and for 2 and 3 than for the first alternative.

<table><caption>**Table 2.** Test results of the pairwise comparisons</caption><thead><tr><th>Variable</th><th>Test statistic</th><th>0 – 1</th><th>0 – 2</th><th>0 – 3</th><th>1 – 2</th><th>1 – 3</th><th>2 – 3</th></tr></thead><tbody><tr><td>CE</td><td>MD</td><td>0.5988</td><td>0.6397</td><td>1.7433<sup>***</sup></td><td>0.0409</td><td>1.1445<sup>***</sup></td><td>1.1036<sup>***</sup></td></tr><tr><td>A</td><td>U</td><td>311<sup>*</sup></td><td>263<sup>**</sup></td><td>128<sup>***</sup></td><td>429</td><td>296<sup>**</sup></td><td>279<sup>**</sup></td></tr><tr><td>T</td><td>U</td><td>275<sup>**</sup></td><td>337</td><td>182<sup>***</sup></td><td>333</td><td>284<sup>**</sup></td><td>214<sup>***</sup></td></tr><tr><td>ME</td><td>U</td><td>406.5</td><td>370.5</td><td>310<sup>*</sup></td><td>352</td><td>312.5<sup>**</sup></td><td>379</td></tr><tr><td>PEOU</td><td>U</td><td>323.5</td><td>361.5</td><td>356.5</td><td>320</td><td>310.5<sup>**</sup></td><td>382</td></tr><tr><td>PU</td><td>U</td><td>402</td><td>369</td><td>271<sup>**</sup></td><td>373</td><td>365<sup>*</sup></td><td>260<sup>**</sup></td></tr><tr><td>IU</td><td>U</td><td>400.5</td><td>286.5</td><td>267.5<sup>**</sup></td><td>286.5<sup>**</sup></td><td>273<sup>**</sup></td><td>360</td></tr></tbody></table>

Note: MD = mean difference (Tukey HSD); U = Mann-Whitney U

*P<0.05; **P<0.01; ***P<0.001.
# 6 Discussion

The experiment results demonstrate that the last visual notation is clearly the best alternative. First of all, this notation is cognitively more effective than the others. Besides, the respondents of this notation have indicated a high perceived usefulness and intention to use. It is therefore advised to implement this notation.

Several statements can be made in the context of this experiment:

1. When alternative 1 is compared to the current notation, the conclusion can be made that incorporating semiotic clarity, perceptual discriminability, semantic transparency and visual expressiveness improves the accuracy and speed of the answers. However, the change in cognitive effectiveness is not significant due to the variable mental effort, which is not significantly improved.

2. When, on top of these principles, complexity management is applied, an impressive difference can be observed. Adding this principle results in a significant increase in the cognitive effectiveness of the notation. This can be concluded when alternative 3 is compared to the other visualisations.

3. Enhancing the aesthetics of the notation does not improve the cognitive effectiveness of it, nor one of its component variables (alternative 2 vs. 1).

4. However, ameliorating the aesthetics does lead to a higher intention to use. The results for this variable are significantly better for alternative 3 compared to the current notation and for 2 and 3 compared to the first alternative.

5. Integrating all five considered principles leads to a higher perceived usefulness of the notation. As the PU is not improved when the first four principles are applied, the idea rises that complexity management causes the increase in PU.

Overall, it can be said that both Moody's principles and aesthetics have a positive influence on the notation, and this in a complementary way. Moody's principles improve the comprehensibility of the notation and lead to an increase in perceived usefulness. Aesthetics on the other hand augment the intention to use the notation.

# 7 Conclusion and Future Research

This research has investigated the visual notation of CHOOSE, which is an EA approach developed by Bernaert et al. with the aim to facilitate the implementation of EA in the context of SMEs [7]. The current visual notation has been evaluated and alternatives have been established, after which the different visualisations have been compared in an experiment. Based on this experiment, an advice has been made to implement one of the notations in the CHOOSE approach.

The result of the investigation facilitates a cognitively effective interpretation of CHOOSE diagrams on the one hand, and improves the perceived usefulness and the intention to use the notation on the other hand. In practice, this should lead to an effective and time efficient way to deal with EA and hence improve its adoption rate in SMEs. However, as the experiment is conducted appealing to students, this aspect is ought to be further analysed in future work by means of executing case studies or experiments in SMEs. Although the students subjected to the experiment have several
characteristics in common with employees of SMEs – they have for example a keen interest in business topics and are novices in enterprise modelling – it is difficult to extrapolate the results of this investigation to the target group of SMEs.

Besides these practical implications, this paper also provides a validation for the Physics of Notations. The research reveals that applying its principles significantly improves the comprehensibility of the notation. On top of this, it becomes clear that aesthetics should not be neglected, as this increases the intention to use the notation.

At last, this paper suggests a methodology to evaluate visual notations and develop improved versions. Although this research is conducted in the context of CHOOSE, the positive outcome of this case might motivate researchers to consider following the same path.

## References

1. Maes, R.: An Integrative Perspective on Information Management. In: Huizing, A., Vries, E. J. (eds.) Information Management: Setting the Scene. pp. 11-26. Elsevier, Oxford, UK (2007)

2. Bernaert, M., et al.: Enterprise Architecture for Small and Medium-Sized Enterprises: A Starting Point for Bringing Ea to Smes, Based on Adoption Models. In: Devos, J., Van Landeghem, H., Deschoolmeester, D. (eds.) Information Systems and Small and Medium-sized Enterprises: State of art of IS research in SMEs. pp. 67-96. Springer (2013)

3. Lankhorst, M., et al.: Enterprise Architecture at Work. Springer, New York (2013)

4. Bhagwat, R., Sharma, M. K.: Information System Architecture: A Framework for a Cluster of Small- and Medium-Sized Enterprises (Smes). Production Planning Control 18(4), 283-296 (2007)

5. Dehbokry, S. G., Chew, E. K.: The Strategic Requirements for an Enterprise Business Architecture Framework by Smes. LNIT 2(1), 32-38 (2014)

6. Jacobs, D., et al.: Enterprise Architecture for Small and Medium Enterprise Growth. In: Albani, A., Dietz, J., Verelst, J. (eds.) EEWC 2011. LNBIP, vol. 79, pp. 61-75. Springer, Heidelberg (2011)

7. Bernaert, M., et al.: Choose: Towards a Metamodel for Enterprise Architecture in Small and Medium-Sized Enterprises. Ghent University, K.U. Leuven, University of Antwerp. (2013)

8. Ingelbeen, D., Bernaert, M., Poels, G.: Enterprise Architecture Software Tool Support for Small and Medium-Sized Enterprises: Ease. In: 19th Americas Conference on Information Systems (AMCIS 2013), Chicago, USA (2013)

9. Dumeez, J., Bernaert, M., Poels, G.: Development of Software Tool Support for Enterprise Architecture in Small and Medium-Sized Enterprises. In: Franch, X., Soffer, P. (eds.) Advanced Information Systems Engineering Workshops. LNBIP, vol. 148, pp. 87-98. Springer, Heidelberg (2013)

10. Bernaert, M., Maes, J., Poels, G.: An Android Tablet Tool for Enterprise Architecture Modeling in Small and Medium-Sized Enterprises. In: Grabis, J., Kirikova, M., Zdravkovic, J., Stirna, J. (eds.) POEM 2013. LNBIP, vol. 165, pp. 145-160. Springer, Heidelberg (2013)

11. Zutterman, S.: Development of a Tool for Business Architecture Modeling in Eclipse. Ghent University (2013)

12. Moody, D.: The "Physics" of Notations: Towards a Scientific Basis for Constructing Visual Notations in Software Engineering. IEEE Transactions on Software Engineering 35(5), 756-778 (2009)
13. Larkin, J. H., Simon, H. A.: Why a Diagram Is (Sometimes) Worth Ten Thousand Words. Cognitive Science 11(1), 65-100 (1987)

14. Recker, J. C., et al.: Do Process Modelling Techniques Get Better? A Comparative Ontological Analysis of Bpmn. In: Campbell, B., Underwood, J., Bunker, D. (eds.) 16th Australasian Conference on Information Systems, Sydney, Australia (2005)

15. Opdahl, A. L., Henderson-Sellers, B.: Ontological Evaluation of the Uml Using the Bunge-Wand-Weber Model. Software & Systems Modeling 1(1), 43-67 (2002)

16. Green, T. R. G., et al.: Cognitive Dimensions: Achievements, New Directions, and Open Questions. Journal of Visual Languages & Computing 17(4), 328-365 (2006)

17. Krogstie, J., Sindre, G., Havard, J.: Process Models Representing Knowledge for Action: A Revised Quality Framework. European Journal of Information Systems 15(1), 91-102 (2006)

18. Genon, N., Heymans, P., Amyot, D.: Analysing the Cognitive Effectiveness of the Bpmn 2.0 Visual Notation. In: Malloy, B., Staab, S., van den Brand, M. (eds.) SLE 2010. LNCS, vol. 6563, pp. 377-396. Springer, Heidelberg (2011)

19. Moody, D.: Theory Development in Visual Language Research: Beyond the Cognitive Dimensions of Notations. In: IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 151-154, Corvallis, USA (2009)

20. Sweller, J.: Cognitive Load Theory, Learning Difficulty, and Instructional Design. Learning and Instruction 4(4), 295-312 (1994)

21. The Open Group: Archimate 2.0 Specification. Van Haren Publishing (2012)

22. Bernaert, M., et al.: Bridging Ea for Smes to Ea for Large Enterprises: Mapping Choose on the Archimate Standard. Ghent University, K.U. Leuven, University of Antwerp. (2013)

23. Moody, D., Hillegersberg, J.: Evaluating the Visual Syntax of Uml: An Analysis of the Cognitive Effectiveness of the Uml Family of Diagrams. In: Gašević, D., Lämmel, R., Van Wyk, E. (eds.) SLE 2008. LNCS, vol. 5452, pp. 16-34. Springer, Heidelberg (2009)

24. Moody, D., Heymans, P., Matulevičius, R.: Visual Syntax Does Matter: Improving the Cognitive Effectiveness of the I* Visual Notation. Requirements Engineering 15(2), 141-175 (2010)

25. Genon, N., Amyot, D., Heymans, P.: Analysing the Cognitive Effectiveness of the Ucm Visual Notation. In: Kraemer, F. A., Herrmann, P. (eds.) SAM 2010. LNCS, vol. 6598, pp. 221-240. Springer, Heidelberg (2011)

26. Gopalakrishnan, S., Krogstie, J., Sindre, G.: Adapting Uml Activity Diagrams for Mobile Work Process Modelling: Experimental Comparison of Two Notation Alternatives. In: van Bommel, P., Hoppenbrouwers, S., Overbeek, S., Proper, E., Barjis, J. (eds.) POEM 2010. LNBIP, vol. 68, pp. 145-161. Springer, Heidelberg (2010)

27. Huang, W., Eades, P., Hong, S.-H.: Measuring Effectiveness of Graph Visualizations: A Cognitive Load Perspective. Information Visualization 8(3), 139-152 (2009)

28. Sonderegger, A., Sauer, J.: The Influence of Design Aesthetics in Usability Testing: Effects on User Performance and Perceived Usability. Applied Ergonomics 41(3), 403-410 (2010)

29. Brown, S. R., Melamed, L. E.: Experimental Design and Analysis. SAGE, Newbury Park (1990)

30. Davis, F. D.: Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13(3), 319-340 (1989)

31. Paas, F. G.: Training Strategies for Attaining Transfer of Problem-Solving Skill in Statistics: A Cognitive-Load Approach. Journal of Educational Psychology 84(4), 429 (1992)

32. Tuovinen, J., Paas, F.: Exploring Multidimensional Approaches to the Efficiency of Instructional Conditions. Instructional Science 32(1-2), 133-152 (2004)

33. Skovlund, E., Fenstad, G. U.: Should We Always Choose a Nonparametric Test When Comparing Two Apparently Nonnormal Distributions? Journal of Clinical Epidemiology 54(1), 86-92 (2001)
Appendix

<figure><img src="image_9.png" /><figcaption>Fig. 8. Legend: symbols applied in the different visual notations</figcaption></figure>