# How to measure business value of business model ontologies? A proposal

Hichem Benaissa Anouar Badsi, Leila Zemmouchi Ghomari, Abdessamed Réda Ghomari

## ► To cite this version:

Hichem Benaissa Anouar Badsi, Leila Zemmouchi Ghomari, Abdessamed Réda Ghomari. How to measure business value of business model ontologies? A proposal. International Journal of Business Information Systems, 2015, 19 (2), pp.241. 10.1504/IJBIS.2015.069433. hal-04566102

HAL Id: hal-04566102

https://hal.science/hal-04566102

Submitted on 2 May 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>
# How to measure business value of business model ontologies? A proposal

Hichem Benaissa Anouar Badsi,
Leila Zemmouchi-Ghomari* and
Abdessamed Réda Ghomari

LMCS Laboratory,
Ecole nationale Supérieure d'Informatique (ESI) (ex INI),
P.O. Box 16309, Oued Smar, Algiers, Algeria

Email: h_badsi@esi.dz

Email: l_zemmouchi@esi.dz

Email: a_ghomari@esi.dz

*Corresponding author

**Abstract:** A business model is an attractive concept for business management. It can benefit from ontology paradigms that are described more formally and more precisely. However, case studies of ontologies describing enterprise business models have rarely been reported in the literature, although there are a few indications of the impact of these ontologies on business models; however, these indications do not ease the assessment of the added value of this type of ontology. The aim of this paper is two-fold; on one hand, we highlight the lack of task-based assessment of enterprise business model ontologies via some concrete case studies, and on the other hand, we aim to tackle this issue by proposing indicators intended for measuring the impact of business model ontologies in the framework of enterprise business models.

**Keywords:** ontology evaluation; ontology task-based assessment; business model representation; business model ontology; BMO; ontology business value; business information systems.

**Reference** to this paper should be made as follows: Badsi, H.B.A., Zemmouchi-Ghomari, L. and Ghomari, A.R. (2015) ‘How to measure business value of business model ontologies? A proposal’, *Int. J. Business Information Systems*, Vol. 19, No. 2, pp.241–264.

**Biographical notes:** Hichem Benaissa Anouar Badsi is a Computer Science Engineer. He received his Research Master in Computer Science from ESI, Algiers, Algeria, in June 2013. He is currently a PhD student and a member of LMCS (the Laboratory of Systems Design Methodologies). His research interests focus on business ontology engineering, business models and multi-agents systems.

Leila Zemmouchi-Ghomari received her PhD in Computer Science from ESI, Algiers, in January 2014. She is a member of LMCS (the Laboratory of Systems Design Methodologies). Her research interests focus on ontology engineering, knowledge engineering and semantic web. She is currently a Lecturer at UMBB: M’hamed Bougara Boumerdes University, Algeria.
# How to measure business value of business model ontologies? A proposal

Hichem Benaissa Anouar Badsi,
Leila Zemmouchi-Ghomari* and
Abdessamed Réda Ghomari

LMCS Laboratory,

Ecole nationale Supérieure d'Informatique (ESI) (ex INI),

P.O. Box 16309, Oued Smar, Algiers, Algeria

Email: h_badsi@esi.dz

Email: l_zemmouchi@esi.dz

Email: a_ghomari@esi.dz

*Corresponding author

**Abstract:** A business model is an attractive concept for business management. It can benefit from ontology paradigms that are described more formally and more precisely. However, case studies of ontologies describing enterprise business models have rarely been reported in the literature, although there are a few indications of the impact of these ontologies on business models; however, these indications do not ease the assessment of the added value of this type of ontology. The aim of this paper is two-fold; on one hand, we highlight the lack of task-based assessment of enterprise business model ontologies via some concrete case studies, and on the other hand, we aim to tackle this issue by proposing indicators intended for measuring the impact of business model ontologies in the framework of enterprise business models.

**Keywords:** ontology evaluation; ontology task-based assessment; business model representation; business model ontology; BMO; ontology business value; business information systems.

**Reference** to this paper should be made as follows: Badsi, H.B.A., Zemmouchi-Ghomari, L. and Ghomari, A.R. (2015) ‘How to measure business value of business model ontologies? A proposal’, *Int. J. Business Information Systems*, Vol. 19, No. 2, pp.241–264.

**Biographical notes:** Hichem Benaissa Anouar Badsi is a Computer Science Engineer. He received his Research Master in Computer Science from ESI, Algiers, Algeria, in June 2013. He is currently a PhD student and a member of LMCS (the Laboratory of Systems Design Methodologies). His research interests focus on business ontology engineering, business models and multi-agents systems.

Leila Zemmouchi-Ghomari received her PhD in Computer Science from ESI, Algiers, in January 2014. She is a member of LMCS (the Laboratory of Systems Design Methodologies). Her research interests focus on ontology engineering, knowledge engineering and semantic web. She is currently a Lecturer at UMBB: M'hamed Bougara Boumerdes University, Algeria.
Abdessamed Réda Ghomari received his PhD in Computer Science from ESI, Algiers, in 2008. From 2001 to 2014, he is Information System Management Team Head at LMCS (the Laboratory of Systems Design Methodologies). His research interests focus on cooperative knowledge engineering, Enterprise 2.0 and collective intelligence. He is currently an Associate Professor at ESI (National Superior School of Computer Science), Algiers, Algeria.

# 1 Introduction

A business model describes the rationale of how an organisation creates, delivers, and captures value (Osterwalder and Pigneur, 2010), and is “oriented toward total value creation for all parties involved” (Zott and Amit, 2010). Furthermore, business models are used by managers to explore possibilities for future development (Osterwalder, 2004).

Nevertheless, considering the increasing external pressures enterprises face, their individual business models are becoming increasingly complex and more difficult to manage (Osterwalder, 2004; Yasin et al., 2006; Routray and Mishra, 2009).

Researchers have established that ontologies are an effective way of representing and sharing knowledge of several domains (Chandrasekaran et al., 1999; Staab and Studer, 2009; Dietz, 2006) such as business domain.

Ontologies today have matured significantly since their inception (Staab and Studer, 2009), and they are increasingly being integrated in enterprises' activities. Concrete applications of ontologies in an enterprise context (Chandrasekaran et al., 1999; Obrst, 2003; Milton et al., 2010; Oberle, 2013; Asharaf and Abdulla, 2013) include the integration of heterogeneous databases, semantic interoperability, semantics-driven product/service discovery, and recommendation mechanisms and business intelligence (because ontologies allow reasoning).

The integration of ontologies in enterprises is not a trivial task (Gordijn and Akkermans, 2001a; Kort and Gordijn, 2008), and several researchers such as Gordijn and Akkermans (2001a), Geerts and McCarthy (2002), and Osterwalder (2004) support the idea that business model ontologies can play a significant role in the improvement of enterprise business models and hence enhance the profitability of enterprises.

However, the literature does not provide tangible evidence to substantiate the positive impact of ontologies. In fact, this claim is essentially based on the descriptive method (Hevner et al., 2004) applied from the evaluation perspective of a design-science research process, which is not encouraged except when other methods (observational, analytical, and experimental) are not possible. Consequently, there is a need of measurable indicators that allow for evaluation via more preferable assessment methods such as the traditional case study approach (observational), which we have used.

To achieve task-based assessment of ontologies for business models, we selected three different cases – E3-value ontology in online news provisioning (ONP), business model ontology (BMO) in Montreux Jazz festival (MJF), and the extended E3-value ontology in banking strategic partnership (BSP).

Despite sustained efforts, we were unable to extract metrics from the selected case studies for several reasons such as the fact that the application of ontologies to business models remains theoretical in all three cases. In fact, these ontologies have not been
Abdessamed Réda Ghomari received his PhD in Computer Science from ESI, Algiers, in 2008. From 2001 to 2014, he is Information System Management Team Head at LMCS (the Laboratory of Systems Design Methodologies). His research interests focus on cooperative knowledge engineering, Enterprise 2.0 and collective intelligence. He is currently an Associate Professor at ESI (National Superior School of Computer Science), Algiers, Algeria.

# 1 Introduction

A business model describes the rationale of how an organisation creates, delivers, and captures value (Osterwalder and Pigneur, 2010), and is “oriented toward total value creation for all parties involved” (Zott and Amit, 2010). Furthermore, business models are used by managers to explore possibilities for future development (Osterwalder, 2004).

Nevertheless, considering the increasing external pressures enterprises face, their individual business models are becoming increasingly complex and more difficult to manage (Osterwalder, 2004; Yasin et al., 2006; Routray and Mishra, 2009).

Researchers have established that ontologies are an effective way of representing and sharing knowledge of several domains (Chandrasekaran et al., 1999; Staab and Studer, 2009; Dietz, 2006) such as business domain.

Ontologies today have matured significantly since their inception (Staab and Studer, 2009), and they are increasingly being integrated in enterprises’ activities. Concrete applications of ontologies in an enterprise context (Chandrasekaran et al., 1999; Obrst, 2003; Milton et al., 2010; Oberle, 2013; Asharaf and Abdulla, 2013) include the integration of heterogeneous databases, semantic interoperability, semantics-driven product/service discovery, and recommendation mechanisms and business intelligence (because ontologies allow reasoning).

The integration of ontologies in enterprises is not a trivial task (Gordijn and Akkermans, 2001a; Kort and Gordijn, 2008), and several researchers such as Gordijn and Akkermans (2001a), Geerts and McCarthy (2002), and Osterwalder (2004) support the idea that business model ontologies can play a significant role in the improvement of enterprise business models and hence enhance the profitability of enterprises.

However, the literature does not provide tangible evidence to substantiate the positive impact of ontologies. In fact, this claim is essentially based on the descriptive method (Hevner et al., 2004) applied from the evaluation perspective of a design-science research process, which is not encouraged except when other methods (observational, analytical, and experimental) are not possible. Consequently, there is a need of measurable indicators that allow for evaluation via more preferable assessment methods such as the traditional case study approach (observational), which we have used.

To achieve task-based assessment of ontologies for business models, we selected three different cases – E3-value ontology in online news provisioning (ONP), business model ontology (BMO) in Montreux Jazz festival (MJF), and the extended E3-value ontology in banking strategic partnership (BSP).

Despite sustained efforts, we were unable to extract metrics from the selected case studies for several reasons such as the fact that the application of ontologies to business models remains theoretical in all three cases. In fact, these ontologies have not been
tested in real world situations. Additionally, these ontologies have been used for very specific purposes.

To address this issue, we propose an approach to derive indicators from a list of performance criteria that are used by international consulting and auditing organisations that regularly evaluate companies belonging to several business domains.

The present study fits into the ontology evaluation domain, particularly BMO in the context of enterprise business models.

The aim of this paper is two-fold. First, we study the impact of ontologies for business models on enterprise business models through some case studies. In doing so, we seek to illustrate that these ontologies are deficient in terms of task-based assessment. Second, we propose an approach for deriving indicators that facilitate measuring the impact of ontologies for business models on enterprise business models, following which we demonstrate their applicability to the aforementioned case studies.

The rest of this paper is organised as follows: Section 2 briefly defines the paradigms used in this study – that is, business model, ontology, and BMO, and describes and compares the main business model ontologies reported in the literature (Section 3). Section 4 describes the case studies that illustrate the use of ontologies for business models in enterprise business models and discusses the lessons drawn from these case studies. Section 5 discusses related studies from the literature. Section 6 presents our proposed approach for deriving indicators that allows us to measure the impact of ontologies for business models on enterprises’ business models. Next, we apply these derived metrics to the previously described case studies to assess the ontologies used in these cases (Section 7). Finally, our conclusions and perspectives are presented in Section 8.

## 2 Definitions

Understanding business concepts is a *sine qua non* condition to develop good quality business applications and to ensure that there is no violation of the domain rules. It also facilitates applications adaptation to requirements modification without major changes in the overall architecture. It is known that companies are under pressures from external environment that push for change. Generally, these changes occur naturally or in a random way. Regardless of any prior, this instability can lead to project failures. That is why; finding a way to control the execution of changes following a well-designed, prepared and tested plan would be useful, such as: business model artefact (Dietz, 2006).

### 2.1 Business model

From the 103 publications related to business models investigated by Zott et al. (2011), more than a third (37%) did not define the concept at all, taking all meaning more or less for granted. Less than half (44%) defined it explicitly or conceptualised it by the enumeration of its main components. The remaining publications (19%) referred to the work of other researchers definitions of concepts.

Moreover, existing definitions overlap only partially, giving a rise to a multitude of interpretations.
tested in real world situations. Additionally, these ontologies have been used for very specific purposes.

To address this issue, we propose an approach to derive indicators from a list of performance criteria that are used by international consulting and auditing organisations that regularly evaluate companies belonging to several business domains.

The present study fits into the ontology evaluation domain, particularly BMO in the context of enterprise business models.

The aim of this paper is two-fold. First, we study the impact of ontologies for business models on enterprise business models through some case studies. In doing so, we seek to illustrate that these ontologies are deficient in terms of task-based assessment. Second, we propose an approach for deriving indicators that facilitate measuring the impact of ontologies for business models on enterprise business models, following which we demonstrate their applicability to the aforementioned case studies.

The rest of this paper is organised as follows: Section 2 briefly defines the paradigms used in this study – that is, business model, ontology, and BMO, and describes and compares the main business model ontologies reported in the literature (Section 3). Section 4 describes the case studies that illustrate the use of ontologies for business models in enterprise business models and discusses the lessons drawn from these case studies. Section 5 discusses related studies from the literature. Section 6 presents our proposed approach for deriving indicators that allows us to measure the impact of ontologies for business models on enterprises’ business models. Next, we apply these derived metrics to the previously described case studies to assess the ontologies used in these cases (Section 7). Finally, our conclusions and perspectives are presented in Section 8.

## 2 Definitions

Understanding business concepts is a *sine qua non* condition to develop good quality business applications and to ensure that there is no violation of the domain rules. It also facilitates applications adaptation to requirements modification without major changes in the overall architecture. It is known that companies are under pressures from external environment that push for change. Generally, these changes occur naturally or in a random way. Regardless of any prior, this instability can lead to project failures. That is why; finding a way to control the execution of changes following a well-designed, prepared and tested plan would be useful, such as: business model artefact (Dietz, 2006).

### 2.1 Business model

From the 103 publications related to business models investigated by Zott et al. (2011), more than a third (37%) did not define the concept at all, taking all meaning more or less for granted. Less than half (44%) defined it explicitly or conceptualised it by the enumeration of its main components. The remaining publications (19%) referred to the work of other researchers definitions of concepts.

Moreover, existing definitions overlap only partially, giving a rise to a multitude of interpretations.
Actually, business models are still not well comprehended and the knowledge about business models is fragmented over different disciplines (Gordijn and Akkermans, 2003b; Al-Debei and Avison, 2010), where both business and information systems researchers testify to the importance of the business model concept to the success of companies. This concept remains fuzzy and vague, and lacks of consensus on its definition and its components (Al-Debei and Fitzgerald, 2010; Al-Debei and Avison, 2010; Morris et al., 2005; Shafer et al., 2005).

The mismatch in business model signification is more perceptible between business experts and technology experts. A survey reported in Osterwalder et al. (2005) shows that the business model is generally perceived as a business value/customer approach while technology experts have a more focused role/activity perspective.

In fact, most business model definitions belong to the business value perspective, such as Timmers' (1998) definition that describes business model as:

1. architecture for product, service and information flows, including a description of the various business actors and their roles

2. description of the potential benefits for the various actors in the business

3. description of resources and incomes.

In the same current of thoughts, Osterwalder (2004) defines the business model as: a conceptual tool that contains a set of elements and their relationships that allows expressing the logic of a business to earn money. In other words, it is a description of the architecture of the enterprise and its network of partners for creating, marketing and delivering this value in order to generate profitable and sustainable revenue streams (Figure 1).

<figure><img src="image_1.png" /><figcaption>**Figure 1** The business model components</figcaption></figure>

Source: Osterwalder (2004)

The role/activity perspective of the business model is conveyed by Chesbrough's definition (Chesbrough and Rosenbloom, 2002; Chesbrough, 2010) who considers that a business model must fulfil the following functions:
Actually, business models are still not well comprehended and the knowledge about business models is fragmented over different disciplines (Gordijn and Akkermans, 2003b; Al-Debei and Avison, 2010), where both business and information systems researchers testify to the importance of the business model concept to the success of companies. This concept remains fuzzy and vague, and lacks of consensus on its definition and its components (Al-Debei and Fitzgerald, 2010; Al-Debei and Avison, 2010; Morris et al., 2005; Shafer et al., 2005).

The mismatch in business model signification is more perceptible between business experts and technology experts. A survey reported in Osterwalder et al. (2005) shows that the business model is generally perceived as a business value/customer approach while technology experts have a more focused role/activity perspective.

In fact, most business model definitions belong to the business value perspective, such as Timmers' (1998) definition that describes business model as:

1. architecture for product, service and information flows, including a description of the various business actors and their roles

2. description of the potential benefits for the various actors in the business

3. description of resources and incomes.

In the same current of thoughts, Osterwalder (2004) defines the business model as: a conceptual tool that contains a set of elements and their relationships that allows expressing the logic of a business to earn money. In other words, it is a description of the architecture of the enterprise and its network of partners for creating, marketing and delivering this value in order to generate profitable and sustainable revenue streams (Figure 1).

<figure><img src="image_2.png" /><figcaption>**Figure 1** The business model components</figcaption></figure>

Source: Osterwalder (2004)

The role/activity perspective of the business model is conveyed by Chesbrough's definition (Chesbrough and Rosenbloom, 2002; Chesbrough, 2010) who considers that a business model must fulfil the following functions:
* to articulate the value proposition (i.e., the value created for users by offering a technology)

* to identify market segments and clarify the mechanism of income generation (i.e., users for whom the technology is useful and for what purpose)

* to define the structure of the value chain necessary to create and distribute and supply additional assets required to support the position in the chain

* to detail the mechanism(s) of income by which the company will be paid for what it offers

* to estimate the cost structure and profit potential (proposal given value and the structure of the value chain)

* to describe the position of the company in the value network linking suppliers and customers (including the identification of potential partners and competitors)

* to formulate the competitive strategy by which the innovating firm will gain and maintain an edge over its rivals.

In the scope of this paper, we will use a widely agreed definition of the business model which includes the notion of value, explicitly: "A business model describes the rationale of how an organization creates, delivers, and captures value" (Osterwalder and Pigneur, 2010). Indeed, this definition is broad enough to cover all kind of business models (Burkhart et al., 2011).

Research work, related to enterprise business models, conducted until now attempt to identify all aspects of the business domain. For example, representing entities and services that makes up business processes, or aspects of business value that can be perceived through the business model.

In fact, we are involved in the representation and the evaluation of business models research field that consists of

> "criteria that describe the constitutive components of business models and resulting relationships between them. It also depicts the ways of representing and evaluating business models. As a result, existing gaps in the representation and evaluation of business models should be revealed because these aspects are particularly important for the purpose of business models, namely the communication and analysis of companies' business logic." (Burkhart et al., 2011)

## 2.2 Ontology

One of the most consensual ontology definitions is that an ontology is a formal, explicit specification of a shared conceptualisation (Studer et al., 1998), it provides a machine readable, standardised nomenclature and semantics of a part of the real world.

This knowledge representation technology is intended to promote: share, reuse and communication between human and artificial agents (Milton et al., 2010).
* to articulate the value proposition (i.e., the value created for users by offering a technology)

* to identify market segments and clarify the mechanism of income generation (i.e., users for whom the technology is useful and for what purpose)

* to define the structure of the value chain necessary to create and distribute and supply additional assets required to support the position in the chain

* to detail the mechanism(s) of income by which the company will be paid for what it offers

* to estimate the cost structure and profit potential (proposal given value and the structure of the value chain)

* to describe the position of the company in the value network linking suppliers and customers (including the identification of potential partners and competitors)

* to formulate the competitive strategy by which the innovating firm will gain and maintain an edge over its rivals.

In the scope of this paper, we will use a widely agreed definition of the business model which includes the notion of value, explicitly: "A business model describes the rationale of how an organization creates, delivers, and captures value" (Osterwalder and Pigneur, 2010). Indeed, this definition is broad enough to cover all kind of business models (Burkhart et al., 2011).

Research work, related to enterprise business models, conducted until now attempt to identify all aspects of the business domain. For example, representing entities and services that makes up business processes, or aspects of business value that can be perceived through the business model.

In fact, we are involved in the representation and the evaluation of business models research field that consists of

> "criteria that describe the constitutive components of business models and resulting relationships between them. It also depicts the ways of representing and evaluating business models. As a result, existing gaps in the representation and evaluation of business models should be revealed because these aspects are particularly important for the purpose of business models, namely the communication and analysis of companies' business logic." (Burkhart et al., 2011)

## 2.2 Ontology

One of the most consensual ontology definitions is that an ontology is a formal, explicit specification of a shared conceptualisation (Studer et al., 1998), it provides a machine readable, standardised nomenclature and semantics of a part of the real world.

This knowledge representation technology is intended to promote: share, reuse and communication between human and artificial agents (Milton et al., 2010).
It is worth mentioning that ontologies, since they are formal representations, permit: automatic interpretation of the domain model, inferencing via reasoners to derive new rules and querying via formal query languages.

For example, an ontology can describe a company according to different perspectives: static (components), kinematic (processes), dynamic (action rules), and the prospect of construction (composition, environment and structure) (Dietz, 2005), in order to bring more transparency to the system in general.

Ontologies allow, as stated by Oberle (2013), an interesting combination of technology features that are not permitted by other technologies, "There exists a plethora of technologies that offer the features of conceptual modelling but only ontologies combine this feature with web compliance, formality and reasoning possibilities".

Therefore, ontology is an essential artefact for the concretisation of enterprises semantic interoperability, where the important issue is the preservation of semantics across databases and applications. An example is product and service ontologies that have been developed for business-to-business applications in various market sectors and in which domain ontologies are used for property-parameterised product search and buying (Obrst, 2003; Rodrigues et al., 2013).

## 2.3 Business model ontology

According to Osterwalder (2004), research conducted in business models area addressed the lack of tools and concepts allowing managers to: capture, understand, communicate, design, analyse and modify the business logic of their companies. This statement highlights some of the important roles that can be played by business models knowledge representations, and therefore business model ontologies in order to enhance usefulness of business management in general.

The primary role of BMO is to represent key concepts related to business domain, such as customers or suppliers, as well as the relationships between these concepts.

Ontologies for business models are intended to represent the business model in a formal way that allows users or agents (humans and artificial) to understand its semantics and hence better communicate between them even if they belong to different communities or organisations. This understanding will potentially produce business value that can help in reacting more easily to external pressures.

An overview of literature on ontologies for business models shows that there are three main ontologies which tackled the description of enterprise business model in a formal way, namely: resource event agent (REA) ontology (Geerts and McCarthy, 1999), BMO (Osterwalder, 2004) and E3-value ontology (Gordijn and Akkermans, 2003b).

These ontologies have been compared to each other in several papers, such as Gordijn et al. (2005), Andersson et al. (2006) and Ilayperuma (2007) in which they were merged to develop a new reference ontology for business model.

# 3 Ontologies for business models

A bibliographic exploration in the literature on ontologies for business models highlights three ontologies related to our domain of interest: REA, BMO and E3-value.
It is worth mentioning that ontologies, since they are formal representations, permit: automatic interpretation of the domain model, inferencing via reasoners to derive new rules and querying via formal query languages.

For example, an ontology can describe a company according to different perspectives: static (components), kinematic (processes), dynamic (action rules), and the prospect of construction (composition, environment and structure) (Dietz, 2005), in order to bring more transparency to the system in general.

Ontologies allow, as stated by Oberle (2013), an interesting combination of technology features that are not permitted by other technologies, "There exists a plethora of technologies that offer the features of conceptual modelling but only ontologies combine this feature with web compliance, formality and reasoning possibilities".

Therefore, ontology is an essential artefact for the concretisation of enterprises semantic interoperability, where the important issue is the preservation of semantics across databases and applications. An example is product and service ontologies that have been developed for business-to-business applications in various market sectors and in which domain ontologies are used for property-parameterised product search and buying (Obrst, 2003; Rodrigues et al., 2013).

## 2.3 Business model ontology

According to Osterwalder (2004), research conducted in business models area addressed the lack of tools and concepts allowing managers to: capture, understand, communicate, design, analyse and modify the business logic of their companies. This statement highlights some of the important roles that can be played by business models knowledge representations, and therefore business model ontologies in order to enhance usefulness of business management in general.

The primary role of BMO is to represent key concepts related to business domain, such as customers or suppliers, as well as the relationships between these concepts.

Ontologies for business models are intended to represent the business model in a formal way that allows users or agents (humans and artificial) to understand its semantics and hence better communicate between them even if they belong to different communities or organisations. This understanding will potentially produce business value that can help in reacting more easily to external pressures.

An overview of literature on ontologies for business models shows that there are three main ontologies which tackled the description of enterprise business model in a formal way, namely: resource event agent (REA) ontology (Geerts and McCarthy, 1999), BMO (Osterwalder, 2004) and E3-value ontology (Gordijn and Akkermans, 2003b).

These ontologies have been compared to each other in several papers, such as Gordijn et al. (2005), Andersson et al. (2006) and Ilayperuma (2007) in which they were merged to develop a new reference ontology for business model.

# 3 Ontologies for business models

A bibliographic exploration in the literature on ontologies for business models highlights three ontologies related to our domain of interest: REA, BMO and E3-value.
## 3.1 Resource event agent

The structure of the ontology is based on the REA accounting model (McCarthy, 1982), it is commonly used for teaching in the case of accounting information systems, but it is less used in real commercial applications (Braccini, 2010).

The basic REA ontology is a stereotypical representation of an economic exchange as a basic economic phenomenon (Geerts and McCarthy, 2002). This exchange is executed between parts inside and outside the boundaries of a company and follows a particular object pattern (Geerts and McCarthy, 2002). In order to conceptualise this model, REA ontology provides a virtual representation of the reality of business through three fundamental concepts, namely:

1. resources, such as goods, services or money

2. events, such as business transactions or agreements that allocate resources

3. agents such as human actors or human organisations (like corporations)

The original REA model was extended by McCarthy and Geerts in a framework for the architecture of enterprise information systems and in an ontology for business systems (Geerts and McCarthy, 2002).

## 3.2 Business model ontology

In 2004, Osterwalder (2004) introduced ‘the business model ontology’ or BMO, in order to provide an approach to build business models by applying a more rigorous detailed and precise analysis of the business of a company. This work has been inspired by the balanced scorecards of Kaplan and Norton (1992) and of the work of Markides (1999).

<table><caption>**Table 1** The four business model pillars and building blocks of BMO</caption><thead><tr><th rowspan="2">Balanced scorecard</th><th rowspan="2">Markides</th><th colspan="2">Business model ontology</th></tr><tr><th>Pillar</th><th>Building blocks</th></tr></thead><tbody><tr><td>Innovation and learning perspective</td><td>What?</td><td>Product</td><td>Value proposition</td></tr><tr><td rowspan="3">Customer perspective</td><td rowspan="3">Who?</td><td rowspan="3">Customer interface</td><td>Target customer</td></tr><tr><td>Distribution channel</td></tr><tr><td>Relationship</td></tr><tr><td rowspan="3">Internal business perspective</td><td rowspan="3">How?</td><td rowspan="3">Infrastructure management</td><td>Value configuration</td></tr><tr><td>Capability</td></tr><tr><td>Partnership</td></tr><tr><td rowspan="2">Financial perspective</td><td rowspan="2">/</td><td rowspan="2">Financial aspects</td><td>Cost structure</td></tr><tr><td>Revenue model</td></tr></tbody></table>

Source: Osterwalder (2004)

Norton and Kaplan (Kaplan and Norton, 1992) identify four perspectives necessary to conduct successful business. In the customer perspective, the focus is on customer satisfaction. In the internal perspective, the focus is on company main activity. In the innovation and learning perspective, the focus is on future projects dedicated to development and creation of value. In the financial perspective, the focus is on how the
## 3.1 Resource event agent

The structure of the ontology is based on the REA accounting model (McCarthy, 1982), it is commonly used for teaching in the case of accounting information systems, but it is less used in real commercial applications (Braccini, 2010).

The basic REA ontology is a stereotypical representation of an economic exchange as a basic economic phenomenon (Geerts and McCarthy, 2002). This exchange is executed between parts inside and outside the boundaries of a company and follows a particular object pattern (Geerts and McCarthy, 2002). In order to conceptualise this model, REA ontology provides a virtual representation of the reality of business through three fundamental concepts, namely:

1. resources, such as goods, services or money

2. events, such as business transactions or agreements that allocate resources

3. agents such as human actors or human organisations (like corporations)

The original REA model was extended by McCarthy and Geerts in a framework for the architecture of enterprise information systems and in an ontology for business systems (Geerts and McCarthy, 2002).

## 3.2 Business model ontology

In 2004, Osterwalder (2004) introduced ‘the business model ontology’ or BMO, in order to provide an approach to build business models by applying a more rigorous detailed and precise analysis of the business of a company. This work has been inspired by the balanced scorecards of Kaplan and Norton (1992) and of the work of Markides (1999).

<table><caption>**Table 1** The four business model pillars and building blocks of BMO</caption><thead><tr><th rowspan="2">Balanced scorecard</th><th rowspan="2">Markides</th><th colspan="2">Business model ontology</th></tr><tr><th>Pillar</th><th>Building blocks</th></tr></thead><tbody><tr><td>Innovation and learning perspective</td><td>What?</td><td>Product</td><td>Value proposition</td></tr><tr><td rowspan="3">Customer perspective</td><td rowspan="3">Who?</td><td rowspan="3">Customer interface</td><td>Target customer</td></tr><tr><td>Distribution channel</td></tr><tr><td>Relationship</td></tr><tr><td rowspan="3">Internal business perspective</td><td rowspan="3">How?</td><td rowspan="3">Infrastructure management</td><td>Value configuration</td></tr><tr><td>Capability</td></tr><tr><td>Partnership</td></tr><tr><td rowspan="2">Financial perspective</td><td rowspan="2">/</td><td rowspan="2">Financial aspects</td><td>Cost structure</td></tr><tr><td>Revenue model</td></tr></tbody></table>

Source: Osterwalder (2004)

Norton and Kaplan (Kaplan and Norton, 1992) identify four perspectives necessary to conduct successful business. In the customer perspective, the focus is on customer satisfaction. In the internal perspective, the focus is on company main activity. In the innovation and learning perspective, the focus is on future projects dedicated to development and creation of value. In the financial perspective, the focus is on how the
company is evaluated by its owners. Osterwalder considers that these perspectives are suitable as a starting point for BMO.

Markides (1999) explains that a company should focus on ‘who’ it aims as clients? ‘What’ offer to its customers? And ‘how’ they should do to satisfy their customers? Osterwalder found that financial aspect are lacking in this proposal.

The following table lists the components of each pillar of BMO (Osterwalder, 2004) beside the balanced scorecard (Kaplan and Norton, 1992; Markides, 1999) vision.

The structure of BMO is obviously more complex than elements described in Table 1. Such a structure allows several types of alternative representations of a business model that is modelled using it. In fact, the above description is low in terms of granularity and can be much more detailed according to ontology application purposes.

### 3.3 E3-value

The e3-value ontology proposed by Gordijn and Akkermans (2001b) aims to identify the exchange of value objects between actors in a business case. The original idea subsequently evolved into a family of ontologies for business called e3-family, which is a set of ontological approaches for modelling value constellation network (Fielt, 2011).

The e3-value ontology includes some basic concepts and defines the relationships between them. Its main concepts are:

* the actors (an autonomous economic entity)

* the value objects (such as services, goods or related exchanged between actors)

* the value ports (actors use it to show his willingness to exchange value objects)

* the value interfaces (set of value ports offering or asking for value objects)

* the value exchange (a potential exchange of value objects between value ports).

All these constructs can be linked among each other's with different elements: dependency elements, connection elements, stimulus elements and value interfaces revisited (Braccini, 2010).

### 3.4 Comparison of business model ontologies

The previously mentioned ontologies have been compared by Gordijn et al. (2005) and Braccini (2010). Gordijn et al. (2005) used the four main components of a business model defined by Osterwalder (Figure 1) as a basis for this assessment as presented in Table 2.

It is clear that the structure of the business model represented by the BMO is much wider in scope and in structure compared to the business model representations of either the e3-value or the REA ontology

Moreover, Braccini compared them according to different dimensions (Table 3).

According to origin criterion, only the BMO has been designed for the area of research on business model, and is directly targeting this domain. The other ontologies have been created for different purposes but, due to their structure and to the constructs they include, they can also be used to represent business models.
company is evaluated by its owners. Osterwalder considers that these perspectives are suitable as a starting point for BMO.

Markides (1999) explains that a company should focus on ‘who’ it aims as clients? ‘What’ offer to its customers? And ‘how’ they should do to satisfy their customers? Osterwalder found that financial aspect are lacking in this proposal.

The following table lists the components of each pillar of BMO (Osterwalder, 2004) beside the balanced scorecard (Kaplan and Norton, 1992; Markides, 1999) vision.

The structure of BMO is obviously more complex than elements described in Table 1. Such a structure allows several types of alternative representations of a business model that is modelled using it. In fact, the above description is low in terms of granularity and can be much more detailed according to ontology application purposes.

### 3.3 E3-value

The e3-value ontology proposed by Gordijn and Akkermans (2001b) aims to identify the exchange of value objects between actors in a business case. The original idea subsequently evolved into a family of ontologies for business called e3-family, which is a set of ontological approaches for modelling value constellation network (Fielt, 2011).

The e3-value ontology includes some basic concepts and defines the relationships between them. Its main concepts are:

* the actors (an autonomous economic entity)

* the value objects (such as services, goods or related exchanged between actors)

* the value ports (actors use it to show his willingness to exchange value objects)

* the value interfaces (set of value ports offering or asking for value objects)

* the value exchange (a potential exchange of value objects between value ports).

All these constructs can be linked among each other's with different elements: dependency elements, connection elements, stimulus elements and value interfaces revisited (Braccini, 2010).

### 3.4 Comparison of business model ontologies

The previously mentioned ontologies have been compared by Gordijn et al. (2005) and Braccini (2010). Gordijn et al. (2005) used the four main components of a business model defined by Osterwalder (Figure 1) as a basis for this assessment as presented in Table 2.

It is clear that the structure of the business model represented by the BMO is much wider in scope and in structure compared to the business model representations of either the e3-value or the REA ontology

Moreover, Braccini compared them according to different dimensions (Table 3).

According to origin criterion, only the BMO has been designed for the area of research on business model, and is directly targeting this domain. The other ontologies have been created for different purposes but, due to their structure and to the constructs they include, they can also be used to represent business models.
<table><caption>**Table 2** Comparison of ontologies for business models according to Gordijn</caption><thead><tr><th>Constructs</th><th>BMO</th><th>e3-value</th><th>REA</th></tr></thead><tbody><tr><td>Offer related constructs</td><td>Value proposition</td><td>Value objects value interface value port</td><td>Resource</td></tr><tr><td>Customer related constructs</td><td>Target customer distribution channel relationship mechanism</td><td>Actor value exchange</td><td>Actor event</td></tr><tr><td>Network related constructs</td><td>Value configuration resources/core capability partnership agreement actor</td><td></td><td>Event</td></tr><tr><td>Financial related constructs</td><td>Revenue stream cost account</td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 3** Comparison of ontologies for business models according to Braccini</caption><thead><tr><th>Dimensions</th><th>BMO</th><th>e3-value</th><th>REA</th></tr></thead><tbody><tr><td>Origin</td><td>Business model research</td><td>E-business</td><td>Accounting</td></tr><tr><td>Theoretical perspective</td><td>Resource-based view (RBV)</td><td>Value network</td><td>Value chains</td></tr><tr><td>Value configuration</td><td>Chain/shop/network</td><td>Network</td><td>Chain</td></tr><tr><td>Focus</td><td>Internal</td><td>External</td><td>Internal</td></tr></tbody></table>

# 4 Case studies

To the best of our knowledge, there are a few case studies in the literature that have reported feedback evaluation of BMO on the ground (Zott et al., 2011). Nonetheless, we present in this section, three interesting real world cases of the use of ontologies for business models.

## 4.1 Online news provisioning

The business idea concretised in 2001 consisted of providing news articles to subscribers over internet (Gordijn and Akkermans, 2001a). The innovative aspect was financial, since connection fees were exploited to fund the service of online articles. This project was achieved on behalf of PCM Interactive Media, which is a subsidiary of a publisher of daily newspapers in the Netherlands.

The e3-value methodology (Gordijn, 2002), which is based on the ontology of the same name, was used to transform the previously mentioned basic business idea into an understandable and profitable business model. This business model was aimed to allow distinction between different value activities and to develop several forecasting variations of the model. Given these potential benefits, the project has been accepted by the leaders of PCM.

However, the limited use of this service and hence the lack of incomes, led PCM to stop the service. The analysis of this failure reported in Gordijn and Akkermans (2003a) shows that the e3-value methodology is not involved, and rather would have predicted the fiasco.
<table><caption>**Table 2** Comparison of ontologies for business models according to Gordijn</caption><thead><tr><th>Constructs</th><th>BMO</th><th>e3-value</th><th>REA</th></tr></thead><tbody><tr><td>Offer related constructs</td><td>Value proposition</td><td>Value objects value interface value port</td><td>Resource</td></tr><tr><td>Customer related constructs</td><td>Target customer distribution channel relationship mechanism</td><td>Actor value exchange</td><td>Actor event</td></tr><tr><td>Network related constructs</td><td>Value configuration resources/core capability partnership agreement actor</td><td></td><td>Event</td></tr><tr><td>Financial related constructs</td><td>Revenue stream cost account</td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 3** Comparison of ontologies for business models according to Braccini</caption><thead><tr><th>Dimensions</th><th>BMO</th><th>e3-value</th><th>REA</th></tr></thead><tbody><tr><td>Origin</td><td>Business model research</td><td>E-business</td><td>Accounting</td></tr><tr><td>Theoretical perspective</td><td>Resource-based view (RBV)</td><td>Value network</td><td>Value chains</td></tr><tr><td>Value configuration</td><td>Chain/shop/network</td><td>Network</td><td>Chain</td></tr><tr><td>Focus</td><td>Internal</td><td>External</td><td>Internal</td></tr></tbody></table>

# 4 Case studies

To the best of our knowledge, there are a few case studies in the literature that have reported feedback evaluation of BMO on the ground (Zott et al., 2011). Nonetheless, we present in this section, three interesting real world cases of the use of ontologies for business models.

## 4.1 Online news provisioning

The business idea concretised in 2001 consisted of providing news articles to subscribers over internet (Gordijn and Akkermans, 2001a). The innovative aspect was financial, since connection fees were exploited to fund the service of online articles. This project was achieved on behalf of PCM Interactive Media, which is a subsidiary of a publisher of daily newspapers in the Netherlands.

The e3-value methodology (Gordijn, 2002), which is based on the ontology of the same name, was used to transform the previously mentioned basic business idea into an understandable and profitable business model. This business model was aimed to allow distinction between different value activities and to develop several forecasting variations of the model. Given these potential benefits, the project has been accepted by the leaders of PCM.

However, the limited use of this service and hence the lack of incomes, led PCM to stop the service. The analysis of this failure reported in Gordijn and Akkermans (2003a) shows that the e3-value methodology is not involved, and rather would have predicted the fiasco.
In fact, the original business model has been radically altered after negotiations of PCM managers with their partners. In fact, the modest use of the online service is one of the various proposed assessment scenarios of the business model and therefore applied business model could not generate any direct or indirect income.

Clearly, this failure has been caused at least partially by modifications that occurred on the initial business idea. This negative overall result leads us to highlight the importance of supporting variations in all project phases when designing business model-based approaches. The lack of flexible technologies, that facilitates the management and the supervision of a newly developed business model, is in our opinion one of the main reasons of the aforementioned failure.

In order to overcome this limitation, e3-value designers; Gordijn and his team developed a tool: e3-value editor (http://www.e3value.com) based on the e3-value ontology that has subsequently been used in a European project business models in a world characterized by distributed generation (BUSMOD) (Gordijn and Akkermans, 2007).

## 4.2 Montreux Jazz festival

The MJF is an important festival launched in 1967 with nearly 240,000 visitors, 326 groups, and 1200 persons for the organisation in 2003.

A reliable demonstration of the applicability of BMO in real-world cases is the main objective of Osterwalder's (2004) doctoral thesis.

BMO provided an exhaustive description of each element of the festival, such as: value proposition, distribution channels, value configuration and clients.

Osterwalder (2004) adapted the syntax and the semantics of BMO (initially dedicated to be only a framework) into a representation written in XML language called BM²L, in order to generate reports and representations of the business model in different sizes and in different formats.

In fact, the ontology has not been applied directly in the festival business model; it has been presented to the festival manager who pointed out the expressional and analytical qualities of the model.

The use of BMO in the context of a real case proved the feasibility of the approach and facilitated the understanding of the related business model despite its complexity.

This case study shows that BMO is potentially applicable on the ground and it might represent a useful artefact for improving communication and understanding among actors of this festival.

However, since it has not been really experimented, no concrete conclusions on its added value could be drawn because they might be no more than theoretical assumptions.

## 4.3 Banking strategic partnership

This is a case study conducted in a bank 'X Bank' (Kort and Gordijn, 2008) represented in several countries around the world. The bank wanted to expand its services in countries where its direct presence is lacking. In order to solve this problem, it exploits partnerships with local banks.

Kort and Gordijn built a conceptual model for strategic partnership by extending the e3-value ontology. The newly developed concepts were about: equity investments, joint ventures, value activities conducted mutually and supply contracts.
In fact, the original business model has been radically altered after negotiations of PCM managers with their partners. In fact, the modest use of the online service is one of the various proposed assessment scenarios of the business model and therefore applied business model could not generate any direct or indirect income.

Clearly, this failure has been caused at least partially by modifications that occurred on the initial business idea. This negative overall result leads us to highlight the importance of supporting variations in all project phases when designing business model-based approaches. The lack of flexible technologies, that facilitates the management and the supervision of a newly developed business model, is in our opinion one of the main reasons of the aforementioned failure.

In order to overcome this limitation, e3-value designers; Gordijn and his team developed a tool: e3-value editor (http://www.e3value.com) based on the e3-value ontology that has subsequently been used in a European project business models in a world characterized by distributed generation (BUSMOD) (Gordijn and Akkermans, 2007).

## 4.2 Montreux Jazz festival

The MJF is an important festival launched in 1967 with nearly 240,000 visitors, 326 groups, and 1200 persons for the organisation in 2003.

A reliable demonstration of the applicability of BMO in real-world cases is the main objective of Osterwalder's (2004) doctoral thesis.

BMO provided an exhaustive description of each element of the festival, such as: value proposition, distribution channels, value configuration and clients.

Osterwalder (2004) adapted the syntax and the semantics of BMO (initially dedicated to be only a framework) into a representation written in XML language called BM²L, in order to generate reports and representations of the business model in different sizes and in different formats.

In fact, the ontology has not been applied directly in the festival business model; it has been presented to the festival manager who pointed out the expressional and analytical qualities of the model.

The use of BMO in the context of a real case proved the feasibility of the approach and facilitated the understanding of the related business model despite its complexity.

This case study shows that BMO is potentially applicable on the ground and it might represent a useful artefact for improving communication and understanding among actors of this festival.

However, since it has not been really experimented, no concrete conclusions on its added value could be drawn because they might be no more than theoretical assumptions.

## 4.3 Banking strategic partnership

This is a case study conducted in a bank 'X Bank' (Kort and Gordijn, 2008) represented in several countries around the world. The bank wanted to expand its services in countries where its direct presence is lacking. In order to solve this problem, it exploits partnerships with local banks.

Kort and Gordijn built a conceptual model for strategic partnership by extending the e3-value ontology. The newly developed concepts were about: equity investments, joint ventures, value activities conducted mutually and supply contracts.
To assess whether these extensions were useful for understanding strategic partnerships, the extended e3-value ontology has been experimented in a partnership between aforementioned bank ‘X Bank’ and a local bank ‘Y Bank’. The resulting business model submitted to banks’ managers have undergone several changes until a satisfactory model was found.

In fact, some aspects of the initially developed model were considered unnecessary or
too complex, such as: value networks, profitability of actors, value objects, value
interfaces and value transaction.

As a result, the developed e3-value-based business model predicted that this
partnership will be financially negative but qualitatively positive. In other terms,
‘X Bank’ will earn less money than ‘Y Bank’ but will enhance its quality of service.

This case illustrates the flexibility of an ontology for business models that has been
extended to meet the requirements of a new situation: the partnership.

Actually, E3-value is primarily an analytical tool that has permitted in this particular case, to avoid engagement in an unprofitable project for the ‘X Bank’.

## 4.4 Discussion

Development and deployment of technologies in enterprises are often time consuming and cost effective. Hence, the evaluation of technologies potential impact is essential for their applicability at a wider scale.

Through the study of the aforementioned cases, we attempted to deduce the influence of ontologies for business models in their respective application contexts.

Indeed, except for established ontology advantages: i.e., expressivity, communication and versioning, no measurable indications have been reported on the impact of the usage of ontologies for business models in all described cases.

From our point of view, the reasons are twofold, on one hand the ontology has not been experimented in a real world business model:

* The business model applied in ONP case is not the initially developed model. In fact, e3-value business model remained theoretical.

* The MJF model has been evaluated by festival managers after the festival and it has never been reported further experimentations of the BMO in subsequent festival editions.

* The extended e3-value model did not recommend partnership between ‘X bank’ and ‘Y bank’ but how to evaluate the impact of this prediction?

On the other hand, we found that task-based assessment of these ontologies was not the goal of these works, purposes were more specific, i.e.:

* The purpose of applying e3-value in ONP business model was to demonstrate the capabilities of e3-value in terms of predictability of business models scenarios.

* The purpose of applying BMO in MJF business model was to demonstrate the applicability of the ontology in a real world situation in terms of representativeness and expressivity.
To assess whether these extensions were useful for understanding strategic partnerships, the extended e3-value ontology has been experimented in a partnership between aforementioned bank ‘X Bank’ and a local bank ‘Y Bank’. The resulting business model submitted to banks’ managers have undergone several changes until a satisfactory model was found.

In fact, some aspects of the initially developed model were considered unnecessary or
too complex, such as: value networks, profitability of actors, value objects, value
interfaces and value transaction.

As a result, the developed e3-value-based business model predicted that this
partnership will be financially negative but qualitatively positive. In other terms,
‘X Bank’ will earn less money than ‘Y Bank’ but will enhance its quality of service.

This case illustrates the flexibility of an ontology for business models that has been
extended to meet the requirements of a new situation: the partnership.

Actually, E3-value is primarily an analytical tool that has permitted in this particular case, to avoid engagement in an unprofitable project for the ‘X Bank’.

## 4.4 Discussion

Development and deployment of technologies in enterprises are often time consuming and cost effective. Hence, the evaluation of technologies potential impact is essential for their applicability at a wider scale.

Through the study of the aforementioned cases, we attempted to deduce the influence of ontologies for business models in their respective application contexts.

Indeed, except for established ontology advantages: i.e., expressivity, communication and versioning, no measurable indications have been reported on the impact of the usage of ontologies for business models in all described cases.

From our point of view, the reasons are twofold, on one hand the ontology has not been experimented in a real world business model:

* The business model applied in ONP case is not the initially developed model. In fact, e3-value business model remained theoretical.

* The MJF model has been evaluated by festival managers after the festival and it has never been reported further experimentations of the BMO in subsequent festival editions.

* The extended e3-value model did not recommend partnership between ‘X bank’ and ‘Y bank’ but how to evaluate the impact of this prediction?

On the other hand, we found that task-based assessment of these ontologies was not the goal of these works, purposes were more specific, i.e.:

* The purpose of applying e3-value in ONP business model was to demonstrate the capabilities of e3-value in terms of predictability of business models scenarios.

* The purpose of applying BMO in MJF business model was to demonstrate the applicability of the ontology in a real world situation in terms of representativeness and expressivity.
* The purpose of applying the extended e3-value ontology in BSP business model was to demonstrate the potential of ontology support in strategic partnerships and hence prove ontology flexibility and predictability with regard to various scenarios.

Furthermore and beyond the described case studies, a broader and a more detailed benchmark is required in order to draw sound conclusions on the overall impact of ontologies for business models in a real world enterprise context.

# 5 Related work

Measuring the impact of ontologies for business models performance in enterprises belongs to the broader field of research on IT business value that examines the impact of information technology on organisational performance. Though, the assessment, quantification, or measure of the value created by IT remains extremely difficult (Nevo and Wade, 2010). Indeed, “The studies of IT value contribute descriptive knowledge in the form of theories about value generation through IT, but the literature lacks contributions to prescriptive knowledge that provide concrete orientations for measuring the economic value created by IT” (Vom Brocke et al., 2013).

This issue has been addressed among others by Melville et al. (2004) who proposed an economic value model of ITs founded on RBV or the RBV of the firm. This model is decomposed in three parts: focal firm, competitive environment and macro environment. Although the proposed model has been criticised for being too focused on the economic perspective, we consider it as relevant in the context of the present research work; in fact, the RBV model provides a robust framework for viewing the sources of competitive advantages within firms.

Nonetheless, our emphasis in the scope of this paper is on a particular category of IT, namely: ontologies.

According to Oberle (2013), the benefits of ontologies for enterprise applications are:

* new business scenarios

* increased productivity of information workers

* improved enterprise information management

* increased productivity of software engineering.

These benefits are sorted from the most reliable to the most hypothetical from Oberle's point of view. It is worth mentioning that these features have not yet been experimentally confirmed. He also provides a list of challenges that are likely to appear once ontologies are adopted in enterprises, namely:

* cost-benefit ratio

* training of employees

* recruitment of ontology experts

* demonstrate that ontology-based applications are more efficient than the other applications (how to measure the benefits?)
* The purpose of applying the extended e3-value ontology in BSP business model was to demonstrate the potential of ontology support in strategic partnerships and hence prove ontology flexibility and predictability with regard to various scenarios.

Furthermore and beyond the described case studies, a broader and a more detailed benchmark is required in order to draw sound conclusions on the overall impact of ontologies for business models in a real world enterprise context.

# 5 Related work

Measuring the impact of ontologies for business models performance in enterprises belongs to the broader field of research on IT business value that examines the impact of information technology on organisational performance. Though, the assessment, quantification, or measure of the value created by IT remains extremely difficult (Nevo and Wade, 2010). Indeed, “The studies of IT value contribute descriptive knowledge in the form of theories about value generation through IT, but the literature lacks contributions to prescriptive knowledge that provide concrete orientations for measuring the economic value created by IT” (Vom Brocke et al., 2013).

This issue has been addressed among others by Melville et al. (2004) who proposed an economic value model of ITs founded on RBV or the RBV of the firm. This model is decomposed in three parts: focal firm, competitive environment and macro environment. Although the proposed model has been criticised for being too focused on the economic perspective, we consider it as relevant in the context of the present research work; in fact, the RBV model provides a robust framework for viewing the sources of competitive advantages within firms.

Nonetheless, our emphasis in the scope of this paper is on a particular category of IT, namely: ontologies.

According to Oberle (2013), the benefits of ontologies for enterprise applications are:

* new business scenarios

* increased productivity of information workers

* improved enterprise information management

* increased productivity of software engineering.

These benefits are sorted from the most reliable to the most hypothetical from Oberle's point of view. It is worth mentioning that these features have not yet been experimentally confirmed. He also provides a list of challenges that are likely to appear once ontologies are adopted in enterprises, namely:

* cost-benefit ratio

* training of employees

* recruitment of ontology experts

* demonstrate that ontology-based applications are more efficient than the other applications (how to measure the benefits?)
* technical integration of the ontology (or several ontologies) in the enterprise environment for multiple users

* ontology modelling by non-ontology experts

* and the lack of tools intended to this type of developers.

From a theoretical perspective, it appears that measuring the impact of ontologies in enterprises can be achieved by calculating the ratio between ontology benefits and ontology drawbacks or the challenges that arise with the use of ontologies.

Simperl et al. (2006) proposed a model ontology cost model (ONTOCOM) to estimate parametric costs for ontologies by identifying relevant cost factors that have a direct impact on the efforts invested in the construction of the ontology. They evaluate the model a priori and a posteriori.

This model is interesting to consider as the financial aspect is important for the performance of the company. However, Bürger and Simperl (2008) identified that the complementarity of non-financial assessment with the previous model, is essential to create value for the company. They compared several assessment studies of the value of IT systems methods, the authors opted for an evaluation based on user satisfaction throughout a questionnaire.

Braccini (2010) sought to measure the impact of IT using ontologies. This work aimed to develop a method and a tool to identify the relationships between IT resources and activities that generate value by combining IT business value and the business model. He combined two ontologies: the ontology for linking processes and IT infrastructure (OLPIT) (Vom Brocke et al., 2009) and BMO (Osterwalder, 2004) because they complement each other.

The meta-model obtained was intended to identify, represent and communicate the relationship between IT resources and value generating activities in an organisation. The author demonstrates his model on a real-world case; however, the results must be viewed critically, because they might be different according to different business and/or IT resources.

Actually, to the best of our knowledge, Gordijn and Akkermans (2003a) were the only researchers that addressed the issue of whether the design of business models is really useful for enterprises. For this purpose, they applied their proposed approach on a real world business case, in order to develop, analyse and evaluate business models.

They demonstrate the capabilities of the ontology for business models for improving the understanding and the analysis of business models focusing mainly on the value of the company. However, the model originally designed by (Gordijn and Akkermans, 2003a) was not the one that was actually applied. Therefore, theoretical predictions have not been compared with practical results for a reliable evaluation.

# 6 Proposed approach for deriving the indicators

In the previous section, we highlighted that several attempts to measure the impact of IT artefacts on enterprises’ business models, such as ontologies, have failed either because they were theoretical or because they concern a unique case study that may not be generalisable to different situations. Therefore, we did not have sufficient information to assess this type of ontology.
* technical integration of the ontology (or several ontologies) in the enterprise environment for multiple users

* ontology modelling by non-ontology experts

* and the lack of tools intended to this type of developers.

From a theoretical perspective, it appears that measuring the impact of ontologies in enterprises can be achieved by calculating the ratio between ontology benefits and ontology drawbacks or the challenges that arise with the use of ontologies.

Simperl et al. (2006) proposed a model ontology cost model (ONTOCOM) to estimate parametric costs for ontologies by identifying relevant cost factors that have a direct impact on the efforts invested in the construction of the ontology. They evaluate the model a priori and a posteriori.

This model is interesting to consider as the financial aspect is important for the performance of the company. However, Bürger and Simperl (2008) identified that the complementarity of non-financial assessment with the previous model, is essential to create value for the company. They compared several assessment studies of the value of IT systems methods, the authors opted for an evaluation based on user satisfaction throughout a questionnaire.

Braccini (2010) sought to measure the impact of IT using ontologies. This work aimed to develop a method and a tool to identify the relationships between IT resources and activities that generate value by combining IT business value and the business model. He combined two ontologies: the ontology for linking processes and IT infrastructure (OLPIT) (Vom Brocke et al., 2009) and BMO (Osterwalder, 2004) because they complement each other.

The meta-model obtained was intended to identify, represent and communicate the relationship between IT resources and value generating activities in an organisation. The author demonstrates his model on a real-world case; however, the results must be viewed critically, because they might be different according to different business and/or IT resources.

Actually, to the best of our knowledge, Gordijn and Akkermans (2003a) were the only researchers that addressed the issue of whether the design of business models is really useful for enterprises. For this purpose, they applied their proposed approach on a real world business case, in order to develop, analyse and evaluate business models.

They demonstrate the capabilities of the ontology for business models for improving the understanding and the analysis of business models focusing mainly on the value of the company. However, the model originally designed by (Gordijn and Akkermans, 2003a) was not the one that was actually applied. Therefore, theoretical predictions have not been compared with practical results for a reliable evaluation.

# 6 Proposed approach for deriving the indicators

In the previous section, we highlighted that several attempts to measure the impact of IT artefacts on enterprises’ business models, such as ontologies, have failed either because they were theoretical or because they concern a unique case study that may not be generalisable to different situations. Therefore, we did not have sufficient information to assess this type of ontology.
Further, as discussed in Section 4.4, we could not address the issue of the task-based
assessment of ontologies for business models using the case study approach. Therefore,
we propose an approach (Figure 2) that identifies measurable indicators from a different
source to quantify the impact of these ontologies in the enterprise business model context.
This type of approach is supported by the statements of Peffers et al. (2007) who declared
that “depending on the nature of the problem venue and the artefact, evaluation could
take many forms. It could include objective quantitative performance measures, such as
budgets or items produced, the results of satisfaction surveys or simulations. It could
include quantifiable measures of system performance, such as response time or
availability”.

In fact, an indicator is defined as “any significant measure relative or not, used for
assessing achievements, resource utilization, the progress of the work or the external
context state” (National-Institute-of-Standards-and-Technology, 2011–2012).

<figure><img src="image_2.png" /><figcaption>**Figure 2** Approach for obtaining indicators intended to measure ontology use impact on enterprise business model</figcaption></figure>
Further, as discussed in Section 4.4, we could not address the issue of the task-based
assessment of ontologies for business models using the case study approach. Therefore,
we propose an approach (Figure 2) that identifies measurable indicators from a different
source to quantify the impact of these ontologies in the enterprise business model context.
This type of approach is supported by the statements of Peffers et al. (2007) who declared
that “depending on the nature of the problem venue and the artefact, evaluation could
take many forms. It could include objective quantitative performance measures, such as
budgets or items produced, the results of satisfaction surveys or simulations. It could
include quantifiable measures of system performance, such as response time or
availability”.

In fact, an indicator is defined as “any significant measure relative or not, used for
assessing achievements, resource utilization, the progress of the work or the external
context state” (National-Institute-of-Standards-and-Technology, 2011–2012).

<figure><img src="image_3.png" /><figcaption>**Figure 2** Approach for obtaining indicators intended to measure ontology use impact on enterprise business model</figcaption></figure>
<table><caption>**Table 4** Relevant criteria for business model ontologies</caption><thead><tr><th>Category</th><th>Article</th><th>Domain</th><th>Criterion</th></tr></thead><tbody><tr><td>Leadership</td><td>Senior leadership</td><td>Vision, values, and mission<br>Communication and organisational performance</td><td>Vision and values<br>Communication</td></tr><tr><td>Measurement, analysis, and knowledge management</td><td>Measurement, analysis, and improvement of organisational performance<br>Management of information, knowledge, and information technology</td><td>Performance measurement<br>Performance improvement<br>Data, information, and knowledge management</td><td>Performance measures<br>Best-practice sharing<br>Data and information availability<br>Knowledge management</td></tr><tr><td>Operations focus</td><td>Work systems</td><td>Work system design</td><td>Design concepts<br>Work system requirements</td></tr></tbody></table>

The proposed approach for deriving performance indicators for measuring the impact of the use of ontologies for business models in the management of a given company's business model, is to draw these indicators from a list of performance criteria that are standard or valid for a maximum of businesses. From this list, we retain the most concerned criteria by the use of ontologies for business models.

To the best of our knowledge, there is no list of performance indicators that is effective for all businesses, and the best starting point in our opinion, is provided by consulting and auditing organisations that exploit performance indicators lists in order to evaluate different types of corporate clients. Among these assessment structures, we use the one proposed by a non-profit and recognised US Government organisation, namely: The Malcolm Baldrige National Quality Award (http://www.nist.gov/baldrige/), which ranks and rewards every year the most successful US companies in the sectors of trade, health and education (D'Auteuil and Bonneau, 2003).

Of the 70 criteria proposed by this organisation divided into these categories: leadership, strategic planning, measurement-analysis-knowledge management and operations focus, we selected eight metrics that are more likely to be influenced by ontology usage from our point of view. These criteria are presented in Table 4.

For every assumed criterion we have to answer to two key questions, namely:

* How the criterion is influenced by the ontology usage? In order to justify its selection (see Table 5).

* How to measure this criterion? In order to deduce the corresponding indicator(s) (see Table 6).

In the following table, we carried out a mapping between the criteria impacted by ontology usage, as stated in the literature, with the criteria retained by the consulting firm that measure the performance of the audited organisation.

An effort for grouping them has been achieved in order to obtain a more manageable number of criteria.
<table><caption>**Table 4** Relevant criteria for business model ontologies</caption><thead><tr><th>Category</th><th>Article</th><th>Domain</th><th>Criterion</th></tr></thead><tbody><tr><td>Leadership</td><td>Senior leadership</td><td>Vision, values, and mission<br>Communication and organisational performance</td><td>Vision and values<br>Communication</td></tr><tr><td>Measurement, analysis, and knowledge management</td><td>Measurement, analysis, and improvement of organisational performance<br>Management of information, knowledge, and information technology</td><td>Performance measurement<br>Performance improvement<br>Data, information, and knowledge management</td><td>Performance measures<br>Best-practice sharing<br>Data and information availability<br>Knowledge management</td></tr><tr><td>Operations focus</td><td>Work systems</td><td>Work system design</td><td>Design concepts<br>Work system requirements</td></tr></tbody></table>

The proposed approach for deriving performance indicators for measuring the impact of the use of ontologies for business models in the management of a given company's business model, is to draw these indicators from a list of performance criteria that are standard or valid for a maximum of businesses. From this list, we retain the most concerned criteria by the use of ontologies for business models.

To the best of our knowledge, there is no list of performance indicators that is effective for all businesses, and the best starting point in our opinion, is provided by consulting and auditing organisations that exploit performance indicators lists in order to evaluate different types of corporate clients. Among these assessment structures, we use the one proposed by a non-profit and recognised US Government organisation, namely: The Malcolm Baldrige National Quality Award (http://www.nist.gov/baldrige/), which ranks and rewards every year the most successful US companies in the sectors of trade, health and education (D'Auteuil and Bonneau, 2003).

Of the 70 criteria proposed by this organisation divided into these categories: leadership, strategic planning, measurement-analysis-knowledge management and operations focus, we selected eight metrics that are more likely to be influenced by ontology usage from our point of view. These criteria are presented in Table 4.

For every assumed criterion we have to answer to two key questions, namely:

* How the criterion is influenced by the ontology usage? In order to justify its selection (see Table 5).

* How to measure this criterion? In order to deduce the corresponding indicator(s) (see Table 6).

In the following table, we carried out a mapping between the criteria impacted by ontology usage, as stated in the literature, with the criteria retained by the consulting firm that measure the performance of the audited organisation.

An effort for grouping them has been achieved in order to obtain a more manageable number of criteria.
Based on these criteria, we deduce the potential indicators that are as measurable (quantifiable) as possible.

The next step is to generate questions based on the derived indicators in order to collect answers or data that allow evaluators to measure with a relative precision the impact of the BMO in the business model of the considered organisation.

<table><caption>**Table 5** Correlation between selected criteria and ontology usage</caption><thead><tr><td>Ontology usage</td><td>Selected criterion</td><td>Criterion objective¹</td></tr></thead><tbody><tr><td>Ontologies are representations of particular viewpoints of real world knowledge of their developers (Chandrasekaran et al., 1999). So if decision makers are involved in the development of the ontology, the resulted representation will certainly reflect their visions and values.</td><td>Vision and values</td><td>How do leaders deploy company vision and values to all stakeholders as appropriate?</td></tr><tr><td>Ontology should facilitate communication among people and organisations to achieve interoperability (Ilayperuma, 2007); ontology should make domain assumptions explicit.</td><td>Communication common understanding</td><td>How communicate all stakeholders and make Frank exchange of opinions and ideas between them?</td></tr><tr><td>Formal representation of domain key concepts (Staab and Studer, 2009).</td><td>Representation of performance measures</td><td>What are indicators for tracking the achievement and effectiveness of business plans?</td></tr><tr><td>Save time and effort in building similar knowledge systems (Zemmouchi-Ghomari and Ghomari, 2012).</td><td>Sharing/best practice sharing reuse</td><td>How to share best practices and performance review findings?</td></tr><tr><td>Support retrieval and integration of information from different, distributed sources (Staab and Studer, 2009)</td><td>Data availability data retrieval</td><td>How do you make needed data available to all stakeholders as appropriate?</td></tr><tr><td>Provide background knowledge for query expansion or query rewriting (Bodner and Song, 1996)</td><td>Knowledge management</td><td>How do you manage collection and transfer of organisational knowledge?</td></tr><tr><td>Ontology for business models should represent the main aspects of the organisation's business model (Andersson et al., 2006): management infrastructure, financial aspects, relationship with clients and product value, so it should represent at least some work system requirements or needs.</td><td>Design concepts</td><td>How do you design your overall work systems?</td></tr><tr><td></td><td>Work system requirements</td><td>How do you determine key work system requirements, incorporating input from customers, suppliers, partners, and collaborators, as appropriate?</td></tr></tbody></table>

Note: According to the consulting firm.
Based on these criteria, we deduce the potential indicators that are as measurable (quantifiable) as possible.

The next step is to generate questions based on the derived indicators in order to collect answers or data that allow evaluators to measure with a relative precision the impact of the BMO in the business model of the considered organisation.

<table><caption>**Table 5** Correlation between selected criteria and ontology usage</caption><thead><tr><td>Ontology usage</td><td>Selected criterion</td><td>Criterion objective¹</td></tr></thead><tbody><tr><td>Ontologies are representations of particular viewpoints of real world knowledge of their developers (Chandrasekaran et al., 1999). So if decision makers are involved in the development of the ontology, the resulted representation will certainly reflect their visions and values.</td><td>Vision and values</td><td>How do leaders deploy company vision and values to all stakeholders as appropriate?</td></tr><tr><td>Ontology should facilitate communication among people and organisations to achieve interoperability (Ilayperuma, 2007); ontology should make domain assumptions explicit.</td><td>Communication common understanding</td><td>How communicate all stakeholders and make Frank exchange of opinions and ideas between them?</td></tr><tr><td>Formal representation of domain key concepts (Staab and Studer, 2009).</td><td>Representation of performance measures</td><td>What are indicators for tracking the achievement and effectiveness of business plans?</td></tr><tr><td>Save time and effort in building similar knowledge systems (Zemmouchi-Ghomari and Ghomari, 2012).</td><td>Sharing/best practice sharing reuse</td><td>How to share best practices and performance review findings?</td></tr><tr><td>Support retrieval and integration of information from different, distributed sources (Staab and Studer, 2009)</td><td>Data availability data retrieval</td><td>How do you make needed data available to all stakeholders as appropriate?</td></tr><tr><td>Provide background knowledge for query expansion or query rewriting (Bodner and Song, 1996)</td><td>Knowledge management</td><td>How do you manage collection and transfer of organisational knowledge?</td></tr><tr><td>Ontology for business models should represent the main aspects of the organisation's business model (Andersson et al., 2006): management infrastructure, financial aspects, relationship with clients and product value, so it should represent at least some work system requirements or needs.</td><td>Design concepts</td><td>How do you design your overall work systems?</td></tr><tr><td></td><td>Work system requirements</td><td>How do you determine key work system requirements, incorporating input from customers, suppliers, partners, and collaborators, as appropriate?</td></tr></tbody></table>

Note: According to the consulting firm.
We distinguish two types of indicators:

* Indicators related to representational aspects: specific to the BMO, hence the evaluators are mainly ontology developers and end-users.

* Indicators related to organisation's aspects: to be applied before and after using the ontology for business models, for example one or several fiscal years later. Preferably, this period has to be determined empirically. The evaluators are organisation's deciders and employees.

Obviously, the underneath list of questions is non-exhaustive and can be extended or adapted with regard to specific organisation's needs.

<table><caption>**Table 6** Indicators derived from criteria</caption><thead><tr><td>Criterion</td><td>Corresponding measurable indicators</td></tr></thead><tbody><tr><td>Vision and values</td><td>Consensus of all stakeholders (deciders and employees) on the representation.</td></tr><tr><td>Communication common understanding</td><td><ul><li>Process formalisation and standardisation</li><li>Results standardisation</li><li>Qualifications standardisation</li></ul>The above indicators are more or less significant according to activity sector and other criteria such as: organisation size and work complexity (Mintzberg, 1983)</td></tr><tr><td>Representation of performance measures</td><td>Representation of Key performance indicators, Kaplan and Norton (1992) that pertains to financial perspective (cash flow, working capital, investment capital), customer perspective (service quality, offer customisation), internal process (process quality), innovation and learning(access and reuse of project resources)</td></tr><tr><td>Sharing/best practice sharing</td><td>Relationship with customers (customer satisfaction)<br/>Time of tasks execution (more generally, personal productivity)</td></tr><tr><td>Data availability</td><td>Response time of the query to the knowledge base<br/>Knowledge base documentation</td></tr><tr><td>Data retrieval</td><td>Precision and recall of obtained responses (based on a gold standard)</td></tr><tr><td>Knowledge management and design concepts</td><td>Quality of the representation:<ul><li>Does the ontology represent correctly the organisation business model?</li><li>Does the ontology represent the performance indicators of the BM (management infrastructure, financial aspects, relationship with customers and product value)?</li></ul></td></tr><tr><td>Reuse</td><td>Modularity, documentation, standardisation of the representation (not specific)</td></tr><tr><td>Work system requirements</td><td>Quality of work processes, which role is to deliver customer value and achieve organisational success and sustainability</td></tr></tbody></table>
We distinguish two types of indicators:

* Indicators related to representational aspects: specific to the BMO, hence the evaluators are mainly ontology developers and end-users.

* Indicators related to organisation's aspects: to be applied before and after using the ontology for business models, for example one or several fiscal years later. Preferably, this period has to be determined empirically. The evaluators are organisation's deciders and employees.

Obviously, the underneath list of questions is non-exhaustive and can be extended or adapted with regard to specific organisation's needs.

<table><caption>**Table 6** Indicators derived from criteria</caption><thead><tr><td>Criterion</td><td>Corresponding measurable indicators</td></tr></thead><tbody><tr><td>Vision and values</td><td>Consensus of all stakeholders (deciders and employees) on the representation.</td></tr><tr><td>Communication common understanding</td><td><ul><li>Process formalisation and standardisation</li><li>Results standardisation</li><li>Qualifications standardisation</li></ul>The above indicators are more or less significant according to activity sector and other criteria such as: organisation size and work complexity (Mintzberg, 1983)</td></tr><tr><td>Representation of performance measures</td><td>Representation of Key performance indicators, Kaplan and Norton (1992) that pertains to financial perspective (cash flow, working capital, investment capital), customer perspective (service quality, offer customisation), internal process (process quality), innovation and learning(access and reuse of project resources)</td></tr><tr><td>Sharing/best practice sharing</td><td>Relationship with customers (customer satisfaction)<br/>Time of tasks execution (more generally, personal productivity)</td></tr><tr><td>Data availability</td><td>Response time of the query to the knowledge base<br/>Knowledge base documentation</td></tr><tr><td>Data retrieval</td><td>Precision and recall of obtained responses (based on a gold standard)</td></tr><tr><td>Knowledge management and design concepts</td><td>Quality of the representation:<ul><li>Does the ontology represent correctly the organisation business model?</li><li>Does the ontology represent the performance indicators of the BM (management infrastructure, financial aspects, relationship with customers and product value)?</li></ul></td></tr><tr><td>Reuse</td><td>Modularity, documentation, standardisation of the representation (not specific)</td></tr><tr><td>Work system requirements</td><td>Quality of work processes, which role is to deliver customer value and achieve organisational success and sustainability</td></tr></tbody></table>
<table><caption>**Table 7** Representational indicators</caption><thead><tr><td>Indicator</td><td>How to measure this indicator?</td></tr></thead><tbody><tr><td>Consensus of all stakeholders (deciders and employees) on the representation</td><td>We recommend here to apply ontology human assessment approach (Zemmouchi-Ghomari and Ghomari, 2013) that allows deriving natural language questions from the ontology. The obtained questionnaire is more adequate to non-ontologists evaluation and reflects the main components of an ontology.</td></tr><tr><td>Representation of Key performance indicators</td><td>Are main KPIs represented by the ontology? Due to the important number of KPIs, a core list of unescapable metrics has to be defined in order to facilitate this assessment.</td></tr><tr><td>Response time of the query to the knowledge base</td><td>Assuming that there is a gold standard of ideal answers according to a set of standard (frequent)queries</td></tr><tr><td>Precision and recall of obtained responses</td><td>What are the values of either precision and recall of the answers to these queries asked to the knowledge base (the instantiated ontology)?</td></tr><tr><td>Ontology documentation</td><td>Is the ontology easily understood? Does the ontology contain annotations of its several components, such as: definitions, comments, labels and versioning information? Is the ontology well documented?</td></tr></tbody></table>

<table><caption>**Table 8** Organisational indicators</caption><thead><tr><td>Indicator</td><td>How to measure this indicator?</td></tr></thead><tbody><tr><td>Process formalisation and standardisation (Mintzberg, 1983)</td><td>What referential of quality is adopted for the development of a service and/or a product?<br/>What is the production capacity?<br/>What is the rate of product rejects?<br/>What is the average time of product manufacturing?</td></tr><tr><td>Results standardisation (Mintzberg, 1983)</td><td>What is the precision degree of final product/service characteristics?</td></tr><tr><td>Qualifications standardisation (Mintzberg, 1983)</td><td>What training sessions have been made by employees?<br/>What competences are expected for a function occupied by an employee?<br/>What is the level of an employee autonomy in his work?</td></tr><tr><td>Relationship with clients (Carton, 2004)</td><td>What is the volume of: Sales, purchasing, number of clients; profit generated by a customer or a customer category?<br/>Does the company maintain lasting relationships with its customers?<br/>What the number is of attracted or earned clients?</td></tr><tr><td>Personal productivity (Michaels, 2009)</td><td>What is the level of the user autonomy? What is the mastering level of office tools?</td></tr></tbody></table>
<table><caption>**Table 7** Representational indicators</caption><thead><tr><td>Indicator</td><td>How to measure this indicator?</td></tr></thead><tbody><tr><td>Consensus of all stakeholders (deciders and employees) on the representation</td><td>We recommend here to apply ontology human assessment approach (Zemmouchi-Ghomari and Ghomari, 2013) that allows deriving natural language questions from the ontology. The obtained questionnaire is more adequate to non-ontologists evaluation and reflects the main components of an ontology.</td></tr><tr><td>Representation of Key performance indicators</td><td>Are main KPIs represented by the ontology? Due to the important number of KPIs, a core list of unescapable metrics has to be defined in order to facilitate this assessment.</td></tr><tr><td>Response time of the query to the knowledge base</td><td>Assuming that there is a gold standard of ideal answers according to a set of standard (frequent)queries</td></tr><tr><td>Precision and recall of obtained responses</td><td>What are the values of either precision and recall of the answers to these queries asked to the knowledge base (the instantiated ontology)?</td></tr><tr><td>Ontology documentation</td><td>Is the ontology easily understood? Does the ontology contain annotations of its several components, such as: definitions, comments, labels and versioning information? Is the ontology well documented?</td></tr></tbody></table>

<table><caption>**Table 8** Organisational indicators</caption><thead><tr><td>Indicator</td><td>How to measure this indicator?</td></tr></thead><tbody><tr><td>Process formalisation and standardisation (Mintzberg, 1983)</td><td>What referential of quality is adopted for the development of a service and/or a product?<br/>What is the production capacity?<br/>What is the rate of product rejects?<br/>What is the average time of product manufacturing?</td></tr><tr><td>Results standardisation (Mintzberg, 1983)</td><td>What is the precision degree of final product/service characteristics?</td></tr><tr><td>Qualifications standardisation (Mintzberg, 1983)</td><td>What training sessions have been made by employees?<br/>What competences are expected for a function occupied by an employee?<br/>What is the level of an employee autonomy in his work?</td></tr><tr><td>Relationship with clients (Carton, 2004)</td><td>What is the volume of: Sales, purchasing, number of clients; profit generated by a customer or a customer category?<br/>Does the company maintain lasting relationships with its customers?<br/>What the number is of attracted or earned clients?</td></tr><tr><td>Personal productivity (Michaels, 2009)</td><td>What is the level of the user autonomy? What is the mastering level of office tools?</td></tr></tbody></table>
## 7 Applying the indicators to the case studies

An important step in the evaluation of a proposed approach is its application to some case studies reported in the literature, as stated by Peffers et al. (2007).

However, we believe that to assess the use of BMO in a real-world organisation's business model, our study period should be spread over several years of measuring the proposed indicators before and after the use of this ontology to draw solid conclusions on their effective impact. Here, we refer to the indicators related to various aspects of the organisation, such as: process formalisation and standardisation or client satisfaction.

Regarding the first type of indicators proposed, namely, representational metrics; we attempt to assess their impact based on the elements describing these ontologies as reported in the relevant literature (Table 9).

The answers can have the following values: yes, no, partially or N/A (not available) combined respectively with these weightings: (1), (-1), (0.5) and (0).

<table><caption>**Table 9** Application of representational indicators to the cases</caption><thead><tr><td>Indicator/case study</td><td>ONP online news provisioning</td><td>MJF Montreux Jazz Festival</td><td>BSP banking strategic partnership</td></tr></thead><tbody><tr><td>Consensus of all stakeholders on the representation of the business model</td><td>No, the business model didn't reflect the adopted business model because the latter has been changed</td><td>Yes, BMO has been totally approved by festival organisers</td><td>Yes, The representation have undergone several changes until a satisfactory model was found</td></tr><tr><td>Representation of key performance indicators</td><td>Partially, according to ontologies comparison (Table 2), this ontology represents only offer and customer related constructs</td><td>Yes, according to ontologies comparison (Table 2), BMO has very good coverage of main KPIs</td><td>Partially, even if it is an extended version of e3-value, several aspects necessary to represent KPIs were pruned as explained in Section 3.3</td></tr><tr><td>Precision and recall of obtained responses to the queries</td><td>N/A, The ontology has neither been used nor queried</td><td>N/A, BMO has not been really experimented</td><td>N/A, this ontology has been used for a profitability prediction of a new situation (simulation only)</td></tr><tr><td>Ontology documentation</td><td>Yes, the e3-value is a very well documented ontology in several research papers (Gordijn and Akkermans, 2001b; Akkermans et al., 2004; Gordijn et al., 2006; Kartseva et al., 2009) and has its own website: http://e3value.few.vu.nl/</td><td>Yes, BMO is very well described in Osterwalder's doctoral thesis and other research papers (Osterwalder and Pigneur, 2010; Osterwalder, 2004; Gordijn et al., 2005; Hruby, 2006)</td><td>Yes, since it is an extended version of e3-value, we can attest that its documentation is extensively available</td></tr><tr><td>Overall</td><td>0.5</td><td>3</td><td>2.5</td></tr></tbody></table>
## 7 Applying the indicators to the case studies

An important step in the evaluation of a proposed approach is its application to some case studies reported in the literature, as stated by Peffers et al. (2007).

However, we believe that to assess the use of BMO in a real-world organisation's business model, our study period should be spread over several years of measuring the proposed indicators before and after the use of this ontology to draw solid conclusions on their effective impact. Here, we refer to the indicators related to various aspects of the organisation, such as: process formalisation and standardisation or client satisfaction.

Regarding the first type of indicators proposed, namely, representational metrics; we attempt to assess their impact based on the elements describing these ontologies as reported in the relevant literature (Table 9).

The answers can have the following values: yes, no, partially or N/A (not available) combined respectively with these weightings: (1), (-1), (0.5) and (0).

<table><caption>**Table 9** Application of representational indicators to the cases</caption><thead><tr><th>Indicator/case study</th><th>ONP online news provisioning</th><th>MJF Montreux Jazz Festival</th><th>BSP banking strategic partnership</th></tr></thead><tbody><tr><td>Consensus of all stakeholders on the representation of the business model</td><td>No, the business model didn't reflect the adopted business model because the latter has been changed</td><td>Yes, BMO has been totally approved by festival organisers</td><td>Yes, The representation have undergone several changes until a satisfactory model was found</td></tr><tr><td>Representation of key performance indicators</td><td>Partially, according to ontologies comparison (Table 2), this ontology represents only offer and customer related constructs</td><td>Yes, according to ontologies comparison (Table 2), BMO has very good coverage of main KPIs</td><td>Partially, even if it is an extended version of e3-value, several aspects necessary to represent KPIs were pruned as explained in Section 3.3</td></tr><tr><td>Precision and recall of obtained responses to the queries</td><td>N/A, The ontology has neither been used nor queried</td><td>N/A, BMO has not been really experimented</td><td>N/A, this ontology has been used for a profitability prediction of a new situation (simulation only)</td></tr><tr><td>Ontology documentation</td><td>Yes, the e3-value is a very well documented ontology in several research papers (Gordijn and Akkermans, 2001b; Akkermans et al., 2004; Gordijn et al., 2006; Kartseva et al., 2009) and has its own website: http://e3value.few.vu.nl/</td><td>Yes, BMO is very well described in Osterwalder's doctoral thesis and other research papers (Osterwalder and Pigneur, 2010; Osterwalder, 2004; Gordijn et al., 2005; Hruby, 2006)</td><td>Yes, since it is an extended version of e3-value, we can attest that its documentation is extensively available</td></tr><tr><td>Overall</td><td>0.5</td><td>3</td><td>2.5</td></tr></tbody></table>
If we consider that all indicators are equal in terms of weightings, it appears that BMO in MJF has been the most beneficial from the perspective of representativeness compared to the other ontologies in the context of the case studies.

Many more case studies are needed to establish an overall that can be considered as a threshold of an acceptable level of representativeness concerning this type of ontology.

## 8 Conclusions and future work

An overview of the most popular ontologies for business models and some case studies of their applications in enterprise business models have helped us establish that business modelling using ontologies is a useful approach to understand and analyse a business model.

Nonetheless, this does not allow us to make robust assumptions on the real impact of the usefulness and application of these ontologies.

Specifically, in most cases, the ontology has not been firmly integrated into the business model; consequently, the conclusions drawn were mainly theoretical even if they were globally positive.

In addition, the purpose of the case studies was to target specific aspects of ontology usability, such as their representativeness or flexibility. This was therefore not useful in determining their overall impact on business model ontologies.

It is clear that more practical case studies are required to assess how business model ontologies can actually be applied.

Even so, this is an ongoing study that aims to produce a non-exhaustive list of indicators for measuring the impact of business model ontologies in the enterprise context. Such indicators should provide a framework to facilitate the assessment of these types of ontologies when used in real world business models.

We start with a list of performance criteria that are valid for many businesses – such as the ones used by auditing firms – and we retain some that can potentially measure the impact of BMO in the context of an economic organisation.

To justify the selection of these criteria, we mapped the selected criteria to the potential use of the BMO. We then deduced measurable metrics for each criterion that allow us to quantify the influence of the ontology use in an organisational business model.

There are two types of indicators: indicators related to representational aspects or specific to the BMO, and indicators related to organisational aspects that would be assessed before and after using business model ontologies. The necessary time period could be determined empirically.

Regarding the representational indicators, we assessed the impact of described ontologies based on the facts reported in the literature to demonstrate their applicability.

As part of our ongoing research, for assessing the applicability of organisational indicators, we propose the use of multi-agent systems technology to simulate the BMO in the framework of an organisation business model. In other words, we intend to examine, IT artefacts (ontology) using artificial data, as recommended in the experimental category of the evaluation in the design science research methodology, as there are a growing number of agent-based simulation applications (North and Macal, 2007; Macal and North, 2011) across a variety of domains and disciplines.
If we consider that all indicators are equal in terms of weightings, it appears that BMO in MJF has been the most beneficial from the perspective of representativeness compared to the other ontologies in the context of the case studies.

Many more case studies are needed to establish an overall that can be considered as a threshold of an acceptable level of representativeness concerning this type of ontology.

## 8 Conclusions and future work

An overview of the most popular ontologies for business models and some case studies of their applications in enterprise business models have helped us establish that business modelling using ontologies is a useful approach to understand and analyse a business model.

Nonetheless, this does not allow us to make robust assumptions on the real impact of the usefulness and application of these ontologies.

Specifically, in most cases, the ontology has not been firmly integrated into the business model; consequently, the conclusions drawn were mainly theoretical even if they were globally positive.

In addition, the purpose of the case studies was to target specific aspects of ontology usability, such as their representativeness or flexibility. This was therefore not useful in determining their overall impact on business model ontologies.

It is clear that more practical case studies are required to assess how business model ontologies can actually be applied.

Even so, this is an ongoing study that aims to produce a non-exhaustive list of indicators for measuring the impact of business model ontologies in the enterprise context. Such indicators should provide a framework to facilitate the assessment of these types of ontologies when used in real world business models.

We start with a list of performance criteria that are valid for many businesses – such as the ones used by auditing firms – and we retain some that can potentially measure the impact of BMO in the context of an economic organisation.

To justify the selection of these criteria, we mapped the selected criteria to the potential use of the BMO. We then deduced measurable metrics for each criterion that allow us to quantify the influence of the ontology use in an organisational business model.

There are two types of indicators: indicators related to representational aspects or specific to the BMO, and indicators related to organisational aspects that would be assessed before and after using business model ontologies. The necessary time period could be determined empirically.

Regarding the representational indicators, we assessed the impact of described ontologies based on the facts reported in the literature to demonstrate their applicability.

As part of our ongoing research, for assessing the applicability of organisational indicators, we propose the use of multi-agent systems technology to simulate the BMO in the framework of an organisation business model. In other words, we intend to examine, IT artefacts (ontology) using artificial data, as recommended in the experimental category of the evaluation in the design science research methodology, as there are a growing number of agent-based simulation applications (North and Macal, 2007; Macal and North, 2011) across a variety of domains and disciplines.
## References

Akkermans, H., Baida, Z., Gordijn, J., Peña, N., Altuna, A. and Laresgoiti, I. (2004) ‘Value webs: ontology-based bundling of real-world services’, *IEEE Intelligent Systems*, Vol. 19, No. 44, pp.23–32.

Al-Debei, M.M. and Avison, D. (2010) ‘Developing a unified framework of the business model concept’, *European Journal of Information Systems*, Vol. 19, No. 3, pp.359–376.

Al-Debei, M.M. and Fitzgerald, G. (2010) ‘The design and engineering of mobile data services: developing an ontology based on business model thinking’, in J. Pries-Heje, J. Venable and J. De Gross (Eds.): *Human Benefits Through the Diffusion of Information Systems Design Science Research*, Vol. 318, pp.28–51, Springer, Boston.

Andersson, B., Bergholtz, M., Edirisuriya, A., Ilayperuma, T., Johannesson, P., Gordijn, J., Grégoire, B., Schmitt, M., Dubois, E., Abels, S., Hahn, A., Wangler, B. and Weigand, H. (2006) ‘Towards a reference ontology for business models’, *Proceedings of 25th International Conference on Conceptual Modeling*, Tucson, AZ, USA, pp.482–496.

Asharaf, S. and Abdulla, M.S. (2013) ‘Scope of ontological annotation in e-commerce’, *International Journal of Business Information Systems*, Vol. 14, No. 3, pp.335–347, Inderscience Publisher.

Bodner, R.C. and Song, F. (1996) *Knowledge-based Approaches to Query Expansion in Information Retrieval*, pp.146–158, Springer, Berlin, Heidelberg.

Braccini, A.M. (2010) *On the Impact of IT on Value Generating Activities in Organisations: An Ontology Based Approach*, Unpublished PhD thesis, Université LUISS Guido Carli, Department of Business and Management, Rome.

Bürger, T. and Simperl, E. (2008) ‘Measuring the benefits of ontologies’, *On the Move to Meaningful Internet Systems: OTM 2008 Workshops*.

Burkhart, T., Krumeich, J., Werth, D. and Loos, P. (2011) ‘Analyzing the business model concept – a comprehensive classification of literature’, *Proceedings of the 32nd International Conference on Information Systems*, Shanghai, China.

Chandrasekaran, B., Josephson, J.R. and Benjamins, V.R. (1999) ‘What are ontologies, and why do we need them?’, *Intelligent Systems and their Applications*, Vol. 14, No. 1, pp.20–26, IEEE.

Chesbrough, H. (2010) ‘Business model innovation: opportunities and barriers’, long range planning – business models’, Vol. 43, No. 2, pp.354–363, Elsevier.

Chesbrough, H. and Rosenbloom, R.S. (2002) ‘The role of the business model in capturing value from innovation: evidence from Xerox Corporation’s technology spin-off companies’, *Industrial and Corporate Change*, Vol. 11, No. 3, pp.529–555.

D’Auteuil, C. and Bonneau, C. (2003) *Guide sur les indicateurs, modernisation de la gestion publique*, Rapport du Secrétariat du Conseil du trésor, Québec, Canada.

Dietz, J.L.G. (2005) ‘System ontology and its role in software development’, *Advanced Information Systems Engineering Wokshops, EMOI-INTEROP*, Vol. 5.

Dietz, J.L.G. (2006) *Enterprise Ontology*, Springer, Berlin, Heidelberg.

Fielt, E. (2011) *Understanding Business Models*, Business Service Management, Whitepaper, Vol. 3, Smart Services CRC, Brisbane, Australia.

Geerts, G.L. and McCarthy, W.E. (1999) *Use of an Accounting Object Infrastructure for Knowledge-Based Enterprise Models*, Association for the Advancement of Artificial Intelligence (AAAI) Technical Report.

Geerts, G.L. and McCarthy, W.E. (2002) ‘An ontological analysis of the economic primitives of the extended-REA enterprise information architecture’, *International Journal of Accounting Information Systems*, Vol. 3, No. 1, pp.1–16.

Gordijn, J. (2002) *Value-based Requirements Engineering: Exploring Innovative E-Commerce Ideas*, Unpublished PhD thesis, Vrije Universiteit: Amsterdam.
## References

Akkermans, H., Baida, Z., Gordijn, J., Peña, N., Altuna, A. and Laresgoiti, I. (2004) ‘Value webs: ontology-based bundling of real-world services’, *IEEE Intelligent Systems*, Vol. 19, No. 44, pp.23–32.

Al-Debei, M.M. and Avison, D. (2010) ‘Developing a unified framework of the business model concept’, *European Journal of Information Systems*, Vol. 19, No. 3, pp.359–376.

Al-Debei, M.M. and Fitzgerald, G. (2010) ‘The design and engineering of mobile data services: developing an ontology based on business model thinking’, in J. Pries-Heje, J. Venable and J. De Gross (Eds.): *Human Benefits Through the Diffusion of Information Systems Design Science Research*, Vol. 318, pp.28–51, Springer, Boston.

Andersson, B., Bergholtz, M., Edirisuriya, A., Ilayperuma, T., Johannesson, P., Gordijn, J., Grégoire, B., Schmitt, M., Dubois, E., Abels, S., Hahn, A., Wangler, B. and Weigand, H. (2006) ‘Towards a reference ontology for business models’, *Proceedings of 25th International Conference on Conceptual Modeling*, Tucson, AZ, USA, pp.482–496.

Asharaf, S. and Abdulla, M.S. (2013) ‘Scope of ontological annotation in e-commerce’, *International Journal of Business Information Systems*, Vol. 14, No. 3, pp.335–347, Inderscience Publisher.

Bodner, R.C. and Song, F. (1996) *Knowledge-based Approaches to Query Expansion in Information Retrieval*, pp.146–158, Springer, Berlin, Heidelberg.

Braccini, A.M. (2010) *On the Impact of IT on Value Generating Activities in Organisations: An Ontology Based Approach*, Unpublished PhD thesis, Université LUISS Guido Carli, Department of Business and Management, Rome.

Bürger, T. and Simperl, E. (2008) ‘Measuring the benefits of ontologies’, *On the Move to Meaningful Internet Systems: OTM 2008 Workshops*.

Burkhart, T., Krumeich, J., Werth, D. and Loos, P. (2011) ‘Analyzing the business model concept – a comprehensive classification of literature’, *Proceedings of the 32nd International Conference on Information Systems*, Shanghai, China.

Chandrasekaran, B., Josephson, J.R. and Benjamins, V.R. (1999) ‘What are ontologies, and why do we need them?’, *Intelligent Systems and their Applications*, Vol. 14, No. 1, pp.20–26, IEEE.

Chesbrough, H. (2010) ‘Business model innovation: opportunities and barriers’, long range planning – business models’, Vol. 43, No. 2, pp.354–363, Elsevier.

Chesbrough, H. and Rosenbloom, R.S. (2002) ‘The role of the business model in capturing value from innovation: evidence from Xerox Corporation’s technology spin-off companies’, *Industrial and Corporate Change*, Vol. 11, No. 3, pp.529–555.

D’Auteuil, C. and Bonneau, C. (2003) *Guide sur les indicateurs, modernisation de la gestion publique*, Rapport du Secrétariat du Conseil du trésor, Québec, Canada.

Dietz, J.L.G. (2005) ‘System ontology and its role in software development’, *Advanced Information Systems Engineering Wokshops, EMOI-INTEROP*, Vol. 5.

Dietz, J.L.G. (2006) *Enterprise Ontology*, Springer, Berlin, Heidelberg.

Fielt, E. (2011) *Understanding Business Models*, Business Service Management, Whitepaper, Vol. 3, Smart Services CRC, Brisbane, Australia.

Geerts, G.L. and McCarthy, W.E. (1999) *Use of an Accounting Object Infrastructure for Knowledge-Based Enterprise Models*, Association for the Advancement of Artificial Intelligence (AAAI) Technical Report.

Geerts, G.L. and McCarthy, W.E. (2002) ‘An ontological analysis of the economic primitives of the extended-REA enterprise information architecture’, *International Journal of Accounting Information Systems*, Vol. 3, No. 1, pp.1–16.

Gordijn, J. (2002) *Value-based Requirements Engineering: Exploring Innovative E-Commerce Ideas*, Unpublished PhD thesis, Vrije Universiteit: Amsterdam.
Gordijn, J. and Akkermans, H. (2001a) ‘Designing and evaluating e-business models’, *IEEE Intelligent Systems*, Vol. 16, No. 4, pp.11–17.

Gordijn, J. and Akkermans, H. (2001b) ‘Ontology-based operators for e-business model de- and re-construction’, *Proceedings of the First International Conference on Knowledge Capture*, Canada, 21–23 October, pp.60–67.

Gordijn, J. and Akkermans, H. (2003a) ‘Does e-business modeling really help?’, *International Conference on System Sciences, Proceedings of the 36th Annual Hawaii – HICSS36*, p.10.

Gordijn, J. and Akkermans, J.M. (2003b) ‘Value-based requirements engineering: exploring innovative e-commerce ideas’, *Requirements Engineering Journal*, Vol. 8, No. 2, pp.114–134.

Gordijn, J. and Akkermans, H. (2007) ‘Business models for distributed generation in a liberalized market environment’, *Electric Power Systems Research Journal*, Vol. 77, No. 9, pp.1178–1188.

Gordijn, J., Osterwalder, A. and Pigneur, Y. (2005) ‘Comparing two business model ontologies for designing e-business models and value constellations’, *Proceedings of BLED2005 Conference*, Bled, Slovenia, pp.6–8.

Gordijn, J., Yu, E. and van der Raadt, B. (2006) ‘E-service design using i* and e/sup 3/value modeling’, *Software*, Vol. 23, No. 3, pp.26–33, IEEE.

Hevner, A.R., March, S.T., Park, J. and Ram, S. (2004) ‘Design science in information systems research’, *MIS Quarterly*, Vol. 28, No. 1, pp.75–105.

Hruby, P. (2006) *Model-Driven Design Using Business Patterns*, Springer-Verlag, New York, USA.

Ilayperuma, T. (2007) *Reference Ontology for Business Models: Towards Interoperability between Business Modelling Ontologies*, Unpublished PhD thesis, Université de Stockholm, Department of Computer and Systems Sciences, Suède.

Kaplan, R.S. and Norton, D.P. (1992) ‘The balanced scorecard – measures that drive performance’, *Harvard Business Review*, January–February, pp.71–79.

Kartseva, V., Gordijn, J. and Tan, Y.H. (2009) ‘Designing value-based inter-organizational controls using patterns’, in *Design Requirements Engineering: A Ten-Year Perspective*, pp.276–301, Springer, Berlin, Heidelberg.

Kort, C. and Gordijn, J. (2008) ‘Modeling strategic partnerships using the e3-value ontology: a field study in the banking industry’, *Handbook of Ontologies for Business Interaction*.

Macal, C.M. and North, M.J. (2011) ‘Introductory tutorial: agent-based modeling and simulation’, *The Winter Simulation Conference (WSC)*, pp.1451–1464.

Markides, C. (1999) *All the Right Moves: A Guide to Crafting Breakthrough Strategy*, Harvard Business School Press, Boston.

McCarthy, W.E. (1982) ‘The REA accounting model: a generalized framework for accounting systems in a shared data environment’, *The Accounting Review*, Vol. 57, No. 3, pp.554–578.

Melville, N., Kraemer, K. and Gurbaxani, V. (2004) ‘Review: information technology and organizational performance: an integrative model of IT business value’, *Journal MIS Quarterly*, Vol. 28, No. 2, pp.283–322.

Michaels, T. (2009) *Handbook of Research on Web 2.0 and Second Language Learning*, Information Science Reference, IGI Global, Pennsylvania, USA.

Milton, S.K., Keen, C.D. and Kurnia, S. (2010) ‘Understanding the benefits of ontology use for Australian industry: a conceptual study’, *Australasian Conference on Information Systems, ACIS 2010 Proceedings*, Brisbane, Australia.

Mintzberg, H. (1983) *Structure in Fives: Designing Effective Organizations*, Prentice Hall, Inc., New Jersey, USA.

Morris, M., Schindehutte, M. and Allen, J. (2005) ‘The entrepreneur’s business model: toward a unified perspective’, *Journal of Business Research*, Vol. 58, No. 6, pp.726–735.
Gordijn, J. and Akkermans, H. (2001a) ‘Designing and evaluating e-business models’, *IEEE Intelligent Systems*, Vol. 16, No. 4, pp.11–17.

Gordijn, J. and Akkermans, H. (2001b) ‘Ontology-based operators for e-business model de- and re-construction’, *Proceedings of the First International Conference on Knowledge Capture*, Canada, 21–23 October, pp.60–67.

Gordijn, J. and Akkermans, H. (2003a) ‘Does e-business modeling really help?’, *International Conference on System Sciences, Proceedings of the 36th Annual Hawaii – HICSS36*, p.10.

Gordijn, J. and Akkermans, J.M. (2003b) ‘Value-based requirements engineering: exploring innovative e-commerce ideas’, *Requirements Engineering Journal*, Vol. 8, No. 2, pp.114–134.

Gordijn, J. and Akkermans, H. (2007) ‘Business models for distributed generation in a liberalized market environment’, *Electric Power Systems Research Journal*, Vol. 77, No. 9, pp.1178–1188.

Gordijn, J., Osterwalder, A. and Pigneur, Y. (2005) ‘Comparing two business model ontologies for designing e-business models and value constellations’, *Proceedings of BLED2005 Conference*, Bled, Slovenia, pp.6–8.

Gordijn, J., Yu, E. and van der Raadt, B. (2006) ‘E-service design using i* and e/sup 3/value modeling’, *Software*, Vol. 23, No. 3, pp.26–33, IEEE.

Hevner, A.R., March, S.T., Park, J. and Ram, S. (2004) ‘Design science in information systems research’, *MIS Quarterly*, Vol. 28, No. 1, pp.75–105.

Hruby, P. (2006) *Model-Driven Design Using Business Patterns*, Springer-Verlag, New York, USA.

Ilayperuma, T. (2007) *Reference Ontology for Business Models: Towards Interoperability between Business Modelling Ontologies*, Unpublished PhD thesis, Université de Stockholm, Department of Computer and Systems Sciences, Suède.

Kaplan, R.S. and Norton, D.P. (1992) ‘The balanced scorecard – measures that drive performance’, *Harvard Business Review*, January–February, pp.71–79.

Kartseva, V., Gordijn, J. and Tan, Y.H. (2009) ‘Designing value-based inter-organizational controls using patterns’, in *Design Requirements Engineering: A Ten-Year Perspective*, pp.276–301, Springer, Berlin, Heidelberg.

Kort, C. and Gordijn, J. (2008) ‘Modeling strategic partnerships using the e3-value ontology: a field study in the banking industry’, *Handbook of Ontologies for Business Interaction*.

Macal, C.M. and North, M.J. (2011) ‘Introductory tutorial: agent-based modeling and simulation’, *The Winter Simulation Conference (WSC)*, pp.1451–1464.

Markides, C. (1999) *All the Right Moves: A Guide to Crafting Breakthrough Strategy*, Harvard Business School Press, Boston.

McCarthy, W.E. (1982) ‘The REA accounting model: a generalized framework for accounting systems in a shared data environment’, *The Accounting Review*, Vol. 57, No. 3, pp.554–578.

Melville, N., Kraemer, K. and Gurbaxani, V. (2004) ‘Review: information technology and organizational performance: an integrative model of IT business value’, *Journal MIS Quarterly*, Vol. 28, No. 2, pp.283–322.

Michaels, T. (2009) *Handbook of Research on Web 2.0 and Second Language Learning*, Information Science Reference, IGI Global, Pennsylvania, USA.

Milton, S.K., Keen, C.D. and Kurnia, S. (2010) ‘Understanding the benefits of ontology use for Australian industry: a conceptual study’, *Australasian Conference on Information Systems, ACIS 2010 Proceedings*, Brisbane, Australia.

Mintzberg, H. (1983) *Structure in Fives: Designing Effective Organizations*, Prentice Hall, Inc., New Jersey, USA.

Morris, M., Schindehutte, M. and Allen, J. (2005) ‘The entrepreneur’s business model: toward a unified perspective’, *Journal of Business Research*, Vol. 58, No. 6, pp.726–735.
National-Institute-of-Standards-and-Technology (2011–2012) *Criteria for Performance Excellence*, Baldrige Performance Excellence Program, Gaithersburg, USA.

Nevo, S. and Wade, M.R. (2010) ‘The formation and value of IT-enabled resources: antecedents and consequence of synergistic relationships’, *MIS Quarterly*, Vol. 34, No. 1, pp.163–83.

North, M.J. and Macal, C.M. (2007) *Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation*, Oxford University Press, Oxford, UK.

Oberle, D. (2013) *How Ontologies Benefit Enterprise Applications*, Semantic Web, IOS Press, Amsterdam, The Netherlands.

Obrst, L. (2003) ‘Ontologies for semantically interoperable systems’, *The International Conference on Information and Knowledge Management, CIKM ’03*, ACM Press, New York, NY, USA, pp.366–369.

Osterwalder, A. (2004) *The Business Model Ontology – A Proposition in a Design Science Approach*, Unpublished PhD thesis, Ecole des Hautes Etudes Commerciales, Université de Lausanne.

Osterwalder, A. and Pigneur, Y. (2010) *Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers*, John Wiley & Sons, New Jersey, USA.

Osterwalder, A., Ondrus, J. and Pigneur, Y. (2005) *Skype’s Disruptive Potential in the Telecom Market: A Systematic Comparison of Business Models*, University of Lausanne, Information Systems Institute, Lausanne, Working Paper.

Peffers, K., Tuunanen, T., Rothenberger, M.a. and Chatterjee, S. (2007) ‘A design science research methodology for information systems research’, *Journal of Management Information Systems*, Vol. 24, No. 3, pp.45–77.

Rodrigues, J.M., Schulz, S., Rector, A., Spackman, K., Ustün, B., Chute, C.G. and Kalra, D. (2013) ‘Do we need a common ontology between ICD 11 and SNOMED CT to ensure seamless re-use and semantic interoperability?’, Vol. 192, p.1234, *Studies in Health Technology and Informatics Series*, IOS Press, Amsterdam, the Netherlands.

Routray, S. and Mishra, S. (2009) ‘Designing a value-added e-business model through information aggregation: an Indian case study’, *International Journal of Business Information Systems*, Vol. 4, No. 3, pp.311–323, Inderscience Publisher.

Shafer, S.M., Smith, H.J. and Linder, J.C. (2005) ‘The power of business models’, *Business Horizons*, Vol. 48, No. 3, pp.199–207.

Simperl, E.P.B., Tempich, C. and Sure, Y. (2006) ‘Ontocom: a cost estimation model for ontology engineering’, *The Semantic Web-ISWC 2006*, pp.625–639.

Staab, S. and Studer, R. (2009) *Handbook on Ontologies*, Springer Berlin Heidelberg, Germany.

Studer, R., Benjamins, V.R. and Fensel, D. (1998) ‘Knowledge engineering: principles and methods’, *Data & Knowledge Engineering*, Vol. 25, No. 1, pp.161–197.

Timmers, P. (1998) ‘Business models for electronic markets’, *Electronic Markets*, Vol. 8, No. 2, pp.3–8.

Vom Brocke, J., Braccini, A. M., Sonnenberg, C. and Spagnoletti, P. (2013) ‘Living IT infrastructures – an ontology-based approach to aligning IT infrastructure capacity and business needs’, *International Journal of Accounting Information Systems*, in press [online] http://dx.doi.org/10.1016/j.accinf.2013.10.004 (accessed 26 April 2014).

Vom Brocke, J., Braccini, A.M., Sonnenberg, C. and Ender, E. (2009) ‘A process oriented assessment of the IT infrastructure value: a proposal of an ontology based approach’, in *Business Information Systems*, pp.13–24, Springer, Berlin, Heidelberg.

Yasin, M.M., Czuchry, A.J., Gonzales, M. and Bayes, P.E. (2006) ‘E-commerce implementation challenges: small to medium-sized versus large organisations’, *International Journal of Business Information Systems*, Vol. 1, No. 3, pp.256–275, Inderscience Publisher.

Zemmouchi-Ghomari, L. and Ghomari, A.R. (2012) ‘Ontology versus terminology from the perspective of ontologists’, *International Journal of Web Science*, Vol. 1, No. 4, pp.315–331, Inderscience Publisher.
National-Institute-of-Standards-and-Technology (2011–2012) *Criteria for Performance Excellence*, Baldrige Performance Excellence Program, Gaithersburg, USA.

Nevo, S. and Wade, M.R. (2010) ‘The formation and value of IT-enabled resources: antecedents and consequence of synergistic relationships’, *MIS Quarterly*, Vol. 34, No. 1, pp.163–83.

North, M.J. and Macal, C.M. (2007) *Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation*, Oxford University Press, Oxford, UK.

Oberle, D. (2013) *How Ontologies Benefit Enterprise Applications*, Semantic Web, IOS Press, Amsterdam, The Netherlands.

Obrst, L. (2003) ‘Ontologies for semantically interoperable systems’, *The International Conference on Information and Knowledge Management, CIKM ’03*, ACM Press, New York, NY, USA, pp.366–369.

Osterwalder, A. (2004) *The Business Model Ontology – A Proposition in a Design Science Approach*, Unpublished PhD thesis, Ecole des Hautes Etudes Commerciales, Université de Lausanne.

Osterwalder, A. and Pigneur, Y. (2010) *Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers*, John Wiley & Sons, New Jersey, USA.

Osterwalder, A., Ondrus, J. and Pigneur, Y. (2005) *Skype’s Disruptive Potential in the Telecom Market: A Systematic Comparison of Business Models*, University of Lausanne, Information Systems Institute, Lausanne, Working Paper.

Peffers, K., Tuunanen, T., Rothenberger, M.a. and Chatterjee, S. (2007) ‘A design science research methodology for information systems research’, *Journal of Management Information Systems*, Vol. 24, No. 3, pp.45–77.

Rodrigues, J.M., Schulz, S., Rector, A., Spackman, K., Ustün, B., Chute, C.G. and Kalra, D. (2013) ‘Do we need a common ontology between ICD 11 and SNOMED CT to ensure seamless re-use and semantic interoperability?’, Vol. 192, p.1234, *Studies in Health Technology and Informatics Series*, IOS Press, Amsterdam, the Netherlands.

Routray, S. and Mishra, S. (2009) ‘Designing a value-added e-business model through information aggregation: an Indian case study’, *International Journal of Business Information Systems*, Vol. 4, No. 3, pp.311–323, Inderscience Publisher.

Shafer, S.M., Smith, H.J. and Linder, J.C. (2005) ‘The power of business models’, *Business Horizons*, Vol. 48, No. 3, pp.199–207.

Simperl, E.P.B., Tempich, C. and Sure, Y. (2006) ‘Ontocom: a cost estimation model for ontology engineering’, *The Semantic Web-ISWC 2006*, pp.625–639.

Staab, S. and Studer, R. (2009) *Handbook on Ontologies*, Springer Berlin Heidelberg, Germany.

Studer, R., Benjamins, V.R. and Fensel, D. (1998) ‘Knowledge engineering: principles and methods’, *Data & Knowledge Engineering*, Vol. 25, No. 1, pp.161–197.

Timmers, P. (1998) ‘Business models for electronic markets’, *Electronic Markets*, Vol. 8, No. 2, pp.3–8.

Vom Brocke, J., Braccini, A. M., Sonnenberg, C. and Spagnoletti, P. (2013) ‘Living IT infrastructures – an ontology-based approach to aligning IT infrastructure capacity and business needs’, *International Journal of Accounting Information Systems*, in press [online] http://dx.doi.org/10.1016/j.accinf.2013.10.004 (accessed 26 April 2014).

Vom Brocke, J., Braccini, A.M., Sonnenberg, C. and Ender, E. (2009) ‘A process oriented assessment of the IT infrastructure value: a proposal of an ontology based approach’, in *Business Information Systems*, pp.13–24, Springer, Berlin, Heidelberg.

Yasin, M.M., Czuchry, A.J., Gonzales, M. and Bayes, P.E. (2006) ‘E-commerce implementation challenges: small to medium-sized versus large organisations’, *International Journal of Business Information Systems*, Vol. 1, No. 3, pp.256–275, Inderscience Publisher.

Zemmouchi-Ghomari, L. and Ghomari, A.R. (2012) ‘Ontology versus terminology from the perspective of ontologists’, *International Journal of Web Science*, Vol. 1, No. 4, pp.315–331, Inderscience Publisher.
Zemmouchi-Ghomari, L. and Ghomari, A.R. (2013) ‘A new approach for human assessment of ontologies’, in *Proceedings of the Third International Conference of Information Systems and Technologies, ICIST’2013*, Tangier, Morocco.

Zott, C. and Amit, R. (2010) ‘Business model design: an activity system perspective’, *Long Range Planning*, Vol. 43, No. 2, pp.216–226.

Zott, C., Amit, R. and Massa, L. (2011) ‘The business model: recent developments and future research’, *Journal of Management*, Vol. 37, No. 4, pp.1019–1042.
Zemmouchi-Ghomari, L. and Ghomari, A.R. (2013) ‘A new approach for human assessment of ontologies’, in *Proceedings of the Third International Conference of Information Systems and Technologies, ICIST’2013*, Tangier, Morocco.

Zott, C. and Amit, R. (2010) ‘Business model design: an activity system perspective’, *Long Range Planning*, Vol. 43, No. 2, pp.216–226.

Zott, C., Amit, R. and Massa, L. (2011) ‘The business model: recent developments and future research’, *Journal of Management*, Vol. 37, No. 4, pp.1019–1042.