<figure><img src="image_1.png" /></figure>

HAL
open science

# Ensuring Quality of Large Scale Industrial Process Collections: Experiences from a Case Study

Merethe Heggset, John Krogstie, Harald Wesenberg

► To cite this version:

Merethe Heggset, John Krogstie, Harald Wesenberg. Ensuring Quality of Large Scale Industrial Process Collections: Experiences from a Case Study. 7th IFIP Working Conference on The Practice of Enterprise Modeling (PoEM), Nov 2014, Manchester, United Kingdom. pp.11-25, 10.1007/978-3-662-45501-2_2. hal-01281965

HAL Id: hal-01281965

https://inria.hal.science/hal-01281965

Submitted on 3 Mar 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Ensuring Quality of Large Scale Industrial Process Collections: Experiences from a Case Study

Merethe Heggset¹, John Krogstie¹ and Harald Wesenberg²

¹Norwegian University of Science & Technology - NTNU, Norway
merethhe@gmail.com, krogstie@idi.ntnu.no

²Statoil ASA, Norway
hwes@statoil.com

**Abstract:** As approaches and tools for process and enterprise modelling are maturing, these techniques are in an increasing number of organizations being taken into use on a large scale. In this paper we report on the use over many years of process-modelling in connection to the quality system of Statoil, a large Norwegian oil-company, in particular on the aspects found necessary to emphasis to achieve the right quality of the models in this organisation. The Statoil-guidelines for enterprise structure and use of standard notation are mapped to the levels of SEQUAL, a generic framework for understanding the quality of models. Guidelines for modelling are found on most levels. More detailed guidelines than in general work on quality of business process models are found in particular on the physical, empirical, and syntactic level, where the number of detailed guidelines in Statoil has increased over the years due to needs identified.

**Keywords:** Enterprise process modelling, case study, experience paper

## 1 Introduction

Statoil is a global oil company headquartered in Norway. It has more than 20.000 employees in more than 30 countries worldwide, and has spent significant resources for enterprise modelling over the years. They report to have achieved a fair success with enterprise modelling in its corporate management system [29] where workflow models are used extensively to communicate requirements and best practices throughout the enterprise. The current management system contains more than 2000 business process and workflow models with associated requirements and best practices, all available through a corporate web portal from anywhere in the company. The models are used daily in large parts of the organization, and are a significant contributor in reducing operational, environmental and safety risks. As an example, the important SIF-index (Serious Injury Frequency) which counts the number of incidents per million
work hours has been reduced from 6 to around 0.8 in the period since the models where introduced. Every week Statoil employees perform approximately 2 million work hours. That said the process model is only one approach to risk mitigation. One also experience that the process models could be utilized even better.

From the start, Statoil has been aware of the need to balance different levels of quality of the models. According to [26, 30] Statoil have found that it makes sense to talk about three dimensions of model quality: Syntactic quality (how well the model uses the modelling language), semantic quality (how well the model reflects the real world) and pragmatic quality (how well the model is understood by the target audience), building upon distinctions first described in [12], which is a predecessor to the current SEQUAL framework on quality of models and modelling languages [6]. In enterprise models the balance between these dimensions becomes very important based on the goal of modelling; else the model will not be used by its intended target audience in the right way.

Enterprise models being part of a quality system exists over a longer period of time, and is distributed widely throughout the enterprise [30]. Enterprise models thus must be managed properly. They need to be subject to strict versioning routines, configuration management practices and release plans. In many ways enterprise models used in this manner are similar to source code, and should be subject to similar professional practices. If the models are not managed properly they will not be trusted and they will subsequently fail to achieve their full potential as enterprise models.

To manage the development and evolution of their enterprise process models, Statoil has developed detailed requirements for modelling. We have in this case study looked upon these requirements in the light of the SEQUAL-framework for understanding quality of models. Other frameworks for evaluating quality of process models exist including Guidelines of Modeling (GoM) and Quality of Modeling (QoMo). GoM is focused on managing the subjectivism involved when building models. The framework consists of two dimensions; one for the range of model use, and one for the degree of precision or concretion [22]. QoMo is focused on knowledge state transitions, cost management, and goal structure [2]. Our choice of SEQUAL rests on that Statoil has already used a subset of SEQUAL. Quite detailed overviews of quality of process models in the light of SEQUAL exist [7] and the aim of this experience paper is to illustrate the additional level of detail on such guidelines that is needed to gain value from these kinds of models in professional practice. The case study has taken as outset different version of official modelling guidelines/requirements in Statoil, and mapped this to SEQUAL by an expert of the framework, partly by document study, and partly in interaction with the developers of requirements in Statoil. General background on modelling and SEQUAL is provided in section 2. In section 3 we describe the Statoil quality system in more detail, before we in section 4 describe the mapping of the Statoil modelling guidelines to SEQUAL. Ideas on further work on understanding the trade-off on different quality aspects are found in section 5.
## 2 Background on Modeling and Quality of Models

According to general model theory [24] there are three common characteristics of models: *Representation*, *Simplification* and *Pragmatic orientation*.

* *Representation:* Models represents something else.

* *Simplification:* Models possess a reductive trait in that they map only a subset of attributes of the phenomenon being modelled.

* *Pragmatic orientation:* Models have a substitutive function in that they substitute a certain phenomenon as being conceptualized by a certain subject in a given temporal space with a certain *incentive* or operation in mind

Thus a model is not just a representation of something else, it is a conscious construction. Enterprise process modelling is always done in some organizational setting. One can look upon an organization and its information system abstractly to be in a state (the current state, often represented as a descriptive 'as-is' model) that are often to be evolved to some future wanted state (represented as a prescriptive 'to be' model).

The state includes the existing processes, organization and computer systems. These states are often modelled, and the state of the organization is perceived (differently) by different persons through these models. Different usage areas of conceptual models as described in [6, 19] are:

1. Human sense-making: The descriptive model of the current state can be useful for people to make sense of and learn about the current perceived situation.

2. Communication between people in the organization: Models can have an important role in human communication. Thus, in addition to support the sense-making process for the individual, a model can act as a common framework supporting communication both relative to descriptive and prescriptive models.

3. Computer-assisted analysis: This is used to gain knowledge about the organization through simulation or deduction, often by comparing a model of the current state and a model of a future, potentially better state.

4. Quality assurance, ensuring compliance e.g. that the organization acts according to a certified process developed for instance as part of an ISO-certification process.

5. Model deployment and activation: To integrate the model of the future state in a new information system directly. Models can be activated in three ways:

a. Through people, (manual activation) with no active tool-support.

b. Automatically, where the system plays an active role, as in most automated workflow systems.

c. Interactively, where the computer and the users co-operate in bringing the process forward.

6. To be a prescriptive model to be used in a traditional system development project, without being directly activated.

SEQUAL [6] is a framework for assessing and understanding the quality of models and modelling languages. It builds on early work on quality of model [12, 15], but has been extended based on theoretical results [16, 17, 20] and practical experiences [6, 10] with the original framework. It has earlier been used for evaluation of modelling and modelling languages of a large number of perspectives, including data [8],
ontologies [3], process [7, 11], enterprise [9], topological [18] and goal-oriented modelling [4, 5]. Quality has been defined referring to the correspondence between statements belonging to the following sets:

* G, the set of goals of the modelling task.

* D, the domain, i.e., the set of all statements that can be stated about the situation. The goal of modelling typically restricts the domain to only those things relevant to achieve this goal

* L, the language extension, i.e. what can be expressed by the modelling language used.

* M, the externalized model itself.

* K, the explicit knowledge that the audience (both modelers and model interpreters) have of the domain.

* I, the social actor (human) interpretation of the model

* T, the technical actor (tool) interpretation of the model

The main quality types are:

* Physical quality: The basic quality goal is that the externalized model *M* is available to the relevant actors (and not others) for interpretation (*I* and *T*).

* Empirical quality deals with comprehensibility of the model *M*.

* Syntactic quality is the correspondence between the model *M* and the language extension *L*. Is the language used correctly in the model?

* Semantic quality is the correspondence between the model *M* and the domain *D*.

* Perceived semantic quality is the similar correspondence between the social actor interpretation *I* of a model *M* and his or hers current knowledge *K* of domain *D*.

* Pragmatic quality is the correspondence between the model *M* and the actor interpretation (*I* and *T*) of it. Thus whereas empirical quality focus on if the model is understandable according to some objective measure that has been discovered empirically in e.g., cognitive science, we at this level look on to what extend the model has actually been understood.

* The goal defined for social quality is agreement among social actor's interpretations of the models.

* The deontic quality of the model relates to that all statements in the model *M* contribute to fulfilling the goals of modelling *G*, and that all the goals of modelling *G* are addressed through the model *M*.

When we structure different quality aspects according to these levels, one will find that there might be conflicts between the levels (e.g. what is good for semantic quality might be bad for pragmatic quality), thus to make a trade-off between achieving the different quality levels is important for achieving the main goals of modelling.

## 3 Case-environment - Statoil Quality Management System

The Statoil Management System is "the set of principles, policies, processes and requirement which support our organization in fulfilling the tasks required to achieve
our goals" [27]. It defines how work is done within the company, and all employees
are required to act according to relevant governing documentation.

The Management System consists of three main parts:

* ARIS, the modelling solution from which all governing documentation is accessed by the end users.

* Docmap, used for handling and publishing textual governing documentation

* Disp, a tool which supports the process of handling applications for deviation permits in cases where compliance with a requirement is difficult or impossible to achieve.

The three main objectives of the Statoil Management System are

1. Contributing to safe, reliable and efficient operations and enabling compliance with external and internal requirements

2. Helping the company incorporating their values, people and leadership principles into everything they do

3. Supporting business performance through high-quality decision-making, fast and precise execution and continuous learning

Governing documentation describes what is to be achieved, how to execute tasks,
and ensures standardization. Each process area has governing documentation in the
form of documents and/or process models, accessible from the Management System
start page on the Statoil intranet. This is kept up to date by the management system
function [28]. The main purpose of the management system function is to ensure that:

* The management system is developed and improved based on learning and business needs.

* Governing documentation is understood and implemented.

* Compliance with requirements is monitored.

This is done in close collaboration with line management and owners of governing
documentation. The governing documentation is managed in a systematic five step
cycle: Assess and plan, design, implement, use, and monitor and control.

The enterprise process model is created according to a set of rules for structuring
and use of notation, and can be used for a variety of purposes, such as compliance
management, competence management, portfolio management, decision making and
performance analysis. There are three levels of abstraction in the enterprise model:
The contextual level, the conceptual level and the logical level, including the follow-
ing interrelated diagrams:

* The top-level diagram as a mandatory navigational diagram visualizing core value chain processes, management processes, and support processes, capturing what they in Statoil term the contextual level. This is similar to what others have termed a process map [13].

* The navigation diagram(s) are optional diagrams to support more tailored access to the processes than the top-level diagram.
* Model diagram: Is a mandatory diagram that visualizes the model of one process area in the organization.

* Process navigation diagram is an optional model for navigational support on the conceptual level.

* Workflow diagram - BPMN models [23] on the logical levels.

When designing diagrams in the enterprise model, requirements in TR0002 - Enterprise structure and standard notation [25] shall be met. We will look at these in more detail in the next section as they map to the levels of SEQUAL, with focus on the guidelines for development of workflow models on the logical level.

# 4 Development of the Understanding of Quality of Models in Statoil

As indicated in the introduction, modelling has been used for a number of years in Statoil. The requirements for modelling to achieve a balance of syntactic, semantic and pragmatic quality has through this period evolved based on concrete needs identified through the quality cycle described above. Thus, although we here in particular look upon the current requirements (Version 3, valid from Dec. 5 2013) [25] we also look on the development, in particular relative to version 1 of the requirements, that was made available Feb. 12 2009[26]. Although the levels of syntactic, semantic, and pragmatic quality are emphasized, the existing requirements are not structured according to these levels. As we will see, also other levels of SEQUAL are relevant, partly since the original SEQUAL-categories have been divided in sub-areas in the later versions of the framework (e.g. splitting pragmatic quality into empirical quality (for aspects that at least in theory can be fully evaluated objectively by a tool) and pragmatic quality for aspects of understanding that has to take the human interpreter into account). Looking first at the sets of SEQUAL in the light of the case of the Statoil management system, we have the following:

* M: The models we look upon here are in particular the workflow-model part of the overall model-framework. Relative to the description of purpose of modelling in Section 2 the models are meant to be as-is models, to support communication on the current process, manual activation (i.e. supporting human action in the organization according to the models), and checking of compliance (area 4: quality assurance).

* G: Whereas the general requirements for the quality system was described in Section 3, five main more concrete usage areas are listed as:

1. Compliance management: To monitor and control that the way of working is compliant with the standards set for the way to work. This enables producing predictable output from work.

2. Competence management: Document the competency profiles needed to perform tasks, compare required competency profiles with competence represented in the organization, and therefore manage the competency gap.

3. Portfolio management: Gain an overview of the current portfolio of e.g. processes, information systems, and technologies. This gives opportunities for ana-
lyzing whether the existing portfolio will meet future needs, and to plan the roadmap to get from the current to the future portfolio.

4. Analysis and decision making: The model and its subsets enables analysis of the relationships between different objects in the models and how changes to one object (e.g. a process) will impact other objects (e.g. the information systems used by that process or relations between different work processes)

5. Performance analysis: Monitoring of these results to get experience and data on the quality. This information can be used to analyze if the way of working produce the best possible result.

Even if several possible purposes are listed, one model always has one primary purpose, with potentially a (set of) secondary purpose. The current primary purpose of the enterprise model is compliance management therefore the priority is given on achieving the right quality of governing documentation models with corresponding governing elements, roles and responsibilities. We notice that two of these goals were not in version 1 of the requirements (competency management and performance analysis). Rather than being an example of 'goal creep' (that models used for one purpose over time is used also for other things not originally envisioned [10]) it is because the models have to be current as-is models (due to focus on compliance). First recently the underlying infrastructure to support competency management and performance analysis has been put in production.

* D: Domain: The work processes in Statoil.

* L: The language for workflow modelling is a subset of BPMN2.0. In the original version of the requirements [26] it was a similar sub-set of BPMN 1.

* A: The target audience comes from the whole company. It is therefore necessary to do a stakeholder analyses to ensure that models have the right abstraction level, complexity, terminology suitable for the target audience.

* K: The relevant explicit knowledge of the actors (A).

* T: The tool currently used is ARIS. We note that two other tools (APOS and QLM/BPM) were used in version 1 of the requirements, since ARIS was introduced at a later stage.

* I: Relates to how easy it is for the different actors to interpret the data as it can be presented in ARIS.

## 4.1 Physical Quality

Physical quality relates to if the model is:

* Available to the right people in a physical form (through the ARIS-tool) when needed for interpretation.

* People are able to find the right model (e.g. through navigation and search), knowing if all relevant parts of the model is found.

* Availability of both the current and previous versions of the model.

* Possibility to store relevant meta-data e.g., on purpose and validity (what part of the organization the model is valid for).
* Only available for those that should have access in case of there being security aspects.

Each governing documentation model and governing element shall have only one documented, published, and valid version that is properly numbered. Old versions must be kept available though. There are two types of updates of governing documentation models and other governing elements: regular and minor.

To support the storage and presentation of models, ARIS is used. Some guidelines for how to use ARIS are described (the usage of the different aspects are described under other quality levels as appropriate).

In ARIS, before publishing you shall select the relevant increase option. Only regular update increases trigger the publication workflow mechanism. Once the update is approved the system automatically increases the value and publishes the model or an element with the new version number.

In ARIS the information regarding the deviation handling process shall be given in the field "Deviation Permit". The following options are available:

* Level 1: Owner acceptance and line manager approval required

* Level 2: Line manager approval required

The descriptive field "Validity" is used to provide information about who the model applies to, using a validity register. The validity register is used to store and maintain a list of locations and organizational entities.

The purpose of the governing documentation model shall be represented by the "Purpose" attribute in work process model and basic document model

## 4.2 Empirical Quality

We here focus on naming and language conventions described in [25]. Few concrete guidelines for graph layout are provided in the Statoil requirements.

**General naming conventions:** To ensure a common naming practice across the management system, one shall use names according to the following set of rules:

* Names on symbols and expressions shall be formulated in singular form.

* Avoid names with more than four words if possible.

* A name shall not be a detailed description.

* The first letter in the first word of a symbol name shall be in upper case. All other letters shall be in lower case.

* Proper names shall start with upper case letters.

* If the same concept has several alternative names, the Statoil official name shall be used in the models. The other names are synonyms and can be presented as such if suitable for the business.

* Abbreviations should be avoided.

* Avoid names starting with: 'receive', 'send', 'manage', and 'process', as they do not reflect value added.
In documents with additional information there are also guidelines on the lan-
guage. Whereas one in [6] mainly mention the use of readability indexes as a tech-
nique here, the Statoil guidelines mention a broader set of guidelines.

* Address the reader - write rather 'When you submit...' than 'When someone submits...'.

* Use words and phrasings familiar to all users - e.g. 'present' rather than 'prevailing'.

* Ensure that content is sufficiently explained.

* Mindfully use the word “focus” - rather than write 'The purpose of this information is to focus on safety' write 'The purpose of this information is to highlight safety'.

* Apply negative confirmation. When the reader expect to find content in the text, but no content exist, apply negative conformation by using 'not applicable' or 'none'.

* Use active sentences - write 'The process owner representatives handle improvement suggestions' rather than 'Improvement suggestions are handled in the Process Owner dimension'.

* Use verbs (do not use heavy nouns) - write' when the role actor complied with the specific requirement....' rather than 'When there has not been a deviation to the specific requirement by the role actor...'.

* Organize your message content. Extract information by using verbs and pronouns. Divide information into suitably sized pieces and use periods wherever possible (cf. readability index). Postpone restrictions and additions to the next sentence.

* Use lists where possible.

Process role name shall consist of following elements:

* Use process role name that is qualified against the RACI-principle (Responsible, Accountable, Consulted, Informed).

* Write competence or focus area as a free text.

The name of a work process role must be a noun in the singular form or an expres-
sion in the singular form starting with a noun or an adjective. The process role name
shall be written in full, no abbreviations shall be used. The process role name shall
contain no organization units or terms indicating organization of tasks or services. The
process role can exist only once within one work process.

In addition to its name, a process role is characterized through its role description and competence requirements, which represent more detailed description of the competency profile. Process roles are represented in the workflow diagram through a lane. The lane shall have as minimum the identifier attribute. The identifier is necessary when linking the role to one or more actors.

The role description and competence requirements of specific process role are de-
scribed and visualized through an attribute in a form of descriptive text connected to
the lane. From earlier work, we know that the lane-concept in BPMN is overloaded
[21], thus having detailed guidelines for how to use this is very important.

Responsibilities represented by specific process role are carried out by an actor.
An actor is an object in the organizational structure. It is used to group positions in order to ease assignment of process roles to persons. One process role can be linked to one or more actors. Assignment of two or more process roles to the same actor in the same work process has to be carefully evaluated. Actor name indicates organizational assignment, operated asset type and type of operation.

Some additional naming conventions are

* Task: You shall define the title of a task as 'verb imperative noun', where the verb reflects the activity performed in order to add value to an asset. The noun shall reflect the asset.

* Start event: The title is a noun reflecting the asset and a verb past particle that reflect the activity performed in order to add value to the asset.

* End event: The label follows the same structure as a start event.

* Diverging exclusive gateway: The title consists of 2 parts. A term 'control' (or a similar term e.g. check, verify, evaluate, clarify) and a noun that reflects the object submitted to control.

* Converging exclusive gateway. No label.

* A sequence flow can be given a title that describes the flow between a source and a target. It is mandatory to add a title after a diverging exclusive gateway, which should be the adjective reflecting the result of the control. If possible the text shall be placed over the flow close to the arrow exit.

* Data: The title is a noun/noun expression in singular.

## 4.3 Syntactic Quality

Diagrams (used for work process model and document model) shall be designed in accordance with the requirements and symbols table available in the requirements. A subset of BPMN is used, following mostly the BPMN visual notation. This is similar to the analytical subset of BPMN [23], although not having support of intermediate events (this was supported in earlier versions of the standard, also we see that the use of the different concepts is supported with more detailed guidelines in the current version of the requirements). In addition some extensions to standard BPMN are included, and a number of specific requirements are introduced.

* Task: A task symbol represents what actors do as “individuals” in their process roles and thus shall be limited to a specific lane only. Tasks can be optional (dotted border). You shall not connect any governing elements classified as requirement to an optional task. A task can be collapsed, i.e. a decomposition as a separate workflow diagram can exist with the same title as the collapsed task. One should not introduce new roles in a decomposition. The sequence flow inputs to and outputs from the collapsed sub-process workflow diagram shall match start events and end events of the sub-process workflow diagram.

* Call task, to be able to reuse sub-tasks between different process models can be defined indicating this with a special border.
* A collaborative activity is a group of activities executed across lanes. These activities should not be sequenced in time or have other dependencies. Note that this is a particular extension to BPMN which is arguably poor at depicting (multi-party) collaborations [1]. The name of the collaboration activity symbol shall be unique and you shall not name the collaboration activity with names that have been used in the tasks framed by the collaboration activity symbol. Each of the tasks framed by a collaboration activity symbol must have a unique title clarifying different type of activities performed by different roles. You shall not place an optional task, a call task or a sub-process within a collaboration activity.

* Start event: Describe the state of the asset that triggers work. You shall not connect any governing elements to an event as no assigned person will be accountable for complying with them. An event shall be placed inside a lane.

* End event describe the state when terminating the workflow.

* Parallel gateway. Visualize the parallel divergence and convergence. It can split the flow into two or more parallel flows. "Event", "Exclusive gateway" or an activity related symbol can be used as preceding or resulting symbols of the parallel gateway symbol. It is not allowed to leave split parallel flows not merged again in the same work process.

* Diverging exclusive gateway. Indicate a choice of path in the workflow. Event, any type of gateway or an activity related symbol can be used before resulting symbols of the exclusive gateway symbol. Each exclusive flow may have different end events.

* Converging exclusive gateway. Match a diverging exclusive gateway.

* Sequence flow: Sequence flows to and from collaboration activities are connected to and from the same task within that collaboration activity. The flow is connected to and from the task performed by the role that is responsible for the output of the collaboration activity. You shall not use more than one sequence flow arrow from an activity. You shall not connect sequence flows to an optional task.

* Data: Used to describe a physical collection of information. Data association: To link data to the rest of the model.

* Association: To link text annotations together with other symbols.

* Lane: Represent a process role.

* Presence of requirement: A Statoil-specific symbol being a triangle with an exclamation mark to show the presence of one or more governing elements classified as requirements. Symbols representing presence of requirements or information shall be placed at the lower right corner of an activity related symbol. Any activity symbol and gateway symbol except a collapsed activity can have requirements linked to them.

* Presence of information: Use to show presence of one or more governing elements classified as information.

## 4.4 Semantic Quality

How the model represents the real world, it is a model of. Any model is an abstraction of the real world for a given purpose [24] and can never be a complete representation
of the world. The semantic quality of the model is based on how well the model reflects the real world in light of the goal of the model.

The content of a governing element shall explain scope, adhere to the purpose and be described with necessary level of details. Special rules apply for describing the content of a key control. This description shall include:

* Control activity

* Actions in case of deviations

* Audit trail

* Key control characteristics

Process role represents a method of grouping of activities and decision gates according to responsibility and competence within a work process. The purpose of process role is to:

* Secure necessary segregation of duties

* Achieve efficient recognition and allocation of the competence in the work process

It is important that the end-users easily recognize the process role names. Process role is organization and location independent and helps different process users to better relate to their work processes and it is indicating which activities are performed by the role itself. The categorization of process roles have been established to secure necessary segregation of duties. Categorization is based on the RACI (Responsible-Accountable-Consulted- Informed) principle as described above.

## 4.5 Pragmatic Quality

A number of the guidelines listed under empirical and syntactic quality above are made to support the development of understandable models. In addition it is important to be clear on the intention of the model. Each governing documentation model shall as a minimum have a defined purpose that includes:

* Risk – a description of the risk that the model mitigates.

* Objective – a description of the intended result (output).

* Target group – the main end-users of the process and the main users of the result.

## 4.6 Social Quality

Each governing documentation model and governing element shall have a documented validity (i.e. organizational area where it applies). There are 2 validity dimensions: location and organization. The location validity is based on geography. The organizational validity is based on business area. The following rules apply when defining validity:

* If validity is set for a specific organizational entity then location validity is by default unspecified (covers all locations)

* If validity is set for specific location then organizational validity by default remains unspecified (covers all organizational entities)
Note that validity indicates who needs to agree on the model. The deviation attrib-
ute is used to document the deviation approval method of the governing documenta-
tion model, its workflows or governing element categorized as requirement.

Each governing documentation model and governing element shall have docu-
mented one single owner and minimum one owner representative. The ownership at-
tribute is used to identify who is responsible for the right quality of a model or ele-
ment attributes, references, links as well as to enable deviations and improvement
proposals handling process, and thus who has the last say when not all agree.

4.7 Deontic Quality

As discussed in Section 3, the main goal of the models is to fulfil the goals of the
quality systems which are:

1. Contributing to safe, reliable and efficient operations and enabling compliance with external and internal requirements.

2. Helping the company incorporating their values, people and leadership principles into everything they do.

3. Supporting business performance through high-quality decision-making, fast and precise execution and continuous learning.

A straightforward relation between the different goals of modelling, different qual-
ity aspects and the goals of the quality systems above are not explicitly written in the
requirements. Neither are the cost/benefit tradeoffs between effort used and sufficient
quality achieved. As for quality trade-off, it is clearly stated already in [26] that prag-
matic quality is the most important whereas syntactic and semantic quality is primarily
a mean to achieve pragmatic quality.

5 Final Discussion and Further Work

The quality system of Statoil is developed supporting in particular compliance to re-
duce risk, an area where large improvements have been observed over the last decade.
Quality maturity can be claimed to be high, with a balanced concern of syntactic, se-
mantic and pragmatic (including empirical) quality. When mapping the guidelines for
modelling to the current SEQUAL framework, we find relevant aspects on all areas,
not only on the core syntactic, semantic and pragmatic levels. Most of the aspects re-
lated to in the Statoil-guidelines as pragmatic are in current SEQUAL-terms rather to
be classified as empirical, supporting the achievement of understanding of the model
in general. The paper illustrates that much more detailed guidelines are devised on es-
pecially the physical (linked to the particular modelling environment used), empirical
and syntactic level than what is found in generic overviews of quality of business
process models [7, 14]. On the other hand, we observe that links of the quality features
to the main goal of the models and trade-off as for resource usage vs. model quality
(deontic quality) is discussed in less detail.

A recent evaluation of the models also points to potentials for improvements: Dur-
ing the end of 2013 and the beginning of 2014, a large-scale user survey was con-
ducted in Statoil in order to identify the most prominent challenges related to the man-
agement system and governing documentation. 4828 employees took part in the sur-
vey, which was about half of those invited. The survey indicated opportunities for im-
provements on the physical level (related to finding all relevant models), empirical
level (relative to the use of abbreviations), semantic level (supporting efficient feed-
back and learning from users of the model for model evolution), pragmatic level (lack
of clarity and being aware of the intention of the model), social level (through leader-
ship support for using the model), and deontic level (by supporting the full range of
goals of the organization, not only safety and reliability).

As this is a case study, there are challenges relative to threat to validity of the re-
sults. Since there is one main informant (co-writer of the paper) one can claim some
limitations relative to internal validity, as representatives of all involved roles have not
been interviewed thoroughly. As for descriptive validity (what happened in specific
situations) the close day to day interaction with the development of the quality system
by one of the researchers gives us confidence on the accuracy here. As for the inter-
pretive validity (what it means to the people involved) we have again in-depth, ac-
counts from central people in main roles. The same can be said on evaluative validity
(judgements of the worth and value of actions and meaning). That we find many re-
sults that fit the categories of existing theoretical frameworks (SEQUAL) gives us
confidence on the theoretical validity of the results. A main issue as with all case stud-
ies is the external validity i.e. the generalizability of the results.

In future work, we will follow how the results from the assessment can be ad-
dressed by updating the requirements and models, and in particular

* Do changes to the Management System have a measurable effect on efficient model use, e.g. as for higher achievement of the overall goals such as less incidents due to non-compliance?

* How to balance the different types of model quality to get the best support of the organizational goals of modelling as defined in section 3.

Also doing similar case studies in other organization would be very interesting.

References

1. Aagesen, G., Krogstie, J.: Analysis and design of business processes using BPMN. In Jan vom Brocke & Michael Rosemann (Eds.), *Handbook on Business Process Management*: Springer (2010)

2. van Bommel, P., Hoppenbrouwers, S.J.B.A., Proper, H.A., van der Weide, T.: QOMO: A model-
ing process quality framework based on SEQUAL. In Proceedings of EMMSAD (2007)

3. Hella, L, Krogstie J.: A Structured Evaluation to Assess the Reusability of Models of User Profiles. Paper presented at the EMMSAD Hammamet, Tunis, 7-8/6 (2010)

4. Krogstie, J.: Using Quality Function Deployment in Software Requirements Specification. Paper presented at the Fifth International Workshop on Requirements Engineering: Foundations for Software Quality (REFSQ'99), Heidelberg, Germany, June 14-15 (1999)

5. Krogstie, J.: Integrated Goal, Data and Process Modeling: From TEMPORA to Model-Generated Work-Places. In: Johannesson P, Søderstrøm E (eds) Information Systems Engineering From Data Analysis to Process Networks. IGI, pp 43-65 (2008)

6. Krogstie, J.: Model-based development and evolution of information systems: A quality approach, Springer, London (2012)
7. Krogstie, J.: Quality of Business Process Models. Proceedings PoEM 2012, Rostock Germany LNBIP (2012)

8. Krogstie, J.: Quality of Conceptual Data Models. Proceedings 14th ICISO, Stockholm Sweden (2013)

9. Krogstie, J., Arnesen, S.: Assessing Enterprise Modeling Languages using a Generic Quality Framework. In: Krogstie J, Siau K, Halpin T (eds) Information Modeling Methods and Methodologies. Idea Group Publishing (2004)

10. Krogstie, J., Dalberg, V., Jensen, S. M.: Process modeling value framework. In Yannis Manolopoulos, Joaquim Filipe, Panos Constantopoulos & Jose Cordeiro (Eds.), *Selected papers from 8th International Conference, ICEIS 2006* (Vol. LNBIP 3, pp. 309-321). Paphos, Cyprus: Springer. (2008)

11. Krogstie, J., Jørgensen, H.D. Quality of Interactive Models. in First International Workshop on Conceptual Modelling Quality (IWCMQ'02). Tampere, Finland: Springer Verlag (2002)

12. Lindland, O. I., Sindre, G., Sølvberg, A.: Understanding Quality in Conceptual Modelling,” IEEE Software, vol. 11, no. 2, pp. 42--49 (1994)

13. Malinova, M., Leopold, H., Mendling, J.: A Meta-Model for Process Map Design. In: CAISE Forum 2014, June 16-20, Thessaloniki, Greece (2014)

14. Mendling J, Reijers H.A., van der Aalst W.M.P.: Seven Process Modeling Guidelines (7PMG). Information and Software Technology (IST). Volume 52, Number 2, pages 127-136 (2010)

15. Moody, D. L., Shanks, G. G.: What Makes a Good Data Model? Evaluating the Quality of Entity Relationship Models. In: Proceedings of the 13th International Conference on the Entity-Relationship Approach (ER'94), pages 94-111, Manchester, England (1994)

16. Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: Current state and future directions. Data and Knowledge Engineering 55 243-276 (2005)

17. Nelson, H.J, Poels, G., Genero, M., Piattini, M. A conceptual modeling quality framework. Software Quality Journal 20:201-228 (2012)

18. Nossum, A., Krogstie, J.: Integrated Quality of Models and Quality of Maps, Enterprise, 14th International Conference, EMMSAD 2009, Held at CAiSE 2009, Amsterdam, the Netherlands, June 8-9, Proceedings, Springer, p. 264. (2009)

19. Nysetvold, A. G., Krogstie, J.: Assessing Business Process Modeling Languages Using a Generic Quality Framework, in K. Siau (Ed.) Advanced Topics in Database Research Vol. 5, Hershey, Pennsylvania: Idea Group, pp. 79-93. (2006)

20. Price, R., Shanks, G.: A semiotic information quality framework: Development and comparative analysis. Journal of Information Technology, 20 (2), 88-102 (2005)

21. Recker, J., Rosemann, M., Krogstie, J.: Ontology- versus pattern-based evaluation of process modeling language: A comparison. Communications of the Association for Information Systems 20:774-799 (2007)

22. Schuette, R., Rotthowe, T.: The guidelines of modeling–an approach to enhance the quality in information models. In Conceptual Modeling-ER'98, pages 240–254. Springer (1998)

23. Silver, B. BPMN Method and Style. Cody-Cassidy Press (2012)

24. Stachowiak, H.: Allgemeine Modelltheorie. Springer, Wien, Austria (1973)

25. Statoil: TR0002 Enterprise Structure and Standard Notation. version 3 (2013)

26. Statoil: TR0002 Enterprise Structure and Standard Notation. version 1 (2009)

27. Statoil: Statoilboken http://www.statoil.com/no/About/TheStatoilBook/Downloads/Statoil-Boken.pdf (2014)

28. Statoil FR20 Management System (2014)

29. Wesenberg, H., Landre, E.: Using the corporate Management System to communicate architecture, Presentation SATURN2011, San Francisco, California, May http://www.sei.cmu.edu/saturn/2011/abstracts-presentations.cfm#25 (2011)

30. Wesenberg, H. Enterprise Modeling in an Agile World PoEM 2011, Proceedings of the 4th conference on Practice of Enterprise Modeling, Oslo, Norway, November 2-3 (2011)