# Organizational Learning to Leverage Benefits Realization Management; Evidence from a Municipal eHealth Effort

Kirsti Askedal, Leif Skiftenes Flak, Hans Solli-Sæther, Detmar W. Straub

► To cite this version:

Kirsti Askedal, Leif Skiftenes Flak, Hans Solli-Sæther, Detmar W. Straub. Organizational Learning to Leverage Benefits Realization Management; Evidence from a Municipal eHealth Effort. 16th International Conference on Electronic Government (EGOV), Sep 2017, St. Petersburg, Russia. pp.142-153, 10.1007/978-3-319-64677-0_12 . hal-01702993

HAL Id: hal-01702993

https://inria.hal.science/hal-01702993

Submitted on 7 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Organizational Learning to Leverage Benefits Realization Management; Evidence from a Municipal eHealth Effort

Kirsti Askedal<fnref n="1" />, Leif Skiftenes Flak<fnref n="1" />, Hans Solli-Sæther<fnref n="2" />, and Detmar W. Straub<fnref n="3" />

**Abstract.** While work with benefits realization requires organizational learning to be effective, emphasis on organizational learning is hard to find in benefits realization studies. To remedy this research gap, we study how organizational learning theory can contribute to improve benefits realization processes. A qualitative approach was used to gain in depth understanding of benefits realization in an ICT healthcare services project. We found that individual learning is present, but organizational learning has not been given explicit attention neither in the project nor in the literature of benefits realization management. We argue that the individual learning in the project forms an excellent basis for organizational learning, i.e., in the form of organizational structures, routines, and methods for benefits realization.

**Keywords:** Benefits Management, Organizational Learning Theory, Complex Organizations, Public Sector, eHealth.

## 1 Introduction

To prepare for the rapid demographic changes and the increased number of citizens suffering from non-communicable and compounded diseases [1, 2], the healthcare sector is dependent on innovation to manage future service-provision. This, among other topics, is emphasized by the European Commission when they included *Health, Demographic Change and Wellbeing* in their framework for research and innovation, *Horizon 2020* [3].

Where will this innovation occur? Information and communication technologies (ICTs), a wide range of which are being implemented into the healthcare sector [4, 5], are interventions supporting people in living safe and independent in their own homes; they can also improve quality of life and provide efficient and effective services. Even though there is enthusiasm to use information and communication technology (ICT) in healthcare services [6], adoption often occurs without a true understanding of the added value of ICT to healthcare service or a comprehensive evaluation of the health impact [4, 6, 7].

In the field of eHealth, it seems difficult to realize expected benefits [5, 8, 9] and varying levels of effects are reported by patients and healthcare professionals [6, 10].

<footnote n="1">¹ University of Agder, Gimlemoen 25, 4604 Kristiansand, Norway
kirsti.askedal@uia.no</footnote>

<footnote n="2">² Norwegian University of Science and Technology, Larsgårdsvegen 2, 6009 Ålesund, Norway</footnote>

<footnote n="3">³ Georgia State University, Atlanta, GA 30302-4015, U.S.A.</footnote>
Hofmann [11] argues it should be seen as a moral problem, i.e., not having knowledge of the effects of technology, as ICT is rapidly being adopted into many countries' healthcare services. Authorities have been hesitant in making benefit realization approaches a requirement, but are eager to better understand the potential benefits and how to produce them [12].

Several benefits realization tools for public sector have been developed and these are increasingly being adopted by praxis [12, 13]. There is, however, little empirical evidence of the benefits realization process as it occurs in practice [14]. As technology is seen as a helping tool for managing the future challenges in the healthcare services and are progressively being integrated into the healthcare services, there is a need for research to document whether ICT contributes and how the public sector should work to secure such gains.

Learning to use benefits management tools and methods is generally related to a common understanding of those representatives involved in the effort. They are typically healthcare professionals with little or no experience with benefits realization management. However, to increase benefits realization, means identifying potential benefits and manage the process. Thus, knowledgeable representatives are key. For health care professionals to become knowledgeable they must learn and experience from the process. Our approach to learning and knowledge is based on how individual knowledge is central in the organizational learning[15].

The research question for our study is: How can organizational learning affect complex benefits realization?

## 2 Theory

This section introduces Benefits Management [16] and organizational learning theory [17] as appropriate analytic lenses for our study. Benefits management emphasizes organizational development and innovation, includes a wide range of potential benefits, and looks at what is appropriate for addressing the complexity in public sector relevant to explicit stakeholder foci. Organization learning theory states that, in order to be competitive in a changing environment, organizations must change their goals and actions to reach these goals. In the public sector, individual learning transforms into organizational learning when information is shared and stored in the organization memory in such a way that it influences rules, values, attitudes and actions.

### 2.1 Benefits Management Model

In the middle of 1990s, a process model of Benefits Management was developed through a research project in benefits management at the Cranefield School of Management Information System Research Centre (ISRS) [18]. With experiences from many organizations, this model has been extended and refined, and presented in detail in the book to Ward and Daniel [16]; Benefits Management: *Delivering Value from IS & IT Investments*.

Working with benefits realization, through the model to Ward and Daniel [16] is like an iterative process. The model emphasizes organization development and innovation
and consists of five stages, with different activities related to each stage, illustrated in
Figure 1.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Benefits Management Model [16, p. 105].</figcaption></figure>

Ward and Daniel [16] point out that there is an inherent interdependency of benefits realization and change management in their approach and that is the reason why they call it Benefits Management. This state that it is not only about the implementation of technology, but also changes in the organizational processes, the roles and working practices individually or in team inside the organizations and in some cases outside the organizations. The term Benefits Management is defined by [16, p. 36] as: "*The process of organizing and managing such that the potential benefits arising from the use of IS/IT are actually realized*".

Even though there are different models of the benefits management process, the main principles are often similar to the Ward and Daniels model [16] and their model has also been an inspiration for the Norwegian work in that field [12, 13].

It is important to understand the strategic context in which IT investments are being made[16], and for this reason, we state that the context for our research is municipal health organizations. A characterizing feature of public organizations is the diversity of different stakeholders and competing interests [19]. Unlike the private sector, the public sector must strive to develop services which can be used by everyone in the community [16].

A critical issue in enabling organizations to realize benefits from IT investments, is the ability of the organization to embed individual learning into organizational structures and routines [16]. During the benefits realization process, learning occurs on the individual level among the people that carry out the various analyses comprising the benefits realization method. However, translating these insights into organizational learning does not happen automatically but require specific attention from the organization.
## 2.2 Organizational Learning Theory

Organizational learning occurs when individuals within an organization experience a problematic situation and inquire into it on behalf of the organization. In order to transcend to the organizational level, learning that results from organizational inquiries must become embedded in the images of organization held in its members' minds and/or in the epistemological artefacts (e.g., the cognitive maps, memories or programs) embedded in the organizational environment [17]. Single loop learning adjusts the action, but not the objectives behind the activity. Double loop learning alters or rejects the established governing objectives and produces a major and fundamental change in the organization's mission. Double loop learning is thus closely linked to an organization's ability to develop and increase their performance, e.g. by realizing benefits from IS & IT investments.

Senge [20] points out that learning organizations engaged in systematic organizational development depend on five conditions for success. These five conditions are: (1) to facilitate personal mastery; (2) to create mental models; (3) to build a shared vision; (4) to develop group learning through good leadership; and (5) to engage in systems thinking. The idea is that the whole will be greater than the sum of the parts. This can be done e.g. by including employees in benefits realization and change management. Ownership to the process will facilitate individual learning, which can build group learning (project) and ultimately organizational learning.

Nonaka and Takeuchi [21] introduced the SECI-model which has become the cornerstone of knowledge creation and transfer theory, illustrated in Figure 2.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The SECI- process [22, p. 12].</figcaption></figure>

The four dimensions of the model – socialization, externalization, combination, and internalization – explain how tacit and explicit knowledge are converted into organizational learning. The first dimension, socialization, is explained to be the process of converting tacit knowledge through shared experiences like spending time together. When tacit knowledge is articulated into explicit knowledge it is called externalization, who
is the second dimension in the SECI- process. Explicit knowledge can be shared with
others, e.g. in processes and routines, and become basis of new knowledge. The third
dimension is called combination, and occurs when explicit knowledge is converted into
more systematic and complex sets of explicit knowledges, and distributed to the mem-
bers of the organization. Internalization is the fourth dimension, and happens when ex-
plicit knowledge created and shared in the organization is converted into individual
tacit knowledge. When individual tacit knowledge is shared with others, it can start a
new spiral of knowledge creation [22].

Organizations that share knowledge and experience contribute to innovation and learning across organizational boundaries and thus create benefits for one or more partners. Knowledge sharing is focused both on creating new knowledge, sharing knowledge, and applying knowledge. Sometimes knowledge sharing is perceived to be difficult to carry out. There can be structural, political, personal or cultural obstacles or barriers that must be overcome. Legislation can be such an obstacle for ICT in healthcare services.

To synthesize our brief review of the benefits management and organizational learn-
ing literature, we suggest that a benefits management model for improving benefits
realization in an organization can be combined with organization learning. The first
challenge is to properly understand the strategic context and conduct the activities of
identification, planning, execution, reviewing, and establishing potential for further
benefits. The second challenge is to move from individual learning to organizational
learning. This challenge involves probing how organizations can take interpreted
knowledge held by individuals and use it to change organizational actions/goals.

3 Method

Based on the research question a qualitative approach for data collection was consid-
ered most appropriate for this project. The purpose of a qualitative approach is to obtain
a richer description of the problem setting and this approach is especially useful when
investigating a phenomenon to which little prior attention has been paid [23].

Case study is one of the most important sources for theory development in social science [24], and can be seen as a non-proactive approach, who “study the phenomenon after the fact” [25, p. 326]. It is best suited when “how” or “why” questions are being sat and when focus is a contemporary phenomenon within a real-life context [26]. There are different definitions for this research method [27], and we apply the definition of case study by Eisenhardt [28, p. 534]: “*The case study is a research strategy which focuses on understanding the dynamics present within single settings*”.

Based on the need for knowledge about benefits realization process, this project is
designed as a single-case study, with an interpretive approach. We have followed the
five components of case study research design proposed by Yin [26, p. 29] where the
unit of analysis is the knowledge creation process in complex benefits realization set-
ting, within a municipal healthcare context. Data is collected through participant-obs-
servation (see section 3.1. for details about the role of the researcher), and field notes
are analyzed as an interactive process among the researchers with use of different interrelated elements illustrated in Creswell [29, p. 185].

## 3.1 Case Description

In 2015, one municipality in Norway, on behalf of two counties (made up of 30 municipalities), was asked by the central government to establish a Response Central for managing safety alarms and other sensors for recipients of municipal healthcare services.

After the business plan for the Response Central was developed and parallel to other important clarifications (i.e., how to cooperate with other municipalities in the region, and preparation for procurement), it was decided by the steering committee to focus on benefits realization. One of the researchers was given the task of managing the benefits realization process, hence referred to as the benefits realization process manager. As it was considered to be extensive and time-consuming to agree on a common benefits realization plan across the potential cooperation partners (municipalities), the initial aim was to develop a general benefits realization plan for one of the municipalities, with an intention to share the document with the cooperation partners as a starting point for them to manage benefits realization process in their own organizations.

Different methodologies for benefits realization were reviewed. The KommIT methodology [30] was considered by the benefits realization process manager to be the most transparent and useful for this project. This methodology is inspired by the work of Ward and Daniel [16]. Table 1 illustrate the different stages from the two stated methodologies and how they relate. The project is still running with only results from stage 1 and part of stage 2 of the methodology being completed.

<table><caption>**Table 1.** Overview of the Stages for Benefits Management Model [16] and KommIT Methodology [30]</caption><thead><tr><td>Benefit Management Model</td><td>KommIT methodology</td></tr></thead><tbody><tr><td>1. Identify and structure benefits</td><td>1. Concept; identify and assess benefits</td></tr><tr><td>2. Plan benefits realization</td><td>2. Plan; Plan benefits realization</td></tr><tr><td>3. Execute benefits plan</td><td>3. Execute; manage benefits realization during project</td></tr><tr><td>4. Review and evaluate results</td><td>4. Hand over; hand over benefits realization from project to operation</td></tr><tr><td>5. Establish potential for further benefits</td><td>5. Realize; Benefits realization in operation</td></tr></tbody></table>

## 4 Results

During a three-month period, a number of activities were conducted following the KommIT methodology. This resulted in important and necessary discussions among key stakeholders. Several inputs were fruitful for benefits realization in this specific case, but the core discussion was related to the benefits realization process in general. It was the first time this specific methodology was used in this sector and the benefits
realization process manager had no practical experiences with it in advance. Thus, the
project was dependent on and tried to strictly follow the methodology. Based on expe-
riences to a given point in time, some minor changes were made to secure progress and
maintain the schedule.

In the following, the purpose and challenges of the two stages will be outlined. Then,
an overview of individual learning related to the stages from the perspective of the
benefits realization process manager will be presented (Table 1).

4.1 Stage 1 - Concept; Identify and Assess Benefits

According to the KommIT methodology, the purpose of this partial stage is to analyze potential benefits linked to the specific ICT-project. What kind of positive effects can the municipality expect? Will there be changes in work-processes? Who are the stakeholders? Are the changes sufficient to justify the project?

One of the main challenges in managing this stage was related to stakeholders’ in-
security about the purpose for the benefits realization process. The decision to establish
the Response Central was taken before the project were started and was the driver for
this process. Some of the stakeholders expressed skepticism based on experiences from
similar processes, where identified benefits and assumptions for savings have had a
directly negative impact on their budgets without taking the necessary prerequisites into
account. Questions like: “Is the process just a cover for justifying the investment” arose.

Given the skepticism in the organization towards change and the fact that the project
affected several departments, all the units were invited to process for identifying bene-
fits during this stage. Some of the stakeholders were concerned that this would be just
another shadow process. However, it seems that all of the stakeholders were satisfied
with the thorough review of the concept and the possibility of asking clarifying ques-
tions. This involvement led to project ownership and important stakeholders were iden-
tified. However, it seemed difficult to achieve the desired openness, due to a major
stakeholder focus on prerequisites and emphasizing that the defined benefits merely
showed a potential. Because of this suspicion, some vital information may have been
held back.

4.2 Stage 2 - Plan; Plan Benefits Realization

The KommIT methodology next suggests that the planning stage purpose is to link
identified benefits to specific targets, define measurement indicators, actions, and as-
sign responsibility for benefits realization to stakeholders in the organization. This
phase starts after the project is accepted based on the benefits analysis in the previous
phase.

The principles underpinning the development of the benefits realization plan appear
simple and easy to implement. Developing a benefits realization plan across different
units within one organization was, however, challenging in praxis because the plan
needed to be broadly accepted in the organization to ensure benefits realization. The
stakeholders had different perspectives to the identified benefits. Some were only will-
ing to pay attention to qualitative effects, like safety and service quality, but others were
willing to discuss direct or indirect economic benefits as well. This may be related to organizational roles or professional background. Most of the identified benefits proved to be qualitative as the organizational changes and ICT investment will affect the budget in a negative way the next years. In short term, this project will cost a lot of money, but in long term, the investment can help to prepare for the future challenges the healthcare services are facing. When it was experienced to be challenging in one organization with different units, developing the same plan for a consortium of organizations, thought to be the overall goal at the start, is obviously even more challenging.

Since this was the first time a benefits realization process was conducted systematically in the healthcare services in the municipality, there were no established structures for where to discuss and ask for advice throughout the process. The benefits realization process manager had to rely on the method and justify for stakeholders both “why focus on benefits realization in general” and facilitating the benefits realization process in the specific circumstance. General organizational guidance for managing processes like this would have been very useful in a project which involves several departments in one organization/across different organizations.

Table 2 summarizes the individual learning in the project based on experiences from stages 1 and 2 from the perspective of the benefits realization process manager.

<table><caption>**Table 2. Individual Learning from the KommIT Methodology Stages in Praxis**</caption><thead><tr><td>Stage</td><td>Individual learning from stage</td></tr></thead><tbody><tr><td>1. Concept:<br/>Identify and consider benefits.</td><td>1. An agreement of purpose for the benefits realization process and the investment is critical. To communicate a clear problem understanding at the grass root level is needed.<br/>2. A combination of competence (e.g. healthcare, technical and innovation) is necessary for modeling current and future work-processes.<br/>3. Analyzing changes in work-processes and identifying benefits are important activities for stakeholder involvement and ownership of the benefits realization process and the project in general.<br/>4. The identified benefits at this point outlines potential, and it is important to identify and be aware of the prerequisites.<br/>5. Due to a constantly evolving project, stakeholder analysis must be seen as an iterative process.<br/>6. A thorough stakeholder analysis is critical to ensure an adequate change management process and high degree of realization of the identified benefits.<br/>7. If an action (here the Response Central) to a challenge is determined in advance, an analysis of benefits is a demanding activity due to the stakeholders' uncertainty about the motive for the benefits realization process.</td></tr><tr><td>2. Plan:<br/>Plan benefits realization.</td><td>8. Organizational support is needed to manage a benefits realization process in complex projects and organizations.<br/>9. A distinct unit for managing processes like this had been very useful in a project who involves several departments in one organization/across different organizations.<br/>10. A benefits realization plan has limited value unless accepted broadly in the organization. This requires substantial effort.</td></tr></tbody></table>
# 5 Discussion

Organizational learning capability is related to both organizational and managerial characteristics and factors that enable the organizational learning process [31]. Dimensions of a learning organization consist of: continuous learning, dialogue and inquiry, collaboration and team learning, systems to capture learning, empowered employees, connected organizations, and strategic leadership [32].

The issue of organizational learning has not been given explicit attention in the benefits realization literature. We argue that this is a major shortcoming and that organizational learning is instrumental in enabling organizations to realize benefits from their ICT investments. We consider organizational learning theory to be a valuable contribution to the benefits realization literature and propose that the practical benefits realization methods should incorporate mechanisms for organizational learning.

The individual learning outlined in Table 2 provides a good basis and can give input to necessary organizational learning. E.g. the need for a broad competence base when modelling processes in Stage 1 indicate that the organization should facilitate exactly this in future endeavors. Further, the expressed need for a distinct coordination unit in Stage 2 suggest that the organization needs to establish such a unit to support similar future efforts. Gladly, the organization in the present case are these days planning to establish a portfolio office, who will be responsible for coordinate and manage projects and help department managers to run processes like this. More examples of how individual learning can be transferred into organizational learning can be found in Table 3.

Results presented from this case can be seen in relation with three of the dimensions presented in the SECI-process [22]. The trigger for the knowledge creating process was the steering committee's focus on benefits realization, and the available methodologies (e.g. KommIT methodology) for running such processes in public sector provided by other organizations (internalization). The benefits realization process manager had some tacit knowledge and this were converted through shared experiences when stakeholders in the project spending time together through this process (socialization). The individual tacit knowledge gained from the process has in this paper being articulated into explicit knowledge (externalization). One part of this dimension is illustrated in Table 2, and another can be viewed in Table 3, where suggestions of how to transfer individual learning (tacit knowledge) into organizational learning (explicit knowledge) is presented. The suggestions to organizational learning from this case can be used for input to the portfolio office, and maybe be implemented in future projects and revised methodologies for benefits realization in public sector (combination).

In summary, we propose the following two additions to existing benefits realization methods: 1) Individual learning should be specified and 2) Individual learning should be translated into organizational learning.

Table 2 summarized the individual learning from the case. Table 3 illustrates how individual learning can be transformed into organizational learning.
<table><caption>**Table 3.** Examples of Transferring Individual Learning into Organizational Learning</caption><thead><tr><th>Stage</th><th>Individual learning from stage</th><th>Suggestions to organizational learning</th></tr></thead><tbody><tr><td>1. Concept:<br>Identify<br>and consider<br>benefits.</td><td>1. An agreement of purpose for the benefits realization process and the investment is critical. To communicate a clear problem understanding at the grass root level is needed.<br>2. A combination of competence (e.g. healthcare, technical and innovation) is necessary for modeling current and future work-processes.<br>3. Analyzing changes in work-processes and identifying benefits are important activities for stakeholder involvement and ownership of the benefits realization process and the project in general.<br>4. The identified benefits at this point outlines potential, and it is important to identify and be aware of the prerequisites.<br>5. Due to a constantly evolving project, stakeholder analysis must be seen as an iterative process.<br>6. A thorough stakeholder analysis is critical to ensure an adequate change management process and high degree of realization of the identified benefits.<br>7. If an action (here the Response Central) to a challenge is determined in advance, an analysis of benefits is a demanding activity due to the stakeholders' uncertainty about the motive for the benefits realization process.</td><td><ul><li>Stimulate the organization to be adaptable to change.</li><li>Communicate accurate and clear information at different levels in the organization.</li><li>Use standardized methodology for project- and benefits realization.</li><li>Ensure that persons involved in the project (in different stages and activities) have the right skills and competence for the tasks.</li><li>Allocate sufficient resources, both human and economical.</li></ul></td></tr><tr><td>2. Plan:<br>Plan benefits<br>realization.</td><td>8. Organizational support is needed to manage a benefits realization process in complex projects and organizations.<br>9. A distinct unit for managing processes like this had been very useful in a project who involves several departments in one organization/across different organizations.<br>10. A benefits realization plan has limited value unless accepted broadly in the organization. This requires substantial effort.</td><td><ul><li>Clarify roles and descriptions of who is responsible for change management, benefits realization management. This needs to be communicated and well known in the organization.</li><li>Establish a unit for support and advise in such processes (e.g. a portfolio office)</li></ul></td></tr></tbody></table>

## 6 Conclusion

This study explored the research question “How can organizational learning affect complex benefits realization?”. Based on a qualitative case study of a complex benefits realization effort in a health care context, we derived several individual learning points based on the benefits realization process manager's experiences. The nature of the
learning points suggests that the organization would benefit from embedding these in-
sights into revised practice in future benefits realization efforts or put another way;
ignoring the individual learning would be likely to cause frustration and low organiza-
tional performance in future efforts. On this basis, we suggest two contributions to the
benefits realization methods: 1) Individual learning should be specified and 2) Individ-
ual learning should be translated into organizational learning. We used the case to il-
lustrate how individual learning can be transformed into organizational learning.

7 Implications

Although it is developed several benefits realization tools for public sector, there is
little evidence on the benefits realization process in practice [14]. This study highlights
the process, focusing on municipal health- and care services. It also sees a benefits
realization method in the perspective of organizational learning theory. The result can
be used as a guide for enabling organizations to realize benefits from IT investments
and how they can embed individual learning into organizational structures and routines.
This project will hopefully lead to better benefits realization processes when imple-
menting technology in practice, and to develop already existing benefits realization
tools.

References

1. Barnett, K., et al., Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet, 2012. **380**(9836): p. 37-43.

2. World Health Organization. 10 facts of ageing and the life course. 2014; Available from: http://www.who.int/features/factfiles/ageing/en/.

3. European Commission. Health, Demographic Change and Wellbeing. 2014; Available from: http://ec.europa.eu/programmes/horizon2020/en/h2020-section/health-demographic-change-and-wellbeing.

4. Martin, S., et al., Smart home technologies for health and social care support. Cochrane Database of Systematic Reviews, 2008. 4.

5. Henderson, C., et al., Cost-effectiveness of telecare for people with social care needs: the Whole Systems Demonstrator cluster randomised trial. Age And Ageing, 2014. **43**(6): p. 794-800.

6. Wootton, R., Twenty years of telemedicine in chronic disease management–an evidence synthesis. Journal of telemedicine and telecare, 2012. **18**(4): p. 211-220.

7. World Health Organization. Connecting for health: Global Vision, Local Insight. . 2005;
Available from: http://www.who.int/ehealth/resources/wsis_report/en/.

8. Essén, A. and M. Conrick, New e-service development in the homecare sector: Beyond implementing a radical technology. International Journal of Medical Informatics, 2008. 77(10): p. 679-688.

9. Henderson, C., et al., Cost effectiveness of telehealth for patients with long term conditions (Whole Systems Demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trial. BMJ, 2013. 346(1035).

10. Steventon, A., et al., Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. Bmj, 2012. 344: p. e3874.
11. Hofmann, B., Ethical challenges with welfare technology: a review of the literature. Science and engineering ethics, 2013. 19(2): p. 389-406.

12. Flak, L., Gevinstrealisering og offentlige IKT-investeringer. 2012: Oslo: Universitetsforlaget AS.

13. Hellang, Ø., L.S. Flak, and T. Päivärinta, Diverging approaches to benefits realization from public ICT investments: A study of benefits realization methods in Norway. Transforming Government: People, Process and Policy, 2013. 7(1): p. 93-108.

14. Ashurst, C., N. Doherty, and J. Peppard, Improving the impact of IT development projects: the benefits realization capability model, in European Journal of Information Systems. 2008. p. 352-370.

15. Chiva, R. and J. Alegre, Organizational learning and organizational knowledge: towards the integration of two approaches. Management learning, 2005. 36(1): p. 49-68.

16. Ward, J. and E. Daniel, Benefits management: delivering value from IS & IT investments. Wiley series in information systems. 2006, Chichester: Wiley.

17. Argyris, C.S. and D. Schön, Organizational learning II: Theory, method and practice. 1996, Reading, PA: Addison-Wesley.

18. Ward, J., P. Taylor, and P. Bond, Evaluation and realisation of IS/IT benefits: an empirical study of current practice. European Journal of Information Systems, 1996. 4(4): p. 214-225.

19. Pang, M.-S., G. Lee, and W.H. DeLone, IT resources, organizational capabilities, and value creation in public-sector organizations: a public-value management perspective. Journal of Information Technology, 2014. 29(3): p. 187-205.

20. Senge, P., The fifth discipline: The art and practice of the learning organization. 2006: Crown Pub.

21. Nonaka, I. and H. Takeuchi, The knowledge creating company: How Japanese companies create the dynamics of innovation. 1995, New York, NY: Oxford University Press.

22. Nonaka, I., R. Toyama, and N. Konno, SECI, Ba and leadership: a unified model of dynamic knowledge creation. Long range planning, 2000. 33(1): p. 5-34.

23. Johannessen, A., P.A. Tufte, and L. Kristoffersen, Introduksjon til samfunnsvitenskapelig metode. 2005, Oslo: Abstrakt forlag as

24. Andersen, S.S., Casestudier: forskningsstrategi, generalisering og forklaring. 2013: Fagbokforlaget.

25. Cole, R., et al., Being proactive: where action research meets design research. 2005.

26. Yin, R.K., Case study research: Design and methods. 2013: Sage publications.

27. Gerring, J., What is a case study and what is it good for? American political science review, 2004. 98(02): p. 341-354.

28. Eisenhardt, K.M., Building theories from case study research. Academy of management review, 1989. 14(4): p. 532-550.

29. Creswell, J.W., Research design: Qualitative, quantitative, and mixed methods approaches. Third edition ed. 2009: Sage publications.

30. The Norwegian Association of Local and Regional Authorities, Gevinstkokebok for IKT-prosjekter i norske kommuner. 2013, KS: http://www.ks.no/contentassets/af1d839033564d188081b64e8ee02a8/13224-ks-kommit-gevinstkokebok.pdf.

31. Chiva, R., J. Alegre, and R. Lapiedra, Measuring organisational learning capability among the workforce. International Journal of Manpower, 2007. 28(3/4): p. 224-242.

32. Marsick, V.J., The dimensions of a learning organization questionnaire: Introduction to the special issue examining DLOQ use over a decade. Advances in Developing Human, 2013. 15(2): p. 127-132.