# A Critical Analysis of the Implementation of Health Information Systems for Public Healthcare Service Delivery in Resource-Constrained Environments: A South African Study

Mourine Achieng, Ephias Ruhode

► To cite this version:

Mourine Achieng, Ephias Ruhode. A Critical Analysis of the Implementation of Health Information Systems for Public Healthcare Service Delivery in Resource-Constrained Environments: A South African Study. 15th International Conference on Social Implications of Computers in Developing Countries (ICT4D), May 2019, Dar es Salaam, Tanzania. pp.568-578, 10.1007/978-3-030-18400-1_47. hal-02285262

HAL Id: hal-02285262

https://inria.hal.science/hal-02285262

Submitted on 12 Sep 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A critical analysis of the implementation of health information systems for public healthcare service delivery in resource-constrained environments: A South African study

Mourine Achieng<fnref n="1" />[0000-0001-6368-8533] and Ephias Ruhode <fnref n="2" />[0000-0002-5727-7460]

**Abstract.** The use of technological solutions is argued to offer quality enhancing efficiencies in the delivery process of healthcare services. For this to be realized certain objectives of these technological solutions have to be achieved. The public health system in South Africa faces various challenges such as poor coordination of most technological solutions therefore fragmented; a lack of interoperability between different systems, haphazard procurement processes and poor information management capabilities. The argument in this study establishes that the causes are not purely logistical but also linked to gaps in the adequacy of the health information systems (HISs) and the implementation process of these technological solutions. A case study strategy, whose empirical home is rural South Africa, was employed using qualitative methodology. Activity Analysis and Development (ActAD) framework was used as a theoretical lens within critical realist paradigm. The study was conducted in a public healthcare facility within a resource constrained environment. Semi-structured interviews was used as data collection method and analysis of data was through narrative and explanatory analyses, employing thematic analysis tool. The paper intends to come up with a framework that can be used to inform the implementation process of HIS(s) across board. With a focus on the context of a divergent national service delivery reality that impacts the right to healthcare service in under-served communities.

**Keywords:** Health information systems, Healthcare service delivery, Resource constrained environments, and underserved contexts.

## 1 Introduction

Effective healthcare service delivery in the public healthcare sector is built on the back of adequate use of clinical and administration systems implemented in health-care facilities. Reichertz [1] argues that the most common administrative support functions of these systems include collection, processing, storage and retrieval of data using various innovative Information and communication Technology (ICTs) tools. The healthcare domain incorporates a wide range of technological tools that are

<footnote n="1">¹ Cape Peninsula University of Technology, Cape Town 8001, South Africa
sachiengm@gmail.com</footnote>

<footnote n="2">² Cape Peninsula University of Technology, Cape Town 8001, South Africa</footnote>
rapidly diversifying in their purposes. These technologies present great opportunities in the transformation of healthcare service delivery process. Readily identifiable opportunities include easy access to integrated real time, relevant administrative and clinical health data and information necessary for adequate service delivery of healthcare and decision making at various government levels [2]. Moreover, improved health outcomes, administrative efficiency, cost effectiveness, or users' experience are also some of the perceived benefits. The significance of these interventions, and their intended beneficial effect on healthcare personnel and health outcomes, it is imperative to ensure that optimal results are achieved, and any unforeseen outcomes identified.

This paper looks at HISs as one of the many technological innovations within the public healthcare domain. HIS is used in this paper to incorporate an all-inclusive range of information and communication technologies interconnected across a health system. Technologies are often viewed as enablers in a healthcare system in improving service provision processes, by aiding institutions to do more with lesser resources, quicker and cost effectively [3, 4]. This point would be ideal for healthcare facilities in resource constrained environments-- this are contexts that are plagued with shortage of human resource, lack of or limited skills and capabilities to formulate policies and strategies for the development or implementation of novel ideas. However, the perceived benefits of HISs are only possible to achieve when the implementation and infusion of these systems into the work processes of healthcare personnel are done adequately [5]. This study sought to address the question of why the existing implementation of HISs in public healthcare facilities is not adequately facilitating provision of care services in resource constrained environments.

In South Africa, an example of HIS existing in the public healthcare domain is the District Health Information system (DHIS) which was established of as a routine system for tracking health service delivery in the public health sector in 1996/97. The system, developed by the Health Information Systems Programme (HISP) was implemented to play a pivotal role in the collection, capturing, storage, analysis and reporting of routine data [6, 7]. Despite the great achievements of the DHIS, increase in the demand for routine information exposed weakness to the system such as poor data quality, dataflow bottlenecks [8] and reporting discrepancies at different levels. A web-based DHIS2 was introduced to mitigate some of the challenges with the earlier version. However, this was only done in one province whilst the rest of the provinces continued with the DHIS.

Many HIS implementations are usually done in a silo ad hoc manner resulting in fragmented systems with limited interoperability and software reuse, and a plethora of small pilots that are often not scalable [9]. This leads to duplication of data and information [10], a high dependency on technical support from external donors and many more challenges. These challenges that are more prevalent at public healthcare facilities in resource constrained environments where data collection is still a significant burden [11]. Resulting to inhibited adequacies as well as analysis and decision making [12, 13].

This paper adopts Damschroder et al [14] concept of implementation broadly summing it up as encompassing the processes involved in the HISs operating
satisfactorily in varied healthcare contexts. This would include analysing healthcare
facilities requirements in varied contexts, installation and configuration,
customization, running, testing systems integration, user training and making
necessary changes to the system.

## 2 Literature Review

### 2.1 Information Systems in Healthcare

Innovations within the health care sector according to [15] and [16] are directed at improving a healthcare system's efficiency and responding more effectively to patients' care needs thus improving health outcomes. The various types of information systems (IS) in this sector are designed to assist healthcare practitioners with managing daily tasks and patients' information in the delivery of care services to a population [17]. They include the following : (i) *Operational and tactical systems* designed to ease the process of classifying information, (ii) *Clinical and administrative systems* designed for managing patient details on an administrative level, (iii) *Subject and task based systems* such as Electronic Medical Records (EMRs) and (iv) *Financial systems* for tracking revenue and managing billing submissions.

The provision and management of the necessary information is highlighted as an
essential ingredient to the system of health care governance [18]. Further arguments
are that the delivery of care services is highly dependent on information [19],
consequently the management and ultimately adequate use of the technology enablers
are of paramount importance [20]. Healthcare institutions rely largely on information
about the science of care, individual patients, care provided, result of care, as well as
its performance to provide, coordinate, and integrate services [21]. According to [22]
and [23], HIS emphasises on optimizing and utilizing information to increase
efficiency and effectiveness in healthcare institutions. The systems are also designed
to integrate data management and use for the improvement of patient healthcare
services provision through better management of patient data at all level of
implementation [24]. Most health systems around the world still struggle with
implementation, and experience major delays in diffusing innovations [25].

## 2.2 Healthcare Information Systems Implementation

The implementation of HIS has been at the forefront in the field of information sys-
tems, with literature revealing an on-going debate between the impact of the technical
aspects Vis-a`-Vis human behavioral aspects. Authors like Keen and Morton [26] ar-
gue that the IS implementation process is an intuitive skill and that the best way to
deal with it is to be technically competent. Contrary to Keen and Morton, [27] and
[28] argue that implementation process depends on user involvement, how the proto-
typing of the system is done, the information analysis done on the nature for the im-
plementation and change agents. The later argument on user involvement seems to
carry across literature on IS implementation. Involvement of users in the implementa-
tion process is crucial because they can point out where the system could fail. The au-
thors also note other important factors such as top management commitment and
training have an effect in the implementation process (ibid). According to [29]the im-
plementation process of IS includes the following four activities; (i) defining the in-
novation problem and goal setting; (ii) planning activity; (iii) implementation activity;
and (iv)use and development activity. The shaping of the planning and implementa-
tion process of information systems in the user organization can be seen to be affected
by the development mechanism proceeding in two ways: (i) the strategic goal setting
and the definition of planning and implementation practices performed by the man-
agement, as well as by the planners; and (ii) the user activity [30; 31; 32; 33].

3 Methodology

The study was aligned to qualitative research methodology because of the exploratory and explanatory nature of phenomenon under investigation. The research paradigm aligned to this study was the critical realist’s perspective. This is because knowledge is considered socially constructed and through retroduction, it provides causal explanations of actual and empirical evidence. The study adopted the critical realist’s ontology as it focuses on the generative mechanisms, causal powers, deep structures and reality of entities [34] which have implications on the healthcare service delivery and the implementation of health information systems.

Both positivism and interpretivism paradigms are argued to suffer from the ‘epis-
temic fallacy’ i.e. they reduce statements about reality to statements about human
knowledge of that reality. Moreover, the two paradigms assume that what exists is
only what we observe and experience. It is on the basis of this that critical realism was
chosen for this study. Further, the study adopts the use of critical realism paradigm as
a means to gain an in-depth understanding of the relation between HIS implementa-
tion and its adequate use in the facilitation of care service delivery for an improved
health outcome. Arguments are that critical realism simultaneously confronts the cen-
tral concern of both the natural and social science regimes. It is on the basis of this
reasoning that makes critical realism of particular interest in the study of information
systems which bears significant relevance to natural science (due to their application
in deeply human context such as organizations). The critical realists’ assumptions and
feature of realism correspondingly matchup with the theoretical lenses upon which
this study is hinged: Activity Analysis and Development (ActAD).

## 3.1 The use of ActAD framework in this paper

ActAD framework was employed in this paper as an analytical lens through which the interplay between HIS and the society within which it is implemented is explored. The framework focuses on a work activity and the key features (object, actions, mediation, subject, actors, tools, transformation, rules, and outcome) of the work system are as illustrated in Fig. 1. The work activity portrays an activity as a collective phenomenon with a shared motive which stems from the organizational objectives or
strategic goals. These activities are mediated by mediators such as tools, signs, artefacts, context and conditions. The collective activities involves actions carried out by *actors/subjects* who collectively or individually are directed by rules and norms established by organizations to achieve a shared *motive or goal*. Context, culture, tools and rules also mediate the relationship between subjects and objects as well as amongst the subjects. The delivery of healthcare services through the use of HIS is seen as the key activity object in the framework. The main activities are therefore the interaction between the actors, tools mediators and the actual healthcare delivery process.

The transformation of the object into an *outcome*, renewal, alteration or abortion of the object have a predetermined timeframe in a work activity. This could be because of the failure of the object to achieve a desired outcome, perhaps as a result of contextual inhibitors or mediator tensions. Therefore, [35] argues that a successful interplay between the object from the mediation process and actions result into outcomes, where the objects undergo a successful *transformation* into an outcome. In Fig. 1, the object-representing a common purpose held by all actors (Dept. Of health, Medical and clinical personnel, nurses and administrators/clerks) in a public healthcare sector is to provide patients with adequate healthcare services. The object is then broken down into different goals, activities and responsibilities for each actor in the system. The goal and related responsibilities of the main actor, the national Department of Health (D.o.H) in South Africa, is to improve patient care by providing and enabling adequate healthcare services provision in public healthcare facilities. One of the main mediators based on the framework, is enabling factors- public healthcare facilities require enabling tools to be able to provide efficient healthcare services.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The ActAD Framework: the structure and relations of a work activity as a systemic entity [37].</figcaption></figure>
The key problem this study sought to address was the perceived gap in the explana-
tion of the failures of the implemented HISs to adequately facilitate healthcare service
delivery in public healthcare facilities within a resource constrained environment. The
key features of the work system in the conceptualized ActAD framework which
draws on from activity theory, was developed to operationalize and contextualize
healthcare activities at public healthcare facilities as shown in Fig. 1. From an activity
systems based approach, healthcare service delivery process can be analyzed as a ser-
vice provision with an objective, mediators, actions, mediator tensions, work activity
as a transformation and the activity outcomes.

<figure><img src="image_3.png" /><figcaption>Fig. 1. ActAD analytical framework</figcaption></figure>

3.2 Empirical Case

The healthcare facility used in the study is located in the OR Tambo Municipality in
the Eastern Cape province of South Africa. Both the municipality and the healthcare
facilities bear characteristics of a resource constrained environment. The facilities are
government funded and provide tertiary healthcare services to the region. The health-
care facilities are plagued with shortage of clinical staff. The selection criterion of the
facilities was based on the availability of health information systems such as the Dis-
trict health information system, patient records management system, laboratory infor-
mation system and radiological information system.

3.3 Data Collection and Sample Size

Data was be collected through in-depth open ended semi-structured interviews to
gather information about stakeholders' knowledge, experience and, interpretation on
the use and implementation of HIS in healthcare facility. Document reviews, observa-
tion and review of the software systems in use were also used as data collection meth-
ods. For data analysis, ActAD framework was used as an analytical lens together with
thematic and content analysis (for document analysis). A purposive sample was
drawn from the provincial Dept. of health, hospital and clinic. The research sample
consisted of senior hospital and clinic managers (as they manage the healthcare facili-
ties and are involved in the implementation of various health information systems);
clinical and medical staff including nurses (as they are the users of these systems for
clinical work to provide services); administrators and clerks (as they use the systems
to capture patient details at the entry point); HIS managers and the information sys-
tems managers (a representative from the provincial department of health- as they
oversee the adoption and implementation of health information systems at healthcare
facilities). In Total 21 participants were interviewed over the course of 6 weeks.

3.4 Data Analysis and Findings

This section presents the initial data analysis and preliminary findings on the factors
of HIS implementation and use in facilitating health care service delivery in public
healthcare facilities within resource constrained environments in South Africa's East-
ern Cape Province, based on the 21 interviews. The study built the semi-structured
interview questions around the ActAD framework, suggesting that healthcare service
delivery processes are determined by the interplay between mediators, tools (HIS) and
actors. Further questions employed different factors that mediate healthcare service
provision in the facilities -dependent variables such as diverse HIS usage or under-
utilization and whether the use adequately facilitates the work processes depend on
the interplay between the enabling and inhibiting factors. This paper presents the
preliminary findings in three themes of investigation: usage of tools, user
skills/competencies and functionality (sub-categorised into system functionality,
degree to which the system is used and how the failures affect other functions). The
section draws on Fig. 1 (as an analytical tool) to present the public healthcare service
delivery processes as an object activity system.

**Usage of tools (Health Information Systems)**

. Based on the initial findings, there is prevalent evidence of usage of tools in the
facility for facilitating health care service delivery in the hospital and clinic. Some of
the systems have been in use since the foundation of the hospital. The hospital and
clinic section are using a combination of electronic systems such as “*Delta9 (an
electronic health patient registration system)* (PRT1¹), *DHIS, Picture Archiving and
Communication System (PACS) and Radiology Information System (RIS)* (PRT1,
PRT2). In the laboratory unit of the hospital “...we use electronic gate keeping system
(in short we call it EGK) the system is used by the all the doctors to request for
patients’ laboratory test” (PRT3). The Delta9 system is an electronic system used for
patient registration “...at this unit of the hospital (patient registration), we use the
system to only capture and store patient biographical details and record the purpose
of the visit” (PRT1), “...the patients’ record unit also makes use of this system as
well... and also the revenue department makes use of this department for billing
purposes..." (PRT1). A major contradiction is the usage of paper-based system-
healthcare facilities still uses the paper folders to store patient clinical data – which
limits the sharing of data by multiple users, as paper files can only be in one point and
cannot be used in multiple areas at the same time. "... although we have these systems
here we are still required to record the things we do manually" (PRT3). Although the
electronic system is available the paper based system is still mainly used to store "...
the medical information that the doctors and clinicians have written or things like
laboratory reports they have requested on the patient" (PRT4). The clinics make use
of the Primary Healthcare information system (PHCIS) for patient data management
and reporting.

User Skills/ Competencies

. The availability and usage of relevant systems in various hospital units reflect a positive development in the findings, however these systems can only add value if their potentials are exploited fully. The analytical framework Fig. 1 suggests that there a need not only for the presence of the mediating enabling factors such as adequate ICT infrastructure and networks, but also the will of the subjects/actors to put their skills into effective use. User competencies in this paper is used as the understanding, literacy and ability to put a system into effective use [36]. For instance, framework the goal of a medical personnel and clerks/administrators is to provide health care services delivery through effective use of tools, rules and processes. For clerks, information systems such as Delta9 and the EGK systems based on the findings are the main tools (and mediators) - to effectively collect, capture, store, share and maintain the integrity and accuracy of patient data/information in the process of delivering healthcare. This same tools are also utilized by medical personnel to efficiently record, share patient diagnoses and prognoses at the facilities. The framework suggests that for the realization of these operational goals to take place depends on the interplay of the adequacy of ICT skilled personnel, relevant frequent training and readily available technical support for the users. The logic here is that unless the users are competent to effectively use the available tools, they cannot put it into use effective use. For instance PRT 4 indicates that "sometimes when the clerks enter the doctor's request into the system you find that they make mistakes that they should not be making like capturing duplicate requests or capturing lab requests that are not made by doctors".

Functionality

. The analytical framework emphasises the significance of relevant systems with
continuous availability and uninterrupted functionality. Findings reveal that a reliable
systems improve confidence in the pursuit of user's operational objectives. Also
revealed is that restricted infrastructure capacity, inadequate coordination of
networked systems and restricted technical support are the negative mediators
(inhibitors) of HIS usage by users in the hospital. Citing resource limitations such as
"...disjointed systems, interruptions of network, technical support for the users..."
PRT5 complains that they have a tendency of interrupting the provision of care.

## System functionality

Given the significance of systems reliability-continuous availability and functionality, healthcare facilities are expected not only to put appropriate systems in place but also to ensure their undisrupted presence. Continuous updates therefore, are important in ensuring relevance, so that systems can always react to the current operational needs of the organization. In Fig. 1, the functionality of systems is presented as one of the key enablers (mediating factors) of system usage, without which users cannot effectively achieve their goals. Unfortunately, the findings reveal limitation in the functionality of existing HISs some sections/units of the hospital. For instance, "...the radiology information system at radiology unit of the hospital had been non-functional for the past six months..." (PRT6). Similarly, users in other units within the hospital complain of challenges with the systems such going off-line or having technical issues. An administrator admits that the systems "...are very helpful, however there are many times when the systems go off-line and we have to wait for external people to come and fix the network... so we cannot do our jobs" Functionality failures are serious with various implications to the work processes of the users within the activity system. These include "long waiting periods for patients... sometime the patients have to referred to other hospitals or told to come back the next day, and the backlog of work" (PRT7). What has emerged from these findings is that, in spite of these functionality limitations, the overall user-attitudes towards the system have not been significantly impacted. In fact, it emerged in the findings that users are very fond of the Delta9 and LHIS systems. Opinions are that the issues with the systems, "Delta9 is very helpful in terms of looking up patients' information in a faster manner than the paper based system where you would be required to go into a store a manually go through stacks of paper work..." (PRT3). Another respondent said that apart from the network slowness issues, Delta9 "...makes my work very easy..." (PRT4).

## Degree of Utilization (tools)

As discussed earlier, system functionalities, based on the analytical framework, can add full value to the medical personnel and other users' work processes only if their potential is fully exploited. Having relevant reliable systems in place to support the daily operation of the users is paramount. To this, one participant mentions that "...especially with the Delta9 system, there are some modules of the system that we are not using that would even make our job better..." (PRT6). From the medical personnel point of a participant confirms that "...although we can receive lab result on our mobile phone, some of us rarely utilise that functionality" (PRT8). There is evidence in a correlation between functionality and usage limitations in the healthcare systems. Systems functionality failures have an inhibiting effect on the adequacy of health care delivery process in the public healthcare facilities. Apart from the system users that are affected by the system usage, other practitioners- because of the networking relationship in the working system, who depend on other system users for
relevant information. The head of ICT services at the hospital notes that "... we do
get a lot of frustrated users who log in call for issues they are facing..." (Participant
#7). Consultations with patients can be adversely affected.

In summary, ActAD framework suggests that, whilst usage of HIS to facilitate health-
care service delivery is effected by enabling mediators, inadequate usage is an out-
come of a negative interplay between the mediating factors, activities and actors.
There are a number of constraints such as limited capacity of the network relative to
the ever-increasing traffic and human errors as some of the reasons to the causes
which lead to many of the systems functionality failures. There is still evidence of
segmentation of infrastructure connectivity. Also, there seems to be a disconnection
between e-health strategies/policies and the actual practice of the implementation and
use of HISs in public healthcare facilities. With a focus on the context of a divergent
national service delivery reality that impacts the right to healthcare service in under-
served communities. This may suggest a need of a regulatory implementation frame-
work guiding the implementation process of the HISs in public healthcare facilities
within resource constrained environments.

References

1. Reichertz, Peter L. "Hospital information systems—Past, present, future." International journal of medical informatics 75, no. 3-4 (2006): 282-299.

2. Sharmin, S., Faith, B., Prieto Martin, P. & Ramalingam, B. The Contribution of Digital Technologies to Service Delivery: An Evidence Review, IDS Evidence Report 221, Brighton: (2017) IDS. http://opendocsids.ac.uk/opendocs/handle/123456789/12862.

3. Smedley, Alison. "The importance of informatics competencies in nursing: an Australian perspective." CIN: Computers, Informatics, Nursing 23, no. 2 (2005): 106-110.

4. Liu, Gordon Guoen, Yiqun Chen, and Xuezheng Qin. "Transforming rural health care through information technology: an interventional study in China." Health policy and planning 29, no. 8 (2013): 975-985.

5. Calligaro, G.L., Zijenah, L.S., Peter, J.G., Theron, G., Buser, V., McNerney, R., Bara, W., Bandason, T., Govender, U. & Tomasicchio, M. Effect of new tuberculosis diagnostic technologies on community-based intensified case finding: a multicentre randomised controlled trial. The Lancet Infectious Diseases, 17(4) (2017): 441–450.

6. Jacucci E, Shaw V, Braa J. Standardization of Health Information Systems in South Africa: the Challenge of Local Sustainability. In: Abiodun OB (ed). Proceedings of the IFIP 9.4 Working Conference on Enhancing Human Resource Development through ICT. Abuja, Nigeria, May 2005.

7. Venter S Hospital Information Systems: Producing accurate hospital data – A myth or pos-
sibility? Presentation at the Health Systems Trust Conference, 11 October 2007.

8. Mchunu, N.N. Adequacy of healthcare information systems to support data quality in the public healthcare sector, in the Western Cape, South Africa. 2013

9. Stansfield, Sally, Nosa Orobaton, David Lubinski, Steven Uggowitzer, and Henry Mwanyika. "The case for a national health information system architecture; a missing link to guiding national development and implementation." Making the eHealth Connection, Bellagio (2008).
10. BAKAR, Abubakar, Yahya SHEIKH, and Bakar SULTAN. "Opportunities and challenges of open source software integration in developing countries: case of Zanzibar health sector." Journal of Health Informatics in Developing Countries 6, no. 2 (2012).

11. Lucas, Henry. "Information and communications technology for future health systems in developing countries." Social science & medicine 66, no. 10 (2008): 2122-2132.

12. Shortliffe, Edward H., and Marsden S. Blois. "The computer meets medicine and biology: emergence of a discipline." In Biomedical Informatics, pp. 3-45. Springer, New York, NY, 2006.

13. Katuu, Shadrack, and Mpho Ngoepe. "Managing digital records within South Africa's legislative and regulatory framework." In 3rd International Conference on Cloud Security and Management ICCSM-2015, Tacoma, WA, University of Washington-Tacoma, pp. 59-70. 2015.

14. Damschroder, Laura J., et al. "Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science." Implementation science 4.1 (2009): 50.

15. Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P. & Kyriakidou, O. 'Diffusion of Innovations in Service Organisations: Systematic Review and Recommendations'. The Milbank Quarterly, 82(4) (2004): 581-629.

16. Omachonu, Vincent K., and Norman G. Einspruch. "Innovation in healthcare delivery systems: a conceptual framework." The Innovation Journal: The Public Sector Innovation Journal 15.1 (2010): 1-20.

17. Weeks, Richard. "Health care management: An e-health perspective." Editorial Advisory Committee (2012): 34.

18. Mladovsky, Philipa, Divya Srivastava, Jonathan Cylus, Marina Karanikolos, Tamás Evettovits, Sarah Thomson, and Martin McKee. "Health policy in the financial crisis." Eurohealth 18, no. 1 (2012): 3-6.

19. Bose, Ranjit. "Knowledge management-enabled health care management systems: capabilities, infrastructure, and decision-support." Expert systems with Applications 24, no. 1 (2003): 59-71.

20. Chassin, Mark R., and Robert W. Galvin. "The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality." Jama 280, no. 11 (1998): 1000-1005.

21. Rezazadeh, Esmaeil, Peyman Rezaei Hachesu, Aziz Rezapoor, and Khaksarian Alireza. "Evidence based medicine: going beyond improving care provider viewpoints, using and challenges upcoming." Journal of evidence-based medicine 7, no. 1 (2014): 26-31.

22. Reichertz, Peter L. "Hospital information systems—Past, present, future." International journal of medical informatics 75, no. 3-4 (2006): 282-299.

23. Häyrinen, Kristiina, Kaija Saranto, and Pirkko Nykänen. "Definition, structure, content, use and impacts of electronic health records: a review of the research literature." International journal of medical informatics 77, no. 5 (2008): 291-304.

24. Paul, Ray J., Inas Ezz, and Jasna Kuljis. "Healthcare information systems: a patient-user perspective." Health Systems 1, no. 2 (2012): 85-95.

25. Heeks, Richard. Implementing and managing eGovernment: an international text. Sage, 2005.

26. Keen, Peter GW, and Scott Morton. "MS (1978)." Decision support systems: An organizational perspective (1978).

27. Hirschheim, Rudy. "Information systems epistemology: An historical perspective." Research methods in information systems (1985): 13-35.
28. Friedman, Andrew L., and Dominic S. Cornford. Computer Systems Development: History Organization and Implementation. John Wiley & Sons, Inc., 1989.

29. Hyötyläinen, Raimo. Implementation of technical change as organizational problem-solving process: management and user activities. VTT Technical Research Centre of Finland, 1998.

30. Norros, L., K. Toikka, and R. Hyötyläinen. "Constructing Skill Based FMS-Lessons for Design and Implementation." IFAC Proceedings Volumes 23, no. 7 (1990): 151-156.

31. Hyötyläinen, R., L. Norros, and K. Toikka. "Constructing skill-based FMS-a new approach to design and implementation." IFAC Proceedings Volumes 23, no. 8 (1990): 53-58.

32. Hyötyläinen, Raimo. Implementation of technical change as organizational problem-solving process: management and user activities. VTT Technical Research Centre of Finland, 1998.

33. Kautz, Karlheinz. "Investigating the design process: participatory design in agile software development." Information Technology & People 24, no. 3 (2011): 217-235.

34. Vandenberghe, Frédéric. "Avatars of the collective: A realist theory of collective subjectivities." Sociological Theory 25, no. 4 (2007): 295-324.

35. Mursu, A., Luukkonen, I., Toivanen, M., and Korpela, M. Activity Theory in Information Systems Research and Practice: Theoretical Underpinnings for an Information Systems Development Model. Computer Science and IT Centre, Koupio University, Kuopio, Finland, Information Research (IR), 12 (3), April 2007 [Online]. Available from: http://informationr.net/ir/123/paper311.html [Accessed on 13 September 2018]

36. American Library Association, Information Literacy Competency Standards for Higher Education. The Association of College and Research Libraries. Chicago, Illinois. 2000.

37. Korpela, Mikko, et al. "Information systems research and development by activity analysis and development: dead horse or the next wave?." Information Systems Research. Springer, Boston, MA, 2004. 453-471.
i PRT = Participant