<figure><img src="image_1.png" /></figure>

HAL
open science

# Challenges in Information Systems Procurement in the Norwegian Public Sector

Carl Erik Moe, Tero Päivärinta

► To cite this version:

Carl Erik Moe, Tero Päivärinta. Challenges in Information Systems Procurement in the Norwegian Public Sector. 10th Electronic Government (EGOV), Aug 2011, Delft, Netherlands. pp.404-417, 10.1007/978-3-642-22878-0_34. hal-01589077

HAL Id: hal-01589077

https://inria.hal.science/hal-01589077

Submitted on 18 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Challenges in Information Systems Procurement in the
Norwegian Public Sector

Carl Erik Moe¹, Tero Päivärinta¹

¹Department of Information Systems, University of Agder,
Gimlemoen 25, 4604 Kristiansand, Norway
Carl.E.Moe@uia.no, Tero.Paivarinta@uia.no

**Abstract.** Public procurement of information systems (IS) and IS services provides several challenges to the stakeholders involved in the procurement processes. This paper reports initial results from a Delphi study, which involved 46 experienced procurement managers, chief information officers, and vendor representatives in the Norwegian public sector. The participants identified altogether 98 challenges related to IS procurement, divided further into 13 categories: requirements specification, change management, cooperation among stakeholders, competence, competition, contracting, inter-municipal cooperation, governmental management, procurement process, rules and regulations, technology and infrastructure, vendors, and IT governance. The results contribute by supporting a few previous findings from conceptual and case-based studies, and by suggesting additional issues which deserve both further research and managerial and governmental attention. As such, the results provide altogether a rich overview of the IS procurement challenges in the Norwegian public context.

**Key words:** Public procurement, procurement of information systems, Delphi study

## 1 Introduction

Public procurement, i.e. "the acquisition (through buying or purchasing) of goods and services by government or public organizations" [1], of information systems (IS) involves significant investments, where costs are covered by the taxpayers. For example, in Norway, a country of roughly 5 million inhabitants, the central government, municipalities and county governments used 6.6 billion NOK (825 million Euros) for purchasing hardware and software alone, in 2006 [2]. Procurement of IS consulting has been simultaneously increasing [3] due to a broader view of how IT is deployed in organizations.

Procurement of IS from the market is challenging, if compared to acquisition of more standardized goods and services. The systems often need to be customized to the needs of the public sector [4]. Moreover, procurement decisions are made early in
the process, when requirements are still uncertain [5]. The buyer may face challenges
to evaluate the bids against requirement specifications and to compare between
potentially competing, complex system options. By information systems we limit
ourselves here to software and systems implemented for specific organizational
purposes, such as enterprise resource planning and e-services tailored for the buyer's
needs. We thus exclude acquisition of off-the-shelf software, such as standard text
processing, e-mail, and other generic applications, from this study. In IS procurement,
the procurer also often needs to weigh between alternative IS implementation
strategies (e.g. [6]), such as in-house development, tailoring from proprietary
software, or adhering to open-source software ecosystems. Procurement of related IS
consulting services can also involve complexity, including difficulties in specifying
requirements, risks of incurring costs, and problems inherent in outsourcing
confidential knowledge to external consultants [7].

The public sector may face procurement challenges slightly different from the private business community. It is often bound by strict regulations concerning procurement and public tendering. For example, most European countries are required to publicly announce their call for tenders for all procurements above a certain threshold value; this applies to member states in EU, and in the EEA area. In addition, public projects are often large, both in terms of scope and of volume, which makes them risky. There are several incidents of significant overruns in time and costs, such as the Golf project for the Norwegian defense [8] or the NHS Programme for IT in England [9]. The above-mentioned issues make the field of public IS procurement a relevant, but challenging, area of research.

Our research question is: What challenges and dilemmas are typically faced in the procurement of information systems and related services in the public sector?

The paper reports results from two initial phases of a Delphi study among three expert panels related to IS procurement in the Norwegian public sector (procurement managers, CIOs, and vendor representatives). The experts identified a list of 98 challenges in public IS procurement projects, which were further divided into 13 categories. Based on the initial list, the experts and researchers came up with a narrowed-down list of 19 major challenges. A discussion of the major challenges contributes to the previous literature by confirming a few (mainly conceptual and case-based) findings and suggesting additional issues which deserve managerial and governmental attention for improving public IS procurement.

The rest of the article is structured as follows. Section 2 summarizes previous challenges of IS procurement identified in the public sector, as a basis for discussion of the results. Section 3 describes the research process and section 4 summarizes the results. Section 5 discusses the results in light of the previous research and section 6 concludes with suggestions for further research.

## 2 Previous work

A considerable body of research on procurement in the IS field has focused on on-line procurement or e-procurement, both in the public and in the private sectors [10-13]. E-procurement appears most beneficial when the purchased items have explicit
requirements or are manufactured according to common quality standards, and when there is a large pool of suppliers [12]. Hence this line of research seems to offer little insight into public procurement of more complex information systems and IS services. There is also a stream of research covering procurement policies in general. This line of research questions whether and how procurement can be used as an instrument for specific policies (such as stimulating innovations or development of green products). Other researchers cover how procurement of technology can be carried out, whereas a few has focused on the procurement process of IS itself.

The findings from previous research on public IS procurements and associated challenges can be summed up in a limited number of different issues or research topics (Table 1). These issues are discussed in more detail below.

<table><caption>**Table 1.** Summary of the findings from previous research</caption><thead><tr><td>Challenge</td><td>Reference</td></tr></thead><tbody><tr><td>Various stakeholders</td><td>[15], [16], [17]</td></tr><tr><td>Opportunism, from consultants</td><td>[7]</td></tr><tr><td>Limited interest from vendors, due to payment model and standard government contracts</td><td>[18], [19]</td></tr><tr><td>Balance between different socio-economic objectives, e.g. between partnership and fair competition</td><td>[20], [21], [22]</td></tr><tr><td>Specifying requirements before announcing tender</td><td>[19]</td></tr><tr><td>Focusing on life cycle cost and not just initial procurement costs</td><td>[23]</td></tr></tbody></table>

## 2.1 Stakeholders

A large body of general-level procurement literature addresses how the process itself is or should be carried out. Organizational buying involves multiple participants [14] and represents a decision process where many purchasing decisions are influenced by various members of the buying centre [15]. The public sector also involves the complexity of satisfying different needs of different stakeholders. The main conventional distinction between public and private organizations resides in ownership; whereas a limited group of entrepreneurs or shareholders owns private businesses, public agencies are owned collectively by members of political communities [16]. Organizations subject to political rather than economic controls are likely to face multiple and potentially conflicting sources of authority [16]. In general, public IS acquisitions involve several stakeholders with the challenges of balancing among their goals, which gives a starting point for this research.

## 2.2. Opportunism

Dawson et al. [7] discuss information asymmetry when clients procure services from IS consultants. They use the agency theory to identify possible manifestations of opportunism, and how this can be constrained. Whereas their work is highly conceptual, it introduces the challenge of opportunism between clients and consultants. Pan [17] uses a stakeholder analysis in a case study to analyse how a procurement process of an e-procurement system was abandoned. He claims the study
provides useful insight for practitioners on how to manage stakeholders in IS
development projects.

**2.3. Contract issues**

Research on policy for public acquisitions has focused on issues related to both
competition and contract types. These issues are expected to apply to public
procurement of IS as well. A large study of the choice of contracts in Indian software
industry find that reputation matters in terms of whether fixed price or cost-plus
contracts is used [24]. Bajari and Tadelis [25] develop a model inspired by data from
the private-sector construction industry and show that cost-plus contracts are
preferred when buyer and contractor share uncertainty about many important design
changes that occur after the contract is signed and production begins. These findings
should be relevant for procurement of information systems in instances when
requirements are incomplete.

However, findings from the public sector indicate that the regulations and contract arrangements are protective of the government customer, through particular payment models [26] and the use of standard government contracts [19]. This is suggested to limit the vendors' interest to participate in public tenders [19].

**2.4. Balance between socio-economic objectives**

One line of previous work has focused on challenges or conflicting goals in public
procurement. Thai states that public procurement must deal with a broad range of
issues [20], such as:

* Balancing the dynamic tension between a) competing socioeconomic objectives, and b) national economic interests and global competition as required by regional and international trade agreements;

* Satisfying the requirements of fairness, equity and transparency;

* Maintaining an overarching focus on maximizing competition; and

* Utilizing new technology to enhance procurement efficiency, including e-procurement and purchase cards.

However, Thai [20] provides little empirical evidence, and others have questioned the traditional way of doing procurement and suggest moving towards public procurement partnerships. Complexity of procuring information technology, software and IT-services is one of the reasons for such a move [21]. In a recent paper Loader [22] finds through a survey of 105 authorities in UK that 63% of local authorities have to some extent partnership with some suppliers, two thirds of the respondents recognizing partnership as being more likely to produce best value, rather than switching supplier through a tendering process. However the term partnership was not defined to the respondents, and we do not know to what extent they do practice partnership.
In spite of the goal of transparency and fair competition, there may be further reasons to believe that the process is not altogether transparent, and that not all competing vendors have equal opportunities. A qualitative study of IT procurement processes in private companies in four European countries [27] shows that the choice of vendors may be based on limited information. The findings indicate that the search for supplier is usually initiated either through contacting a known supplier or one that has been recommended, and on average a rather small amount of suppliers were actually contacted in this study.

As the public sector is large, the effect of public sector procurement may be big. The European Union (EU) policymakers have increasingly encouraged “public procurement of innovative products and services” as a policy instrument for raising private sector R&D investment in member states [1]. The documented outcomes indicate that public procurement has significant positive effects on innovation [28]. It is less clear if and how this normative recommendation is operationalized in IS procurements. Research has been carried out through surveys with large samples (in Germany, on effects on innovation).

**2.5. Specifying requirements**

The findings from Moe et al. [19] indicate tensions or dilemmas concerning creating requirement specification up front and doing the system specification as an integral part of the procurement process, and other dilemmas related to negotiations and fixed price contracts. The paper reports two case studies on procurement of fairly complex information systems in a Norwegian municipality. Whereas the general-level IS procurement literature recommends the in-house development strategy in case of unclear requirements [5], it may not be a realistic option for those public organizations with minimal internal IS development resources.

**2.6. Focus on initial procurement costs**

Tyssedal [23] finds that even though the Norwegian ministry of Defense has stated that the lowest life cycle cost should be preferred, still procurement in some projects are reviewed on the basis of initial procurement costs alone. Tyssedal [23] uses agency theory to explain this phenomenon, and suggest that agents may be more risk-averse than the principal. He finds only limited support for information symmetry between principal and agent affecting the use of life cycle cost for decision making.

The literature thus identifies a number of potential challenges of IS procurement, while giving a limited account of empirical data to validate the conceptual and normative recommendations. Moreover, there exists little systematic research on what other challenges there may be, or which of the challenges would be most prominent. One of the challenges that is referred to, is the issue of different stakeholders, and this may be more important in the public sector, but there is limited research on how this issue adds to the other challenges.
## 3 Research method, data collection

As the contemporary literature on the IS procurement challenges in the public sector remains without largely established theoretical or empirical grounds, we chose a research approach which orientates towards mapping the state-of-the-art expert opinions in the field. Hence, we chose the Delphi study method, which is often used in the field of information systems in order to find key issues and to explore an emerging topic [29], with three expert panels: municipal and regional procurement managers, chief information officers (CIOs) and representatives of vendors providing their systems and services to the public sector. This paper presents the preliminary results of the two first steps of the study. In general, we follow the process steps recommended for Delphi studies by Schmidt [30] and Okoli & Pawlowski [29].

The first activity was to select the experts for the study. We limited ourselves to inviting experts that are practitioners, representing the three different stakeholder groups mentioned above. We selected practitioners from different types of public entities of a reasonable size (municipalities, government run entities such as hospitals and entities in central government), where they possess a recognizably good level of experience. Likewise we selected experts from different categories of vendors with a considerable portion of the market for the public sector, including consultants, software houses offering systems targeted to public sector (e.g. systems for social services) and general software houses (e.g. offering ERP systems or systems for accounting and HRM). Experts were identified partly through suggestions from other experts, and partly through contacting the major vendors and municipalities.

In order to qualify as an expert, we set a minimum level of at least 3 years of experience in the current position or in a similar position with responsibility for either procurement or, on the vendor's side, selling. We chose to include procurement managers and CIO's, as both these groups should in general have expertise on IS procurement. We decided not to include user representatives as it turned out to be hard to find expert participants from this group. We set a requirement of having taken part in minimum 3 procurement processes of information systems or services. The procurement manager panel included 18, the CIO panel 17, and the vendor panel 11 participants. However we lost one procurement manager before the study was completed; hence there were 45 participants in total. Most of the CIO's and procurers are from municipalities (28), the rest (7) being employed in either health care organizations, counties (fylker) or state government.

The first phase of the actual Delphi study with the selected panels was the brainstorming of issues related to the research question. In this phase, we treated the experts as individuals. Each expert was asked to list at least 6 challenges or dilemmas of public IS procurement. Each issue has a shorter "name", definition, and a brief reasoning why this is important according to the expert in question. That is, what are the consequences of the challenge, if it is not managed, and what causes it? The experts e-mailed their lists to the researchers, thus remaining anonymous to each other. After gathering the challenges from the participants, the researchers unified the list, removed exact duplicates and unified terminology. The consolidated list of 96 identified challenges was sent back to experts who gave feedback to validate that the researchers had not dropped out any in this phase and that the researchers had not misinterpreted or changed meanings. The feedback resulted in the list which was
added with 2 challenges. Some of these fit perhaps less within a text book definition of the procurement phase, e.g. "Change of work processes and benefit realization", or "Management of problems after delivery". However all challenges in the consolidated list were identified by experienced procurers and we thus decided to include them, instead of relying on our own interpretations or prejudices about the relevant challenges within the procurement theme.

The second round narrowed down the brainstormed list to a manageable number of the most important issues, to be ranked within each panel. Now, we divided the experts into the three panels described above. In each panel, the experts defined around 20 issues that they considered as "most important". The presentation order of these 98 factors was randomized to the varying panel members to avoid bias related e.g. towards choosing factors from the top of the list. This study reports these preliminary results.

The third phase of the Delphi study will aim at a consensual ranking of the relative importance of the issues identified as most important in the narrowing-down phase. Despite the fact that the Delphi research was in process while writing this paper, proceeding towards the panels ranking the issues, the collection of the top issues already provide us with food for discussion and theory creation.

# 4 Results

The consolidated list from the brainstorming phase consisted of 98 challenges and dilemmas. The challenges are divided further into 13 higher-level categories, which help to organize and to get an overview of the results. Whereas no readily theorized category structure for the theme of procurement existed, the researchers grouped the challenges in this phase according to more common themes which they interpreted to emerge from the brainstorming data. The categorization has a purely pragmatic role for the further process, and it should thus not be considered as a "theorizing" effort as such. The narrowing-down-phase included altogether 19 issues (table 1), which were selected as follows. Firstly, we selected a "top ten" list based on the votes in total. This resulted in a selection of 12 issues (the 10th place was shared by three issues).

Then we had to find out whether there were large differences between the groups. Kendall's tau values, which measure the agreement between the panels [30], from this phase showed some positive and significant correlations between the panel-wise selections for the narrowed-down lists:

* procurement managers – CIOs: 0,474, sig 0,000;

* procurement managers – vendors: 0,205, sig 0,006; and

* CIOs – vendors: 0,234, sig 0,004.

However, as all the Kendall's tau values between the panels were less than 0,5 (values above 0,5 would have indicated strong inter-panel agreement) we decided to form a narrowed-down list, which would include the most important issues identified in each panel, in addition to those issues identified important across the panels. Hence, we decided to include the panel-wise challenges chosen by more than 50% [30] of members in each panel (Table 2).
<table><caption>**Table 2.** Major challenges. The list shows the 19 challenges that were selected after the narrowing-down phase. The numbers in brackets show how many from the different groups that selected this challenge as one of the more important ones.</caption><thead><tr><th>#</th><th>Total votes, N=45</th><th>Procurement managers, N=17</th><th>CIOs, N=17</th><th>Vendors, N=11</th></tr></thead><tbody><tr><td>1</td><td>2.1 Change of work processes and benefits realization (30)</td><td>4.1 Procurement competence (13)</td><td>2.1 Change of work processes and benefits realization (12)</td><td>1.3 Sober requirements (7)</td></tr><tr><td>2</td><td>9.1 Lack of coordination and standardization (of the procurement process) (24)</td><td>2.1 Change of work processes and benefits realization (12)</td><td>10.1 Complex regulations (12)</td><td>10.5 Tendering obligation may conflict with long-term planning (switching costs) (7)</td></tr><tr><td>3</td><td>4.1 Procurement competence (23)</td><td>1.1 Clear requirements (10)</td><td>9.1 Lack of coordination and standardization (of processes / services) (11)</td><td>5.6 The vendor is not given an opportunity to show its qualities (7)</td></tr><tr><td>4</td><td>10.1 Complex regulations (22)</td><td>1.2 Complete requirements (9)</td><td>11.2 Integration, compatibility (11)</td><td>13.9 Too much focus on costs (7)</td></tr><tr><td>5</td><td>11.2 Integration, compatibility (20)</td><td>11.2 Integration, compatibility (9)</td><td>6.6 Framework contracts (9)</td><td>2.1 Change of work processes and benefits realization (6)</td></tr><tr><td>5</td><td>3.3 Co-operation between different stakeholders (20)</td><td>12.2. Vendors “oversell” (9)</td><td></td><td>9.1 Lack of coordination and standardization (of processes/services) (6)</td></tr><tr><td>5</td><td>5.3 Weighing / Prioritization between vendor evaluation criteria (20)</td><td>5.5 Monopoly-resembling vendor conditions (9)</td><td></td><td>3.3 Co-operation between different stakeholders (6)</td></tr><tr><td>8</td><td>1.1 Clear requirements (19)</td><td></td><td></td><td>5.3 Weighing / prioritization between vendor criteria (6)</td></tr><tr><td>9</td><td>10.5 Tendering obligation may conflict with long-term planning (switching costs) (18)</td><td></td><td></td><td>10.3 Partnership and innovation is hindered (6)</td></tr><tr><td>10</td><td>7.1 Municipal cooperation is challenging (17)</td><td></td><td></td><td></td></tr><tr><td>10</td><td>12.2. Vendors “oversell” (17)</td><td></td><td></td><td></td></tr></tbody></table>
<table><tr><td>10</td><td>10.3 Partnership and innovation is hindered (17)</td><td></td><td></td></tr><tr><td>10</td><td>1.10 & 5.2 Finding good criteria for vendor evaluation (17)</td><td></td><td></td></tr></table>

The differences between the three groups did result in some challenges being included due to one group only finding them among the most important. For example, “Vendors oversell” is an issue, which none of the participants from the vendor side found worth listing, whereas “The vendor is not given an opportunity to show its qualities” was listed by more than half the vendor participants but by none of the others. “Sober requirements” is another example along the same vein.

This heuristic resulted in additional six challenges to be included in further analysis. In this phase, we also realized that two challenges (1.10 and 5.2) were so close to each other that they could be merged to one issue of “Finding good criteria for vendor evaluation. Then, we calculated individual votes altogether for this merged criteria – and it was, indeed, making itself as an additional (13th) criterion to be included among the “top 10”.

In general, the narrowed-down list involves all but one (governmental management) of the 13 categories identified in the first phase. That is, the major challenges relate to many issues, and require holistic attention among the stakeholders. Three of the categories had more than one issue among the top 19: ensuring competition, requirements specification, and rules and regulations. However, the number of issues per category does not necessarily indicate the relative importance of the category. The experts highlight also significant challenges related to change management, cooperation among different stakeholders of procurement, competence, contracting, inter-municipal co-operation, the procurement process, technology and infrastructure, vendors, and organizational IT governance in general.

<table><caption>**Table 3.** The narrowed-down list of 19 major challenges divided into the 13 categories</caption><thead><tr><th>#</th><th>Category</th><th>Top 19 Challenges and Dilemmas</th></tr></thead><tbody><tr><td>1.</td><td>Requirements specification</td><td>1.1 Clear requirements<br>1.2 Complete requirements<br>1.3 Sober requirements</td></tr><tr><td>2.</td><td>Change management</td><td>2.1 Change of work processes and benefits realization</td></tr><tr><td>3.</td><td>Different stakeholders, cooperation</td><td>3.3 Co-operation between different stakeholders</td></tr><tr><td>4.</td><td>Competence</td><td>4.1 Procurement competence</td></tr><tr><td>5.</td><td>Competition</td><td>5.2 (& 1.10) Finding good criteria for vendor evaluation.<br>5.3 Weighing/prioritization of vendor evaluation criteria<br>5.5 Monopoly-resembling vendor conditions<br>5.6 The vendor is not given an opportunity to show its qualities</td></tr><tr><td>6.</td><td>Contracting issues</td><td>6.6 Framework contracts</td></tr><tr><td>7.</td><td>Cooperation between municipalities</td><td>7.1 Municipal cooperation is challenging</td></tr><tr><td>8.</td><td>Governmental management</td><td></td></tr></tbody></table>
<table><tr><td>9.</td><td>Procurement process</td><td>9.1 Lack of coordination and standardization (of the work processes / services)</td></tr><tr><td>10.</td><td>Rules and regulations</td><td>10.1 Complex regulations<br>10.3 Partnership and innovation is hindered<br>10.5 Tendering obligation may conflict with long-term planning (switching cost to change vendor)</td></tr><tr><td>11.</td><td>Technology and infrastructure</td><td>11.2 Integration, compatibility</td></tr><tr><td>12.</td><td>Vendors</td><td>12.2. Vendors “oversell”</td></tr><tr><td>13.</td><td>Governance of IT and the organization</td><td>13.9 Too much focus on costs</td></tr></table>

Among the five issues with most overall votes from all experts across the panels, the most voted one was change management of work processes and benefits realization, which thus should be considered already during the procurement. The next issue was lack of coordination and standardization of the procurement process, being especially in the top agenda of CIOs and vendors. The third issue, procurement competence, was especially highlighted by the procurement managers. Complexity of regulations was also among the top five of CIOs, whereas the fifth issue concerning technological integration and compatibility challenges of the purchased systems was in the agenda of both CIOs and procurement managers.

In addition to the above-mentioned issues, the procurement managers especially highlighted the challenges of getting clear and complete requirements for the procurement process. They also mentioned two challenges related to vendors. That is, the procurement managers experience that in some areas particular vendors may have a monopoly-like position, whereas sometimes the customers also experience the vendors “overselling” and thus creating exaggerated expectations.

The CIO panel lifted up the challenges related to the framework contracts in general, such as decreased flexibility to choose the best partners and systems for particular needs. The vendor panel seems to have a slightly differing view on the challenges, highlighting sober and realistic requirements from their customers, problems with the need for opening new projects for tendering instead of longer-term co-operations, problems with foci of requirements hindering the vendors from showing their qualities, and the overall focus on costs only among the public sector procurers.

Co-operation challenges exist between different stakeholders within the public sector customers but also between the municipalities; sometimes networks of municipalities pursue joint acquisitions of systems. In addition, rules and regulations are regarded to hinder longer-term vendor-customer partnerships, when new projects need to be opened for public requests of bids. Finally, solid vendor evaluation criteria are hard to find and prioritize.

## 5 Discussion

The results, i.e. the major 19 challenges identified in public sector procurement of IS, confirm a few previous findings (see table 1), but also reveal a group of new challenges, which have not been discussed in detail in previous research. There was
also one previously identified issue, which was not directly among the challenges of
this study.

The issue of the vendors “overselling” their ability to deliver products and services denotes potential for opportunistic vendor behaviour, confirming the ideas of Dawson et al. [7]. Alongside with the overall differences among the panel prioritizations, our study thus supports their suggestions for improving stakeholder management in public IS procurements. The previously identified challenge of focusing too much on initial procurement costs in a case study from the Norwegian ministry of Defence [23] is also present in our list.

The issues of finding good vendor evaluation criteria and to balance among different objectives (such as fair competition and partnership) [20-22] were also well visible in our list of major challenges. Transparency for ensuring fair competition between the potential vendors is clearly a public-sector-specific challenge, whereas the private firms can be more pragmatic on these issues. The classic challenge of coordinating between various stakeholders in procurement in general [15] and in the public sector IS investments [17] was also one of the major challenges. Our results thus support the previous calls for more focus on managing these challenges in public procurement practices and processes. In addition, municipal cooperation is found to be one of the major challenges, and some of the participants are from municipalities that are highly involved in networks with neighbour municipalities where they try to use procurement as an instrument to negotiate better prizes, and to move to a more shared portfolio of systems.

The experts highlight importance on clear, complete, and sober requirements specifications, which has been perhaps less recognized as an issue for public sector procurement, if compared to the field of systems development in general. However, increased focus on requirements specifications may be especially important for the public sector, as procurement regulations specify tendering as the default instrument for enhancing fair competition and requirements must often be clarified up-front before talking to vendors [19]. This regulation-initiated challenge may be controversial to the observations made in the software engineering and information systems literature since the end of 1970s, which have recognized the difficulties for defining “complete” and “clear” ex-ante requirements and the fact that requirements tend to change during development (e.g. [31]). Varying IS procurement strategies for coping with “standard” requirements versus complex projects with uncertain requirements has also been suggested for some time [5]. However, many public-sector organizations can rarely develop systems internally, despite such a strategy for procurement being recommended for projects with unclear requirements [5]. Our data thus confirms that this dilemma of ex-ante requirements definitions for IS procurement largely remains to be solved in the Norwegian public sector.

To our slight surprise, the issue of facilitating change in work processes and benefits realization was regarded as an important challenge that relates to procurement as well. We regard this as a new finding in the field of public IS procurement. As such, this finding supports the benefits realization literature, which highlights the importance of planning for the expected benefits from IS investments from early on (e.g. [32]). This issue has not been identified in the previous literature of the public IS procurement. However, the challenge of establishing benefits realization practices from IT investments in the Norwegian public sector has been
recently identified in other studies [33, 34]. Our results thus motivate further research, development and education initiatives on public benefits realization practices, linking them also to the procurement process of IS investments.

A need for coordination and standardization of IS procurement processes (among organizations) is probably specific to the public sector. In the public sector, among the altogether 430 municipalities plus other public sector organizations in Norway, such coordination may be possible in the first place. However, procurement standardization would probably represent a big public project in itself, to be coordinated by the state-level government. The issue of procurement competence was also highly ranked, which has surely some implications for both governmental and educational institutions. This is not surprising as IS procurement is a complex field due to the risk associated with large IS projects and due to the complex regulations.

The three issues related to rules and regulations (complex rules, hindering partnerships and long-term planning) for public IS procurement has also got less attention in the previous literature. This issue surely relates both to the dilemma of ex-ante requirement definitions and the call for increased competence. Interestingly, increased competence comes out top on the list from procurement personnel whereas complex rules comes out near the top from CIO's, it may well be that procurement personnel sees the need for increased competence due to the complex rules, whereas CIO's are more challenged by the rules themselves. Framework contracts, on the other hand, may hinder flexibility to adopt new ideas from the non-included vendors. However, these issues may also give food for thought for the policy-makers and the framework-contractors, in order to make the current rules better suited for information systems projects. These challenges may be present also in parts of the private sector; rules and regulations may be just as complex in larger corporations.

The challenge of considering technological integration and compatibility of the systems highlights the importance to involve the IT and operations experts of the customer organization in the procurement process. In Norway, which is a small country, some vendors may have also reached a "monopoly-like" position in some niche areas specific to the public sector. On the other hand, due the regulations and strong focus on ex-ante requirements defined by the customer, some vendors may have few opportunities to show their specific qualities, if the customers cannot ask for that. The challenge of inter-municipal co-operation may be also a case characteristic to the Norwegian context, where the municipalities have been so far rather independent with regard to their IT/IS implementations. Whereas 2/3 of CIOs and procurement managers in our panels were employed in the municipal sector, but not all municipalities participate in inter-municipal procurement networks, this issue may appear a bit contextual for particular organizational settings.

Interestingly enough, the challenge of "uninterested vendors" due to complex regulations and practices in the public sector, identified in the previous research [19], was not present among the 98 identified challenges *per se*. This might have something to do with our selection of panels – i.e., by involving only vendors which already are active with the public sector. On the other hand, the issue of "monopoly-resembling vendor conditions", identified by the customer organization representatives, likely relates to this issue, taken that the scarcity of the vendor selection would be a consequence of such disinterest to overcome the public bureaucracy threshold.
# 6 Conclusions and further work

This Delphi study revealed typical challenges of IS procurement in the Norwegian public sector. Three expert panels, involving procurement managers, CIOs, and vendors, defined altogether 98 challenges and dilemmas, divided into 13 categories: requirements specification, change management, co-operation among stakeholders, competence, competition, contracting, inter-municipal co-operation, governmental management, procurement process, rules and regulations, technology and infrastructure, vendors, and IT governance. The results provide a rich overview on the challenges and complement the previous, largely conceptual and case-based, literature on public IS procurement challenges. The study supports the previously identified challenges related to stakeholders, vendor opportunism, standard government contracts, balancing between objectives, requirement specification for tendering, and plain focus on costs. In addition, the study revealed previously less discussed challenges of public IS procurement, such as aligning benefits realization to procurement, coordinating and standardizing public procurement processes, procurement competence, complex and constraining regulations, rigid framework contracts, issues of technological integration and compatibility, monopolistic vendor positions, unawareness of particular vendor qualities, and inter-municipal co-operation. In general, the identified major issues provide food for thought for Norwegian researchers, policy-makers, and practitioners in the field.

Our further research efforts will focus on finalizing the Delphi study with panel-
wise prioritizations of the issues in order to see whether the experts would reach
further consensus about the definitely most important issues, and whether the
stakeholder prioritizations vary. We plan also to conduct in-depth studies to form
theories of causes and effects related to the most important issues. Whereas the
Delphi analysis alone reveals little or no relationships between the issues, our further
work will focus on creation of cause-effect relations between the most commonly
observed issues through qualitative analyses of the brainstorming data and additional
field work. Another natural avenue for further work resides in cross-country studies,
which might reveal more information about generalizability of these results beyond
the Norwegian context.

References

1. Hommen, L. and M. Rolfstam, *Public Procurement and Innovation: Towards a Taxonomy*. Journal of Public Procurement, 2009. 9(1): p. 17-56.

2. SSB. Offentlig sektor kjøpte datautstyr og programvare for 6,6 milliarder i 2006. 2008-08-07-01]; Available from: http://www.ssb.no/magasinet/analyse/art-2008-05-07-01.

3. Iyer, G., S. Ravindran, and P.M.J. Reckers, *Procurement of IT Consulting Services and Firm-Specific Characteristics*. Journal of the Association of Information Systems, 2006. 7(4): p. 207-239.
4. Keiichiro, I. and M. Hajime. *How Local Governments Should Procure Information Systems? (Abstract in English)*. in *Proceedings of the 19th Annual Conference on JSAI*. 2005.

5. Saarinen, T. and A.P.J. Vepsäläinen, *Procurement Strategies for Information Systems*. Journal of Management Information Systems, 1994. **11**(2): p. 187-208.

6. Hallikainen, P. and H. Kivijärvi. *Determining Information Systems Procurement Strategies - AHP Approach*. in *The Fifth International Symposium on The Analytic Hierarchy Process*. 1999. Kobe, Japan.

7. Dawson, G.S., R.T. Watson, and M.-C. Boudreau, *Information Asymmetry in IS Consulting: Towards a Theory of Relationship Constraints*. Journal of Management Information Systems, 2011. **27**(3): p. 143-178.

8. Riksrevisjonen. *Riksrevisjonens oppfølging av forvaltningsrevisjoner behandlet av Stortinget*. 2008 [cited 2009 12.08]; Available from: http://www.riksrevisjonen.no/NR/rdonlyres/3AC09C1F-1BFD-40D3-B278-5186BB6D548D/0/Dok_3_1_2007_2008.pdf.

9. BBC. *Warning over fresh NHS IT delays*. 2009 Retrieved 12.08, 2009]; Available from: http://news.bbc.co.uk/2/hi/health/7850619.stm.

10. Davila, A., M. Gupta, and R.J. Palmer, *Moving Procurement Systems to the Internet: The Adoption and Use of E-procurement Technology Models*. European Management Journal, 2003. **21**(1): p. 11-23.

11. Gebauer, J., C. Beam, and A. Segev, *Impact of the Internet on Procurement*. Acquisition Review Quarterly, 1998: p. 167-184.

12. Hsiao, R.-L. and T.S.H. Teo, *Delivering on the Promise of E-Procurement*. MIS Quarterly Executive, 2005. **4**(3): p. 343-360.

13. Henriksen, H.Z. and V. Mahnke, *E-Procurement Adoption in the Danish Public Sector: The Influence of Economic and Political Rationality*. Scandinavian Journal of Information Systems, 2005. **17**(2): p. 85-106.

14. Wind, Y. and R.J. Thomas, *Conceptual and methodological Issues in Organisational Buying Behaviour*. European Journal of Marketing, 2001. **14**(5/6): p. 239-263.

15. Spekman, R.E. and L.W. Stern, *Environmental Uncertainty and Buying Group Structure: Empirical Investigation*. Journal of Marketing, 1979. **43**(Spring): p. 55-64.

16. Boyne, G.A., *Public and Private Management: What's the difference?*. Journal of Management Studies, 2002. **39**(1): p. 97-122.

17. Pan, G.S.C., *Information systems project abandonment: a stakeholder analysis*. International Journal of Information Management, 2005. **25**(2): p. 173-184.

18. Doshi, B., *The New OGC guidance: The future raodmap for government IT procurement*. Computer Law and Security Report, 2005. **21**(4): p. 344-348.

19. Moe, C.E., A.K. Risvand, and M.K. Sein, *Limits of Public Procurement: Information systems acquisition, in Electronic Government - 5th International Conference, EGOV 2006*. 2006, Springer Verlag. p. 281-292.

20. Thai, K.V., *Advancing Public Procurement: Practices, Innovation and Knowledge Sharing*, K.V. Thai and G. Piga, Editors. 2006, PrAcademics Press.
21. Lawther, W.C. and L.L. Martin, *Innovative practices in public procurement partnerships: The case of the United States*. Journal of Purchasing and Supply Management, 2005. **11**(5-6): p. 212-220.

22. Loader, K., *Is local authority 'lean'? An exploration to determine if 'lean' can explain a useful explanation of practice*. Journal of Purchasing and Supply Management, 2010. **16**(1): p. 41-50.

23. Tyssedal, B.E., *Life cycle cost based procurement decisions. A case study of Norwegian Defence Procurement projects*. International Journal of Project Management, 2008. **26**(4): p. 366-375.

24. Banerjee, A.V. and E. Duflo, *Reputation Effects and the Limits of Contracting: A Study of Indian Software Industry*. Quarterly Journal of Economics, 2000. **115**: p. 989-1018.

25. Bajari, P. and S. Tadelis, *Incentives versus transaction costs: a theory of procurement contracts*. RAND Journal of Economics, 2001. **32**: p. 387-407.

26. Doshi, B., *The new OGC guidance: The future roadmap for government IT procurement*. Computer Law and Security Report, 2005. **21**(4): p. 344-348.

27. Schiessl, M. and S. Duda. *A Qualitative Oriented Study About IT Procurement Processes: Comparison of 4 European Countries*. in *Second International Conference on Usability and Internationalization, UI-HCII 2007, Held as Part of HCI International 2007*. Beijing, China: Springer Verlag.

28. Aschhoff, B. and W. Sofka, *Innovation on demand - Can public procurement drive market success of innovations? Research Policy*, 2009. **38**: p. 1235-1247.

29. Okoli, C. and S.D. Pawlowski, *The Delphi method as a research tool: an example, design considerations and applications*. Information & Management, 2004. **42**: p. 15-29.

30. Schmidt, R.C., Managing Delphi Surveys Using Nonparametric Statistical Techniques. Decision Sciences, 1997. **28**(3): p. 763-774.

31. Parnas, D., *Designing Software for Ease of Extension and Contraction*. IEEE Transactions on Software Engineering, 1979. **5**(2): p. 128-138.

32. Ward, J. and E. Daniel, *Benefits Management: Delivering Value from IS & IT Investments*. 2006, Chichester: Wiley.

33. Päivärinta, T., W. Dertz, and L. Flak. *Issues of Adopting Benefits Management Practices of IT Investments in Municipalities: A Delphi Study in Norway*. in *Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS)*. 2007: IEEE.

34. Päivärinta, T. and W. Dertz, *Pre-Determinants of Implementing IT Benefits Management in Norwegian Municipalities: Cultivate the Context*. In: *Electronic Government - 7th International Conference, in EGOV 2008*. 2008, Springer Verlag, LCNS 5184. p. 111-123.