# 1_EN_autre_document_travail_service_part1_v4.docx

european flagEUROPEAN COMMISSION

Brussels, 19.4.2016

SWD(2016) 108 final

COMMISSION STAFF WORKING DOCUMENT

Implementation and Evaluation report

Accompanying the document

Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions

EU eGovernment Action Plan 2016 - 2020Accelerating the digital transformation of government

{COM(2016) 179 final}{SWD(2016) 109 final}

Table of contents

1.Introduction

1.1.The purpose of the evaluation

1.2.The scope of the evaluation

2.Background to the EU eGovernment Action Plan 2011 - 20156

2.1.Description of the European eGovernment Action plan 2011 – 2015 and its objectives6

2.2.Instruments8

2.2.1.Political Activities including exchange of experience and Commission Communications8

2.2.2.Funding for eGovernment9

2.2.3.Legislation10

2.3.Intervention logic11

2.4.Situation in 2010/Baseline12

2.4.1.Baseline of target: 50% of citizens (Digital Agenda) and 80% of businesses make use of eGovernment services13

2.4.2.Baseline of target: a number of key cross-border services be offered online by 2015 (Digital Agenda)13

2.4.3.Baselines in the four policy priority areas13

3.Evaluation Questions/Criteria15

4.Method and sources16

4.1.Mid-term evaluation16

4.2.Annual benchmarking reports16

4.3.The online public consultation17

4.4.Limitations18

5.Implementation of the EU eGovernment Action Plan 2011 – 2015 – state of play of the policy priorities19

5.1.General target: 50% of citizens (Digital Agenda) and 80% of businesses make use of eGovernment services19

5.2.General target: a number of key cross-border services be offered online by 2015 (Digital Agenda)19

5.3.The four priorities and 40 actions of the EU eGovernment Action Plan22

5.4.Governance22

6.Answers to the evaluation questions24

6.1.Effectiveness24

6.1.1.Objective on using eGovernment (take-up)24

6.1.2.Key cross-border services online25

6.1.3.Effectiveness of the four policy priorities26

6.2.Efficiency36

6.2.1.Efficiency of the related budget36

6.2.2.Costs and benefits to stakeholders38

6.3.Relevance39

6.3.1.Relevance: User Empowerment40

6.3.2.Relevance: Internal market42

6.3.3.Relevance: "Efficiency and Effectiveness of Governments and Administrations"45

6.3.4.Relevance: Pre-conditions48

6.4.Coherence49

6.5.EU-added value51

7.Conclusions54

8.ANNEXES57

Annex 1.Procedural information57

Annex 2.The Online public consultation59

Annex 3.The individual actions of the eGovernment Action Plan 2011 - 201589

Annex 4.Implementation of the individual actions of the eGovernment Action Plan 2011 – 2015 – State of play of the actions93

Priority: User Empowerment93

Priority: Internal Market96

Priority: Efficiency and Effectiveness of Governments and Administrations98

Priority: Pre-conditions for developing eGovernment100

Annex 5.Acronyms103

Annex 6.REFERENCES AND DATA SOURCES105

1.Introduction

1.1.The purpose of the evaluation

In December 2015, European Commission's Directorate-General Communications Networks, Content and Technology launched an implementation and evaluation assessment of the European eGovernment Action Plan 2011 – 2015
  1
  ("EU eGovernment Action Plan 2011 – 2015"), adopted on 15th December 2010. The purpose of this evaluation is to describe the progress of implementation of individual actions of the Action Plan and assess its functioning. In the broadest sense, this evaluation exercise will provide answers to how well the European public administrations have progressed towards meeting the Malmö vision (November 2009 Malmö Ministerial declaration
  2
  ), which is included in this eGovernment Action Plan:

"by 2015 European public administrations will be recognised for being open, flexible and collaborative in their relations with citizens and businesses. They use eGovernment to increase their efficiency and effectiveness and to constantly improve public services in a way that caters for user's different needs and maximises public value, thus supporting the transition of Europe to a leading knowledge-based economy."

The EU eGovernment Action plan 2011-2015 did not indicate the need to carry out a final evaluation as such. According to the eGovernment Action Plan, the overall progress would be measured annually, using an appropriate mix of instruments and methods. However, according to the Commission's Better Regulation guidelines, retrospective performance evaluations support policy preparation
  3
  . "Evaluations gather evidence to assess how well a specific intervention has performed (or is working) - -. An evaluation also draws conclusions on whether the EU intervention continues to be justified or should be modified to improve its effectiveness, relevance and coherence or to eliminate excessive burdens or inconsistencies or simply be repealed." This evaluation is carried out in so far as possible in this context, in order to provide a set of lessons learnt to guide the work of the new eGovernment Action Plan 2016 – 2020 announced in the Digital Single Market Strategy
  4
  .

1.2.The scope of the evaluation

The scope of the evaluation is the objectives and measures set out in the EU eGovernment Action Plan 2011 – 2015. It covers the level of progress in eGovernment policy in the EU in the period starting from the beginning of 2011 (Action Plan was adopted 15 December 2010) until and including 2015, subject to the availability of most recent data.

The Action Plan 2011 – 2015 aimed to contribute to achieving two important targets of the Digital Agenda in Europe
  5
  : 1) "By 2015, 50% of EU citizens will have used eGovernment services"; and 2) offering a number of cross-border services online. The Action Plan also aimed that "by 2015, 80% of enterprises will have used eGovernment".

Furthermore, the evaluation focuses on the four policy priorities (agreed by Commission and Member States in the Malmö Ministerial Declaration) of the EU eGovernment Action Plan 2011 – 2015:

1.User Empowerment

2.Internal Market

3.Efficiency and Effectiveness of Governments and Administrations

4.Pre-conditions for developing eGovernment

The Action Plan included 40 individual actions (listed in Annex 4) divided amongst these four priorities. Five additional actions were related to the governance.

The evaluation focuses on the Action Plan as a political instrument to achieve its objectives. It does not evaluate in detail individual actions identified in the Action Plan. An assessment of the completion of actions will be done separately via a study planned for 2016. The results of the study will be used to update to complete this evaluation. Furthermore, since the scope of the evaluation is the policy priorities eGovernment Action Plan 2011 – 2015, it will not cover eGovernment related activities in Member States at all levels of government and all sectors.

For the evaluation of the eGovernment policy priorities in the EU it is important to note that eGovernment has also been supported through a number of funding programmes (listed in section 2.2.2). The evaluation of these funding programmes falls outside the scope of this evaluation but their findings are referred to in this evaluation when relevant to address the evaluation criteria.

The evaluation criteria used are in line with the Commission's Better Regulation Guidelines and assess the effectiveness, efficiency, relevance, coherence and the EU added value of the EU eGovernment Action Plan 2011 - 2015.

2.Background to the EU eGovernment Action Plan 2011 - 2015

2.1.Description of the European eGovernment Action plan 2011 – 2015 and its objectives

The Europe 2020 strategy that was put forward by the Commission on 3 March 2010
  6
  set out a strategy for smart, sustainable and inclusive growth to face the immediate challenges resulting from the financial crisis and long-term challenges such as resource scarcity, ageing and globalisation. One of the so-called seven flagship initiatives, the Digital Agenda for Europe
  7
  , outlined a number of actions supporting the further development of eGovernment in the EU. The actions can be found from the eGovernment chapter of the Digital Agenda Communication and from the eGovernment Action Plan that was the successor of the i2010 Action Plan 2006-2010
  8
  .

The Action Plan 2011 – 2015 aimed to contribute to achieving two important targets of the Digital Agenda in Europe: 1) that 50% of citizens make use of eGovernment services; and 2) that a number of key cross-border services will be available online by 2015. The Action Plan also aimed that "by 2015, 80% of enterprises will have used eGovernment". Furthermore, the objective was to translate into concrete measures the four political priorities of the Malmö Ministerial Declaration on eGovernment of December 2009. Subsequently, the eGovernment Action Plan had the following four policy priorities (see also

  Figure 1

  ):

"1.
     User Empowerment

Empowerment means increasing the capacity of citizens, businesses and other organisations to be pro-active in society through the use of new technological tools. Public services can gain in efficiency and users in satisfaction by meeting the expectations of users better and being designed around their needs and in collaboration with them and third parties whenever possible. Empowerment also means that governments should provide easy access to public information, improve transparency and allow effective involvement of citizens and businesses in the policy-making process.

2.
     Internal Market

Most public online services do not work across borders or involve cumbersome procedures to be accessible. People from one EU country cannot easily apply for public services in another country than the one in which they are established, using for instance national electronic identity cards. This seriously reduces the mobility of businesses and citizens. To support the Internal Market, governments should develop ‘seamless’ services for entrepreneurs to set up and run a business anywhere in Europe and allowing individuals to study, work, reside, receive health care and retire anywhere in the European Union.

3.
     Efficiency and Effectiveness of Governments and Administrations

The aim was to use ICT and enabling organisational changes to deliver better, less intrusive, more sustainable and faster public services, by reducing the administrative burden, improving organisational processes and promoting a sustainable low-carbon economy.

4.
     Pre-conditions for developing eGovernment

A number of technical and legal pre-conditions need to be put in place to enable the implementation of the actions that will enhance eGovernment services in Europe. These include the promotion of interoperability across borders, which would allow - among others - sharing of information, deployments of one-stop-shop approaches, Europe wide use of (national) electronic identity solutions and payment schemes. Interoperability is supported through open specifications and the development of key enablers such as electronic identity management and stimulation of innovation in eGovernment."

Figure 1 The four policy priorities of the eGovernment Action plan 2011 – 2015 and their related objectives.

To achieve these objectives, the Action Plan proposed 40 individual actions (listed in Annex 4 to the present document) distributed under each of the policy priorities. According to the Action Plan, "the actions can also be categorised in three groups, depending on the actors involved and their competence defined in the Treaty:

1.Where Member States are leading and rely on their own resources, the Commission will help by supporting and coordinating activities. The measures proposed will focus on setting targets with the Member States and on how to achieve these targets by means of measures such as exchanging best practice and information, conducting studies and benchmarking.

2.Where the Commission and the Member States work jointly to develop, deploy or improve cross-border services, the Commission will take the lead in activities where joint resources are used, while the Member States will bear the final responsibility for implementing activities using their own resources. The measures proposed will include research and development, pilot projects, collaborative development of services by Member States and transfer of knowledge to the market.

3.Where the Commission can create enabling conditions, the measures proposed will include adopting legal instruments, setting standards, formulating common frameworks, implementing generic tools, providing (re-usable) technical building blocks and ensuring interoperability."

Of the 40 actions in the eGovernment Action Plan, the Commission was exclusively responsible for 23 and the Member States for 10. Seven actions were under shared responsibility between the Member States and the Commission. Five additional actions, two for the Commission and three for the Member States, were included for the governance.

2.2.Instruments

The instruments used to implement the Action Plan were political, financial and legislative activities.

2.2.1.Political Activities including exchange of experience and Commission Communications

The political activities refer to dialogue with Member States, Ministerial meeting, exhibitions, awards and benchmarking. Their aim was to provide political steer, enhance mutual learning through exchange of experience and awareness rising.

Dialogue with Member States

eGovernment High-Level Expert Group of Member States representatives (activity under governance): The Commission and Member States have engaged in collaborating and in regular dialogue during the eGovernment Action Plan 2011 – 2015 through the eGovernment High-Level Expert Group. In addition, there have been other fora to engage with Member States' representatives: including the ISA (Interoperability Solutions for European Public Administrations) Programme Management Committee.

Ministerial meetings

Ministerial Conferences (together with a series of eGovernment Presidency events/conferences) helped to take stock of where eGovernment was and provide a vision for where it should go in the coming years, mainly in the form of Ministerial Declarations. The 2009 Malmö Ministerial Declaration resulted in the current eGovernment Action Plan. The conferences also provided exhibitions of projects and awards for best eGovernment projects. The 6th Ministerial meeting was held on November 2011 in Poznan. As well as the conference and the exhibition, it featured an Informal Council of Ministers meeting.

Benchmarking

Since 2001, the EC has been measuring the progress made in on-line public services. This annual benchmarking has become a worldwide reference for international comparison of the development of eGovernment. Since 2012 a new benchmarking framework has been in place, aligned to the four policy priorities of the current Action Plan.

The European Interoperability Framework and the European Interoperability Strategy

The European Interoperability Strategy
  9
  (EIS) and a European Interoperability Framework
  10
  (EIF) provide guidance in delivering services across borders and sectors. The EIF, in particular, has been translated into National Interoperability Frameworks in more than 20 Member States. It is now being reviewed and extended in the context of the Digital Single Market Strategy.

Cloud computing

The Commission Communication "Unleashing the Potential of Cloud Computing in Europe"
  11
  , provided a European strategy designed to speed up and increase the use of cloud computing across all economic sectors.

ICT standardisation

A number of ICT standardisation activities supported the EU policy activities, including the areas of eGovernment, eProcurement, Public Sector Information, Open Data and Big Data.

2.2.2.Funding for eGovernment

The Action Plan did not have funding, but served as a guide to prioritise a number of investments from EU funding programmes:

The past Competitiveness and Innovation Framework Programme ICT Policy Support Programme
  12
  (CIP ICT-PSP) has supported (some projects on-going) piloting of cross-border eGovernment solutions and supported piloting of concepts, like the "cloud of public services"
  13
  ;

Under the Connecting Europe Facility (CEF), Digital Services Infrastructures (CEF DSI)
  14
  deploy digital public services that work across borders;

Horizon 2020's Societal Challenge 6 'Europe in a changing world – inclusive, innovative and reflective societies' funds research and innovation actions on ICT-enabled public sector innovation
  15
  ;

The past Seventh EU Framework Programme for Research and Technological Development (FP7) supported eGovernment-related projects;

The ISA programme
  16
  (Interoperability solutions for European Public Administrations) provides a framework that allows Member States to work together to create efficient and effective electronic cross-border public services for the benefit of citizens and businesses;

The European Structural and Investment Funds (ESIF as of 2014
  17
  ) provide investments in the field of eGovernment in less favoured regions, as this is seen as a strategic component of their economic and social development. One of the thematic priorities
  18
  for the 2014 – 2020 programming period is enhancing access, use and quality of ICT (thematic objective 2), including strengthening ICT applications for e-government. Another thematic objective is on administrative capacity building (thematic objective 11).

The exact amounts invested to policy priorities of the Action Plan are in some cases challenging to measure. Sometimes investments were done directly by the Member States without reporting to the Commission. Furthermore, it is not always obvious what would be considered an eGovernment–related investment in programmes with various information technology or information society projects.

2.2.3.Legislation

Legislative instruments have been used to achieve the eGovernment Action Plan's objectives where needed. Some of these legislative instruments existed already in the beginning of the Action Plan and needed amendment, while new legislation was also required, these included:

The planned revision of the PSI (Public Sector Information) Directive was indicated in the eGovernment Action Plan. On 12 December 2011 presented a proposal in this respect
  19
  . The revised Directive was adopted on 26 June 2013.
  20
  The reuse policy of the Commission was revised by a Decision of 2011
  21
  .

The revision of the eSignature Directive and the proposal for a Decision for mutual recognition of eIdentification and eAuthentication were indicated in the eGovernment Action Plan and led to the eIDAS Regulation on electronic identification and trust services (Regulation 910/2014).

The update of the Public Procurement Directives presented by the EC in the end of 2011 proposed a gradual but ambitious transition towards e-procurement in the EU. In February 2014, two Public Procurement Directives
  22
  were revised and a Directive on concession contracts was adopted
  23
  . The new rules will enter into force as of 18 April 2016.

The CEF regulation in the area of telecommunications infrastructure
  24
  (CEF Telecom Regulation).

The reform of the EU data protection legal framework – the General Data Protection Regulation and the Data Protection Directive for Police and Criminal Justice Authorities. Both will be adopted in early 2016 and followed by a two year transition period
  25
  .

The proposal for a directive on the accessibility of public sector bodies' websites, which has not yet been adopted
  26
  .

The new ISA2 programme adopted by the co-legislators in November 2015 on interoperability between public administrations and extended to local and regional administrations and the links to businesses and citizens is a key instrument for the development of eGovernment in Europe.
  27

2.3.Intervention logic

As stated in the Staff Working Document of 2010 accompanying the eGovernment Action Plan 2011 - 2015
  28
  , this eGovernment Action Plan 2011 – 2015 responded to a call from Member States, made in the Malmö Declaration, for a shared eGovernment policy in the European Union in order to build on past achievements and increase collaboration on eGovernment. This included joint action between Member States and close collaboration with the European Commission The simplified intervention logic figure (

  Figure 2

  ) explains how the different parts of the Action Plan fit together.

Figure 2 Simplified intervention logic.

2.4.Situation in 2010/Baseline

The 2010 Staff Working Document, referred to above, provides an overview of the situation in 2010
  29
  , focusing at the time on the availability of basic services online:

"The online delivery of basic services
  30
  has continued to increase steadily in recent years: their full online availability
  31
  went from 21% in 2001 to 71% in 2009
  32
  . However, there is a marked difference between services for businesses and services for citizens. Services for businesses reach 83% availability whereas services for citizens are still only at 63% availability."

There were considerable discrepancies between Member States. The gap between the best (with 100% availability of basic services) and worst performer (40% availability) was large. The EU average figure was around 70%. Four Member States had a full online availability for all the 20 basic services considered in 2009 in the benchmark measurement report.

2.4.1.Baseline of target: 50% of citizens (Digital Agenda) and 80% of businesses make use of eGovernment services

The Eurostat figure of the percentage of Europeans (EU27) aged 16 to 74 who had used the Internet in the last 3 months for interaction with public authorities was 31 % in 2010
  33
  . If extending to cover the last 12 months, the Eurostat figure collected in 2010 was 41 %
  34
  .

Regarding advanced modes of interacting with public administrations, the Staff Working Document of the 2011 – 2015 Action Plan noted that in 2009 only 17% of EU citizens had downloaded official forms from a public authority website and only 12% had used the Internet to send back completed forms.

In 2010 76% of enterprises used at least one eGovernment service
  35
  . Take-up was also relatively high for advanced ways of interacting with the public administrations: 54% of enterprises used the Internet to return completed forms and 43% interacted with the public administration entirely using electronic transactions.
  36

2.4.2.Baseline of target: a number of key cross-border services be offered online by 2015 (Digital Agenda)

The eGovernment Action Plan 2011 – 2015 Communication stated, describing the situation in 2010, that cross-border eGovernment services are few and, even where eGovernment services were offered, the majority of EU citizens were reluctant to use them. (In the Digital Agenda Communication in 2010 no specific baseline was given for the target of "by 2015 online availability of all the key cross-border public services contained in the list to be agreed by Member States by 2011".)

In 2010 the work towards cross-border services had started with three Large Scale Pilots (LSPs) co-funded by the EC under the "ICT Policy Support Programme"
  37
  as part of the Competitiveness and Innovation Framework Programme (CIP ICT PSP). PEPPOL
  38
  had started in 2008 to work towards a European system of eProcurement; STORK
  39
  had started in 2008 and aimed to enable citizens and businesses to identify themselves when interacting with their own and other national administrations over the internet; SPOCS
  40
  had started in 2009 and was related to the Services Directive. Around 36 million Euros
  41
  investment had been planned for in these three pilot projects at the time.

2.4.3.Baselines in the four policy priority areas

Data from 2010 is not available for the four policy priority areas of the Action Plan, because the benchmarking measurement of the four priorities started only with the new method as of 2012 with a first complete measurement available in 2013.

Some data relevant to the Action Plan 2011 – 2015 were given in the eGovernment Benchmarking report of 2010
  42
  that reported that Europe had made substantial advances in the 20 basic services. For businesses, a traditional 20 services assessment covered eight services for businesses. Out of these, the services ‘Registration of a company’ and ‘Obtaining an Environmental Permit’ were highly available online and displayed sophistication and full online availability scores of 90%/77% and 78%/63% for the EU27+ respectively. Furthermore, the eGovernment Benchmarking report 2011 measured the availability of different enablers in the EU27+ countries (

  Figure 3

  below)

Figure 3 Frequency of enablers in EU27+. Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark Measurement , 2010.

3.Evaluation Questions/Criteria

In order to structure the evaluation exercise, the current analysis sets out to answer a series of questions that will provide insight into the effects produced by the EU eGovernment Action Plan 2011-2015. The evaluation criteria used are from the EC’s Better Regulation Guidelines: effectiveness, efficiency, coherence, relevance and EU added value

Effectiveness analysis considers how successful the eGovernment Action Plan has been in achieving and progressing towards achieving the targets on the usage of eGovernment services and the availability of cross-border services, in addition to achieving the objectives of the four policy priorities on user empowerment, internal market, efficiency and effectiveness and pre-conditions. This criterion assesses the progress made and the role of the Action Plan in delivering the changes and/or the extent to which progress has fallen short of the target and what factors have influenced why something has not been achieved. Have users' needs been sufficiently catered for under the lifetime of the Action Plan?

The EU eGovernment Action Plan 2011 – 2015 did not have funding, but it served as a guide to prioritise investment from a number of funding programmes. How does the development and deployment of these projects support the overall aim of the eGovernment Action Plan? The efficiency analysis aims to, to the extent possible; consider the relationship between the resources used by the Action Plan and the changes generated by the funding intervention.

Relevance looks at the relationship between the needs and problems in society and the objectives of the EU eGovernment Action Plan: Was the vision in the Malmö declaration on which the eGovernment Action Plan is based appropriate for developing the EU eGovernment 2011 - 2015? Do the original objectives still correspond to existing needs within the EU, which of the objectives of the Action Plan continue to be still relevant and identifies also some new or changed needs.

The evaluation of coherence looks at how well different actions of the eGovernment Action Plan work together (internal coherence) and how the Action Plan relates to activities outside the Action plan (external coherence), including other EU policies. To what extent have Member States developed eGovernment services and policies in line with the overall principles set out by the current eGovernment Action Plan?

What is the EU added value resulting from the EU intervention compared with what could be achieved by Member States at national and/or regional levels? This assessment relates to the added value of the EU eGovernment Action Plan, why action at the EU level was necessary and what would have happened without an Action Plan? Which of the eGovernment policy priorities could not have been achieved without an EU-level Action Plan?

4.Method and sources

The evaluation has been coordinated by the EC's Directorate-General Communications Networks, Content and Technology with the support of an Inter-Service Group (with representatives of Commission Directorate-Generals Agriculture and Rural Development; Communication; Competition; Informatics; Economic and Financial Affairs; Employment; Social Affairs and Inclusion; Energy; Environment; Internal Market, Industry, Entrepreneurship and SMEs; Migration and Home Affairs, Joint Research Centre; Publications Office; Justice and Consumers; Mobility and Transport; Regional and Urban Policy; Research and Innovation; Health and Food Safety; Secretariat-General; Taxation; and Customs Union and Trade), which steered and monitored progress of the exercise, ensuring the necessary quality, impartiality and usefulness of the evaluation.

For the evaluation, a broad literature review was conducted covering various reports, EU studies and a number of international references. Important sources in the desk research were the mid-term evaluation of the eGovernment Action Plan 2011 - 2015
  43
  and the annual eGovernment Benchmarking reports. Furthermore, additional information and data (listed in the in the footnotes and in the "References and data sources" part in the end of this document) were used to answer the evaluation questions. Their methods are available in the original referenced reports. The eGovernment factsheets published on the Joinup platform
  44
  give an overview of eGovernment in each Member State.

A 12 week online public consultation was conducted to collect primary data and complemented by statistical data (Eurostat data and Digital Economy and Society Index DESI
  45
  ) and Digital Agenda Scoreboard reports
  46
  .

An effort was made to “triangulate” the data used throughout the analysis. This means that findings presented in the evaluation are supported by evidence from different data sources whenever additional data was available. Any contradicting evidence has been weighed according to its strength and quality before reaching conclusions.

4.1.Mid-term evaluation

The first monitoring of the implementation of the EU eGovernment Action Plan was done through the mid-term evaluation that carried out a monitoring and stocktaking exercise (progress assessment), with the purpose of providing an overview of the extent and modalities of implementation of the eGovernment Action Plan, its four priorities, 40 actions and governance chapter.

The Mid-term evaluation of the eGovernment Action Plan was carried out between December 2013 and June 2014 by external contractors. The eGovernment Action Plan states that "A midterm evaluation of implementation of this Action Plan will be conducted in 2013. "

4.2.Annual benchmarking reports

Since 2001, the Commission measures the progress made in the availability of on-line public services.. In 2012, this benchmarking was aligned
  47
  with the priorities of the eGovernment Action Plan 2011-2015. The benchmarking method examines a set of indicators related to each of the four political priorities of the Malmö Ministerial Declaration and the Action Plan. The latest benchmarking report
  48
  dates from June 2015. This 12th eGovernment Benchmark report is the third edition of the measurement made according to the new

  eGovernment Benchmark Framework 2012-2015

  . Unless otherwise specified, the benchmarking report references in this evaluation refer to the "EU28+", which refers to 33 European countries: the EU, Iceland, Norway, Serbia, Switzerland and Turkey.

With regard to the methodology used for the 2015 benchmarking report, each of the priority areas of the eGovernment Action Plan was assessed by one or more top level benchmarks. Two methods were used for data collection: Mystery Shopping and User Survey
  49
  .

Benchmarking report

| Benchmarking report | Year of data collection |
| --- | --- |
| eGovernment Benchmarking report 2012 (May 2013): Public Services Online. "Digital by Default or by Detour" | 2012 |
| eGovernment Benchmark Report 2014 -11th report (June 2014): Delivering on the European advantage? "How European governments can and should benefit from innovative public services" | 2013 |
| eGovernment Benchmarking Report 2015 (June 2015): Future-proofing eGovernment for a Digital Single Market. | 2014 |
| eGovernment Benchmarking Report 2016 (to be published) | 2015 |

Year of data collection

eGovernment Benchmarking report 2012 (May 2013): Public Services Online. "Digital by Default or by Detour"

2012

eGovernment Benchmark Report 2014 -11th report (June 2014): Delivering on the European advantage? "How European governments can and should benefit from innovative public services"

2013

eGovernment Benchmarking Report 2015 (June 2015): Future-proofing eGovernment for a Digital Single Market.

2014

eGovernment Benchmarking Report 2016 (to be published)

2015

4.3.The online public consultation

The online public consultation gathered input from citizens, businesses and civil servants over 12 weeks from 30 October 2015 till 22 January 2016. Its objective was to inform and help define the new eGovernment Action Plan 2016 - 2020. The consultation was promoted through the eGovernment network (comprising 5000 entities), the EC website and on Twitter.

The questionnaire also addressed the EU eGovernment Action Plan 2011 – 2015 as regards the improvement of cross-border eGovernment services and the success of the four political priories. The respondents were asked to provide answers in the form of agreeing to predefined statement on a scale from "not likely" to "very likely", in addition to a few open questions.

The results are provided in annex 2.

4.4.Limitations

The evaluation faced some limitations in the collection of data, whose impact was mitigated to a maximum possible extent:

Measuring the impact of the eGovernment Action Plan is a challenging exercise as it is almost impossible to isolate the impact of the Action Plan from other developments in the public sector, such as ICT-policies or the increased use of the Internet in general. Furthermore, some objectives of the Action Plan, such as increased collaboration, are non-tangible and, as a result, difficult to quantify. The evaluation has used benchmarking to measure the progress in the different policy priority areas of the Action Plan and assessed the effectiveness criteria qualitatively.

The Action Plan did not have a dedicated budget but relied on funding from other programmes for a large part of its actions. Other actions were financed by the Member States. As a result, the efficiency analysis (in particular the value for money assessment) was difficult to carry out.

Despite being prompted in a number of occasions by the EC, there is no systematic measuring or reporting requirement for the Member States of cross-border public services use. To overcome this, the evaluation used data from other sources, including case studies.

Given the multiplicity of the tools used to gather evidence, the results obtained are of different nature (for instance, eGovernment related data stems from the mid-term evaluation of the Action Plan and the benchmarking reports while the online public consultation provided insights in order to prepare for the new  eGovernment Action Plan 2016-2020).

The evaluation takes into account the inherent limitations of the findings of public consultations. Firstly, as in all surveys, the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who has a stake in this domain. Secondly, stakeholders' views convey an individual rather than a holistic perspective.

The mid-term evaluation did not systematically cover the five evaluation criteria. Furthermore, the methodology had strict requirements. For an action to qualify as "delayed" it was sufficient that one indicator is negative (e.g. one Member State answered ‘no’ to the question, or one action was delayed in one Member State) can sometimes give only a partial overview of progress and it should be understood as an evaluation of the actions as defined by the action plan, not an assessment of the policy priority as such. In addition, the mid-term evaluation did not capture the last 18 months of the Action Plan. This is partly mitigated using the 2015 benchmark study and partly the evaluation questions in the 2015/2016 online consultation on the eGovernment Action Plan and other recent reports and information.

Based on the elements above, the evaluation has been carried out on the basis of the best available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate and possibly counter-balanced with qualitative data and considerations.

5.Implementation of the EU eGovernment Action Plan 2011 – 2015 – state of play of the policy priorities

The monitoring of the implementation of the Action Plan was carried out by mid-term evaluation and supported by the annual eGovernment benchmarking. An assessment of the completion of actions will be done separately in a planned 2016 study on the assessment of the implementation of the Action Plan. The benchmarking reports are not intended to act as a benchmarking of the implementation of the Action Plan as such, but its results are informative for monitoring the progress of eGovernment in the EU.

5.1.General target: 50% of citizens (Digital Agenda) and 80% of businesses make use of eGovernment services

In 2014 eGovernment services were used by 47% of the EU population (38 % in 2009
  50
  )
  51
  . The Eurostat figure
  52
  for individuals (aged 16 – 74) having used the Internet for interaction with public authorities within the last 12 months was 46 % in 2015 (41 % in 2010).

Figure 4 Individuals in the EU that had used the Internet in the last 12 months for interaction with public authorities.

eGovernment usage by firms (EU 28) has increased steadily from 76% in 2010 to 84% in 2011
  53
  and 88 per cent in 2013
  54
  .

5.2.General target: a number of key cross-border services be offered online by 2015 (Digital Agenda)

The eGovernment Action Plan was to contribute to two key objectives of the Digital Agenda: by 2015 a number of key cross-border services will be available on line – enabling a) entrepreneurs to set up and run a business anywhere in Europe independently of their original location, and b) allowing citizens to study, work, reside and retire anywhere in the EU. The Digital Agenda had specified that the key cross-border public services would be contained in a list to be agreed by Member States by 2011.

The identification of the key cross-border services was discussed at the Ministerial Roundtable in Poznan (Poland) in 2011
  55
  . EU Ministers responsible for eGovernment discussed a number of areas of the Digital Single Market from which potential cross-border services could identify for future roll out. The conclusions
  56
  noted that:

The Ministers concluded that cross–border eServices should be built from components that can be shared and re–used. The Large Scale Pilots have already contributed to this goals by developing building blocks.

The Ministers concluded that at least five key cross–border services should be selected and implemented between 2012 – 2014 where possible domains have been put forward for consideration. These cross–border eServices should be built from components that can be shared and re–used.

The Member States' eGovernment Expert Group later agreed to endorse a number of focus areas and key enablers for digital cross-border public services. Because of divergent views, Member States did not reach formal consensus on the exact services. However, the work continued through the The Competitiveness and Innovation Programme ICT Policy Support Programme
  57
  (CIP ICT-PSP) LSPs (STORK, SPOCS, PEPPOL, epSOS
  58
  , eCodex
  59
  and eSENS
  60
  ) towards the development of the building blocks that can be used by Member States to produce their own cross-border services according their needs. Most of the Member States have participated with co-funding in one or several of these projects to develop cross-border solutions for the public services.

At the mid-term evaluation it was noted that, "while not all Member States have identified their priority key cross-border public service, and not all Member States have implemented or are planning to implement cross-border services, there are several examples of Member States implementing cross-border services."

To support the work on cross-border services a detailed study
  61
  was concluded by the Commission in 2013. The study identified the cross-border services with the highest potential impact. In determining these services Member States were recommended to refer to the PESTEL (Political, Economic, Social, Technological, Environmental and Legal factors) analysis and estimated volume of users. Furthermore, the study noted that quick wins for Member States can be achieved by re-using the building blocks from the CIP ICT-PSP large-scale pilots.

The CEF Telecom Regulation
  62
   acknowledges the role of the CIP LSPs such as "PEPPOL, STORK, epSOS, eCODEX or SPOCS, in validating key cross-border digital services in the internal market, based on common building blocks, which are being consolidated by the project eSENS". The regulation's annex specifically identifies a number of building blocks:

Electronic identification and authentication

Electronic delivery of documents

Automated translation

Critical digital infrastructures support

Electronic invoicing

The CEF building blocks
  63
  that are now available are: eID, eSignature, eDelivery, eInvoicing and Automated Translation. The deployment of eDelivery, eID and eSignatures at the end of 2015
  64
  is described in the table below (excluding countries outside the EU and EEA):

| eDelivery

  Countries that have deployed an eDelivery Access Point | eID

  Countries that are connected to the eID network via STORK | eSignatures |
| --- | --- | --- |
| Austria


  Belgium


  Czech Rep.


  Denmark


  Finland


  France


  Germany


  Greece


  Iceland


  Ireland


  Italy


  Netherlands


  Norway


  Spain


  Sweden


  UK | Austria


  Belgium


  Czech Rep.


  Estonia


  Iceland


  Italy


  Latvia


  Lithuania


  Luxemburg


  Portugal


  Slovakia


  Slovenia


  Spain


  Sweden | The trusted list enabling the use of eSignature as part of the trust backbone has been deployed in the 28 Member States (as required by the regulation). |

eDelivery

  Countries that have deployed an eDelivery Access Point

eID

  Countries that are connected to the eID network via STORK

eSignatures

Austria

Belgium

Czech Rep.

Denmark

Finland

France

Germany

Greece

Iceland

Ireland

Italy

Netherlands

Norway

Spain

Sweden

UK

Austria

Belgium

Czech Rep.

Estonia

Iceland

Italy

Latvia

Lithuania

Luxemburg

Portugal

Slovakia

Slovenia

Spain

Sweden

The trusted list enabling the use of eSignature as part of the trust backbone has been deployed in the 28 Member States (as required by the regulation).

Furthermore, the "Cross Border Mobility" benchmark of the eGovernment Benchmarking indicates to what extent European users can use online services in another country. This top level benchmark, which is split into Citizen Mobility and Business Mobility, measures the availability and usability of cross border services in a number of life events under four sub-criteria: availability, usability, ease of use and speed of use. According to the eGovernment Benchmarking Report 2015
  65
  , across Europe, on average, the score for the indicator of cross-border mobility for business is better than the citizen mobility indicator (58% as opposed to 43% for citizens). There is some improvement from the previous measurement (2012/2013, data not available earlier) from, respectively, 53% and 38%.

According to the eGovernment Benchmarking Report 2015, only 57% of the assessed services (life events) are available to cross border businesses and only 41% are available to cross border citizens
  66
  in 2014. This represents an improvement of 5 points for both groups in the assessed life events (compared to previous measurement 2012/2013, data not available earlier).

5.3.The four priorities and 40 actions of the EU eGovernment Action Plan

The 40 actions of the Action Plan (listed and numbered in Annex 4 to this document) were grouped under four priorities. The answers of each Member State (that answered to the survey) for each of the action in the mid-term evaluation of the eGovernment Action Plan are provided in a separate dashboard
  67
  . The last updates on the dashboard were in the first quarter of 2014. Non-implementation by one Member State was sufficient to mark an action either as delayed or at risk of delay. The mid-term evaluation was finished in June 2014.

The detailed implementation status of the 40 actions is provided in Annex 4 to this report. The status is provided on basis of the mid-term evaluation report findings. A number of actions were completed after the end of the evaluation and new progress in Commission actions is reported in Annex 4; detailed data from each Member States' progress after the mid-term evaluation was not available.

The mid-term evaluation concluded that of the 40 actions, 16 were completed at the time and six were on track. Three actions were at risk of delay and 15 delayed. The delay meant that an action had not been completed by one or several Member States or Commission within the deadline year and there was thus a delay from one to two years. The reasons and the possible impact (where an analysis was already possible) of these delays are assessed under the evaluation questions in part seven. The large majority of the EC’s 30 Actions were completed or were on track. Of these, there were 23 for which the EC has exclusive responsibility, while, for the remaining seven, responsibility was shared with Member States.

5.4.Governance

The High-Level Expert Group of Member State representatives was set up, and rules of procedures were adopted in 2011 (action 41).

According to the mid-term evaluation (2014) of the Action Plan, almost all Member States have a National eGovernment Strategy, which incorporates priorities from the Malmö Declaration and the eGovernment Action Plan (action 42). Only two Member States had not yet updated their national eGovernment strategies.

According to the mid-term evaluation, most of the Member States (17) have incorporated the political priorities of the Malmö Declaration and eGovernment Action Plan in their national strategies (action 43) and only four countries indicated in 2014 that this action was delayed in their case.

A mid-term evaluation of the Action Plan was carried out and its results and the results of this evaluation accompany the Communication on the EU eGovernment Action Plan 2016 – 2020 (action 44). Furthermore, Member States have informed the EC and the High-Level Expert Group on how the political priorities of the Malmö Declaration have been achieved, while six Member States (of the twenty-four responding in the mid-term evaluation) have not yet done so. Many countries have already provided some lessons learned within the implementation process (action 45).

6.Answers to the evaluation questions

6.1.Effectiveness

The effectiveness analysis assesses how successful the EU eGovernment Action Plan has been in achieving or progressing towards its objectives. What was the role of the EU eGovernment Action Plan 2011 – 2015 in delivering the changes? What factors have influenced the progress? To what extent progress fell short of the target?

The objectives to be analysed in this part are, first, the two targets of the Digital Agenda in Europe: (1) 50% of citizens make use of eGovernment services by 2015; and (2) that a number of key cross-border services be offered online by 2015. The Action Plan also aimed that "by 2015, 80% of enterprises will have used eGovernment". Secondly, the analysis includes the four policy priorities.

The mid-term evaluation of the Action Plan described a wide range of eGovernment effects across the different Member States with various case studies as examples, including examples in the area of user empowerment, open data, interoperability, eID and transparency.

However, the eGovernment Action Plan was a strategic plan to give the policies of EU Member States a direction towards commonly agreed goals. Without a detailed individual assessment of each action in each Member State it is not possible to associate the quantitative and qualitative effects on the Member States level directly to the Action Plan.

6.1.1.Objective on using eGovernment (take-up)

The 80% target for eGovernment usage by firms (EU 28) was achieved already in in 2011 (84%)
  68
  and it grew to 88 per cent in 2013
  69
  .

The action plan objective of "By 2015, 50% of EU citizens will have used eGovernment" was nearly achieved. In 2014 eGovernment services were used by 47% of the EU population
  70
  . The Eurostat figure
  71
  for individuals (aged 16 – 74) having used the Internet for interaction with the public authorities within the last 12 months was 46 per cent in 2015.

The achievement of the full 50 % usage by citizens was still expected in 2011. In the Digital Agenda Scoreboard report 2011
  72
  it was assessed that "the take-up of eGovernment in 2010 has reached 41 % of the EU population and, given the current trends, could be quite in accord with the target of the Digital Agenda of 50% of population using it by 2015." At the same time, the rise of the use of eGovernment services has also been driven by the increase of regular Internet users.

Finding 1

The Action Plan target of by 2015, 80% of enterprises will have used eGovernment was achieved early into the Action Plan. The target of by 2015, 50% of EU citizens will have used eGovernment was nearly achieved (46% in 2014).

6.1.2.Key cross-border services online

During the Action plan duration, cross-border eGovernment services were addressed in the high-level political discussions and in the work of the LSP projects. This was a direct result of the eGovernment Action Plan. The Action Plan played a key role as a basis and political framework for bringing Member States together to discuss and agree on these initiatives.

The Action Plan objective was to contribute towards fulfilling the Digital Agenda objective: "By 2015, a number of key cross-border services will be available on line – enabling entrepreneurs to set up and run a business anywhere in Europe independently of their original location, and allowing citizens to study, work, reside and retire anywhere in the European Union."

The contribution towards this objective was achieved by LSP projects implemented under the CIP ICT PSP programme. The evaluation
  73
  of the ICT PSP Programme underlined, among other aspects, its innovative role as its projects design new platforms for innovative cross-border services in areas of public interest, in particular in areas where there are continuing systemic and organisational risks.

Finding 2

The Large Scale Pilot projects had an innovative role as projects designing new platforms for cross-border services in areas of public interest. The activities on cross-border services went beyond the Action Plan's objectives.

Through the financial support from the CIP ICT-PSP Programme to the LSPs' work on what has become the CEF building blocks, the Action Plan has contributed to achieving also regulatory objectives: the eIDAS Regulation and transition to the electronic public procurement.

Through the financial support from the CIP ICT-PSP Programme to the LSPs' work on what has become the CEF building blocks, the Action Plan has contributed to achieving also regulatory objectives: the eIDAS Regulation and transition to the electronic public procurement.

The eIDAS Regulation
  74
  got important leverage from the STORK project that piloted an interoperability solution and that is referred to in the Regulation recitals.

The eIDAS Regulation

74

got important leverage from the STORK p

roject that piloted an interoperability solution and that is referred to in the Regulation recitals.

In the new public procurement directives, e-procurement was one of the important changes introduced. The PEPPOL LSP (Pan-European Public Procurement Online) sought to make it easier for companies to bid for public sector contracts throughout the EU and provided supporting experiences and solutions for cross-border e-procurement.

In the new public procurement directives, e-procurement was one of the important changes introduced. The PEPPOL LSP (Pan-European Public Procurement Online) sought to make it easier for companies to bid for public sector contracts throughout the EU and provided supporting experiences and solutions for cross-border e-procurement.

In the context of cross-border services, the Points of Single Contact, required by the Services Directive, cover an increasing number of administrative formalities for businesses. The SPOCS LSP (Simple Procedures Online for Cross-Border Services) sought to improve the cross-border electronic procedures for businesses. In the environmental area, the EC has developed an EU "Shared Environmental Information System" (SEIS).

In the context of cross-border services, the Points of Single Contact, required by the Services Directive, cover an increasing number of administrative formalities for businesses. The SPOCS LSP (Simple Procedures Online for Cross-Border Services) sought to improve the cross-border electronic procedures for businesses. In the environmental area, the EC has developed an EU "Shared Environmental Information System" (SEIS).

The aim of the Action Plan was to contribute towards the availability of cross-border services. To this date, life events for businesses and citizens are not yet covered by cross-border services and not all Member States have rolled-out the building blocks. For example, the objective that entrepreneurs should be able to set up and run business anywhere in Europe independently of their original location has not been met.

The aim of the Action Plan was to contribute towards the availability of cross-border services. To this date, life events for businesses and citizens are not yet covered by cross-border services and not all Member States have rolled-out the building blocks. For example, the objective that entrepreneurs should be able to set up and run business anywhere in Europe independently of their original location has not been met.

In the online public consultation (Annex 2) the majority of the respondents did not know whether the current eGovernment Action Plan has improved cross-border eGovernment services overall or did not agree that the Action Plan has improved them services. Around 25% of the respondents agreed that the Action Plan has improved cross border services overall.

In the online public consultation (Annex 2) the majority of the respondents did not know whether the current eGovernment Action Plan has improved cross-border eGovernment services overall or did not agree that the Action Plan has improved them services. Around 25% of the respondents agreed that the Action Plan has improved cross border services overall.

Figure 5 Online public consultation: Has the current eGovernment Action Plan improved cross-border eGovernment services overall?

Figure 5 Online public consultation: Has the current eGovernment Action Plan improved cross-border eGovernment services overall?

Finding 3

The eGovernment Action Plan has shown effectiveness to mobilise resources in particular in areas where the cross-border services objectives have been shared between both EU technical/operational and regulatory measures, e.g. cross-border eID and e-procurement. However, take-up has varied between Member States: entrepreneurs are not yet able to set-up and run a business anywhere in Europe independently of their location nor are citizens able to complete administrative procedures cross-border for all life events on-line. As reported under point 6.2, according to the eGovernment Benchmarking Report 2015, only 57% of the assessed services (life events) are available to cross border businesses and only 41% are available to cross border citizens.

6.1.3.Effectiveness of the four policy priorities

6.1.3.

Effectiveness of the four policy priorities

The progress in four of the main areas (User empowerment, Internal Market and pre-conditions, key enablers/pre condition) is visible in the below figure (

  Figure 6

  ) from the 2015 eGovernment Benchmark background report "Future-proofing eGovernment for the Digital Single Market".

The progress in four of the main areas (User empowerment, Internal Market and pre-conditions, key enablers/pre condition) is visible in the below figure (

Figure 6

) from the 2015 eGovernment Benchmark background report "Future-proofing eGovernment for the Digital Single Market".

Figure 6 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmark report 2015. Top-level benchmarks EU28+ 2012/2013 versus 2013/2014 (%). (The top level benchmarks Effective Government and User Centricity (user survey) were measured once,

Figure 6 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmark report 2015. Top-level benchmarks EU28+ 2012/2013 versus 2013/2014 (%). (The top level benchmarks Effective Government and User Centricity (user survey) were measured once,

6.1.3.1.Effectiveness of the User Empowerment policy priority

6.1.3.1.

Effectiveness of the User Empowerment policy priority

In the Action Plan, empowerment was referring to increasing the capacity of citizens, businesses and other organisations to be pro-active in society through the use of new technological tools. Governments should provide easy access to public information, improve transparency and allow effective involvement of citizens and businesses in the policy-making process.

In the Action Plan, empowerment was referring to increasing the capacity of citizens, businesses and other organisations to be pro-active in society through the use of new technological tools. Governments should provide easy access to public information, improve transparency and allow effective involvement of citizens and businesses in the policy-making process.

According to the mid-term evaluation
  75
  , the actions under the User Empowerment priority require further attention. Important targets have been achieved in implementing inclusive services, in involving citizens and business in the policy-making process, and notable progress has been made in creating collaborative services. There are also good achievements of the Action Plan targets under sub-priority on re-use of public sector information. At the same time, there are delays in general in drawing up and adopting common targets, indicators and measurement frameworks. Details of the achievements under each of the 14 actions under this priority are in Annex 4.

According to the mid-term evaluation

75

, the actions under the User Empowerment priority

require further attention. Important targets have been achieved in implementing inclusive services, in involving citizens and business in the policy-making process, and notable progress has been made in creating collaborative services. There are also good

achievements of the Action Plan targets under sub-priority on re-use of public sector information. At the same time, there are delays in general in drawing up and adopting common targets, indicators and measurement frameworks. Details

of the achievements u

nder each of the 14 actions under this priority are in Annex 4.

Services designed around users' needs and Inclusive Services

Services designed around users' needs and Inclusive Services

The Action Plan has supported exchanges of practice in the field of user-centred, inclusive and accessible eGovernment services (through the Joinup
  76
  platform), though no agreement had been reached with Member States on common targets and evaluation criteria for those services.

The Action Plan has supported exchanges of practice in the field of user-centred, inclusive and accessible eGovernment services

(through the Joinup

76

platform), though no agreement had been reached with Member States on common targets and evaluation criteria for those services.

The Commission adopted its proposal for a directive on the accessibility of public sector bodies' websites on 3 December 2012
  77
  . The MeAC (Measuring progress of eAccessibility in Europe) reports provided evidence and analysis to help understand and compare the approaches followed by the European countries, with a view to identifying issues and challenges, good practices and future priorities in the web accessibility field
  78
  .

The Commission adopted its proposal for a directive on the accessibility of public sector bodies' websit

es on 3 December 2012

77

.

The MeAC (Measuring progress of eAccessibility in Europe) reports provided evidence and analysis to help understand and compare the approaches followed by the European countries, with a view to identifying issues and challenges, goo

d practices and future priorities in the web accessibility field

78

.

According to the mid-term evaluation report, most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. The majority of the Member States that have already introduced personalised services offer them via multiple channels, such as mobile access via mobile apps and helplines.

According to the mid-term evaluation report, most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. The majority of the Member States that have already introduced personalised services offer them via multiple channels, such as mobile access via mobile apps and helplines.

Re-use of Public Sector Information (PSI)

Re-use of Public Sector Information (PSI)

Under the Action Plan priority on re-use of public sector information, among other things (details in Annex 4), the Commission has reviewed and subsequently revised the PSI Directive improving re-usability of data held by Member States' authorities and has put in place an Open Data Portal for data of the EU institutions
  79
  . The portal, launched in 2012, has currently more than 8000 datasets available.

Under the Action Plan

priority on re-use of public sector information, among other things (details in Annex 4), the Commission has reviewed and subsequently revised the PSI Directive improving re-usability of data held by Member States' authorities and has put in place an Ope

n Data Portal for data of the EU institutions

79

. The portal, launched in 2012, has currently more than 8000 datasets available.

The Action Plan achievements under the sub-priority Re-use of Public Sector Information (see Annex 4 and achievements under Action 6, 7 and 8, including the review of the PSI Directive and the set-up of the Open Data Portal) are good, apart from the formal achievement on agreement on indicators. The Action Plan has supported the achievements and has given them political visibility. There appears to be increased awareness of the importance, including economic importance, of open data as shown, for example, by the adoption of the G8 Open Data Charter and the commitment of various EU Member States to the Open Government Partnership.

The study "Open Data Maturity in Europe 2015
  80
  " reported that 27 countries (of EU28+) have a national Open Data portal and an open data policy is in place in 71% of the countries, often as part of a more generic Digital Strategy or eGovernment programme. It is also worth noting that beyond the objectives of the Action Plan, but supporting the sub-priority on re-use of public sector information, the EU endorsed the G8 Open Data Charter in 2013 and committed to promoting the application of the principles of the G8 Open Data Charter to all EU Member States within the context of a range of ongoing activities
  81
  .

Improvement of Transparency

On the sub-priority on Transparency the mid-term evaluation found that the actions under transparency were delayed. Fourteen Member States were using some transparency targets and most Member States provide online access to information on government laws and regulations (see Annex 4 for details). Member States also provide access to citizens' to their personal data, but not all provide it in electronic form as seen in the Action Plan (30% of Member States did at the time of the mid-term evaluation).

At the same time, the eGovernment benchmarking report 2015
  82
  stated that the transparency benchmark shows improvement from the previous measurement, but is still quite unsatisfactory, as it is at 51%. The Transparency indicator of the benchmarking report examines the extent to which governments are transparent about their own responsibilities and performance, the service delivery process, and the personal data involved
  83
  .

It is also positive that users have gained better access to personal data that is handled on the governments’ websites, but they still face considerable barriers when it comes to the clarity of the service delivery process. The Action Plan supported the delivery of a number of transparency objectives, but this remains an area for continued work.

Involvement of citizens and businesses in policy-making processes

In the sub-priority Involvement of citizens and businesses in policy-making processes progress is very good according to the mid-term evaluation (see Annex 4 for details). The Action Plan supported and gave a guiding framework for the work for governance and policy modelling under the Seventh EU Framework Programme for Research (FP7) and for eParticipation projects that were supported under the Competitiveness and Innovation Framework programme (CIP).

Finding 4

The Action Plan delivered mixed results regarding its contribution to the 4 policy objectives.

While it was effective in achieving and contributing to targets in inclusiveness, re-use of public sector information and in involving citizens and businesses in the policy making processes – particularly where both voluntary and regulatory actions at EU-level shared the same objective such as in re-use of public sector information -, it did not fully meet its objectives under the sub-priority on transparency where the benchmarking report findings indicate that the transparency of governments could be future improved.

Not reaching an agreement on common targets was reported by the mid-term evaluation to be an issue requiring further attention in relation to the collaborative production of services, PSI re-use indicators and targets and indicators on transparency.

Not reaching an agreement on common targets was reported by the mid-term evaluation to be an issue requiring further attention in relation to the collaborative production of services, PSI re-use indicators and targets and indicators on transparency.

When looking into reasons of why common targets or indicators were not agreed in the different areas, it should be noted that sometimes similar activities were already taking place elsewhere, sometimes making the agreement under the Action Plan unnecessary, or similar issues were discussed elsewhere:

When looking into reasons of why common targets or indicators were not agreed in the different areas, it should be noted that sometimes similar activities were already taking place elsewhere, sometimes making the agreement under the Action Plan unnecessary, or similar issues were discussed elsewhere:

The Web Accessibility Directive proposal (2012) aims to approximate the laws, regulations and administrative provisions of the Member States on the accessibility of websites of public sector bodies. A directive proposal goes beyond the (non-binding) Action Plan target of agreeing on common targets.



The Web Accessibility Directive proposal (2012) aims to approximate the laws, regulations and administrative provisions of the Member States on the accessibility of websites of public sector bodies. A directive proposal goes beyond the (non-binding) Action Plan target of agreeing on common targets.

For the collaborative production of services, workshops were organised to discuss recommendations but these did not lead to agreement on common targets for the roll-out.



For the collaborative production of services, workshops were organised to discuss recommendations but these did not lead to agreement on common targets for the roll-out.

For the PSI re-use indicators, the majority of the Member States agreed as to the usefulness of the indicators of the PSI Scoreboard
  84
  . However, given the existence of other benchmarking tools (Global Open Data Index, Open Data Barometer) and the fact that the PSI Directive does not mandate adopting specific indicators, it was difficult to ensure a uniform adoption of any given set of indicators for all EU Member States. Furthermore, the Europan Data Portal
  85
  , has developed its own approach and conducts a 'landscaping study' every year to measure open data maturity across Europe.



For the PSI re-use indicators,

the majority of the Member States agreed as to the us

efulness of the indicators of the PSI Scoreboard

84

. However, given the existence of other benchmarking tools (Global Open Data Index, Open Data Barometer) and the fact that the PSI Directive does not mandate adopting specific indicators, it was difficult to

ensure a uniform adoption of any given set of indicators for all EU Member States. Furthermore, the Europan Data Portal

85

, has developed its own approach and conducts a 'landscaping study' every year to measure open data maturity across Europe.

For transparency, the mid-term evaluation found that some Member States were using the Open Government Partnership indicators.



For transparency, the mid-term evaluation found that some Member States were using the Open Government Partnership indicators.

Finding 5

A number of common targets and indicators were not agreed by MS due to similar work and agreements made in other fora. The Action Plan could not be adjusted to take this into account.

Looking beyond the specific actions of the eGovernment Action Plan to the broader development of user empowerment in the EU with regards to eGovernment services, the eGovernment Benchmarking Report 2015 indicates that governments’ efforts to improve the quality of the online experience have focused on increasing the availability of their online services and on improving the mechanisms for online support and feedback. Users are also more empowered with medium increases in User Centricity and Transparency. Their performance is weaker at the user friendliness and time efficiency of the online services.

Looking beyond the specific actions of the eGovernment Action Plan to the broader development of user empowerment in the EU with regards to eGovernment services, the eGovernment Benchmarking Report 2015 indicates that governments’ efforts to improve the quality of the online experience have focused on increasing the availability of their online services and on improving the mechanisms for online support and feedback. Users are also more empowered with medium increases in User Centricity and Transparency. Their performance is weaker at the user friendliness and time efficiency of the online services.

The eGovernment Benchmarking report 2015 indicates that tests of 'user centricity' (to what extent a service is provided online and how it is perceived) show good results with a score of 73 out of a possible 100 points
  86
  across the EU28+. There is, however, a big difference between the compound indicators of user centricity, with much better performances for usability and online availability of services than for the ease and speed of using those services. While availability was up six points in 2014 (48% of all services are completely online and available through a portal
  87
  ), ease and speed of use did not change between 2012 and 2014. This indicates that many Member States are not focusing enough on the quality of the user’s experience
  88
  . This again suggests, according to the 2015 benchmark report, a trend of quantity over quality. The differences between citizens and businesses are not too large, suggesting that while there are significantly more information and services available to businesses, the quality of these does not differ much from citizen oriented eGovernment.

The online public consultation results reflect low awareness about the work carried out in the User Empowerment area. For User Empowerment, 27% of the respondents assessed that the measures had been successful; 48% said that these measure of the action were not successful and 24% did not know. It should be noted that the Action Plan activities were typically not directed directly towards citizens and businesses, but activities were carried out by the Member States and the Commission with a view to, for example, creating enabling conditions which are not directly visible to individual citizens as eGovernment Action Plan activities.

6.1.3.2.Effectiveness of the internal market policy priority

The internal market dimension of the Action Plan covers some of the most relevant accomplishments in the advancement of the cross-border delivery of services for citizens and businesses. The Action Plan supported the internal market objective through activities in "seamless services for businesses", "personal mobility" and in "EU-wide implementation of cross-border services". The detailed results are included in annex 4 of this report.

According to the mid-term evaluation of the Action Plan, most of the Member States have incorporated the political priorities of the Malmö Declaration, which is the basis of the Action Plan, in their national eGovernment strategies. As a consequence, aspects of internal market related eGovernment activities have been included in national strategies.

The role of the Action Plan in driving the cross-border services was discussed above in 7.1.2. The effectiveness of the work of the LSP projects was acknowledged also in the Commission report
  89
  on "Evaluations of the Competitiveness and Innovation Framework programme" where the evaluation panel recommended that the top-down policy-driven approach in the LSP projects (Pilot A) should be continued, as it is a working model of how to enhance more widespread uptake of new innovative services at EU level.
  90

Also in e-procurement the achievements went beyond the objectives of the Action Plan. In addition to the objective stated in the latter, the mandatory transition to e-procurement is driven by the new procurement directives and facilitated with a wide array of policies and support measures.

Finding 6

Due to the lack of flexibility of the Action Plan, it did not adapt to changing or new policy priorities and this reduced its effectiveness in terms of setting new and advanced priorities for the Member States and the European Commission.

Consecutive assessments of the Points of Single Contact in 2011/12, 2013 and 2014/15 show continuous but slow progress towards fulfilling the requirements of the Services Directive and becoming an effective tool for further integration of the Single Market. The most visible improvement has been made with regards to the availability of online procedures, although the gap is still significant. Furthermore, efforts to improve general usability of the web portals and accessibility to foreign users are recorded, but still insufficient to produce needed effects.
  91

Consecutive

assessments of the Points of Single Contact in 2011/12, 2013 and 2014/15 show continuous but slow progress towards fulfilling the requirements of the Services Directive and becoming an effective tool for further i

ntegration of the Single Market. The most visible improvement has been made with regards to the availability of online procedures, although the gap is still significant. Furthermore, efforts to improve general usability of the web portals and accessibility

to foreign users are recorded, but still insufficient to produce needed effects.

91

As for the effectiveness of cross-border eEnvironment services, the Mid-Term technical evaluation
  92
  on the implementation the INSPIRE Directive lists a number of effective eEnvironment cross-border services already at the mid-term implementation stage of INSPIRE. It emphasises that effective cross-border eEnvironment services will depend to a large extent on further ensuring the coherent and timely implementation of INSPIRE across the Member States in the 2016-2020 period.

As for the effectiveness of cross-border eEnvironment services, the Mid-Term technical evaluation

92

on the implementation the INSPIRE Directive lists a number of effective e

Environment cross-border services already at the mid-term implementation stage of INSPIRE. It emphasises that effective cross-border eEnvironment services will depend to a large extent on further ensuring the coherent and timely implementation of INSPIRE a

cross the Member States in the 2016-2020 period.

Going beyond the actions of the Action Plan, the eGovernment Benchmarking Report 2015 measured the cross-border mobility of businesses through studying the life event "Starting up a business". Scores on all four indicators (speed and ease of use, online availability and usability) are generally much higher than those for other life events. The score on Usability is excellent at 81% but the scores for Online Availability, Ease of Use and Speed of Use leave room for improvement.
  93

Going beyond the actions of the Action Plan, the eGovernment Benchmarking Report 2015 measured the

cross-border mobility of businesses

through studying the life event "

Starting up a business

". Scores on al

l four indicators (speed and ease of use, online availability and usability) are generally much higher than those for other life events. The score on Usability is excellent at 81% but the scores for Online Availability, Ease of Use and Speed of Use leave r

oom for improvement

.

93

Personal mobility

In the Action Plan sub-priority on personal mobility activities
  94
  have been undertaken to support exchange of good practices and to coordinate the Member States’ efforts to jointly develop and set-up cross-border and interoperable eDelivery services. The Action Plan set-out the framework and commitment for this voluntary work. According to the mid-term evaluation of the eGovernment Action Plan, more than 50% of the Member States already provide or are planning to provide those services. The CEF Telecom programme has financed the deployment of the cross-border eDelivery service.

According to the eGovernment Benchmarking Report 2015
  95
  , eGovernment services are still far from giving adequate support to citizens moving to another EU country. The cross-border mobility compound benchmark (beyond the actions of the Action Plan)_measures the ease and speed of use, online availability and usability of online public services for nationals of another EU country giving a score of maximum 100. This stands at 58% for businesses and at 43% for citizens
  96
  . Although there is improvement from previous year's benchmarking report (from, respectively, 53% and 38%), these figures are lower than the corresponding ones for domestic services. Cross-border transactional services (where a citizen can complete an entire process online) are rare. At the same time, significant increases of both the Business Mobility and Citizens Mobility score of the benchmarking report indicate that it is gradually becoming easier for people working across borders to access eGovernment services.

Finding 7

In the Action Plan's internal market priority, the cross-border delivery of public services for citizens and businesses in a number of sectors identified in the Action Plan was advanced. In e-procurement, the achievements go beyond the objectives of the Action Plan, since similar to findings 2 and 3, both EU voluntary and regulatory actions reinforced each other. However, availability of cross-border public services for citizens and businesses remain still insufficient for a fully functioning Internal Market.

6.1.3.3.Effectiveness of the "Effeciency and Effectiveness" policy priority

6.1.3.3.

Effectiveness of the "Effeciency and Effectiveness" policy priority

According to the eGovernment Action Plan mid-term review significant progress was being made in improving organisational processes with a "great deal of activity within Member States both at national and local level, focusing on better coordination and cooperation across different public organisations, and therefore improved governance"
  97
  . The Action Plan played an essential role in supporting the exchanges of experience and the sharing of new approaches through the Expert Group meetings, workshops and the Joinup (ePractice) portal. The programme for staff exchanges between administrations in different Member States was not carried out given the current budgetary cuts due to the economic situation.

Ac

cording to the eGovernment Action Plan mid-term review significant progress was being made in improving organisational processes with a "great deal of activity within Member States both at national and local level, focusing on better coordination and coope

ration across different public organisations, and therefore improved governance"

97

. The Action Plan played an essential role in supporting the exchanges of experience and the sharing of new approaches through the Expert Group meetings, workshops and the Joi

nup (ePractice) portal. The programme for staff exchanges between administrations in different Member States was not carried out given the current budgetary cuts due to the economic situation.

In the area of the reduction of administrative burden, the Action Plan included a number of actions related to the once-only principle. The sharing of experiences was done in a dedicated workshop and at the High-Level Expert Group meetings. A cost-benefit analysis study was carried out
  98
  and it gave important input to continuing the work on the once-only principle. The study provided also a roadmap for implementation of the once-only principle. The once-only policy gained further momentum through its inclusion in the Council Conclusions 2013
  99
  . This supported further steering of Commission policy and resources towards the once-only principle. The principle was included in the Digital Single Market strategy and a call for a new LSP project focusing on EU-wide once-only principle for businesses was included in the H2020 Work Programme 2016 – 2017
  100
  adopted in 2015. The effects of the Action Plan in went beyond the original objectives, and were influenced by the increased political visibility.

In the area of the reduction of administrative burden, the Acti

on Plan included a number of actions related to the once-only principle. The sharing of experiences was done in a dedicated workshop and at the High-Level Expert Group meetings. A cost-benefit analysis study was carried out

98

and it gave important input to

continuing the work on the once-only principle. The study provided also a roadmap for implementation of the once-only principle. The once-only policy gained further momentum through its inclusion in the Council Conclusions 2013

99

. This supported further s

teering of Commission policy and resources towards the once-only principle. The principle was included in the Digital Single Market strategy and a call for a new LSP project focusing on EU-wide once-only principle for businesses was included in the H2020 W

ork Programme 2016 – 2017

100

adopted in 2015. The effects of the Action Plan in went beyond the original objectives, and were influenced by the increased political visibility.

In the area of Green Government, the Action Plan did not succeed in achieving its objectives in assessing the possibilities of eGovernment in reducing the carbon footprint. The study was not carried out due to insufficient data (study on the potential of eGovernment to reducing carbon footprint of government). The mid-term evaluation also found that although there were interesting developments in the area of sustainability, most Member States did not have indicators or evaluation procedures in "Green Government". An agreement on indicators and evaluation procedures for measuring the reduction of carbon footprint as a result of eGovernment between Member States would not have been realistic.

In the area of Green Government, the Action Plan did not succeed in achieving its objectives in assessing the possibilities of eGovernment in reducing the carbon footprint. The study was not carried out due to insufficient data (study on the potential of eGovernment to reducing carbon footprint of government). The mid-term evaluation also found that although there were interesting developments in the area of sustainability, most Member States did not have indicators or evaluation procedures in "Green Government". An agreement on indicators and evaluation procedures for measuring the reduction of carbon footprint as a result of eGovernment between Member States would not have been realistic.

In 2015 the REFIT Fitness Check of the environmental reporting process
  101
  was launched contributing to the reduction of the carbon footprint of governments. It aims to “Launch a broad review of reporting requirements to see how burdens can be alleviated. This review will have a particularly strong focus on areas where stakeholders have recently indicated their concerns, such as agriculture, energy, environment and financial services." With regard to environment the technical Mid-term evaluation report
  102
  on the INSPIRE implementation emphasised the need to prioritise in the further implementation period of INSPIRE the connection between EU environmental reporting obligations and INSPIRE so as to reduce overall administrative burden and make reporting easier and more efficient for the Member States.

In 2015 the REFIT Fitness Check of the environmental reporting process

101

was launched contributing to the reduction of the carbon footprint of governm

ents. It aims to “Launch a broad review of reporting requirements to see how burdens can be alleviated. This review will have a particularly strong focus on areas where stakeholders have recently indicated their concerns, such as agriculture, energy, envir

onment and financial services." With regard to environment the technical Mid-term evaluation report

102

on the INSPIRE implementation emphasised the need to prioritise in the further implementation period of INSPIRE the connection between EU environmental rep

orting obligations and INSPIRE so as to reduce overall administrative burden and make reporting easier and more efficient for the Member States.

Finding 8

The Action Plan was effective in contributing to significant progress in improving organisational processes. The once-only principle policy gained political visibility from related Council Conclusions, which increased the overall effectiveness of the Action Plan priority. The reduction of carbon footprint continued to have a high visibility during the duration of the Action Plan, but in many Member States it was not linked specifically to eGovernment services and the Action Plan did not reach its objectives in these actions.

6.1.3.4.Effectiveness of the preconditions policy priority

6.1.3.4.

Effectiveness of the preconditions policy priority

The EU eGovernment Action Plan's policy objectives for the preconditions priority supported putting in place technical and legal pre-conditions for eGovernment. The mid-term evaluation noted that coordination of national efforts to support the development of interoperable solutions and key enablers is ongoing. The actions that have been supported by the Action Plan include:

The EU eGovernment Action Plan's policy objectives for the preconditions priority supported putting in place technical and legal pre-conditions for eGovernment. The mid-term evaluation noted that coordination of national efforts to support the development of interoperable solutions and key enablers is ongoing. The actions that have been supported by the Action Plan include:

European projects (such as PEPPOL and other LSPs) piloted shared cross-border solutions that many Member States are implementing;

The alignment of Member States with the European Interoperability Framework (EIF) (combined with a revision of the European Interoperability Strategy (EIS)); The European Interoperability Framework promotes and supports the delivery of European public services by fostering cross-border and cross-sectoral interoperability and has strongly contributed to the Action Plan objectives in the preconditions priority.

The adoption of a Regulation on Mutual recognition of Identification and Authentication; with the eIDAS Regulation and the STORK solution the eGovernment Action Plan proved very effective. To complete the legal framework, the Commission adopted several implementing acts under eIDAS Regulation.

Launch of pilot projects for innovative architecture and technologies in eGovernment; under sub-priority Innovative eGovernment, the mid-term evaluation assessed the progress as very satisfactory.

On-going work on the deployment of the CEF building blocks.

The Action Plan pushed Member States administrations towards the deployment of the pre-conditions, as shown, for example, in the EIF alignment figures (

  Figure 7

  ) below and in the eID roll-out figures that indicate that 13 Member States are connected to the eID network via STORK (details are in Annex 4 under action 37).

However, in the online public consultation, the majority of the respondents did not know the measures in the pre-conditions area or rated them not successful. At the same time, 46% of public administrations that responded to the survey rated the measures related to the preconditions for developing eGovernment successful.

As an example of effective work supported by the Action Plan, the Member States national interoperability frameworks were aligned to the EIF (

  Figure 7

  ). This has helped in achieving a common understanding of interoperability in the public sector and how to achieve it.

Figure 7 NIF-EIF alignment overview for each country assessed. State of play of interoperability in Europe – Report 2014.

Some of the above interoperability activities, like the implementation of the EIF, STORK or PEPPOL were supported also by the ISA programme which has also funded various interoperability activities. It has contributed substantially also to the implementation of a number of activities in the Maintenance and Implementation Work Programme of INSPIRE. INSPIRE is an important element in cross-sector interoperability between public administrations.

The eGovernment benchmarking report 2015 found that in the key enablers measured in the report (measuring also enablers beyond the Action Plan), there was only a marginal gain in 2013/2014 compared to 2012/2013
  103
  in the EU28+. Europe has been slow in terms of adopting key enabling technologies for supporting public online services. Some Member States score well with some of technologies but less for others showing that countries have different priorities in the adoption of these enablers. Most countries showed almost no progress from 2012-13 to 2013-14. The most common of these technologies (eIdentification) was deployed in 63% of the cases examined in the eGovernment benchmarking report in 2014.

Finding 9

The European Interoperability Framework (EIF) has strongly contributed to the Action Plan objectives in interoperability. In the Action Plan's key enablers work, a legal framework (eIDAS) was provided on electronic identification and trust services and the eID rollout of eIDAS compatible technical solutions is progressing, but mostly within certain Member States, and only in certain cases across borders.

6.1.3.5.Effectiveness of the "Governance" actions of the Action Plan

6.1.3.5.

Effectiveness of the "Governance" actions of the Action Plan

Important effects of the Action Plan arise from the governance chapter of the eGovernment Action Plan. These included the creation of an eGovernment High-Level Expert Group and the update of national eGovernment strategies incorporating the principles of the Malmö declaration. Actually, in most Member States, the national eGovernment strategies incorporate the Malmö priorities (see above 6.4). The effect has been that in most Member States the national eGovernment strategies thus include Action Plan's political objectives. The mid-term evaluation gives some examples that show how the national eGovernment work has been influenced by the Action Plan
  104
  . The Expert Group has been a forum for exchange of information and discussion on eGovernment EU-wide.

Important effects of the Action Plan arise from the governance chapter of the eGovernment Action Plan. These included the creation of an eGovernment High-Level Expert Group and the update of national eGovernment

strategies incorporating the principles of the Malmö declaration. Actually, in most Member States, the national eGovernment strategies incorporate the Malmö priorities (see above 6.4). The effect has been that in most Member States the national eGovernment

strategies thus include Action Plan's political objectives. The mid-term evaluation gives some examples that show how the national eGovernment work has been influenced by the Action Plan

104

. The Expert Group has been a forum for exchange of information and

discussion on eGovernment EU-wide.

Finding 10.

An EU eGovernment Action Plan (based on the Malmö Declaration) has been a "mobiliser" instrument for national eGovernment policies. This gave further strength and visibility to the objectives of the Action Plan. The Action Plan's governance established an EU-wide forum for exchange of information and discussion on eGovernment.

6.2.Efficiency

6.2.

Efficiency

6.2.1.Efficiency of the related budget

6.2.1.

Efficiency of the related budget

The eGovernment Action Plan had no dedicated budget, therefore efficiency related to budget may only be linked to actions under the Action Plan for which spending has been allocated under an EU funding scheme. For the same reason, no systematic monitoring of resources has been carried out for each of the priorities of the eGovernment Action Plan. Likewise, across the 28 Member States the implementation of the Action Plan has been carried out in a variety of different manners on the national level.

The eGovernment Action Plan had no dedicated budget, therefore efficiency related to budget may only be linked to actions under the Action Plan for which spending has been allocated under an EU funding scheme. For the same reason, no systematic monitoring of resources has been carried out for each of the priorities of the eGovernment Action Plan. Likewise, across the 28 Member States the implementation of the Action Plan has been carried out in a variety of different manners on the national level.

eGovernment is supported through a number of funding programmes (see also under 3.2.2.):

eGovernment is supported through a number of funding programmes (see also under 3.2.2.):

The past Competitiveness and Innovation Programme ICT Policy Support Programme
  105
  (CIP ICT-PSP);



The

past Competitiveness and Innovation Programme ICT Policy Support Programme

105

(

CIP ICT-PSP

);

CEF, Digital Services Infrastrctures (CEF DSI)
  106
  ;



CEF

, Digital Services Infrastrctures (CEF DSI)

106

;

Horizon 2020's Societal Challenge 6
  107
  ;



Horizon 2020

's Societal Challenge 6

107

;

The past Seventh EU Framework Programme for Research (FP7);



The past Seventh EU Framework Programme for Research (

FP7

);

The ISA programme
  108
  ;



The

ISA

programme

108

;

The European Structural and Investment Funds ESIF (ERDF, CF, ESF, YEI and EAFRD
  109
  ); before 2014: Structural Funds.



The European Structural and Investment Funds

ESIF (ERDF, CF, ESF, YEI and EAFRD

109

); before 2014: Structural Funds.

CIP LSP projects

CIP LSP projects

The CIP ICT Policy Support Programme funded the following LSPs projects (with the approximate total costs, including the 50 percent EU co-funding): STORK (eIdentification – 26 + 18,6 M EUR), eCodex (e-justice – 24 M EUR), PEPPOL (e-procurement – 30,8 M EUR), SPOCS (Points of Single Contact – 24 M EUR), ePSOS (e-health – 36,5 M EUR) and eSENS (consolidation of results – 27 M EUR). In the analysis below, the eHealth related activities and pilot are excluded as they are covered by a separate eHealth Action Plan
  110
  .

The CIP ICT Policy Support Programme funded the following LSPs projects (with the approximate total costs, incl

uding the 50 percent EU co-funding): STORK (eIdentification – 26 + 18,6 M EUR), eCodex (e-justice – 24 M EUR), PEPPOL (e-procurement – 30,8 M EUR), SPOCS (Points of Single Contact – 24 M EUR), ePSOS (e-health – 36,5 M EUR) and eSENS (consolidation of resul

ts – 27 M EUR). In the analysis below, the eHealth related activities and pilot are excluded as they are covered by a separate eHealth Action Plan

110

.

The CIP Implementation reports
  111
  provide overviews on the LSPs. The total cost of the eGovernment LSPs (excluding ePSOS) is around 150 million EUR. The Commission has co-funded around 50 per cent of this amount the other half was invested by the participating Member States (and countries associated to the CIP programme). The Commission report
  112
  on "Evaluations of the Competitiveness and Innovation Framework programme" referred to the final evaluation
  113
  of the CIP programme: "The evaluation underlined the improvements arising from the delegation of the management of substantial parts of CIP to the EACI
  114
   concerning the efficiency of the programme management, both in relation with costs to the Commission and efficiency of the services provided, as measured in terms of number of contracts signed, the period to contract and payment delays."

The CIP Implementation reports

111

provide overviews on the LSPs. The total cost of the eGovernment LSPs (exc

luding ePSOS) is around 150 million EUR. The Commission has co-funded around 50 per cent of this amount the other half was invested by the participating Member States (and countries associated to the CIP programme). The Commission report

112

on "Evaluations

of the Competitiveness and Innovation Framework programme" referred to the final evaluation

113

of the CIP programme: "

The

evaluation underlined the improvements arising from the delegation of the management of substantial parts of CIP to the

EACI

114

concerning

the efficiency of the programme management, both in relation with costs to the Commission and efficiency of the services provided, as measured in terms of number of contracts signed, the period to contract and payment delays

."

The results achieved through these pilots in the form of re-usable technical building blocks that are now starting to be deployed in the EU have been described above in 7.1.2. The work contributed towards having key cross-border services online. The participation to these pilots was (and is) voluntary of which reason the costs varied among the Member States. The pilots typically involved between 10 – 20 Member States.

The results achieved through these pilots in the form of re-usable technical building blocks that are now starting to be deployed in the EU have been described above in 7.1.2. The work contributed towards having key cross-border services online. The participation to these pilots was (and is) voluntary of which reason the costs varied among the Member States. The pilots typically involved between 10 – 20 Member States.

The 2013 study "The feasibility and scenarios for the long-term sustainability of the LSPs" included a cost-benefit analysis of the large-scale pilots. Cost-benefits analysis calculations were underpinned by an evidence based estimation of the potential use levels for the nine core service platforms (CSPs) and building blocks (BBs) - eID, eProcurement, eBusiness, eHealth, eJustice, eSignature, eDocument, eDelivery. The role of the public sector in financing the building blocks and CSP/BBs may diminish where a business for private sector involvement is more apparent.

The 2013 study "The feasibility and scenarios for the long-term sustainability of the LSPs" included a cost-benefit analysis of the large-scale pilots.

Cost-benefits analysis calculations were underpinned by an evidence based estimation of the potential use levels for the nine core service platforms (CSPs) and building blocks (BBs) - eID, eProcurement, eBusiness, eHealth, eJustice, eSignature, eDocument, eDelivery.

The role of the public sector in financing the building blocks and CSP/BBs may diminish where a business for private sector involvement is more apparent.

ESIF

ESIF funding (as from 2014) for eGovernment can be assessed by extracting the amount of potential investments (this does not necessarily reflect final investment figures) in the intervention category "eGovernment services and applications" (including e-procurement, ICT measures supporting the reform of public administration, cybersecurity, trust and privacy measures, e-justice and e-democracy)
  115
  . For 2014 Operational Programmes, the amount is 3,427 million EUR potential investments.

Research and Innovation funding

The Action Plan has also led to financing initiatives under the Research and Innovation programmes of the Commission. In particular, research and innovation actions were targeting the understanding the transformation of public administration in engaging more stakeholders in the policy making process, the reduction of administrative burden and the service delivery of public services. Under the FP7 the programme invested over 75M EUR between 2011 and 2013 in co-financed research and innovation actions. Since the launch of the new programme, H2020, over 27M EUR were invested in co-financed research and innovation actions.

Finding 11

The Action Plan had no dedicated budget. Despite this, the Action Plan worked by providing a guiding framework for funding. The Action Plan mobilised resources in the Commission and Member States (and countries associated to the CIP programme), that co-funded 50 per cent of the LSPs to develop solutions and common understanding in several cross-border policies.

6.2.2.Costs and benefits to stakeholders

6.2.2.

Costs and benefits to stakeholders

The efficiency can also be assessed as the capacity of the Action Plan to mobilise resources (human and financial) versus the potential savings in the intervention domain. Investments in the modernisation of the European public sector have to be seen also in the wider context.

The efficiency can also be assessed as the capacity of the Action Plan to mobilise resources (human and financial) versus the potential savings in the intervention domain. Investments in the modernisation of the European public sector have to be seen also in the wider context.

In the EU, public expenditure accounts for almost 50% of GDP and the public sector represents about 17% of total employment
  116
  .



In the EU, public expenditure accounts for almost 50% of GDP and the public sector represents about 17% of total employment

116

.

Public administrations have a powerful means to pull innovation; in the EU, in 2010, public authorities in the EU spent over € 2 400 billion on supplies, works, and services – amounting to around 19% of EU GDP
  117
  .



Public administrations have a powerful means to pull innovation; in the EU, in 2010,

public authorities in the EU spent over € 2 400 billion on supplies, works, and services – amounting to around 19% of EU GDP

117

.

The potential of these investments for savings can also be seen in different sectors mentioned in the Action Plan, for example:

The potential of these investments for savings can also be seen in different sectors mentioned in the Action Plan, for example:

eIdentification

  It has been assessed that replacing the current paper based identification elements used across the private sector could reduce costs by 90% if eID replaced them
  118
  . In addition, eID is also the most important service to access other digital public services, a key enabler.

eIdentification

It has been assessed that

replacing the current paper based identification elements used across the private sector could reduce costs by 90% if eID replaced them

118

. In addition, eID is also the most important service to access other digital

public services, a key enabler.

e-procurement

  Contracting authorities and entities that have successfully completed the transition to e-procurement commonly report savings between 5 and 20%; whereby experience also shows that investment costs related to transition to e-procurement can be rapidly recouped. Given the size of the total procurement market in the EU, each 5% saved could return around €100 billion to the public purse
  119
  . Based on the extrapolation of the national evaluations of benefits, the EC estimates that the adoption of e-invoicing in public procurement across the EU could generate savings of up to €2.3 bn
  120
  . Switching from paper to fully automated e-invoicing can cut the costs of receiving an invoice from €30-50 to €1
  121
  .

e-procurement

Contracting authorities and entities that have successfully completed the transition to e-procurement commonly report savings between 5 and 20%; whereby experience also shows that investment costs related to

transition to e-procurement can be rapidly recouped. Given the size of the total procurement market in the EU, each 5% saved could return around €100 billion to the public purse

119

. Based on the extrapolation of the national evaluations of benefits, the EC e

stimates that the adoption of e-invoicing in public procurement across the EU could generate savings of up to €2.3 bn

120

. Switching from paper to fully automated e-invoicing can cut the costs of receiving an invoice from €30-50 to €1

121

.

The once-only principle

The once-only principle

The extension of this principle, in compliance with data protection legislation, would likely generate an annual net saving at the EU level of around EUR 5 billion per year by 2017
  122
  .

The extension of this principle, in compliance with data protection legislation, would likely generate an annual net saving at the EU level of around EUR 5 billion per year by 2017

122

.

As reported in the mid-term evaluation, for the Member States public administrations, achieving this type of financial gains and improving the efficiency of administrative processes were among the main drivers for eGovernment projects, even more so given the current economic climate and budget pressures. Financial savings are expected to translate into efficiency gains in a variety of ways, including a reduction of administrative burden and savings in opportunity costs.

The mid-term evaluation found in the eGovernment case studies that financial impacts are not always clear before implementing a project. In general, the mid-term evaluation found that the decision to implement a project depended in only a few cases on a clear business case.

At the same time, the mid-term evaluation found that it was uncommon to have users of the service paying a fee for it. In fact, financial benefits for the public sector are expected to come mostly from savings generated by reducing the production costs for a service and/or by the rationalisation of public expenditure. This meant that for the other stakeholders, citizens and businesses, the Action Plan activities did not incur direct costs and they were able to benefit from, for example, improved cross-border availability of eGovernment services.

Finding 12

Different sectors report significant potential savings from the use of eGovernment. Given the important size of public expenditure in these and other sectors, the resources that the Action Plan mobilised towards the modernisation and increasing efficiency of the public administrations seem justified.

6.3.Relevance

6.3.

Relevance

The public sector represents a significant and unique part of Europe’s economy, has a key responsibility for delivering citizens' quality of life and is also a vital actor in efficiently and effectively delivering policy objectives.

The public sector represents a significant and unique part of Europe’s economy, has a key responsibility for delivering citizens' quality of life and is also a vital actor in efficiently and effectively delivering policy objectives.

Economic and budgetary pressures force governments to be ever more efficient, effective, reduce costs and be more competitive in a multi-polar world
  123
  . The, contribution of eGovernment to jobs and growth remains a very relevant high-level objective. Repeated analysis during the European Semester has shown that the modernisation of public administrations can contribute to tackling economic challenges. The recent Annual Growth Surveys
  124
  recognise the importance of a modern and efficient public administration for a business-friendly environment and to ensure fast and high-quality services for citizens.

Economic and budgetary pressures force governments to be ever more efficient, effective, reduce costs and be more competitive in a multi-polar world

123

. The, contribution of eGovernment t

o jobs and growth remains a very relevant high-level objective. Repeated analysis during the European Semester has shown that the modernisation of public administrations can contribute to tackling economic challenges.

The recent Annual Growth Surveys

124

recognise the importance of a modern and efficient public administration for a business-friendly environment and to ensure fast and high-quality services for citizens.

Additionally, in many EU countries the work to implement digital end-to-end services and to achieve cross-border interoperability continues, so further pursuing the eGovernment activities remains relevant. Online public services continue in relevance for reducing business costs and increasing the efficiency and the quality of the services provided to citizens and businesses
  125
  . According to a recent study
  126
  , if interactions with public authorities can be made as transparent, as fast and as cost-efficient as in the private sector, then the potential benefits will materialise.

Additionally, in many EU countries the work to implement digital end-to-end services

and to achieve cross-border interoperability continues, so further pursuing the eGovernment activities remains relevant. Online public services continue in relevance for reducing business costs and increasing the efficiency and the quality of the services

provided to citizens and businesses

125

. According to a recent study

126

, if interactions with public authorities can be made as transparent, as fast and as cost-efficient as in the private sector, then the potential benefits will materialise.

6.3.1.Relevance: User Empowerment

6.3.1.

Relevance: User Empowerment

As noted earlier, according to the 2015 eGovernment Benchmarking report, user centricity is confirmed as the most advanced indicator. However, many Member States are not focusing enough on the quality of the users' experience.

As noted earlier, according to the 2015 eGovernment Benchmarking report, user centricity is confirmed as the most advanced indicator. However, many Member States are not focusing enough on the quality of the users' experience.

According to the mid-term evaluation (on Action 2), most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. The relevance of one-stop-shops continues with new needs for cross-border approaches..

According to the mid-term evaluation (on Action 2), most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. The relevance of one-stop-shops continues with new needs for cross-border approaches

.

.

The Action Plan objectives on re-use of public sector information have been found to be relevant. The Action Plan's objectives on user needs, inclusive services, collaborative services, transparency and the involvement of citizens and businesses in policy-making were assessed to be appropriate for user empowerment. This is also due to advancing technology that brings, for example, increased interactions through new tools and social networking.

The Action Plan objectives on re-use of public sector information have been found to be relevant. The Action Plan's objectives on user needs, inclusive services, collaborative services, transparency and the involvement of citizens and businesses in policy-making were assessed to be appropriate for user empowerment. This is also due to advancing technology that brings, for example, increased interactions through new tools and social networking.

Transparency and participation continue as important principles in the modernisation of public administrations. According to the eGovernment benchmarking study users are provided with information about the duration of the process in only 39% of measured cases. The figure for maximum delivery timelines for government is 46%. At the same time, administrations rarely give an account of their service performance (in 38% of cases). Only 1 in 3 websites inform visitors about their ability to participate in policy-making processes
  127
  .

Transparency and participation c

ontinue as important principles in the modernisation of public administrations. According to the eGovernment benchmarking study users are provided with information about the duration of the process in only 39% of measured cases. The figure for maximum deli

very timelines for government is 46%. At the same time, administrations rarely give an account of their service performance (in 38% of cases). Only 1 in 3 websites inform visitors about their ability to participate in policy-making processes

127

.

Users of Member States' justice systems are an example of user empowerment through new technological tools Here, there are currently gaps regarding the deployment of e-justice or more specifically the use of ICT at courts, for example, concerning the possibility to communicate electronically between the courts and the parties as well as the use of ICT in court management system as identified by the EU Justice Scoreboard
  128
  .

Users of M

ember States' justice systems are an example of user empowerment through new technological tools Here, there are currently gaps

regarding the deployment of e-justice or more specifically the use of ICT at courts, for example, concerning the possibility to

communicate electronically between the courts and the parties as well as the use of ICT in court management system as identified by the EU Justice Scoreboard

128

.

The online public consultation also asked how stakeholders should be enabled to contribute to, make proposals on and publicly exchange their views on new initiatives emerging under the eGovernment Action Plan. Most respondents said that using an online social media platform was a good idea. Using a social media platform was viewed as useful to reach out to a large audience, citizens as well as businesses.

The online public consultation

also asked how stakeholders should be enabled to contribute to, make proposals on and publicly exchange their views on new initiatives emerging under the eGovernment Action Plan. Most respondents said that using an online social media platform was a good idea. Using a social media platform was viewed as useful to reach out to a large audience, citizens as well as businesses.

Finding 13

The eGovernment Action Plan priorities on user empowerment have been found to be relevant. However the Action Plan could benefit from stronger user empowerment through a mechanism to enable stakeholders to contribute to the initiatives of the Action Plan.

The Action Plan objective on the electronic service to support citizens' initiatives was achieved. However, the general goal of involvement of citizens and the rights of citizens continues to be relevant. According to the Citizenship Report 2013, only one in three citizens (36%) say they are well informed about their EU rights and just under a quarter (24%) feel fairly or very well informed about what they can do when their EU rights are not respected
  129
  .

The Action Plan objective on the electronic service to support citizens' initiatives was achieved. However, the general goal of involvement of citizens and

the rights of citizens

conti

nues to be relevant. A

ccording to the Citizenship Report 2013, only one in three citizens (36%) say they are well informed about their EU rights and just under a quarter (24%) feel fairly or very well informed about what they can do when their EU rights ar

e not respected

129

.

Building on the achievements of the Action Plan on the sub-priority on the re-use of public sector information further work was relevant and continues to be so. The mid-term evaluation finding of the lack of agreement on the common set of Public Sector Information (PSI) re-use indicators can also feed into future work, including in possible future adaptations of the PSI Directive
  130
  . The policy evaluations of the PSI Directive
  131
  and the Infrastructure for Spatial Information in the EU (INSPIRE) Directive
  132
  demonstrated the need for public authorities to continue improving the sharing and re-use of their data.

Building on the achievements of the Action Plan on the sub-priority on the re-use of public sector information further work was relevant and continues to be so. The mid-term evaluation finding of the lack of agreement on the common set o

f Public Sector Information (PSI) re-use indicators can also feed into future work, including in possible future adaptations of the PSI Directive

130

. The policy evaluations of the PSI Directive

131

and the Infrastructure for Spatial Information in the EU (INSPI

RE) Directive

132

demonstrated the need for public authorities to continue improving the sharing and re-use of their data.

Despite challenges remaining in implementing the INSPIRE Directive, both the Member States and the consulted stakeholders reported qualitative benefits ranging from more efficient information services to citizens and the improved collaboration between public authorities, as well as building up skills, to cost savings in various areas and improved data quality. Estimates are available for a number of countries
  133
  ,
  134
  . Other illustrations are more effective environmental risk management
  135
  , improved cross-border and cross-sector collaboration, spatial planning and lower costs for environmental impact assessments
  136
  . Moreover, synergies
  137
  were reported between the EU and national strategies on Open Data
  138
  and eGovernment where the availability of INSPIRE spatial data and services is used increasingly for a wide range of government-to-government and government-to-public applications
  139
  .

Despite challenges remaining in implementing the INSPIRE Directive, both the Member States and the consulted stakeholders reported qua

litative benefits ranging from more efficient information services to citizens and the improved collaboration between public authorities, as well as building up skills, to cost savings in various areas and improved data quality. Estimates are available for

a number of countries

133

,

134

. Other illustrations are more effective environmental risk management

135

, improved cross-border and cross-sector collaboration, spatial planning and lower costs for environmental impact assessments

136

. Moreover, synergies

137

were reported between the EU and national strategies on Open Data

138

and eGovernment where the availability of INSPIRE spatial data and services is used increasingly for a wide range of government-to-government and government-to-public applications

139

.

The Commission has put in place an Open Data Portal where re-usable data from the EU institutions is are referenced
  140
  and has currently more than 8000 datasets available. As a next step to opening up its own data, the Commission is actively promoting the discoverability and re-use of PSI in a cross-border and cross-lingual context, notably by the creation of the European Data Portal
  141
  , a digital service infrastructure under CEF. The importance of open data and re-use of data continues relevant for contributing to transparency and for economic growth. According to a recent study "Creating Value through Open Data"
  142
  the market size for Open Data is expected to increase by 36,9 % between 2016 – 2020 to a value of 75,7 bn EUR in 2020. The study also forecasts and a 32 % growth in open data jobs in the 5-year period and important cost savings.

The

Commission has put in place an Open Data Portal where re-usable data from the EU institutions is are referenced

140

and has currently more than 8000 datasets available. As a next step to opening up its own data, the Commission is actively promoting the discov

erability and re-use of PSI in a cross-border and cross-lingual context, notably by the creation of the European Data Portal

141

, a digital service infrastructure under CEF. The importance of open data and re-use of data continues relevant for contributing to

transparency and for economic growth. According to a recent study "Creating Value through Open Data"

142

the market size for Open Data is expected to increase by 36,9 % between 2016 – 2020 to a value of 75,7 bn EUR in 2020. The study also forecasts and a 32

% growth in open data jobs in the 5-year period and important cost savings.

The relevance of the Action Plan sub-priority on transparency, including data protection has continued to increase. In addition to the consent-based approaches, users demand to be in control of their data and provide only the amount of data necessary for a certain transaction. The future General Data Protection Regulation on which co-legislators reached a political agreement
  143
  in December 2015 updates and modernises the Data Protection rules currently contained in the 1995 Data Protection Directive and will have an effect also on the public sector.

The relevance of the Action Plan sub-priority on transparency, including data protection has continued to increase. In addition to the consent-based approaches, users demand to b

e in control of their data and provide only the amount of data necessary for a certain transaction. The future General Data Protection Regulation on which co-legislators reached a political agreement

143

in December 2015

updates and modernises the Data Protec

tion rules currently contained in the 1995 Data Protection Directive and will have an effect also on the public sector.

Finding 14

There are continued needs in the Action Plan priorities on citizens' rights, re-use of public sector information, transparency, participation and data protection.

6.3.2.Relevance: Internal market

6.3.2.

Relevance: Internal market

The Action Plan objectives of seamless services for businesses, personal mobility and EU-wide implementation of cross-border services still correspond to the needs of the single market. The Digital Single Market of over 500 million people cannot be completed without public services. In 2014, there were 17.9 million persons who had been born in a different EU Member State from the one where they were resident
  144
  . From the perspective of a citizenship, the figure is still high, in 2014 more than 14 million EU citizens lived in another EU Member State than where they had their citizenship
  145
  . The trends of intra-EU mobility are steadily increasing
  146
  . According to a 2013 study
  147
  , over 1,5 million citizens and 314,000 businesses are likely to use cross-border eGovernment services each year in 2020. The same study estimates that the potential volume of the top five public services for businesses is 2,490,000 cross-border users annually.

The Action Plan objectives of seamless services for businesses, personal mobility and EU-wide implementation of cross-border services still correspond to the needs of the single market.

The Digital Single Market of over 500 million people cannot be completed without public services. In 2014, there were 17.9 million persons who had been born in a different EU Member State from the one where they were resident

144

. From the perspective of a ci

tizenship, the figure is still high, in 2014 more than 14 million EU citizens lived in another EU Member State than where they had their citizenship

145

. The trends of intra-EU mobility are steadily increasing

146

. According to a 2013 study

147

, over 1,5 million ci

tizens and 314,000 businesses are likely to use cross-border eGovernment services each year in 2020. The same study estimates that

the potential volume of the top five public services for businesses is 2,490,000 cross-border users annually.

The mid-term evaluation of the Action Plan noted that under half of Member States had rolled out LSP projects, which highlights the need for continued action with the CEF DSIs.

The mid-term evaluation of the Action Plan noted that under half of Member States had rolled out LSP projects, which highlights the need for continued action with the CEF DSIs.

There are also increasing needs for cross-border company procedures and the related objectives are still highly relevant, for example in the areas of e-procurement, company law, e-justice, Points of Single Contact and information portals:

There are also increasing needs for cross-border company procedures and the related objectives are still highly relevant, for example in the areas of e-procurement, company law, e-justice, Points of Single Contact and information portals:

E-procurement

E-procurement

As outlined above, transition to the e-procurement presents a potential of significant savings enabled by streamlining, simplification and opening up of public procurement procedures. Only adoption of e-invoicing in public procurement across the EU could generate savings of up to €2.3 billion
  148
  . The work in this area delivered beyond the objectives of the Action Plan. In 2014, the European Commission adopted new public procurement directives, which enter into force in April 2016
  149
  and make e-procurement gradually compulsory, with e-Submission being mandatory for all public buyers as of October 2018. The policy aiming at supporting transition to full e-procurement is also building on a 2012 Communication on "A strategy for e-procurement" and a 2013 Communication on "End-to-end procurement to modernise public administration"
  150
  . A wide range of support measures have been put in place to facilitate the transition to e-procurement in all Member States. This will allow the businesses, in particular, SMEs, to benefit as early as possible from new business opportunities opening up at the European Single Market thanks to the electronic public procurement, in particular simplification of the pre-qualification phase.

Company law

The objective of the 'internal market' priority of the action plan aimed at facilitating the running of companies anywhere in the EU. The Digital Single Market Staff Working Document
  151
  noted that existing rules under company law do not sufficiently integrate the benefits of digital technologies. Companies are still faced with paper-based formalities. For example, online registration of companies is only possible in 16 Member States. Though a number of measures have been taken at national level, a national approach does not remove the obstacles that companies face if they consider setting up or operating a company across borders
  152
  . In this context, the Single Market Strategy
  153
  further mentioned that the Commission will consider further ways of achieving simpler and less burdensome rules for companies — while continuing to act against letterbox companies — including making digital solutions available throughout a company’s lifecycle, in particular in relation to their registration and to the filing of company documents and information.

E-justice

The action plan foresaw in the 'EU wide implementation of cross border services' activities in the eJustice area as there are continuing needs in cross-border e-justice. The Digital Single Market strategy (Staff Working Document) noted that European e-Justice
  154
  facilitates access to justice and cross-border judicial proceedings and makes it easier for citizens to find a lawyer/notary in the EU and for businesses to search for insolvent entities through interconnected insolvency registers. However, in a cross-border context issues identified at national level are amplified, as cross-border cooperation in this area entails that the Member States are at equal levels of development.

In this context, the work started by the CIP pilot e-CODEX remains highly relevant. Although the pilot project has created a strong backbone for communications between justice administrations, which can be used for many different purposes, extension of this work will be needed in three dimensions. Firstly, the number of Member States which currently participate in this work remains limited, and should be expanded to cover all Member States. Secondly, the e–CODEX backbone provides many opportunities to further support cross-border exchanges, including between judicial professionals such as laywers, notaries and judicial officers. Thirdly, expansion can also take place in terms of the number of judicial cooperation instruments covered. A first pilot project implementing this technology on the European e-Justice Portal will go live in 2016. Further work on this topic could include efforts to realise the cross-border elecronic service of documents. Furthermore, as of 2017, the business registers interconnection system, which will be composed of Member States' business registers, a European central platform, and the European e-Justice portal serving as the European electronic access point will facilitate access to information on companies for citizens and businesses throughout the EU
  155
  . In addition, the interconnection of national land registers will be developed to go live on the European e-Justice Portal in the course of 2017.

Points of Single Contact

Extending the scope of the Points of Single Contact as indicated in the Action Plan could further contribute to simplification, savings for public administration and more coherent approach in providing information and e-services to businesses. Information gaps increase costs for business, in particular for SMEs
  156
  . Procedural streamlining via Points of Single Contact (PSC) could generate up to 0.15% of GDP in the medium run (5-year horizon) and up to 0.21% of GDP in the long run
  157
  .

In line with Action Plan action 18 to transit into "Second Generation" Points of Single Contact (PSC), that would function as fully-fledged eGovernment centres, a first assessment of the PSCs was carried out in 2011-2012
  158
  . The results revealed a hybrid landscape, with some PSCs more advanced than others in providing the information and services required. This first assessment resulted in the Communication on the implementation of the Services Directive "A partnership for new growth in services 2012-2015”
  159
  . This Communication encouraged Member States to develop by the end of 2014 Second Generation Points of Single Contacts, which should “1) cover all procedures during the business life cycle, 2) be multilingual, and 3) be more user-friendly”.

These recommendations were subsequently included into the PSC Charter
  160
  , endorsed by the Council in 2013. The Charter encourages the Member States to bring additional features into their PSCs so that the latter respond more effective to the business life cycle of a company in the services sector. It also includes the benchmarks facilitating the assessment of the PSCs with a view to ensure their continuous improvement and ultimately, to provide better public online services to business. The assessment of the PSCs performance against of the criteria set out in the Charter was carried out in 2014 and published in 2015 in a study
  161
  . The results of the study indicate that PSCs are still far from delivering what is expected from them. The overall score of the PSCs on the EU28+ level indicates moderate performance with considerable room for improvement. Although PSCs provide basic information on general requirements, information on sector specific requirements is insufficient. The study reveals also a considerable gap in online availability of the procedures associated with general requirements and the ones associated with specific requirements. Although the general requirements are fully transactional on about half of the PSCs. The access to PSC services by foreign users were assessed as poor. The transactionality of online procedures for foreigners is one of the weakest points of the PSC.

Portals and networks

Furthermore, there are a number of existing portals and networks for finding information and initiate and complete transactions with Member States’ administrations across the EU (including Points of Single contact, Product contact points, Construction product contact points, Your Europe, Your Europe Advice, SOLVIT and Enterprise Europe Network). However, currently Single Market-related information, problem-solving mechanisms, contact points and procedures do not operate as a whole but are dispersed and not sufficiently inter-connected – both at EU and national level. It is therefore difficult for users to find the right information and assistance required. This implies that higher transaction costs have to be incurred before engaging in a cross-border activity. More synergies between actions, such as providing information, assistance and e-Government services, can be achieved at EU, national and regional level if efforts are better coordinated. An area for further work is to link up relevant EU and national content, services and procedures and offer users a streamlined, comprehensive portal to find information and initiate as well as complete transactions with Member States’ administrations across the EU (Single Digital Gateway as included in the Digital Single Market strategy).

The online public consultation (Annex 2) reflected a need for increasing awareness about the on-going work and availability of cross-border public services. The majority of the respondents did not know whether the current eGovernment Action Plan has improved cross-border eGovernment services overall or did not agree that the Action Plan has improved them.

Finding 15

The intra-EU mobility is increasing and citizens and businesses would benefit from connected cross-border public services in the internal market. The Action Plan had objectives for seamless services for businesses, personal mobility and EU-wide cross-border services. The relevance of these eGovernment services for the internal market continues to be high. At the same time, there is not awareness about the cross-border eGovernment services.

6.3.3.Relevance: "Efficiency and Effectiveness of Governments and Administrations"

6.3.3.

Relevance: "Efficiency and Effectiveness of Governments and Administrations"

Under the sub-priority on reduction of administrative burden, the eGovernment Action Plan initiated the work towards the implementation of the once-only principle.

Under the sub-priority on reduction of administrative burden, the eGovernment Action Plan initiated the work towards the implementation of the once-only principle.

This objective is ever more relevant as a policy priority. The European Council Conclusions of October 2013 stated that "EU legislation should be designed to facilitate digital interaction between citizens and businesses and the public authorities. Efforts should be made to apply the principle that information is collected from citizens only once, in due respect of data protection rules."

This objective is ever more relevant as a policy priority. The European Council Conclusions of October 2013 stated that "EU legislation should be designed to facilitate digital interaction between citizens and businesses and the public authorities. Efforts should be made to apply the principle that information is collected from citizens only once, in due respect of data protection rules."

Citizens and businesses are still all too often required to submit information that governments already hold. Only in 48% of cases do public administrations reuse information about the citizen that is already in their possession without asking for it again
  162
  . According to the eGovernment benchmarking report 2015
  163
  , the use of Authentic Sources has actually decreased. The use of authentic sources means that public administrations get data from sources such as National register, Tax registers, Company registers etc., instead of asking citizens and businesses to provide the data (again). The decrease in their use is disappointing as it also indicates that European governments still struggle to re-use data. As a result, citizens and businesses are still all too often required to submit information that governments already hold.

Citizens and businesses are still all too often required to submit information that governments already hold.

O

nly in 48% of cases do public administrations reuse information about the citizen that is

already in their possession without asking for it again

162

. According to the eGovernment benchmarking report 2015

163

, the use of Authentic Sources has actually decreased. The use of authentic sources means that public administrations get data from sources such

as National register, Tax registers, Company registers etc., instead of asking citizens and businesses to provide the data (again). The decrease in their use is disappointing as it also indicates that European governments still struggle to re-use data. As

a result, citizens and businesses are still all too often required to submit information that governments already hold.

The amount of data that is pre-filled in Public Services' online forms (one way to implement the once-only principle) varies to a great deal within the EU
  164
  :

Figure 8 Pre-filled forms as an indicator of once-only. Amount of data pre-filled in public services forms in seven life events. (Digital Economy and Society Index 2016.)

The percentage of individuals who used the Internet to submit completed forms to public authorities was on average 33% of Internet users within the EU
  165
  in 2014. In 2015 this was slightly lower at 32%
  166
  :

Figure 9 Percentage of Internet users submitting completed forms to public authorities. (Digital Economy and Society Index. 2016.)

The eGovernment benchmarking report measures the availability of "authentic sources", which are base registries used by government to automatically validate or fetch data relating to citizens and businesses. It implies governments re-use data to deliver and facilitates the implementation of the once-only principle. The figure of 2013/2014 in the eGovernment benchmarking
  167
  assessment for EU28+ was 45%, meaning that in 45% of the assessed cases do public administrations reuse information that is already in their possession without asking for it again.

In consideration of these findings and the potential savings, implementing the once-only principle across borders continues to be a highly relevant priority in Europe and could further contribute towards the efficiency of the European public sector.

The relevance of the principle was acknowledged also by the European Parliament in its report Towards a Digital Single Market act in January 2016
  168
  : "a step-by-step sectoral approach to apply the ‘once only principle’ in public administrations according to which citizens and businesses should not be asked for information already provided to a public authority, whilst ensuring citizens' privacy and a high level of data protection - -;"

In addition to the once-only principle, the sub-priority of the Action Plan on improving organisational processes remains relevant. The eCommission Action Plan has led also the EC towards, amongst other things, electronic procurement. The digitalisation of procedures (including digital by default) is also a means for administrative burden reduction and remains relevant. Toolboxes and innovative architectures are also important measures where the EC to promote and present pilots and practices on the quality, efficiency, transparency of public services. The Joinup collaborative eGovernment platform has grown to be a platform for thousands of users.

The mid-term evaluation also recommends that although the action under sub-priority Green Government was not carried out it should be reconsidered: "Looking at eGovernment as an enabler to lower the carbon footprint of administration makes sense in looking at eGovernment in a more holistic way." Furthermore, the mid-term evaluation recommends that the staff exchange programme between administrations in different Member States, though not carried out, could be re-launched and it could be an excellent learning and sharing accelerator for Member States' eGovernment agencies.

Finding 16

The assessment of the use of authentic sources by the public sector shows the continued relevance of the once-only principle. The principle is also receiving important political support. Improving organisational processes remains a relevant goal, also for the EC. The Action Plan's Green Government sub-priority and the staff exchange programme could be re-considered.

6.3.4.Relevance: Pre-conditions

6.3.4.

Relevance: Pre-conditions

This Action Plan objectives on technical pre-conditions is still very pertinent. However, with many of the technical and legal pre-conditions achieved during the eGovernment Action Plan, the benefits to citizens, businesses and public administrations themselves need to be demonstrated. The take-up of these solutions could be increased, making a real impact on people's lives - reducing administrative burden, facilitating mobility in the EU and improving the user experience when dealing with public administrations.

This Action Plan objectives on technical pre-conditions is still very pertinent. However, with many of the technical and legal pre-conditions achieved during the eGovernment Action Plan, the benefits to citizens, businesses and public administrations themselves need to be demonstrated. The take-up of these solutions could be increased, making a real impact on people's lives - reducing administrative burden, facilitating mobility in the EU and improving the user experience when dealing with public administrations.

The eIDAS Regulation on electronic identification and trust services for electronic transactions in the internal market was an objective of the Action Plan. It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where eIDs are available. In addition, it helps to create a European internal market for eTS (Trust Services).  The take up of these services demonstrates the important potential for the future and the continued high relevance. In the end of 2015, 15 countries, including 13 Member States, were connected to the eID network via STORK (see further details in Annex 4 under action 37). In addition, the potential of a trusted identification mechanism goes also beyond the public sector, such as for the financial sector.

The eIDAS Regulation on electronic identification and trust services for electronic transactions in the internal market was an objective of the Action Plan. It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where eIDs are available. In addition, it helps to create a European internal market for eTS (Trust Services).

The take up of these services demonstrates the important potential for the future and the continued high relevance. In the end of 2015, 15 countries, including 13 Member States, were connected to the eID network via STORK (see further details in Annex 4 under action 37)

.

In addition, the potential of a trusted identification mechanism goes also beyond the public sector, such as for the financial sector.

The eGovernment Benchmarking Report 2015 suggests that building blocks, such as electronic identity (eID), that enable online authentication of persons and companies, or authentic sources that encompass registers of personal and other data, and that would allow re-use of that data for other service processes, will boost cross-border and national online services.

The eGovernment Benchmarking Report 2015 suggests that building blocks, such as electronic identity (eID), that enable online authentication of persons and companies, or authentic sources that encompass registers of personal and other data, and that would allow re-use of that data for other service processes, will boost cross-border and national online services.

A lot was achieved under the interoperability objectives of the Action Plan. However, working towards interoperability between Member States at European level continues to be a relevant objective. Interoperability is needed to enable public services to work across borders. 1,500,000 citizens and 300,000 businesses are likely to use cross-border online services each year by 2020
  169
  which will presuppose the existence of interoperable solutions for which reason the creation of interoperable solutions will continue to be of high relevance.

As stated in the DSM staff working document, the lack of interoperability among public entities and private operators restricts the potential for digital end-to-end services, One Stop Shops, the once-only principle, the single data entry principle, the transparency of public services and the full exploitation of public open data. Further progress is needed to improve the interoperability of systems for cross-border delivery of goods and services, as well as for the mobility of people and businesses and for cooperation between public authorities, at national and EU level.

In the area of public services, the European Interoperability Framework, adopted by the EC in 2010
  170
  , promotes and supports the delivery of European public services by fostering cross-border and cross-sectoral interoperability. The majority of the Member States have transposed this Framework, which has brought a common understanding of the basic requirements for interoperability between public services. This common understanding is still relevant and requires updating and extending with other concrete and practical instruments to possibly be shared by national administrations such as the European Interoperability Reference Architecture (EIRA) and the European Interoperability Cartography (EUCart)
  171
  . The revision of the EIF is included in the list of actions of the Digital Single Market Strategy.

The ISA2 programme, recently adopted by the co-legislators, is a key instrument for the development of interoperability of public administrations in Europe and between public administrations and businesses and citizens.

Standardisation is another aspect that remains relevant in the future. Digitisation of administrative formalities offers opportunities to, for example, standardise the documents that businesses have to present to national authorities in different Member States, yielding additional cost savings
  172
  .

Finding 17

While many of the technical and legal pre-conditions were achieved during the eGovernment Action Plan, there is a continued need and relevance to pursue the re-use of common solutions for rationalisation, savings and interoperability and to increase the take-up both in cross-border and national eGovernment services.

6.4.Coherence

6.4.

Coherence

The EU eGovernment Action Plan was based on a collaborative design by the EC and Member States, starting from a common vision (Malmö Declaration), containing agreed policy objectives, a monitoring mechanism and holistic approach with the relevant EC services working together. This wide coordinated approach helped to ensure coherence right from the beginning. This coherence was further strengthened by the inclusion of priorities of the Malmö Declaration and Action Plan in national eGovernment strategies.

The EU eGovernment Action Plan was based on a collaborative design by the EC and Member States, starting from a common vision (Malmö Declaration), containing agreed policy objectives, a monitoring mechanism and holistic approach with the relevant EC services working together. This wide coordinated approach helped to ensure coherence right from the beginning. This coherence was further strengthened by the inclusion of priorities of the Malmö Declaration and Action Plan in national eGovernment strategies.

The various parts and actions of the Action Plan included several different policy areas of the EU. Coherence at the time of the Action Plan preparation was focusing on identifying impactful areas where the introduction of ICT and related interoperability measures across the EU could yield the greatest impact, for example on eID and eProcurement. The coherence of the numerous different areas together was supported through the holistic governance structure of the Action Plan with the involvement of various Commission services.

The various parts and actions of the Action Plan included several different policy areas of the EU. Coherence at the time of the Action Plan preparation was focusing on identifying impactful areas where the introduction of ICT and related interoperability measures across the EU could yield the greatest impact, for example on eID and eProcurement. The coherence of the numerous different areas together was supported through the holistic governance structure of the Action Plan with the involvement of various Commission services.

The governance through the different groups with Member States, including the eGovernment High Level Expert Group, CEF, H2020, FP7, CIP and ISA Committees contributed to the avoidance of overlaps. The governance activities provided for regular meetings and exchanges of experience between Member States and the EC.

The governance through the different groups with Member States, including the eGovernment High Level Expert Group, CEF, H2020, FP7, CIP and ISA Committees contributed to the avoidance of overlaps. The governance activities provided for regular meetings and exchanges of experience between Member States and the EC.

Finding 18

The wide coordinated approach of the preparation of the Action Plan contributed to coherence right from the beginning. The coherence benefited further from a political agreement between the Member States (the Malmö Declaration). Coherence was supported also by the governance structure of the Action Plan.

In order to achieve a snowball effect on investments the Action Plan successfully made coherent use of different funding instruments related to the eGovernment Action Plan. The work carried out on key enablers under the various programmes aimed also at the subsequent deployment of mature services under the CEF Telecom programme. The ISA programme played also a key role in ensuring the maturity and implementation of enablers piloted and/or developed under the H2020 and the CIP programme. The uptake could then be initiated with funding from the CEF Telecom programme. The figure below demonstrates the moving from CIP pilots towards deployment in CEF.

In order to achieve a snowball effect on investments the Action Plan successfully made coherent use of different funding instruments related to the eGovernment Action Plan. The work carried out on key enablers under the various programmes aimed also at the subsequent deployment of mature services under the CEF Telecom programme. The ISA programme played also a key role in ensuring the maturity and implementation of enablers piloted and/or developed under the H2020 and the CIP programme. The uptake could then be initiated with funding from the CEF Telecom programme. The figure below demonstrates the moving from CIP pilots towards deployment in CEF.

Figure 10 Coherence through CEF. Moving from piloting to roll-out; from large-scale pilots to the CEF.

Figure 10 Coherence through CEF. Moving from piloting to roll-out; from large-scale pilots to the CEF.

For the ESIF funding, an ex-ante conditionality on Digital growth
  173
  was established to ensure coherence of the investments with other EU or national funding source. This conditionality aims to foster the development and implementation of national and regional digital growth measures and to assess their consistency with the Digital Agenda for Europe's goals that include e-government. In practice this meant checking against the eGovernment Action Plan priorities to ensure complementarity.

For the ESIF funding,

an ex-ante conditionality on Digital growth

173

was established to ensure coherence of the investments with other EU or national funding source. This conditionality aims to foster the development and implementation of national and regional digital growth meas

ures and to assess their consistency with the Digital Agenda for Europe's goals that include e-government. In practice this meant checking against the eGovernment Action Plan priorities to ensure complementarity.

Finding 19

The different funding mechanisms for eGovernment key enablers aimed also at the subsequent deployment of mature services under the CEF Telecom programme. This coordination of funding mechanisms contributed to coherent funding. For the ESIF funding on eGovernment, coherence with the Action Plan was aimed to by assessing the consistency of the planned measures with the Digital Agenda goals, including eGovernment.

Coherence analysis can also assess the eGovernment related activities of the 28 Member States. In the eGovernment Action Plan, the Member States were responsible for 10 actions while seven actions were under the shared responsibility of the Member States and the EC.

In the Commission, several different policy areas were involved in the eGovernment Action Plan. The different services took part also in the eGovernment Action Plan governance activities, including the Member States groups. This increased the coherence between the different priorities of the Action Plan. Public sector modernisation was discussed in various fora and considering different aspects during the Action Plan duration.

However, as the eGovernment Action Plan was based on voluntary activities of the Member States, full coherence could not be assured, also because of the different implementation approaches used in the Member States.

The aim was not necessarily to achieve full coherence as various other initiatives had to be considered. On the international level, the EC followed the policies of the UN and the OECD and the policies of third countries that sometimes were also taking aboard the Commission approaches. The Member States also followed and participated in various international eGovernment activities.

Finding 20

A wide variety of initiatives at various levels need to be considered in the assessment of coherence of the Action Plan internally and externally. For this reason an assessment of the Action Plan's coherence was not fully achievable.

6.5.EU-added value

6.5.

EU-added value

The EU eGovernment Action Plan 2011-2015 as well as its predecessor have been political instruments for the Commission to advance the agenda of Public Sector Modernisation across the EU. They have been supporting European coordination, collaboration and joint actions on eGovernment, helping to use public resources more efficiently and reducing public expenditures by coordinating and pooling public and private resources. The Action Plan also helped inside the Commission to provide a political context and objectives for many different policy areas and helped to focus investments from different programmes.

The EU eGovernment Action Plan 2011-2015 as well as its predecessor have been political instruments for the Commission to advance the agenda of Public Sector Modernisation across the EU. They have been supporting European coordination, collaboration and joint actions on eGovernment, helping to use public resources more efficiently and reducing public expenditures by coordinating and pooling public and private resources. The Action Plan also helped inside the Commission to provide a political context and objectives for many different policy areas and helped to focus investments from different programmes.

The Action Plan has also been an example for establishing national eGovernment strategies in the Member States. The Action Plan has helped to coordinate investments, share and re-use infrastructures, processes, data, resources, content and tools to avoid duplication and waste and to speed up the roll-out of eGovernment. These types of initiatives continue to be required at EU level.

The Action Plan has also been an example for establishing national eGovernment strategies in the Member States. The Action Plan has helped to coordinate investments, share and re-use infrastructures, processes, data, resources, content and tools to avoid duplication and waste and to speed up the roll-out of eGovernment. These types of initiatives continue to be required at EU level.

Some of the actions of the Action Plan stem from mandatory requirements set down in EU law, which went through the subsidiarity and proportionality test in the legislative process. Here the Action Plan helped to support the collaboration between the Member States. On the other hand, some actions were relevant to or contributed towards new legislation, examples of this are e-procurement (public procurement legislation
  174
  ) and eID-activities leading to the eIDAS Regulation.

Some of the actions of the A

ction Plan stem from mandatory requirements set down in EU law, which went through the subsidiarity and proportionality test in the legislative process. Here the Action Plan helped to support the collaboration between the Member States. On the other hand,

some actions were relevant to or contributed towards new legislation, examples of this are e-procurement (public procurement legislation

174

) and eID-activities leading to the eIDAS Regulation.

The cross-border elements of EU legislation relating to public services (e.g. eProcurement, eIDAS, Public Sector Information, Services Directive) continue to justify, and require collaborative activities at EU level, for example, to support the implementation of legislation through the common development and piloting of technical solutions.

The cross-border elements of EU legislation relating to public services (e.g. eProcurement, eIDAS, Public Sector Information, Services Directive) continue to justify, and require collaborative activities at EU level, for example, to support the implementation of legislation through the common development and piloting of technical solutions.

The eIDAS Regulation ensures secure and seamless cross-border electronic transactions in the EU by promoting the widespread use and uptake of electronic identification and trust services (eIDAS services) across borders. It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where online public services are available. In addition, it helps to create a European internal market for electronic trust services by ensuring that they work across borders and have the same legal status as traditional paper based processes.

The eIDAS Regulation ensures secure and seamless cross-border electronic transactions in the EU by promoting the widespread use and uptake of electronic identification and trust services (eIDAS services) across borders. It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where online public services are available. In addition, it helps to create a European internal market for electronic trust services by ensuring that they work across borders and have the same legal status as traditional paper based processes.

In the eGovernment policy the Commission's responsibility is limited to the cross-border aspects. Yet the Action Plan included various actions that Member States committed to in a voluntary manner and where the Commission did not have a formal legal basis to act. The Action Plan fully respected the organisational set-up of eGovernment activities at national, regional and local levels and was complementary to national Action Plans under the responsibility of the Member States at the same time being instrumental in motivating and leveraging implementation in Member States of voluntary actions.

In the eGovernment policy the Commission's responsibility is limited to the cross-border aspects. Yet the Action Plan included various actions that Member States committed to in a voluntary manner and where the Commission did not have a formal legal basis to act. The Action Plan fully respected the organisational set-up of eGovernment activities at national, regional and local levels and was complementary to national Action Plans under the responsibility of the Member States at the same time being instrumental in motivating and leveraging implementation in Member States of voluntary actions.

The mid-term evaluation of the eGovernment Action Plan acknowledges that the Action Plan "has a positive impact on the development of eGovernment at the European and Member State level even if progress has been better in some areas compared to others. This type of Action plan can be a perfect ‘mobilizer’ instrument in order to help the European Commission and the Member States coordinating their actions". Several examples of this "mobilizer" effect have been given in this evaluation.

Most of the countries also stress that the cross-border dimension of the eGovernment Action Plan has had the strongest impact on their national strategies. It is this effect that makes up the EU-added value. Interoperable cross-border digital solutions can prevent the emergence of new obstacles to the Single Market. The building block approach created in the large-scale pilots and continued under the CEF Telecom programme is doing exactly that by promoting and supporting the re-use of such solutions in support of interoperability across borders.

While the Action Plan did not have a dedicated budget or funding instrument, it has served as a catalyst to coordinate resources. The research, innovation and piloting projects carried out under the umbrella of the eGovernment Action Plan in FP7, H2020, CIP, ISA and CEF programmes helped to pool public resources and coordinating with private resources in order to develop and pilot innovative eGovernment as well as pilot solutions to be deployed as key enablers for eGovernment.

Without the intervention of the eGovernment Action Plan these achievements may not have realised or would have been only partly achieved. Acting at EU level through the eGovernment Action Plan helped the Commission and Member States to coordinate and collaborate in order to ensure interoperability between national systems towards seamless access to digital public services across borders. Continued work is required to encourage the opening up between public administrations at all levels, the re-use of existing solutions to avoid duplication, the reduction of costs multiplication and the optimisation of investments. Coordination of efforts and resources contributes to strengthening the EU as a provider of benefits to citizens and businesses and to meeting their digital expectations of modern governments.

Finding 21

The Action Plan contributed to increased coordination between the Member States and in the Commission aiming at the modernisation of public administrations. An EU level Action Plan helped to realise the potential of cross-border eGovernment services to avoid further fragmentation and to increase interoperability.

7.Conclusions

7.

Conclusions

The findings of this report allow for a relatively clear assessment of the key evaluation criteria and questions despite the limited availability of recent quantitative data and certain limitations regarding the representativeness of responses to the stakeholder consultations.

The findings of this report allow for a relatively clear assessment of the key evaluation criteria and questions despite the limited availability of recent quantitative data and certain limitations regarding the representativeness of responses to the stakeholder consultations.

Effectiveness

Effectiveness

Throughout the analysis it was often impossible to isolate the impact of the Action Plan from other developments in the public sector and therefore to assess its direct impacts. However, the eGovernment Action Plan has been effective in achieving most of its objectives. Already the mid-term evaluation of the Action Plan recognised that the Action Plan had a positive impact on the development of eGovernment at the European and Member State level.

Throughout the analysis it was often impossible to isolate the impact of the Action Plan from other developments in the public sector and therefore to assess its direct impacts. However, the eGovernment Action Plan has been effective in achieving most of its objectives. Already the mid-term evaluation of the Action Plan recognised that the Action Plan had a positive impact on the development of eGovernment at the European and Member State level.

Despite there being varying degrees of progress in the Member States and in the different policy priorities, the Action Plan demonstrated the importance of having common European goals in eGovernment. The Action Plan acted also as a "mobiliser" instrument. According to the mid-term evaluation of the Action Plan, the national eGovernment strategies in almost all Member States incorporate priorities from the Malmö Declaration and the eGovernment Action Plan. Furthermore, most of the Member States (17) have incorporated the political priorities of the Malmö Declaration and eGovernment Action Plan in their national strategies.

Despite there being varying degrees of progress in the Member States and in the different policy priorities, the Action Plan demonstrated the importance of having common European goals in eGovernment. The Action Plan acted also as a "mobiliser" instrument.

According to the mid-term evaluation of the Action Plan, the national eGovernment strategies in almost all Member States incorporate priorities from the Malmö Declaration and the eGovernment Action Plan. Furthermore, most of the Member States (17) have incorporated the political priorities of the Malmö Declaration and eGovernment Action Plan in their national strategies.

Furthermore, in 2012 the annual eGovernment Benchmark reporting was aligned with the priorities of the eGovernment Action Plan 2011-2015. This provided a robust tool to monitor and measure the effectiveness of the results, but the Action Plan had no mechanism to iteratively improve the process and fine-tune the actions according to the findings.

Furthermore, in 2012 the annual eGovernment Benchmark reporting was aligned with the priorities of the eGovernment Action Plan 2011-2015. This provided a robust tool to monitor and measure the effectiveness of the results, but the Action Plan had no mechanism to iteratively improve the process and fine-tune the actions according to the findings.

The eGovernment Action Plan has acted as a focal point for actions on the EU level, in particular related to the support for the development of enablers for cross-border services. The Action Plan had a key role in pushing for the availability of key cross-border services. Since the adoption of the Action Plan in 2010, large-scale piloting actions have been co-funded by the Commission to achieve the transition towards a Digital Single Market. The effects of these pilots have been quantitative (take-up) and qualitative (increased awareness and coordination) and have, for example, contributed towards the adoption of the eIDAS Regulation. The e-procurement piloting through the PEPPOL project has contributed towards the take-up of e-invoicing in some Member States. The work on the large-scale pilots led to the support of the Member States of the CEF Telecom Regulation, aiming at financing the deployment of the cross-border services and the building block approach. This has also had an impact in the work of other sectors, for example, e-justice and e-health. The public online consultation indicated that awareness of cross-border services availability could be further increased.

Some of the eGovernment Action Plan actions have not been implemented. The mid-term evaluation acknowledges that in a rapidly changing world with very fast evolving technology, a static five years period seems too rigid for an Action Plan. This static nature has affected the effectiveness of the Action Plan as it was not possible to identify new actions needed, to adapt actions to changed circumstance or to remove actions that became obsolete, sometimes due to activities taking part elsewhere.

Moreover, the Action Plan, while prioritising user empowerment, did not have a mechanism to enable stakeholders to be part of the decision-making process that identified actions to be taken by Member States or the Commission during the duration of the plan. This could have led to greater acceptance of the results. The online public consultation showed the interest of stakeholders in an online social media platform to facilitate the sharing of ideas.

Areas of eGovernment can gain political visibility arising from political decision-making, which increases the overall effectiveness. On the other hand, effectiveness can suffer from too specific objectives if they cannot be updated during the duration of the Action Plan.

Efficiency

The eGovernment Action Plan had no dedicated budget.. but it mobilised resources in the Member States and the Commission to develop solutions and common understanding in several internal market policies, whicn in turn hase the potential to unleash long term savings from digitalisation.

Relevance

The Action Plan priorities (user empowerment; internal market; efficiency and effectiveness of government and administrations; and pre-conditions for developing eGovernment) remain relevant today. The modernisation of the European Public Sector remains an important goal in the EU.

However, given the changing environment with economic challenges, increasing use of online means and increasing intra-EU mobility the evaluation found that further work is needed, sometimes with evolving new needs. For example, there is a continued need to pursue the re-use of common solutions and to increase the take-up of both cross-border and national eGovernment services. Another example is the indicator "online service completion" being over 80% on average in the EU in 2015
  175
  , the relevance has moved from mere online availability of public services to, for example, quality of the services and cross-border availability.

There are various examples of topics with continued or increased relevance in the future. Some key enablers have been successful and the rollout looks promising, but the strengthening of the key enabler approach, including the take-up, remains relevant for future work. The Action Plan areas of Citizens rights, re-use of public sector information, transparency and data protection are important principles also in the future when modernising public administrations.

The relevance of eGovernment services for the internal market continues, driven also by the increased intra-EU mobility of citizens and businesses. Advancing the once-only principle continues to be a relevant target beyond the duration of the Action Plan. There are also continued needs to pursue the re-use of common solutions for rationalisation, savings and interoperability and to increase take-up.

The online public consultation confirmed the relevance of a number of these areas. There was a clear consensus (supported by 80% or more of the respondents from all categories) on the importance of applying the principles of Privacy by default, Digital by default, Cross-border by default, open by default, the Once-only principle, Online end-to-end services, Inclusive by default and Transparency to eGovernment policies and services in the EU.

Coherence

The governance structure of the Action Plan with regular meetings and exchanges of experience between different Member States and the Commission contributed to coherence and avoidance of overlaps. The Action Plan's coherence benefited from a political consensus (Malmö Declaration) between the Member States. However, full coherence could not be assured, as the Action Plan was based on voluntary activities of the Member States that also had different implementation approaches.

The coherence of the different funding mechanisms for eGovernment key enablers benefited from a common aim towards the deployment of mature services under the CEF Telecom programme.

A wide variety of eGovernment initiatives at various levels were taken during the Action Plan's five year duration. An Action Plan specific coherence was not fully achievable.

EU-added value

The Action Plan led to increase the coordination of investments in the Member States and in the Commission in order to focus on the modernisation of public administrations. An EU level action plan was probably the only way to start to realise the potential of cross-border eGovernment services and to achieve interoperability while avoiding fragmentation. If the eGovernment Action Plan was not to be renewed, there would be a risk of fragmentation of public services solutions, possible overlap of efforts and a risk of low availability of cross-border public services in the Digital Single Market.

8.ANNEXES

Annex 1.Procedural information

In December 2015, European Commission's Directorate-General Communications Networks, Content and Technology (DG CNECT) launched an implementation and evaluation assessment of the European eGovernment Action Plan 2011 – 2015 ("EU eGovernment Action Plan 2011 – 2015"), adopted on 15th December 2010, first and foremost in order to provide a set of lessons learnt to guide the work of the new eGovernment Action Plan 2016 – 2020 announced in the Digital Single Market Strategy.

The evaluation has been coordinated by the EC's Directorate-General Communications Networks, Content and Technology with the support of an Inter-Service Group with representatives of Commission Directorate-Generals Agriculture and Rural Development; Communication; Competition; Informatics; Economic and Financial Affairs; Employment; Social Affairs and Inclusion; Energy; Environment; Internal Market, Industry, Entrepreneurship and SMEs; Migration and Home Affairs, Joint Research Centre; Publications Office; Justice and Consumers; Mobility and Transport; Regional and Urban Policy; Research and Innovation; Health and Food Safety; Secretariat-General; Taxation; and Customs Union and Trade. The Inter-Service group steered and monitored progress of the evaluation exercise, ensuring the necessary quality, impartiality and usefulness of the evaluation. The group met three time during the evaluation process.

For the evaluation, a broad literature review was conducted covering various reports, EU studies and a number of international references. Important sources in the desk research were the mid-term evaluation of the eGovernment Action Plan 2011 - 2015
  176
  and the annual eGovernment Benchmarking reports (for further details see section 4). Furthermore, additional information and data (listed in the in the footnotes and in the "References and data sources" part in the end of this document) were used to answer the evaluation questions. Their methods are available in the original referenced reports. The eGovernment factsheets published on the Joinup platform
  177
  give an overview of eGovernment in each Member State.

A 12 week online public consultation was conducted to collect primary data and complemented by statistical data (Eurostat data and Digital Economy and Society Index DESI
  178
  ) and Digital Agenda Scoreboard reports
  179
  .

An effort was made to “triangulate” the data used throughout the analysis. This means that findings presented in the evaluation are supported by evidence from different data sources whenever additional data was available. Any contradicting evidence has been weighed according to its strength and quality before reaching conclusions

The evaluation faced some limitations in the collection of data, whose impact was mitigated to a maximum possible extent:

Measuring the impact of the eGovernment Action Plan is a challenging exercise as it is almost impossible to isolate the impact of the Action Plan from other developments in the public sector, such as ICT-policies or the increased use of the Internet in general. Furthermore, some objectives of the Action Plan, such as increased collaboration, are non-tangible and, as a result, difficult to quantify. The evaluation has used benchmarking to measure the progress in the different policy priority areas of the Action Plan and assessed the effectiveness criteria qualitatively.

The Action Plan did not have a dedicated budget but relied on funding from other programmes for a large part of its actions. Other actions were financed by the Member States. As a result, the efficiency analysis (in particular the value for money assessment) was difficult to carry out.

Despite being prompted in a number of occasions by the EC, there is no systematic measuring or reporting requirement for the Member States of cross-border public services use. To overcome this, the evaluation used data from other sources, including case studies.

Given the multiplicity of the tools used to gather evidence, the results obtained are of different nature (for instance, eGovernment related data stems from the mid-term evaluation of the Action Plan and the benchmarking reports while the online public consultation provided insights in order to prepare for the new eGovernment Action Plan 2016-2020).

The evaluation takes into account the inherent limitations of the findings of public consultations. Firstly, as in all surveys, the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who has a stake in this domain. Secondly, stakeholders' views convey an individual rather than a holistic perspective.

The mid-term evaluation did not systematically cover the five evaluation criteria. Furthermore, the methodology had strict requirements. For an action to qualify as "delayed" it was sufficient that one indicator is negative (e.g. one Member State answered ‘no’ to the question, or one action was delayed in one Member State) can sometimes give only a partial overview of progress and it should be understood as an evaluation of the actions as defined by the action plan, not an assessment of the policy priority as such. In addition, the mid-term evaluation did not capture the last 18 months of the Action Plan. This is partly mitigated using the 2015 benchmark study and partly the evaluation questions in the 2015/2016 online consultation on the eGovernment Action Plan and other recent reports and information.

Based on the elements above, the evaluation has been carried out on the basis of the best available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate and possibly counter-balanced with qualitative data and considerations.

Annex 2.The Online public consultation

REPORT

on the public consultation and other consultation activities of the European Commission for the preparation of the EU eGovernment Action Plan 2016-2020

1.INTRODUCTION

The Commission launched an online public consultation on 30 October 2015 for 12 weeks to seek views on a forthcoming EU eGovernment Action Plan 2016-2020, one of the actions for the completion the

  Digital Single Market

  . The consultation collected EU citizens' and businesses' needs and expectations from eGovernment services in the EU, and what public administrative bodies can or plan to deliver. It was accompanied by a

  roadmap

  for a new EU eGovernment Action Plan.

In addition, the Commission organised workshops with stakeholders. The consultation, those events, the annual eGovernment benchmark survey and the mid-term review of the previous action plan underpin the preparation of a new EU eGovernment Action Plan 2016–2020.

2.WORKSHOPS AND CONFERENCES WITH EXTERNAL STAKEHOLDERS

2.1.

  Riga, Digital Assembly 2015

The first of these events was organised as part of the Digital Assembly in Riga. The panellists at the workshop pointed to some urging user expectations, resulting in an initial list of possible digital rights, further expanded during the discussion with the audience. The list stresses possible rights related to user-friendly public services:

digital – including the ability to receive and submit documents electronically and eventually through one-stop-shops

intuitive

inclusive

accessible

fast

efficient

multilingual

automated services or at least reducing the number of interactions with public administrations, for example through the once-only principle.

Some of the suggested rights linked to modernising public administration, making them open, transparent and collaborative, allowing for citizen involvement and eDemocracy. Another set of suggested rights related to mobility within the single market, ensuring that data and digital services moved seamlessly across borders and the right to do business anywhere in the EU. Another group of rights pointed to basic pre-conditions that would make this happen i.e. privacy / confidentiality, the right to exist digitally, to control access to personal data, security, access to cheap and fast network, digital literacy and the quality of and access to machine readable data.

Participants ranked the 'once only principle', 'user friendly / intuitive public services' and 'digital literacy' as the three most important rights.

2.2.Workshops on a new EU eGovernment Action Plan, Brussels: 1 July, 5 November 2015, 15 December 2015

The first workshop on a new EU eGovernment Action Plan focused on the overall policy framework (modernising public administration, helping public administrations cooperate with one another and offering better service, citizens' involvement). Many pre-conditions must be achieved before actions are implemented such as privacy, quality and opening up of data, interoperability and security. The current Action Plan's mid-term review showed, that five-year static plans were not fit-for-purpose amidst rapidly changing technology.

At the second Join-up workshop, the Commission focused on the public consultation questions and how to engage citizens so that they propose ideas for the new Action Plan. Participants discussed setting up an online stakeholder platform, raising awareness about it, the need for political support, the use of multiple languages and maintaining contributors' interest in the platform.

The third workshop on a new EU eGovernment Action Plan focused on the online platform and how it could support a new eGovernment Action Plan.

2.3.

  Luxembourg eGovernment conference

  , 1st December 2015

Organised by the Luxembourg presidency, the event showed EU countries' progress on public sector modernisation. Important steps have been taken since the Malmö Ministerial Declaration on eGovernment of November 2009 and the EU eGovernment Action Plan 2011-2015 which lead to concrete measures on the political priorities of the Declaration.

Defining and launching a new coherent EU eGovernment Action Plan was central, aiming at increasing citizen and business engagement in the design of public services and policy-making, among others.

The Luxembourg Presidency concluded that improved coordination between EU countries is essential. An effective European Action Plan must consider all actors and measures affecting the EU eGovernment agenda, yet be centred on real user needs in EU countries.

3.THE ONLINE, PUBLIC CONSULTATION

The 12-week consultation was launched on 30 October 2015 until 22 January 2016. It covered the following topics related to the development of eGovernment services in the EU:

Lessons learned for the current Action Plan

Factors hampering the use of public services

Improving eGovernment services

Mobility and cross-border public services in the EU

Modernizing eGovernment services in the EU

The role of the European Commission

Citizen involvement

Policy principles

3.1.Overview of the respondents
  180

The online questionnaire received 365 replies; 12 position papers came from organisations and government representatives:

65% of the replies came from EU citizens

13% from businesses and organisations

22% from public administrations

Most replies were received from Germany (112), Romania (59), Italy (37), Czech Republic (32) and Spain (29). In the citizens group most replies came from Romanians (46), Italians (31) and Germans (31). In the Institution/Organisation/Business category, most replies were received from Germany (26) and Belgium (21).

3.2.The topics
  181

3.2.1.Lessons learned from the current Action Plan
  182

This section with mandatory questions addressed the eGovernment Action Plan 2011-2015 (on-going at the time).

Has the current eGovernment Action Plan improved cross-border eGovernment services overall?

43% of respondents replied that they don’t know; 28.5% said that it has not; 24.9% agreed that the plan has improved cross-border services overall. There are not significant differences between the respondent groups with regards to the perceived cross-border effects of the current Action Plan. The answers point to the need to increase awareness of on-going work and availability of cross-border public services in the EU.

How do you rate the measures comprising the current eGovernment Action Plan?

This section had 4 sub-questions addressing the measures of the previous action plan, asking about their success.

The respondents mostly replied that they were either unaware of the measures comprising the Action Plan or rated them as not successful, "not successful" comprising the largest group. For instance, on the User Empowerment measure of the action plan on inclusive services, collaborative production of services, re-use of public sector information, transparency and the involvement of citizens and businesses, 48.1%, said that it was not successful; 24% said that they don’t know; 27% of respondents found it successful. Increasing the efficiency and effectiveness of government and administrative bodies was viewed as the least successful, although by a small margin. Public administrations view the success of the measures addressing user empowerment and preconditions for developing eGovernment as much higher than the overall average. For instance, 46% rated the measures related to the preconditions for developing eGovernment successful, against a 28% average for the 3 respondent groups.

Action Plan activities did not target directly citizens and businesses. They were carried out by the Member States and the Commission to create for example enabling conditions not directly visible to the citizens as eGovernment Action Plan activities.

3.2.2.Factors hampering the use of digital public services
  183

Your contacts with the public administrations in your home country

This section addressed the factors that may/may not hamper the use of digital public services in the respondent's home country.

Citizens highlighted 3 factors not likely to hamper the use of digital public services: limited Internet access (63%), lack of digital skills (74%) and preference to interact with public administrative bodies through traditional channels (55.5%).

Concerning the factors likely or very likely to hamper the use of digital public services, 73% pointed to the lack of user-friendly public services (73%) and that public administrations require the same information more than once ('Once-only principle')(66%); 56% stressed meeting individual needs and 57% - the lack of online feedback mechanisms.

For the other factors e.g. accessibility on mobile devices, replies were more equally divided between factors that were likely and very likely to hamper the use of digital public services in the respondent's home country on the one hand and factors that were less likely and not likely to do so on the other.

Businesses also pointed towards the absence of the 'Once-only principle' as most likely to hamper the use of digital public services (82%). Other hampering factors businesses mentioned were the difficulty to find relevant information (75%), and the poor quality of pre-filled forms (71%). The factor least probable to stop businesses using digital public services in their contact with the public administrations is the preference for personal interaction (71%). One may conclude that businesses prefer to interact digitally with public sector administrations, but that the current state of online services is not fully satisfactory.

In addition to the pre-defined factors listed in the questionnaire, the lack of transparency and the insufficient respect of privacy were mentioned as other factors that could hamper the use of digital public services.

Public administrations listed 3 factors most likely to hamper the use of digital public services: the lack of accepted electronic identities (eIDs), the need to provide the same data more than once and poor quality and user unfriendly online public services. The lack of digital skills in public sector organisation staff was said to be the factor least likely to hamper the use of digital public services.

The questionnaire addressed the supply side of eGovernment services, asking about the likely factors to hold the organisation back from providing more or better government services online. Complex legislation making it difficult to digitise administrative procedures rank as the number one factor most likely to do this, followed by the lack of interconnections between public sector data sets and interoperability. Lack of political priority and leadership does not appear to be a major stumbling block.

Contact with the public administrations in another EU country

Regarding cross-border services, the need to provide paper documents in addition to online information were seen as the factors most likely to stop citizens from using digital public services in their contact with public administrations in another EU country (75%).

3.2.3.Improving eGovernment Services
  184

This section listed measures with the potential to improve eGovernment services, posting the same list of measures for citizens, businesses and public administrations to compare.

Citizens consider most of the listed measures important or highly important, with minor differences between the categories. Improving digital skills, both in business and the general public were seen as the least important factor to improve.

On the measures rated as highly important, making all online public services inclusive and accessible to all got the highest score (64%), giving users access to public services online (63%) and making online public services more trustworthy and secure.

Public administrations emphasised the personalisation of all online public services and making them user-friendly (98% in favour); the once-only provision of data got 90%, giving users access to services online (89%) and 83% for the acceptance of EU wide electronic signatures.

Inclusiveness and accessibility received 100% positive replies from business respondents, 93% to giving users access to public services online. There are hardly any differences between the answers to the different measures.

The different respondent categories rate some measures higher than others, but the differences do not conclusively point towards one or a few measures to address more than others. All measures with the potential to improve eGovernment services should be equally tackled.

Ensuring cross-border availability of public services is important for the smooth functioning of the internal market. 63% of citizens indicate that that the measures promoted at national level also should be promoted at the European level. That is positive for the continuation of eGovernment initiatives at European level. 4.9% said that this should not be the case; 32.1% replied that they don’t know. Businesses and the public sector also believe that national level measures should also be promoted at European level.

3.2.4.Mobility and cross-border services in the EU
  185

Citizens

This section addressed the cross-border aspect of eGovernment services, the expectations and difficulties citizens, businesses and public sector bodies may have had when interacting online with public authorities in another EU country.

54.1% of the respondents (citizens) said that they had had contact with or needed to engage with public authorities in an EU country other than their own.

Among the reasons for that contact education ranked highest (29.5%), followed by moving (22.5%), health-related services (20%), finding a job (16%), declaring taxes (11%), retiring and starting a judicial procedure (both 3.6%). Multiple answers were possible.

Many had indicated other reasons for the need to contact public authorities in another EU country i.e. setting up a business and contact with the European institutions. Most did not explain more about the other reasons they referred to.

When interacting online with a public authority in another EU country, many expected to be able to access all relevant information and start the procedure online using a "one-stop" shop (43%) and that the information would be provided in a language they understood (40%). Many also expected to be able to complete procedures, get help and communicate in a language they understood (37.5%) and use electronic signatures and electronic identifications from their home country (32%). Fewer expected to be able to give electronic access to personal data already provided in the home country (26%). Multiple answers were possible.

On the specific difficulties in transferring information/document/data between the public authorities of the home country and those of another, not accepting national information/documents/data was ranked the highest. In the free text under the "other" option, the difficulty to translate documents for official purposes and the lack of direct exchange of documents between public sector bodies in different countries were mentioned.

Businesses

Over half (52%) of the businesses replied that they had had contact with or needed to engage with public authorities in another EU country. Some reported no difficulties, others said they had had problems with the resubmission of documents already provided to a national authority or that national documents were not accepted.

Public sector bodies

The majority (62%) of public sector bodies reported that they had had contact with or needed to engage with public authorities in another EU country, and more so with authorities from more than one EU country in citizens-related cases. In doing so, 58% said contact was made using other means than the existing EU legal framework (e.g. under EU law on cooperation between national authorities) and in the absence of an established bilateral channel (62%) e.g. Memorandum of Understanding. Concerning potential difficulties in the transfer of information, most respondents replied that they had had problems. Few respondents referred to specific problems. Among those who did, language issues came on top.

3.2.5.Modernising eGovernment services in the EU
  186

Please indicate how important it would be to improve or apply eGovernment services over the next 5 years (for each of the following areas):

According to the respondents (citizens), most listed areas require improving i.e. eGovernment services for healthcare, filing tax declarations, enrolling in higher education, looking for a job and applying for a passport. eGovernment services related to buying and selling a home were seen as the least important.

Another element for the modernisation of eGovernment - the improvement of eGovernment procedures related to businesses - was addressed in the following question. None of the procedures listed were viewed predominantly as less or not important meaning that respondents would like to see all of them addressed. The procedures marked as more important compared to others were improving online services for tax- and insurance-related matters and obtaining government certificates.

When asked if measures to modernise eGovernment services should be proposed for the areas listed, public sector bodies mostly replied yes to all with minor differences in weighting between them.

Respondents would like to see eGovernment services improved for most eGovernment services and online procedures, not only a select few.

3.2.6.The role of the European Commission
  187
  and the new eGovernment Action Plan

Section 6 of the online questionnaire asked respondents how the Commission can help improve public administrations at all levels –national, regional and EU-wide (in free text).

The answers suggest various ways for the Commission to help improve public administrations: imposing EU legislation in specific areas e.g. establish an EU authority for digital certificates, setting standards for cross-border interoperability e.g. by consolidating existing national electronic registers, setting up an EU platform for e-participation, ensuring the exchange of good practice and cross-border cooperation, increase funding to eGovernment development in the regions including financing for the deployment of broadband infrastructures, design eGovernment policies to avoid social exclusion (universal design), harmonise the use of electronic identities including the issuing of an EU eIDs and make mandatory the use of electronic signatures, increase collaboration between governments and citizens in the design of eGovernment services and ensure the availability of open data-sets and open format documents (mandatory).

The setting of standards, interoperability at all levels and the exchange of good practice were mentioned by many as priority areas for the Commission in the future eGovernment Action Plan. The Commission was also asked to lead by example and improve its own internal and external services. Several respondents also underlined the need for the Commission to push for the application of the 'once-only principle' in order to reduce the administrative burden for citizens.

Ideas on the Commission's role and the focus of the new eGovernment Action Plan were contained in more detail in some of the position papers Member States submitted. For instance, the Danish Government sees the new eGovernment Action Plan as an important tool to move public sector digitisation in Member States forward and implement initiatives that will improve cross-border collaboration on public service delivery. The UK government believes it is crucial to improve access for businesses to the single market and that the role of the Commission is to support Member States in the delivery of cross border services, coordinate activities and monitor progress. Estonia also said that EU level activities should target the better functioning of the internal market. On the role of the Commission, the Polish government believes that it should coordinate and contribute to the creation of standards relevant to the digital single market; solutions related to the recognition of qualifications, data exchange between public registers in Member States, defining common document models and imposing interoperable solutions. The Commission should also provide open source solutions to handle electronic signatures, ePrior etc. under the Join-up initiative and lead by example. La poste (France) said that the eGovernment policy set forth by the Commission should facilitate digital interaction between governments and citizens/businesses reduce the administrative burden and leave the citizens in control.

In a follow-on question, all 3 respondent categories ranked accepting electronic signatures for the most significant communication flows as the most urgent. That would allow Commission suppliers and grant participants to send the relevant data and documents once only, and make e-invoicing and pre-award e-Procurement mandatory for all the new Commission market procedures.

3.2.7.The 2016 – 2020 eGovernment Action Plan, Citizen involvement
  188

This section asked how people should be enabled to contribute to, make proposals on and publicly exchange views on new initiatives under the eGovernment Action Plan e.g. should the Commission set up an online social media platform to facilitate sharing of ideas (free text).

A clear majority said that an online social media platform to reach out to a large audience is a good idea, although views were divided on whether the Commission should set up a dedicated one or use existing ones. Some viewed using exiting platforms as the more efficient option. Examples of comparable online platforms were provided e.g. the co-creating platform for budgetary spending in New York, and similar initiatives in Germany and France (Paris). With regards to the setting up of a dedicated online platform, respondents said that it should be designed with the users in mind, allow all languages, include feedback mechanisms, be backed up by e-signatures and allow users to compare eGovernment solutions in different EU countries.

Some argued that using an online social media platform is not sufficient. It has to be linked to going local initiatives and existing national mechanisms for reaching out to citizens. Due to the very technical nature of eGovernment, most likely the platform will only be used by experts and not ordinary citizens.

3.2.8.Policy Principles
  189

How important are the following strategic policy principles?

This section addressed policy principles that may be applied to the development of eGovernment services in the EU, asking respondents to rate them by importance.

Over 80% of the respondents agreed that these principles are important, including to have online end-to end services, that data should be provided only once, that services should be inclusive by default, transparent, privacy by default, No-legacy with regards to the technical infrastructure, cross-border and open by default. Privacy by default was said to be more important relative to the other principles but only marginally so. The replies do not provide evidence that one or several of the policy principle are viewed as significantly more important than any of the others.

Views cautioning the application of some principles referred to the need to adapt technology to the service in question and to user needs e.g. some user groups would be excluded if services were only provided digitally.

In the position papers, Member States referred to the reduction of the administrative burden and the 'Once-only principle' as important to the development of eGovernment services in the EU.

Respondents from the public sector were less favourable than citizens to the No legacy principle which would require governments to renew IT systems and technologies after a certain period of time to keep in line with the changing environment and technology.

Respondents agree that eGovernment policies should be designed with these policy principles in mind.

IV
     CONCLUSIONS

In view of the replies on the previous action plan, the new eGovernment Action Plan should be more visible if it is to reach a broader audience.

Stakeholders responding to consultation report that many factors still hamper the use of eGovernment services. The consultation and the workshops confirm the need to do more to improve eGovernment services in the EU at national and EU level. The consultation also confirms the Commission's role e.g. by promoting standards and ensuring interoperability at all levels. Certain principles should be applied to the setting up of eGovernment policies, including the "Once Only" principle to reduce the administrative burden to businesses and citizens. Overall the responses support the approach taken in the roadmap for the new eGovernment Action Plan.

Public consultation Report Annex 1:
     A: Graphs and associated questions

B: Free text questions

A: Graphs and associated questions

1)Information about the respondents

a)Nationality

b)Place of business

c)Age

2)Lessons learned for the current Action Plan

a) Has the current 2011-2015 eGovernment Action Plan improved cross-border eGovernment services overall

b)How do you rate the measures comprising the current eGovernment Action plan?

3)Factors hampering the use of public services, 1)Citizens, 2) Cross-border, 3)Businesses, and public administrations

a)For each of the following measures how likely is it to stop you using digital public services in your contact with the public administration in your country?

b)For each of the following factors how likely is it to stop you using digital public service in your contact with the public administrations in another EU country

c)For each of the following factors how likely is it to stop you and your business using digital public services in your contact with the public administrations in your country?

d)For each of the following factors how likely is it to stop you and your business using digital public services in your contact with the public administrations in another EU country?

e)As a public administrative body and a user of eGovernment services, please rate for each of the following factors how it is likely to hold back the use of public services at your level of government (EU level, national, regional or local)

f)As a public administrative body providing online public services, please state for each of the following factors how likely it is to hold your organization back from providing more and better government services online?

4)Improving eGovernment services

a)How important are the following measures in terms of improving eGovernment services?

b)Promoting at EU level the measures you have selected above as important or highly important?

5)Mobility and cross border services in the EU

a)Have you ever had contact with or needed to engage with

b)For what purpose?

c)When interacting online with a public authority in another EU country, expected?

d)Have you ever had difficulty transferring information/documents/data between the public authorities of your home country and those of another EU country?

e)Have you ever had contact with, or needed to engage with public authorities in an EU country other than your own?

f)If you have tried to engage with public authorities In another EU country (e.g.) for business purposes, have you ever had difficulties transferring documents/data between the public authorities of your home country and the country where you intended to do business?

g)To provide services to citizens, businesses or other public administrative bodies in your home country or abroad, have you had contact with, or needed to engage with, public authorities in an EU country other than your own?

h)With authorities from?

i)In cases concerning:

j)Did you use existing mechanism (under the EU law on cooperation between national authorities, IMI, Solvit)

k)Is there an established bilateral channel of communication with the authorities in the other EU country? (e.g. under a Memorandum of Understanding)

l)If you have tried to engage with public authorities in another EU country have you had difficulty transferring information?

6)Modernising eGovernment services

a)Please indicate how important it would be to improve eGovernment services over the next 5 years for each of the following areas?

b)Please indicate how important it would be to further improve or introduce each of the following eGovernment related procedures (local, regional, national and EU) over the next 5 years?

c)Should measures be proposed in the following areas?

7.The role of the European Commission

Please indicate for each of the following areas the level of priority for action by the European Commission.

8.Policy principles

How important are the following strategic policy principles?

B: Free text questions

Section 6 – Modernising eGovernment services in the EU

In which areas should eGovernment at all levels (local/regional/national and EU) be improved over the next 5 years?

Section 7 of the online questionnaire – The role of the European Commission

How can the European Commission help improve public administrations in the EU at all levels – regional, national and EU wide?

Section 8 – The 2016 2020 eGovernment Action Plan, Citizens involvement

How should people be enabled to contribute to, make proposals on and publicly exchange views on new initiatives emerging under the eGovernment Action Plan? For instance, should the Commission set up an online social media platform to facilitate the sharing of ideas?

Annex 3.The individual actions of the eGovernment Action Plan 2011 - 2015

Below is a description of the actions per priority and sub priority. Some of the actions were already defined in the Digital Agenda for Europe, they are indicated with the footnotes. The actions are numbered for the evaluation purposes. Of the 40 actions, the Commission was exclusively responsible for 23 (1, 3, 4, 6, 7, 8, 12, 13, 16, 19, 21, 25, 26, 27, 28, 30, 32, 33, 35, 36, 38, 39, 40 +41, 44) and the Member States for 10 (2, 5, 11, 17, 18, 20, 22, 31, 34, 37 +42, 43, 45). Seven actions (9, 10, 14, 15, 23, 24, and 29) were under shared responsibility between the Member States and the Commission.

1. User Empowerment

Services designed around users' needs and Inclusive Services

| 2011-2013 | 1. The Commission will support Member States in developing eGovernment services designed around user needs and in ensuring inclusiveness and accessibility by:


  agreeing common targets and evaluation criteria with the Member States,


  organising exchanges of valuable expertise at national, regional or local level to support additional take-up,


  supporting effective and concrete accessibility solutions, compliant with relevant European and international standards when available, through demonstration |
| --- | --- |
| 2013 | 2. Member States will develop personalised online services, including functions such as monitoring the progress of transactions with public administrations. |

2011-2013

1. The Commission will support Member States in developing eGovernment services designed around user needs and in ensuring inclusiveness and accessibility by:

agreeing common targets and evaluation criteria with the Member States,

organising exchanges of valuable expertise at national, regional or local level to support additional take-up,

supporting effective and concrete accessibility solutions, compliant with relevant European and international standards when available, through demonstration

2013

2. Member States will develop personalised online services, including functions such as monitoring the progress of transactions with public administrations.

Collaborative Production of Services

| 2011 | 3. Based on a study, the Commission will first assess how to involve users actively in design and production of eGovernment services and further elaborate recommendations / guidelines with and for the Member States. |
| --- | --- |
| 2011-2013 | 4. The Commission will facilitate exchanges of knowledge and experience between stakeholders, and, agree with Member States on common targets for the roll out collaborative services. |

2011

3. Based on a study, the Commission will first assess how to involve users actively in design and production of eGovernment services and further elaborate recommendations / guidelines with and for the Member States.

2011-2013

4. The Commission will facilitate exchanges of knowledge and experience between stakeholders, and, agree with Member States on common targets for the roll out collaborative services.

Re-use of Public Sector Information

| 2011 | 5. Member States will agree on a common set of PSI re-use indicators. |
| --- | --- |
| 2011 | 6. The Commission will conduct a study to assess to what extent open data catalogues and/or PSI portals (e.g. data.gov.uk) have been developed and implemented by Member States. |
| 2011-2013 | 7. The Commission will facilitate exchanges of good practice and awareness-raising activities and will adopt its own internal PSI re-use strategy based on a review of the PSI Decision. |
| 2011-2012 | 8. The Commission will review the PSI Directive, as indicated in the Digital Agenda for Europe, and will consider the possibility of an extended strategy for European PSI. |

2011

5. Member States will agree on a common set of PSI re-use indicators.

2011

6. The Commission will conduct a study to assess to what extent open data catalogues and/or PSI portals (e.g. data.gov.uk) have been developed and implemented by Member States.

2011-2013

7. The Commission will facilitate exchanges of good practice and awareness-raising activities and will adopt its own internal PSI re-use strategy based on a review of the PSI Decision.

2011-2012

8. The Commission will review the PSI Directive, as indicated in the Digital Agenda for Europe, and will consider the possibility of an extended strategy for European PSI.

Improvement of Transparency

| 2011 | 9. The Commission and Member States will set common voluntary transparency targets and exchange available experiences. |
| --- | --- |
| 2013 | 10. Member States and the Commission will provide online access to information on government laws and regulations, policies and finance. |
| 2014 | 11. In accordance with Data Protection Directive 95/46/EC, Member States will enable citizens to have electronic access to those personal data that are held on them when available electronically and will inform them electronically whenever such data are being processed by automatic means, in a simple and unambiguous manner. |

2011

9. The Commission and Member States will set common voluntary transparency targets and exchange available experiences.

2013

10. Member States and the Commission will provide online access to information on government laws and regulations, policies and finance.

2014

11. In accordance with Data Protection Directive 95/46/EC, Member States will enable citizens to have electronic access to those personal data that are held on them when available electronically and will inform them electronically whenever such data are being processed by automatic means, in a simple and unambiguous manner.

Involvement of citizens and businesses in policy-making processes

| 2011 | 12. The Commission will collaborate with Member States on developing the electronic service to support ‘citizens initiatives’ (as foreseen by Art. 11 of the Treaty on European Union). |
| --- | --- |
| 2011 | 13. The Commission will assess existing research projects and launch new ones under the ‘ICT for Governance and Policy Modelling’ objective of the 2011-2012 FP7 Work Programme and ensure further exchanges of knowledge and best practice. |
| 2011-2015 | 14. Member States, the Commission and other representative institutions such as parliaments should develop services that involve stakeholders in public debates and decision-making processes building on pilots and demonstration. |

2011

12. The Commission will collaborate with Member States on developing the electronic service to support ‘citizens initiatives’ (as foreseen by Art. 11 of the Treaty on European Union).

2011

13. The Commission will assess existing research projects and launch new ones under the ‘ICT for Governance and Policy Modelling’ objective of the 2011-2012 FP7 Work Programme and ensure further exchanges of knowledge and best practice.

2011-2015

14. Member States, the Commission and other representative institutions such as parliaments should develop services that involve stakeholders in public debates and decision-making processes building on pilots and demonstration.

2. Internal Market

2. Internal Market

Seamless Services for Businesses

Seamless Services for Businesses

| 2011 | 15. Member States and the Commission will assess outcomes of PEPPOL and SPOCS and ensure sustainable follow up. |
| --- | --- |
| 2011 | 16. The Commission will issue a White Paper on practical steps to inter-connect eProcurement capacity across the internal market
  190
  . |
| 2012-2014 | 17. Member States should roll out cross-border services based on the results of PEPPOL and SPOCS. |
| 2013 | 18. Member States will ensure that a ‘second generation’ of points of single contact will function as fully fledged eGovernment centres beyond the requirements and areas covered by the Services Directive.
  191 |

2011

15. Member States and the Commission will assess outcomes of PEPPOL and SPOCS and ensure sustainable follow up.

2011

16. The Commission will issue a White Paper on practical steps to inter-connect eProcurement capacity across the internal market
  190
  .

2012-2014

17. Member States should roll out cross-border services based on the results of PEPPOL and SPOCS.

2013

18. Member States will ensure that a ‘second generation’ of points of single contact will function as fully fledged eGovernment centres beyond the requirements and areas covered by the Services Directive.
  191

Personal Mobility

| 2012-2014 | 19. The Commission will support exchanges of best practice and coordinate the efforts of Member States to jointly develop and set up interoperable eDelivery services. |
| --- | --- |
| 2015 | 20. Member States will provide cross-border and interoperable eDelivery services for citizens, e.g. so that they can study, work, reside, receive health care and retire anywhere in the European Union. |

2012-2014

19. The Commission will support exchanges of best practice and coordinate the efforts of Member States to jointly develop and set up interoperable eDelivery services.

2015

20. Member States will provide cross-border and interoperable eDelivery services for citizens, e.g. so that they can study, work, reside, receive health care and retire anywhere in the European Union.

EU-wide implementation of cross-border services

| 2011 | 21. The Commission will conduct a study with the Member States, of the demand for cross-border services and assess the organisational, legal, technical and semantic barriers. |
| --- | --- |
| 2011 | 22. Member States will agree on a number of key cross-border public services to be rolled out between 2012 and 2015 and will identify appropriate life events/stages
  192
  . |
| 2012-2015 | 23. The Commission will support and coordinate the efforts of Member States to roll out Large Scale Pilot projects and to start new ones, while encouraging coordination and re-use of results and solutions between them. |
| 2012-2015 | 24. The Commission will work with Member States and stakeholders to implement cross-border eEnvironment services
  193
  . |

2011

21. The Commission will conduct a study with the Member States, of the demand for cross-border services and assess the organisational, legal, technical and semantic barriers.

2011

22. Member States will agree on a number of key cross-border public services to be rolled out between 2012 and 2015 and will identify appropriate life events/stages
  192
  .

2012-2015

23. The Commission will support and coordinate the efforts of Member States to roll out Large Scale Pilot projects and to start new ones, while encouraging coordination and re-use of results and solutions between them.

2012-2015

24. The Commission will work with Member States and stakeholders to implement cross-border eEnvironment services
  193
  .

3. Efficiency and Effectiveness of Governments and Administrations

3. Efficiency and Effectiveness of Governments and Administrations

Improving Organisational Processes

Improving Organisational Processes

| 2011 | 25. The Commission will facilitate the exchange of experience, encouraging re-use of successful solutions and applications and exploring new approaches to support the Member States in improving organisational processes. |
| --- | --- |
| 2011-2012 | 26. The Commission will transform the ePractice.eu portal into an effective experience exchange and information tool for Member States’ eGovernment practitioners |
| 2011-2015 | 27. The Commission will implement an ambitious eCommission Action Plan for 2011-2015, including full electronic procurement, a public sector information strategy and a transparency policy
  194
  . |
| 2013 | 28. The Commission, in close cooperation with Member States, will set up a programme for staff exchanges between administrations in different Member States. |

2011

25. The Commission will facilitate the exchange of experience, encouraging re-use of successful solutions and applications and exploring new approaches to support the Member States in improving organisational processes.

2011-2012

26. The Commission will transform the ePractice.eu portal into an effective experience exchange and information tool for Member States’ eGovernment practitioners

2011-2015

27. The Commission will implement an ambitious eCommission Action Plan for 2011-2015, including full electronic procurement, a public sector information strategy and a transparency policy
  194
  .

2013

28. The Commission, in close cooperation with Member States, will set up a programme for staff exchanges between administrations in different Member States.

Reduction of Administrative Burdens

| 2011-2013 | 29. The Commission will organise with Member States the sharing of experiences on the implementation of the 'once-only' registration principle and, on electronic procedures and communications having become a dominant channel for delivering eGovernment services, conduct a cost-benefit analysis and design a roadmap for further implementation. |
| --- | --- |

2011-2013

29. The Commission will organise with Member States the sharing of experiences on the implementation of the 'once-only' registration principle and, on electronic procedures and communications having become a dominant channel for delivering eGovernment services, conduct a cost-benefit analysis and design a roadmap for further implementation.

Green Government

| 2012 | 30. The Commission will conduct a study on the potential of eGovernment to reducing carbon footprint of governments including best practices. |
| --- | --- |
| 2013 | 31. Member States should develop and agree indicators and evaluation procedures for measuring the reduction of the carbon footprint of their administrations as a result of eGovernment services. |

2012

30. The Commission will conduct a study on the potential of eGovernment to reducing carbon footprint of governments including best practices.

2013

31. Member States should develop and agree indicators and evaluation procedures for measuring the reduction of the carbon footprint of their administrations as a result of eGovernment services.

4. Pre-conditions for developing eGovernment

4. Pre-conditions for developing eGovernment

Open Specifications and Interoperability

Open Specifications and Interoperability

| 2011-2015 | 32. The Commission (via the ISA programme) will implement activities to put into action the European Interoperability Framework (EIF) and the European Interoperability Strategy (EIS) (adoption of the EIF and EIS expected in 2010). |
| --- | --- |
| 2012 | 33. The Commission will organise exchanges of expertise and promote the re-use and sharing of solutions to implement interoperable eGovernment services. This includes establishment of interfaces to gain access to and use authentic national sources. |
| 2013 | 34. Member States should have aligned their national interoperability frameworks to the EIF
  195
  . |

2011-2015

32. The Commission (via the ISA programme) will implement activities to put into action the European Interoperability Framework (EIF) and the European Interoperability Strategy (EIS) (adoption of the EIF and EIS expected in 2010).

2012

33. The Commission will organise exchanges of expertise and promote the re-use and sharing of solutions to implement interoperable eGovernment services. This includes establishment of interfaces to gain access to and use authentic national sources.

2013

34. Member States should have aligned their national interoperability frameworks to the EIF
  195
  .

Key Enablers

| 2011 | 35. The Commission will propose a revision of the eSignature Directive with a view to providing a legal framework for cross-border recognition and interoperability of secure eAuthentication systems
  196
  . |
| --- | --- |
| 2012 | 36. The Commission will propose a Council and European Parliament Decision to ensure mutual recognition of eIdentification and eAuthentication across the EU, based on online ‘authentication services’ to be offered in all Member States (which may use the most appropriate official identification documents — issued by the public and private sectors)
  197
  . |
| 2012-2014 | 37. Member States should apply and roll out the eID solutions, based on the results of STORK and other eID-related projects. |

2011

35. The Commission will propose a revision of the eSignature Directive with a view to providing a legal framework for cross-border recognition and interoperability of secure eAuthentication systems
  196
  .

2012

36. The Commission will propose a Council and European Parliament Decision to ensure mutual recognition of eIdentification and eAuthentication across the EU, based on online ‘authentication services’ to be offered in all Member States (which may use the most appropriate official identification documents — issued by the public and private sectors)
  197
  .

2012-2014

37. Member States should apply and roll out the eID solutions, based on the results of STORK and other eID-related projects.

Innovative eGovernment

| 2011 | 38. The Commission will launch a study and recommend action on how to apply emerging technologies and paradigms (such as SOA and clouds of public services) in the public sector. |
| --- | --- |
| 2011 | 39. The Commission will launch activities under the CIP programme to support administrations to pilot the upgrade to IPv6, thereby creating showcases and new momentum for moving to IPv6 on a large scale. |
| 2012 | 40. The Commission will launch pilot projects to demonstrate how public administrations can deliver eGovernment services in a more flexible and efficient way by using innovative architecture and technologies. |

2011

38. The Commission will launch a study and recommend action on how to apply emerging technologies and paradigms (such as SOA and clouds of public services) in the public sector.

2011

39. The Commission will launch activities under the CIP programme to support administrations to pilot the upgrade to IPv6, thereby creating showcases and new momentum for moving to IPv6 on a large scale.

2012

40. The Commission will launch pilot projects to demonstrate how public administrations can deliver eGovernment services in a more flexible and efficient way by using innovative architecture and technologies.

Governance

In addition to the above policy priorities, the Action Plan also includes a section on its governance with five actions. This reflects the close partnership between Member States and the Commission. The Governance chapter is an element of the eGovernment Action Plan that was added to the four priority areas of the Malmö Declaration.

2011

| 2011 | 41. The Commission will establish a High-Level Expert Group of Member States’ representatives and will suggest an appropriate mandate. |
| --- | --- |
| 2012 | 42. Member States will inform the Commission and the High-Level Expert Group how the political priorities of the Malmö Ministerial Declaration have been or will be reflected in their national eGovernment strategies. |
| 2013 | 43. All Member States will have incorporated the political priorities of the Malmö Declaration in their national strategies. |
| 2013 | 44. The eGovernment Action Plan will be evaluated and the findings used to update the Action Plan. |
| 2015 | 45. All Member States will inform the Commission and the High-Level Expert Group how the political priorities of the Malmö Declaration have been achieved. |

41. The Commission will establish a High-Level Expert Group of Member States’ representatives and will suggest an appropriate mandate.

2012

42. Member States will inform the Commission and the High-Level Expert Group how the political priorities of the Malmö Ministerial Declaration have been or will be reflected in their national eGovernment strategies.

2013

43. All Member States will have incorporated the political priorities of the Malmö Declaration in their national strategies.

2013

44. The eGovernment Action Plan will be evaluated and the findings used to update the Action Plan.

2015

45. All Member States will inform the Commission and the High-Level Expert Group how the political priorities of the Malmö Declaration have been achieved.

Annex 4.Implementation of the individual actions of the eGovernment Action Plan 2011 – 2015 – State of play of the actions

Annex 4.

Implementation of the individual actions of the eGovernment Action Plan 2011 – 2015 – State of play of the actions

Priority: User Empowerment

Priority: User Empowerment

The Action Plan priority on user empowerment included 14 actions (listed in Annex 4) across five sub-priorities. The figure below
  198
  provides an overview of the status of implementation of these 14 actions until the beginning of 2014.

The Acti

on Plan priority on user empowerment included 14 actions (listed in Annex 4) across five sub-priorities. The figure below

198

provides an overview of the status of implementation of these 14 actions until the beginning of 2014.

Overview of progress of the User Empowerment priority. Status 6/2014.

Overview of progress of the User Empowerment priority. Status 6/2014.

User Empowerment priority

| User Empowerment priority |
| --- |
| Action number | Status in Mid-term evaluation (2013 - Q1/2014) and updates |
| | Services designed around users' needs and Inclusive Services |
| 1. | Within the Sub-Priority "Services designed around users’ needs and Inclusive Services" action (Action 1) on "Support Member States in developing eGovernment services in relation to common targets and evaluation criteria on the user-centred services" the Commission has supported exchanges of practice in the field of user-centred, inclusive and accessible eGovernment services (through the ePractice/Joinup platform
  199
  ), though there is no agreement with the Member States on common targets and evaluation criteria for those services.


  The Commission adopted its proposal for a directive on the accessibility of public sector bodies' websites on 3 December 2012, in accordance with Action 64 of the Digital Agenda for Europe where the Commission committed to "make proposals by 2011 that will make sure that public sector websites (and websites providing basic services to citizens) are fully accessible by 2015". The proposal aims to approximate the laws, regulations and administrative provisions of the Member States on the accessibility of websites of public sector bodies. The co-legislators are working on the proposal, which has not yet been adopted
  200
  .


  The MeAC (Measuring progress of eAccessibility in Europe) reports provided evidence and analysis to help understand and compare the approaches followed by the European countries, with a view to identifying issues and challenges, good practices and future priorities in the web accessibility field
  201
  . |
| 2. | Action 2 (development of personalised services and multi-channel delivery) is formally delayed; however, most Member States (85%, 22 out of the 26 answers) have developed personalised services and deliver them via multiple channels.


  Furthermore, according to the mid-term evaluation "Most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. Furthermore, the majority of the Member States that have already introduced personalised services offer them via multiple channels, such as mobile access via mobile apps and helplines". |
| | Collaborative production of Services |
| 3. | Action 3 (study on collaborative production in eGovernment
  202
  ) has been completed, and the policy recommendations of the study have been shared and discussed with the Member States. |
| 4. | The common targets for the roll-out of collaborative services have not been agreed upon (Action 4). |
| | Re-use of Public Sector Information (PSI) |
| 5. | According to the mid-term evaluation of 2014, there is no agreement on the common set of Public Sector Information (PSI) re-use indicators (Action 5). Eight out of the 27 countries that replied to this question affirmed to use a set of indicators for the re-use of Public Sector Information
  203
  . Indicators were further discussed in the 2015 study "Creating Value through Open Data" that analysed in-depth three exemplar indicators. |
| 6. | In Action 6, the Commission assessed in a study to what extent open data catalogues and/or PSI portals have been developed and are used in the Member States. A study on the pricing of public sector information/Open Data Portals was commissioned and its results were published in the end of 2011
  204
  . The study noted that "Open data portals have moved over the course of two years from the pioneering stage to the mainstream: they can be considered today as a recognized flagship initiative of government technology policy." A new study, "Creating Value through Open Data" was completed in November 2015
  205
  . The study forecasts benefits of the re-use of open data in the EU. |
| 7. | To complete Action 7, the Commission has set up a PSI Group
  206
  , the ePSI platform and several studies on PSI, and has developed its own PSI re-use strategy. |
| 8. | The reuse policy of the Commission is implemented by a Decision of 12 December 2011
  207
  in line with the objective of Action 8; the Commission has also reviewed the PSI Directive 2003/98/EC on re-use of public sector information (leading to the amending Directive 2013/37/EU) and introduced an Open Data Strategy for Europe in December 2011
  208
  . The Commission has put in place an Open Data Portal (in 2012) where open data from the EU institutions is are referenced
  209
  that has currently more than 8000 datasets available. The European Data Portal
  210
  , federating access to open data from all over Europe (EU, EEA and neighbouring countries) was launched in 2015. |
| | Improvement of Transparency |
| 9. | Fourteen Member States are using some transparency targets. Of those most are using Open Government Partnership indicators via the Independent Reporting Mechanism (IRM)
  211
  ; another common indicator is open data availability. A workshop on transparency targets was held at the end of 2012
  212
  . However, common voluntary transparency targets have not been set between the Commission and Member States. |
| 10. | Action 10 (open data and online access to government laws, actions and policies) is delayed according to the mid-term evaluation. Nevertheless, most Member States provide online access to information on government laws and regulations; fewer Member States provide information on policies. All Member States provide online access to information on government finance. Not all of them provide this information through an Open Data Portal. Nevertheless, the majority (19 Member States) already have Open Data Portals and almost all those countries offer data that are reusable for both non-commercial and commercial purposes. |
| 11. | For Action 11 (citizens’ electronic access to personal data) the mid-term evaluation reports that all Member States provide access to this information but not all provide access in electronic form (30% of Member States do so). |
| | Involvement of citizens and businesses in policy-making processes |
| 12. | The European Citizens' Initiative (ECI), allows one million EU citizens (according to a certain distribution to participate directly in the development of EU policies by calling on the European Commission to make a legislative proposal. Support of the ECI was implemented in April 2012 (action 12). The online system/portal is in place
  213
  . |
| 13. | The research projects under the ICT for Governance and Policy Modelling objective of the 2011-2012 FP7 Work Programme were assessed, and new projects under this objective were launched. |
| 14. | Joint action 14 on involving stakeholders in public debates and decision-making processes is on track. Most of the Member States have completed it. However, at the time of the mid-term evaluation, only a couple of Member States have developed centralised platforms for online consultations. Many Member States run e-petition platforms. However, not many Member States measure the level of take-up of the online consultations in a systematic way. The Commission has funded various eParticipation projects
  214
  . |

User Empowerment priority

Action number

Action number

Status in Mid-term evaluation (2013 - Q1/2014) and updates

Status in Mid-term evaluation (2013 - Q1/2014) and updates

Services designed around users' needs and Inclusive Services

Services designed around users' needs and Inclusive Services

1.

1.

Within the Sub-Priority "Services designed around users’ needs and Inclusive Services" action (Action 1) on "Support Member States in developing eGovernment services in relation to common targets and evaluation criteria on the user-centred services" the Commission has supported exchanges of practice in the field of user-centred, inclusive and accessible eGovernment services (through the ePractice/Joinup platform
  199
  ), though there is no agreement with the Member States on common targets and evaluation criteria for those services.

Within the Sub-Priority "

Services designed

around users’ needs and Inclusive Services

" action (

Action 1

) on "Support Member States in developing eGovernment services in relation to common targets and evaluation criteria on the user-centred services" the Commission has supported exchanges of

practice in the field of user-centred, inclusive and accessible eGovernment services (through the ePractice/Joinup platform

199

), though there is no agreement with the Member States on common targets and evaluation criteria for those services.

The Commission adopted its proposal for a directive on the accessibility of public sector bodies' websites on 3 December 2012, in accordance with Action 64 of the Digital Agenda for Europe where the Commission committed to "make proposals by 2011 that will make sure that public sector websites (and websites providing basic services to citizens) are fully accessible by 2015". The proposal aims to approximate the laws, regulations and administrative provisions of the Member States on the accessibility of websites of public sector bodies. The co-legislators are working on the proposal, which has not yet been adopted
  200
  .

The Commission

adopted its proposal for a directive on the accessibility of public sector bodies' websites on 3 December 2012, in accordance with Action 64 of the Digital Agenda for Europe where the Commission committed to "make proposals by 2011 that will make sure tha

t public sector websites (and websites providing basic services to citizens) are fully accessible by 2015". The proposal aims to approximate the laws, regulations and administrative provisions of the Member States on the accessibility of websites of public

sector bodies.

The co-legislators are working on the proposal, which has not yet been adopted

200

.

The MeAC (Measuring progress of eAccessibility in Europe) reports provided evidence and analysis to help understand and compare the approaches followed by the European countries, with a view to identifying issues and challenges, good practices and future priorities in the web accessibility field
  201
  .

The MeAC (Measuring progress of eAccessibility in Europe) reports provided evidence and analysis to help understand and compare the approaches followed by the

European countries, with a view to identifying issues and challenges, good practices and future priorities in the web accessibility field

201

.

2.

2.

Action 2 (development of personalised services and multi-channel delivery) is formally delayed; however, most Member States (85%, 22 out of the 26 answers) have developed personalised services and deliver them via multiple channels.

Action 2

(development of personalised services and multi-channel delivery) is formally delayed; however, most Member States (85%, 22 out of the 26 answers) have developed personalised services and deliver them via multiple channels.

Furthermore, according to the mid-term evaluation "Most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. Furthermore, the majority of the Member States that have already introduced personalised services offer them via multiple channels, such as mobile access via mobile apps and helplines".

Furthermore, according to the mid-term evaluation "Most Member States have introduced personalised services in their One-Stop-Shops for business and for citizens, as well as in the area of health and tax services. Furthermore, the majority of the Member States that have already introduced personalised services offer them via multiple channels, such as mobile access via mobile apps and helplines".

Collaborative production of Services

Collaborative production of Services

3.

3.

Action 3 (study on collaborative production in eGovernment
  202
  ) has been completed, and the policy recommendations of the study have been shared and discussed with the Member States.

Action 3

(study on collaborative production in eGovernment

202

) has been completed, and the policy recommendations of the study have been shared and discussed with the Member States.

4.

4.

The common targets for the roll-out of collaborative services have not been agreed upon (Action 4).

The common targets for the roll-out of collaborative services have not been agreed upon (

Action 4

).

Re-use of Public Sector Information (PSI)

Re-use of Public Sector Information (PSI)

5.

5.

According to the mid-term evaluation of 2014, there is no agreement on the common set of Public Sector Information (PSI) re-use indicators (Action 5). Eight out of the 27 countries that replied to this question affirmed to use a set of indicators for the re-use of Public Sector Information
  203
  . Indicators were further discussed in the 2015 study "Creating Value through Open Data" that analysed in-depth three exemplar indicators.

According to the mid-term evaluation of 2014, there is no agreement on the common set of Public Sector Information (PSI

) re-use indicators (

Action 5

). Eight out of the 27 countries that replied to this question affirmed to use a set of indicators for the re-use of Public Sector Information

203

. Indicators were further discussed in the 2015 study "Creating Value through Open D

ata" that analysed in-depth three exemplar indicators.

6.

6.

In Action 6, the Commission assessed in a study to what extent open data catalogues and/or PSI portals have been developed and are used in the Member States. A study on the pricing of public sector information/Open Data Portals was commissioned and its results were published in the end of 2011
  204
  . The study noted that "Open data portals have moved over the course of two years from the pioneering stage to the mainstream: they can be considered today as a recognized flagship initiative of government technology policy." A new study, "Creating Value through Open Data" was completed in November 2015
  205
  . The study forecasts benefits of the re-use of open data in the EU.

In

Action 6

, the Commission assessed in a study to what extent open data catalogues and/or PSI portals have been developed and are used in the Member States.

A study on the pricing of public sec

tor information/Open Data Portals was commissioned and its results were published in the end of 2011

204

. The study noted that "Open data portals have moved over the course of two years from the pioneering stage to the mainstream: they can be considered today

as a recognized flagship initiative of government technology policy." A new study, "Creating Value through Open Data" was completed in November 2015

205

. The study forecasts benefits of the re-use of open data in the EU.

7.

7.

To complete Action 7, the Commission has set up a PSI Group
  206
  , the ePSI platform and several studies on PSI, and has developed its own PSI re-use strategy.

To complete

Action 7

, the

Commission has set up a PSI Group

206

, the ePSI platform and several studies on PSI, and has developed its own PSI re-use strategy.

8.

8.

The reuse policy of the Commission is implemented by a Decision of 12 December 2011
  207
  in line with the objective of Action 8; the Commission has also reviewed the PSI Directive 2003/98/EC on re-use of public sector information (leading to the amending Directive 2013/37/EU) and introduced an Open Data Strategy for Europe in December 2011
  208
  . The Commission has put in place an Open Data Portal (in 2012) where open data from the EU institutions is are referenced
  209
  that has currently more than 8000 datasets available. The European Data Portal
  210
  , federating access to open data from all over Europe (EU, EEA and neighbouring countries) was launched in 2015.

The reuse policy of the Commission is implemented by a Decision of 12 December 2011

207

in line with the objective of

Action

8

; the Commission has also reviewed the PSI Directive 2003/98/EC on re-use of public sector information (leading to the amending Directive 2013/37/EU) and introduced an Open Data Strategy for Europe in December 2011

208

.

The Commission has put in place an Op

en Data Portal (in 2012) where open data from the EU institutions is are referenced

209

that has currently more than 8000 datasets available. The European Data Portal

210

, federating access to open data from all over Europe (EU, EEA and neighbouring countries) w

as launched in 2015.

Improvement of Transparency

Improvement of Transparency

9.

9.

Fourteen Member States are using some transparency targets. Of those most are using Open Government Partnership indicators via the Independent Reporting Mechanism (IRM)
  211
  ; another common indicator is open data availability. A workshop on transparency targets was held at the end of 2012
  212
  . However, common voluntary transparency targets have not been set between the Commission and Member States.

Fourteen Member States are using some transparency targets. Of those most are using Open Government Partnership indicators via the Independent Reporting Mechanism (IRM)

211

; another common indicator is

open data availability. A workshop on transparency targets was held at the end of 2012

212

. However, common voluntary transparency targets have not been set between the Commission and Member States.

10.

10.

Action 10 (open data and online access to government laws, actions and policies) is delayed according to the mid-term evaluation. Nevertheless, most Member States provide online access to information on government laws and regulations; fewer Member States provide information on policies. All Member States provide online access to information on government finance. Not all of them provide this information through an Open Data Portal. Nevertheless, the majority (19 Member States) already have Open Data Portals and almost all those countries offer data that are reusable for both non-commercial and commercial purposes.

Action 10

(open data and online access to government laws, actions and policies) is delayed according to the mid-term evaluation. Nevertheless, most Member States provide online access to information on government laws and regulations; fewer Member States provide information on policies. All Member States provide online access to information on government finance. Not all of them provide this information through an Open Data Portal. Nevertheless, the majority (19 Member States) already have Open Data Portals and almost all those countries offer data that are reusable for both non-commercial and commercial purposes.

11.

11.

For Action 11 (citizens’ electronic access to personal data) the mid-term evaluation reports that all Member States provide access to this information but not all provide access in electronic form (30% of Member States do so).

For

Action 11

(citizens’ electronic access to personal data) the mid-term evaluation reports that all Member States provide access to this information but not all provide access in electronic form (30% of Member States do so).

Involvement of citizens and businesses in policy-making processes

Involvement of citizens and businesses in policy-making processes

12.

12.

The European Citizens' Initiative (ECI), allows one million EU citizens (according to a certain distribution to participate directly in the development of EU policies by calling on the European Commission to make a legislative proposal. Support of the ECI was implemented in April 2012 (action 12). The online system/portal is in place
  213
  .

The European Citizens' Initiative (ECI), allows one million EU citizens (according to a certain distribution to participate directly in the development of EU policies by calling on the European Commission to make a

legislative proposal. Support of the ECI was implemented in April 2012 (

action 12

).

The online system/portal is in place

213

.

13.

13.

The research projects under the ICT for Governance and Policy Modelling objective of the 2011-2012 FP7 Work Programme were assessed, and new projects under this objective were launched.

The research projects under the ICT for Governance and Policy Modelling objective of the 2011-2012 FP7 Work Programme were assessed, and new projects under this objective were launched.

14.

14.

Joint action 14 on involving stakeholders in public debates and decision-making processes is on track. Most of the Member States have completed it. However, at the time of the mid-term evaluation, only a couple of Member States have developed centralised platforms for online consultations. Many Member States run e-petition platforms. However, not many Member States measure the level of take-up of the online consultations in a systematic way. The Commission has funded various eParticipation projects
  214
  .

Joint

action 14

on involving stakeholders in public debates and decision-making processes is on track. Most of the Member States have completed it. However, at the time of the mid-term evalu

ation, only a couple of Member States have developed centralised platforms for online consultations. Many Member States run e-petition platforms. However, not many Member States measure the level of take-up of the online consultations in a systematic way.

The Commission has funded various eParticipation projects

214

.

Priority: Internal Market

Priority: Internal Market

Overview of progress of the internal Market priority. Mid-term evaluation of the eGovernment Action Plan. Status 6/2014.

Overview of progress of the internal Market priority. Mid-term evaluation of the eGovernment Action Plan. Status 6/2014.

Internal Market priority

| Internal Market priority |
| --- |
| Action number | Status in Mid-term evaluation (2013 - Q1/2014) and updates |
| | Seamless Services for Businesses |
| 15. | Action 15 required Member States and the Commission to assess the outcomes of the PEPPOL
  215
  (e-procurement) and SPOCS
  216
  (eBusiness single points of contact) pilots, which ran from 2009-2012. Fifteen Member States and Norway took part in the SPOCS project. Nine Member States and Norway took part in the PEPPOL project, The Commission has assessed the outcomes of both projects
  217
  and launched a pilot project e-SENS (Electronic Simple European Networked Services) to consolidate the building blocks of the existing LSPs. |
| 16. | Action 16 on issuing a White Paper on practical steps to inter-connect eProcurement capacity across the internal market has been fully completed. In 2014, the Commission adopted new public procurement directives, which enter into force in April 2016
  218
  and make e-procurement gradually compulsory, with e-Submission being mandatory for all public buyers as of October 2018
  219
  . The policy aiming at supporting transition to full e-procurement is also building on a 2012 Communication on "A strategy for e-procurement" and a 2013 Communication on "End-to-end procurement to modernise public administration"
  220
  . |
| 17. | Two of the LSPs were assessed by the Commission and continued in a consolidated pilot project (eSENS) providing modules for eDocuments, eDelivery, eID, eSignature and semantics (Action 17). At the time of the mid-rem evaluation (2012 – 2013) twelve Member States (and Norway) have rolled out a LSP, while nine Member States (and Norway) have indicated they have plans to roll out a LSP by 2015 (but only two countries plan to do so of those that have not yet one out rolled). More Member States are rolling out PEPPOL services than participated in the pilot
  221
  . In the end of 2015 111 Access Points of eDelivery had been deployed:


          102 as part of the PEPPOL network (eProcurement domain) in over 18 countries (including 15 Member States) which accounts thousands of daily transactions.


  9 as part of the e-CODEX network (eJustice domain) which accounts over 700 exchanged documents in 2015 between 8 countries. |
| 18. | The action 18 required "Second Generation" of Points of Single Contact (PSC) that would function as fully-fledged eGovernment centres. A first assessment of the PSCs was carried out in 2011-2012
  222
  . The results revealed a hybrid landscape, with some PSCs more advanced than others in providing the information and services required. This first assessment resulted in the Communication on the implementation of the Services Directive "A partnership for new growth in services 2012-2015”
  223
  . This Communication encouraged Member States to develop by the end of 2014 Second Generation Points of Single Contacts, which should “1) cover all procedures during the business life cycle, 2) be multilingual, and 3) be more user-friendly”.


  These recommendations were subsequently included into the PSC Charter
  224
  , endorsed by the Council in 2013. The assessment of the PSCs performance against of the criteria set out in the Charter was carried out in 2014 and published in 2015 in a study
  225
  . |
| | Personal Mobility |
| 19. | e-Codex
  226
  LSP on e-justice consolidated the standard for eDelivery (action 19), that will enable citizens, businesses and public administrations to come together to do business online on the European market. |
| 20. | In action 20, the objective for the Member States was to provide cross-border and interoperable eDelivery services for citizens. According to the mid-term evaluation of the eGovernment Action Plan, more than 50% of the Member States already provide or are planning to provide those services. The CEF Telecom programme has financed the deployment of the cross-border eDelivery service. |
| | EU-wide implementation of cross-border services
  227 |
| 21. | For action 21, the Commission carried out a study on the demand for cross-border services
  228
  . |
| 22. | Related to action 22 Member States did not agree on a list. The CEF Telecom Regulation identifies the key cross-border services . |
| 23. | The action 23 focused on Commission to support and coordinate the efforts of Member States to roll out LSPs and to start new ones while encouraging coordination and re-use of results and solutions between them. The LSP eCodex
  229
  had started at the end of 2010 and a new initiative e-SENS was started during the Action Plan. In 2015, a new Large Scale Pilot was envisioned for the development of a solution of applying the once-only principle at European level for businesses
  230
  . The Commission has supported the coordination between the different pilots both in the development and re-use of solutions and in dissemination activities.


  The mid-term evaluation noted that under half of Member States had rolled out LSP projects. |
| 24. | The action 24 on cross-border eEnvironment services relates to the step-wise implementation (2007-2021) of the INSPIRE Directive
  231
  establishing an infrastructure for spatial information in the EU. With the support
  232
  of i.a the ISA and CIP programmes, several projects allowing public authorities to work more efficiently together on environmental issues across administrative and jurisdictional boundaries at regional
  233
  , national
  234
  and European
  235
  scales contributed to this action. A survey of the use of spatial information in eGovernment carried out by the ISA funded EULF
  236
  project, which builds on INSPIRE identified, in addition a large number of different public services using location information, many of which have the potential for integration in cross-border services. The technical Mid-term evaluation report
  237
  on the INSPIRE implementation, however, reports on a general delay and differences between the rate of INSPIRE implementation in the Member States affects the further roll-out of cross-border eEnvironment services. |

Action number

Status in Mid-term evaluation (2013 - Q1/2014) and updates

Seamless Services for Businesses

15.

Action 15 required Member States and the Commission to assess the outcomes of the PEPPOL
  215
  (e-procurement) and SPOCS
  216
  (eBusiness single points of contact) pilots, which ran from 2009-2012. Fifteen Member States and Norway took part in the SPOCS project. Nine Member States and Norway took part in the PEPPOL project, The Commission has assessed the outcomes of both projects
  217
  and launched a pilot project e-SENS (Electronic Simple European Networked Services) to consolidate the building blocks of the existing LSPs.

16.

Action 16 on issuing a White Paper on practical steps to inter-connect eProcurement capacity across the internal market has been fully completed. In 2014, the Commission adopted new public procurement directives, which enter into force in April 2016
  218
  and make e-procurement gradually compulsory, with e-Submission being mandatory for all public buyers as of October 2018
  219
  . The policy aiming at supporting transition to full e-procurement is also building on a 2012 Communication on "A strategy for e-procurement" and a 2013 Communication on "End-to-end procurement to modernise public administration"
  220
  .

17.

Two of the LSPs were assessed by the Commission and continued in a consolidated pilot project (eSENS) providing modules for eDocuments, eDelivery, eID, eSignature and semantics (Action 17). At the time of the mid-rem evaluation (2012 – 2013) twelve Member States (and Norway) have rolled out a LSP, while nine Member States (and Norway) have indicated they have plans to roll out a LSP by 2015 (but only two countries plan to do so of those that have not yet one out rolled). More Member States are rolling out PEPPOL services than participated in the pilot
  221
  . In the end of 2015 111 Access Points of eDelivery had been deployed:

        102 as part of the PEPPOL network (eProcurement domain) in over 18 countries (including 15 Member States) which accounts thousands of daily transactions.

9 as part of the e-CODEX network (eJustice domain) which accounts over 700 exchanged documents in 2015 between 8 countries.

18.

The action 18 required "Second Generation" of Points of Single Contact (PSC) that would function as fully-fledged eGovernment centres. A first assessment of the PSCs was carried out in 2011-2012
  222
  . The results revealed a hybrid landscape, with some PSCs more advanced than others in providing the information and services required. This first assessment resulted in the Communication on the implementation of the Services Directive "A partnership for new growth in services 2012-2015”
  223
  . This Communication encouraged Member States to develop by the end of 2014 Second Generation Points of Single Contacts, which should “1) cover all procedures during the business life cycle, 2) be multilingual, and 3) be more user-friendly”.

These recommendations were subsequently included into the PSC Charter
  224
  , endorsed by the Council in 2013. The assessment of the PSCs performance against of the criteria set out in the Charter was carried out in 2014 and published in 2015 in a study
  225
  .

Personal Mobility

19.

e-Codex
  226
  LSP on e-justice consolidated the standard for eDelivery (action 19), that will enable citizens, businesses and public administrations to come together to do business online on the European market.

20.

In action 20, the objective for the Member States was to provide cross-border and interoperable eDelivery services for citizens. According to the mid-term evaluation of the eGovernment Action Plan, more than 50% of the Member States already provide or are planning to provide those services. The CEF Telecom programme has financed the deployment of the cross-border eDelivery service.

EU-wide implementation of cross-border services
  227

21.

For action 21, the Commission carried out a study on the demand for cross-border services
  228
  .

22.

Related to action 22 Member States did not agree on a list. The CEF Telecom Regulation identifies the key cross-border services .

23.

The action 23 focused on Commission to support and coordinate the efforts of Member States to roll out LSPs and to start new ones while encouraging coordination and re-use of results and solutions between them. The LSP eCodex
  229
  had started at the end of 2010 and a new initiative e-SENS was started during the Action Plan. In 2015, a new Large Scale Pilot was envisioned for the development of a solution of applying the once-only principle at European level for businesses
  230
  . The Commission has supported the coordination between the different pilots both in the development and re-use of solutions and in dissemination activities.

The mid-term evaluation noted that under half of Member States had rolled out LSP projects.

24.

The action 24 on cross-border eEnvironment services relates to the step-wise implementation (2007-2021) of the INSPIRE Directive
  231
  establishing an infrastructure for spatial information in the EU. With the support
  232
  of i.a the ISA and CIP programmes, several projects allowing public authorities to work more efficiently together on environmental issues across administrative and jurisdictional boundaries at regional
  233
  , national
  234
  and European
  235
  scales contributed to this action. A survey of the use of spatial information in eGovernment carried out by the ISA funded EULF
  236
  project, which builds on INSPIRE identified, in addition a large number of different public services using location information, many of which have the potential for integration in cross-border services. The technical Mid-term evaluation report
  237
  on the INSPIRE implementation, however, reports on a general delay and differences between the rate of INSPIRE implementation in the Member States affects the further roll-out of cross-border eEnvironment services.

Priority: Efficiency and Effectiveness of Governments and Administrations

The figure below from the mid-term evaluation of the eGovernment Action Plan provides an overview of the seven actions included in the Efficiency and Effectiveness of governments and administrations priority.

Overview of progress of the Efficiency and Effectiveness priority. Mid-term evaluation of the eGovernment Action Plan. Status 6/2014. Action 26 was completed 2014. Action 29 (EC) was completed in 2014 – 2015.

| Efficiency and Effectiveness priority |
| --- |
| Action number | Status in Mid-term evaluation (2013/Q1/2014) and updates |
| | Improving Organisational Processes |
| 25. | Under the sub-priority Improving Organisational Processes, several meetings have been organised to facilitate exchange and re-use of experiences (action 25) and a study on “eGovernment and the Reduction of Administrative Burden”
  238
   was carried out to explore new approaches to improving organisational processes. |
| 26. | In December 2014, ePractice.eu was migrated to the collaborative platform Joinup
  239
  and action 26 fulfilled. The number of Joinup users is growing and is now above 25 000. |
| 27. | For Action 27 (on the implementation of the eCommission Action Plan), a Communication entitled "Delivering user-centric digital services"
  240
  was adopted in August 2012 and the Public Sector Information internal strategy has been drawn up. The implementation of the transparency policy is almost completed and the work on fully electronic procurement is on track. Fully electronic e-invoicing takes place in the EC
  241
  . |
| 28. | The programme for staff exchanges between administrations in different Member States (Action 28) was not carried out. |
| | Reduction of Administrative Burdens |
| 29. | For sub-priority "Reduction of administrative Burdens" a range of actions has been implemented both by the Commission and the Member States. The results of the study on 'eGovernment and the reduction of administrative burden' focusing on the once-only principle were published in 2014
  242
  including a cost-benefit analysis and a proposed roadmap for further implementation (Action 29). The European Council Conclusions
  243
   25 October 2013 stated that "EU legislation should be designed to facilitate digital interaction between citizens and businesses and the public authorities. Efforts should be made to apply the principle that information is collected from citizens only once, in due respect of data protection rules."


  The new H2020 Work Programme for 2016 – 2017, drafted and adopted in 2015, in the area of "Europe in a changing world – inclusive, innovative and reflective Societies" includes the once-only principle under topic CO-CREATION-05-2016 - Co-creation between public administrations: once-only principle
  244
  . This topic includes calls for two types of actions: a piloting action and a coordination and support action. |
| | Green Government |
| 30. | Under sub-priority on Green Government the Commission launched a survey with the Member States and a workshop on the topic (Action 30). No study was launched in this area on the potential of eGovernment to reducing carbon footprint of governments. |
| 31. | For developing indicators and evaluation procedures for "measuring the reduction of the carbon footprint (Action 31) of administrations" number of countries provided examples of evaluation projects and methods, but there was no Member States agreement on indicators and evaluation procedures. |

Efficiency and Effectiveness priority

Action number

Status in Mid-term evaluation (2013/Q1/2014) and updates

Improving Organisational Processes

25.

Under the sub-priority Improving Organisational Processes, several meetings have been organised to facilitate exchange and re-use of experiences (action 25) and a study on “eGovernment and the Reduction of Administrative Burden”
  238
   was carried out to explore new approaches to improving organisational processes.

26.

In December 2014, ePractice.eu was migrated to the collaborative platform Joinup
  239
  and action 26 fulfilled. The number of Joinup users is growing and is now above 25 000.

27.

For Action 27 (on the implementation of the eCommission Action Plan), a Communication entitled "Delivering user-centric digital services"
  240
  was adopted in August 2012 and the Public Sector Information internal strategy has been drawn up. The implementation of the transparency policy is almost completed and the work on fully electronic procurement is on track. Fully electronic e-invoicing takes place in the EC
  241
  .

28.

The programme for staff exchanges between administrations in different Member States (Action 28) was not carried out.

Reduction of Administrative Burdens

29.

For sub-priority "Reduction of administrative Burdens" a range of actions has been implemented both by the Commission and the Member States. The results of the study on 'eGovernment and the reduction of administrative burden' focusing on the once-only principle were published in 2014
  242
  including a cost-benefit analysis and a proposed roadmap for further implementation (Action 29). The European Council Conclusions
  243
   25 October 2013 stated that "EU legislation should be designed to facilitate digital interaction between citizens and businesses and the public authorities. Efforts should be made to apply the principle that information is collected from citizens only once, in due respect of data protection rules."

The new H2020 Work Programme for 2016 – 2017, drafted and adopted in 2015, in the area of "Europe in a changing world – inclusive, innovative and reflective Societies" includes the once-only principle under topic CO-CREATION-05-2016 - Co-creation between public administrations: once-only principle
  244
  . This topic includes calls for two types of actions: a piloting action and a coordination and support action.

Green Government

30.

Under sub-priority on Green Government the Commission launched a survey with the Member States and a workshop on the topic (Action 30). No study was launched in this area on the potential of eGovernment to reducing carbon footprint of governments.

31.

For developing indicators and evaluation procedures for "measuring the reduction of the carbon footprint (Action 31) of administrations" number of countries provided examples of evaluation projects and methods, but there was no Member States agreement on indicators and evaluation procedures.

Priority: Pre-conditions for developing eGovernment

Overview of progress of the Pre-conditions priority. Mid-term evaluation of the eGovernment Action Plan. Status 6/2014. Action 32 has been completed 2014 – 2015.

| Pre-conditions priority |
| --- |
| Action number | Status in Mid-term evaluation (2013/Q1/2014) and updates |
| | Open specifications and Interoperability |
| 32. | The level of implementation (action 32) of the European Interoperability Framework (EIF) is evaluated via NIFO
  245
  , the National Interoperability Framework Observatory. The NIFO is operational and Member States have been provided with guidance and support on aligning their national frameworks with the principles of the EIF (within the framework of the ISA programme implemented by DG DIGIT). |
| 33. | The exchanges of expertise, promotion of re-use of interoperable solutions, as well as interfaces for these exchanges (action 33) have been facilitated through the ISA programme
  246
  . |
| 34. | The NIFO report of 2014
  247
  stated that the majority of countries have taken into account the EIF in their National Interoperability Frameworks (NIFs) (action 34) and there is overall a very good NIF-EIF alignment. The interoperability situation in each Member State individually is provided in separate factsheets
  248
  that are regularly updated. Furthermore, the revision and extension of the EIF was included in the Digital Single Market Strategy in 2015. |
| | Key Enablers |
| 35.


  36. | Actions (35 and 36) on eSignatures and eIdentification have been completed. The eIDAS Regulation on electronic identification and trust services was adopted 2014 and came into force in September 2014
  249
  . The regulation sets a general legal and technical framework for electronic transactions. The aim is to achieve secure and seamless cross-border electronic transactions by promoting the widespread use and uptake of electronic identification and trust services (eIDAS services). It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where eIDs are available. In addition, it helps to create a European internal market for eTS (Trust Services) - namely electronic signatures, electronic seals, time stamp, electronic delivery service and website authentication - by ensuring that they will work across borders and have the same legal status as traditional paper based processes. |
| 37. | eID solutions have been applied and rolled out (action 37). While initiated in the STORK
  250
  projects (with around 22 M EUR EU co-funding), the eID building block is now part of the CEF Telecom Programme. The STORK eID has contributed to the eIDAS technical specifications.


  The status at the end of 2014 was
  251
  :


  In 6 Member States a framework had been developed for an interoperable service allowing foreign citizens (using their eID credentials) to notify all relevant entities of an address change. This was achieved without modifying current procedures in each Member State.


  12 Member States had integrated

  STORK

  with the EC's Authentication Service (ECAS). This integration allowed citizens from those Member States to use their nationally issued eIDs to access electronic services of the EC.


  5 Member States were using the

  STORK

  solution in their eDelivery applications, allowing citizens from other Member States to access the service with their own eID credentials.


  In 5 Member States foreign students could access online administrative and academic services offered by European Universities with their eID.


  10 Member States allowed foreign citizens to register for social security with their eID credentials.


  In 2015 the following developments were reported:


  15 countries, including 13 Member States, are connected to the eID network via STORK
  252
  .


  There were over 200 downloads of the sample software of the eID technical specifications compliant to eIDAS v0.9. This version was released on Joinup on 21 September 2015.


  There were over 1000 downloads of the sample software of the eID technical specifications compliant to eIDAS v1.00. This version was released on Joinup on 27 November 2015. |
| | Innovative eGovernment |
| 38. | The results of a study on Cloud and Service Oriented Architectures (SOA) for eGovernment
  253
  was published in 2011 (action 38). The results contributed to the development of a new vision of public services
  254
  . |
| 39. | Work for supporting administrations in piloting the upgrade to IPv6 (action 39) has been done in the CIP programme
  255
  . |
| 40. | The concept of cloud of public services was tested in the Competitiveness and Innovation Programme pilot projects
  256
  (action 40). |

Pre-conditions priority

Action number

Status in Mid-term evaluation (2013/Q1/2014) and updates

Open specifications and Interoperability

32.

The level of implementation (action 32) of the European Interoperability Framework (EIF) is evaluated via NIFO
  245
  , the National Interoperability Framework Observatory. The NIFO is operational and Member States have been provided with guidance and support on aligning their national frameworks with the principles of the EIF (within the framework of the ISA programme implemented by DG DIGIT).

33.

The exchanges of expertise, promotion of re-use of interoperable solutions, as well as interfaces for these exchanges (action 33) have been facilitated through the ISA programme
  246
  .

34.

The NIFO report of 2014
  247
  stated that the majority of countries have taken into account the EIF in their National Interoperability Frameworks (NIFs) (action 34) and there is overall a very good NIF-EIF alignment. The interoperability situation in each Member State individually is provided in separate factsheets
  248
  that are regularly updated. Furthermore, the revision and extension of the EIF was included in the Digital Single Market Strategy in 2015.

Key Enablers

35.

36.

Actions (35 and 36) on eSignatures and eIdentification have been completed. The eIDAS Regulation on electronic identification and trust services was adopted 2014 and came into force in September 2014
  249
  . The regulation sets a general legal and technical framework for electronic transactions. The aim is to achieve secure and seamless cross-border electronic transactions by promoting the widespread use and uptake of electronic identification and trust services (eIDAS services). It ensures that people and businesses can use their own national electronic identification schemes (eIDs) to access public services in other EU countries where eIDs are available. In addition, it helps to create a European internal market for eTS (Trust Services) - namely electronic signatures, electronic seals, time stamp, electronic delivery service and website authentication - by ensuring that they will work across borders and have the same legal status as traditional paper based processes.

37.

eID solutions have been applied and rolled out (action 37). While initiated in the STORK
  250
  projects (with around 22 M EUR EU co-funding), the eID building block is now part of the CEF Telecom Programme. The STORK eID has contributed to the eIDAS technical specifications.

The status at the end of 2014 was
  251
  :

In 6 Member States a framework had been developed for an interoperable service allowing foreign citizens (using their eID credentials) to notify all relevant entities of an address change. This was achieved without modifying current procedures in each Member State.

12 Member States had integrated

  STORK

  with the EC's Authentication Service (ECAS). This integration allowed citizens from those Member States to use their nationally issued eIDs to access electronic services of the EC.

5 Member States were using the

  STORK

  solution in their eDelivery applications, allowing citizens from other Member States to access the service with their own eID credentials.

In 5 Member States foreign students could access online administrative and academic services offered by European Universities with their eID.

10 Member States allowed foreign citizens to register for social security with their eID credentials.

In 2015 the following developments were reported:

15 countries, including 13 Member States, are connected to the eID network via STORK
  252
  .

There were over 200 downloads of the sample software of the eID technical specifications compliant to eIDAS v0.9. This version was released on Joinup on 21 September 2015.

There were over 1000 downloads of the sample software of the eID technical specifications compliant to eIDAS v1.00. This version was released on Joinup on 27 November 2015.

Innovative eGovernment

38.

The results of a study on Cloud and Service Oriented Architectures (SOA) for eGovernment
  253
  was published in 2011 (action 38). The results contributed to the development of a new vision of public services
  254
  .

39.

Work for supporting administrations in piloting the upgrade to IPv6 (action 39) has been done in the CIP programme
  255
  .

40.

The concept of cloud of public services was tested in the Competitiveness and Innovation Programme pilot projects
  256
  (action 40).

Annex 5.Acronyms

AGRI
     
     The Directorate General responsible for the Common Agricultural Policy.

BBs
     
     Building Blocks

CEF
     
     Connecting Europe Facility

CF
     
     Cohesion Fund

CIP 
     
     Competitiveness and Innovation Framework Programme

CIPA
     
     Common Infrastructure for Public Administrations Sustainability

CONNECT
     The European Commission Directorate General for Communications

Networks, Content & Technology

COM
     
     Communication from the Commission

COMM
     The Directorate General responsible for Communication

COMP
     The Directorate General in charge of Competition Policy in the European Union

CSP
     Core Service platforms

DAE 
     
     Digital Agenda for Europe

DESI
     
     Eurostat data and Digital Economy and Society Index

DIGIT
     
     Directorate General for Informatics

DG
     
     Directorate General

DSI 
     
     Digital Service Infrastructure

DSM 
     
     Digital Single Market

EAFRD 
     European Agricultural Fund for Rural Development

ECFIN
     Directorate General for Economic and Financial Affairs

DG CNECT 
     Communications Networks, Content and Technology DG (ex DG INFSO)

EACI
     
     Executive Agency on Competitiveness and Innovation

EC 
     
     European Commission

ECAS
     
     European Commission Authentication Service

ECI 
     
     European Citizens’ initiative

e-CODEX 
     e-Justice Communication via Online Data Exchange

e-Sens 
     
     Electronic Simple European Networked Services

EASME
     Executive Agency for SMEs

EEA
     
     European Economic Area

EFTA
     
     European Free Trade Association

EID
     
     Electronic Identity

eIDAS
     Electronic identification and trust services for electronic transactions in the internal market

EIF 
     
     European Interoperability Framework

EIS 
     
     European Interoperability Strategy

EIRA 
     
     European Interoperability Reference Architecture

EMPL
     
     Directorate for Employment, Social Affairs & Inclusion

ENER
     
     Directorate General for Energy

ENV
     
     Directorate General for the Environment

epSOS 
     European Patients - Smart Open Services

ERDF 
     
     European Regional Development Fund

ESF 
     
     European Social Fund

ESIF 
     
     European Structural and Investment Funds

ESPD
     
     European Single Procurement Document

eTS
     
     electornic Trust Services

EU 
     
     European Union

EULF
     
     European Union Location Network

FP7
     
     EU 7th Framework Programme for Research and Technological Development

GDP
     
     Gross Domestic Product

GROW
     Directorate General for Internal Market, Industry, Entrepreneurship and SMEs

HOME
     Directorate General for Migration and Home Affairs

ICT 
     
     Information and Communication Technology

ICT PSP 
     Information and Communication Technology Policy Support Programme

INSPIRE
     Infrastructure for Spatial Information for Europe Directive

IPv6
     
     Internet protocol version 6

IRM
     
     Independent Reporting Mechanisms

ISA 
     
     Interoperability Solutions for European public administrations

ISA2 
     Interoperability solutions for European public administrations, businesses and citizens

JRC
     EC Joint Research Centre

JUST
     Directorate for Justice and Consumers

KPIs
     Key Performance Indicators

LSPs 
     Large Scale Pilots

NIF 
     
     National Interoperability Framework

NIFO 
     
     National Interoperability Framework Observatory

MeAC
     
     Measuring progress of eAccessibility in Europe

MOVE
     Directorate General for Mobility and Transport

NGOs
     
     Non-Governmental Organisations

OECD
     
     Organisation for Economic Co-operation and Development

OP
     
     Publication Office of the European Union

PEPPOL
     Pan-European Public Procurement OnLine

PESTEL
     Political, Economic, Social, Technological, Environmental and Legal factors

PSC 
     
     Point of Single Contact

PSI 
     
     Public Sector Information

REFIT
     Regulatory Fitness and Performance of EU Legislation

REGIO
     Directorate General for Regional and Urban policy

SANTE
     Directorate General for Health and Food Safety

SEC
     Register of Commission Document: Documents which cannot be classified in any of the other series.

SEIS
     
     Shared Environmental Information System

SG
     
     Secretariat-General of the European Commission

SMEs
     
     Small and Medium Size companies

SOA
     
     Cloud and Service Oriented Architectures

SOLVIT
     Service provided by the national administrations in each EEA EFTA country

SPOCS
     Simple Procedures Online for Cross-border Services

STORK 
     Secure Identity Across Borders Linked

SWD
     
     Staff Working Document

TAXUD
     Directorate for Taxation

TRADE
     Directorate for Trade

UN
     
     United Nations

VAT
     
     Value Added Tax

WP 
     
     Work Programme

YEI
     
     Youth Employment Initiative

Annex 6.REFERENCES AND DATA SOURCES

-Ashfield District Council, INSPIRE and Open Data, March 2015 [available at:

  http://www.ashfield-dc.gov.uk/residents/democracy,-elections-and-legal/data-information-data-protection,-freedom-of-information-etc/inspire-and-open-data.aspx

  ]

-Austrian Institute of Economic Research (WIFO), Center for European Economic Research and IDEA Consult for the European Commission, Study Excellence in public administration for competitiveness in EU Member States, 2012 [available at: http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=7970]

-Capgemini Consulting for the European Commission, Creating Value through Open Data: Study on the Impact of Re-use of Public Data Resources, 2015 [available at:

  http://www.europeandataportal.eu/sites/default/files/edp_creating_value_through_open_data_0.pdf

  ]

-Capgemini Consulting for the European Commission, Open Data Maturity in Europe 2015 Insights into the European state of play, 2015 [available at:

  http://www.europeandataportal.eu/en/content/open-data-maturity-europe

  ]

-Capgemini et al. for the European Commission, Delivering the European Advantage? How European governments can and should benefit from innovative public services, 11th eGovernment Benchmark Report, 2014 [available at: https://ec.europa.eu/digital-single-market/en/news/eu-egovernment-report-2014-shows-usability-online-public-services-improving-not-fast]

-Capgemini et al. for the European Commission, eGovernment Benchmark Framework 2012 – 2015. Method paper, July 2012 [available at: https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/eGovernment%20Benchmarking%20method%20paper%20published%20version_0.pdf]

-Capgemini et al. for the European Commission, Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013 [available at

  https://ec.europa.eu/digital-single-market/news/final-report-study-analysis-needs-cross-border-services-and-assessment-organisational-legal

  ]

-CapGemini et al. for the European Commission, Smarter, Faster, Better eGovernment: 8th Benchmark Measurement, 2009 [available at

  https://joinup.ec.europa.eu/sites/default/files/files_epractice/sites/Smarter%2C%20Faster%2C%20Better%20eGovernment%20-%208th%20Benchmark%20Measurement.pdf

  ]

-Capgemini et al. for the European Commission, Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark Measurement , 2010 [available at:

  https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/egov_report.pdf

  ]

-Capgemini et al. for the European Commission, Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015 [available at:

  https://ec.europa.eu/digital-single-market/news/eu-egovernment-report-2015-shows-online-public-services-europe-are-smart-could-be-smarter

  ]

-Capgemini Consulting and Eurochambres for the European Commission, The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015 [available at:

  http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=8342

  ]

-Carlyle, S. Environment Agency for England and Wales, Reducing environmental risk through INSPIRE (quantified benefits) 2013 [available at:

  http://www.poweredbyinspire.eu/documents/0403-sustainability-carlyle.pdf

  ]

-Centre for strategy & Evaluation Services for the European Commission (CSES), Study Final Evaluation of the Competitiveness and Innovation Framework Programme, December 2011 [available at:

  http://ec.europa.eu/cip/files/cip/cip_final_evaluation_final_report_en.pdf

  ]

-Craglia M., Pavanello L. and Smith R. S., European Commission, Joint Research Centre Institute for Environment and Sustainability, The use of spatial Data for the Preparation of Environmental Reports in Europe, 2010 [available at:

  http://ec.europa.eu/environment/eia/pdf/jrc_technical%20report_2009%20eia-sea%20survey.pdf

  ]

-Ctrl-Shift, The economics of identity, June 2014 [

  https://www.ctrl-shift.co.uk/news/general/2014/06/09/the-economics-of-identity/

  ]

-Deloitte for the European Commission, The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' evaluation, 2013 [available at:

  https://ec.europa.eu/digital-single-market/news/final-report-study-feasibility-and-scenarios-long-term-sustainability-large-scale-pilots

  ]

-Deloitte for the European Commission, Study on cloud and service oriented architectures for eGovernment, Final report, 2011 [available at https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/smart2010-0074finalreport.pdf]

-Deloitte Consulting and Tech4i2 for the European Commission, Mid-term Evaluation of the e-Government Action Plan 2011–2015, 2014 [available at:

  https://ec.europa.eu/digital-single-market/news/mid-term-evaluation-e-government-action-plan-2011-2015-implementation-smart-2012-006020

  ]

-Deloitte and Tech4i2 for the European Commission, Pricing Of Public Sector Information Study on Models of Supply and Charging for Public Sector Information, 2011 [available at: 

  https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=1158

  ]

-Deloitte Consulting and Tech4i2 for the European Commission, Study on The functioning and usability of the Points of Single Contact under the Services Directive, 2012 [available at: 

  http://ec.europa.eu/internal_market/services/docs/services-dir/study_on_points/final_report_en.pdf

  ]

-E-Codex project [

  http://www.e-codex.eu/about-the-project.html

  ]

-ECORYS Nederland BV & Grontmij Nederland BV, Costs-benefits analysis INSPIRE in The Netherland, 2009. [available at:

  http://www.geonovum.nl/sites/default/files/nkba_engelse_vertaling.pdf

  ]

-EEA Technical report 17/2014. Mid-term evaluation report of INSPIRE evaluation. Joint EEA-JRC report. [available at: http://www.eea.europa.eu/publications/midterm-evaluation-report-on-inspire-implementation]

-Ernst & Young and Danish Technological Institute for the European Commission, Study on eGovernment and the Reduction of Administrative Burden, 2014 [available at

  https://ec.europa.eu/digital-single-market/en/news/final-report-study-egovernment-and-reduction-administrative-burden-smart-20120061

  ]

-EY for the European Commission, Study on Evaluation of the impact of the free movement of EU citizens at local level, 2014 [available at

  http://ec.europa.eu/justice/citizen/files/dg_just_eva_free_mov_final_report_27.01.14.pdf

  ]

-European Commission, A vision for public services, 2013 [available at:

  https://ec.europa.eu/digital-single-market/news/vision-public-services

  ]

-European Commission, Better Regulation Guidelines [available at:

  http://ec.europa.eu/smart-regulation/guidelines/ug_chap1_en.htm

  ]

-European Commission, CEF Monitoring report Q4/2015 [available at:

  https://joinup.ec.europa.eu/community/cef/news/first-cef-monitoring-report-available

  ]

-European Commission, Charter for the electronic Points of Single Contact under the Services Directive [available at:

  http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf

  ]

-European Commission, Communication to the Institutions - A Digital Agenda for Europe [COM(2010) 245 final; available at:

  http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52010DC0245R(01)

  ]

-European Commission, Communication to the Institutions - A Digital Single Market Strategy for Europe [COM(2015) 192; available at:

  http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1447773803386&uri=CELEX:52015DC0192

  ]

-European Commission, Communication to the Institutions - A strategy for e-procurement - [COM/2012/0179 final; available at:

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0179:FIN:EN:PDF

  ]

-European Commission, Communication from the Commission - Annual Growth Survey 2013 [COM(2012) 750 final; available at:

  http://ec.europa.eu/europe2020/pdf/ags2013_en.pdf

  ]

-European Commission, Communication to the Institutions - Annual Growth Survey 2015 [COM(2014) 902 final; available at:

  http://ec.europa.eu/europe2020/pdf/2015/ags2015_en.pdf

  ]

-European Commission, Communication to the Institutions - Annual Growth Survey 2016 [COM(2015) 690; available at: http://ec.europa.eu/europe2020/pdf/2016/ags2016_annual_growth_survey.pdf]

-European Commission, Communication to the Institutions - Better regulation for better results - An EU agenda [COM(2015)215 final of 19 May 2015; available at:

  http://ec.europa.eu/smart-regulation/better_regulation/documents/com_2015_215_en.pdf

  ]

-European Commission, Communication to the Institutions - End-to-end e-procurement to modernise public administration [COM/2013/0453 final; available at:

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2013:0453:FIN:EN:PDF

  ]

-European Commission, Communication to the Institutions – Europe 2020 strategy [COM(2010)2020; available at:

  http://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-%20Europe%202020%20-%20EN%20version.pdf

  ]

-European Commission, Communication to the Institutions, Towards interoperability for European public services [COM(2010) 744 final, available at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0744:FIN:EN:PDF]

-European Commission, Communication to the Institutions, European Interoperability Framework (EIF) for European public services [COM(2010) 744 final, Annex 2; available at

  http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-3action_en.htm

  ]

-European Commission, Communications to Institutions - Evaluations of the Competitiveness and Innovation Framework Programme [COM(2013)2; available at:

  http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm

  ]

-European Commission, Communication to the Institutions - i2010 eGovernment Action Plan: Accelerating eGovernment in Europe for the Benefit of All [COM(2006) 173 final; available at:

  http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52006DC0173&from=EN

  ]

-European Commission, Communication to the Institutions - On the implementation of the Services Directive. A partnership for new growth in services 2012-2015 [COM(2012) 261 final; available at:

  http://ec.europa.eu/internal_market/services/docs/services-dir/implementation/report/COM_2012_261_en.pdf

  ]

-European Commission, Communication to the Institutions - Open data - An engine for innovation, growth and transparent governance [COM(2011) 882 final; available at

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0882:FIN:EN:PDF

  ]

-European Commission, Communication to the Institutions - The European eGovernment Action Plan 2011-2015, Harnessing ICT to promote smart, sustainable & innovative Government [COM(2010) 743; available at:

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0743:FIN:EN:PDF

  ]

-European Commission, Communication to the Institutions - Towards interoperability for European public services [COM(2010) 744 final; available at:

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0744:FIN:EN:PDF

  ]

-European Commission, Communication to the Institutions - Unleashing the Potential of Cloud Computing in Europe [COM(2012)0529 final; available at:

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0529:FIN:EN:PDF

  ]

-European Commission, Communication to the Institutions -Upgrading the Single Market: more opportunities for people and business [COM(2015)550; available at 

  https://ec.europa.eu/transparency/regdoc/rep/1/2015/EN/1-2015-550-EN-F1-1.PDF

  ]

-European Commission, Competitiveness and Innovation Framework Programme ICT Policy Support Programme [Archived site:

  http://ec.europa.eu/cip/ict-psp/index_en.htm

  ]

-European Commission, Competitiveness and Innovation Framework Programme (CIP)- Implementation Reports [available at:

  http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm

  ]

-European Commission, Connecting Europe Facility [https://ec.europa.eu/digital-single-market/connecting-europe-facility]

-European Commission, Contributions and preliminary trends of the public consultation on the eGovernment Action Plan 2016 – 2020, 2016 [available at:

  https://ec.europa.eu/digital-single-market/news/contributions-and-preliminary-trends-public-consultation-egovernment-action-plan-2016-2020

  ]

-European Commission, DG Internal Market and Services: Management Plan 2013, 2012 [

  http://ec.europa.eu/dgs/internal_market/docs/amp_2013_en.pdf

  ]

-European Commission, Digital Agenda, Action 91: Member States to agree a common list of key cross-border public services [https://ec.europa.eu/digital-single-market/pillar-vii-ict-enabled-benefits-eu-society/action-91-member-states-agree-common-list-key-cross]

-European Commission, Digital Agenda Scoreboard report 2011, Pillar 7 [available at:

  https://ec.europa.eu/digital-single-market/download-scoreboard-reports

  ]

-European Commission, Digital Agenda Scoreboard report 2012. [available at

  https://ec.europa.eu/digital-single-market/download-scoreboard-reports

  ]

-European Commission, Digital Agenda Scoreboard reports [available at:

  https://ec.europa.eu/digital-single-market/download-scoreboard-reports

  ]

-European Commission, Digital Agenda Targets Progress report, 2015 [available at: 

  https://ec.europa.eu/digital-single-market/download-scoreboard-reports

  ]

-European Commission, Digital Economy and Society Index DESI [available at https://ec.europa.eu/digital-single-market/digital-economy-and-society-index-desi]

-European Commission, Digital Single Market, Connecting Europe Facility [

  https://ec.europa.eu/digital-single-market/connecting-europe-facility

  ]

-European Commission, e-Commission 2012 – 2015, Communication from VP Šefčovič to the Commission -Delivering user-centric digital services [SEC(2012) 492; available at: 

  http://ec.europa.eu/dgs/informatics/ecomm/doc/communication_sefcovic_tothecom.pdf

  ]

-European Commission, eGovernment Action Plan 2016-2020, Roadmap [available at:

  http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_cnect_006_e_government_action_plan_en.pdf

  ]

-European Commission, eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century [https://ec.europa.eu/digital-single-market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century]

-European Commission, e-PRIOR Supplier Portal [

  http://ec.europa.eu/dgs/informatics/supplier_portal/index_en.htm

  ]

-European Commission, ePSIplatform [

  http://epsiplatform.eu/

  and

  http://www.epsiplatform.eu/content/european-psi-scoreboard

  ]

-European Commission, EU eGovernment Report 2015 shows that online public services in Europe are smart but could be smarter, 23rd June 2015 [https://ec.europa.eu/digital-single-market/news/eu-egovernment-report-2015-shows-online-public-services-europe-are-smart-could-be-smarter]

-European Commission, EU implementation of the G8 Open Data Charter [https://ec.europa.eu/digital-single-market/news/eu-implementation-g8-open-data-charter]

-European Commission, European citizens' initiative (ECI) [

  http://ec.europa.eu/citizens-initiative/public/welcome

  ]

-European Commission, European Union Open Data Portal [

  https://open-data.europa.eu

  ]

-European Commission, Europe's Digital Competitiveness Report 2010 [available at: https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/european_competitiveness_report_2010.pdf]

-European Commission, Eurostat, E-government usage by individuals [available at:

  http://ec.europa.eu/eurostat/en/web/products-datasets/-/TSDGO330

  ]

-European Commission, Eurostat, Foreign citizens living in the EU Member States, 2015 [available at: http://ec.europa.eu/eurostat/documents/2995521/7113991/3-18122015-BP-EN.pdf/d682df12-8a77-46a5-aaa9-58a00a8ee73e]

-European Commission, Eurostat, ICT survey of Households and Individuals, 2014 [available at:

  http://ec.europa.eu/eurostat/statistics-explained/index.php/Information_society_statistics_-_households_and_individuals

  ]

-European Commission, Eurostat, Information society indicator "Enterprises using the internet for interacting with public authorities"

-European Commission, Eurostat, Individuals using the internet for interaction with public authorities [available at:

  http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&plugin=1&pcode=tin00012&language=en

  ]

-European Commission, Eurostat, Migration and migrant population statistics [available at: http://ec.europa.eu/eurostat/statistics-explained/index.php/Migration_and_migrant_population_statistics]

-European Commission, GROWTH, E-Certis [https://ec.europa.eu/growth/tools-databases/ecertis/]

-European Commission, GROWTH, E-procurement [

  http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm

  ]

-European Commission, GROWTH, European Single Procurement Document (ESPD) [

  https://ec.europa.eu/growth/tools-databases/espd

  ]

-European Commission, GROWTH, New rules on public procurement and concession contracts as of April 2016 [

  http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

  ]

-European Commission, GROWTH, Single Market and Standards – New rules on Public procurement and concession contracts as of April 2016 [

  http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

  ]

-European Commission, Guidance on Ex ante Conditionalities for the European Structural and Investment Funds, 2014 [available at:

  http://ec.europa.eu/regional_policy/sources/docgener/informat/2014/eac_guidance_esif_part2_en.pdf

  ]

-European Commission, High Level Group on Business Services, Final Report, 2014 [available at:

  http://ec.europa.eu/growth/industry/policy/renaissance/high-level-group/index_en.htm

  ]

-European Commission, Horizon 2020 Work Programme 2016 – 2017, Europe in a changing world – inclusive, innovative and reflective Societies [European Commission Decision C (2015)6776 of 13 October 2015; available at:

  http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-societies_en.pdf

  ]

-European Commission, INSPIRE, MIG, National Implementation Seminars [

  https://ies-svn.jrc.ec.europa.eu/projects/mig-inspire/wiki/National_implementation_webinars

  ]

-European Commission, Interoperability Solutions for European Public Administrations [

  http://ec.europa.eu/isa/

  ]

-European Commission, ISA programme, Common Infrastructure for Public Administrations Sustainability [

  http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-6_1-19action_en.htm

  ]

-European Commission, Joinup platform [

  https://joinup.ec.europa.eu/

  ]

-European Commission, Joinup, Connecting Europe Facility, Catalogue of Building Blocks, 2014 [

  https://joinup.ec.europa.eu/community/cef/og_page/catalogue-building-blocks

  ]

-European Commission, Justice, Protection of personal data [

  http://ec.europa.eu/justice/data-protection/

  ]

-European Commission, Justice, Reform of EU data protection rules [

  http://ec.europa.eu/justice/data-protection/reform/index_en.htm

  ]

-European Commission, Justice, EU Justice Scoreboard [http://ec.europa.eu/justice/effective-justice/scoreboard/index_en.htm]

-European Commission, MEMO/10/200, 19th May 2010, Digital Agenda for Europe: key initiatives [available at:

  http://europa.eu/rapid/press-release_MEMO-10-200_en.htm?locale=en

  ]

-European Commission, Mid-Term Evaluation of the eGovernment Action Plan - Evaluation of the eGovernment Action Plan [

  http://www.egovap-evaluation.eu

  ]

-European Commission, National Interoperability Framework Observatory, eGovernment Factsheets 2015 (published in 2016) [available at:

  https://joinup.ec.europa.eu/community/nifo/og_page/egovernment-factsheets

  ]

-European Commission, National Interoperability Framework Observatory, Factsheets (published in 2015) [available at:

  https://joinup.ec.europa.eu/community/nifo/og_page/nifo-factsheets

  ]

-European Commission, National Interoperability Framework Observatory (NIFO), JoinUp community [

  https://joinup.ec.europa.eu/community/nifo/home

  ]

-European Commission, Performance of the Points of Single Contact - an assessment against the PSC Charter, 29.6.2015 [

  http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=8342

  ]

-European Commission, Press Release IP/11/1524, 12 December 2011 [available at:

  http://europa.eu/rapid/press-release_IP-11-1524_en.htm?locale=en

  ]

-European Commission, Press Release IP/13/608, June 2013 "E-invoicing in public procurement: another step towards end-to-end e-procurement and e-government in Europe" [

  http://europa.eu/rapid/press-release_IP-13-608_en.htm

  ]

-European Commission, Proposal for a Directive of the European Parliament and of the Council on the accessibility of public sector bodies' websites [COM(2012) 721 final; available at

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0721:FIN:EN:PDF

  ]

-European Commission, PSI Scoreboard Indicator List [

  http://www.epsiplatform.eu/content/psi-scoreboard-indicator-list

  ]

-European Commission, Public Services, Cross-border solutions [https://ec.europa.eu/digital-single-market/node/77121]

-European Commission, Public Services, eParticipation [https://ec.europa.eu/digital-single-market/eparticipation]

-European Commission, Public Services, ICT-enabled public sector innovation in Horizon 2020 [https://ec.europa.eu/digital-single-market/ict-enabled-public-sector-innovation-horizon-2020]

-European Commission, Public Services, Towards a Cloud of Public Services [https://ec.europa.eu/digital-single-market/towards-cloud-public-services]

-European Commission, Regional Policy- European Structural and Investment Funds [

  http://ec.europa.eu/regional_policy/en/funding/

  ]

-European Commission, Regional Policy, Thematic objectives [

  http://ec.europa.eu/regional_policy/en/policy/what/glossary/t/thematic-objectives

  ]

-European Commission, Staff Working Document – A Digital Single Market Strategy for Europe - Analysis and Evidence – [SWD (2015) 100 final; available at:

  http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52015SC0100&from=EN

  ]

-European Commission, Staff Working Document – i2010 eGovernment Action Plan: Accelerating eGovernment in Europe for the Benefit of All [SEC(2006) 511; available at:

  http://ec.europa.eu/smart-regulation/impact/ia_carried_out/docs/ia_2006/sec_2006_0511_en.pdf

  ]

-European Commission, Staff Working Document, Impact assessment, Proposal for a Directive of the European Parliament and of the Council on Single-member private limited liability companies [SWD(2014) 124 final; available at:

  http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52014SC0124&qid=1458297454242&from=EN

  ]

-European Commission, Staff Working Document - The European eGovernment Action Plan 2011-2015 –Harnessing ICT to promote smart, sustainable & innovative Government [SEC(2010) 1539 final; available at

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=SEC:2010:1539:FIN:EN:PDF

  ]

-European Commission, The European e-Justice Portal [

  https://e-justice.europa.eu/home.do

  ]

-European Commission, Workshop "Improving Transparency in Government Decision Making" 6 November 2012 [https://ec.europa.eu/digital-single-market/news/workshop-improving-transparency-government-decision-making-6112012-results-presentations]

-European Council, Conclusions, 24/25 October 2013; [EUCO 169/13, available at:

  http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ec/139197.pdf

  ]

-European Parliament, European Parliament resolution of 19 January 2016 on Towards a Digital Single Market Act [2015/2147(INI); available at:

  http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-//EP//TEXT+TA+P8-TA-2016-0009+0+DOC+XML+V0//EN

  ]

-Fornefeld, M. et al., Assessment of the Re-use of Public Sector Information (PSI) in the Geographical information, Meteorological Information and Legal Information Sectors, MICUS Management Consulting GmbH for the European Commission, 2008 [available at:

  https://ec.europa.eu/digital-single-market/news/assessment-re-use-public-sector-information-psi-geographical-information-meteorological

  ]

-Funka et al. for the European Commission, Study Measuring Progress of eAccessibility in Europe – MeAC, 2013 [available at:

  http://www.funka.com/en/our-assignments/position-of-trust/archive---commission-of-trust/measuring-progress-of-eaccessibility-in-europe---meac2/

  ]

-GEN6 - Governments ENabled with IPv6 [

  http://www.gen6.eu/home

  ]

-Geopunt, the Flemish Geoportal [

  http://inspire.ec.europa.eu/events/conferences/inspire_2013/pdfs/26-06-2013_ROOM-4_09.00%20-%2010.30_95-Joeri%20Robbrecht_Joeri-Robbrecht.pdf

  ]

-KPMG for the European Commission; Gatti, R. Carbone, L. Mezzapesa,V, Study State of play of interoperability in Europe – Report 2014 [available at: 

  http://ec.europa.eu/isa/documents/publications/2014-report-on-state-of-play-of-interoperability.pdf

  ]

-London Economics for the European Parliament study, A European Single Point of Contact, 2013; [available at:

  http://www.europarl.europa.eu/RegData/etudes/etudes/join/2013/507453/IPOL-IMCO_ET(2013)507453_EN.pdf

  ]

-Ministerial Declaration on eGovernment, Malmö, Sweden, on 18 November 2009 [available at:

  https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/ministerial-declaration-on-egovernment-malmo.pdf

  ]

-Open Data Barometer [http://www.opendatabarometer.org/]

-Open Government Partnership, Independent Progress Mechanism Reports [

  http://www.opengovpartnership.org/independent-reporting-mechanism

  ]

-Organisation for Economic Co-operation and Development (OECD)- Recommendation on Digital Government Strategies, 2014 [available at:

  http://www.oecd.org/gov/public-innovation/recommendation-on-digital-government-strategies.htm

  ]

-Sixth European Ministerial eGovernment Conference “Borderless eGovernment Services for Europeans”, Conference proceedings, November 2011 [available at:

  https://mac.gov.pl/files/wp-content/uploads/2011/12/6th-BeGSE-folder-A4-CP-EN.pdf

  ]

-STORK2 project [

  www.eid-stork2.eu

  ]

-Tech4i2 and Deloitte for the European Commission, Study on collaborative production in eGovernment, 2012 [available at:

  https://ec.europa.eu/digital-single-market/news/study-collaborative-production-egovernment-smart-20100075

  ]

-TNS Opinion & Social for the European Commission- Flash Eurobarometer 365, European Union citizenship, 2013 [available at:  

  http://ec.europa.eu/public_opinion/flash/fl_365_en.pdf

  ]

-Tumin, Z, Fung, A. From Government 2.0 to Society 2.0: Pathways to Engagement, Collaboration and Transformation, October 2011 [available at:

  http://belfercenter.ksg.harvard.edu/publication/21475/from_government_20_to_society_20.html

  ]

-UK Environment Agency, Reducing environmental risk through INSPIRE (quantified benefits), 2013 [available at: http://www.poweredbyinspire.eu/documents/0403-sustainability-carlyle.pdf]

-UK Location Programme - Benefits Realisation Strategy Final, 2012 [available at:

  https://data.gov.uk/sites/default/files/Benefits%20Realisation%20Strategy%20v2.0%20Final_10.pdf

  ]

-Vickery, G. Review of recent studies on PSI re-use and related market developments, Information Economics for the European Commission, 2011 [available at:

  https://ec.europa.eu/digital-single-market/news/review-recent-studies-psi-reuse-and-related-market-developments

  ]

-Vickery,G. Luukkonen,T. Radosevic,S and Fisher, R, for the European Commission, CIP ICT PSP Final (Second Interim) Evaluation-Final report, 2011 [available at:

  http://ec.europa.eu/cip/files/cip/docs/cip_ict_psp_interim_evaluation_report_2011_en.pdf

  ]

-World Wide Web Foundation, Open data barometer Second Edition, January 2015 [

  http://www.opendatabarometer.org/

  ]

Legislation:

-Decision (EU) 2015/2240 Of the European Parliament and of the Council of 25 November 2015 establishing a programme on interoperability solutions and common frameworks for European public administrations, businesses and citizens (ISA2 programme) as a means for modernising the public sector

-Directive 2004/18/EC of the European Parliament and the Council of 31 March 2004 on the coordination of procedures for the award of public works contracts, public supply contracts and public service contracts

-Directive 2007/2/EC Of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE)

-Directive 2012/17/EU Of the European Parliament and of the Council of 13 June 2012 amending Council Directive 89/666/EEC and Directives 2005/56/EC and 2009/101/EC of the European Parliament and of the Council as regards the interconnection of central, commercial and companies registers.

-Directive 2014/25/EU of the European Parliament and the Council of 26 February 2014 on procurement by entities operating in the water, energy, transport and postal services sectors

-European Commission, Decision of 12 December 2011 on the reuse of Commission documents [2011/833/EU]

-Regulation(EU) No 283/2014 of the European Parliament and of the Council of 11 March 2014 on guidelines for trans-European networks in the area of telecommunications infrastructure and repealing Decision No 1336/97/EC

-Regulation (EU) No 910/2014 of the European Parliament and of the Council of 23 July 2014 on electronic identification and trust services for electronic transactions in the internal market and repealing Directive 1999/93/EC

COM(2010) 743

https://ec.europa.eu/digital-agenda/sites/digital-agenda/files/ministerial-declaration-on-egovernment-malmo.pdf

http://ec.europa.eu/smart-regulation/guidelines/ug_chap1_en.htm

COM(2015) 0192

COM(2010) 245

COM(2010) 2020

COM(2010) 245

COM(2006) 173 final.

COM(2010) 744

COM(2010) 744 final, Annex 2

COM(2012) 0529 final

http://ec.europa.eu/cip/ict-psp/index_en.htm

  and https://ec.europa.eu/digital-single-market/node/77121

https://ec.europa.eu/digital-single-market/towards-cloud-public-services

https://ec.europa.eu/digital-single-market/connecting-europe-facility

https://ec.europa.eu/digital-single-market/ict-enabled-public-sector-innovation-horizon-2020

http://ec.europa.eu/isa/

Before 2014 "European Structural funds"

http://ec.europa.eu/regional_policy/en/policy/what/glossary/t/thematic-objectives

COM(2011) 877 final.

Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of public sector information, OJ L 175/1.

European Commission, Decision of 12 December 2011 on the re-use of Commission documents,, repealing Decision 2006/291/EC, Euratom.

Directive on procurement in the water, energy, transport and postal services sectors; and Directive on public works, supply and service contracts

http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

Regulation (EU) No 283/2014..

http://ec.europa.eu/justice/data-protection/

COM(2012) 721 final

Decision (EU) 2015/2240

SEC(2010) 1539 final

Source: Smarter, Faster, Better eGovernment: 8th Benchmark Measurement, 2009.

Basic Services refer to the 20 services (12 for citizens, 8 for businesses) used to benchmark online availability of public services. These are: income taxes, job search, social security benefits, personal documents, car registration, building permission, declaration to police, public libraries, certificates, enrolment in higher education, announcement of moving, health-related services (citizens), social contributions, corporate tax, VAT, company registration, statistical data, customs declaration, environment-related permits, public procurement (businesses).

Full online availability is an indicator which measures whether the service is delivered in a completely electronic way without need of interacting through traditional (i.e. paper, face-to-face) channels. It corresponds to level 4 and above of the sophistication indicator introduced below. The composite indicator is an average of the values taken in the 20 services.

The small decrease in 2004 was due to an enlargement of the sample to New Member States: until 2003 the sample included only EU15 countries.

Eurostat. E-government usage by individuals.

Eurostat. Individuals using the internet for interaction with public authorities and Digital Agenda Scoreboard report 2011. Pillar 7

Digital Agenda Scoreboard report 2012.

SEC(2010) 1539

Archived site: http://ec.europa.eu/information_society/activities/ict_psp/index_en.htm

Pan-European Public Procurement OnLine

Secure idenTity acrOss boRders LinKed

Simple Procedures Online for Cross-border Services

This figure is based on the total estimated eligible costs of STORK, PEPPOL and SPOCS for the entire duration of the projects including the share of co-funding of participating Member States and other partners.

Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark measurement. 2010.

Mid-term Evaluation of the e-Government Action Plan 2011–2015. 2014.

NIFO eGovernment factsheets (edition 2015 published in 2016).

http://ec.europa.eu/digital-agenda/en/digital-economy-and-society-index-desi

https://ec.europa.eu/digital-agenda/download-scoreboard-reports

eGovernment Benchmark Framework 2012 – 2015. Method paper, July 2012.

Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmark report, 2015.

The Mystery Shopping had consultants simulate the user journey of citizens dealing with the public administration in order to get some services. The tested services (both for services and enterprises) were seven different families of services grouped around a specific user need (e.g. services around starting a business) embodied by the concept of life event. The User Survey was conducted on an online panel of 26000 citizens across 32 countries, and participants were asked about their level of interaction with public administrations both online and offline and about their degree of satisfaction with the online channel. The User Survey has been conducted only in 2012, while the Mystery Shopping took place in all the years, but alternating the measurement of half of life events each year.

(50)


  Digital Agenda Europe baseline: in 2009, 38% of individuals aged 16 – 74 had used eGovernment services in the last 12 months.

Digital Agenda Targets Progress report. 2015.

Eurostat. Individuals using the internet for interaction with public authorities.

Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark measurement. 2010; Digital Agenda Scoreboard Report 2012.

Eurostat: "Enterprises using the internet for interacting with public authorities".

https://ec.europa.eu/digital-single-market/pillar-vii-ict-enabled-benefits-eu-society/action-91-member-states-agree-common-list-key-cross

Sixth European Ministerial eGovernment Conference “Borderless eGovernment Services for Europeans”, Conference proceedings, November 2011.

ICT Policy Support Programme (ICT PSP) was one of three specific programmes of the Competitiveness and Innovation Framework Programme (CIP).

European Patients Smart Open Services project

E-Justice Communication via Online Data Exchange project

Electronic Simple European Networked Services

Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013.

Regulation (EU) No 283/2014..

EC, Joinup, Connecting Europe Facility, Catalogue of Building Blocks, 2014.

CEF Monitoring report Q4/2015.

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmark report, 2015

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmark report, 2015. Background report, p. 32.

http://www.egovap-evaluation.eu/

Digital Agenda Scoreboard Report 2012.

As reported by Eurostat on data "Enterprises using the internet for interacting with public authorities"

Digital Agenda Targets Progress report, 2015.

Eurostat, Individuals using the internet for interaction with public authorities

Digital Agenda Scoreboard report 2011, Pillar 7.

CIP ICT PSP Final (Second Interim) Evaluation-Final report, 2011 and COM (201302.

Regulation (EU) No 910/2014

Mid-term evaluation of the eGovernment Action Plan 2011 - 2015 (p. 34).

https://joinup.ec.europa.eu/

COM(2012) 721 final

Study Measuring Progress of eAccessibility in Europe – MeAC, 2013

https://open-data.europa.eu

Open Data Maturity in Europe 2015 Insights into the European state of play, 2015

https://ec.europa.eu/digital-single-market/en/news/eu-implementation-g8-open-data-charter

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Insight report, p. 25

The data is collected through life event assessment by "Mystery Shoppers". More details of the method are in chapter 5.2.

http://www.epsiplatform.eu/content/european-psi-scoreboard

http://www.europeandataportal.eu/

Method used: "Mystery Shopping". This top level (compound) benchmark assesses the availability and usability of public eServices.

Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmarking report 2015, Background report, p. 23

Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmarking report 2015, Insight report, p.19.

COM(2013)2

The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' evaluation, 2013

The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015, p. 50

EEA Technical report No 17/2014

  . Joint EEA-JRC report.

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Background report. Pp. 46 - 47

For example, the ISA programme activity on Common Infrastructure for Public Administrations Sustainability

http://ec.europa.eu/digital-agenda/en/news/eu-egovernment-report-2015-shows-online-public-services-europe-are-smart-could-be-smarter

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Insight report, p. 25.

Mid-term evaluation of the eGovernment Action Plan 2011 – 2015, 2014, p. 41

The study extrapolated that the extension of the Danish approach to implement the “once-only” principle is likely to generate an annual net saving at the EU 28 level, amounting to around € 5 billion per year by 2017. This highly positive impact depends on a complex system of registries being freely accessible by users (citizens and businesses) for commercial purposes, which additionally might foster growth in some economic sectors.

European Council, Conclusions, 24/25 October 2013

H2020 Work Programme 2016 – 2017, Europe in a changing world – inclusive, innovative and reflective societies. Co-creation-05-2016: Co-creation between public administrations: once-only principle.

COM(2015)215

EEA Technical report No 17/2014

  . Joint EEA-JRC report.

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Background report. Insight report, p. 30

Mid-term Evaluation of the e-Government Action Plan 2011–2015, pp. 48 – 49.

http://ec.europa.eu/cip/ict-psp/index_en.htm

  and https://ec.europa.eu/digital-single-market/en/node/77121

https://ec.europa.eu/digital-single-market/en/connecting-europe-facility

https://ec.europa.eu/digital-single-market/en/ict-enabled-public-sector-innovation-horizon-2020

http://ec.europa.eu/isa/

European Regional Development Fund, Cohesion Fund, European Social Fund, Youth Employment Initiative and European Agricultural Fund for Rural Development

https://ec.europa.eu/digital-agenda/en/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century

http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm

COM(2013)2

Final Evaluation of the Competitiveness and Innovation Framework Programme , 2011

Executive Agency on Competitiveness and Innovation, as of 2014, Executive Agency for SMEs (EASME)

Tool available at

  http://ec.europa.eu/regional_policy/en/funding/

Annual Growth Survey 2013, COM(2012) 750 final

DG Internal Market and Services: Management Plan 2013, 2012

The economics of identity, June 2014

A strategy for e-procurement, COM(2012)0179 final;

  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0179:FIN:EN:PDF

End-to-end e-procurement to modernise public administration, COM(2013)0453 final

E-invoicing in public procurement: another step towards end-to-end e-procurement and e-government in Europe (Press Release, June 2013)

Study on eGovernment and the Reduction of Administrative Burden, 2014

Excellence in public administration for competitiveness in EU Member States, 2012

COM(2014) 902 and COM(2015) 690

A Digital Single Market Strategy for Europe – Analysis and Evidence, SWD(2015) 100

Study on eGovernment and the Reduction of Administrative Burden, 2014

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Insight report.

The forthcoming EU Justice Scoreboard 2016.

Flash Eurobarometer 365, European Union citizenship, 2013

Directive (2013/37/EU), article 13.1.

Review of recent studies on PSI re-use and related market developments, 2011

Assessment of the Re-use of Public Sector Information (PSI) in the Geographical information, Meteorological Information and Legal Information Sectors, 2008

UK Location Programme - Benefits Realisation Strategy Final, 2012 - estimates the quantifiable INSPIRE benefits at 70-130 m£/year to UK environmental protection activities and the quantifiable benefits across UK government departments at 470-510 m£/year.

Costs-benefits analysis INSPIRE in The Netherland, 2009

Reducing environmental risk through INSPIRE, UK Environment Agency, 2013

The use of spatial Data for the Preparation of Environmental Reports in Europe, 2010

  EUR24327 EN - 2010

For example: 57% of INSPIRE data is OPEN in Finland and served through INSPIRE services (INSPIRE, MIG, National Implementation Seminars); Ashfield District Council (INSPIRE and Open Data, 2015) publishes data under the European INSPIRE Directive and as Open Data.

COM(2011) 882 final

For example: INSPIRED eGovernment Apps - Geopunt, the Flemish Geoportal.

http://open-data.europa.eu/

http://

  www.europeandataportal.eu

Creating Value through Open Data: Study on the Impact of Re-use of Public Data Resources, 2015

http://ec.europa.eu/justice/data-protection/reform/index_en.htm

Eurostat, Migration and migrant population statistics, 2015

Eurostat, Foreign citizens living in EU Member States, 2015

Study on Evaluation of the impact of the free movement of EU citizens at local level, 2014

Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013, p. 45 and figure 26.

COM(2013) 453 final

http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm

SWD (2015) 100 final

SWD(2014) 124 final, p. 27 and examples of differing national reforms in company law p. 21 et seq.

COM(2015)550, p. 5.

The European e-Justice Portal is available at: https://e-justice.europa.eu/home.do

In line with Directive 2012/17/EU.

European Parliament study, A European Single Point of Contact, 2013; European Commission, High Level Group on Business Services - Final Report, 2014

SWD(2012) 148 final

Study on The functioning and usability of the Points of Single Contact under the Services Directive, 2012

http://ec.europa.eu/internal_market/services/docs/services-dir/implementation/report/COM_2012_261_en.pdf

http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf

The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015

Delivering the European Advantage? How European governments can and should benefit from innovative public services, 11th eGovernment Benchmark Report, 2014, p. 49

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015, Background report, p. 35

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015

Eurostat, ICT survey of Households and Individuals, 2014; European Commission, Digital Agenda Scoreboard – the indicator has a 12-month reference period and concerns the use of web applications for uploading completed forms or transmitting web forms with details filled in directly.

EC, Digital Economy and Society Index DESI, February 2016

Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015, p. 35.

European Parliament resolution of 19 January 2016 on Towards a Digital Single Market Act

Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013

COM(2010) 744 final

The Interoperability Solutions for European Public Administrations (ISA) programme (2010-2015) monitored and supported the EIF implementation in Europe; it was followed by the ISA2 programme

Digital Single Market Communication 2015. Staff Working Document, SWD (2015) 100 final.

Guidance on Ex ante Conditionalities for the European Structural and Investment Funds, 2014

http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

Digital Economy and Society Index DESI, 2016. Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. Share of the steps in a public services life event that can be completed online, seven life events.

Mid-term Evaluation of the e-Government Action Plan 2011–2015. 2014.

NIFO eGovernment factsheets (edition 2015 published in 2016).

http://ec.europa.eu/digital-agenda/en/digital-economy-and-society-index-desi

https://ec.europa.eu/digital-agenda/download-scoreboard-reports

See Annex 2, report annex 1 A, section 1

The evaluation takes into account the inherent limitations of the findings of public consultations. Firstly as in all surveys, the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who has a stake in this domain. Secondly, stakeholders views convey an individual rather than a more general and representative perspective.

See Annex 2, report annex 1 A, section 2

See Annex 2, report annex 1 A, section 3

See Annex 2, report annex 1A, section 4

See Annex 2, report annex 1A, section 5

See Annex 2, report annex 1A, section 6

See Annex 2, report annex 1A, section 7

For the full question see Annex 2, report annex 1B

See Annex 2, report annex 1 A, section 8

This action is defined in the Digital Agenda for Europe

This action is defined in the Digital Agenda for Europe

This action is defined in the Digital Agenda for Europe.

This action is defined in the Digital Agenda for Europe.

This action is defined in the Digital Agenda for Europe.

This action is defined in the Digital Agenda for Europe.

This action is defined in the Digital Agenda for Europe.

This action is defined in the Digital Agenda for Europe.

Mid-term Evaluation of the e-Government Action Plan 2011–2015, 2014.

https://joinup.ec.europa.eu/

COM(2012) 721 final

Study Measuring Progress of eAccessibility in Europe – MeAC, 2013

Study on collaborative production in eGovernment, 2012

In many cases using the PSI Scoreboard Indicators

Pricing Of Public Sector Information Study on Models of Supply and Charging for Public Sector Information 2011

http://www.europeandataportal.eu/sites/default/files/edp_creating_value_through_open_data_0.pdf

http://epsiplatform.eu/

European Commission, Decision of 12 December 2011 on the reuse of Commission documents [2011/833/EU]

COM(2011) 882 final

http://open-data.europa.eu/

http://

  www.europeandataportal.eu

http://www.opengovpartnership.org/independent-reporting-mechanism

https://ec.europa.eu/digital-agenda/en/news/workshop-improving-transparency-government-decision-making-6112012-results-presentations

http://ec.europa.eu/citizens-initiative/public/welcome

https://ec.europa.eu/digital-agenda/en/eparticipation

http://www.peppol.eu

http://www.eu-spocs.eu

The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' evaluation, 2013

http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm

http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm

http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm

Mid-term Evaluation of the e-Government Action Plan 2011–2015, 2014, p. 36 and country-specific details at

  http://www.egovap-evaluation.eu

Study on The functioning and usability of the Points of Single Contact under the Services Directive, 2012

COM(2012) 261

http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf

The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015

http://www.e-codex.eu

Note: the progress under the sub-priority on EU-wide implementation of cross-border services is linked to the general objective of the Action Plan on a number of key cross-border services to be offered online by 2015, see above at 5.2.

Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013

http://www.e-codex.eu

The topic was included in the H2020 WP 2016-2017 that was adopted in 2015.

More information at

  http://inspire.ec.europa.eu/

  and Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE). The INSPIRE Directive entered into force on the 15th May 2007

ISA Action 1.17

  Improving the cross-border exchange of spatial information

  : A Reusable INSPIRE Reference Platform.

Providing INSPIRE-compliant access to utility services: the case of sewage networks in Flanders, Belgium

Dutch cadastre INSPIRE conform

  ,

  The German Marine Data Infrastructure

  ,

  UK National Biodiversity Network to deliver INSPIRE compliance for species data

  ,

  Geodateninfrastruktur Deutschland: INSPIRE success story – Implementing e-reporting of air quality based on INSPIRE at national level

One-Geology Europe

  , European Location Framework,

  Reporting and exchanging air quality information using e-Reporting

ISA ACTION 2.13

  European Union Location Framework (EULF)

EEA Technical report No 17/2014

  . Joint EEA-JRC report.

Study on eGovernment and the Reduction of Administrative Burden, 2014

https://joinup.ec.europa.eu/

e-Commission 2012 – 2015. SEC(2012) 492.

http://ec.europa.eu/dgs/informatics/supplier_portal/index_en.htm

  - link to Supplier portal requires ECAS login.

Study on eGovernment and the Reduction of Administrative Burden, 2014

European Council, Conclusions, 24/25 October 2013

Horizon 2020 Work Programme 2016 – 2017, Europe in a changing world – inclusive, innovative and reflective Societies

http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-3action_en.htm

  and NIFO JoinUp community https://joinup.ec.europa.eu/community/nifo/home

http://ec.europa.eu/isa/

State of play of interoperability in Europe – Report 2014

https://joinup.ec.europa.eu/community/nifo/og_page/nifo-factsheets

Regulation (EU) No 910/2014. On eIDAS, see http://ec.europa.eu/digital-agenda/en/trust-services-and-eid

www.eid-stork2.eu

https://joinup.ec.europa.eu/community/cef/og_page/catalogue-building-blocks

CEF Monitoring Report Q4/2015

Study on cloud and service oriented architectures for eGovernment, Final report, 2011

A vision for public services, 2013

http://www.gen6.eu/home

https://ec.europa.eu/digital-agenda/en/towards-cloud-public-services