COMMISSION STAFF WORKING DOCUMENT Implementation and Evaluation report Accompanying the document Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions EU eGovernment Action Plan 2016 - 2020 Accelerating the digital transformation of government
Date: 2016-04-19
Institutions: Directorate-General for Communications Networks, Content and Technology, European Commission
Subjects: action programme, administrative formalities, data protection, data transmission, digital technology, electronic government, information technology, information technology applications, single market

EN 

   EN 

   EUROPEAN COMMISSION  Brussels, 19.4.2016  SWD(2016) 108 final   COMMISSION STAFF WORKING DOCUMENT Implementation and Evaluation report Accompanying the document Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions EU eGovernment Action Plan 2016 - 2020 Accelerating the digital transformation of government {COM(2016) 179 final} {SWD(2016) 109 final}  
 
 
TABLE OF CONTENTS 

1. 

INTRODUCTION ....................................................................................................... 4 

1.1.  The purpose of the evaluation ........................................................................... 4 
1.2.  The scope of the evaluation ............................................................................... 4 

2.  BACKGROUND TO THE EU EGOVERNMENT ACTION PLAN 2011 - 2015 .... 6 

2.1.  Description of the European eGovernment Action plan 2011 – 2015 and its 

2.2. 

objectives ........................................................................................................... 6 
Instruments ........................................................................................................ 8 
2.2.1.  Political  Activities 

including  exchange  of  experience  and 

Commission Communications ............................................................ 8 
2.2.2.  Funding for eGovernment ................................................................... 9 
2.2.3.  Legislation ......................................................................................... 10 
2.3. 
Intervention logic ............................................................................................ 11 
2.4.  Situation in 2010/Baseline .............................................................................. 12 

2.4.1.  Baseline  of  target:  50%  of  citizens  (Digital  Agenda)  and  80%  of 

businesses make use of eGovernment services ................................. 13 

2.4.2.  Baseline  of  target:  a  number  of  key  cross-border  services  be 

offered online by 2015 (Digital Agenda) .......................................... 13 
2.4.3.  Baselines in the four policy priority areas ......................................... 13 

3.  EVALUATION QUESTIONS/CRITERIA .............................................................. 15 

4.  METHOD AND SOURCES ..................................................................................... 16 

4.1.  Mid-term evaluation ........................................................................................ 16 
4.2.  Annual benchmarking reports ......................................................................... 16 
4.3.  The online public consultation ........................................................................ 17 
4.4.  Limitations ...................................................................................................... 18 

5. 

IMPLEMENTATION OF THE EU EGOVERNMENT  ACTION  PLAN 2011  – 
2015 – STATE OF PLAY OF THE POLICY PRIORITIES .................................... 19 

5.1.  General  target:  50%  of  citizens  (Digital  Agenda)  and  80%  of  businesses 

make use of eGovernment services ................................................................. 19 

5.2.  General target: a number of key cross-border services be offered online by 

2015 (Digital Agenda) ..................................................................................... 19 
5.3.  The four priorities and 40 actions of the EU eGovernment Action Plan ........ 22 
5.4.  Governance ...................................................................................................... 22 

6.  ANSWERS TO THE EVALUATION QUESTIONS .............................................. 24 

6.1.  Effectiveness ................................................................................................... 24 
6.1.1.  Objective on using eGovernment (take-up) ...................................... 24 
6.1.2.  Key cross-border services online ...................................................... 25 
6.1.3.  Effectiveness of the four policy priorities ......................................... 26 
6.2.  Efficiency ........................................................................................................ 36 
6.2.1.  Efficiency of the related budget ........................................................ 36 
6.2.2.  Costs and benefits to stakeholders..................................................... 38 

2 

 
 
6.3.  Relevance ........................................................................................................ 39 
6.3.1.  Relevance: User Empowerment ........................................................ 40 
6.3.2.  Relevance: Internal market ................................................................ 42 
6.3.3.  Relevance:  "Efficiency  and  Effectiveness  of  Governments  and 

Administrations" ................................................................................ 45 
6.3.4.  Relevance: Pre-conditions ................................................................. 48 
6.4.  Coherence ........................................................................................................ 49 
6.5.  EU-added value ............................................................................................... 51 

7.  CONCLUSIONS ....................................................................................................... 54 

8.  ANNEXES ................................................................................................................ 57 

Annex 1. 
Annex 2. 
Annex 3. 

2015 

Annex 4. 

Procedural information ...................................................................... 57 
The Online public consultation ......................................................... 59 
The  individual  actions  of  the  eGovernment  Action  Plan  2011  - 
89 
Implementation  of  the  individual  actions  of  the  eGovernment 

Action Plan 2011 – 2015 – State of play of the actions .................................. 93 
Priority: User Empowerment ....................................................................... 93 
Priority: Internal Market ............................................................................. 96 
Priority:  Efficiency  and  Effectiveness  of  Governments  and 

Administrations ............................................................................... 98 
Priority: Pre-conditions for developing eGovernment ............................ 100 
Acronyms ........................................................................................ 103 
REFERENCES AND DATA SOURCES ....................................... 105 

Annex 5. 
Annex 6. 

3 

 
 
 
 
 
1. 

INTRODUCTION 

1.1.  The purpose of the evaluation  

In December 2015, European Commission's Directorate-General Communications Networks, 
Content  and  Technology  launched  an  implementation  and  evaluation  assessment  of  the 
European  eGovernment  Action  Plan  2011  –  20151  ("EU  eGovernment  Action  Plan  2011  – 
2015"),  adopted  on  15th  December  2010.  The  purpose  of  this  evaluation  is  to  describe  the 
progress of implementation of individual actions of the Action Plan and assess its functioning. 
In the broadest sense, this evaluation exercise will provide answers to how well the European 
public administrations have progressed towards meeting the Malmö vision (November 2009 
Malmö Ministerial declaration2), which is included in this eGovernment Action Plan:  

"by  2015  European  public  administrations  will  be  recognised  for  being  open,  flexible  and 
collaborative  in  their  relations  with  citizens  and  businesses.  They  use  eGovernment  to 
increase their efficiency and effectiveness and to constantly improve public services in a way 
that  caters  for  user's  different  needs  and  maximises  public  value,  thus  supporting  the 
transition of Europe to a leading knowledge-based economy." 

The  EU  eGovernment  Action  plan  2011-2015  did  not  indicate  the  need  to  carry  out  a  final 
evaluation as such. According to the eGovernment Action Plan, the overall progress would be 
measured  annually,  using  an  appropriate  mix  of  instruments  and  methods.  However, 
according  to  the  Commission's  Better  Regulation  guidelines,  retrospective  performance 
evaluations  support  policy  preparation3.  "Evaluations  gather  evidence  to  assess  how  well  a 
specific intervention has performed (or is working) - -. An evaluation also draws conclusions 
on whether the EU intervention continues to be justified or should be modified to improve its 
effectiveness, relevance and coherence or to eliminate excessive burdens or inconsistencies or 
simply  be  repealed."  This  evaluation  is  carried  out  in  so  far  as  possible  in  this  context,  in 
order to provide a set of lessons learnt to guide the work of the new eGovernment Action Plan 
2016 – 2020 announced in the Digital Single Market Strategy4.  

1.2.  The scope of the evaluation 

The  scope  of  the  evaluation  is  the  objectives  and  measures  set  out  in  the  EU  eGovernment 
Action Plan 2011 – 2015. It covers the level of progress in eGovernment policy in the EU in 
the period starting from the beginning of 2011 (Action Plan was adopted 15 December 2010) 
until and including 2015, subject to the availability of most recent data.  

The  Action Plan  2011  –  2015  aimed  to  contribute  to  achieving  two  important  targets  of  the 
Digital  Agenda  in  Europe5:  1)  "By  2015,  50%  of  EU  citizens  will  have  used  eGovernment 
services";  and  2)  offering  a  number  of  cross-border  services  online.  The  Action  Plan  also 
aimed that "by 2015, 80% of enterprises will have used eGovernment". 

Furthermore, the evaluation focuses on the four policy priorities (agreed by Commission  and 
Member  States  in  the  Malmö  Ministerial  Declaration)  of  the  EU  eGovernment  Action  Plan 
2011 – 2015: 

1 COM(2010) 743 
2 https://ec.europa.eu/digital-agenda/sites/digital-agenda/files/ministerial-declaration-on-egovernment-
malmo.pdf 
3 http://ec.europa.eu/smart-regulation/guidelines/ug_chap1_en.htm 
4 COM(2015) 0192   
5 COM(2010) 245 

4 

 
 
 
                                                 
1.  User Empowerment 
2.  Internal Market 
3.  Efficiency and Effectiveness of Governments and Administrations 
4.  Pre-conditions for developing eGovernment 

The Action Plan included 40 individual actions (listed in Annex 4) divided amongst these four 
priorities. Five additional actions were related to the governance.  

The evaluation focuses on the Action Plan as a political instrument to achieve its objectives. It 
does not evaluate in detail individual actions identified in the Action Plan. An assessment of 
the completion of actions will be done separately via a study planned for 2016. The results of 
the study will be used to update to complete this evaluation. Furthermore, since the scope of 
the evaluation is the policy priorities eGovernment Action Plan 2011 – 2015, it will not cover 
eGovernment related activities in Member States at all levels of government and all sectors.  

For the evaluation of the eGovernment policy priorities in the EU it is important to note that 
eGovernment  has  also  been  supported  through  a  number  of  funding  programmes  (listed  in 
section  2.2.2).  The  evaluation  of  these  funding  programmes  falls  outside  the  scope  of  this 
evaluation  but  their  findings  are  referred  to  in  this  evaluation  when  relevant  to  address  the 
evaluation criteria.  

The evaluation criteria used are in line with the Commission's Better Regulation Guidelines 
and assess the effectiveness, efficiency, relevance, coherence and the EU added value of the 
EU eGovernment Action Plan 2011 - 2015. 

5 

 
 
 
 
 
 
2.  BACKGROUND TO THE EU EGOVERNMENT ACTION PLAN 2011 - 2015 

2.1.  Description  of  the  European  eGovernment  Action  plan  2011  –  2015  and  its 

objectives 

The Europe 2020 strategy that was put forward by the Commission on 3 March 20106 set out a 
strategy for smart, sustainable and inclusive growth to face the immediate challenges resulting 
from  the  financial  crisis  and  long-term  challenges  such  as  resource  scarcity,  ageing  and 
globalisation. One of the so-called seven flagship initiatives, the Digital Agenda for Europe7, 
outlined a number of actions supporting the further development of eGovernment in the EU. 
The  actions  can  be  found  from  the  eGovernment  chapter  of  the  Digital  Agenda 
Communication and from the eGovernment Action Plan that was the successor of the i2010 
Action Plan 2006-20108. 

The  Action Plan  2011  –  2015  aimed  to  contribute  to  achieving  two  important  targets  of  the 
Digital Agenda in Europe: 1) that 50% of citizens make use of eGovernment services; and 
2) that a number of key cross-border services will be available online by 2015. The Action 
Plan  also  aimed  that  "by  2015,  80%  of  enterprises  will  have  used  eGovernment". 
Furthermore, the objective was to translate into concrete measures the four political priorities 
of the Malmö Ministerial Declaration on eGovernment of December 2009. Subsequently, the 
eGovernment Action Plan had the following four policy priorities (see also Figure 1): 

"1. 

User Empowerment 

Empowerment  means  increasing  the  capacity  of  citizens,  businesses  and  other 
organisations  to  be  pro-active  in  society  through  the  use  of  new  technological  tools. 
Public  services  can  gain  in  efficiency  and  users  in  satisfaction  by  meeting  the 
expectations of users better and being designed around their needs and in collaboration 
with  them  and  third  parties  whenever  possible.  Empowerment  also  means  that 
governments should provide easy access to public information, improve transparency 
and  allow  effective  involvement  of  citizens  and  businesses  in  the  policy-making 
process.  

2. 

Internal Market 

Most  public  online  services  do  not  work  across  borders  or  involve  cumbersome 
procedures  to  be  accessible.  People  from  one  EU  country  cannot  easily  apply  for 
public services in another country than the one in which they are established, using for 
instance  national  electronic  identity  cards.  This  seriously  reduces  the  mobility  of 
businesses and citizens. To support the Internal Market, governments should develop 
‘seamless’ services for entrepreneurs to set up and run a business anywhere in Europe 
and  allowing  individuals  to  study,  work,  reside,  receive  health  care  and  retire 
anywhere in the European Union.  

3. 

Efficiency and Effectiveness of Governments and Administrations 

6 COM(2010) 2020 
7 COM(2010) 245 
8 COM(2006) 173 final. 

6 

 
 
 
 
                                                 
The  aim  was  to  use  ICT  and  enabling  organisational  changes  to  deliver  better,  less 
intrusive, more sustainable and faster public services, by reducing the administrative 
burden,  improving  organisational  processes  and  promoting  a  sustainable  low-carbon 
economy.  

4. 

Pre-conditions for developing eGovernment 

A number of technical and legal pre-conditions need to be put in place to enable the 
implementation  of  the  actions  that  will  enhance  eGovernment  services  in  Europe. 
These  include  the promotion of interoperability across  borders, which would allow  - 
among  others  -  sharing  of  information,  deployments  of  one-stop-shop  approaches, 
Europe  wide  use  of  (national)  electronic  identity  solutions  and  payment  schemes. 
Interoperability is supported through open specifications and the development of key 
enablers  such  as  electronic  identity  management  and  stimulation  of  innovation  in 
eGovernment." 

Figure 1 The four policy priorities of the eGovernment Action plan 2011 – 2015 and their related objectives. 

To achieve these objectives, the Action Plan proposed 40 individual actions (listed in Annex 4 
to  the  present  document)  distributed  under  each  of  the  policy  priorities.  According  to  the 
Action  Plan,  "the  actions  can  also  be  categorised  in  three  groups,  depending  on  the  actors 
involved and their competence defined in the Treaty:  

1.  Where  Member  States  are  leading  and  rely  on  their  own  resources,  the  Commission 
will help by supporting and coordinating activities. The measures proposed will focus 
on  setting  targets  with  the  Member  States  and  on  how  to  achieve  these  targets  by 
means  of  measures  such  as  exchanging  best  practice  and  information,  conducting 
studies and benchmarking. 

2.  Where  the  Commission  and  the  Member  States  work  jointly  to  develop,  deploy  or 
improve cross-border services, the Commission will take the lead in activities where 

7 

 
 
 
 
joint resources are used, while the Member States will bear the final responsibility for 
implementing  activities  using  their  own  resources.  The  measures  proposed  will 
include  research  and  development,  pilot  projects,  collaborative  development  of 
services by Member States and transfer of knowledge to the market.  

3.  Where  the  Commission  can  create  enabling  conditions,  the  measures  proposed  will 
include  adopting 
formulating  common 
frameworks,  implementing  generic  tools,  providing  (re-usable)  technical  building 
blocks and ensuring interoperability." 

instruments,  setting  standards, 

legal 

Of  the  40  actions  in  the  eGovernment  Action  Plan,  the  Commission  was  exclusively 
responsible  for  23  and  the  Member  States  for  10.  Seven  actions  were  under  shared 
responsibility between the Member States and the Commission. Five additional actions, two 
for the Commission  and three for the Member States, were included for the governance.  

2.2.  Instruments 

The  instruments  used  to  implement  the  Action  Plan  were  political,  financial  and  legislative 
activities. 

2.2.1.  Political  Activities 

including 

exchange  of 

experience  and  Commission 

Communications 

The political activities refer to dialogue with Member States, Ministerial meeting, exhibitions, 
awards and benchmarking. Their aim was to provide political steer, enhance mutual learning 
through exchange of experience and awareness rising. 

Dialogue with Member States 

eGovernment  High-Level  Expert  Group  of  Member  States  representatives  (activity 
in 
under  governance):  The  Commission  and  Member  States  have  engaged 
collaborating  and  in  regular  dialogue  during  the  eGovernment  Action  Plan  2011  – 
2015 through the eGovernment High-Level Expert Group. In addition, there have been 
other  fora  to  engage  with  Member  States'  representatives:  including  the  ISA 
(Interoperability  Solutions  for  European  Public  Administrations)  Programme 
Management Committee.  

Ministerial meetings 

Ministerial  Conferences  (together  with  a  series  of  eGovernment  Presidency 
events/conferences)  helped  to  take  stock  of  where  eGovernment  was  and  provide  a 
vision for where it should  go in  the coming  years, mainly  in the form of Ministerial 
Declarations.  The  2009  Malmö  Ministerial  Declaration  resulted  in  the  current 
eGovernment Action Plan. The conferences also provided exhibitions of projects and 
awards  for  best  eGovernment  projects.  The  6th  Ministerial  meeting  was  held  on 
November 2011 in Poznan. As well as the conference and the exhibition, it featured an 
Informal Council of Ministers meeting. 

Benchmarking 

8 

 
 
 
 
 
 
 
 
 
 
 
Since 2001, the EC has been measuring the progress made in on-line public services. 
This  annual  benchmarking  has  become  a  worldwide  reference  for  international 
comparison  of  the  development  of  eGovernment.  Since  2012  a  new  benchmarking 
framework has been in place, aligned to the four policy priorities of the current Action 
Plan.  

The European Interoperability Framework and the European Interoperability Strategy 

The  European  Interoperability  Strategy9  (EIS)  and  a  European  Interoperability 
Framework10 (EIF) provide guidance in delivering services across borders and sectors. 
The EIF, in particular, has been translated into National Interoperability Frameworks 
in more than 20 Member States. It is now being reviewed and extended in the context 
of the Digital Single Market Strategy. 

Cloud computing 

The  Commission  Communication  "Unleashing  the  Potential  of  Cloud  Computing  in 
Europe"11, provided a European strategy designed to speed up and increase the use of 
cloud computing across all economic sectors. 

ICT standardisation 

A  number  of  ICT  standardisation  activities  supported  the  EU  policy  activities, 
including the areas of eGovernment, eProcurement, Public Sector  Information, Open 
Data and Big Data. 

2.2.2.  Funding for eGovernment  

The  Action  Plan  did  not  have  funding,  but  served  as  a  guide  to  prioritise  a  number  of 
investments from EU funding programmes: 

  The  past  Competitiveness  and  Innovation  Framework  Programme  ICT  Policy  Support 
Programme12 (CIP ICT-PSP) has supported (some projects on-going) piloting of cross-
border  eGovernment  solutions  and  supported  piloting  of  concepts,  like  the  "cloud  of 
public services"13; 

  Under  the  Connecting  Europe  Facility  (CEF),  Digital  Services  Infrastructures  (CEF 

DSI)14 deploy digital public services that work across borders; 

  Horizon  2020's  Societal  Challenge  6  'Europe  in  a  changing  world  –  inclusive, 
innovative  and  reflective  societies'  funds  research  and  innovation  actions  on  ICT-
enabled public sector innovation15; 

9 COM(2010) 744 
10 COM(2010) 744 final, Annex 2 
11 COM(2012) 0529 final 
12 http://ec.europa.eu/cip/ict-psp/index_en.htm and  https://ec.europa.eu/digital-single-market/node/77121 
13 https://ec.europa.eu/digital-single-market/towards-cloud-public-services 
14  https://ec.europa.eu/digital-single-market/connecting-europe-facility 
15 https://ec.europa.eu/digital-single-market/ict-enabled-public-sector-innovation-horizon-2020 

9 

 
 
 
 
 
 
 
 
 
 
                                                 
  The  past  Seventh  EU  Framework  Programme  for  Research  and  Technological 

Development (FP7) supported eGovernment-related projects; 

  The ISA programme16 (Interoperability solutions for European Public Administrations) 
provides  a  framework  that  allows  Member  States  to  work  together  to  create  efficient 
and  effective  electronic  cross-border  public  services  for  the  benefit  of  citizens  and 
businesses;  

  The  European  Structural  and  Investment  Funds  (ESIF  as  of  201417)  provide 
investments  in  the  field  of  eGovernment  in  less  favoured  regions,  as  this  is  seen  as  a 
strategic  component  of  their  economic  and  social  development.  One  of  the  thematic 
priorities18  for  the  2014  –  2020  programming  period  is  enhancing  access,  use  and 
quality  of  ICT  (thematic  objective  2),  including  strengthening  ICT  applications  for  e-
government.  Another  thematic  objective  is  on  administrative  capacity  building 
(thematic objective 11).  

The  exact  amounts  invested  to  policy  priorities  of  the  Action  Plan  are  in  some  cases 
challenging  to  measure.  Sometimes  investments  were  done  directly  by  the  Member  States 
without  reporting  to  the  Commission.  Furthermore,  it  is  not  always  obvious  what  would  be 
considered  an  eGovernment–related  investment  in  programmes  with  various  information 
technology or information society projects.  

2.2.3.  Legislation 

Legislative instruments have been used to achieve the eGovernment Action Plan's objectives 
where needed. Some of these legislative instruments existed already in the beginning of the 
Action Plan and needed amendment, while new legislation was also required, these included:  

  The  planned  revision  of  the  PSI  (Public  Sector  Information)  Directive  was 
indicated  in  the  eGovernment  Action  Plan.  On  12  December  2011  presented  a 
proposal in  this  respect19.  The  revised  Directive was  adopted  on  26  June  2013.20 
The reuse policy of the Commission was revised by a Decision of 201121. 

  The revision of the eSignature Directive and the proposal for a Decision for mutual 
recognition  of  eIdentification  and  eAuthentication  were 
the 
eGovernment  Action  Plan  and  led  to  the  eIDAS  Regulation  on  electronic 
identification and trust services (Regulation 910/2014).  

indicated 

in 

  The update of the Public Procurement Directives presented by the EC in the end of 
2011  proposed  a  gradual  but  ambitious  transition  towards  e-procurement  in  the 
EU.  In  February  2014,  two  Public  Procurement  Directives22  were  revised  and  a 

16 http://ec.europa.eu/isa/ 
17 Before 2014 "European Structural funds" 
18 http://ec.europa.eu/regional_policy/en/policy/what/glossary/t/thematic-objectives 
19 COM(2011) 877 final.  
20 Directive 2013/37/EU amending Directive 2003/98/EC on the re-use of public sector information, OJ L 175/1. 
21 European Commission, Decision of 12 December 2011 on the re-use of Commission documents,, repealing 
Decision 2006/291/EC, Euratom. 
22 Directive on procurement in the water, energy, transport and postal services sectors; and Directive on public 
works, supply and service contracts 

10 

 
 
 
 
 
                                                 
Directive  on  concession  contracts  was  adopted23.  The  new  rules  will  enter  into 
force as of 18 April 2016.  

  The  CEF  regulation  in  the  area  of  telecommunications  infrastructure24  (CEF 

Telecom Regulation). 

  The  reform  of  the  EU  data  protection  legal  framework  –  the  General  Data 
Protection  Regulation  and  the  Data  Protection  Directive  for  Police  and  Criminal 
Justice Authorities. Both will be adopted in early 2016 and followed by a two year 
transition period25. 

  The proposal for a directive on the accessibility of public sector bodies' websites, 

which has not yet been adopted26.  

  The  new  ISA2  programme  adopted  by  the  co-legislators  in  November  2015  on 
interoperability between public administrations and extended to local and regional 
administrations and the links to businesses and citizens is a key instrument for the 
development of eGovernment in Europe.27 

2.3.  Intervention logic  

As  stated  in  the  Staff  Working  Document  of  2010  accompanying  the  eGovernment  Action 
Plan  2011  -  201528,  this  eGovernment  Action  Plan  2011  –  2015  responded  to  a  call  from 
Member  States,  made  in  the  Malmö  Declaration,  for  a  shared  eGovernment  policy  in  the 
European  Union  in  order  to  build  on  past  achievements  and  increase  collaboration  on 
eGovernment. This included joint action between Member States and close collaboration with 
the  European  Commission  The  simplified  intervention  logic  figure  (Figure  2)  explains  how 
the different parts of the Action Plan fit together.  

23 http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm  
24 Regulation (EU) No 283/2014.. 
25 http://ec.europa.eu/justice/data-protection/ 
26 COM(2012) 721 final 
27 Decision (EU) 2015/2240 
28 SEC(2010) 1539 final 

11 

 
 
 
                                                 
Figure 2 Simplified intervention logic. 

2.4.  Situation in 2010/Baseline 

The 2010 Staff Working Document, referred to above, provides an overview of the situation 
in 201029, focusing at the time on the availability of basic services online:  

"The  online  delivery  of  basic  services30  has  continued  to  increase  steadily  in  recent 
years:  their  full  online  availability31  went  from  21%  in  2001  to  71%  in  200932. 
However,  there  is  a  marked  difference  between  services  for  businesses  and  services 
for  citizens.  Services  for  businesses  reach  83%  availability  whereas  services  for 
citizens are still only at 63% availability." 

There  were  considerable  discrepancies  between  Member  States.  The  gap  between  the  best 
(with 100% availability of basic services) and worst performer (40% availability) was large. 
The EU average figure was around 70%. Four Member States had a full online availability for 
all the 20 basic services considered in 2009 in the benchmark measurement report. 

29 Source: Smarter, Faster, Better eGovernment: 8th Benchmark Measurement, 2009. 
30 Basic Services refer to the 20 services (12 for citizens, 8 for businesses) used to benchmark online availability 
of  public  services.  These  are:  income  taxes,  job  search,  social  security  benefits,  personal  documents,  car 
registration,  building  permission,  declaration  to  police,  public  libraries,  certificates,  enrolment  in  higher 
education, announcement of moving, health-related services (citizens), social contributions, corporate tax, VAT, 
company  registration,  statistical  data,  customs  declaration,  environment-related  permits,  public  procurement 
(businesses). 
31  Full  online  availability  is  an  indicator  which  measures  whether  the  service  is  delivered  in  a  completely 
electronic way without need of interacting through traditional (i.e. paper, face-to-face) channels. It corresponds 
to level 4 and above of the sophistication indicator introduced below. The composite indicator is an average of 
the values taken in the 20 services. 
32 The small decrease in 2004 was due to an enlargement of the sample to New Member States: until 2003 the 
sample included only EU15 countries. 

12 

 
 
 
                                                 
2.4.1.  Baseline of target: 50% of citizens (Digital Agenda) and 80% of businesses make use 

of eGovernment services  

The Eurostat figure of the percentage of Europeans (EU27) aged 16 to 74 who had used the 
Internet  in  the  last  3  months  for  interaction  with  public  authorities  was  31  %  in  201033.  If 
extending to cover the last 12 months, the Eurostat figure collected in 2010 was 41 %34.   

Regarding  advanced  modes  of  interacting  with  public  administrations,  the  Staff  Working 
Document of the 2011 – 2015 Action Plan noted that in 2009 only 17% of EU citizens had 
downloaded  official  forms  from  a  public  authority  website  and  only  12%  had  used  the 
Internet to send back completed forms.  

In  2010  76%  of  enterprises  used  at  least  one  eGovernment  service35.  Take-up  was  also 
relatively  high  for  advanced  ways  of  interacting  with  the  public  administrations:  54%  of 
enterprises  used  the  Internet  to  return  completed  forms  and  43%  interacted  with  the  public 
administration entirely using electronic transactions.36 

2.4.2.  Baseline  of  target:  a  number  of  key  cross-border  services be  offered  online  by  2015 

(Digital Agenda) 

The eGovernment Action Plan 2011 – 2015 Communication stated, describing the situation in 
2010, that cross-border eGovernment services are few and, even where eGovernment services 
were offered, the majority of EU citizens were reluctant to use them. (In the Digital Agenda 
Communication  in  2010  no  specific  baseline  was  given  for  the  target  of  "by  2015  online 
availability  of  all  the  key  cross-border  public  services  contained  in  the  list  to  be  agreed  by 
Member States by 2011".) 

In  2010  the  work  towards  cross-border  services  had  started  with  three  Large  Scale  Pilots 
(LSPs)  co-funded  by  the  EC  under  the  "ICT  Policy  Support  Programme"37  as  part  of  the 
Competitiveness  and  Innovation  Framework  Programme  (CIP  ICT  PSP).  PEPPOL38  had 
started in 2008 to work towards a European system of eProcurement; STORK39 had started in 
2008 and aimed to enable citizens and businesses to identify themselves when interacting with 
their own and other national administrations over the internet; SPOCS40 had started in 2009 
and  was  related  to  the  Services  Directive.  Around  36  million  Euros41  investment  had  been 
planned for in these three pilot projects at the time. 

2.4.3.  Baselines in the four policy priority areas 

Data from 2010 is not available for the four policy priority areas of the Action Plan, because 
the benchmarking measurement of the four priorities started only with the new method as of 
2012 with a first complete measurement available in 2013.  

33 Eurostat. E-government usage by individuals.  
34 Eurostat. Individuals using the internet for interaction with public authorities and Digital Agenda Scoreboard 
report 2011. Pillar 7 
35 Digital Agenda Scoreboard report 2012.  
36 SEC(2010) 1539  
37 Archived site: http://ec.europa.eu/information_society/activities/ict_psp/index_en.htm 
38 Pan-European Public Procurement OnLine 
39 Secure idenTity acrOss boRders LinKed 
40 Simple Procedures Online for Cross-border Services 
41  This  figure  is  based  on  the  total  estimated  eligible  costs  of  STORK,  PEPPOL  and  SPOCS  for  the  entire 
duration of the projects including the share of co-funding of participating Member States and other partners. 

13 

 
 
 
 
 
 
                                                 
Some  data  relevant  to  the  Action  Plan  2011  –  2015  were  given  in  the  eGovernment 
Benchmarking report of 201042 that reported that Europe had made substantial advances in the 
20 basic services. For businesses, a traditional 20 services assessment covered eight services 
for  businesses.  Out  of  these,  the  services  ‘Registration  of  a  company’  and  ‘Obtaining  an 
Environmental  Permit’  were  highly  available  online  and  displayed  sophistication  and  full 
online  availability  scores  of  90%/77%  and  78%/63%  for  the  EU27+  respectively. 
Furthermore,  the  eGovernment  Benchmarking  report  2011  measured  the  availability  of 
different enablers in the EU27+ countries (Figure 3 below) 

Figure  3  Frequency  of  enablers  in  EU27+.  Digitizing  Public  Services  in  Europe:  Putting  ambition  into  action.  9th 
Benchmark Measurement , 2010. 

42 Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark measurement. 2010. 

14 

 
 
 
 
 
                                                 
3.  EVALUATION QUESTIONS/CRITERIA 

In order to structure the evaluation exercise, the current analysis sets out to answer a series of 
questions that will provide insight into the effects produced by the EU eGovernment Action 
Plan 2011-2015. The evaluation criteria used are from the EC’s Better Regulation Guidelines: 
effectiveness, efficiency, coherence, relevance and EU added value  

Effectiveness  analysis  considers  how  successful  the  eGovernment  Action  Plan  has  been  in 
achieving  and  progressing  towards  achieving  the  targets  on  the  usage  of  eGovernment 
services and the availability of cross-border services, in addition to achieving the objectives of 
the four policy priorities on user empowerment, internal market, efficiency and effectiveness 
and pre-conditions. This criterion assesses the progress made and the role of the Action Plan 
in delivering the changes and/or the extent to which progress has fallen short of the target and 
what factors have influenced why something has not been achieved. Have users' needs been 
sufficiently catered for under the lifetime of the Action Plan?   

The EU eGovernment Action Plan 2011 – 2015 did not have funding, but it served as a guide 
to  prioritise  investment  from  a  number  of  funding  programmes.  How  does  the  development 
and  deployment  of  these  projects  support  the  overall  aim  of  the  eGovernment  Action  Plan? 
The efficiency analysis aims to, to the extent possible; consider the relationship between the 
resources used by the Action Plan and the changes generated by the funding intervention.  

Relevance  looks  at  the  relationship  between  the  needs  and  problems  in  society  and  the 
objectives of the EU eGovernment Action Plan: Was the vision in the Malmö declaration on 
which the eGovernment Action Plan is based appropriate for developing the EU eGovernment 
2011  -  2015?  Do  the  original  objectives  still  correspond  to  existing  needs  within  the  EU, 
which of the objectives of the Action Plan continue to be still relevant and identifies also some 
new or changed needs.  

The evaluation of coherence looks at how well different actions of the eGovernment Action 
Plan work together (internal coherence) and how the Action Plan relates to activities outside 
the  Action  plan  (external  coherence),  including  other  EU  policies.  To  what  extent  have 
Member  States  developed  eGovernment  services  and  policies  in  line  with  the  overall 
principles set out by the current eGovernment Action Plan?  

What is the EU added value resulting from the EU intervention compared with what could be 
achieved by Member States at national and/or regional levels? This assessment relates to the 
added value of the EU eGovernment Action Plan, why action at the EU level was necessary 
and  what  would  have  happened  without  an  Action  Plan?  Which  of  the  eGovernment  policy 
priorities could not have been achieved without an EU-level Action Plan? 

15 

 
 
 
 
 
4.  METHOD AND SOURCES 

The  evaluation  has  been  coordinated  by  the  EC's  Directorate-General  Communications 
Networks,  Content  and  Technology  with  the  support  of  an  Inter-Service  Group  (with 
representatives  of  Commission    Directorate-Generals  Agriculture  and  Rural  Development; 
Communication;  Competition;  Informatics;  Economic  and  Financial  Affairs;  Employment; 
Social  Affairs  and 
Industry, 
Entrepreneurship  and  SMEs;  Migration  and  Home  Affairs,  Joint  Research  Centre; 
Publications  Office;  Justice  and  Consumers;  Mobility  and  Transport;  Regional  and  Urban 
Policy; Research and Innovation; Health and Food Safety; Secretariat-General; Taxation; and 
Customs  Union  and Trade), which steered and monitored progress of the exercise, ensuring 
the necessary quality, impartiality and usefulness of the evaluation. 

Inclusion;  Energy;  Environment; 

Internal  Market, 

For  the  evaluation,  a  broad  literature  review  was  conducted  covering  various  reports,  EU 
studies and a number of international references. Important sources in the desk research were 
the  mid-term  evaluation  of  the  eGovernment  Action  Plan  2011  -  201543  and  the  annual 
eGovernment Benchmarking reports. Furthermore, additional information and data (listed in 
the in the footnotes and in the "References and data sources" part in the end of this document) 
were  used  to  answer  the  evaluation  questions.  Their  methods  are  available  in  the  original 
referenced  reports.  The  eGovernment  factsheets  published  on  the  Joinup  platform44  give  an 
overview of eGovernment in each Member State.  

A  12  week  online  public  consultation  was  conducted  to  collect  primary  data  and 
complemented  by  statistical  data  (Eurostat  data  and  Digital  Economy  and  Society  Index 
DESI45) and Digital Agenda Scoreboard reports46. 

An  effort  was  made  to  “triangulate”  the  data  used  throughout  the  analysis.  This  means  that 
findings  presented  in  the  evaluation  are  supported  by  evidence  from  different  data  sources 
whenever  additional  data  was  available.  Any  contradicting  evidence  has  been  weighed 
according to its strength and quality before reaching conclusions.  

4.1.  Mid-term evaluation 

The  first  monitoring  of  the  implementation  of  the  EU  eGovernment  Action  Plan  was  done 
through  the  mid-term  evaluation  that  carried  out  a  monitoring  and  stocktaking  exercise 
(progress assessment), with the purpose of providing an overview of the extent and modalities 
of  implementation  of  the  eGovernment  Action  Plan,  its  four  priorities,  40  actions  and 
governance chapter.  

The Mid-term evaluation of the eGovernment Action Plan was carried out between December 
2013  and  June  2014  by  external  contractors.  The  eGovernment  Action  Plan  states  that  "A 
midterm evaluation of implementation of this Action Plan will be conducted in 2013. " 

4.2.  Annual benchmarking reports 

Since 2001, the Commission measures the progress made in the availability of on-line public 
services..  In  2012,  this  benchmarking  was  aligned47  with  the  priorities  of  the  eGovernment 

43 Mid-term Evaluation of the e-Government Action Plan 2011–2015. 2014.  
44 NIFO eGovernment factsheets (edition 2015 published in 2016). 
45 http://ec.europa.eu/digital-agenda/en/digital-economy-and-society-index-desi 
46 https://ec.europa.eu/digital-agenda/download-scoreboard-reports 

16 

 
 
 
 
 
 
                                                 
Action  Plan  2011-2015.  The  benchmarking  method  examines  a  set  of  indicators  related  to 
each of the four political priorities of the Malmö Ministerial Declaration and the Action Plan. 
The latest benchmarking report48 dates from June 2015. This 12th eGovernment Benchmark 
report  is  the  third  edition  of  the  measurement  made  according  to  the  new  eGovernment 
Benchmark  Framework  2012-2015.  Unless  otherwise  specified,  the  benchmarking  report 
references in this evaluation refer to the "EU28+", which refers to 33 European countries: the 
EU, Iceland, Norway, Serbia, Switzerland and Turkey. 

With regard to the methodology used for the 2015 benchmarking report, each of the priority 
areas of the eGovernment Action Plan was assessed by one or more top level benchmarks. 
Two methods were used for data collection: Mystery Shopping and User Survey49.  

Benchmarking report 

Year of data 
collection 

eGovernment  Benchmarking  report  2012  (May  2013):  Public  Services 
Online. "Digital by Default or by Detour" 

2012 

eGovernment  Benchmark  Report  2014  -11th  report  (June  2014): 
Delivering on the European advantage? "How European governments can 
and should benefit from innovative public services" 

2013 

eGovernment  Benchmarking  Report  2015  (June  2015):  Future-proofing 
eGovernment for a Digital Single Market.  

2014 

eGovernment Benchmarking Report 2016 (to be published) 

2015 

4.3.  The online public consultation   

The online public consultation gathered input from citizens, businesses and civil servants over 
12  weeks  from  30  October  2015  till  22  January  2016.  Its  objective  was  to  inform  and  help 
define  the  new  eGovernment  Action  Plan  2016  -  2020.  The  consultation  was  promoted 
through the eGovernment network (comprising 5000 entities), the EC website and on Twitter.  

47 eGovernment Benchmark Framework 2012 – 2015. Method paper, July 2012. 
48 Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmark report, 2015.  
49  The  Mystery  Shopping  had  consultants  simulate  the  user  journey  of  citizens  dealing  with  the  public 
administration in order to get some services. The tested services (both for services and enterprises) were seven 
different  families  of  services  grouped  around  a  specific  user  need  (e.g.  services  around  starting  a  business) 
embodied  by  the  concept  of  life  event.  The  User  Survey  was  conducted  on  an  online  panel  of  26000  citizens 
across 32 countries, and participants were asked about their level of interaction with public administrations both 
online  and  offline  and  about  their  degree  of  satisfaction  with  the  online  channel.  The  User  Survey  has  been 
conducted only in 2012, while the Mystery Shopping took place in all the years, but alternating the measurement 
of half of life events each year. 

17 

 
 
 
 
 
                                                                                                                                                         
 
The questionnaire also addressed the EU eGovernment Action Plan  2011 – 2015 as regards 
the improvement of cross-border eGovernment services and the success of the four political 
priories. The respondents were asked to provide answers in the form of agreeing to predefined 
statement on a scale from "not likely" to "very likely", in addition to a few open questions. 

The results are provided in annex 2. 

4.4.  Limitations  

The evaluation faced some limitations in the collection of data, whose impact was mitigated to 
a maximum possible extent: 

  Measuring the impact of the eGovernment Action Plan is a challenging exercise as it is 
almost impossible to isolate the impact of the Action Plan from other developments in 
the public sector, such as ICT-policies or the increased use of the Internet in general. 
Furthermore, some objectives of the Action Plan, such as increased collaboration, are 
non-tangible  and,  as  a  result,  difficult  to  quantify.  The  evaluation  has  used 
benchmarking  to  measure  the  progress  in  the  different  policy  priority  areas  of  the 
Action Plan and assessed the effectiveness criteria qualitatively.   

  The  Action  Plan  did  not  have  a  dedicated  budget  but  relied  on  funding  from  other 
programmes for a large part of its actions. Other actions were financed by the Member 
States.  As  a  result,  the  efficiency  analysis  (in  particular  the  value  for  money 
assessment) was difficult to carry out.   

  Despite  being  prompted  in  a  number  of  occasions  by  the  EC,  there  is  no  systematic 
measuring  or  reporting  requirement  for  the  Member  States  of  cross-border  public 
services use. To overcome this, the evaluation used data from other sources, including 
case studies.   

  Given the multiplicity of the tools used to gather evidence, the results obtained are of 
different  nature  (for  instance,  eGovernment  related  data  stems  from  the  mid-term 
evaluation  of  the  Action  Plan  and  the  benchmarking  reports  while  the  online  public 
consultation  provided  insights  in  order  to  prepare  for  the  new   eGovernment  Action 
Plan 2016-2020).  

  The  evaluation  takes  into  account  the  inherent  limitations  of  the  findings  of  public 
consultations.  Firstly,  as  in  all  surveys,  the  answers  received  reflect  the  views  of  a 
sample of relevant stakeholders and not those of the entire population who has a stake 
in  this  domain.  Secondly,  stakeholders'  views  convey  an  individual  rather  than  a 
holistic perspective. 

  The  mid-term  evaluation  did  not  systematically  cover  the  five  evaluation  criteria. 
Furthermore,  the  methodology  had  strict  requirements.  For  an  action  to  qualify  as 
"delayed"  it  was  sufficient  that  one  indicator  is  negative  (e.g.  one  Member  State 
answered  ‘no’  to  the  question,  or  one  action  was  delayed  in  one  Member  State)  can 
sometimes give only a partial overview of progress and it should be understood as an 
evaluation of the actions as defined by the action plan, not an assessment of the policy 
priority  as  such.  In  addition,  the  mid-term  evaluation  did  not  capture  the  last  18 
months  of  the  Action  Plan.  This  is  partly  mitigated  using  the  2015  benchmark  study 
and  partly  the  evaluation  questions  in  the  2015/2016  online  consultation  on  the 
eGovernment Action Plan and other recent reports and information.   

Based  on  the  elements  above,  the  evaluation  has  been  carried  out  on  the  basis  of  the  best 
available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate 
and possibly counter-balanced with qualitative data and considerations.  

18 

 
 
 
5. 

IMPLEMENTATION OF THE EU EGOVERNMENT ACTION PLAN 2011 – 2015 – STATE OF 
PLAY OF THE POLICY PRIORITIES 

The  monitoring  of  the  implementation  of  the  Action  Plan  was  carried  out  by  mid-term 
evaluation  and  supported  by  the  annual  eGovernment  benchmarking.  An  assessment  of  the 
completion of actions will be done separately in a planned 2016 study on the assessment of 
the implementation of the Action Plan. The benchmarking reports are not intended to act as a 
benchmarking of the implementation of the Action Plan as such, but its results are informative 
for monitoring the progress of eGovernment in the EU. 

5.1.  General target: 50% of citizens (Digital Agenda) and 80% of businesses make use 

of eGovernment services  

In 2014 eGovernment services were used by 47% of the EU population (38 % in 200950)51. 
The Eurostat  figure52 for individuals (aged 16  – 74)  having used  the Internet for interaction 
with public authorities within the last 12 months was 46 % in 2015 (41 % in 2010).  

Figure 4 Individuals in the EU that had used the Internet in the last 12 months for interaction with public authorities. 

eGovernment  usage  by  firms  (EU  28)  has  increased  steadily  from  76%  in  2010  to  84%  in 
201153 and 88 per cent in 201354.  

5.2.  General  target:  a  number  of  key  cross-border  services  be  offered  online  by  2015 

(Digital Agenda) 

The eGovernment Action Plan was to contribute to two key objectives of the Digital Agenda: 
by  2015  a  number  of  key  cross-border  services  will  be  available  on  line  –  enabling  a) 
entrepreneurs to set up and run a business anywhere in Europe independently of their original 
location, and b) allowing citizens to study, work, reside and retire anywhere in the EU. The 

50 Digital Agenda Europe baseline: in 2009, 38% of individuals aged 16 – 74 had used eGovernment services in 
the last 12 months.  
51 Digital Agenda Targets Progress report. 2015.  
52 Eurostat. Individuals using the internet for interaction with public authorities.  
53 Digitizing Public Services in Europe: Putting ambition into action. 9th Benchmark measurement. 2010; Digital 
Agenda Scoreboard Report 2012.  
54 Eurostat: "Enterprises using the internet for interacting with public authorities". 

19 

 
 
 
 
 
                                                 
Digital Agenda had specified that the key cross-border public services would be contained in 
a list to be agreed by Member States by 2011. 

The  identification  of  the  key  cross-border  services  was  discussed  at  the  Ministerial 
Roundtable  in  Poznan  (Poland)  in  201155.  EU  Ministers  responsible  for  eGovernment 
discussed a number of areas of the Digital Single Market from which potential cross-border 
that: 
services 

conclusions56  noted 

roll  out.  The 

identify 

future 

could 

for 

The Ministers concluded that cross–border eServices should be built from components 
that can be shared and re–used. The Large Scale Pilots have already contributed to 
this goals by developing building blocks. 
The Ministers concluded that at least five key cross–border services should be selected 
and implemented between 2012 – 2014 where possible domains have been put forward 
for consideration. These cross–border eServices should be built from components that 
can be shared and re–used. 

The  Member  States'  eGovernment  Expert  Group  later  agreed  to  endorse  a  number  of  focus 
areas  and  key  enablers  for  digital  cross-border  public  services.  Because  of  divergent  views, 
Member  States  did  not  reach  formal  consensus  on  the  exact  services.  However,  the  work 
continued  through  the  The  Competitiveness  and Innovation Programme  ICT  Policy Support 
Programme57  (CIP  ICT-PSP)  LSPs  (STORK,  SPOCS,  PEPPOL,  epSOS58,  eCodex59  and 
eSENS60) towards the development of the building blocks that can be used by Member States 
to produce their own cross-border services according their needs. Most of the Member States 
have participated with co-funding in one or several of these projects to develop cross-border 
solutions for the public services.  

At  the  mid-term  evaluation  it  was  noted  that,  "while  not  all  Member  States  have  identified 
their priority key cross-border public service, and not all Member States have implemented or 
are planning to implement cross-border services, there are several examples of Member States 
implementing cross-border services."  

To  support  the  work  on  cross-border  services  a  detailed  study61was  concluded  by  the 
Commission in 2013. The study identified the cross-border services with the highest potential 
impact.  In  determining  these  services  Member  States  were  recommended  to  refer  to  the 
PESTEL  (Political,  Economic,  Social,  Technological,  Environmental  and  Legal  factors) 
analysis  and  estimated  volume  of  users.  Furthermore,  the  study  noted  that  quick  wins  for 
Member States can be achieved by re-using the building blocks from the CIP ICT-PSP large-
scale pilots. 

55 https://ec.europa.eu/digital-single-market/pillar-vii-ict-enabled-benefits-eu-society/action-91-member-states-
agree-common-list-key-cross  
56  Sixth  European  Ministerial  eGovernment  Conference  “Borderless  eGovernment  Services  for  Europeans”, 
Conference proceedings, November 2011.  
57 ICT Policy Support Programme (ICT PSP) was one of three specific programmes of the Competitiveness and 
Innovation Framework Programme (CIP). 
58 European Patients Smart Open Services project  
59 E-Justice Communication via Online Data Exchange project  
60 Electronic Simple European Networked Services  
61 Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, 
Technical and Semantic Barriers, 2013.  

20 

 
 
 
 
 
 
 
                                                 
The  CEF  Telecom  Regulation62  acknowledges  the  role  of  the  CIP  LSPs  such  as  "PEPPOL, 
STORK, epSOS, eCODEX or  SPOCS, in  validating key cross-border digital services in  the 
internal  market,  based  on  common  building  blocks,  which  are  being  consolidated  by  the 
project eSENS". The regulation's annex specifically identifies a number of building blocks: 

  Electronic identification and authentication  
  Electronic delivery of documents  
  Automated translation 
  Critical digital infrastructures support 
  Electronic invoicing 

The CEF building blocks63 that are now available are: eID, eSignature, eDelivery, eInvoicing 
and Automated Translation. The deployment of eDelivery, eID and eSignatures at the end of 
201564 is described in the table below (excluding countries outside the EU and EEA): 

eDelivery 
Countries that have deployed 
an eDelivery Access Point 

Austria 
Belgium 
Czech Rep. 
Denmark 
Finland 
France 
Germany 
Greece 
Iceland 
Ireland 
Italy 
Netherlands 
Norway 
Spain 
Sweden 
UK 

eSignatures 

The trusted list enabling the 
use of eSignature as part of 
the trust backbone has been 
deployed in the 28 Member 
States (as required by the 
regulation). 

eID 
Countries that are connected 
to the eID network via 
STORK 
Austria 
Belgium 
Czech Rep. 
Estonia 
Iceland 
Italy 
Latvia 
Lithuania 
Luxemburg 
Portugal 
Slovakia 
Slovenia 
Spain 
Sweden 

Furthermore,  the  "Cross  Border  Mobility"  benchmark  of  the  eGovernment  Benchmarking 
indicates to what extent European users can use online services in another country. This top 
level  benchmark,  which  is  split  into  Citizen  Mobility  and  Business  Mobility,  measures  the 
availability and usability of cross border services in a number of life events under four sub-
criteria:  availability, usability, ease of use  and speed of use.  According to  the eGovernment 
Benchmarking Report 201565, across Europe, on average, the score for the indicator of cross-
border mobility for business is better than the citizen mobility indicator (58% as opposed to 

62 Regulation (EU) No 283/2014.. 
63 EC, Joinup, Connecting Europe Facility, Catalogue of Building Blocks, 2014.  
64 CEF Monitoring report Q4/2015.  
65 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmark report, 2015 

21 

 
 
 
 
 
 
                                                 
43%  for  citizens).  There  is  some  improvement  from  the  previous  measurement  (2012/2013, 
data not available earlier) from, respectively, 53% and 38%.  

According to the eGovernment Benchmarking Report 2015, only 57% of the assessed services 
(life  events)  are  available  to  cross  border  businesses  and  only  41%  are  available  to  cross 
border citizens66 in 2014. This represents an improvement of 5 points for both groups in the 
assessed  life  events  (compared  to  previous  measurement  2012/2013,  data  not  available 
earlier).  

5.3.  The four priorities and 40 actions of the EU eGovernment Action Plan 

The 40 actions of the Action Plan (listed and numbered in Annex 4 to this document) were 
grouped  under  four  priorities.  The  answers  of  each  Member  State  (that  answered  to  the 
survey) for each of the action in the mid-term evaluation of the eGovernment Action Plan are 
provided in a separate dashboard67. The last updates on the dashboard were in the first quarter 
of 2014. Non-implementation by one Member State was sufficient to mark an action either as 
delayed or at risk of delay. The mid-term evaluation was finished in June 2014.  

The detailed implementation status of the 40 actions is provided in Annex 4 to this report. The 
status  is  provided  on  basis  of  the  mid-term  evaluation  report  findings.  A  number  of  actions 
were completed after the end of the evaluation and new progress in Commission  actions is 
reported  in  Annex  4;  detailed  data  from  each  Member  States'  progress  after  the  mid-term 
evaluation was not available.   

The mid-term evaluation concluded that of the 40 actions, 16 were completed at the time and 
six were on track. Three actions were at risk of delay and 15 delayed. The delay meant that an 
action had not been completed by one or several Member States or Commission  within the 
deadline year and there was thus a delay from one to two years. The reasons and the possible 
impact  (where  an  analysis  was  already  possible)  of  these  delays  are  assessed  under  the 
evaluation questions in part seven. The large majority of the EC’s 30 Actions were completed 
or were on track. Of these, there were 23 for which the EC has exclusive responsibility, while, 
for the remaining seven, responsibility was shared with Member States.  

5.4.  Governance 

The  High-Level  Expert  Group  of  Member  State  representatives  was  set  up,  and  rules  of 
procedures were adopted in 2011 (action 41).  

According  to  the  mid-term  evaluation  (2014)  of  the  Action  Plan,  almost  all  Member  States 
have  a  National  eGovernment  Strategy,  which  incorporates  priorities  from  the  Malmö 
Declaration and the eGovernment Action Plan (action 42). Only two Member States had not 
yet updated their national eGovernment strategies. 

According to the mid-term evaluation, most of the Member States (17) have incorporated the 
political priorities of the Malmö Declaration and eGovernment Action Plan in their national 
strategies (action 43) and only four countries indicated in 2014 that this action was delayed in 
their case.  

66  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmark  report,  2015.  
Background report, p. 32. 
67 http://www.egovap-evaluation.eu/ 

22 

 
 
 
 
 
 
                                                 
A mid-term evaluation of the Action Plan was carried out and its results and the results of this 
evaluation accompany the Communication on the EU eGovernment Action Plan 2016 – 2020 
(action  44).  Furthermore,  Member  States  have  informed  the  EC  and  the  High-Level  Expert 
Group on how the political priorities of the Malmö Declaration have been achieved, while six 
Member States (of the twenty-four responding in the mid-term evaluation) have not yet done 
so.  Many  countries  have  already  provided  some  lessons  learned  within  the  implementation 
process (action 45).  

23 

 
 
 
 
 
6.  ANSWERS TO THE EVALUATION QUESTIONS 

6.1.  Effectiveness 

The effectiveness analysis assesses how successful the EU eGovernment Action Plan has been 
in achieving or progressing towards its objectives. What was the role of the EU eGovernment 
Action  Plan  2011  –  2015  in  delivering  the  changes?  What  factors  have  influenced  the 
progress? To what extent progress fell short of the target?  

The objectives to be analysed in this part are, first, the two targets of the Digital Agenda in 
Europe:  (1)  50%  of  citizens  make  use  of  eGovernment  services  by  2015;  and  (2)  that  a 
number of key cross-border services be offered online by 2015. The Action Plan also aimed 
that  "by  2015,  80%  of  enterprises  will  have  used  eGovernment".  Secondly,  the  analysis 
includes the four policy priorities. 

The mid-term evaluation of the Action Plan described a wide range of eGovernment effects 
across the different Member States with various case studies as examples, including examples 
in the area of user empowerment, open data, interoperability, eID and transparency.  

However,  the  eGovernment  Action  Plan  was  a  strategic  plan  to  give  the  policies  of  EU 
Member  States  a  direction  towards  commonly  agreed  goals.  Without  a  detailed  individual 
assessment of each action in each Member State it is not possible to associate the quantitative 
and qualitative effects on the Member States level directly to the Action Plan.  

6.1.1.  Objective on using eGovernment (take-up) 

The  80%  target  for  eGovernment  usage  by  firms  (EU  28)  was  achieved  already  in  in  2011 
(84%)68 and it grew to 88 per cent in 201369.   

The action plan objective of "By 2015, 50% of EU citizens will have used eGovernment" was 
nearly  achieved.  In  2014  eGovernment  services  were  used  by  47%  of  the  EU  population70. 
The  Eurostat  figure71  for  individuals  (aged  16  –  74)  having  used  the  Internet  for  interaction 
with the public authorities within the last 12 months was 46 per cent in 2015.  

The achievement of the full 50 % usage by citizens was still expected in 2011. In the Digital 
Agenda Scoreboard report 201172 it was assessed that "the take-up of eGovernment in  2010 
has reached 41 % of the EU population and, given the current trends, could be quite in accord 
with  the  target  of  the  Digital  Agenda  of  50%  of  population  using  it  by  2015."  At  the  same 
time,  the  rise  of  the  use  of  eGovernment  services  has  also  been  driven  by  the  increase  of 
regular Internet users. 

Finding 1 
The  Action  Plan  target  of  by  2015,  80%  of  enterprises  will  have  used  eGovernment  was 
achieved early into the Action Plan. The target of by 2015, 50% of EU citizens will have used 
eGovernment was nearly achieved (46% in 2014). 

68 Digital Agenda Scoreboard Report 2012.  
69 As reported by Eurostat on data "Enterprises using the internet for interacting with public authorities"  
70 Digital Agenda Targets Progress report, 2015.  
71 Eurostat, Individuals using the internet for interaction with public authorities 
72 Digital Agenda Scoreboard report 2011, Pillar 7.  

24 

 
 
 
 
                                                 
6.1.2.  Key cross-border services online 

During  the  Action  plan  duration,  cross-border  eGovernment  services  were  addressed  in  the 
high-level political discussions and in the work of the LSP projects. This was a direct result of 
the  eGovernment  Action  Plan.  The  Action  Plan  played  a  key  role  as  a  basis  and  political 
framework for bringing Member States together to discuss and agree on these initiatives.  

The Action Plan objective was to contribute towards fulfilling the Digital Agenda objective: 
"By  2015,  a  number  of  key  cross-border  services  will  be  available  on  line  –  enabling 
entrepreneurs  to  set  up  and  run  a  business  anywhere  in  Europe  independently  of  their 
original  location,  and  allowing  citizens  to  study,  work,  reside  and  retire  anywhere  in  the 
European Union."  

The contribution towards this objective was achieved by LSP projects implemented under the 
CIP  ICT  PSP  programme.  The  evaluation73  of  the  ICT  PSP  Programme  underlined,  among 
other  aspects,  its  innovative  role  as  its  projects  design  new  platforms  for  innovative  cross-
border  services  in  areas  of  public  interest,  in  particular  in  areas  where  there  are  continuing 
systemic and organisational risks.  

Finding 2 
The Large Scale Pilot projects had an innovative role as projects designing new platforms for 
cross-border services in areas of public interest. The activities on cross-border services went 
beyond the Action Plan's objectives.  

Through the financial support from the CIP ICT-PSP Programme to the LSPs' work on what 
has  become  the  CEF  building  blocks,  the  Action  Plan  has  contributed  to  achieving  also 
regulatory  objectives:  the  eIDAS  Regulation  and  transition  to  the  electronic  public 
procurement. 

The eIDAS Regulation74 got important leverage from the STORK project that piloted 
an interoperability solution and that is referred to in the Regulation recitals. 

In  the  new  public  procurement  directives,  e-procurement  was  one  of  the  important 
changes  introduced.  The  PEPPOL  LSP  (Pan-European  Public  Procurement  Online) 
sought to make it easier for companies to bid for public sector contracts throughout the 
EU  and  provided  supporting  experiences  and  solutions  for  cross-border  e-
procurement.  

In the context of cross-border services, the Points of Single Contact, required by the Services 
Directive,  cover  an  increasing  number  of  administrative  formalities  for  businesses.  The 
SPOCS  LSP  (Simple  Procedures  Online  for  Cross-Border  Services)  sought  to  improve  the 
cross-border  electronic  procedures  for  businesses.  In  the  environmental  area,  the  EC  has 
developed an EU "Shared Environmental Information System" (SEIS). 

The aim of the Action Plan was to contribute towards the availability of cross-border services. 
To  this  date,  life  events  for  businesses  and  citizens  are  not  yet  covered  by  cross-border 
services  and  not  all  Member  States  have  rolled-out  the  building  blocks.  For  example,  the 

73 CIP ICT PSP Final (Second Interim) Evaluation-Final report, 2011 and COM (201302. 
74 Regulation (EU) No 910/2014 

25 

 
 
 
 
 
 
 
                                                 
objective  that  entrepreneurs  should  be  able  to  set  up  and  run  business  anywhere  in  Europe 
independently of their original location has not been met.  

In  the  online  public  consultation  (Annex  2)  the  majority  of  the  respondents  did  not  know 
whether  the  current  eGovernment  Action  Plan  has  improved  cross-border  eGovernment 
services  overall  or  did  not  agree  that  the  Action  Plan  has  improved  them  services.  Around 
25%  of  the  respondents  agreed  that  the  Action  Plan  has  improved  cross  border  services 
overall.  

Figure 5 Online public consultation: Has the current eGovernment Action Plan improved cross-border eGovernment 
services overall? 

Finding 3 
The eGovernment Action Plan has shown effectiveness to mobilise resources in particular in 
areas  where  the  cross-border  services  objectives  have  been  shared  between  both  EU 
technical/operational  and  regulatory  measures,  e.g.  cross-border  eID  and  e-procurement. 
However, take-up has varied between Member States: entrepreneurs are not yet able to set-up 
and run a business anywhere in Europe independently of their location nor are citizens able to 
complete administrative procedures cross-border for all life events on-line. As reported under 
point  6.2,  according  to  the  eGovernment  Benchmarking  Report  2015,  only  57%  of  the 
assessed  services  (life  events)  are  available  to  cross  border  businesses  and  only  41%  are 
available to cross border citizens.  

6.1.3.  Effectiveness of the four policy priorities 

The  progress  in  four  of  the  main  areas  (User  empowerment,  Internal  Market  and  pre-
conditions, key enablers/pre condition) is visible in the below figure (Figure 6) from the 2015 
eGovernment  Benchmark  background  report  "Future-proofing  eGovernment  for  the  Digital 
Single Market".  

26 

 
 
 
 
Figure  6  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmark  report  2015.  Top-
level benchmarks EU28+ 2012/2013 versus 2013/2014 (%). (The top level benchmarks Effective Government and User 
Centricity (user survey) were measured once, 

6.1.3.1.  Effectiveness of the User Empowerment policy priority 

In  the  Action  Plan,  empowerment  was  referring  to  increasing  the  capacity  of  citizens, 
businesses  and  other  organisations  to  be  pro-active  in  society  through  the  use  of  new 
technological tools. Governments should provide easy access to public information, improve 
transparency and allow effective involvement of citizens and businesses in the policy-making 
process.  

According  to  the  mid-term  evaluation75,  the  actions  under  the  User  Empowerment  priority 
require  further  attention.  Important  targets  have  been  achieved  in  implementing  inclusive 
services,  in  involving  citizens  and  business  in  the  policy-making  process,  and  notable 
progress has been made in creating collaborative services. There are also good achievements 
of  the  Action  Plan  targets  under  sub-priority  on  re-use  of  public  sector  information.  At  the 
same time, there are delays in general in drawing up and adopting common targets, indicators 
and measurement frameworks. Details of the achievements under each of the 14 actions under 
this priority are in Annex 4.  

Services designed around users' needs and Inclusive Services 

The  Action  Plan  has  supported  exchanges  of  practice  in  the  field  of  user-centred,  inclusive 
and  accessible  eGovernment  services  (through  the  Joinup76  platform),  though  no  agreement 
had  been  reached  with  Member  States  on  common  targets  and  evaluation  criteria  for  those 
services.  

75 Mid-term evaluation of the eGovernment Action Plan 2011 - 2015 (p. 34). 
76 https://joinup.ec.europa.eu/ 

27 

 
 
 
 
 
 
                                                 
The  Commission  adopted  its  proposal  for  a  directive  on  the  accessibility  of  public  sector 
bodies' websites on 3 December 201277. The MeAC (Measuring progress of eAccessibility in 
Europe)  reports  provided  evidence  and  analysis  to  help  understand  and  compare  the 
approaches  followed  by  the  European  countries,  with  a  view  to  identifying  issues  and 
challenges, good practices and future priorities in the web accessibility field78. 

According  to  the  mid-term  evaluation  report,  most  Member  States  have  introduced 
personalised services in their One-Stop-Shops for business and for citizens, as well as in the 
area  of  health  and  tax  services.  The  majority  of  the  Member  States  that  have  already 
introduced personalised services offer them via multiple channels, such as mobile access via 
mobile apps and helplines. 

Re-use of Public Sector Information (PSI) 

Under  the  Action  Plan  priority  on  re-use  of  public  sector  information,  among  other  things 
(details  in  Annex  4),  the  Commission    has  reviewed  and  subsequently  revised  the  PSI 
Directive  improving  re-usability  of  data  held  by  Member  States'  authorities  and  has  put  in 
place an Open Data Portal for data of the EU institutions79. The portal, launched in 2012, has 
currently more than 8000 datasets available. 

The Action Plan achievements under the sub-priority Re-use of Public Sector Information (see 
Annex 4 and achievements under Action 6, 7 and 8, including the review of the PSI Directive 
and  the  set-up  of  the  Open  Data  Portal)  are  good,  apart  from  the  formal  achievement  on 
agreement on indicators. The Action Plan has supported the achievements and has given them 
political  visibility.  There  appears  to  be  increased  awareness  of  the  importance,  including 
economic importance, of open data as shown, for example, by the adoption of the G8 Open 
Data  Charter  and  the  commitment  of  various  EU  Member  States  to  the  Open  Government 
Partnership.    

The study "Open Data Maturity in Europe 201580" reported that 27 countries (of EU28+) have 
a national Open Data portal and an open data policy is in place in 71% of the countries, often 
as part of a more generic Digital Strategy or eGovernment programme. It is also worth noting 
that  beyond  the  objectives  of  the  Action  Plan,  but  supporting  the  sub-priority  on  re-use  of 
public sector information, the EU endorsed the G8 Open Data Charter in 2013 and committed 
to promoting the application of the principles of the G8 Open Data Charter to all EU Member 
States within the context of a range of ongoing activities81. 

Improvement of Transparency 

On  the  sub-priority  on  Transparency  the  mid-term  evaluation  found  that  the  actions  under 
transparency  were  delayed.  Fourteen  Member  States  were  using  some  transparency  targets 
and  most  Member  States  provide  online  access  to  information  on  government  laws  and 
regulations (see Annex 4 for details). Member States also provide access to citizens' to their 
personal  data,  but  not  all  provide  it  in  electronic  form  as  seen  in  the  Action  Plan  (30%  of 
Member States did at the time of the mid-term evaluation).  

77 COM(2012) 721 final 
78 Study Measuring Progress of eAccessibility in Europe – MeAC, 2013  
79 https://open-data.europa.eu 
80 Open Data Maturity in Europe 2015 Insights into the European state of play, 2015 
81 https://ec.europa.eu/digital-single-market/en/news/eu-implementation-g8-open-data-charter 

28 

 
 
 
 
 
 
  
 
                                                 
At the same time, the eGovernment benchmarking report 201582 stated that the transparency 
benchmark  shows  improvement  from  the  previous  measurement,  but  is  still  quite 
unsatisfactory,  as  it  is  at  51%.  The  Transparency  indicator  of  the  benchmarking  report 
examines the extent to which governments are transparent about their own responsibilities and 
performance, the service delivery process, and the personal data involved83. 

It is also positive that users have gained better access to personal data that is handled on the 
governments’ websites, but they still face considerable barriers when it comes to the clarity of 
the  service  delivery  process.  The  Action  Plan  supported  the  delivery  of  a  number  of 
transparency objectives, but this remains an area for continued work.  

Involvement of citizens and businesses in policy-making processes 

In the sub-priority Involvement of citizens and businesses in policy-making processes progress 
is very good according to the mid-term evaluation (see Annex 4 for details). The Action Plan 
supported and gave a guiding framework for the work for governance and policy modelling 
under  the  Seventh  EU  Framework  Programme  for  Research  (FP7)  and  for  eParticipation 
projects  that  were  supported  under  the  Competitiveness  and  Innovation  Framework 
programme (CIP).  

Finding 4 
The Action Plan delivered mixed results regarding its contribution to the 4 policy objectives. 

While  it  was  effective  in  achieving  and  contributing  to  targets  in  inclusiveness,  re-use  of 
public  sector  information  and  in  involving  citizens  and  businesses  in  the  policy  making 
processes – particularly  where both voluntary and regulatory actions at EU-level shared the 
same  objective  such  as  in  re-use  of  public  sector  information  -,  it  did  not  fully  meet  its 
objectives  under  the  sub-priority  on  transparency  where  the  benchmarking  report  findings 
indicate that the transparency of governments could be future improved. 

Not reaching an agreement on common targets was reported by the mid-term evaluation to be 
an issue requiring further attention in relation to the collaborative production of services, PSI 
re-use indicators and targets and indicators on transparency.  

When  looking  into  reasons  of  why  common  targets  or  indicators  were  not  agreed  in  the 
different areas, it should be noted that sometimes similar activities were already taking place 
elsewhere,  sometimes  making  the  agreement  under  the  Action  Plan  unnecessary,  or  similar 
issues were discussed elsewhere:  

  The  Web  Accessibility  Directive  proposal  (2012)  aims  to  approximate  the  laws, 
regulations and administrative provisions of the Member States on the accessibility of 
websites of public sector bodies. A directive proposal goes beyond the (non-binding) 
Action Plan target of agreeing on common targets.  

82  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmarking  report,  2015. 
Insight report, p. 25 
83 The data is collected through life event assessment by "Mystery Shoppers". More details of the method are in 
chapter 5.2. 

29 

 
 
 
 
 
 
 
 
 
 
 
 
 
                                                 
  For  the  collaborative  production  of  services,  workshops  were  organised  to  discuss 
recommendations but these did not lead to agreement on common targets for the roll-
out.  

  For  the  PSI  re-use  indicators,  the  majority  of  the  Member  States  agreed  as  to  the 
usefulness of the indicators of the PSI Scoreboard84. However, given the existence of 
other  benchmarking  tools  (Global  Open  Data  Index,  Open  Data  Barometer)  and  the 
fact  that  the  PSI  Directive  does  not  mandate  adopting  specific  indicators,  it  was 
difficult to ensure a uniform adoption of any given set of indicators for all EU Member 
States.  Furthermore,  the  Europan  Data  Portal85,  has  developed  its  own  approach  and 
conducts  a  'landscaping  study'  every  year  to  measure  open  data  maturity  across 
Europe. 

  For transparency, the mid-term evaluation found that some Member States were using 

the Open Government Partnership indicators.  

Finding 5 
A number of common targets and indicators were not agreed by MS due to similar work and 
agreements  made  in  other  fora.  The  Action  Plan  could  not  be  adjusted  to  take  this  into 
account.  

Looking  beyond  the  specific  actions  of  the  eGovernment  Action  Plan  to  the  broader 
development  of  user  empowerment  in  the  EU  with  regards  to  eGovernment  services,  the 
eGovernment Benchmarking Report 2015 indicates that governments’ efforts to improve the 
quality  of  the  online  experience  have  focused  on  increasing  the  availability  of  their  online 
services  and  on  improving  the  mechanisms  for  online  support  and  feedback.  Users  are  also 
more  empowered  with  medium  increases  in  User  Centricity  and  Transparency.  Their 
performance is weaker at the user friendliness and time efficiency of the online services.  

The eGovernment Benchmarking report 2015 indicates that tests of 'user centricity' (to what 
extent a service is provided online and how it is perceived) show good results with a score of 
73  out  of  a  possible  100  points86  across  the  EU28+.  There  is,  however,  a  big  difference 
between  the  compound  indicators  of  user  centricity,  with  much  better  performances  for 
usability  and  online  availability  of  services  than  for  the  ease  and  speed  of  using  those 
services.  While  availability  was  up  six  points  in  2014  (48%  of  all  services  are  completely 
online and available through a portal87), ease and speed of use did not change between 2012 
and 2014. This indicates that many Member States are not focusing enough on the quality of 
the user’s experience88. This again suggests, according to the 2015 benchmark report, a trend 
of  quantity  over  quality.  The  differences  between  citizens  and  businesses  are  not  too  large, 
suggesting  that  while  there  are  significantly  more  information  and  services  available  to 
businesses, the quality of these does not differ much from citizen oriented eGovernment.  

84 http://www.epsiplatform.eu/content/european-psi-scoreboard 
85 http://www.europeandataportal.eu/ 
86 Method used: "Mystery Shopping". This top level (compound) benchmark assesses the availability and 
usability of public eServices. 
87 Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmarking report 2015, 
Background report, p. 23 
88 Future-proofing eGovernment for the Digital Single Market. eGovernment Benchmarking report 2015, Insight 
report, p.19. 

30 

 
 
 
 
 
 
 
 
                                                 
The online public consultation results reflect low awareness about the work carried out in the 
User Empowerment area. For User Empowerment, 27% of the respondents assessed that the 
measures had been successful; 48% said that these measure of the action were not successful 
and 24% did not know. It should be noted that the Action Plan activities were typically not 
directed  directly  towards  citizens  and  businesses,  but  activities  were  carried  out  by  the 
Member States and the Commission with a view to, for example, creating enabling conditions 
which are not directly visible to individual citizens as eGovernment Action Plan activities.  

6.1.3.2.  Effectiveness of the internal market policy priority  

The  internal  market  dimension  of  the  Action  Plan  covers  some  of  the  most  relevant 
accomplishments in the advancement of the cross-border delivery of services for citizens and 
businesses.  The  Action  Plan  supported  the  internal  market  objective  through  activities  in 
"seamless  services  for  businesses",  "personal  mobility"  and  in  "EU-wide  implementation  of 
cross-border services". The detailed results are included in annex 4 of this report.   

According  to  the  mid-term  evaluation  of  the  Action  Plan,  most  of  the  Member  States  have 
incorporated the political priorities of the Malmö Declaration, which is the basis of the Action 
Plan, in their national eGovernment strategies. As a consequence, aspects of internal market 
related eGovernment activities have been included in national strategies.    

The role of the Action Plan in driving the cross-border services was discussed above in 7.1.2. 
The effectiveness of the work of the LSP projects was acknowledged also in the Commission  
report89  on  "Evaluations  of  the  Competitiveness  and  Innovation  Framework  programme" 
where  the  evaluation  panel  recommended  that  the  top-down  policy-driven  approach  in  the 
LSP projects (Pilot A) should be continued, as it is a working model of how to enhance more 
widespread uptake of new innovative services at EU level.90 

Also  in  e-procurement  the  achievements  went  beyond  the  objectives  of  the  Action  Plan.  In 
addition  to  the  objective  stated  in  the  latter,  the  mandatory  transition  to  e-procurement  is 
driven  by  the  new  procurement  directives  and  facilitated  with  a  wide  array  of  policies  and 
support measures.  

Finding 6 

Due to  the lack of flexibility of  the Action Plan, it did  not adapt to  changing or  new policy 
priorities and this reduced its effectiveness in terms of setting new and advanced priorities for 
the Member States and the European Commission.  

Consecutive  assessments of the Points of Single Contact in 2011/12, 2013 and 2014/15 show 
continuous but slow progress towards fulfilling the requirements of the Services Directive and 
becoming  an  effective  tool  for  further  integration  of  the  Single  Market.  The  most  visible 
improvement  has  been  made  with  regards  to  the  availability  of  online  procedures,  although 
the gap is still significant. Furthermore, efforts to improve general usability of the web portals 
and  accessibility  to  foreign  users  are  recorded,  but  still  insufficient  to  produce  needed 
effects.91 

89 COM(2013)2 
90  The  feasibility  and  scenarios  for  the  long-term  sustainability  of  the  Large  Scale  Pilots,  including  'ex-ante' 
evaluation, 2013  
91 The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015, p. 50 

31 

 
 
 
 
 
 
                                                 
As  for  the  effectiveness  of  cross-border  eEnvironment  services,  the  Mid-Term  technical 
evaluation92  on  the  implementation  the  INSPIRE  Directive  lists  a  number  of  effective 
eEnvironment  cross-border  services  already  at  the  mid-term  implementation  stage  of 
INSPIRE.  It  emphasises  that  effective  cross-border  eEnvironment  services  will  depend  to  a 
large extent on further ensuring the coherent and timely implementation of INSPIRE across 
the Member States in the 2016-2020 period.  

Going beyond  the actions of the Action Plan,  the eGovernment  Benchmarking  Report 2015 
measured the cross-border mobility of businesses through studying the life event "Starting up 
a  business".  Scores  on  all  four  indicators  (speed  and  ease  of  use,  online  availability  and 
usability) are generally much higher than those for other life events. The score on Usability is 
excellent at 81% but the scores for Online Availability, Ease of Use and Speed of Use leave 
room for improvement.

93

Personal mobility 

In  the  Action  Plan  sub-priority  on  personal  mobility  activities94  have  been  undertaken  to 
support  exchange  of  good  practices  and  to  coordinate  the  Member  States’  efforts  to  jointly 
develop and set-up cross-border and interoperable eDelivery services. The Action Plan set-out 
the  framework  and  commitment  for  this  voluntary  work.  According  to  the  mid-term 
evaluation  of  the  eGovernment  Action  Plan,  more  than  50%  of  the  Member  States  already 
provide or are planning to provide those services. The CEF Telecom programme has financed 
the deployment of the cross-border eDelivery service. 

According to the eGovernment Benchmarking Report 201595, eGovernment services are still 
far from giving adequate support to citizens moving to another EU country. The cross-border 
mobility compound benchmark (beyond the actions of the Action Plan)_measures the ease and 
speed of use, online availability and usability of online public services for nationals of another 
EU country giving a score of maximum 100. This stands at 58% for businesses and at 43% for 
citizens96.  Although  there  is  improvement  from  previous  year's  benchmarking  report  (from, 
respectively, 53% and 38%), these figures are lower than the corresponding ones for domestic 
services. Cross-border transactional services (where a citizen can complete an entire process 
online)  are  rare.  At  the  same  time,  significant  increases  of  both  the  Business  Mobility  and 
Citizens  Mobility  score  of  the  benchmarking  report  indicate  that  it  is  gradually  becoming 
easier for people working across borders to access eGovernment services.  

Finding 7 
In the Action Plan's internal market priority, the cross-border delivery of public services for 
citizens and businesses in a number of sectors identified in the Action Plan was advanced. In 
e-procurement, the achievements go beyond the objectives of the Action Plan, since similar to 
findings 2 and 3, both EU voluntary and regulatory actions reinforced each other.  However, 

92 EEA Technical report No 17/2014. Joint EEA-JRC report. 
93  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmarking  report,  2015. 
Background report. Pp. 46 - 47 
94 For example, the ISA programme activity on Common Infrastructure for Public Administrations Sustainability 
95 http://ec.europa.eu/digital-agenda/en/news/eu-egovernment-report-2015-shows-online-public-services-europe-
are-smart-could-be-smarter 
96  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmarking  report,  2015. 
Insight report, p. 25.  

32 

 
 
 
 
 
 
 
 
 
                                                 
availability of cross-border public services for citizens and businesses remain still insufficient 
for a fully functioning Internal Market.  

6.1.3.3.  Effectiveness of the "Effeciency and Effectiveness" policy priority 

According to the eGovernment Action Plan  mid-term review significant progress was being 
made  in  improving  organisational  processes  with  a  "great  deal  of  activity  within  Member 
States both at national and local level, focusing on better coordination and cooperation across 
different public organisations, and therefore improved governance"97. The Action Plan played 
an essential role in supporting the exchanges of experience and the sharing of new approaches 
through  the  Expert  Group  meetings,  workshops  and  the  Joinup  (ePractice)  portal.  The 
programme for staff exchanges between administrations in different Member States was not 
carried out given the current budgetary cuts due to the economic situation. 

In the area of the reduction of administrative burden, the Action Plan included a number of 
actions related to the once-only principle. The sharing of experiences was done in a dedicated 
workshop  and  at  the  High-Level  Expert  Group  meetings.  A  cost-benefit  analysis  study  was 
carried out98 and it gave important input  to  continuing  the work  on the once-only  principle. 
The study provided also a roadmap for implementation of the once-only principle. The once-
only policy gained further momentum through its inclusion in the Council Conclusions 201399  
. This supported further steering of Commission policy and resources towards the once-only 
principle.  The  principle  was  included  in  the  Digital  Single  Market  strategy  and  a  call  for  a 
new LSP project focusing on EU-wide once-only principle for businesses was included in the 
H2020 Work Programme 2016 – 2017100 adopted in 2015. The effects of the Action Plan in 
went beyond the original objectives, and were influenced by the increased political visibility. 

In the area of Green Government, the Action Plan did not succeed in achieving its objectives 
in assessing the possibilities of eGovernment in reducing the carbon footprint. The study was 
not  carried  out  due  to  insufficient  data  (study  on  the  potential  of  eGovernment  to  reducing 
carbon footprint of government). The mid-term evaluation also found that although there were 
interesting  developments  in  the  area  of  sustainability,  most  Member  States  did  not  have 
indicators or evaluation procedures in "Green Government". An agreement on indicators and 
evaluation  procedures  for  measuring  the  reduction  of  carbon  footprint  as  a  result  of 
eGovernment between Member States would not have been realistic.  

In  2015  the  REFIT  Fitness  Check  of  the  environmental  reporting  process101  was  launched 
contributing  to  the  reduction  of  the  carbon  footprint  of  governments.  It  aims  to  “Launch  a 
broad review of reporting requirements to see how burdens can be alleviated. This review will 
have  a  particularly  strong  focus  on  areas  where  stakeholders  have  recently  indicated  their 
concerns,  such  as  agriculture,  energy,  environment  and  financial  services."  With  regard  to 
environment  the  technical  Mid-term  evaluation  report102  on  the  INSPIRE  implementation 

97 Mid-term evaluation of the eGovernment Action Plan 2011 – 2015, 2014, p. 41  
98 The study extrapolated that the extension of the Danish approach to implement the “once-only” principle is 
likely to generate an annual net saving at the EU 28 level, amounting to around € 5 billion per year by 2017. This 
highly positive impact depends on a complex system of registries being freely accessible by users (citizens and 
businesses) for commercial purposes, which additionally might foster growth in some economic sectors. 
99 European Council, Conclusions, 24/25 October 2013  
100  H2020  Work  Programme  2016  –  2017,  Europe  in  a  changing  world  –  inclusive,  innovative  and  reflective 
societies. Co-creation-05-2016: Co-creation between public administrations: once-only principle.  
101 COM(2015)215 
102 EEA Technical report No 17/2014. Joint EEA-JRC report. 

33 

 
 
 
                                                 
emphasised  the  need  to  prioritise  in  the  further  implementation  period  of  INSPIRE  the 
connection  between  EU  environmental  reporting  obligations  and  INSPIRE  so  as  to  reduce 
overall  administrative  burden  and  make  reporting  easier  and  more  efficient  for  the  Member 
States. 

Finding 8 
The  Action  Plan  was  effective  in  contributing  to  significant  progress  in  improving 
organisational  processes.  The  once-only  principle  policy  gained  political  visibility  from 
related  Council  Conclusions,  which  increased  the  overall  effectiveness  of  the  Action  Plan 
priority.  The  reduction  of  carbon  footprint  continued  to  have  a  high  visibility  during  the 
duration  of  the  Action  Plan,  but  in  many  Member  States  it  was  not  linked  specifically  to 
eGovernment services and the Action Plan did not reach its objectives in these actions.  

6.1.3.4.  Effectiveness of the preconditions policy priority 

The EU eGovernment Action Plan's policy objectives for the preconditions priority supported 
putting in place technical and legal pre-conditions for eGovernment. The mid-term evaluation 
noted  that  coordination  of  national  efforts  to  support  the  development  of  interoperable 
solutions  and  key  enablers  is  ongoing.  The  actions  that  have  been  supported  by  the  Action 
Plan include:   

  European  projects  (such  as  PEPPOL  and  other  LSPs)  piloted  shared  cross-border 

solutions that many Member States are implementing; 

  The alignment of Member States with the European Interoperability Framework (EIF) 
(combined  with  a  revision  of  the  European  Interoperability  Strategy  (EIS));  The 
European Interoperability Framework promotes and supports the delivery of European 
public  services  by  fostering  cross-border  and  cross-sectoral  interoperability  and  has 
strongly contributed to the Action Plan objectives in the preconditions priority. 

  The  adoption  of  a  Regulation  on  Mutual  recognition  of  Identification  and 
Authentication; with the eIDAS Regulation and the STORK solution the eGovernment 
Action Plan proved very effective. To complete the legal framework, the Commission 
adopted several implementing acts under eIDAS Regulation. 

  Launch of pilot projects for innovative architecture and technologies in eGovernment; 
under  sub-priority  Innovative  eGovernment,  the  mid-term  evaluation  assessed  the 
progress as very satisfactory.  

  On-going work on the deployment of the CEF building blocks. 

The Action Plan  pushed Member States  administrations towards the deployment of the pre-
conditions, as shown, for example, in the EIF alignment figures (Figure 7) below and in the 
eID roll-out figures that indicate that 13 Member States are connected to the eID network via 
STORK (details are in Annex 4 under action 37).   

However, in the online public consultation, the majority of the respondents did not know the 
measures in the pre-conditions area or rated them  not successful. At the same time,  46% of 
public  administrations  that  responded  to  the  survey  rated  the  measures  related  to  the 
preconditions for developing eGovernment successful.  

34 

 
 
 
 
 
As an example of effective work supported by the Action Plan, the Member States national 
interoperability frameworks were aligned to the EIF (Figure 7). This has helped in achieving a 
common understanding of interoperability in the public sector and how to achieve it. 

Figure 7 NIF-EIF alignment overview for each country assessed. State of play of interoperability in Europe – Report 
2014. 

Some of the above interoperability activities, like the implementation of the EIF, STORK or 
PEPPOL  were  supported  also  by  the  ISA  programme  which  has  also  funded  various 
interoperability  activities.  It  has  contributed  substantially  also  to  the  implementation  of  a 
number of activities in the Maintenance and Implementation Work Programme of INSPIRE. 
INSPIRE 
interoperability  between  public 
administrations. 

important  element 

in  cross-sector 

is  an 

The eGovernment benchmarking report 2015 found that in the key enablers measured in the 
report (measuring also enablers beyond the  Action Plan),  there  was only  a marginal  gain in 
2013/2014  compared  to  2012/2013103  in  the  EU28+.  Europe  has  been  slow  in  terms  of 
adopting  key  enabling  technologies  for  supporting  public  online  services.  Some  Member 
States  score  well  with  some  of  technologies  but  less  for  others  showing  that  countries  have 
different  priorities  in  the  adoption  of  these  enablers.  Most  countries  showed  almost  no 
progress from 2012-13 to 2013-14. The most common of these technologies (eIdentification) 
was  deployed  in  63%  of  the  cases  examined  in  the  eGovernment  benchmarking  report  in 
2014. 

Finding 9 
The European Interoperability Framework (EIF) has strongly contributed to the Action Plan 
objectives  in  interoperability.  In  the  Action  Plan's  key  enablers  work,  a  legal  framework 
(eIDAS)  was  provided  on  electronic  identification  and  trust  services  and  the  eID  rollout  of 

103  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmarking  report,  2015. 
Background report. Insight report, p. 30 

35 

 
 
 
 
 
                                                 
eIDAS  compatible  technical  solutions  is  progressing,  but  mostly  within  certain  Member 
States, and only in certain cases across borders.  

6.1.3.5.  Effectiveness of the "Governance" actions of the Action Plan 

Important  effects  of  the  Action  Plan  arise  from  the  governance  chapter  of  the  eGovernment 
Action  Plan.  These  included  the  creation  of  an  eGovernment  High-Level  Expert  Group  and 
the  update  of  national  eGovernment  strategies  incorporating  the  principles  of  the  Malmö 
declaration. Actually, in most Member States, the national eGovernment strategies incorporate 
the  Malmö  priorities  (see  above  6.4).  The  effect  has  been  that  in  most  Member  States  the 
national eGovernment strategies thus include Action Plan's political objectives. The mid-term 
evaluation  gives  some  examples  that  show  how  the  national  eGovernment  work  has  been 
influenced  by  the  Action  Plan104.  The  Expert  Group  has  been  a  forum  for  exchange  of 
information and discussion on eGovernment EU-wide.  

Finding 10. 
An EU eGovernment Action Plan (based on the Malmö Declaration) has been a "mobiliser" 
instrument for national eGovernment policies. This gave further strength and visibility to the 
objectives  of  the  Action  Plan.  The  Action  Plan's  governance  established  an  EU-wide  forum 
for exchange of information and discussion on eGovernment.   

6.2.  Efficiency 

6.2.1.  Efficiency of the related budget 

The eGovernment Action Plan had no dedicated budget, therefore efficiency related to budget 
may  only  be  linked  to  actions  under  the  Action  Plan  for which  spending  has  been  allocated 
under an EU funding scheme. For the same reason, no systematic monitoring of resources has 
been carried out for each of the priorities of the eGovernment Action  Plan. Likewise, across 
the 28 Member States the implementation of the Action Plan has been carried out in a variety 
of different manners on the national level.  

eGovernment is supported through a number of funding programmes (see also under 3.2.2.):  

  The  past  Competitiveness  and 
Programme105 (CIP ICT-PSP);  

Innovation  Programme 

ICT  Policy  Support 

  CEF, Digital Services Infrastrctures (CEF DSI)106;  

  Horizon 2020's Societal Challenge 6107; 

  The past Seventh EU Framework Programme for Research (FP7);  

  The ISA programme108;  

104 Mid-term Evaluation of the e-Government Action Plan 2011–2015, pp. 48 – 49. 
105 http://ec.europa.eu/cip/ict-psp/index_en.htm and https://ec.europa.eu/digital-single-market/en/node/77121 
106 https://ec.europa.eu/digital-single-market/en/connecting-europe-facility 
107 https://ec.europa.eu/digital-single-market/en/ict-enabled-public-sector-innovation-horizon-2020 

36 

 
 
 
 
 
                                                 
  The  European  Structural  and  Investment  Funds  ESIF  (ERDF,  CF,  ESF,  YEI  and 

EAFRD109); before 2014: Structural Funds. 

CIP LSP projects  

The  CIP  ICT  Policy  Support  Programme  funded  the  following  LSPs  projects  (with  the 
approximate total costs, including the 50 percent EU co-funding): STORK (eIdentification – 
26  +  18,6  M  EUR),  eCodex  (e-justice  –  24  M  EUR),  PEPPOL  (e-procurement  –  30,8  M 
EUR), SPOCS (Points of Single Contact – 24 M EUR), ePSOS (e-health – 36,5 M EUR) and 
eSENS  (consolidation  of  results  –  27  M  EUR).  In  the  analysis  below,  the  eHealth  related 
activities and pilot are excluded as they are covered by a separate eHealth Action Plan110.  

The  CIP  Implementation  reports111  provide  overviews  on  the  LSPs.  The  total  cost  of  the 
eGovernment LSPs (excluding ePSOS) is around 150 million EUR. The Commission has co-
funded  around  50  per  cent  of  this  amount  the  other  half  was  invested  by  the  participating 
Member States (and countries associated to the CIP programme). The Commission  report112 
on  "Evaluations  of  the  Competitiveness  and  Innovation  Framework  programme"  referred  to 
the final evaluation113 of the CIP programme: "The evaluation underlined the improvements 
arising  from  the  delegation  of  the  management  of  substantial  parts  of  CIP  to  the  EACI114 
concerning  the  efficiency  of  the  programme  management,  both  in  relation  with  costs  to  the 
Commission  and  efficiency  of  the  services  provided,  as  measured  in  terms  of  number  of 
contracts signed, the period to contract and payment delays." 

The  results  achieved  through  these  pilots  in  the  form  of  re-usable  technical  building  blocks 
that are now starting to be deployed in the EU have been described above in 7.1.2. The work 
contributed towards having key cross-border services online. The participation to these pilots 
was (and is) voluntary of which reason the costs varied among the Member States. The pilots 
typically involved between 10 – 20 Member States.  

The  2013  study  "The  feasibility  and  scenarios  for  the  long-term  sustainability  of  the  LSPs" 
included  a  cost-benefit  analysis  of  the  large-scale  pilots.  Cost-benefits  analysis  calculations 
were underpinned by an evidence based estimation of the potential use levels for the nine core 
service platforms (CSPs) and building blocks (BBs) - eID, eProcurement, eBusiness, eHealth, 
eJustice,  eSignature,  eDocument,  eDelivery.  The  role  of  the  public  sector  in  financing  the 
building blocks and CSP/BBs may diminish where a business for private sector involvement 
is more apparent. 

ESIF 

ESIF funding (as from 2014) for eGovernment can be assessed by extracting the amount of 
potential  investments  (this  does  not  necessarily  reflect  final  investment  figures)  in  the 
intervention  category  "eGovernment  services  and  applications"  (including  e-procurement, 
ICT measures supporting the reform of public administration, cybersecurity, trust and privacy 

108 http://ec.europa.eu/isa/ 
109 European Regional Development Fund, Cohesion Fund, European Social Fund, Youth Employment Initiative 
and European Agricultural Fund for Rural Development 
110https://ec.europa.eu/digital-agenda/en/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century 
111 http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm 
112 COM(2013)2 
113 Final Evaluation of the Competitiveness and Innovation Framework Programme , 2011 
114 Executive Agency on Competitiveness and Innovation, as of 2014, Executive Agency for SMEs (EASME)  

37 

 
 
 
 
                                                                                                                                                         
measures,  e-justice  and  e-democracy)115.  For  2014  Operational  Programmes,  the  amount  is 
3,427 million EUR potential investments.  

Research and Innovation funding 

The  Action  Plan  has  also  led  to  financing  initiatives  under  the  Research  and  Innovation 
programmes of the Commission. In particular, research and innovation actions were targeting 
the understanding the transformation of public administration in engaging more stakeholders 
in the policy making process, the reduction of administrative burden and the service delivery 
of public services. Under the FP7 the programme invested over 75M EUR between 2011 and 
2013 in co-financed research and innovation actions. Since the launch of the new programme, 
H2020, over 27M EUR were invested in co-financed research and innovation actions. 

Finding 11 
The Action Plan had no dedicated budget. Despite this, the Action Plan worked by providing a 
guiding framework for funding. The Action Plan mobilised resources in the Commission and 
Member States (and countries associated to the CIP programme), that co-funded 50 per cent of 
the LSPs to develop solutions and common understanding in several cross-border policies. 

6.2.2.  Costs and benefits to stakeholders 

The efficiency can also be assessed as the capacity of the Action Plan to mobilise resources 
(human and financial) versus the potential savings in the intervention domain. Investments in 
the modernisation of the European public sector have to be seen also in the wider context.  

 

In the EU, public expenditure accounts for almost 50% of GDP and the public sector 
represents about 17% of total employment116.  

  Public administrations have a powerful means to pull innovation; in the EU, in 2010, 
public authorities in the EU spent over € 2 400 billion on supplies, works, and services 
– amounting to around 19% of EU GDP117.  

The potential of these investments for savings can also be seen in different sectors mentioned 
in the Action Plan, for example:  

eIdentification 
It has been assessed that replacing the current paper based identification elements used 
across  the  private  sector  could  reduce  costs  by  90%  if  eID  replaced  them118.  In 
addition, eID is also the most important service to access other digital public services, 
a key enabler. 

e-procurement 
Contracting authorities and entities that have successfully completed the transition to 
e-procurement commonly report savings between 5 and 20%; whereby experience also 
shows  that  investment  costs  related  to  transition  to  e-procurement  can  be  rapidly 
recouped.  Given  the  size  of  the  total  procurement  market  in  the  EU,  each  5%  saved 

115 Tool available at http://ec.europa.eu/regional_policy/en/funding/    
116 Annual Growth Survey 2013, COM(2012) 750 final 
117 DG Internal Market and Services: Management Plan 2013, 2012 
118 The economics of identity, June 2014 

38 

 
 
 
 
 
 
                                                 
could return around €100 billion to the public purse119. Based on the extrapolation of 
the national evaluations of benefits, the EC estimates that the adoption of e-invoicing 
in  public  procurement  across  the  EU  could  generate  savings  of  up  to  €2.3  bn120. 
Switching from paper to fully automated e-invoicing can cut the costs of receiving an 
invoice from €30-50 to €1121. 

The once-only principle 
The extension of this principle, in compliance with data protection legislation, would 
likely generate an annual net saving at the EU level of around EUR 5 billion per year 
by 2017122. 

As  reported  in  the  mid-term  evaluation,  for  the  Member  States  public  administrations, 
achieving this type of financial gains and improving the efficiency of administrative processes 
were  among  the  main  drivers  for  eGovernment  projects,  even  more  so  given  the  current 
economic  climate  and  budget  pressures.  Financial  savings  are  expected  to  translate  into 
efficiency  gains  in  a  variety  of  ways,  including  a  reduction  of  administrative  burden  and 
savings in opportunity costs.  

The mid-term evaluation found in the eGovernment case studies that financial impacts are not 
always clear before implementing a project. In general, the mid-term evaluation found that the 
decision to implement a project depended in only a few cases on a clear business case.  

At the same time, the mid-term evaluation found that it was uncommon to have users of the 
service paying a fee for it. In fact, financial benefits for the public sector are expected to come 
mostly  from  savings  generated  by  reducing  the  production  costs  for  a  service  and/or  by  the 
rationalisation of public expenditure. This meant that for the other stakeholders, citizens and 
businesses, the Action Plan activities did not incur direct costs and they were able to benefit 
from, for example, improved cross-border availability of eGovernment services. 

Finding 12 
Different sectors report significant potential savings from the use of eGovernment. Given the  
important size of public expenditure in these and other sectors, the resources that the Action 
Plan  mobilised  towards  the  modernisation  and  increasing  efficiency  of  the  public 
administrations seem justified. 

6.3.  Relevance 

The  public  sector  represents  a  significant  and  unique  part  of  Europe’s  economy,  has  a  key 
responsibility for delivering citizens' quality of life and is also a vital actor in efficiently and 
effectively delivering policy objectives.  

Economic  and  budgetary  pressures  force  governments  to  be  ever  more  efficient,  effective, 
reduce  costs  and  be  more  competitive  in  a  multi-polar  world123.  The,  contribution  of 

119 A strategy for e-procurement, COM(2012)0179 final; http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0179:FIN:EN:PDF  
120 End-to-end e-procurement to modernise public administration, COM(2013)0453 final 
121  E-invoicing  in  public  procurement:  another  step  towards  end-to-end  e-procurement  and  e-government  in 
Europe (Press Release, June 2013) 
122 Study on eGovernment and the Reduction of Administrative Burden, 2014 
123  Excellence in public administration for competitiveness in EU Member States, 2012 

39 

 
 
 
                                                 
eGovernment  to  jobs  and  growth  remains  a  very  relevant  high-level  objective.  Repeated 
analysis  during  the  European  Semester  has  shown  that  the  modernisation  of  public 
administrations  can  contribute  to  tackling  economic  challenges.  The  recent  Annual  Growth 
Surveys124  recognise  the  importance  of  a  modern  and  efficient  public  administration  for  a 
business-friendly environment and to ensure fast and high-quality services for citizens.   

Additionally, in many EU countries the work to implement digital end-to-end services and to 
achieve  cross-border  interoperability  continues,  so  further  pursuing  the  eGovernment 
activities remains relevant. Online public services continue in relevance for reducing business 
costs  and  increasing  the  efficiency  and  the  quality  of  the  services  provided  to  citizens  and 
businesses125.  According  to  a  recent  study126,  if  interactions  with  public  authorities  can  be 
made  as  transparent,  as  fast  and  as  cost-efficient  as  in  the  private  sector,  then  the  potential 
benefits will materialise. 

6.3.1.  Relevance: User Empowerment 

As noted earlier, according to the 2015 eGovernment Benchmarking report, user centricity is 
confirmed  as  the  most  advanced  indicator.  However,  many  Member  States  are  not  focusing 
enough on the quality of the users' experience.  

According  to  the  mid-term  evaluation  (on  Action  2),  most  Member  States  have  introduced 
personalised services in their One-Stop-Shops for business and for citizens, as well as in the 
area of health and tax services. The relevance of one-stop-shops continues with new needs for 
cross-border approaches..  

The  Action  Plan  objectives  on  re-use  of  public  sector  information  have  been  found  to  be 
relevant.  The  Action  Plan's  objectives  on  user  needs,  inclusive  services,  collaborative 
services, transparency and the involvement of citizens and businesses in policy-making were 
assessed to be appropriate for user empowerment.  This is also due to advancing technology 
that brings, for example, increased interactions through new tools and social networking.  

Transparency  and  participation  continue  as  important  principles  in  the  modernisation  of 
public administrations. According to the eGovernment benchmarking study users are provided 
with information about the duration of the process in only 39% of measured cases. The figure 
for  maximum  delivery  timelines  for  government  is  46%.  At  the  same  time,  administrations 
rarely give an account of their service performance (in 38% of cases). Only 1 in 3 websites 
inform visitors about their ability to participate in policy-making processes127.  

Users of Member States' justice systems are an example of user empowerment through new 
technological  tools  Here,  there  are  currently  gaps  regarding  the  deployment  of  e-justice  or 
more  specifically  the  use  of  ICT  at  courts,  for  example,  concerning  the  possibility  to 
communicate  electronically  between  the  courts  and  the  parties  as  well  as  the  use  of  ICT  in 
court management system as identified by the EU Justice Scoreboard128.  

124 COM(2014) 902 and COM(2015) 690  
125 A Digital Single Market Strategy for Europe – Analysis and Evidence, SWD(2015) 100 
126 Study on eGovernment and the Reduction of Administrative Burden, 2014 
127 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015. 
Insight report.  
128 The forthcoming EU Justice Scoreboard 2016.  

40 

 
 
 
 
 
                                                 
The online public consultation  also asked how stakeholders should be enabled to contribute 
to, make proposals on and publicly exchange their views on new initiatives emerging under 
the  eGovernment  Action  Plan.  Most  respondents  said  that  using  an  online  social  media 
platform was a good idea. Using a social media platform was viewed as useful to reach out to 
a large audience, citizens as well as businesses.  

Finding 13 
The  eGovernment  Action  Plan  priorities  on  user  empowerment  have  been  found  to  be 
relevant. However the Action Plan could benefit from stronger user empowerment through a 
mechanism to enable stakeholders to contribute to the initiatives of the Action Plan.  

The  Action  Plan  objective  on  the  electronic  service  to  support  citizens'  initiatives  was 
achieved.  However,  the  general  goal  of  involvement  of  citizens  and  the  rights  of  citizens 
continues to be relevant. According to the Citizenship Report 2013, only one in three citizens 
(36%) say they are well informed about their EU rights and just under a quarter (24%) feel 
fairly or very well informed about what they can do when their EU rights are not respected129.  

Building on the achievements of the Action Plan on the sub-priority on the re-use of public 
sector information further work was relevant and continues to be so. The mid-term evaluation 
finding of the lack of agreement on the common set of Public Sector Information (PSI) re-use 
indicators can also feed into future work, including in possible future adaptations of the PSI 
Directive130. The policy evaluations of the PSI Directive131 and the Infrastructure for Spatial 
Information in the EU (INSPIRE) Directive132 demonstrated the need for public authorities to 
continue improving the sharing and re-use of their data.  

Despite  challenges  remaining  in  implementing  the  INSPIRE  Directive,  both  the  Member 
States and the consulted stakeholders reported qualitative benefits ranging from more efficient 
information services to citizens and the improved collaboration between public authorities, as 
well  as  building  up  skills,  to  cost  savings  in  various  areas  and  improved  data  quality. 
Estimates are available for a number of countries133,134. Other illustrations are more effective 
environmental  risk  management135,  improved  cross-border  and  cross-sector  collaboration, 
spatial  planning  and  lower  costs  for  environmental  impact  assessments136.  Moreover, 
synergies137  were  reported  between  the  EU  and  national  strategies  on  Open  Data138  and 
eGovernment where the availability of INSPIRE spatial data and services is used increasingly 
for a wide range of government-to-government and government-to-public applications139.  

129 Flash Eurobarometer 365, European Union citizenship, 2013  
130 Directive (2013/37/EU), article 13.1. 
131 Review of recent studies on PSI re-use and related market developments, 2011 
132 Assessment of the Re-use of Public Sector Information (PSI) in the Geographical information, Meteorological 
Information and Legal Information Sectors, 2008 
133 UK Location Programme - Benefits Realisation Strategy Final, 2012 - estimates the quantifiable INSPIRE 
benefits at 70-130 m£/year to UK environmental protection activities and the quantifiable benefits across UK 
government departments at 470-510 m£/year. 
134 Costs-benefits analysis INSPIRE in The Netherland, 2009 
135 Reducing environmental risk through INSPIRE, UK Environment Agency, 2013 
136 The use of spatial Data for the Preparation of Environmental Reports in Europe, 2010 EUR24327 EN - 2010  
137 For example: 57% of INSPIRE data is OPEN in Finland and served through INSPIRE services (INSPIRE, 
MIG, National Implementation Seminars); Ashfield District Council (INSPIRE and Open Data, 2015) publishes 
data under the European INSPIRE Directive and as Open Data. 
138 COM(2011) 882 final 
139 For example: INSPIRED eGovernment Apps - Geopunt, the Flemish Geoportal. 

41 

 
 
 
 
 
                                                 
The  Commission  has  put  in  place  an  Open  Data  Portal  where  re-usable  data  from  the  EU 
institutions is are referenced140 and has currently more than 8000 datasets available. As a next 
step to opening up its own data, the Commission is actively promoting the discoverability and 
re-use  of  PSI  in  a  cross-border  and  cross-lingual  context,  notably  by  the  creation  of  the 
European Data Portal141, a digital service infrastructure under CEF. The importance of open 
data and re-use of data continues relevant for contributing to transparency and for economic 
growth. According to a recent study "Creating Value through Open Data"142 the market size 
for Open Data is expected to increase by 36,9 % between 2016 – 2020 to a value of 75,7 bn 
EUR  in  2020.  The  study  also  forecasts  and  a  32  %  growth  in  open  data  jobs  in  the  5-year 
period and important cost savings.  

The relevance of the Action Plan sub-priority on transparency, including data protection has 
continued  to  increase.  In  addition  to  the  consent-based  approaches,  users  demand  to  be  in 
control of their data and provide only the amount of data necessary for a certain transaction. 
The  future  General  Data  Protection  Regulation  on  which  co-legislators  reached  a  political 
agreement143  in  December  2015  updates  and  modernises  the Data  Protection  rules  currently 
contained  in  the  1995  Data  Protection  Directive  and  will  have  an  effect  also  on  the  public 
sector.  

Finding 14 
There  are  continued  needs  in  the  Action  Plan  priorities  on  citizens'  rights,  re-use  of  public 
sector information, transparency, participation and data protection.    

6.3.2.  Relevance: Internal market 

The  Action  Plan  objectives  of  seamless  services  for  businesses,  personal  mobility  and  EU-
wide  implementation  of  cross-border  services  still  correspond  to  the  needs  of  the  single 
market. The Digital  Single Market of over 500 million people cannot be completed without 
public services. In 2014, there were 17.9 million persons who had been born in a different EU 
Member  State  from  the  one  where  they  were  resident144.  From  the  perspective  of  a 
citizenship, the figure is still high, in 2014 more than 14 million EU citizens lived in another 
EU  Member  State  than where  they had  their  citizenship145.  The  trends  of  intra-EU  mobility 
are steadily increasing146. According to a 2013 study147, over 1,5 million citizens and 314,000 
businesses are likely to use cross-border eGovernment services each year in 2020. The same 
study  estimates  that  the  potential  volume  of  the  top  five  public  services  for  businesses  is 
2,490,000 cross-border users annually.  

The mid-term evaluation of the Action Plan noted that under half of Member States had rolled 
out LSP projects, which highlights the need for continued action with the CEF DSIs. 

140 http://open-data.europa.eu/ 
141 http://www.europeandataportal.eu 
142 Creating Value through Open Data: Study on the Impact of Re-use of Public Data Resources, 2015 
143 http://ec.europa.eu/justice/data-protection/reform/index_en.htm 
144 Eurostat, Migration and migrant population statistics, 2015  
145 Eurostat, Foreign citizens living in EU Member States, 2015  
146 Study on Evaluation of the impact of the free movement of EU citizens at local level, 2014  
147  Study  on  Analysis  of  the  Needs  for  Cross-Border  Services  and  Assessment  of  the  Organisational,  Legal, 
Technical and Semantic Barriers, 2013, p. 45 and figure 26. 

42 

 
 
 
 
 
 
                                                 
There  are  also  increasing  needs  for  cross-border  company  procedures  and  the  related 
objectives are still highly relevant, for example in the areas of e-procurement, company law, 
e-justice, Points of Single Contact and information portals:  

E-procurement 

As outlined above, transition to the e-procurement presents a potential of significant savings 
enabled  by  streamlining,  simplification  and  opening  up  of  public  procurement  procedures. 
Only adoption of e-invoicing in public procurement across the EU could generate savings of 
up to €2.3 billion148. The work in this area delivered beyond the objectives of the Action Plan. 
In 2014, the European Commission adopted new public procurement directives, which enter 
into force in April 2016149 and make e-procurement gradually compulsory, with e-Submission 
being  mandatory  for  all  public  buyers  as  of  October  2018.  The  policy  aiming  at  supporting 
transition to full e-procurement is also building on a 2012 Communication on "A strategy for 
e-procurement" and a 2013 Communication on "End-to-end procurement to modernise public 
administration"150. A wide range of support measures have been put in place to facilitate the 
transition to e-procurement in all Member States. This will allow the businesses, in particular, 
SMEs,  to  benefit  as  early  as  possible  from  new  business  opportunities  opening  up  at  the 
European  Single  Market  thanks  to  the  electronic  public  procurement,  in  particular 
simplification of the pre-qualification phase.  

Company law 

The  objective  of  the  'internal  market'  priority  of  the  action  plan  aimed  at  facilitating  the 
running  of  companies  anywhere  in  the  EU.  The  Digital  Single  Market  Staff  Working 
Document151  noted  that  existing  rules  under  company  law  do  not  sufficiently  integrate  the 
benefits  of  digital  technologies.  Companies  are  still  faced  with  paper-based  formalities.  For 
example,  online  registration  of  companies  is  only  possible  in  16  Member  States.  Though  a 
number of measures have been taken at national level, a national approach does not remove 
the obstacles that companies face if they consider setting  up or  operating a company across 
borders152.  In  this  context,  the  Single  Market  Strategy153  further  mentioned  that  the 
Commission  will  consider  further  ways  of  achieving  simpler  and  less  burdensome  rules  for 
companies — while continuing to act against letterbox companies — including making digital 
solutions  available  throughout  a  company’s  lifecycle,  in  particular  in  relation  to  their 
registration and to the filing of company documents and information. 

E-justice 

The action plan foresaw in the 'EU wide implementation of cross border services' activities in 
the  eJustice  area  as  there  are  continuing  needs  in  cross-border  e-justice.  The  Digital  Single 
Market strategy (Staff Working Document) noted that European e-Justice154 facilitates access 
to  justice  and  cross-border  judicial  proceedings  and  makes  it  easier  for  citizens  to  find  a 
lawyer/notary  in  the  EU  and  for  businesses  to  search  for  insolvent  entities  through 
interconnected  insolvency  registers.  However,  in  a  cross-border  context  issues  identified  at 

148 COM(2013) 453 final 
149 http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm  
150 http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm 
151 SWD (2015) 100 final  
152 SWD(2014) 124 final, p. 27 and examples of differing national reforms in company law p. 21 et seq. 
153 COM(2015)550, p. 5. 
154 The European e-Justice Portal is available at: https://e-justice.europa.eu/home.do 

43 

 
 
                                                 
national level are amplified, as cross-border cooperation in this area entails that the Member 
States are at equal levels of development. 

In this context, the work started by the CIP pilot e-CODEX remains highly relevant. Although 
the  pilot  project  has  created  a  strong  backbone  for  communications  between  justice 
administrations, which can be used for many different purposes, extension of this work will 
be  needed  in  three  dimensions.  Firstly,  the  number  of  Member  States  which  currently 
participate in this work remains limited, and should be expanded to cover all Member States. 
Secondly,  the  e–CODEX  backbone  provides  many  opportunities  to  further  support  cross-
border  exchanges,  including  between  judicial  professionals  such  as  laywers,  notaries  and 
judicial  officers.  Thirdly,  expansion  can  also  take  place  in  terms  of  the  number  of  judicial 
cooperation  instruments  covered.  A  first  pilot  project  implementing  this  technology  on  the 
European e-Justice Portal will go live in 2016. Further work on this topic could include efforts 
to  realise  the  cross-border  elecronic  service  of  documents.  Furthermore,  as  of  2017,  the 
business  registers  interconnection  system,  which  will  be  composed  of  Member  States' 
business registers, a European central platform, and the European e-Justice portal serving as 
the  European  electronic  access  point  will  facilitate  access  to  information  on  companies  for 
citizens and businesses throughout the EU155. In addition, the interconnection of national land 
registers will be developed to go live on the European e-Justice Portal in the course of 2017. 

Points of Single Contact 

Extending  the  scope  of  the  Points  of  Single  Contact  as  indicated  in  the  Action  Plan  could 
further  contribute  to  simplification,  savings  for  public  administration  and  more  coherent 
approach  in  providing  information  and  e-services  to  businesses.  Information  gaps  increase 
costs  for  business,  in  particular  for  SMEs156.  Procedural  streamlining  via  Points  of  Single 
Contact (PSC) could generate up to 0.15% of GDP in the medium run (5-year horizon) and up 
to 0.21% of GDP in the long run157.  

In  line  with  Action  Plan  action  18  to  transit  into  "Second  Generation"  Points  of  Single 
Contact (PSC), that would function as fully-fledged eGovernment centres, a first assessment 
of  the  PSCs  was  carried  out  in  2011-2012158.  The  results  revealed  a  hybrid  landscape,  with 
some  PSCs  more  advanced  than  others  in  providing  the  information  and  services  required. 
This  first  assessment  resulted  in  the  Communication  on  the  implementation  of  the  Services 
Directive  "A  partnership  for  new  growth  in  services  2012-2015”159.  This  Communication 
encouraged Member States to develop by the end of 2014 Second Generation Points of Single 
Contacts,  which  should  “1)  cover  all  procedures  during  the  business  life  cycle,  2)  be 
multilingual, and 3) be more user-friendly”.  

These recommendations were subsequently included into the PSC Charter160, endorsed by the 
Council in 2013. The Charter encourages the Member States to bring additional features into 
their PSCs so that the latter respond more effective to the business life cycle of a company in 
the services sector.   It also  includes the benchmarks facilitating the assessment of the PSCs 
with a view to ensure their continuous improvement and ultimately, to provide better public 

155 In line with Directive 2012/17/EU. 
156 European Parliament study, A European Single Point of Contact, 2013; European Commission,  High Level 
Group on Business Services - Final Report, 2014 
157  SWD(2012) 148 final 
158 Study on The functioning and usability of the Points of Single Contact under the Services Directive, 2012   
159 http://ec.europa.eu/internal_market/services/docs/services-dir/implementation/report/COM_2012_261_en.pdf 
160 http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf 

44 

 
 
                                                 
online services to business. The assessment of the PSCs performance against of the criteria set 
out in the Charter was carried out in 2014 and published in 2015 in a study161. The results of 
the  study  indicate  that  PSCs  are  still  far  from  delivering  what  is  expected  from  them.  The 
overall  score  of  the  PSCs  on  the  EU28+  level  indicates  moderate  performance  with 
considerable  room  for  improvement.  Although  PSCs  provide  basic  information  on  general 
requirements,  information  on  sector  specific  requirements  is  insufficient.  The  study  reveals 
also  a  considerable  gap  in  online  availability  of  the  procedures  associated  with  general 
requirements  and  the  ones  associated  with  specific  requirements.  Although  the  general 
requirements are fully transactional on about half of the PSCs. The access to PSC services by 
foreign users were assessed as poor. The transactionality of online procedures for foreigners is 
one of the weakest points of the PSC. 

Portals and networks 

Furthermore, there are a number of existing portals and networks for finding information and 
initiate  and  complete  transactions  with  Member  States’  administrations  across  the  EU 
(including  Points  of  Single  contact,  Product  contact  points,  Construction  product  contact 
points,  Your  Europe,  Your  Europe  Advice,  SOLVIT  and  Enterprise  Europe  Network). 
However, currently Single Market-related information, problem-solving mechanisms, contact 
points and procedures do not operate as a whole but are dispersed and not sufficiently inter-
connected – both at EU and national level. It is therefore difficult for users to find the right 
information  and  assistance  required.  This  implies  that  higher  transaction  costs  have  to  be 
incurred before engaging in a cross-border activity. More synergies between actions, such as 
providing information, assistance and e-Government services, can be achieved at EU, national 
and  regional  level  if  efforts  are  better  coordinated.  An  area  for  further  work  is  to  link  up 
relevant  EU  and  national  content,  services  and  procedures  and  offer  users  a  streamlined, 
comprehensive  portal  to  find  information  and  initiate  as  well  as  complete  transactions  with 
Member  States’  administrations  across  the  EU  (Single  Digital  Gateway  as  included  in  the 
Digital Single Market strategy).  

The online public consultation (Annex 2) reflected a need for increasing awareness about the 
on-going  work  and  availability  of  cross-border  public  services.  The  majority  of  the 
respondents did not know whether the current eGovernment Action Plan has improved cross-
border eGovernment services overall or did not agree that the Action Plan has improved them.  

Finding 15 
The intra-EU mobility is increasing and citizens and businesses would benefit from connected 
cross-border  public  services  in  the  internal  market.  The  Action  Plan  had  objectives  for 
seamless  services  for  businesses,  personal  mobility  and  EU-wide  cross-border  services.  The 
relevance of these eGovernment services for the internal market continues to be high. At the 
same time, there is not awareness about the cross-border eGovernment services.  

6.3.3.  Relevance: "Efficiency and Effectiveness of Governments and Administrations" 

Under the sub-priority on reduction of administrative burden, the eGovernment Action Plan 
initiated the work towards the implementation of the once-only principle.  

161 The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015  

45 

 
 
 
                                                 
This objective is ever more relevant as a policy priority. The European Council Conclusions 
of October 2013 stated that "EU legislation should be designed to facilitate digital interaction 
between citizens and businesses and the public authorities. Efforts  should be made  to  apply 
the  principle  that  information  is  collected  from  citizens  only  once,  in  due  respect  of  data 
protection rules." 

Citizens and businesses are still all too often required to submit information that governments 
already  hold.  Only  in  48%  of  cases  do  public  administrations  reuse  information  about  the 
citizen  that  is  already  in  their  possession  without  asking  for  it  again162.  According  to  the 
eGovernment  benchmarking  report  2015163,  the  use  of  Authentic  Sources  has  actually 
decreased.  The  use  of  authentic  sources  means  that  public  administrations  get  data  from 
sources  such  as  National  register,  Tax  registers,  Company  registers  etc.,  instead  of  asking 
citizens and businesses to provide the data (again). The decrease in their use is disappointing 
as it also indicates that European governments still struggle to re-use data. As a result, citizens 
and businesses are still all too often required to submit information that governments already 
hold. 

The amount of data that is pre-filled in Public Services' online forms (one way to implement 
the once-only principle) varies to a great deal within the EU164:  

Figure 8 Pre-filled forms as an indicator of once-only. Amount of data pre-filled in public services forms in seven life 
events. (Digital Economy and Society Index 2016.) 

The  percentage  of  individuals  who  used  the  Internet  to  submit  completed  forms  to  public 
authorities was on average 33% of Internet users within the EU165 in 2014. In 2015 this was 
slightly lower at 32%166: 

162  Delivering  the  European  Advantage?  How  European  governments  can  and  should  benefit  from  innovative 
public services, 11th eGovernment Benchmark Report, 2014, p. 49 
163  Future-proofing  eGovernment  for  the  Digital  Single  Market,  eGovernment  Benchmarking  report,  2015, 
Background report, p. 35 
164 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015 

46 

 
 
 
 
 
                                                 
Figure 9 Percentage of Internet users submitting completed forms to public authorities. (Digital Economy and Society 
Index. 2016.) 

The  eGovernment  benchmarking  report  measures  the  availability  of  "authentic  sources", 
which are base registries used by government to automatically validate or fetch data relating 
to  citizens  and  businesses.  It  implies  governments  re-use  data  to  deliver  and  facilitates  the 
implementation  of  the  once-only  principle.  The  figure  of  2013/2014  in  the  eGovernment 
benchmarking167 assessment for EU28+ was 45%, meaning that in 45% of the assessed cases 
do public administrations reuse information that is already in their possession without asking 
for it again.  

In  consideration  of  these  findings  and  the  potential  savings,  implementing  the  once-only 
principle across borders continues to be a highly relevant priority in Europe and could further 
contribute towards the efficiency of the European public sector.  

The  relevance  of  the  principle  was  acknowledged  also  by  the  European  Parliament  in  its 
report  Towards  a  Digital  Single  Market  act  in  January  2016168:  "a  step-by-step  sectoral 
approach  to  apply  the  ‘once  only  principle’  in  public  administrations  according  to  which 
citizens  and  businesses  should  not  be  asked  for  information  already  provided  to  a  public 
authority, whilst ensuring citizens' privacy and a high level of data protection - -;" 

In  addition  to  the  once-only  principle,  the  sub-priority  of  the  Action  Plan  on  improving 
organisational processes remains relevant. The eCommission Action Plan has led also the EC 
towards,  amongst  other  things,  electronic  procurement.  The  digitalisation  of  procedures 
(including digital by default) is also a means for administrative burden reduction and remains 
relevant. Toolboxes and innovative architectures are also important measures where the EC to 
promote  and  present  pilots  and  practices  on  the  quality,  efficiency,  transparency  of  public 

165 Eurostat, ICT survey of Households and Individuals, 2014; European Commission, Digital Agenda 
Scoreboard – the indicator has a 12-month reference period and concerns the use of web applications for 
uploading completed forms or transmitting web forms with details filled in directly. 
166 EC, Digital Economy and Society Index DESI, February 2016  
167 Future-proofing eGovernment for the Digital Single Market, eGovernment Benchmarking report, 2015, p. 35. 
168 European Parliament resolution of 19 January 2016 on Towards a Digital Single Market Act 

47 

 
 
 
 
                                                                                                                                                         
services.  The  Joinup  collaborative  eGovernment  platform  has  grown  to  be  a  platform  for 
thousands of users.    

The mid-term evaluation also recommends that although the action under sub-priority Green 
Government  was  not  carried  out  it  should  be  reconsidered:  "Looking  at  eGovernment  as  an 
enabler  to  lower  the  carbon  footprint  of  administration  makes  sense  in  looking  at 
eGovernment in a more holistic way." Furthermore, the mid-term evaluation recommends that 
the  staff  exchange  programme  between  administrations  in  different  Member  States,  though 
not  carried  out,  could  be  re-launched  and  it  could  be  an  excellent  learning  and  sharing 
accelerator for Member States' eGovernment agencies. 

Finding 16 
The  assessment  of  the  use  of  authentic  sources  by  the  public  sector  shows  the  continued 
relevance  of  the  once-only  principle.  The  principle  is  also  receiving  important  political 
support.  Improving  organisational  processes  remains  a  relevant  goal,  also  for  the  EC.  The 
Action Plan's Green Government sub-priority and the staff exchange programme could be re-
considered. 

6.3.4.  Relevance: Pre-conditions 

This Action Plan objectives on technical pre-conditions is still very pertinent. However, with 
many of the technical and legal pre-conditions achieved during the eGovernment Action Plan, 
the  benefits  to  citizens,  businesses  and  public  administrations  themselves  need  to  be 
demonstrated.  The  take-up  of  these  solutions  could  be  increased,  making  a  real  impact  on 
people's lives - reducing administrative burden, facilitating mobility in the EU and improving 
the user experience when dealing with public administrations.  

The  eIDAS  Regulation  on  electronic  identification  and  trust  services  for  electronic 
transactions in the internal market was an objective of the Action Plan. It ensures that people 
and businesses can use their own national electronic identification schemes (eIDs) to access 
public services in other EU countries where eIDs are available. In addition, it helps to create a 
European  internal  market  for  eTS  (Trust  Services).    The  take  up  of  these  services 
demonstrates  the  important  potential  for  the  future  and  the continued  high  relevance.  In  the 
end of 2015, 15 countries, including 13 Member States, were connected to the eID network 
via  STORK  (see  further  details  in  Annex  4  under  action  37).  In  addition,  the  potential  of  a 
trusted identification mechanism goes also beyond the public sector, such as for the financial 
sector.  

The  eGovernment  Benchmarking  Report  2015  suggests  that  building  blocks,  such  as 
electronic  identity  (eID),  that  enable  online  authentication  of  persons  and  companies,  or 
authentic sources  that encompass registers of personal and other  data, and that  would allow 
re-use  of  that  data  for  other  service  processes,  will  boost  cross-border  and  national  online 
services.  

A  lot  was  achieved  under  the  interoperability  objectives  of  the  Action  Plan.  However, 
working towards interoperability between Member States at European level continues to be a 
relevant objective. Interoperability is needed to enable public services to work across borders. 
1,500,000 citizens and 300,000 businesses are likely to use cross-border online services each 

48 

 
 
 
year  by  2020169  which  will  presuppose  the  existence  of  interoperable  solutions  for  which 
reason the creation of interoperable solutions will continue to be of high relevance.  
As  stated  in  the  DSM  staff  working  document,  the  lack  of  interoperability  among  public 
entities and private operators restricts the potential for digital end-to-end services, One Stop 
Shops,  the  once-only  principle,  the  single  data  entry  principle,  the  transparency  of  public 
services and the full exploitation of public open data. Further progress is needed to improve 
the interoperability of systems for cross-border delivery of goods and services, as well as for 
the  mobility  of  people  and  businesses  and  for  cooperation  between  public  authorities,  at 
national and EU level.  

In the area of public services, the European Interoperability Framework, adopted by the EC in 
2010170, promotes and supports the delivery of European public services by fostering cross-
border and cross-sectoral interoperability. The majority of the Member States have transposed 
this  Framework,  which  has  brought  a  common  understanding  of  the  basic  requirements  for 
interoperability  between  public  services.  This  common  understanding  is  still  relevant  and 
requires updating and extending with other concrete and practical instruments to possibly be 
shared  by  national  administrations  such  as  the  European  Interoperability  Reference 
Architecture (EIRA) and the European Interoperability Cartography (EUCart)171. The revision 
of the EIF is included in the list of actions of the Digital Single Market Strategy. 

The  ISA2  programme,  recently  adopted  by  the  co-legislators,  is  a  key  instrument  for  the 
development  of  interoperability  of  public  administrations  in  Europe  and  between  public 
administrations and businesses and citizens. 

Standardisation  is  another  aspect  that  remains  relevant  in  the  future.  Digitisation  of 
administrative formalities offers opportunities to, for example, standardise the documents that 
businesses  have  to  present  to  national  authorities  in  different  Member  States,  yielding 
additional cost savings172.  

Finding 17 
While many of the technical and legal pre-conditions were achieved during the eGovernment 
Action Plan, there is a continued need and relevance to pursue the re-use of common solutions 
for  rationalisation,  savings  and  interoperability  and  to  increase  the  take-up  both  in  cross-
border and national eGovernment services.  

6.4.  Coherence 

The  EU  eGovernment  Action  Plan  was  based  on  a  collaborative  design  by  the  EC  and 
Member  States,  starting  from  a  common  vision  (Malmö  Declaration),  containing  agreed 
policy  objectives,  a  monitoring  mechanism  and  holistic  approach  with  the  relevant  EC 
services working together. This wide coordinated approach helped to ensure coherence right 
from the beginning. This coherence was further strengthened by the inclusion of priorities of 
the Malmö Declaration and Action Plan in national eGovernment strategies.  

169  Study on Analysis of the Needs for Cross-Border Services and Assessment of the Organisational, Legal, 

Technical and Semantic Barriers, 2013 

170  COM(2010) 744 final 
171   The Interoperability Solutions for European Public Administrations (ISA) programme (2010-2015) 

monitored and supported the EIF implementation in Europe; it was followed by the ISA² programme 

172  Digital Single Market Communication 2015. Staff Working Document, SWD (2015) 100 final. 

49 

 
 
 
                                                 
The various parts and actions of the Action Plan included several different policy areas of the 
EU.  Coherence  at  the  time  of  the  Action  Plan  preparation  was  focusing  on  identifying 
impactful areas where the introduction of ICT and related interoperability measures across the 
EU could yield the greatest impact, for example on eID and eProcurement. The coherence of 
the numerous different areas together was supported through the holistic governance structure 
of the Action Plan with the involvement of various Commission services.  

The governance through the different groups with Member States, including the eGovernment 
High  Level  Expert  Group,  CEF,  H2020,  FP7,  CIP  and  ISA  Committees  contributed  to  the 
avoidance of overlaps. The governance activities provided for regular meetings and exchanges 
of experience between Member States and the EC.  

Finding 18 
The wide coordinated approach of the preparation of the Action Plan contributed to coherence 
right from the beginning. The coherence benefited further from a political agreement between 
the  Member  States  (the  Malmö  Declaration).  Coherence  was  supported  also  by  the 
governance structure of the Action Plan. 

In  order  to  achieve  a  snowball  effect  on  investments  the  Action  Plan  successfully  made 
coherent  use  of  different  funding  instruments  related  to  the  eGovernment  Action  Plan.  The 
work carried out on key enablers under the various programmes aimed also at the subsequent 
deployment  of  mature  services  under  the  CEF  Telecom  programme.  The  ISA  programme 
played also a key role in ensuring the maturity and implementation of enablers piloted and/or 
developed under the H2020 and the CIP programme. The uptake could then be initiated with 
funding from the CEF Telecom programme. The figure below demonstrates the moving from 
CIP pilots towards deployment in CEF.  

Figure 10 Coherence through CEF. Moving from piloting to roll-out; from large-scale pilots to the CEF. 

For the ESIF funding, an ex-ante conditionality on Digital growth173 was established to ensure 
coherence  of  the  investments  with  other  EU  or  national  funding  source.  This  conditionality 

173 Guidance on Ex ante Conditionalities for the European Structural and Investment Funds, 2014 

50 

 
 
 
 
                                                 
aims  to  foster  the  development  and  implementation  of  national  and  regional  digital  growth 
measures  and  to  assess  their  consistency  with  the  Digital  Agenda  for  Europe's  goals  that 
include e-government. In practice this meant checking against the eGovernment Action Plan 
priorities to ensure complementarity. 

Finding 19 
The  different  funding  mechanisms  for  eGovernment  key  enablers  aimed  also  at  the 
subsequent  deployment  of  mature  services  under  the  CEF  Telecom  programme.  This 
coordination  of  funding  mechanisms  contributed  to  coherent  funding.  For  the  ESIF  funding 
on eGovernment, coherence with the Action Plan was aimed to by assessing the consistency 
of the planned measures with the Digital Agenda goals, including eGovernment.  

Coherence  analysis  can  also  assess  the  eGovernment  related  activities  of  the  28  Member 
States. In the eGovernment Action Plan, the Member States were responsible for 10 actions 
while seven actions were under the shared responsibility of the Member States and the EC.  

In the Commission, several different policy areas were involved in the eGovernment Action 
Plan.  The  different  services  took  part  also  in  the  eGovernment  Action  Plan  governance 
activities,  including  the  Member  States  groups.  This  increased  the  coherence  between  the 
different priorities of the Action Plan.  Public sector modernisation was discussed in various 
fora and considering different aspects during the Action Plan duration.   

However, as the eGovernment Action Plan was based on voluntary activities of the Member 
States,  full  coherence  could  not  be  assured,  also  because  of  the  different  implementation 
approaches used in the Member States.  

The  aim  was  not  necessarily  to  achieve  full  coherence  as  various  other  initiatives  had  to  be 
considered. On the international level, the EC followed the policies of the UN and the OECD 
and  the  policies  of  third  countries  that  sometimes  were  also  taking  aboard  the  Commission 
approaches.  The  Member  States  also  followed  and  participated  in  various  international 
eGovernment activities.  

Finding 20 
A  wide  variety  of  initiatives  at  various  levels  need  to  be  considered  in  the  assessment  of 
coherence of the Action Plan  internally  and externally. For this reason an assessment of the 
Action Plan's coherence was not fully achievable.  

6.5.  EU-added value  

The  EU eGovernment Action Plan 2011-2015 as  well as its predecessor  have  been political 
instruments for the Commission to advance the agenda of Public Sector Modernisation across 
the EU. They have been supporting European coordination, collaboration and joint actions on 
eGovernment,  helping  to  use  public  resources  more  efficiently  and  reducing  public 
expenditures by coordinating and pooling public and private resources. The Action Plan also 
helped inside the Commission to provide a political context and objectives for many different 
policy areas and helped to focus investments from different programmes. 

The Action Plan has also been an example for establishing national eGovernment strategies in 
the Member States. The Action Plan has helped to coordinate investments, share and re-use 
infrastructures,  processes,  data,  resources,  content  and  tools  to  avoid  duplication  and  waste 

51 

 
 
 
 
 
 
and to speed up the roll-out of eGovernment. These types of initiatives continue to be required 
at EU level.  

Some  of  the  actions  of  the  Action  Plan  stem  from  mandatory  requirements  set  down  in  EU 
law,  which  went  through  the  subsidiarity  and  proportionality  test  in  the  legislative  process. 
Here the Action Plan helped to support the collaboration between the Member States. On the 
other hand, some actions were relevant to or contributed towards new legislation, examples of 
this  are  e-procurement  (public  procurement  legislation174)  and  eID-activities  leading  to  the 
eIDAS Regulation.  

The  cross-border  elements  of  EU  legislation  relating  to  public  services  (e.g.  eProcurement, 
eIDAS,  Public  Sector  Information,  Services  Directive)  continue  to  justify,  and  require 
collaborative activities at EU level, for example, to support the implementation of legislation 
through the common development and piloting of technical solutions.  

The eIDAS Regulation ensures secure and seamless cross-border electronic transactions in the 
EU by promoting the widespread use and uptake of electronic identification and trust services 
(eIDAS  services)  across  borders.  It  ensures  that  people  and  businesses  can  use  their  own 
national  electronic  identification  schemes  (eIDs)  to  access  public  services  in  other  EU 
countries where online public services are available. In addition, it helps to create a European 
internal  market  for  electronic  trust  services  by  ensuring  that  they  work  across  borders  and 
have the same legal status as traditional paper based processes.  

In  the  eGovernment  policy  the  Commission's  responsibility  is  limited  to  the  cross-border 
aspects. Yet the Action Plan included various actions that Member States committed to in a 
voluntary  manner  and  where  the  Commission  did  not  have  a  formal  legal  basis  to  act.  The 
Action  Plan  fully  respected  the  organisational  set-up  of  eGovernment  activities  at  national, 
regional  and  local  levels  and  was  complementary  to  national  Action  Plans  under  the 
responsibility  of  the  Member  States  at  the  same  time  being  instrumental  in  motivating  and 
leveraging implementation in Member States of voluntary actions.  

The mid-term evaluation of the eGovernment Action Plan acknowledges that the Action Plan 
"has  a  positive  impact  on  the  development  of  eGovernment  at  the  European  and  Member 
State level  even  if  progress  has  been  better  in  some  areas  compared  to  others.  This  type  of 
Action  plan  can  be  a  perfect  ‘mobilizer’  instrument  in  order  to  help  the  European 
Commission  and  the  Member  States  coordinating  their  actions".  Several  examples  of  this 
"mobilizer" effect have been given in this evaluation.   

Most of the countries also stress that the cross-border dimension of the eGovernment Action 
Plan has had the strongest impact on their national strategies. It is this effect that makes up the 
EU-added  value.  Interoperable  cross-border  digital  solutions  can  prevent  the  emergence  of 
new  obstacles  to  the  Single  Market.  The  building  block  approach  created  in  the  large-scale 
pilots and continued under the CEF Telecom programme is doing exactly that by promoting 
and supporting the re-use of such solutions in support of interoperability across borders.  

While the Action Plan did not have a dedicated budget or funding instrument, it has served as 
a catalyst to coordinate resources. The research, innovation and piloting projects carried out 
under  the  umbrella  of  the  eGovernment  Action  Plan  in  FP7,  H2020,  CIP,  ISA  and  CEF 

174 http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm 

52 

 
 
 
 
 
 
 
 
 
                                                 
programmes helped to pool public resources and coordinating with private resources in order 
to develop and pilot innovative eGovernment as well as pilot solutions to be deployed as key 
enablers for eGovernment.   

Without the intervention of the eGovernment Action Plan these achievements may not have 
realised  or  would  have  been  only  partly  achieved.  Acting  at  EU  level  through  the 
eGovernment  Action  Plan  helped  the  Commission  and  Member  States  to  coordinate  and 
collaborate  in  order  to  ensure  interoperability  between  national  systems  towards  seamless 
access to digital public services across borders. Continued work is required to encourage the 
opening  up  between  public  administrations  at  all  levels,  the  re-use  of  existing  solutions  to 
avoid duplication,  the  reduction of costs multiplication and the optimisation of investments. 
Coordination  of  efforts  and  resources  contributes  to  strengthening  the  EU  as  a  provider  of 
benefits  to  citizens  and  businesses  and  to  meeting  their  digital  expectations  of  modern 
governments. 

Finding 21 
The Action Plan contributed to increased coordination between the Member States and in the 
Commission aiming at the modernisation of public administrations. An EU level Action Plan 
helped  to  realise  the  potential  of  cross-border  eGovernment  services  to  avoid  further 
fragmentation and to increase interoperability. 

53 

 
 
 
 
7.  CONCLUSIONS 

The findings of this report allow for a relatively clear assessment of the key evaluation criteria 
and questions despite the limited availability of recent quantitative data and certain limitations 
regarding the representativeness of responses to the stakeholder consultations.  

Effectiveness 

Throughout the analysis it was often impossible to isolate the impact of the Action Plan from 
other  developments  in the public sector and therefore to assess  its direct impacts. However, 
the eGovernment Action Plan has been effective in achieving most of its objectives. Already 
the  mid-term  evaluation  of  the  Action  Plan  recognised  that  the  Action  Plan  had  a  positive 
impact on the development of eGovernment at the European and Member State level.  

Despite  there  being  varying  degrees  of  progress  in  the  Member  States  and  in  the  different 
policy priorities,  the Action Plan demonstrated the importance of having common European 
goals in eGovernment. The Action Plan acted also as a "mobiliser" instrument. According to 
the mid-term evaluation of the Action Plan, the national eGovernment strategies in almost all 
Member  States  incorporate  priorities  from  the  Malmö  Declaration  and  the  eGovernment 
Action  Plan.  Furthermore,  most  of  the  Member  States  (17)  have  incorporated  the  political 
priorities of the Malmö Declaration and eGovernment Action Plan in their national strategies.  

Furthermore,  in  2012  the  annual  eGovernment  Benchmark  reporting  was  aligned  with  the 
priorities of the eGovernment Action Plan 2011-2015.  This provided a robust tool to monitor 
and  measure  the  effectiveness  of  the  results,  but  the  Action  Plan  had  no  mechanism  to 
iteratively improve the process and fine-tune the actions according to the findings. 

The  eGovernment  Action  Plan  has  acted  as  a  focal  point  for  actions  on  the  EU  level,  in 
particular related to the support for the development of enablers for cross-border services. The 
Action Plan had a key role in pushing for the availability of key cross-border services. Since 
the adoption of the Action Plan in 2010, large-scale piloting actions have been co-funded by 
the Commission to achieve the transition towards a Digital Single Market. The effects of these 
pilots have been quantitative (take-up) and qualitative (increased awareness and coordination) 
and  have,  for  example,  contributed  towards  the  adoption  of  the  eIDAS  Regulation.  The  e-
procurement piloting through the PEPPOL project has contributed towards the take-up of e-
invoicing in some Member States. The work on the large-scale pilots led to the support of the 
Member  States  of  the  CEF  Telecom  Regulation,  aiming  at  financing  the  deployment  of  the 
cross-border  services  and  the  building  block  approach.  This  has  also  had  an  impact  in  the 
work  of  other  sectors,  for  example,  e-justice  and  e-health.  The  public  online  consultation 
indicated that awareness of cross-border services availability could be further increased. 

Some  of  the  eGovernment  Action  Plan  actions  have  not  been  implemented.  The  mid-term 
evaluation acknowledges that in a rapidly changing world with very fast evolving technology, 
a static five years period seems too rigid for an Action Plan. This static nature has affected the 
effectiveness of the Action Plan as it was not possible to identify new actions needed, to adapt 
actions to changed circumstance or to remove actions that became obsolete, sometimes due to 
activities taking part elsewhere.  

Moreover, the Action Plan, while prioritising user empowerment, did not have a mechanism 
to enable stakeholders to be part of the decision-making process that identified actions to be 
taken by Member States or the Commission during the duration of the plan. This could have 

54 

 
 
 
 
 
 
 
led to greater acceptance of the results. The online public consultation showed the interest of 
stakeholders in an online social media platform to facilitate the sharing of ideas.  

Areas  of  eGovernment  can  gain  political  visibility  arising  from  political  decision-making, 
which increases the overall effectiveness. On the other hand, effectiveness can suffer from too 
specific objectives if they cannot be updated during the duration of the Action Plan.  

Efficiency 

The  eGovernment  Action  Plan  had  no  dedicated  budget..  but  it  mobilised  resources  in  the 
Member  States  and  the  Commission  to  develop  solutions  and  common  understanding  in 
several internal market policies, whicn in turn hase the potential to unleash long term savings 
from digitalisation.  

Relevance 

The Action Plan priorities (user empowerment; internal market; efficiency and effectiveness 
of government and administrations; and pre-conditions for developing eGovernment) remain 
relevant today. The modernisation of the European Public Sector remains an important goal in 
the EU. 

However, given the changing environment with economic challenges, increasing use of online 
means  and  increasing  intra-EU  mobility  the  evaluation  found  that  further  work  is  needed, 
sometimes with evolving new needs. For example, there is a continued need to pursue the re-
use  of  common  solutions  and  to  increase  the  take-up  of  both  cross-border  and  national 
eGovernment  services.  Another  example  is  the  indicator  "online  service  completion"  being 
over  80%  on  average  in  the  EU  in  2015175,  the  relevance  has  moved  from  mere  online 
availability  of  public  services  to,  for  example,  quality  of  the  services  and  cross-border 
availability.  

There  are  various  examples  of  topics  with  continued  or  increased  relevance  in  the  future. 
Some  key  enablers  have  been  successful  and  the  rollout  looks  promising,  but  the 
strengthening of the key enabler approach, including the take-up, remains relevant for future 
work.  The  Action  Plan  areas  of  Citizens  rights,  re-use  of  public  sector  information, 
transparency and data protection are important principles also in the future when modernising 
public administrations.  

The relevance of eGovernment services for the internal market continues, driven also by the 
increased  intra-EU  mobility  of  citizens  and  businesses.  Advancing  the  once-only  principle 
continues  to  be  a  relevant  target  beyond  the  duration  of  the  Action  Plan.  There  are  also 
continued  needs  to  pursue  the  re-use  of  common  solutions  for  rationalisation,  savings  and 
interoperability and to increase take-up. 

The online public consultation confirmed the relevance of a number of these areas. There was 
a clear consensus (supported by 80% or more of the respondents from all categories) on the 
importance of applying the principles of Privacy by default, Digital by default, Cross-border 

175 Digital Economy and Society Index DESI, 2016. Future-proofing eGovernment for the Digital Single Market, 
eGovernment Benchmarking report, 2015. Share of the steps in a public services life event that can be completed 
online, seven life events.  

55 

 
 
 
 
 
 
 
 
 
 
                                                 
by default, open by default, the Once-only principle, Online end-to-end services, Inclusive by 
default and Transparency to eGovernment policies and services in the EU. 

Coherence 

The  governance  structure  of  the  Action  Plan  with  regular  meetings  and  exchanges  of 
experience  between  different  Member  States  and  the  Commission  contributed  to  coherence 
and avoidance of overlaps. The Action Plan's coherence benefited from a political consensus 
(Malmö  Declaration)  between  the  Member  States.  However,  full  coherence  could  not  be 
assured, as the Action Plan was based on voluntary activities of the Member States that also 
had different implementation approaches.  

The coherence of the different funding mechanisms for eGovernment key enablers benefited 
from  a  common  aim  towards  the  deployment  of  mature  services  under  the  CEF  Telecom 
programme.  

A  wide  variety  of  eGovernment  initiatives  at  various  levels  were  taken  during  the  Action 
Plan's five year duration. An Action Plan specific coherence was not fully achievable. 

EU-added value 

The Action Plan led to increase the coordination of investments in the Member States and in 
the Commission in order to focus on the modernisation of public administrations. An EU level 
action  plan  was  probably  the  only  way  to  start  to  realise  the  potential  of  cross-border 
eGovernment  services  and  to  achieve  interoperability  while  avoiding  fragmentation.  If  the 
eGovernment Action Plan was not to be renewed, there would be a risk of fragmentation of 
public  services  solutions,  possible  overlap  of  efforts  and  a  risk  of  low  availability  of  cross-
border public services in the Digital Single Market.  

56 

 
 
 
 
 
 
 
8.  ANNEXES  

Annex 1. 

Procedural information 

In December 2015, European Commission's Directorate-General Communications Networks, 
Content  and  Technology  (DG  CNECT)  launched  an  implementation  and  evaluation 
assessment  of  the  European  eGovernment  Action  Plan  2011  –  2015    ("EU  eGovernment 
Action Plan 2011  –  2015"), adopted on 15th  December 2010, first and foremost  in order to 
provide a set of lessons learnt to guide the work of the new eGovernment Action Plan 2016 – 
2020 announced in the Digital Single Market Strategy. 

The  evaluation  has  been  coordinated  by  the  EC's  Directorate-General  Communications 
Networks,  Content  and  Technology  with  the  support  of  an  Inter-Service  Group  with 
representatives  of  Commission    Directorate-Generals  Agriculture  and  Rural  Development; 
Communication;  Competition;  Informatics;  Economic  and  Financial  Affairs;  Employment; 
Industry, 
Social  Affairs  and 
Entrepreneurship  and  SMEs;  Migration  and  Home  Affairs,  Joint  Research  Centre; 
Publications  Office;  Justice  and  Consumers;  Mobility  and  Transport;  Regional  and  Urban 
Policy; Research and Innovation; Health and Food Safety; Secretariat-General; Taxation; and 
Customs  Union  and  Trade.  The  Inter-Service  group  steered  and  monitored  progress  of  the 
evaluation  exercise,  ensuring  the  necessary  quality,  impartiality  and  usefulness  of  the 
evaluation. The group met three time during the evaluation process. 

Inclusion;  Energy;  Environment; 

Internal  Market, 

For  the  evaluation,  a  broad  literature  review  was  conducted  covering  various  reports,  EU 
studies and a number of international references. Important sources in the desk research were 
the  mid-term  evaluation  of  the  eGovernment  Action  Plan  2011  -  2015176  and  the  annual 
eGovernment  Benchmarking  reports  (for  further  details  see  section  4).  Furthermore, 
additional information and data (listed in the in the footnotes and in the "References and data 
sources" part in the end of this document) were used to answer the evaluation questions. Their 
methods  are  available  in  the  original  referenced  reports.  The  eGovernment  factsheets 
published on the Joinup platform177 give an overview of eGovernment in each Member State.  

A  12  week  online  public  consultation  was  conducted  to  collect  primary  data  and 
complemented  by  statistical  data  (Eurostat  data  and  Digital  Economy  and  Society  Index 
DESI178) and Digital Agenda Scoreboard reports179. 

An  effort  was  made  to  “triangulate”  the  data  used  throughout  the  analysis.  This  means  that 
findings  presented  in  the  evaluation  are  supported  by  evidence  from  different  data  sources 
whenever  additional  data  was  available.  Any  contradicting  evidence  has  been  weighed 
according to its strength and quality before reaching conclusions 

The evaluation faced some limitations in the collection of data, whose impact was mitigated 
to a maximum possible extent: 

  Measuring the impact of the eGovernment Action Plan is a challenging exercise as it 
is almost impossible to isolate the impact of the Action Plan from other developments 

176 Mid-term Evaluation of the e-Government Action Plan 2011–2015. 2014.  
177 NIFO eGovernment factsheets (edition 2015 published in 2016). 
178 http://ec.europa.eu/digital-agenda/en/digital-economy-and-society-index-desi 
179 https://ec.europa.eu/digital-agenda/download-scoreboard-reports 

57 

 
 
 
 
 
                                                 
in  the  public  sector,  such  as  ICT-policies  or  the  increased  use  of  the  Internet  in 
general.  Furthermore,  some  objectives  of  the  Action  Plan,  such  as  increased 
collaboration,  are  non-tangible  and,  as  a  result,  difficult  to  quantify.  The  evaluation 
has used benchmarking to measure the progress in the different policy priority areas of 
the Action Plan and assessed the effectiveness criteria qualitatively.   

  The  Action  Plan  did  not  have  a  dedicated  budget  but  relied  on  funding  from  other 
programmes for a large part of its actions. Other actions were financed by the Member 
States.  As  a  result,  the  efficiency  analysis  (in  particular  the  value  for  money 
assessment) was difficult to carry out.   

  Despite  being  prompted  in  a  number  of  occasions  by  the  EC,  there  is  no  systematic 
measuring  or  reporting  requirement  for  the  Member  States  of  cross-border  public 
services use. To overcome this, the evaluation used data from other sources, including 
case studies.   

  Given the multiplicity of the tools used to gather evidence, the results obtained are of 
different  nature  (for  instance,  eGovernment  related  data  stems  from  the  mid-term 
evaluation  of  the  Action  Plan  and  the  benchmarking  reports  while  the  online  public 
consultation  provided  insights  in  order  to  prepare  for  the  new    eGovernment  Action 
Plan 2016-2020).  

  The  evaluation  takes  into  account  the  inherent  limitations  of  the  findings  of  public 
consultations.  Firstly,  as  in  all  surveys,  the  answers  received  reflect  the  views  of  a 
sample of relevant stakeholders and not those of the entire population who has a stake 
in  this  domain.  Secondly,  stakeholders'  views  convey  an  individual  rather  than  a 
holistic perspective. 

  The  mid-term  evaluation  did  not  systematically  cover  the  five  evaluation  criteria. 
Furthermore,  the  methodology  had  strict  requirements.  For  an  action  to  qualify  as 
"delayed"  it  was  sufficient  that  one  indicator  is  negative  (e.g.  one  Member  State 
answered  ‘no’  to  the  question,  or  one  action  was  delayed  in  one  Member  State)  can 
sometimes give only a partial overview of progress and it should be understood as an 
evaluation of the actions as defined by the action plan, not an assessment of the policy 
priority  as  such.  In  addition,  the  mid-term  evaluation  did  not  capture  the  last  18 
months of the Action Plan. This is partly mitigated using the 2015 benchmark study 
and  partly  the  evaluation  questions  in  the  2015/2016  online  consultation  on  the 
eGovernment Action Plan and other recent reports and information.   

Based  on  the  elements  above,  the  evaluation  has  been  carried  out  on  the  basis  of  the  best 
available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate 
and possibly counter-balanced with qualitative data and considerations. 

58 

 
 
 
 
Annex 2. 

The Online public consultation 

REPORT 

on the public consultation and other consultation activities of the European Commission 
for the preparation of the EU eGovernment Action Plan 2016-2020 

1.  INTRODUCTION  

The Commission launched an online public consultation on 30 October 2015 for 12 weeks to 
seek views on a forthcoming EU eGovernment Action Plan 2016-2020, one of the actions for 
the  completion  the  Digital  Single  Market.  The  consultation  collected  EU  citizens'  and 
businesses'  needs  and  expectations  from  eGovernment  services  in  the  EU,  and  what  public 
administrative bodies can or plan to deliver. It was accompanied by a roadmap for a new EU 
eGovernment Action Plan.   

In addition, the Commission organised workshops with stakeholders. The consultation, those 
events, the annual eGovernment benchmark survey and the mid-term review of the previous 
action plan underpin the preparation of a new EU eGovernment Action Plan 2016–2020.  

2.  WORKSHOPS AND CONFERENCES WITH EXTERNAL STAKEHOLDERS  

2.1. Riga, Digital Assembly 2015  

The first of these events was organised as part of the Digital Assembly in Riga. The panellists 
at  the  workshop  pointed  to  some  urging  user  expectations,  resulting  in  an  initial  list  of 
possible  digital  rights,  further  expanded  during  the  discussion  with  the  audience.  The  list 
stresses possible rights related to user-friendly public services:  

  digital  –  including  the  ability  to  receive  and  submit  documents  electronically  and 

eventually through one-stop-shops 
 
intuitive 
 
inclusive 
  accessible 
 
fast 
  efficient 
  multilingual 
  automated  services  or  at  least  reducing  the  number  of  interactions  with  public 

administrations, for example through the once-only principle.  

Some of the suggested rights linked to modernising public administration, making them open, 
transparent and collaborative, allowing for citizen involvement and eDemocracy. Another set 
of suggested rights related to mobility within the single market, ensuring that data and digital 
services moved seamlessly across borders and the right to do business anywhere in the EU. 
Another  group  of  rights  pointed  to  basic  pre-conditions  that  would  make  this  happen  i.e. 
privacy  /  confidentiality,  the  right  to  exist  digitally,  to  control  access  to  personal  data, 
security,  access  to  cheap  and  fast  network,  digital  literacy  and  the  quality  of  and  access  to 
machine readable data. 

Participants  ranked  the  'once  only  principle',  'user  friendly  /  intuitive  public  services'  and 
'digital literacy' as the three most important rights.  

59 

 
 
2.2. Workshops  on  a  new  EU  eGovernment  Action  Plan,  Brussels:  1  July,  5  November 

2015, 15 December 2015 

The  first  workshop  on  a  new  EU  eGovernment  Action  Plan  focused  on  the  overall  policy 
framework (modernising public administration, helping public administrations cooperate with 
one another and offering better service, citizens' involvement). Many pre-conditions must be 
achieved  before  actions  are  implemented  such  as  privacy,  quality  and  opening  up  of  data, 
interoperability  and  security.  The  current  Action  Plan's  mid-term  review  showed,  that  five-
year static plans were not fit-for-purpose amidst rapidly changing technology. 

At  the  second  Join-up  workshop,  the  Commission  focused  on  the  public  consultation 
questions    and  how  to  engage  citizens  so  that  they  propose  ideas  for  the  new  Action  Plan. 
Participants  discussed  setting  up  an  online  stakeholder  platform,  raising  awareness  about  it, 
the  need  for  political  support,  the  use  of  multiple  languages  and  maintaining  contributors' 
interest in the platform. 

The third workshop on a new EU eGovernment Action Plan focused on the online platform 
and how it could support a new eGovernment Action Plan.  

2.3. Luxembourg eGovernment conference, 1st  December 2015 

Organised by the Luxembourg presidency, the event showed EU countries' progress on public 
sector  modernisation.  Important  steps  have  been  taken  since  the  Malmö  Ministerial 
Declaration on eGovernment of November 2009 and the EU eGovernment Action Plan 2011-
2015 which lead to concrete measures on the political priorities of the Declaration. 

Defining and launching a new coherent EU eGovernment Action Plan was central, aiming at 
increasing  citizen  and  business  engagement  in  the  design  of  public  services  and  policy-
making, among others. 

The Luxembourg Presidency concluded that improved coordination between EU countries is 
essential. An effective European Action Plan must consider all actors and measures affecting 
the EU eGovernment agenda, yet be centred on real user needs in EU countries. 

3.  THE ONLINE, PUBLIC CONSULTATION  

The 12-week consultation was launched on 30 October 2015 until 22 January 2016. It covered 
the following topics related to the development of eGovernment services in the EU: 

Improving eGovernment services  

  Lessons learned for the current Action Plan  
  Factors hampering the use of public services  
 
  Mobility and cross-border public services in the EU 
  Modernizing eGovernment services in the EU 
  The role of the European Commission  
  Citizen involvement  
  Policy principles 

60 

 
 
3.1. Overview of the respondents180 

The  online  questionnaire  received  365  replies;  12  position  papers  came  from  organisations 
and government representatives:   

  65% of the replies came from EU citizens 
  13% from businesses and organisations  
  22% from public administrations 

Most  replies  were  received  from  Germany  (112),  Romania  (59),  Italy  (37),  Czech  Republic 
(32)  and  Spain  (29).  In  the  citizens  group  most  replies  came  from  Romanians  (46),  Italians 
(31) and  Germans  (31). In  the Institution/Organisation/Business category, most  replies were 
received from Germany (26) and Belgium (21). 

3.2. The topics181  

3.2.1.  Lessons learned from the current Action Plan182 

This  section  with  mandatory  questions  addressed  the  eGovernment  Action  Plan  2011-2015 
(on-going at the time).  

Has  the  current  eGovernment  Action  Plan  improved  cross-border  eGovernment  services 
overall?  

43%  of  respondents  replied  that  they  don’t  know;  28.5%  said  that  it  has  not;  24.9%  agreed 
that the plan has improved cross-border services overall. There are not significant differences 
between  the  respondent  groups  with  regards  to  the  perceived  cross-border  effects  of  the 
current Action Plan. The answers point to the need to increase awareness of on-going work 
and availability of cross-border public services in the EU.  

How do you rate the measures comprising the current eGovernment Action Plan?  

This section had 4 sub-questions addressing the measures of the previous action plan, asking 
about their success.  

The respondents mostly replied that they were either unaware of the measures comprising the 
Action  Plan  or  rated  them  as  not  successful,  "not  successful"  comprising  the  largest  group. 
For  instance,  on  the  User  Empowerment  measure  of  the  action  plan  on  inclusive  services, 
collaborative production of services, re-use of public sector information, transparency and the 
involvement of citizens and businesses, 48.1%, said that it was not successful; 24% said that 
they  don’t  know;  27%  of  respondents  found  it  successful.  Increasing  the  efficiency  and 
effectiveness  of  government  and  administrative  bodies  was  viewed  as  the  least  successful, 
although  by  a  small  margin.  Public  administrations  view  the  success  of  the  measures 
addressing user empowerment and preconditions for developing eGovernment as much higher 

180  See Annex 2, report annex 1 A, section 1 
181  The evaluation takes into account the inherent limitations of the findings of public consultations. Firstly as in 
all surveys, the answers received reflect the views of a sample of relevant stakeholders and not those of the 
entire population who has a stake in this domain. Secondly, stakeholders views convey an individual rather 
than a more general and representative perspective.  

182  See Annex 2, report annex 1 A, section 2 

61 

 
 
                                                 
than the overall average. For instance, 46% rated the measures related to the preconditions for 
developing eGovernment successful, against a 28% average for the 3 respondent groups.  

Action Plan activities did not target directly citizens and businesses. They were carried out by 
the Member States and the Commission to create for example enabling conditions not directly 
visible to the citizens as eGovernment Action Plan activities.  

3.2.2.  Factors hampering the use of digital public services 183 

Your contacts with the public administrations in your home country 

This section addressed the factors that may/may not hamper the use of digital public services 
in the respondent's home country.  

Citizens highlighted 3 factors not likely to hamper the use of digital public services: limited 
Internet  access  (63%),  lack  of  digital  skills  (74%)  and  preference  to  interact  with  public 
administrative bodies through traditional channels (55.5%). 

Concerning the factors likely or very likely to hamper the use of digital public services, 73% 
pointed  to  the  lack  of  user-friendly  public  services  (73%)  and  that  public  administrations 
require  the  same  information  more  than  once  ('Once-only  principle')(66%);  56%  stressed 
meeting individual needs and 57% - the lack of online feedback mechanisms. 

For the other factors e.g. accessibility on mobile devices, replies were more equally divided 
between factors that were likely and very likely to hamper the use of digital public services in 
the respondent's home country on the one hand and factors that were less likely and not likely 
to do so on the other. 

Businesses  also  pointed  towards  the  absence  of  the  'Once-only  principle'  as  most  likely  to 
hamper  the  use  of  digital  public  services  (82%).  Other  hampering  factors  businesses 
mentioned were the difficulty to find relevant information (75%), and the poor quality of pre-
filled forms (71%). The factor least probable to stop businesses using digital public services in 
their contact with the public administrations is the preference for personal interaction (71%). 
interact  digitally  with  public  sector 
One  may  conclude 
administrations, but that the current state of online services is not fully satisfactory. 

that  businesses  prefer 

to 

In addition to the pre-defined factors listed in the questionnaire, the lack of transparency and 
the insufficient respect of privacy were mentioned as other factors that could hamper the use 
of digital public services. 

Public  administrations  listed  3  factors  most  likely  to  hamper  the  use  of  digital  public 
services: the lack of accepted electronic identities (eIDs), the need to provide the same data 
more  than  once  and  poor  quality  and  user  unfriendly  online  public  services.  The  lack  of 
digital skills in public sector organisation staff was said to be the factor least likely to hamper 
the use of digital public services.  

The  questionnaire  addressed  the  supply  side  of  eGovernment  services,  asking  about  the 
likely  factors  to  hold  the  organisation  back  from  providing  more  or  better  government 

See Annex 2, report annex 1 A, section 3 

183   

62 

 
 
                                                 
services online. Complex legislation making it difficult to digitise administrative procedures 
rank as the number one factor most likely to do this, followed by the lack of interconnections 
between public sector data sets and interoperability. Lack of political priority and leadership 
does not appear to be a major stumbling block. 

Contact with the public administrations in another EU country 

Regarding  cross-border  services,  the  need  to  provide  paper  documents  in  addition  to  online 
information  were  seen  as  the  factors  most  likely  to  stop  citizens  from  using  digital  public 
services in their contact with public administrations in another EU country (75%). 

3.2.3.  Improving eGovernment Services184 

This section listed measures with the potential to improve eGovernment services, posting the 
same list of measures for citizens, businesses and public administrations to compare. 

Citizens  consider  most  of  the  listed  measures  important  or  highly  important,  with  minor 
differences between the categories. Improving digital skills, both in business and the general 
public were seen as the least important factor to improve.  

On  the  measures  rated  as  highly  important,  making  all  online  public  services  inclusive  and 
accessible  to  all  got  the  highest  score  (64%),  giving  users  access  to  public  services  online 
(63%) and making online public services more trustworthy and secure.  

Public  administrations  emphasised  the  personalisation  of  all  online  public  services  and 
making them user-friendly (98% in favour); the once-only provision of data got 90%, giving 
users  access  to  services  online  (89%)  and  83%  for  the  acceptance  of  EU  wide  electronic 
signatures. 

Inclusiveness  and  accessibility  received  100%  positive  replies  from  business  respondents, 
93%  to  giving  users  access  to  public  services  online.  There  are  hardly  any  differences 
between the answers to the different measures. 

The different respondent categories rate some measures higher than others, but the differences 
do  not  conclusively  point  towards  one  or  a  few  measures  to  address  more  than  others.  All 
measures with the potential to improve eGovernment services should be equally tackled. 

Ensuring cross-border availability of public services is important for the smooth functioning 
of  the  internal  market.  63%  of  citizens  indicate  that  that  the  measures  promoted  at  national 
level also should be promoted at the European level. That is positive for the continuation of 
eGovernment initiatives at European level. 4.9% said that this should not be the case; 32.1% 
replied that they don’t know. Businesses and the public sector also believe that national level 
measures should also be promoted at European level. 

3.2.4.  Mobility and cross-border services in the EU185 

Citizens 

184  See Annex 2, report annex 1A, section 4 
185  See Annex 2, report annex 1A, section 5 

63 

 
 
                                                 
This section addressed the cross-border aspect of eGovernment services, the expectations and 
difficulties citizens, businesses and public sector bodies may have had when interacting online 
with public authorities in another EU country. 

54.1% of the respondents (citizens) said that they had had contact with or needed to engage 
with public authorities in an EU country other than their own. 

Among  the  reasons  for  that  contact  education  ranked  highest  (29.5%),  followed  by  moving 
(22.5%),  health-related  services  (20%),  finding  a  job  (16%),  declaring  taxes  (11%),  retiring 
and starting a judicial procedure (both 3.6%). Multiple answers were possible. 

Many  had  indicated  other  reasons  for  the  need  to  contact  public  authorities  in  another  EU 
country  i.e.  setting  up  a  business  and  contact  with  the  European  institutions.  Most  did  not 
explain more about the other reasons they referred to. 

When interacting online with a public authority in another EU country, many expected to be 
able to access all relevant information and start the procedure online using a "one-stop" shop 
(43%)  and  that  the  information  would  be  provided  in  a  language  they  understood  (40%). 
Many  also  expected  to  be  able  to  complete  procedures,  get  help  and  communicate  in  a 
language they understood (37.5%) and use electronic signatures and electronic identifications 
from  their  home  country  (32%).  Fewer  expected  to  be  able  to  give  electronic  access  to 
personal data already provided in the home country (26%). Multiple answers were possible.  

the  home  country  and 

On  the  specific  difficulties  in  transferring  information/document/data  between  the  public 
authorities  of 
those  of  another,  not  accepting  national 
information/documents/data was ranked the highest. In the free text under the "other" option, 
the difficulty to translate documents for official purposes and the lack of direct exchange of 
documents between public sector bodies in different countries were mentioned.  

Businesses 

Over half (52%) of the businesses replied that they had had contact with or needed to engage 
with public authorities in another EU country. Some reported no difficulties, others said they 
had  had  problems  with  the  resubmission  of  documents  already  provided  to  a  national 
authority or that national documents were not accepted.  

Public sector bodies 

The majority (62%) of public sector bodies reported that they had had contact with or needed 
to  engage  with  public  authorities  in  another  EU  country,  and  more  so  with  authorities  from 
more than one EU country in citizens-related cases. In doing so, 58% said contact was made 
using other means than the existing EU legal framework (e.g. under EU law on cooperation 
between national authorities) and in the absence of an established bilateral channel (62%) e.g. 
Memorandum  of  Understanding.  Concerning  potential  difficulties  in  the  transfer  of 
information, most respondents replied that they had had problems. Few respondents referred 
to specific problems. Among those who did, language issues came on top. 

64 

 
 
3.2.5.  Modernising eGovernment services in the EU186  

Please indicate how important it would be to improve or apply eGovernment services over the 
next 5 years (for each of the following areas):  

According to the respondents (citizens), most listed areas require improving i.e. eGovernment 
services for healthcare, filing tax declarations, enrolling in higher education, looking for a job 
and applying for a passport. eGovernment services related to buying and selling a home were 
seen as the least important.  

Another element for the modernisation of eGovernment  - the improvement of eGovernment 
procedures  related  to  businesses  -  was  addressed  in  the  following  question.  None  of  the 
procedures  listed  were  viewed  predominantly  as  less  or  not  important  meaning  that 
respondents  would  like  to  see  all  of  them  addressed.  The  procedures  marked  as  more 
important compared  to  others were improving online  services  for tax- and insurance-related 
matters and obtaining government certificates.  

When asked if measures to modernise eGovernment services should be proposed for the areas 
listed,  public  sector  bodies  mostly  replied  yes  to  all  with  minor  differences  in  weighting 
between them. 

Respondents  would  like  to  see  eGovernment  services  improved  for  most  eGovernment 
services and online procedures, not only a select few.  

3.2.6.  The  role  of  the  European  Commission187  and  the  new  eGovernment  Action 

Plan 

Section  6  of  the  online  questionnaire  asked  respondents  how  the  Commission  can  help 
improve public administrations at all levels –national, regional and EU-wide (in free text).  

to  help 

the  Commission 

The  answers  suggest  various  ways  for 
improve  public 
administrations: imposing EU legislation in specific areas e.g.  establish an EU authority for 
digital  certificates,  setting  standards  for  cross-border  interoperability  e.g.  by  consolidating 
existing national electronic registers, setting up an EU platform for e-participation, ensuring 
the exchange of good practice and cross-border cooperation, increase funding to eGovernment 
development  in  the  regions  including  financing  for  the  deployment  of  broadband 
infrastructures,  design  eGovernment  policies  to  avoid  social  exclusion  (universal  design), 
harmonise  the  use  of  electronic  identities  including  the  issuing  of  an  EU  eIDs  and  make 
mandatory the use of electronic signatures, increase collaboration between governments and 
citizens  in  the  design  of  eGovernment  services  and  ensure  the  availability  of  open  data-sets 
and open format documents (mandatory). 

The setting of standards, interoperability at all levels and the exchange of good practice were 
mentioned by many as priority areas for the Commission in the future eGovernment Action 
Plan. The Commission was also asked to lead by example and improve its own internal and 
external services. Several respondents  also underlined the need  for the Commission  to  push 

186  See Annex 2, report annex 1A, section 6  
187  See Annex 2, report annex 1A, section 7 

65 

 
 
                                                 
for the application of the 'once-only principle' in order to reduce the administrative burden for 
citizens. 

Ideas  on  the  Commission's  role  and  the  focus  of  the  new  eGovernment  Action  Plan  were 
contained  in  more  detail  in  some  of  the  position  papers  Member  States  submitted.  For 
instance, the Danish Government sees the new eGovernment Action Plan as an important tool 
to  move  public  sector  digitisation  in  Member  States  forward  and  implement  initiatives  that 
will  improve  cross-border  collaboration  on  public  service  delivery.  The  UK  government 
believes it is crucial to improve access for businesses to the single market and that the role of 
the  Commission  is  to  support  Member  States  in  the  delivery  of  cross  border  services, 
coordinate  activities  and  monitor  progress.  Estonia  also  said  that  EU  level  activities  should 
target the better functioning of the internal market. On the role of the Commission, the Polish 
government  believes  that  it  should  coordinate  and  contribute  to  the  creation  of  standards 
relevant to the digital single market; solutions related to the recognition of qualifications, data 
exchange between public registers in Member States, defining common document models and 
imposing interoperable solutions. The Commission should also provide open source solutions 
to handle electronic signatures, ePrior etc. under the Join-up initiative and lead by example. 
La  poste  (France)  said  that  the  eGovernment  policy  set  forth  by  the  Commission  should 
facilitate  digital  interaction  between  governments  and  citizens/businesses  reduce  the 
administrative burden and leave the citizens in control. 

In a follow-on question, all 3 respondent categories ranked accepting electronic signatures for 
the most significant communication flows as the most urgent. That would allow Commission 
suppliers and grant participants to send the relevant data and documents once only, and make 
e-invoicing  and  pre-award  e-Procurement  mandatory  for  all  the  new  Commission  market 
procedures.   

3.2.7.  The 2016 – 2020 eGovernment Action Plan, Citizen involvement188  

This  section  asked  how  people  should  be  enabled  to  contribute  to,  make  proposals  on  and 
publicly  exchange  views  on  new  initiatives  under  the eGovernment  Action  Plan  e.g.  should 
the  Commission  set  up  an  online  social  media  platform  to  facilitate  sharing  of  ideas  (free 
text).  

A clear majority said that an online social media platform to reach out to a large audience is a 
good idea, although views were divided on whether the Commission should set up a dedicated 
one or use existing ones. Some viewed  using exiting platforms as the more efficient option. 
Examples  of  comparable  online  platforms  were  provided  e.g.  the  co-creating  platform  for 
budgetary spending in New York, and similar initiatives in Germany and France (Paris). With 
regards  to  the  setting  up  of  a  dedicated  online  platform,  respondents  said  that  it  should  be 
designed  with  the  users  in  mind,  allow  all  languages,  include  feedback  mechanisms,  be 
backed up by e-signatures and allow users to compare eGovernment solutions in different EU 
countries. 

Some argued that using an online social media platform is not sufficient. It has to be linked to 
going local initiatives and existing national mechanisms for reaching out to citizens. Due to 
the  very  technical  nature  of  eGovernment,  most  likely  the  platform  will  only  be  used  by 
experts and not ordinary citizens.   

188  For the full question see Annex 2, report annex 1B 

66 

 
 
                                                 
3.2.8.  Policy Principles189  

How important are the following strategic policy principles? 

This  section  addressed  policy  principles  that  may  be  applied  to  the  development  of 
eGovernment services in the EU, asking respondents to rate them by importance.  

Over  80%  of  the  respondents  agreed  that  these  principles  are  important,  including  to  have 
online  end-to  end  services,  that  data  should  be  provided  only  once,  that  services  should  be 
inclusive by default, transparent, privacy by default, No-legacy with regards to the technical 
infrastructure,  cross-border  and  open  by  default.  Privacy  by  default  was  said  to  be  more 
important relative to the other principles but only marginally so. The replies do not provide 
evidence that one or several of the policy principle are viewed as significantly more important 
than any of the others.  

Views cautioning the application of some principles referred to the need to adapt technology 
to  the  service  in  question  and  to  user  needs  e.g.  some  user  groups  would  be  excluded  if 
services were only provided digitally. 

In the position papers, Member States referred to the reduction of the administrative burden 
and the 'Once-only principle' as important to the development of eGovernment services in the 
EU. 

Respondents  from  the  public  sector  were  less  favourable  than  citizens  to  the  No  legacy 
principle  which  would  require  governments  to  renew  IT  systems  and  technologies  after  a 
certain period of time to keep in line with the changing environment and technology.  

Respondents agree that eGovernment policies should be designed with these policy principles 
in mind. 

IV 

CONCLUSIONS 

In view of the replies on the previous action plan, the new eGovernment Action Plan should 
be more visible if it is to reach a broader audience.   

Stakeholders  responding  to  consultation  report  that  many  factors  still  hamper  the  use  of 
eGovernment  services. The consultation  and the workshops  confirm the need  to  do  more to 
improve  eGovernment  services  in  the  EU  at  national  and  EU  level.  The  consultation  also 
confirms the Commission's role e.g. by promoting standards and ensuring interoperability at 
all  levels.  Certain  principles  should  be  applied  to  the  setting  up  of  eGovernment  policies, 
including  the  "Once  Only"  principle  to  reduce  the  administrative  burden  to  businesses  and 
citizens.    Overall  the  responses  support  the  approach  taken  in  the  roadmap  for  the  new 
eGovernment Action Plan. 

189  See Annex 2, report annex 1 A, section 8  

67 

 
 
 
 
                                                 
Public consultation Report Annex 1:  

A: Graphs and associated questions 

B: Free text questions 

A: Graphs and associated questions  

1)  Information about the respondents  

a)  Nationality 

b)  Place of business 

c)  Age 

68 

 
 
 
 
 
 
 
 
 
 
 
2)  Lessons learned for the current Action Plan  

a) 

 Has  the  current  2011-2015  eGovernment  Action  Plan  improved  cross-border 
eGovernment services overall 

b)  How  do  you  rate  the  measures  comprising  the  current  eGovernment  Action 

plan? 

69 

 
 
 
 
3)  Factors hampering the use of public services, 1)Citizens, 2) Cross-border, 

3)Businesses, and public administrations 
a) For each of the following measures how likely is it to stop you using digital public 

services in your contact with the public administration in your country? 

70 

 
 
 
b) For each of the following factors how likely is it to stop you using digital public 
service in your contact with the public administrations in another EU country  

71 

 
 
 
c)  For  each  of  the  following  factors  how  likely  is  it  to  stop  you  and  your 
business  using  digital  public  services  in  your  contact  with  the  public 
administrations in your country? 

72 

 
 
 
d)  For  each  of  the  following  factors  how  likely  is  it  to  stop  you  and  your 
business  using  digital  public  services  in  your  contact  with  the  public 
administrations in another EU country? 

73 

 
 
 
e)  As a public administrative body and a user of eGovernment services, please 
rate for each of the following factors how it is likely to hold back the use of 
public  services  at  your  level  of  government  (EU  level,  national,  regional  or 
local) 

f)  As  a  public  administrative  body  providing  online  public  services,  please 
state  for  each  of  the  following  factors  how  likely  it  is  to  hold  your 
organization  back  from  providing  more  and  better  government  services 
online? 

74 

 
 
 
 
4)  Improving eGovernment services  

a)  How  important  are  the  following  measures  in  terms  of  improving 

eGovernment services? 

75 

 
 
 
 
b)  Promoting  at  EU  level  the  measures  you  have  selected  above  as  important  or 

highly important? 

76 

 
 
 
 
5) Mobility and cross border services in the EU 

a)  Have you ever had contact with or needed to engage with  

b)  For what purpose? 

77 

 
 
 
  
c)  When  interacting  online  with  a  public  authority  in  another  EU  country, 

expected?  

d)  Have  you  ever  had  difficulty  transferring  information/documents/data  between 
the public authorities of your home country and those of another EU country? 

78 

 
 
 
 
e) 

Have you ever had contact with, or needed to engage with public authorities in an 

EU country other than your own? 

f)  If you have tried to engage with public authorities In another EU country (e.g.) 

for business purposes, have you ever had difficulties transferring documents/data 
between the public authorities of your home country and the country where you 

79 

 
 
 
 
intended to do business? 

g)  To provide services to citizens, businesses or other public administrative bodies 

in your home country or abroad, have you had contact with, or needed to engage 
with, public authorities in an EU country other than your own? 

80 

 
 
 
 
h)  With authorities from? 

i)  In cases concerning: 

81 

 
 
 
 
 
 
 
 
j)  Did you use existing mechanism (under the EU law on cooperation between 

national authorities, IMI, Solvit) 

k)  Is there an established bilateral channel of communication with the authorities in 

the other EU country? (e.g. under a Memorandum of Understanding) 

l)  If you have tried to engage with public authorities in another EU country have 

you had difficulty transferring information?  

82 

 
 
 
 
 
 
6)  Modernising eGovernment services  

a)  Please indicate how important it would be to improve eGovernment services over 

the next 5 years for each of the following areas?  

83 

 
 
 
b)  Please indicate how important it would be to further improve or introduce each 
of the following eGovernment related procedures (local, regional, national and 

84 

 
 
 
 
 
 
 
 
EU) over the next 5 years? 

85 

 
 
 
 
 
c)  Should measures be proposed in the following areas? 

86 

 
 
 
7.  The role of the European Commission  

Please indicate for each of the following areas the level of priority for action by 
the European Commission. 

8.  Policy principles 

How important are the following strategic policy principles? 

Online end-to-end public sector…

Once only

Inclusive by default

Transparency

Privacy by default

Digital by default / Digital first

No legacy

Cross-border by default

Open by default

Not important

Less important

Important

Highly important

0%

20%

40%

60%

80%

100%

B: Free text questions  

Section 6 – Modernising eGovernment services in the EU 

 

In  which  areas  should  eGovernment  at  all  levels  (local/regional/national  and  EU)  be 
improved over the next 5 years?  

Section 7 of the online questionnaire – The role of the European Commission  

87 

 
 
 
 
 
  How can the European Commission help improve public administrations in the EU at 

all levels – regional, national and EU wide?  

Section 8 – The 2016 2020 eGovernment Action Plan, Citizens involvement 

  How  should  people  be  enabled  to  contribute  to,  make  proposals  on  and  publicly 
exchange views on new initiatives emerging under the eGovernment Action Plan? For 
instance,  should  the  Commission  set  up  an  online  social  media  platform  to  facilitate 
the sharing of ideas? 

88 

 
 
 
 
 
 
Annex 3. 

The individual actions of the eGovernment Action Plan 2011 - 2015 

Below is a description of the actions per priority and sub priority. Some of the actions were 
already defined in the Digital Agenda for Europe, they are indicated with the footnotes. The 
actions  are  numbered  for  the  evaluation  purposes.  Of  the  40  actions,  the  Commission  was 
exclusively responsible for 23 (1, 3, 4, 6, 7, 8, 12, 13, 16, 19, 21, 25, 26, 27, 28, 30, 32, 33, 
35, 36, 38, 39, 40  +41, 44) and the Member States for 10 (2, 5, 11, 17, 18, 20, 22, 31, 34, 37 
+42, 43, 45). Seven actions (9, 10, 14, 15, 23, 24, and 29) were under shared responsibility 
between the Member States and the Commission. 

1. User Empowerment 

Services designed around users' needs and Inclusive Services 

2011-
2013 

1.  The  Commission  will  support  Member  States  in  developing  eGovernment  services 
designed around user needs and in ensuring inclusiveness and accessibility by: 

agreeing common targets and evaluation criteria with the Member States, 

organising  exchanges  of  valuable  expertise  at  national,  regional  or  local  level  to  support 
additional take-up,  

supporting effective and concrete accessibility solutions, compliant with relevant European 
and international standards when available, through demonstration 

2013 

2.  Member  States  will  develop  personalised  online  services,  including  functions  such  as 
monitoring the progress of transactions with public administrations. 

Collaborative Production of Services 

2011 

2011-
2013 

3. Based on a study, the Commission will first assess how to involve users actively in design 
and  production  of  eGovernment  services  and  further  elaborate  recommendations  / 
guidelines with and for the Member States.  

4.  The  Commission  will  facilitate  exchanges  of  knowledge  and  experience  between 
stakeholders,  and,  agree  with  Member  States  on  common  targets  for  the  roll  out 
collaborative services. 

Re-use of Public Sector Information 

2011 

5. Member States will agree on a common set of PSI re-use indicators. 

2011 

2011-
2013 

6.  The  Commission  will  conduct  a  study  to  assess  to  what  extent  open  data  catalogues 
and/or  PSI  portals  (e.g.  data.gov.uk)  have  been  developed  and  implemented  by  Member 
States.  

7.  The  Commission  will  facilitate  exchanges  of  good  practice  and  awareness-raising 
activities and will adopt its own internal PSI re-use strategy based on a review of the PSI 
Decision. 

2011-
2012 

8.  The  Commission  will  review  the  PSI  Directive,  as  indicated  in  the  Digital  Agenda  for 
Europe, and will consider the possibility of an extended strategy for European PSI. 

Improvement of Transparency 

9.  The  Commission  and  Member  States  will  set  common  voluntary  transparency  targets 
and exchange available experiences. 

10.  Member  States  and  the  Commission  will  provide  online  access  to  information  on 
government laws and regulations, policies and finance. 

2011 

2013 

89 

 
 
 
 
2014 

11.  In  accordance  with  Data  Protection  Directive  95/46/EC,  Member  States  will  enable 
citizens  to  have  electronic  access  to  those  personal  data  that  are  held  on  them  when 
available  electronically  and  will  inform  them  electronically  whenever  such  data  are  being 
processed by automatic means, in a simple and unambiguous manner. 

 Involvement of citizens and businesses in policy-making processes 

2011 

2011 

2011-
2015 

12.  The  Commission  will  collaborate  with  Member  States  on  developing  the  electronic 
service  to  support  ‘citizens  initiatives’  (as  foreseen  by  Art.  11  of  the  Treaty  on  European 
Union). 

13. The Commission will assess existing research projects and launch new ones under the 
‘ICT  for  Governance  and  Policy  Modelling’  objective  of  the  2011-2012  FP7  Work 
Programme and ensure further exchanges of knowledge and best practice. 

14.  Member  States,  the  Commission  and  other  representative  institutions  such  as 
parliaments  should  develop  services  that  involve  stakeholders  in  public  debates  and 
decision-making processes building on pilots and demonstration.  

2. Internal Market 

Seamless Services for Businesses 

2011 

2011 

2012-
2014 

2013 

15. Member States and the Commission will assess outcomes of PEPPOL and SPOCS and 
ensure sustainable follow up. 

16.  The  Commission  will  issue  a  White  Paper  on  practical  steps  to  inter-connect 
eProcurement capacity across the internal market190. 

17.  Member  States  should  roll  out  cross-border  services  based  on  the  results  of  PEPPOL 
and SPOCS. 

18.  Member  States  will  ensure  that  a  ‘second  generation’  of  points  of  single  contact  will 
function as fully fledged eGovernment centres beyond the requirements and areas covered 
by the Services Directive.191 

Personal Mobility 

2012-
2014 

2015 

19. The Commission will support exchanges of best practice and coordinate the efforts of 
Member States to jointly develop and set up interoperable eDelivery services. 

20.  Member  States  will  provide  cross-border  and  interoperable  eDelivery  services  for 
citizens, e.g. so that they can study, work, reside, receive health care and retire anywhere 
in the European Union. 

EU-wide implementation of cross-border services 

2011 

2011 

2012-
2015 

21.  The  Commission  will  conduct  a  study  with  the  Member  States,  of  the  demand  for 
cross-border services and assess the organisational, legal, technical and semantic barriers. 

22. Member States will agree on a number of key cross-border public services to be rolled 
out between 2012 and 2015 and will identify appropriate life events/stages192. 

23. The  Commission  will  support  and  coordinate  the  efforts of  Member  States  to  roll out 
Large  Scale Pilot  projects  and to  start  new  ones, while  encouraging  coordination  and  re-
use of results and solutions between them. 

2012-

24. The  Commission will work with Member States and stakeholders to  implement cross-

This action is defined in the Digital Agenda for Europe 
This action is defined in the Digital Agenda for Europe 
This action is defined in the Digital Agenda for Europe. 

190 
191 
192 

90 

 
 
 
                                                 
2015 

border eEnvironment services193. 

3. Efficiency and Effectiveness of Governments and Administrations 

Improving Organisational Processes 

2011 

2011-
2012 

2011-
2015 

2013 

25.  The  Commission  will  facilitate  the  exchange  of  experience,  encouraging  re-use  of 
successful  solutions  and  applications  and  exploring  new  approaches  to  support  the 
Member States in improving organisational processes. 

26.  The  Commission  will  transform  the  ePractice.eu  portal  into  an  effective  experience 
exchange and information tool for Member States’ eGovernment practitioners 

27. The Commission will implement an ambitious eCommission Action Plan for 2011-2015, 
including  full  electronic  procurement,  a  public  sector  information  strategy  and  a 
transparency policy194. 

28.  The  Commission,  in  close  cooperation  with  Member  States,  will  set  up  a  programme 
for staff exchanges between administrations in different Member States. 

Reduction of Administrative Burdens 

2011-
2013 

29. The  Commission will organise  with Member States the sharing of experiences on the 
implementation of the 'once-only' registration principle and, on electronic procedures and 
communications having become a dominant channel for delivering eGovernment services, 
conduct a cost-benefit analysis and design a roadmap for further implementation. 

Green Government 

2012 

2013 

30.  The  Commission  will  conduct  a  study  on  the  potential  of  eGovernment  to  reducing 
carbon footprint of governments including best practices. 

31.  Member  States  should  develop  and  agree  indicators  and  evaluation  procedures  for 
measuring  the  reduction  of  the  carbon  footprint  of  their  administrations  as  a  result  of 
eGovernment services. 

4. Pre-conditions for developing eGovernment 

Open Specifications and Interoperability 

2011-
2015 

2012 

32. The  Commission (via the ISA programme) will implement  activities  to  put  into action 
the European Interoperability Framework (EIF) and the European Interoperability Strategy 
(EIS) (adoption of the EIF and EIS expected in 2010). 

33.  The  Commission  will  organise  exchanges  of  expertise  and  promote  the  re-use  and 
sharing  of  solutions  to  implement  interoperable  eGovernment  services.  This  includes 
establishment of interfaces to gain access to and use authentic national sources.  

2013 

34.  Member  States  should  have  aligned  their  national  interoperability  frameworks  to  the 
EIF195. 

Key Enablers 

2011 

35.  The  Commission  will  propose  a  revision  of  the  eSignature  Directive  with  a  view  to 
providing  a  legal  framework  for  cross-border  recognition  and  interoperability  of  secure 
eAuthentication systems196. 

This action is defined in the Digital Agenda for Europe. 
This action is defined in the Digital Agenda for Europe. 
This action is defined in the Digital Agenda for Europe. 
This action is defined in the Digital Agenda for Europe. 

193 
194 
195 
196 

91 

 
 
 
 
                                                 
2012 

36.  The  Commission  will  propose  a  Council  and  European  Parliament  Decision  to  ensure 
mutual  recognition  of  eIdentification  and  eAuthentication  across  the  EU,  based  on  online 
‘authentication  services’  to  be  offered  in  all  Member  States  (which  may  use  the  most 
appropriate official identification documents — issued by the public and private sectors)197. 

2012-
2014 

37.  Member  States  should  apply  and  roll  out  the  eID  solutions,  based  on  the  results  of 
STORK and other eID-related projects. 

Innovative eGovernment  

2011 

2011 

2012 

38. The Commission will launch a study and recommend action on how to apply emerging 
technologies  and  paradigms  (such  as  SOA  and  clouds  of  public  services)  in  the  public 
sector. 

39.  The  Commission  will  launch  activities  under  the  CIP  programme  to  support 
administrations  to  pilot  the  upgrade  to  IPv6,  thereby  creating  showcases  and  new 
momentum for moving to IPv6 on a large scale.  

40.  The  Commission  will  launch  pilot  projects  to  demonstrate  how  public  administrations 
can deliver eGovernment services in a more flexible and efficient way by using innovative 
architecture and technologies. 

Governance 
In  addition  to  the  above  policy  priorities,  the  Action  Plan  also  includes  a  section  on  its 
governance with five actions. This reflects the close partnership between Member States and 
the Commission. The Governance chapter is an element of the eGovernment Action Plan that 
was added to the four priority areas of the Malmö Declaration. 

41.  The  Commission  will  establish  a  High-Level  Expert  Group  of  Member  States’ 
representatives and will suggest an appropriate mandate. 

42.  Member States  will  inform  the  Commission  and  the  High-Level  Expert  Group  how  the 
political priorities of the Malmö Ministerial Declaration have been or will be reflected in their 
national eGovernment strategies.  

43. All Member States will have incorporated the political priorities of the Malmö Declaration 
in their national strategies. 

44.  The  eGovernment  Action  Plan  will  be  evaluated  and  the  findings  used  to  update  the 
Action Plan. 

45. All Member States will inform the Commission and the High-Level Expert Group how the 
political priorities of the Malmö Declaration have been achieved. 

This action is defined in the Digital Agenda for Europe. 

2011 

2012 

2013 

2013 

2015 

197 

92 

 
 
 
 
 
 
 
                                                 
Annex 4. 

Implementation  of  the  individual  actions  of  the  eGovernment 

Action Plan 2011 – 2015 – State of play of the actions 

Priority: User Empowerment 

The Action Plan priority on user empowerment included 14 actions (listed in Annex 4) across 
five sub-priorities. The figure below198 provides an overview of the status of implementation 
of these 14 actions until the beginning of 2014.  

Overview of progress of the User Empowerment priority. Status 6/2014. 

User Empowerment priority 

Action 
number 

1.  

Status in Mid-term evaluation (2013 - Q1/2014) and updates 

Services designed around users' needs and Inclusive Services 

Within the Sub-Priority "Services designed around users’ needs and Inclusive Services" action 
(Action  1)    on  "Support  Member  States  in  developing  eGovernment  services  in  relation  to 
common  targets  and  evaluation  criteria  on  the  user-centred  services"  the  Commission  has 
supported  exchanges  of  practice  in  the  field  of  user-centred,  inclusive  and  accessible 
eGovernment services (through the ePractice/Joinup platform199), though there is no agreement 
with the Member States on common targets and evaluation criteria for those services.  

The  Commission  adopted  its  proposal  for  a  directive  on  the  accessibility  of  public  sector 
bodies' websites on 3 December 2012, in accordance with Action 64 of the Digital Agenda for 
Europe where the Commission committed to "make proposals by 2011 that will make sure that 
public sector websites (and websites providing basic services to citizens) are fully accessible by 
2015". The proposal aims to approximate the laws, regulations and administrative provisions of 

198 Mid-term Evaluation of the e-Government Action Plan 2011–2015, 2014.  
199 https://joinup.ec.europa.eu/ 

93 

 
 
 
 
 
                                                 
the Member States on the accessibility of websites of public sector bodies. The co-legislators 
are working on the proposal, which has not yet been adopted200.  

The  MeAC  (Measuring  progress  of  eAccessibility  in  Europe)  reports  provided  evidence  and 
analysis to help understand and compare the approaches followed by the European countries, 
with a view to identifying issues and challenges, good practices and future priorities in the web 
accessibility field201.  

Action  2  (development  of  personalised  services  and  multi-channel  delivery)  is  formally 
delayed;  however,  most  Member  States  (85%,  22  out  of  the  26  answers)  have  developed 
personalised services and deliver them via multiple channels.  

Furthermore,  according  to  the  mid-term  evaluation  "Most  Member  States  have  introduced 
personalised  services  in  their  One-Stop-Shops  for  business  and  for  citizens,  as  well  as  in  the 
area  of  health  and  tax  services.  Furthermore,  the  majority  of  the  Member  States  that  have 
already  introduced  personalised  services  offer  them  via  multiple  channels,  such  as  mobile 
access via mobile apps and helplines". 

Collaborative production of Services 

Action 3 (study on collaborative production in eGovernment202) has been completed, and the 
policy recommendations of the study have been shared and discussed with the Member States. 

The  common  targets  for  the  roll-out  of  collaborative  services  have  not  been  agreed  upon 
(Action 4). 

Re-use of Public Sector Information (PSI) 

According  to  the  mid-term  evaluation  of  2014,  there  is  no  agreement  on  the  common  set  of 
Public Sector Information (PSI) re-use indicators (Action 5). Eight out of the 27 countries that 
replied  to  this  question  affirmed  to  use  a  set  of  indicators  for  the  re-use  of  Public  Sector 
Information203.  Indicators  were  further  discussed  in  the  2015  study  "Creating  Value  through 
Open Data" that analysed in-depth three exemplar indicators.  

In  Action 6,  the  Commission  assessed  in a  study  to what extent open data  catalogues  and/or 
PSI portals have been developed and are used in the Member States. A study on the pricing of 
public sector information/Open Data Portals was commissioned and its results were published 
in the end of 2011204. The study noted that "Open data portals have moved over the course of 
two  years  from  the  pioneering  stage  to  the  mainstream:  they  can  be  considered  today  as  a 
recognized flagship initiative of government technology policy." A new study, "Creating Value 
through Open Data" was completed in November 2015205. The study forecasts benefits of the 
re-use of open data in the EU.  

To  complete  Action  7,  the  Commission  has  set  up  a  PSI  Group206,  the  ePSI  platform  and 
several studies on PSI, and has developed its own PSI re-use strategy. 

The reuse policy of the Commission is implemented by a Decision of 12 December 2011207 in 
line  with  the  objective  of  Action  8;  the  Commission  has  also  reviewed  the  PSI  Directive 

2. 

3. 

4. 

5. 

6.  

7. 

8.  

200 COM(2012) 721 final 
201 Study Measuring Progress of eAccessibility in Europe – MeAC, 2013  
202 Study on collaborative production in eGovernment,  2012 
203 In many cases using the PSI Scoreboard Indicators  
204

 Pricing Of Public Sector Information Study on Models of Supply and Charging for Public Sector Information 

2011 
205 http://www.europeandataportal.eu/sites/default/files/edp_creating_value_through_open_data_0.pdf 
206 http://epsiplatform.eu/ 
207  European Commission, Decision of 12 December 2011 on the reuse of Commission documents 
[2011/833/EU]  

94 

 
 
 
 
 
 
 
                                                 
2003/98/EC  on  re-use  of  public  sector  information  (leading  to  the  amending  Directive 
2013/37/EU)  and  introduced  an  Open  Data  Strategy  for  Europe  in  December  2011208.  The 
Commission  has  put  in  place  an  Open  Data  Portal  (in  2012)  where  open  data  from  the  EU 
institutions  is  are  referenced209  that  has  currently  more  than  8000  datasets  available.  The 
European  Data  Portal210,  federating  access  to  open  data  from  all  over  Europe  (EU,  EEA  and 
neighbouring countries) was launched in 2015.  

Improvement of Transparency 

Fourteen  Member  States  are  using  some  transparency  targets.  Of  those  most  are  using  Open 
Government  Partnership  indicators  via  the  Independent  Reporting  Mechanism  (IRM)211; 
another  common  indicator  is open  data  availability.  A  workshop on  transparency  targets was 
held at the end of 2012212. However, common voluntary transparency targets have not been set 
between the Commission and Member States.  
Action 10 (open data and online access to government laws, actions and policies) is delayed 
according to the mid-term evaluation. Nevertheless, most Member States provide online access 
to information on government laws and regulations; fewer Member States provide information 
on  policies.  All  Member  States  provide  online  access  to  information  on  government  finance. 
Not  all  of  them  provide  this  information  through  an  Open  Data  Portal.  Nevertheless,  the 
majority  (19  Member  States)  already  have  Open  Data  Portals  and  almost  all  those  countries 
offer data that are reusable for both non-commercial and commercial purposes.  

For Action 11 (citizens’ electronic access to personal data) the mid-term evaluation reports that 
all  Member  States  provide  access  to  this  information  but  not  all  provide  access  in  electronic 
form (30% of Member States do so).  

Involvement of citizens and businesses in policy-making processes 

The European Citizens' Initiative (ECI), allows one million EU citizens (according to a certain 
distribution  to  participate  directly  in  the  development  of  EU  policies  by  calling  on  the 
European Commission to make a legislative proposal. Support of the ECI was implemented in 
April 2012 (action 12). The online system/portal is in place213.  

The  research  projects  under  the  ICT  for  Governance  and  Policy  Modelling  objective  of  the 
2011-2012  FP7 Work  Programme  were  assessed, and new projects  under  this objective were 
launched.  
Joint action 14 on involving stakeholders in public debates and decision-making processes is 
on track. Most of the Member States have completed it. However, at the time of the mid-term 
evaluation,  only  a  couple  of  Member  States  have  developed  centralised  platforms  for  online 
consultations.  Many  Member  States  run  e-petition  platforms.  However,  not  many  Member 
States  measure  the  level  of  take-up  of  the  online  consultations  in  a  systematic  way.  The 
Commission has funded various eParticipation projects214. 

9.  

10.  

11. 

12. 

13.  

14. 

208 COM(2011) 882 final  
209 http://open-data.europa.eu/ 
210 http://www.europeandataportal.eu 
211 http://www.opengovpartnership.org/independent-reporting-mechanism 
212  https://ec.europa.eu/digital-agenda/en/news/workshop-improving-transparency-government-decision-making-
6112012-results-presentations 
213 http://ec.europa.eu/citizens-initiative/public/welcome 
214 https://ec.europa.eu/digital-agenda/en/eparticipation 

95 

 
 
 
 
 
 
 
 
 
                                                 
Priority: Internal Market 

Overview of progress of the internal Market priority. Mid-term evaluation of the eGovernment Action 
Plan. Status 6/2014. 

Internal Market priority 

Action 
number 

15. 

16. 

Status in Mid-term evaluation (2013 -  Q1/2014) and updates 

Seamless Services for Businesses 

Action  15  required  Member  States  and  the  Commission  to  assess  the  outcomes  of  the 
PEPPOL215(e-procurement)  and  SPOCS216  (eBusiness  single  points  of  contact)  pilots,  which  ran 
from  2009-2012.  Fifteen  Member  States  and  Norway  took  part  in  the  SPOCS  project.  Nine 
Member States and Norway took part in the PEPPOL project, The Commission has assessed the 
outcomes  of  both  projects217  and  launched  a  pilot  project  e-SENS  (Electronic  Simple  European 
Networked Services) to consolidate the building blocks of the existing LSPs.   

Action  16  on  issuing  a  White  Paper  on  practical  steps  to  inter-connect  eProcurement  capacity 
across the internal market has been fully completed. In 2014, the Commission adopted new public 
procurement directives, which enter into force in April 2016218 and make e-procurement gradually 
compulsory, with e-Submission being mandatory for all public buyers as of October 2018219. The 
policy  aiming  at  supporting  transition  to  full  e-procurement  is  also  building  on  a  2012 

215 http://www.peppol.eu 
216 http://www.eu-spocs.eu 
217 The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' 
evaluation, 2013  
218 http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm  
219 http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm 

96 

 
 
 
 
 
 
 
 
 
 
                                                 
17. 

18. 

19. 

20. 

Communication  on  "A  strategy  for  e-procurement"  and  a  2013  Communication  on  "End-to-end 
procurement to modernise public administration"220.  
Two of the LSPs were assessed by the Commission and continued in a consolidated pilot project 
(eSENS)  providing  modules  for  eDocuments,  eDelivery,  eID,  eSignature  and  semantics  (Action 
17).  At  the  time  of  the  mid-rem  evaluation  (2012  –  2013)  twelve  Member  States  (and  Norway) 
have rolled out a LSP, while nine Member States (and Norway) have indicated they have plans to 
roll  out  a  LSP  by  2015  (but only  two  countries  plan  to  do  so  of  those  that  have  not  yet  one  out 
rolled). More Member States are rolling out PEPPOL services than participated in the pilot221. In 
the end of 2015 111 Access Points of eDelivery had been deployed: 

        102 as part of the PEPPOL network (eProcurement domain) in over 18 countries (including 15 

Member States) which accounts thousands of daily transactions. 

 

9 as part of the e-CODEX network (eJustice domain) which accounts over 700 exchanged 
documents in 2015 between 8 countries. 

The action 18 required "Second Generation" of Points of Single Contact (PSC) that would function 
as  fully-fledged  eGovernment  centres.  A  first  assessment  of  the  PSCs  was  carried  out  in  2011-
2012222.  The  results  revealed  a  hybrid  landscape,  with  some  PSCs  more  advanced  than  others  in 
providing  the  information  and  services  required.  This  first  assessment  resulted  in  the 
Communication on the implementation of the Services Directive "A partnership for new growth in 
services 2012-2015”223. This Communication encouraged Member States to develop by the end of 
2014 Second Generation Points of Single Contacts, which should “1) cover all procedures during 
the business life cycle, 2) be multilingual, and 3) be more user-friendly”.  

These  recommendations  were  subsequently  included  into  the  PSC  Charter224,  endorsed  by  the 
Council  in  2013.  The  assessment  of  the  PSCs  performance  against  of  the  criteria  set  out  in  the 
Charter was carried out in 2014 and published in 2015 in a study225.  

Personal Mobility 

e-Codex226 LSP on e-justice consolidated the standard for eDelivery (action 19), that will enable 
citizens,  businesses  and  public  administrations  to  come  together  to  do  business  online  on  the 
European market.  

In  action 20,  the  objective  for  the  Member  States  was  to  provide  cross-border  and  interoperable 
eDelivery services for citizens. According to the mid-term evaluation of the eGovernment Action 
Plan,  more  than  50%  of  the  Member  States  already  provide  or  are  planning  to  provide  those 
services. The CEF Telecom programme has financed the deployment of the cross-border eDelivery 
service. 

EU-wide implementation of cross-border services227  

21. 

For action 21, the Commission carried out a study on the demand for cross-border services228. 

220 http://ec.europa.eu/growth/single-market/public-procurement/e-procurement/index_en.htm 
221 Mid-term Evaluation of the e-Government Action Plan 2011–2015, 2014, p. 36 and country-specific details at 
http://www.egovap-evaluation.eu  
222 Study on The functioning and usability of the Points of Single Contact under the Services Directive, 2012  
223 COM(2012) 261 
224 http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf 
225 The Performance of the Points of Single Contact. An Assessment against the PSC Charter, 2015  
226 http://www.e-codex.eu 
227 Note: the progress under the sub-priority on EU-wide implementation of cross-border services is linked to the 
general objective of the Action Plan on a number of key cross-border services to be offered online by 2015, see 
above at 5.2. 
228  Study  on  Analysis  of  the  Needs  for  Cross-Border  Services  and  Assessment  of  the  Organisational,  Legal, 
Technical and Semantic Barriers, 2013 

97 

 
 
 
 
 
 
 
                                                 
22. 

23. 

24. 

Related to action 22 Member States did not agree on a list. The CEF Telecom Regulation identifies 
the key cross-border services . 

The action 23 focused on Commission to support and coordinate the efforts of Member States to 
roll  out  LSPs  and  to  start  new  ones  while  encouraging  coordination  and  re-use  of  results  and 
solutions between them. The LSP eCodex229 had started at the end of 2010 and a new initiative e-
SENS was started during the Action Plan. In 2015, a new Large Scale Pilot was envisioned for the 
development of a solution of applying the once-only principle at European level for businesses230. 
The  Commission  has  supported  the  coordination  between  the  different  pilots  both  in  the 
development and re-use of solutions and in dissemination activities. 

The mid-term evaluation noted that under half of Member States had rolled out LSP projects. 

The  action  24  on  cross-border  eEnvironment  services  relates  to  the  step-wise  implementation 
(2007-2021)  of  the  INSPIRE  Directive231  establishing  an  infrastructure  for  spatial  information  in 
the EU. With the support232 of i.a the ISA and CIP programmes, several projects allowing public 
authorities  to  work  more  efficiently  together  on  environmental  issues  across  administrative  and 
jurisdictional  boundaries  at  regional233,  national234  and  European235  scales  contributed  to  this 
action. A survey of the use of spatial information in eGovernment carried out by the ISA funded 
EULF236  project,  which  builds  on  INSPIRE  identified,  in  addition  a  large  number  of  different 
public  services  using  location  information,  many  of  which  have  the  potential  for  integration  in 
INSPIRE 
technical  Mid-term  evaluation 
cross-border  services.  The 
implementation, however, reports on a general delay and differences between the rate of INSPIRE 
implementation  in  the  Member  States  affects  the  further  roll-out  of  cross-border  eEnvironment 
services. 

report237  on 

the 

Priority: Efficiency and Effectiveness of Governments and Administrations 

The figure below from the mid-term evaluation of the eGovernment Action Plan provides an 
overview  of  the  seven  actions  included  in  the  Efficiency  and  Effectiveness  of  governments 
and administrations priority. 

229 http://www.e-codex.eu 
230 The topic was included in the H2020 WP 2016-2017 that was adopted in 2015. 
231 More information at http://inspire.ec.europa.eu/ and  Directive 2007/2/EC of the European Parliament and of 
the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community 
(INSPIRE). The INSPIRE Directive entered into force on the 15th May 2007 
232  ISA  Action  1.17  Improving  the  cross-border  exchange  of  spatial  information:  A  Reusable  INSPIRE 
Reference Platform. 
233 Providing INSPIRE-compliant access to utility services: the case of sewage networks in Flanders, Belgium 
234  Dutch  cadastre  INSPIRE  conform,  The  German  Marine  Data  Infrastructure,  UK  National  Biodiversity 
Network to deliver INSPIRE compliance for species data, Geodateninfrastruktur Deutschland: INSPIRE success 
story – Implementing e-reporting of air quality based on INSPIRE at national level 
235  One-Geology  Europe,  European  Location  Framework,  Reporting  and  exchanging  air  quality  information 
using e-Reporting 
236 ISA ACTION 2.13 European Union Location Framework (EULF) 
237 EEA Technical report No 17/2014. Joint EEA-JRC report. 

98 

 
 
 
 
 
                                                 
Overview  of  progress  of  the  Efficiency  and  Effectiveness  priority.  Mid-term  evaluation  of  the 
eGovernment Action Plan. Status 6/2014. Action 26 was completed 2014. Action 29 (EC) was completed in 
2014 – 2015. 

Efficiency and Effectiveness priority 

Action 
number 

Status in Mid-term evaluation (2013/Q1/2014) and updates 

Improving Organisational Processes 

25. 

26. 

27. 

28. 

Under the sub-priority Improving Organisational Processes, several meetings have been organised to 
facilitate  exchange  and  re-use  of  experiences  (action  25)  and  a  study  on  “eGovernment  and  the 
Reduction  of  Administrative  Burden”238  was  carried  out  to  explore  new  approaches  to  improving 
organisational processes. 
In December 2014, ePractice.eu was migrated to the collaborative platform Joinup239 and action 26 
fulfilled. The number of Joinup users is growing and is now above 25 000. 

For Action 27 (on the implementation of the eCommission Action Plan), a Communication entitled 
"Delivering  user-centric  digital  services"240  was  adopted  in  August  2012  and  the  Public  Sector 
Information internal strategy has been drawn up. The implementation of the transparency policy is 
almost  completed  and  the  work  on  fully  electronic  procurement  is  on  track.  Fully  electronic  e-
invoicing takes place in the EC241.  
The programme for staff exchanges between administrations in different Member States (Action 28) 
was not carried out. 

Reduction of Administrative Burdens 

238 Study on eGovernment and the Reduction of Administrative Burden, 2014 
239 https://joinup.ec.europa.eu/ 
240 e-Commission 2012 – 2015. SEC(2012) 492.  
241  http://ec.europa.eu/dgs/informatics/supplier_portal/index_en.htm  -  link  to  Supplier  portal  requires  ECAS 
login.  

99 

 
 
 
 
 
 
 
 
 
 
 
                                                 
29. 

30. 

31. 

For  sub-priority  "Reduction  of  administrative  Burdens"  a  range  of  actions  has  been  implemented 
both by the Commission and the Member States. The results of the study on 'eGovernment and the 
reduction  of  administrative  burden'  focusing  on  the  once-only  principle  were  published  in  2014242 
including  a  cost-benefit  analysis  and  a  proposed  roadmap  for  further  implementation  (Action  29). 
The  European  Council  Conclusions243  25  October  2013  stated  that  "EU  legislation  should  be 
designed  to facilitate  digital  interaction between citizens and  businesses  and  the public  authorities. 
Efforts should be made to apply the principle that information is collected from citizens only once, in 
due respect of data protection rules."  

The  new  H2020  Work  Programme  for  2016  –  2017,  drafted  and  adopted  in  2015,  in  the  area  of 
"Europe in a changing world – inclusive, innovative and reflective Societies" includes the once-only 
principle under topic CO-CREATION-05-2016 - Co-creation between public administrations: once-
only  principle244.  This  topic  includes  calls  for  two  types  of  actions:  a  piloting  action  and  a 
coordination and support action.  

Green Government 

Under  sub-priority  on  Green  Government  the  Commission  launched  a  survey  with  the  Member 
States and a workshop on the topic (Action 30). No study was launched in this area on the potential 
of eGovernment to reducing carbon footprint of governments.  

For  developing  indicators  and  evaluation  procedures  for  "measuring  the  reduction  of  the  carbon 
footprint  (Action  31)  of  administrations"  number  of  countries  provided  examples  of  evaluation 
projects  and  methods,  but  there  was  no  Member  States  agreement  on  indicators  and  evaluation 
procedures.  

Priority: Pre-conditions for developing eGovernment 

242 Study on eGovernment and the Reduction of Administrative Burden, 2014 
243 European Council, Conclusions, 24/25 October 2013  
244  Horizon  2020  Work  Programme  2016  –  2017,  Europe  in  a  changing  world  –  inclusive,  innovative  and 
reflective Societies 

100 

 
 
 
 
 
 
 
                                                 
Overview  of  progress  of  the  Pre-conditions  priority.  Mid-term  evaluation  of  the  eGovernment  Action 
Plan. Status 6/2014. Action 32 has been completed 2014 – 2015.  

Pre-conditions priority 

Action 
number 

Status in Mid-term evaluation (2013/Q1/2014) and updates 

Open specifications and Interoperability 

32. 

33. 

34. 

35. 

  36. 

The  level  of  implementation  (action  32)  of  the  European  Interoperability  Framework  (EIF)  is 
evaluated  via  NIFO245,  the  National  Interoperability  Framework  Observatory.  The  NIFO  is 
operational  and  Member  States  have  been  provided  with  guidance  and  support  on  aligning  their 
national  frameworks  with  the  principles  of  the  EIF  (within  the  framework  of  the  ISA  programme 
implemented by DG DIGIT).  

The exchanges of expertise, promotion of re-use of interoperable solutions, as well as interfaces for 
these exchanges (action 33) have been facilitated through the ISA programme246.  

The NIFO report of 2014247 stated that the majority of countries have taken into account the EIF in 
their National Interoperability Frameworks (NIFs) (action 34) and there is overall a very good NIF-
EIF  alignment.  The  interoperability  situation  in  each  Member  State  individually  is  provided  in 
separate factsheets248 that are regularly updated. Furthermore, the revision and extension of the EIF 
was included in the Digital Single Market Strategy in 2015. 
Key Enablers 

Actions (35 and 36) on eSignatures and eIdentification have been completed. The eIDAS Regulation 
on electronic identification and trust services was adopted 2014 and  came into force in September 
2014249. The regulation sets a general legal and technical framework for electronic transactions. The 
aim  is  to  achieve  secure  and  seamless  cross-border  electronic  transactions  by  promoting  the 
widespread use and uptake of electronic identification and trust services (eIDAS services). It ensures 
that  people  and  businesses  can  use  their  own  national  electronic  identification  schemes  (eIDs)  to 
access public services in other EU countries where eIDs are available. In addition, it helps to create a 
European  internal  market  for  eTS  (Trust  Services)  -  namely  electronic  signatures,  electronic  seals, 
time stamp, electronic delivery service and website authentication - by ensuring that they will work 
across borders and have the same legal status as traditional paper based processes.  

37. 

eID solutions have been applied and rolled out (action 37). While initiated in the STORK250 projects 
(with around 22 M EUR EU co-funding), the eID building block is now part of the CEF Telecom 
Programme. The STORK eID has contributed to the eIDAS technical specifications.  

The status at the end of 2014 was251:  

 

 

In 6 Member States a framework had been developed for an interoperable service allowing 
foreign citizens (using their eID credentials) to notify all relevant entities of an address 
change. This was achieved without modifying current procedures in each Member State. 

12 Member States had integrated STORK with the EC's Authentication Service (ECAS). 
This integration allowed citizens from those Member States to use their nationally issued 

245 http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-3action_en.htm and NIFO JoinUp community 
https://joinup.ec.europa.eu/community/nifo/home 
246 http://ec.europa.eu/isa/ 
247 State of play of interoperability in Europe – Report 2014 
248 https://joinup.ec.europa.eu/community/nifo/og_page/nifo-factsheets 
249 Regulation (EU) No 910/2014. On eIDAS, see http://ec.europa.eu/digital-agenda/en/trust-services-and-eid 
250 www.eid-stork2.eu 
251 https://joinup.ec.europa.eu/community/cef/og_page/catalogue-building-blocks 

101 

 
 
 
 
 
 
                                                 
eIDs to access electronic services of the EC. 

 

 

 

5 Member States were using the STORK solution in their eDelivery applications, allowing 
citizens from other Member States to access the service with their own eID credentials. 

In 5 Member States foreign students could access online administrative and academic 
services offered by European Universities with their eID. 

10 Member States allowed foreign citizens to register for social security with their eID 
credentials. 

In 2015 the following developments were reported: 

 

15 countries, including 13 Member States, are connected to the eID network via STORK252.  

  There were over 200 downloads of the sample software of the eID technical specifications 
compliant to eIDAS v0.9. This version was released on Joinup on 21 September 2015. 

  There were over 1000 downloads of the sample software of the eID technical specifications 
compliant to eIDAS v1.00. This version was released on Joinup on 27 November 2015. 

Innovative eGovernment 

38. 

39. 

40. 

The results of a study on Cloud and Service Oriented Architectures (SOA) for eGovernment253 was 
published in 2011 (action 38). The results contributed to the development of a new vision of public 
services254. 

Work for supporting administrations in piloting the upgrade to IPv6 (action 39) has been done in the 
CIP programme255. 

The  concept  of  cloud  of  public  services  was  tested  in  the  Competitiveness  and  Innovation 
Programme pilot projects256 (action 40). 

252 CEF Monitoring Report Q4/2015 
253 Study on cloud and service oriented architectures for eGovernment, Final report, 2011  
254  A vision for public services, 2013 
255 http://www.gen6.eu/home 
256 https://ec.europa.eu/digital-agenda/en/towards-cloud-public-services 

102 

 
 
 
 
 
                                                 
Annex 5. 

Acronyms 

AGRI   
BBs 
CEF 
CF 
CIP  
CIPA   
CONNECT   The European Commission Directorate General for Communications  

The Directorate General responsible for the Common Agricultural Policy. 
Building Blocks 
Connecting Europe Facility 
Cohesion Fund 
Competitiveness and Innovation Framework Programme 
Common Infrastructure for Public Administrations Sustainability 

COM   
COMM 
COMP 

Networks, Content & Technology 
Communication from the Commission  
The Directorate General responsible for Communication  
The  Directorate  General  in  charge  of  Competition  Policy  in  the  European 
Union 
Core Service platforms 
Digital Agenda for Europe 
Eurostat data and Digital Economy and Society Index 
Directorate General for Informatics  
Directorate General 
Digital Service Infrastructure 
Digital Single Market 
European Agricultural Fund for Rural Development 
Directorate General for Economic and Financial Affairs  

CSP 
DAE    
DESI   
DIGIT  
DG 
DSI  
DSM    
EAFRD  
ECFIN 
DG CNECT   Communications Networks, Content and Technology DG (ex DG INFSO) 
EACI   
EC  
ECAS  
ECI  
e-CODEX  
e-Sens   
EASME 
EEA 
EFTA  
EID 
eIDAS 

Executive Agency on Competitiveness and Innovation 
European Commission 
European Commission Authentication Service 
European Citizens’ initiative 
e-Justice Communication via Online Data Exchange 
Electronic Simple European Networked Services 
Executive Agency for SMEs 
European Economic Area 
European Free Trade Association  
Electronic Identity  
Electronic identification and trust services for electronic transactions in the 
internal market 
European Interoperability Framework 
European Interoperability Strategy 
European Interoperability Reference Architecture 
Directorate for Employment, Social Affairs & Inclusion  
Directorate General for Energy  
Directorate General for the Environment  
European Patients - Smart Open Services 
European Regional Development Fund 
European Social Fund 
European Structural and Investment Funds 
European Single Procurement Document 
electornic Trust Services   
European Union 
European Union Location Network 

EIF  
EIS  
EIRA   
EMPL  
ENER  
ENV 
epSOS  
ERDF   
ESF  
ESIF    
ESPD   
eTS 
EU  
EULF  

103 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
EU 7th Framework Programme for Research and Technological Development  
Gross Domestic Product 
Directorate General for Internal Market, Industry, Entrepreneurship and SMEs  
Directorate General for Migration and Home Affairs 
Information and Communication Technology 
Information and Communication Technology Policy Support Programme 
Infrastructure for Spatial Information for Europe Directive 
Internet protocol version 6 
Independent Reporting Mechanisms 
Interoperability Solutions for European public administrations 
Interoperability solutions for European public administrations, businesses and 
citizens 
EC Joint Research Centre 
Directorate for Justice and Consumers  
Key Performance Indicators  
Large Scale Pilots 
National Interoperability Framework 
National Interoperability Framework Observatory 
Measuring progress of eAccessibility in Europe 
Directorate General for Mobility and Transport  
Non-Governmental Organisations  
Organisation for Economic Co-operation and Development  
Publication Office of the European Union  
Pan-European Public Procurement OnLine 
Political, Economic, Social, Technological, Environmental and Legal factors 
Point of Single Contact 
Public Sector Information 
Regulatory Fitness and Performance of EU Legislation 
Directorate General for Regional and Urban policy  
Directorate General for Health and Food Safety   
Register of Commission Document: Documents which cannot be classified in 
any of the other series. 
Shared Environmental Information System 
Secretariat-General of the European Commission  
Small and Medium Size companies  
Cloud and Service Oriented Architectures 
Service provided by the national administrations in each EEA EFTA country 
Simple Procedures Online for Cross-border Services 
Secure Identity Across Borders Linked 
Staff Working Document  
Directorate for Taxation 
Directorate for Trade 
United Nations  
Value Added Tax 
Work Programme 
Youth Employment Initiative 

FP7 
GDP 
GROW 
HOME 
ICT  
ICT PSP  
INSPIRE 
IPv6 
IRM 
ISA  
ISA²  

JRC 
JUST 
KPIs 
LSPs  
NIF  
NIFO   
MeAC  
MOVE 
NGOs  
OECD  
OP 
PEPPOL 
PESTEL 
PSC  
PSI  
REFIT 
REGIO 
SANTE 
SEC 

SEIS 
SG 
SMEs   
SOA 
SOLVIT 
SPOCS 
STORK  
SWD   
TAXUD 
TRADE 
UN 
VAT 
WP  
YEI 

104 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Annex 6. 

REFERENCES AND DATA SOURCES  

-  Ashfield District Council, INSPIRE and Open Data, March 2015 [available at: http://www.ashfield-

dc.gov.uk/residents/democracy,-elections-and-legal/data-information-data-protection,-freedom-of-
information-etc/inspire-and-open-data.aspx] 

-  Austrian Institute of Economic Research (WIFO), Center for European Economic Research and IDEA 

Consult for the European Commission, Study Excellence in public administration for competitiveness in EU 
Member States, 2012 [available at: http://ec.europa.eu/growth/tools-
databases/newsroom/cf/itemdetail.cfm?item_id=7970] 

-  Capgemini Consulting for the European Commission, Creating Value through Open Data: Study on the 

Impact of Re-use of Public Data Resources, 2015 [available at: 
http://www.europeandataportal.eu/sites/default/files/edp_creating_value_through_open_data_0.pdf] 

-  Capgemini Consulting for the European Commission, Open Data Maturity in Europe 2015 Insights into the 
European state of play, 2015  [available at: http://www.europeandataportal.eu/en/content/open-
data-maturity-europe] 

-  Capgemini et al. for the European Commission, Delivering the European Advantage? How European 

governments can and should benefit from innovative public services, 11th eGovernment Benchmark Report, 
2014 [available at: https://ec.europa.eu/digital-single-market/en/news/eu-egovernment-report-2014-shows-
usability-online-public-services-improving-not-fast]  

-  Capgemini et al. for the European Commission, eGovernment Benchmark Framework 2012 – 2015. Method 

paper, July 2012 [available at: https://ec.europa.eu/digital-single-market/sites/digital-
agenda/files/eGovernment%20Benchmarking%20method%20paper%20published%20version_0.pdf]  

-  Capgemini et al. for the European Commission, Study on Analysis of the Needs for Cross-Border Services 

and Assessment of the Organisational, Legal, Technical and Semantic Barriers, 2013 [available at 
https://ec.europa.eu/digital-single-market/news/final-report-study-analysis-needs-cross-border-services-and-
assessment-organisational-legal] 

-  CapGemini et al. for the European Commission, Smarter, Faster, Better eGovernment: 8th Benchmark 

Measurement, 2009 [available at 
https://joinup.ec.europa.eu/sites/default/files/files_epractice/sites/Smarter%2C%20Faster%2C%20Better%2
0eGovernment%20-%208th%20Benchmark%20Measurement.pdf] 

-  Capgemini et al. for the European Commission, Digitizing Public Services in Europe: Putting ambition into 

action. 9th Benchmark Measurement , 2010 [available at: https://ec.europa.eu/digital-single-
market/sites/digital-agenda/files/egov_report.pdf] 

-  Capgemini et al. for the European Commission, Future-proofing eGovernment for the Digital Single 
Market, eGovernment Benchmarking report, 2015 [available at: https://ec.europa.eu/digital-single-
market/news/eu-egovernment-report-2015-shows-online-public-services-europe-are-smart-could-be-
smarter] 

-  Capgemini Consulting and Eurochambres for the European Commission, The Performance of the Points of 

Single Contact. An Assessment against the PSC Charter, 2015 [available at: 
http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=8342] 

105 

 
 
 
 
 
 
 
 
 
 
 
 
 
-  Carlyle, S. Environment Agency for England and Wales, Reducing environmental risk through INSPIRE 
(quantified benefits) 2013 [available at: http://www.poweredbyinspire.eu/documents/0403-sustainability-
carlyle.pdf] 

-  Centre for strategy & Evaluation Services for the European Commission (CSES), Study Final Evaluation of 

the Competitiveness and Innovation Framework Programme, December 2011 [available at: 
http://ec.europa.eu/cip/files/cip/cip_final_evaluation_final_report_en.pdf] 

-  Craglia M., Pavanello L. and Smith R. S., European Commission, Joint Research Centre Institute for 

Environment and Sustainability, The use of spatial Data for the Preparation of Environmental Reports in 
Europe, 2010 [available at: http://ec.europa.eu/environment/eia/pdf/jrc_technical%20report_2009%20eia-
sea%20survey.pdf] 

-  Ctrl-Shift, The economics of identity, June 2014 [https://www.ctrl-shift.co.uk/news/general/2014/06/09/the-

economics-of-identity/] 

-  Deloitte for the European Commission, The feasibility and scenarios for the long-term sustainability of the 
Large Scale Pilots, including 'ex-ante' evaluation, 2013 [available at: https://ec.europa.eu/digital-single-
market/news/final-report-study-feasibility-and-scenarios-long-term-sustainability-large-scale-pilots] 

-  Deloitte for the European Commission, Study on cloud and service oriented architectures for eGovernment, 

Final report, 2011 [available at https://ec.europa.eu/digital-single-market/sites/digital-
agenda/files/smart2010-0074finalreport.pdf] 

-  Deloitte Consulting and Tech4i2 for the European Commission, Mid-term Evaluation of the e-Government 
Action Plan 2011–2015, 2014 [available at: https://ec.europa.eu/digital-single-market/news/mid-term-
evaluation-e-government-action-plan-2011-2015-implementation-smart-2012-006020] 

-  Deloitte and Tech4i2 for the European Commission, Pricing Of Public Sector Information Study on Models 

of Supply and Charging for Public Sector Information, 2011 [available at: 
https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=1158] 

-  Deloitte Consulting and Tech4i2 for the European Commission, Study on The functioning and usability of 

the Points of Single Contact under the Services Directive, 2012 [available at: 
http://ec.europa.eu/internal_market/services/docs/services-dir/study_on_points/final_report_en.pdf] 

-  E-Codex project [http://www.e-codex.eu/about-the-project.html] 

-  ECORYS Nederland BV & Grontmij Nederland BV, Costs-benefits analysis INSPIRE in The Netherland, 

2009. [available at: http://www.geonovum.nl/sites/default/files/nkba_engelse_vertaling.pdf] 

-  EEA Technical report 17/2014. Mid-term evaluation report of INSPIRE evaluation. Joint EEA-JRC report. 
[available at: http://www.eea.europa.eu/publications/midterm-evaluation-report-on-inspire-implementation] 

-  Ernst & Young and Danish Technological Institute for the European Commission, Study on eGovernment 

and the Reduction of Administrative Burden, 2014 [available at https://ec.europa.eu/digital-single-
market/en/news/final-report-study-egovernment-and-reduction-administrative-burden-smart-20120061] 

-  EY for the European Commission, Study on  Evaluation of the impact of the free movement of EU citizens at 

local level, 2014  [available at 
http://ec.europa.eu/justice/citizen/files/dg_just_eva_free_mov_final_report_27.01.14.pdf] 

106 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, A vision for public services, 2013 [available at: https://ec.europa.eu/digital-single-

market/news/vision-public-services]  

-  European Commission, Better Regulation Guidelines [available at: http://ec.europa.eu/smart-

regulation/guidelines/ug_chap1_en.htm] 

-  European 

Commission, 

CEF 

Monitoring 

report 

Q4/2015 

[available 

at: 

https://joinup.ec.europa.eu/community/cef/news/first-cef-monitoring-report-available] 

-  European Commission, Charter for the electronic Points of Single Contact under the Services Directive 

[available at: http://ec.europa.eu/internal_market/services/docs/services-dir/psc-charter_en.pdf] 

-  European Commission, Communication to the Institutions -  A Digital Agenda for Europe [COM(2010) 245 
final; available at: http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52010DC0245R(01)] 

-  European Commission, Communication to the Institutions - A Digital Single Market Strategy for Europe 

[COM(2015) 192; available at: http://eur-lex.europa.eu/legal-
content/EN/TXT/?qid=1447773803386&uri=CELEX:52015DC0192] 

-  European Commission, Communication to the Institutions - A strategy for e-procurement - 

[COM/2012/0179 final; available at:  http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0179:FIN:EN:PDF] 

-  European Commission, Communication from the Commission - Annual Growth Survey 2013 [COM(2012) 

750 final; available at: http://ec.europa.eu/europe2020/pdf/ags2013_en.pdf] 

-  European Commission, Communication to the Institutions - Annual Growth Survey 2015 [COM(2014) 902 

final; available at: http://ec.europa.eu/europe2020/pdf/2015/ags2015_en.pdf] 

-  European Commission, Communication to the Institutions - Annual Growth Survey 2016 [COM(2015) 690; 

available at: http://ec.europa.eu/europe2020/pdf/2016/ags2016_annual_growth_survey.pdf] 

-  European Commission, Communication to the Institutions - Better regulation for better results - An EU 

agenda [COM(2015)215 final of 19 May 2015;  available at: http://ec.europa.eu/smart-
regulation/better_regulation/documents/com_2015_215_en.pdf] 

-  European Commission, Communication to the Institutions - End-to-end e-procurement to modernise public 

administration [COM/2013/0453 final; available at:  http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2013:0453:FIN:EN:PDF] 

-  European Commission, Communication to the Institutions – Europe 2020 strategy [COM(2010)2020; 
available at: http://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-
%20Europe%202020%20-%20EN%20version.pdf] 

-  European Commission, Communication to the Institutions, Towards interoperability for European public 

services [COM(2010) 744 final, available at  http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0744:FIN:EN:PDF] 

-  European Commission, Communication to the Institutions, European Interoperability Framework (EIF) for 

European public services [COM(2010) 744 final, Annex 2; available at  http://ec.europa.eu/isa/actions/04-
accompanying-measures/4-2-3action_en.htm] 

107 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, Communications to Institutions - Evaluations of the Competitiveness and Innovation 
Framework Programme [COM(2013)2; available at: http://ec.europa.eu/cip/documents/implementation-
reports/index_en.htm] 

-  European Commission, Communication to the Institutions - i2010 eGovernment Action Plan: Accelerating 

eGovernment in Europe for the Benefit of All [COM(2006) 173 final; available at: http://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52006DC0173&from=EN] 

-  European Commission, Communication to the Institutions - On the implementation of the Services 

Directive. A partnership for new growth in services 2012-2015  [COM(2012) 261 final; available at: 
http://ec.europa.eu/internal_market/services/docs/services-
dir/implementation/report/COM_2012_261_en.pdf] 

-  European Commission, Communication to the Institutions - Open data - An engine for innovation, growth 

and transparent governance [COM(2011) 882 final; available at http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0882:FIN:EN:PDF]  

-  European Commission, Communication to the Institutions - The European eGovernment Action Plan 2011-
2015, Harnessing ICT to promote smart, sustainable & innovative Government [COM(2010) 743; available 
at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0743:FIN:EN:PDF] 

-  European Commission, Communication to the Institutions - Towards interoperability for European public 

services [COM(2010) 744 final; available at: http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0744:FIN:EN:PDF] 

-  European Commission, Communication to the Institutions - Unleashing the Potential of Cloud Computing in 

Europe [COM(2012)0529 final; available at:  http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0529:FIN:EN:PDF] 

-  European Commission, Communication to the Institutions -Upgrading the Single Market: more 

opportunities for people and business [COM(2015)550; available at 
https://ec.europa.eu/transparency/regdoc/rep/1/2015/EN/1-2015-550-EN-F1-1.PDF] 

-  European Commission, Competitiveness and Innovation Framework Programme ICT Policy Support 

Programme [Archived site: http://ec.europa.eu/cip/ict-psp/index_en.htm]   

-  European Commission, Competitiveness and Innovation Framework Programme (CIP)- Implementation 

Reports [available at: http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm] 

-  European Commission, Connecting Europe Facility [https://ec.europa.eu/digital-single-market/connecting-

europe-facility] 

-  European Commission, Contributions and preliminary trends of the public consultation on the eGovernment 

Action Plan 2016 – 2020, 2016 [available at: https://ec.europa.eu/digital-single-market/news/contributions-
and-preliminary-trends-public-consultation-egovernment-action-plan-2016-2020] 

-  European Commission, DG Internal Market and Services: Management Plan 2013, 2012 

[http://ec.europa.eu/dgs/internal_market/docs/amp_2013_en.pdf] 

-  European Commission, Digital Agenda, Action 91: Member States to agree a common list of key cross-
border public services [https://ec.europa.eu/digital-single-market/pillar-vii-ict-enabled-benefits-eu-
society/action-91-member-states-agree-common-list-key-cross]  

108 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, Digital Agenda Scoreboard report 2011, Pillar 7 [available at: 

https://ec.europa.eu/digital-single-market/download-scoreboard-reports] 

-  European Commission, Digital Agenda Scoreboard report 2012. [available at https://ec.europa.eu/digital-

single-market/download-scoreboard-reports] 

-  European Commission, Digital Agenda Scoreboard reports [available at: https://ec.europa.eu/digital-single-

market/download-scoreboard-reports] 

-  European Commission, Digital Agenda Targets Progress report, 2015 [available at: 

https://ec.europa.eu/digital-single-market/download-scoreboard-reports] 

-  European Commission, Digital Economy and Society Index DESI [available at https://ec.europa.eu/digital-

single-market/digital-economy-and-society-index-desi] 

-  European Commission, Digital Single Market, Connecting Europe Facility [https://ec.europa.eu/digital-

single-market/connecting-europe-facility] 

-  European Commission, e-Commission 2012 – 2015, Communication from VP Šefčovič to the Commission -

Delivering user-centric digital services [SEC(2012) 492; available at: 
http://ec.europa.eu/dgs/informatics/ecomm/doc/communication_sefcovic_tothecom.pdf] 

-  European Commission, eGovernment Action Plan 2016-2020, Roadmap [available at:  

http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_cnect_006_e_government_action_plan_en.pdf] 

-  European Commission, eHealth Action Plan 2012-2020: Innovative healthcare for the 21st century 

[https://ec.europa.eu/digital-single-market/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-
century]  

-  European Commission,  e-PRIOR Supplier Portal 

[http://ec.europa.eu/dgs/informatics/supplier_portal/index_en.htm] 

-  European Commission, ePSIplatform [http://epsiplatform.eu/ and 
http://www.epsiplatform.eu/content/european-psi-scoreboard] 

-  European Commission, EU eGovernment Report 2015 shows that online public services in Europe are 

smart but could be smarter, 23rd June 2015 [https://ec.europa.eu/digital-single-market/news/eu-
egovernment-report-2015-shows-online-public-services-europe-are-smart-could-be-smarter]  

-  European Commission, EU implementation of the G8 Open Data Charter [https://ec.europa.eu/digital-

single-market/news/eu-implementation-g8-open-data-charter] 

-  European Commission, European citizens' initiative (ECI) [http://ec.europa.eu/citizens-

initiative/public/welcome] 

-  European Commission, European Union Open Data Portal [https://open-data.europa.eu] 

-  European Commission, Europe's Digital Competitiveness Report 2010 [available at: 

https://ec.europa.eu/digital-single-market/sites/digital-
agenda/files/european_competitiveness_report_2010.pdf] 

109 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, Eurostat, E-government usage by individuals [available at: 

http://ec.europa.eu/eurostat/en/web/products-datasets/-/TSDGO330] 

-  European Commission, Eurostat, Foreign citizens living  in the EU Member States, 2015 [available at: 

http://ec.europa.eu/eurostat/documents/2995521/7113991/3-18122015-BP-EN.pdf/d682df12-8a77-46a5-
aaa9-58a00a8ee73e] 

-  European Commission, Eurostat, ICT survey of Households and Individuals, 2014 [available at: 
http://ec.europa.eu/eurostat/statistics-explained/index.php/Information_society_statistics_-
_households_and_individuals] 

-  European Commission, Eurostat, Information society indicator "Enterprises using the internet for interacting 

with public authorities"  

-  European Commission, Eurostat, Individuals using the internet for interaction with public authorities 

[available at: 
http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&plugin=1&pcode=tin00012&language=en] 

-  European Commission, Eurostat, Migration and migrant population statistics [available at: 

http://ec.europa.eu/eurostat/statistics-explained/index.php/Migration_and_migrant_population_statistics] 

-  European Commission, GROWTH, E-Certis [https://ec.europa.eu/growth/tools-databases/ecertis/] 

-  European Commission, GROWTH, E-procurement [http://ec.europa.eu/growth/single-market/public-

procurement/e-procurement/index_en.htm] 

-  European Commission, GROWTH, European Single Procurement Document (ESPD) 

[https://ec.europa.eu/growth/tools-databases/espd] 

-  European Commission, GROWTH, New rules on public procurement and concession contracts as of April 

2016 [http://ec.europa.eu/growth/single-market/public-procurement/new/index_en.htm] 

-  European Commission,  GROWTH, Single Market and Standards – New rules on Public procurement and 

concession contracts as of April 2016 [http://ec.europa.eu/growth/single-market/public-
procurement/new/index_en.htm] 

-  European Commission, Guidance on Ex ante Conditionalities for the European Structural and Investment 

Funds, 2014 [available at: 
http://ec.europa.eu/regional_policy/sources/docgener/informat/2014/eac_guidance_esif_part2_en.pdf] 

-  European Commission, High Level Group on Business Services, Final Report, 2014 [available at: 

http://ec.europa.eu/growth/industry/policy/renaissance/high-level-group/index_en.htm] 

-  European Commission, Horizon 2020 Work Programme 2016 – 2017, Europe in a changing world – 

inclusive, innovative and reflective Societies [European Commission Decision C (2015)6776 of 13 October 
2015; available at: http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-
wp1617-societies_en.pdf] 

-  European Commission, INSPIRE, MIG, National Implementation Seminars [https://ies-
svn.jrc.ec.europa.eu/projects/mig-inspire/wiki/National_implementation_webinars] 

-  European Commission, Interoperability Solutions for European Public Administrations 

[http://ec.europa.eu/isa/] 

110 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, ISA programme, Common Infrastructure for Public Administrations Sustainability 

[http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-6_1-19action_en.htm] 

-  European Commission, Joinup platform [https://joinup.ec.europa.eu/] 

-  European Commission, Joinup, Connecting Europe Facility, Catalogue of Building Blocks, 2014 

[https://joinup.ec.europa.eu/community/cef/og_page/catalogue-building-blocks] 

-  European Commission, Justice, Protection of personal data [http://ec.europa.eu/justice/data-protection/] 

-  European Commission, Justice, Reform of EU data protection rules [http://ec.europa.eu/justice/data-

protection/reform/index_en.htm] 

-  European Commission, Justice, EU Justice Scoreboard [http://ec.europa.eu/justice/effective-

justice/scoreboard/index_en.htm] 

-  European Commission, MEMO/10/200, 19th May 2010, Digital Agenda for Europe: key initiatives 

[available at: http://europa.eu/rapid/press-release_MEMO-10-200_en.htm?locale=en] 

-  European Commission, Mid-Term Evaluation of the eGovernment Action Plan - Evaluation of the 

eGovernment Action Plan [http://www.egovap-evaluation.eu] 

-  European Commission, National Interoperability Framework Observatory, eGovernment Factsheets 2015 
(published in 2016) [available at: https://joinup.ec.europa.eu/community/nifo/og_page/egovernment-
factsheets]  

-  European Commission, National Interoperability Framework Observatory, Factsheets (published in 2015) 

[available at: https://joinup.ec.europa.eu/community/nifo/og_page/nifo-factsheets] 

-  European Commission, National Interoperability Framework Observatory (NIFO), JoinUp community 

[https://joinup.ec.europa.eu/community/nifo/home] 

-  European Commission, Performance of the Points of Single Contact - an assessment against the PSC 

Charter, 29.6.2015 [http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=8342] 

-  European Commission, Press Release IP/11/1524, 12 December 2011 [available at:  

http://europa.eu/rapid/press-release_IP-11-1524_en.htm?locale=en] 

-  European Commission, Press Release IP/13/608, June 2013 "E-invoicing in public procurement: another 
step towards end-to-end e-procurement and e-government in Europe" [ http://europa.eu/rapid/press-
release_IP-13-608_en.htm] 

-  European Commission, Proposal for a Directive of the European Parliament and of the Council on the 

accessibility of public sector bodies' websites [COM(2012) 721 final; available at http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0721:FIN:EN:PDF] 

-  European Commission, PSI Scoreboard Indicator List [http://www.epsiplatform.eu/content/psi-scoreboard-

indicator-list] 

-  European Commission, Public Services, Cross-border solutions [https://ec.europa.eu/digital-single-

market/node/77121] 

111 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  European Commission, Public Services, eParticipation [https://ec.europa.eu/digital-single-

market/eparticipation] 

-  European Commission, Public Services, ICT-enabled public sector innovation in Horizon 2020 
[https://ec.europa.eu/digital-single-market/ict-enabled-public-sector-innovation-horizon-2020] 

-  European Commission, Public Services, Towards a Cloud of Public Services [https://ec.europa.eu/digital-

single-market/towards-cloud-public-services] 

-  European Commission, Regional Policy- European Structural and Investment Funds 

[http://ec.europa.eu/regional_policy/en/funding/] 

-  European Commission, Regional Policy, Thematic objectives 

[http://ec.europa.eu/regional_policy/en/policy/what/glossary/t/thematic-objectives] 

-  European Commission, Staff Working Document – A Digital Single Market Strategy for Europe - Analysis 

and Evidence – [SWD (2015) 100 final; available at: http://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52015SC0100&from=EN] 

-  European Commission, Staff Working Document – i2010 eGovernment Action Plan: Accelerating 

eGovernment in Europe for the Benefit of All [SEC(2006) 511; available at: http://ec.europa.eu/smart-
regulation/impact/ia_carried_out/docs/ia_2006/sec_2006_0511_en.pdf] 

-  European Commission, Staff Working Document, Impact assessment, Proposal for a Directive of the 

European Parliament and of the Council  on Single-member private limited liability companies [SWD(2014) 
124 final; available at: http://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52014SC0124&qid=1458297454242&from=EN] 

-  European Commission, Staff Working Document - The European eGovernment Action Plan 2011-2015 –

Harnessing ICT to promote smart, sustainable & innovative Government [SEC(2010) 1539 final; available 
at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=SEC:2010:1539:FIN:EN:PDF] 

-  European Commission, The European e-Justice Portal [https://e-justice.europa.eu/home.do] 

-  European Commission, Workshop "Improving Transparency in Government Decision Making" 6 November 
2012 [https://ec.europa.eu/digital-single-market/news/workshop-improving-transparency-government-
decision-making-6112012-results-presentations] 

-  European Council, Conclusions, 24/25 October 2013; [EUCO 169/13, available at: 

http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ec/139197.pdf] 

-  European Parliament, European Parliament resolution of 19 January 2016 on Towards a Digital Single 
Market Act [2015/2147(INI); available at: http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-
//EP//TEXT+TA+P8-TA-2016-0009+0+DOC+XML+V0//EN] 

- 

Fornefeld, M. et al., Assessment of the Re-use of Public Sector Information (PSI) in the Geographical 
information, Meteorological Information and Legal Information Sectors, MICUS Management Consulting 
GmbH for the European Commission, 2008 [available at: https://ec.europa.eu/digital-single-
market/news/assessment-re-use-public-sector-information-psi-geographical-information-meteorological] 

112 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
- 

Funka et al. for the European Commission, Study Measuring Progress of eAccessibility in Europe – MeAC, 
2013 [available at: http://www.funka.com/en/our-assignments/position-of-trust/archive---commission-of-
trust/measuring-progress-of-eaccessibility-in-europe---meac2/] 

-  GEN6 - Governments ENabled with IPv6 [ http://www.gen6.eu/home] 

-  Geopunt, the Flemish Geoportal [http://inspire.ec.europa.eu/events/conferences/inspire_2013/pdfs/26-06-

2013_ROOM-4_09.00%20-%2010.30_95-Joeri%20Robbrecht_Joeri-Robbrecht.pdf] 

-  KPMG  for the European Commission; Gatti, R. Carbone, L. Mezzapesa,V, Study State of play of 

interoperability in Europe – Report 2014 [available at: http://ec.europa.eu/isa/documents/publications/2014-
report-on-state-of-play-of-interoperability.pdf] 

-  London Economics for the European Parliament study, A European Single Point of Contact, 2013; 
[available at: http://www.europarl.europa.eu/RegData/etudes/etudes/join/2013/507453/IPOL-
IMCO_ET(2013)507453_EN.pdf] 

-  Ministerial Declaration on eGovernment, Malmö, Sweden, on 18 November 2009 [available at: 

https://ec.europa.eu/digital-single-market/sites/digital-agenda/files/ministerial-declaration-on-egovernment-
malmo.pdf] 

-  Open Data Barometer [http://www.opendatabarometer.org/] 

-  Open Government Partnership, Independent Progress Mechanism Reports 
[http://www.opengovpartnership.org/independent-reporting-mechanism] 

-  Organisation for Economic Co-operation and Development (OECD)- Recommendation on Digital 

Government Strategies, 2014 [available at: http://www.oecd.org/gov/public-innovation/recommendation-on-
digital-government-strategies.htm] 

- 

Sixth European Ministerial eGovernment Conference “Borderless eGovernment Services for Europeans”, 
Conference proceedings, November 2011 [available at: https://mac.gov.pl/files/wp-
content/uploads/2011/12/6th-BeGSE-folder-A4-CP-EN.pdf] 

- 

STORK2 project [www.eid-stork2.eu] 

-  Tech4i2 and Deloitte for the European Commission, Study on collaborative production in eGovernment,  
2012 [available at: https://ec.europa.eu/digital-single-market/news/study-collaborative-production-
egovernment-smart-20100075] 

-  TNS Opinion & Social for the European Commission- Flash Eurobarometer 365, European Union 

citizenship, 2013 [available at:  http://ec.europa.eu/public_opinion/flash/fl_365_en.pdf] 

-  Tumin, Z, Fung, A. From Government 2.0 to Society 2.0: Pathways to Engagement, Collaboration and 

Transformation, October 2011 [available at: 
http://belfercenter.ksg.harvard.edu/publication/21475/from_government_20_to_society_20.html] 

-  UK Environment Agency, Reducing environmental risk through INSPIRE (quantified benefits), 2013 

[available at: http://www.poweredbyinspire.eu/documents/0403-sustainability-carlyle.pdf] 

-  UK Location Programme - Benefits Realisation Strategy Final, 2012 [available at: 

https://data.gov.uk/sites/default/files/Benefits%20Realisation%20Strategy%20v2.0%20Final_10.pdf] 

113 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
-  Vickery, G. Review of recent studies on PSI re-use and related market developments, Information 

Economics for the European Commission, 2011 [available at: https://ec.europa.eu/digital-single-
market/news/review-recent-studies-psi-reuse-and-related-market-developments] 

-  Vickery,G. Luukkonen,T. Radosevic,S and Fisher, R, for the European Commission, CIP ICT PSP Final 

(Second Interim) Evaluation-Final report, 2011 [available at: 
http://ec.europa.eu/cip/files/cip/docs/cip_ict_psp_interim_evaluation_report_2011_en.pdf] 

-  World Wide Web Foundation, Open data barometer Second Edition, January 2015 

[http://www.opendatabarometer.org/] 

Legislation: 

-  Decision (EU) 2015/2240 Of the European Parliament and of the Council of 25 November 2015 establishing 
a programme on interoperability solutions and common frameworks for European public administrations, 
businesses and citizens (ISA2 programme) as a means for modernising the public sector  

-  Directive 2004/18/EC of the European Parliament and the Council of 31 March 2004 on the coordination of 
procedures for the award of public works contracts, public supply contracts and public service contracts 

-  Directive 2007/2/EC Of the European Parliament and of the Council of 14 March 2007 establishing an 

Infrastructure for Spatial Information in the European Community (INSPIRE) 

-  Directive 2012/17/EU Of the European Parliament and of the Council of 13 June 2012 amending Council 

Directive 89/666/EEC and Directives 2005/56/EC and 2009/101/EC of the European Parliament and of the 
Council as regards the interconnection of central, commercial and companies registers.  

-  Directive 2014/25/EU of the European Parliament and the Council of 26 February 2014 on procurement by 

entities operating in the water, energy, transport and postal services sectors  

-  European Commission, Decision of 12 December 2011 on the reuse of Commission documents 

[2011/833/EU]  

-  Regulation(EU) No 283/2014 of the European Parliament and of the Council of 11 March 2014 on 

guidelines for trans-European networks in the area of telecommunications infrastructure and repealing 
Decision No 1336/97/EC  

-  Regulation (EU) No 910/2014 of the European Parliament and of the Council of 23 July 2014 on electronic 
identification and trust services for electronic transactions in the internal market and repealing Directive 
1999/93/EC 

114