COMMISSION STAFF WORKING DOCUMENT Evaluation of the Innovation and Networks Executive Agency (INEA) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency
Date: 2020-04-29
Institutions: European Commission, Secretariat-General
Subjects: EU aid, EU financing, EU programme, executive agency, financial control, innovation, performance audit, trans-European network, transport network

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 29.4.2020  SWD(2020) 73 final  COMMISSION STAFF WORKING DOCUMENT Evaluation of the Innovation and Networks Executive Agency (INEA) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency {COM(2020) 184 final} - {SWD(2020) 74 final} - {SWD(2020) 75 final} - {SWD(2020) 76 final} - {SWD(2020) 77 final} - {SWD(2020) 78 final}  
 
 
TABLE OF CONTENTS 

1. 

INTRODUCTION .................................................................................................................... 2 

2.  BACKGROUND ...................................................................................................................... 3 

2.1 Description of the intervention and its objectives ............................................................ 3 

2.2 Baseline and points of comparison ................................................................................... 4 

3. 

IMPLEMENTATION/STATE OF PLAY ....................................................................................... 5 

4.  METHOD .............................................................................................................................. 7 

4.1 Short description of methodology .................................................................................... 7 

4.2 Limitations and robustness of findings ............................................................................. 8 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ................................................. 9 

5.1 

5.2 

Effectiveness ................................................................................................................ 9 

Efficiency ................................................................................................................... 12 

5.3  Coherence .................................................................................................................. 16 

5.4  Retrospective Cost-Benefit Analysis ......................................................................... 17 

6.  CONCLUSIONS .................................................................................................................... 19 

7.  ANNEX: EVALUATION QUESTIONS...................................................................................... 21 

1 

 
 
 
 
 
1. 

INTRODUCTION 

The  purpose  of  this  periodical  (every  three  years)  evaluation  of  the  operation  of  the 
Innovation and Networks Executive Agency (INEA) is  to  assess  INEA’s implementation  of 
the parts of the EU funding programmes (Connecting Europe Facility, Horizon 2020 and the 
Trans-European  Transport  Network  (TEN-T)  and  Marco  Polo  legacy  programmes)  which 
have been entrusted to it.  

INEA is  governed by:  (1) the  Framework Regulation for executive agencies1;  (2) the  INEA 
‘Act of Establishment’2, which creates the Agency and sets out its mandate; (3) the INEA Act 
of Delegation3 which specifies the tasks to be carried out by, and the powers delegated to, the 
Agency in order to perform its mandate; and (4) the decision establishing the INEA Steering 
Committee4.  

In  line  with  the  Commission’s  better  regulation  principles5,  the  evaluation  applies  several 
standard evaluation criteria. The evaluation assesses whether the Agency has fulfilled its tasks 
in  an  effective  and  efficient  way,  whether  there  are  overlaps  /  gaps  /  inconsistencies  in  the 
management  of  the  programme  portfolio  by  the  Agency,  and  whether  there  is  a  clear 
delineation  of  tasks  between  INEA  and  the  parent  DGs  or  other  executive  agencies 
(coherence)6.   

The evaluation also assesses whether the functioning of the Agency has yielded the expected 
positive results as estimated in the Cost-Benefit Analysis for delegating task to the executive 
Agency7 and identify potential areas of improvement. To this end, the estimations of the Cost-
Benefit  Analysis  of  2013  have  been  tested  to  provide  evidence  on  the  validity  of  the 
assumptions  made  in  the  ex-ante  scenario  by  considering  the  actual  costs  and  benefits  of 
programme implementation by the Executive Agency in a structured way. The aspects to be 
covered  by  the  Cost-Benefit  Analysis  are  specified  in  Article  3(1)  of  the  Framework 
Regulation8 and the Guidelines on establishing and operating executive agencies9. 

1  Council  Regulation  58/2003  of  19  December  2003  laying  down  the  statute  for  executive  agencies  to  be 
entrusted with certain tasks in the management of Community programmes, OJ L 11 of 16 January 2003, p. 1. 
2  Commission  Implementing  Decision  2013/801/EU  of  23  December  2013  establishing  the  Innovation  and 
Networks Executive Agency and repealing Decision 2007/60/EC as amended by Decision 2008/593/EC. 
3 Commission decision C (2013) 9235 of 23 December 2013 delegating powers to the Innovation and Networks 
Executive Agency with a view to the performance of tasks linked to the implementation of Union programmes in 
the  field  of  transport,  energy  and  telecommunications  infrastructure  and  in  the  field  of  transport  and  energy 
research  and  innovation  comprising,  in  particular,  implementation  of  appropriations  entered  in  the  general 
budget of the Union. 
4 Commission Decision C(2014)520 of 6 February 2014 appointing the members and observers of the Steering 
Committee  of  the  Innovation  and  Networks  Executive  Agency,  as  last  amended  by  Commission  Decision 
C(2018) 769 final of 13 February 2018. 
5 Commission’s Better Regulation Communication COM (2015) 215 final and its accompanying Staff Working 
Documents SWD (2015) 111 final and SWD (2015) 110 final. 
6 The assessment of the evaluation criterion ‘EU added-value’, i.e. why the EU should act, is not perceived to be 
a relevant criterion for the evaluation of INEA as INEA carries out tasks which the European Commission has 
transferred to it. The EU added-value of the programmes that INEA  manages is assessed in the context of the 
programmes  evaluations.  The  previous  needs  which  INEA  was  meant  to  address  and  whether  they  still  are 
pertinent today (relevance) will be presented in the context of the evaluation. 
7 Cost-Benefit analysis for the delegation of certain  tasks regarding the  implementation of Union Programmes 
2014-2020 to the executive agencies - Final report for the Commission of 19 August 2013. 
8  Identification  of  the  tasks  justifying  outsourcing,  a  cost-benefit  analysis  which  includes  the  costs  of 
coordination  and  checks,  the  impact  on  human  resources,  possible  savings  within  the  general  budgetary 
framework  of  the  European  Union,  efficiency  and  flexibility  in  the    implementation  of  outsourced  tasks, 
simplification of the procedures used, proximity of outsourced activities to  final beneficiaries, visibility of the 

2 

 
 
 
 
 
                                                           
The evaluation does not cover the achievements of the programmes managed by INEA, which 
are  subject  to  mid-term  and  ex-post  evaluations  themselves.  The  evaluation  of  INEA 
nevertheless  provides  useful  input  for  these  programme  evaluations,  considering  that  the 
performance  of  the  Agency  affects  the  efficiency  and  effectiveness  of  the  programmes  it 
manages. 

The evaluation  examines the efficient  use of resources and the  effective achievement of the 
tasks entrusted to the Agency. It looks in particular at whether the alignment of more coherent 
programme  portfolios  with  the  Agency’s  core  competences  and  its  brand  identity  delivered 
the estimated qualitative benefits; whether the assembly of the management  of different  EU 
programmes  delivered  the  estimated  synergies,  simplification  and  economies  of  scale; 
whether the pooling of instruments guaranteed consistent service delivery and whether there 
is scope for simplification and further efficiency gains.  

The evaluation covers all the tasks carried out by the Agency during the period from January 
2014  to  December  2016.  The  evaluation  is  supported  by  a  study  carried  out  by  an  external 
contractor. 

The results of this study are summarised hereafter and this evaluation will be presented to the 
European Parliament, to the Council and to the Court of Auditors in accordance with Article 
25(1) of the Framework Regulation.  

The results will feed into the reflection to assess the opportunity of expanding/ modifying the 
Agency’s  mandate  in  view  of  the  delegation  of  the  implementation  of  2021-2027  EU 
programmes. 

2.  BACKGROUND 

2.1 Description of the intervention and its objectives 

The  outsourcing  of  certain  management  tasks  to  the  Agency,  as  outlined  in  the  Framework 
Regulation and in INEA’s Act of Establishment, intends to: 

  Allow  the  Commission  to  focus  on  its  institutional  tasks,  i.e.  tasks  assigned  to  the 
institutions  by  the  Treaty  which  require  discretionary  powers  in  translating  political 
choices into action. Such institutional tasks should not be outsourced. 

  Enable  the  Commission  to  achieve  the  objectives  of  the  delegated  EU  programmes 
more  effectively.  According  to  the  Cost-Benefit  Analysis  carried  out  in  2013,  the 
delegation of certain programme tasks to the Agency was estimated to be more cost-
efficient than an in-house scenario. The Act of Establishment stated that the alignment 
of more coherent  programme portfolios  with  the Agency’s core competences and its 
brand  identity  would  bring  qualitative  benefits.  In  addition,  it  projected  that 
assembling  the  management  of  different  EU  programmes  would  bring  synergies, 
simplification and economies of scale. 

The original objective of the intervention is entrusting the Agency with the implementation of 
several EU programmes in part or fully:  

EU as promoter of the EU programme concerned and the need to maintain an adequate level of know-how inside 
the Commission. 
9  Appendix  II  of  the  Guidelines  for  the  establishment  and  operation  of  executive  agencies  financed  from  the 
Union budget (C (2014) 9109 from 2 December 2014; pp. 64-72). 

3 

 
 
 
 
 
 
 
                                                                                                                                                                                     
  Connecting Europe Facility: transport, energy and telecommunications,  
  Horizon  2020:  transport  and  energy  research  (Part  III  Societal  challenges  of  the 

Specific Programme),  

  Legacy of the Trans-European Transport Network (TEN-T) programme, 
  Legacy of the Marco Polo II programme (transferred from the Executive Agency for 
Competitiveness  and  Innovation  (EACI),  now  Executive  Agency  for  Small  and 
Medium-Sized Enterprises (EASME). 

As of 1 May 2018, the mandate of the Agency was extended to include implementation of the 
new Wifi4EU initiative as part of CEF Telecoms10. 

The Agency works under the supervision of four parent Directorates-General: 

  DG Mobility and Transport (MOVE) – Lead parent DG; 
  DG Energy (ENER); 
  DG Communications Networks, Content and Technology (CNECT); and 
  DG Research and Innovation (RTD). 

In  implementing  the  programmes  delegated  to  it,  INEA  is  responsible  for  monitoring  the 
projects,  making  the  necessary  checks  and  recovery  procedures,  and  performing  budget 
implementation  tasks  covering  revenue  and  expenditure  within  the  meaning  of  the  EU 
Financial Regulation11, and in particular: 

  Manage the operations and procedures leading to the adoption of Commission award 
decisions and to the conclusion of grant agreements and manage the ensuing decisions 
and agreements; 

  Provide support in programme implementation; 
  Perform all the operations required to launch contests and award prizes in accordance 

with the EU Financial Regulation; 

  Conclude  public  procurement  procedures  and  manages  the  ensuing  contracts, 
including  the  operations  required  to  launch  and  conclude  public  procurement 
procedures. 

2.2 Baseline and points of comparison 

The  current  evaluation  of  INEA  operations  during  2014-2016  assesses  the  actual  costs  and 
benefits of programme implementation by INEA (executive agency scenario) when compared 
with the alternative scenario of management by the Commission services (in-house scenario). 

Accordingly,  the  reference  point  for  the  present  INEA  evaluation  is  the  2013  ex  ante  Cost-
benefit analysis, and the Specific Financial Statement 12 (SFS) of INEA. 

10  Commission  Decision  C(2018)  1281  final  of  27.2.2018  and  C(2018)  6366  final  of  4.10.2018  on  amending 
Decision  C(2013)9235  delegating  powers  to  the  Innovation  and  Networks  Executive  Agency,  as  regards 
promotion of internet connectivity in local communities. 
11 Regulation (EU, Euratom) 2018/1046 of the European Parliament and of the Council  of 18 July 2018 on the 
financial  rules  applicable  to  the  general  budget  of  the  Union,  amending    Regulations    (EU)    No    1296/2013,  
(EU)  No  1301/2013,  (EU)  No  1303/2013, (EU) No  1304/2013,  (EU)  No  1309/2013,  (EU)  No  1316/2013,  
(EU)  No  223/2014,  (EU)  No 283/2014,  and  Decision  No  541/2014/EU  and  repealing  Regulation  (EU,  
Euratom)  No 966/2012, OJ L 193, 30.7.2018, p.1. 
12 INEA’s  SFS  was updated in March 2018 in the  context of the extension of INEA’s  mandate  to include the 
delegation of the Wifi4EU programme under CEF Telecoms. 

4 

 
 
 
 
 
 
 
 
                                                           
Wherever possible, the analysis of the INEA performance during the reference period assesses 
the  progress  achieved  since  the  two  previous  evaluations  (of  the  precedent  TEN-T  EA), 
covering the Agency’s operations during 2008-2011 and 2011-2013. 

It should be noted that over the period 2008-2011 the savings resulting from the delegation of 
tasks to TEN-T EA have been evaluated at around EUR 8.6 million compared to the in-house 
scenario. The estimated efficiency gains over the period 2011-2013 were of EUR 8.8 million 
and of EUR 24.4 million over the period 2014-2016. 

According to the SFS of INEA prepared in 2013 the estimated efficiency gains resulting from 
programme  delegation  to  INEA  were  of  EUR  22.8  million  over  the  period  2014-2016.  The 
total number of full time equivalents (FTEs) required to manage the relevant programmes in 
year 2020, the peak programming year in terms of workload, was estimated at 318 for INEA. 
The total initial operational budget entrusted to INEA in 2020 was estimated at around EUR 
37  billion  in  both  commitment  and  payment  appropriations.  Considerable  efficiency  gains 
were  expected  as  compared  to  the  in-house  scenario  over  the  period,  along  with  non-
quantifiable  benefits  such  as  improved  quality  of  programme  management  and  service 
delivery, improved visibility of the EU programmes and proximity to beneficiaries. 

3. 

IMPLEMENTATION/STATE OF PLAY 

In  2006,  by  Commission  Decision  2007/60/EC13,  the  Commission  established  the  Trans-
European  Transport  Network  Executive  Agency  (TEN-T  EA)  and  entrusted  it  with  the 
management of Community actions in the field of the trans-European transport network. The 
initial duration of the Agency was limited to 31 December 2008. Then the main task of the 
Agency was to manage the remaining open TEN-T projects and financing decisions under the 
Multi-annual Financial Framework (MFF) 2000 – 2006. 

Subsequently, in  2008, by  Decision 2008/593/EC14, the Commission extended the period of 
operation of the TEN-T  EA to  31  December  2015 and  redefined its objectives and tasks  so 
that  it  became  responsible  also  for  the  implementation  of  financial  aid  from  the  trans-
European  transport  network  budget  under  the  MFF  2007-2013.  Both  the  establishment  and 
extension of the TEN-T EA were preceded by dedicated cost-benefit analyses. 

TEN-T EA managed a TEN-T programme budget of EUR 0.8 billion during the 2000-2006 
MFF and EUR 7.8 billion during the 2007-2013 programming period. In the latter period the 
Agency’s project portfolio consisted of 411 projects and by end 2013 its staff was around 100.  

The  Innovation  and  Networks  Executive  Agency  (INEA)  was  set  up  on  23  December  2013 
and is an autonomous legal entity since 1 January 2014. It is responsible for the management 
of parts of the Connecting Europe Facility (CEF) and of Horizon 2020 as well as of the TEN-
T and the Marco Polo legacy programmes. 

13Commission  Decision  of  26  October  2006  establishing  the  Tran-European  Transport  Network  Executive 
Agency pursuant to Council Regulation (EC) No 58/2003 (OJ L 32, 6.2.2007, p.88). 
14Commission Decision of 11 July 2008 amending Decision No 2007/60/EC as regards the modification of the 
tasks  and  the  period  of  operation  of  the  Trans-European  Transport  Network  Executive  Agency  (OJ  L  190, 
18.7.2008, p. 35).  

5 

 
 
 
 
 
 
 
 
 
                                                           
The new mandate also brought changes in the governance of the Agency as three new parent 
DGs joined the Steering Committee. As stipulated in the Act of Delegation, a Memorandum 
of Understanding was signed on 1 October 2014, which defines the modalities and procedures 
of  interaction  between  INEA  and  its  parent  DGs15.  In  addition,  it  brought  a  clearer 
delimitation  of  the  administrative  activities  between  the  Agency  and  its  parent  DGs.  The 
present evaluation assesses whether this distribution of roles has been complied with for the 
2014-2016  period,  as  well  as  the  effects  of  the  intervention  on  the  coherence  and  the 
efficiency of INEA’s action. 

The  total  operational  budget  earmarked  for  INEA  for  the  MFF  2014-2020  is  EUR  33.8 
billion,  of  which  EUR  28.5  billion  is  accounted  for  by  the  CEF  and  EUR  5.3  billion  by 
Horizon 2020. This means that INEA handles the largest budget of all the executive agencies 
in the current financial period of 2014-2020.  

The operational budget managed by INEA in 2014-2016 was approximately EUR 11.9 billion 
in  terms  of  commitments  (EUR  3.6  billion  in  2014,  EUR  3.2  billion  in  2015  and  EUR  5.1 
billion in 2016) and EUR 5.1 billion in terms of payments (EUR 0.6 billion in 2014, EUR 2 
billion  in  2015  and  EUR  2.5  billion  in  2016).  The  Agency’s  administrative  budget  evolved 
from EUR 13.1 million in 2014 to EUR 18.2 million in 2015 and EUR 21.5 million in 2016.  

The total budget per head in terms of payments increased from EUR 3.97 million in 2014 to 
10.75 million in 2015 and to EUR 11.11 million in 2016. The increasing volume of INEA’s 
operational  budget  during  this  period  outpaced  the  increase  in  the  number  of  the  Agency’s 
staff. 

The  number  of  proposals  received  and  evaluated  in  2014-2016  was  in  total  3  490,  i.e.  very 
close to the CBA estimations. The number of proposals managed per head varied between 3.1 
in 2014, 8.3 in 2015 and 6.5 in 2016, which results mainly from the heavy frontloading of the 
CEF Transport budget (2014 and 2015 calls, evaluated respectively in 2015 and 2016). 

The number of INEA running projects increased from 496 in 2014 to 1026 in 2015 and 1383 
in 2016. While the number of projects in 2014 was below the CBA estimations (which related 
to  the  later  than  initially  forecasted  launch  of  the  2014  calls  and  later  conclusion  of  grant 
agreements), in 2015-2016 the actual number of running projects generally reached the CBA 
estimations.  Programme  management  costs  (defined  as  the  ratio  between  the  administrative 
and operational budget in payments) decreased from roughly 2% in 2014 to 0.9% in 2015 and 
0.9% in 2016. 

A  series  of  organisational  and  procedural  changes  were  introduced  to  render  more  efficient 
the  implementation  of  the  delegated  programmes  under  the  MFF  2014-2020,  notably  in  the 
evaluation period: 

15 Memorandum of Understanding between the Directorate-General for Mobility and Transport (DG MOVE), the 
Directorate-General for Energy (DG ENER), the Directorate-General for Communication Networks, Content and 
Technology  (DG  CNECT)  and  the  Innovation  and  Networks  Executive  Agency  (INEA)  of  1  October  2014, 
subsequently  amended.  The  MoU  is  formed  by  four  documents:  Part  I  –  General  Provisions,  Part  IIa  – 
Connecting Europe Facility, Part IIb – Horizon 2020, and Annex – Administrative and logistical support services 
provided by REA. The General Provisions and Horizon 2020 parts of the MoU were updated in December 2015 
to  take  into  account  the  lessons-learned  concerning  programme  implementation  as  well  as  administrative 
developments within the Commission. The CEF part was updated in July 2018 notably also to take into account 
the delegation of the Wifi4EU programme under CEF Telecoms to INEA on 1 May 2018. 

6 

 
 
 
 
 
 
 
 
                                                           
  Further  simplification  and  improvement  of  grant  management  processes  and 
procedures,  notably  concerning 
the  TEN-Tec  portal 
functionalities  concerning  submission  and  evaluation  of  proposals,  preparation  and 
signature  of  Grant  Agreements  and  reporting  (in  2014,  the  Agency  merged  the 
submission of the Strategic Action Plan (SAP) with the Action Status Report (ASR) – 
previously two separate documents); 

the  widespread  use  of 

  The adoption of new business processes, Horizon 2020 IT tools and modes of service 

delivery; 

  The adoption in April 2017 of a multi-annual human resource (HR) strategy, covering 
the years 2016 – 2020, including an action plan for following up on the key findings 
from  the  2016  staff  opinion  survey.  Based  on  this  strategy,  the  Agency  is 
implementing  specific  measures  related  to  staff  engagement,  training,  internal 
communication,  career  development  opportunities,  well-being  and  the  balance 
between work and private life, aimed at improving HR management and increasing the 
effectiveness  of  the  Agency’s  operations.  A  staff  retention  policy  has  since  been 
approved  in  September  2018  to  increase  staff  satisfaction  levels  and  help  reducing 
staff turnover. 

4.  METHOD 

4.1 Short description of methodology 

The  evaluation  was  supported  by  a  study  carried  out  by  an  external  contractor16.  The 
methodology  of  the  INEA  evaluation  was  consistent  with  the  approaches  employed  for 
similar  parallel  evaluations  of  EACEA,  EASME,  ERCEA,  CHAFEA  and  REA.  The  study 
was structured around a series of evaluation questions available in Annex. 
A  combination  of  evaluation  methods  providing  for  the  collection  of  qualitative  and 
quantitative information and evidence included:  

  Documentary  review  and  desk  research  concerning  all  relevant  documentation 
relating to INEA, including the legal basis of the executive agencies and of INEA,  the 
Memorandum  of  Understanding  between  the  parent  DGs  and  INEA  (MoU),  Annual 
Work Programmes, Annual Activity Reports, European Court of Auditors (ECA) and 
Internal Audit Service (IAS) reports and related documents, previous evaluations and 
CBA studies, financial documents, and action plans; the evaluation relied also on the 
results of the 2016 staff opinion survey; 

  Semi-structured  or  in-depth  interviews  with  EU  staff  and  stakeholders  of  the 
delegated programmes. 59 interviews were carried out with: staff of the parent DGs, of 
other  DGs  represented  in  the  INEA  Steering  Committee,  and  of  INEA;  project 
applicants and beneficiaries; external experts contracted to carry out the external phase 
of the CEF and Horizon 2020 evaluations;  

  Survey  addressed  to  stakeholders,  experts  and  Member  States’  officials  involved  in 
the implementation of programmes managed by INEA. 1878 responses were received: 
1404  from  project  applicants  and  beneficiaries;  382  from  external  experts;  79  from 
programme  committee  members;  and  13  from  members  of  INEA’s  CEF  Transport 
Advisory Group; 

16 Study supporting the Evaluation of the Innovation and Networks Executive Agency (INEA) 2014-2016, Final 
report 5 February 2019, performed by the Centre for Strategy & Evaluation Services (CSES), the Public Policy 
and Management Institute (PPMI) and the Centre for Industrial Studies (CSIL). 

7 

 
 
 
                                                           
  A retrospective cost–benefit analysis; 
  A comparative analysis and benchmarking of the various executive agencies was also 
carried out  against a set  of (qualitative and quantitative) indicators. This  enabled the 
evaluation  team  to  compare  the  performance  of  INEA  effectively  with  that  of  other 
executive agencies. 

Case studies were used to complement the desk research and interview /  survey programme 
and to provide insights into the performance of the Agency on issues such as coherence and 
maintenance of the know-how within the Commission, efficiency of simplification measures 
etc. Those case studies have fed the different assessment questions. 

4.2 Limitations and robustness of findings 

A representative sample of Commission and INEA staff and stakeholders (both applicants of 
rejected  proposals  and  beneficiaries)  and  other  officials  /  experts  (external  experts, 
programme committee members and advisers) were consulted in the course of the evaluation. 
Compared to the overall population reached, unsuccessful applicants participated less actively 
in the survey than programme beneficiaries.  

No  sampling  bias  was  observed  as  the  profile  of  the  respondents  to  the  surveys  was  very 
similar to the overall population, guaranteeing statistical representativeness. The non-response 
bias  (not  all  characteristics  of  the  group  that  did  not  reply  had  been  captured  in  full)  was 
mitigated  through  triangulation  with  the  results  of  follow  up  interviews.  The  triangulation 
approach,  using  multi-level  and  multi-stakeholder  dimension  in  the  data  collection,  ensured 
the robustness and reliability of the data and information used to draw up conclusions in the 
supporting study. 

A  mix  of  parent  DGs  and  INEA  managers  and  policy  /  project  officers  were  interviewed 
extensively.  Some  answers  to  evaluation  questions,  notably  in  the  ‘Coherence’  section,  rely 
largely  on  interview  and  survey  data.  Some  views  from  Commission  and  Agency  staff 
gathered  from  the  interviews  and  mentioned  in  the  study  illustrate  personal  experience, 
although they may not be representative of overall relations between the Commission and the 
Agency. 

Also,  it  should  be  noted  that,  in  parallel  to  this  evaluation,  the  European  Court  of  Auditors 
(ECA)  carried  out  a  performance  audit  to  ascertain  if  INEA’s  management  of  the  EU 
delegated  programmes  optimise  their  implementation.  The  audit  focused  on  the  2014-2020 
programming period and covered INEA’s set-up and organisation, and its implementation of 
the CEF programme. It involved inter alia the examination of the regulatory framework, the 
responsibilities  and  the  control  systems  in  place  as  well  as  INEA’s  performance  in  the 
management of the CEF programme and projects. In its report, the ECA inferred that INEA is 
a  well-organised  Agency  and  that  it  is  implementing  the  programmes  delegated  to  it,  in 
particular the CEF, efficiently. However, the report also made a number of recommendations 
to enhance the implementation of the Connecting Europe Facility and the overall operational 
performance  of  the  Agency,  such  as:  a  multi-annual  programming  of  calls  for  proposals; 
increased  synergies  between  CEF  and  Horizon  2020  (Horizon  Europe);  a  performance 
framework  to  measure  programme  and  project  achievements;  an  upgraded  and  harmonised 
procedure for the evaluation and selection of project proposals; to enhance the link between 
funding  and  the  achievement  of  project  milestones;  and  to  ensure  more  flexibility  in  the 
allocation  of  staff  among  programmes  within  the  Agency.  The  Commission  takes  the  view 
that the European Parliament and Council common understanding on the new CEF Regulation 

8 

 
 
 
 
 
 
is  consistent  with  the  ECA’s  recommendations.  The  Commission  and  INEA  are  already 
addressing  the  ECA’s  recommendations  with  a  view  to  improving  programme  management 
under the present MFF and in view of the CEF 2 programme. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

5.1 Effectiveness  

For  the  purpose  of  this  evaluation,  effectiveness  relates  to  how  successful  the  Agency  has 
been in achieving or progressing towards its objectives. 

The  evaluation  inferred  that  during  2014-2016  INEA  operated  according  to  the  legal 
framework that established it. INEA’s activities, as planned in the annual work programmes 
and  reported  in  the  annual  activity  reports,  corresponded  to  the  tasks  set  out  in  the 
Commission’s establishment and delegation Decisions. 

As provided in the delegation Decision, the modalities and procedures of interaction between 
the Agency and its parent DGs are set out in the Memorandum of Understanding (MoU) of 1 
October  2014.  The  MoU  provides  for  a  supervision  strategy  aimed  at  avoiding  gaps  or 
duplication  of  efforts  resulting  from  crossover  between  the  policy-making,  monitoring  and 
supervision tasks of the parent DGs and the execution tasks of the Agency. These provisions 
have generally worked well and all parent DGs and INEA appreciate the effectiveness of the 
collaboration.  

In  the  evaluation  period,  INEA  continuously  improved  its  operations  and  achieved  a  high 
level  of  overall  effectiveness.  The  Agency  was  flexible  and  effective  in  addressing  the  key 
challenges that occurred during the evaluation period, in particular:  

 

 

 

 
 

the continuous phasing-out of its two legacy programmes TEN-T and Marco Polo (the 
latter transferred from EACI, now EASME);  
the  evaluation  and  grant  award  process  for  the  CEF  Transport  calls  for  proposals, 
especially  the  first  and  largest  ever  CEF  Transport  call  in  2014/2015  involving  the 
evaluation of almost 700 proposals for a final selection decision for funding of EUR 
13.1 billion;  
the gradual  assumption of responsibilities on the implementation  of the  CEF Energy 
and  Telecoms  programmes  and  the  Horizon  2020  sub-programmes  concerning 
transport and energy;  
the launch of a CEF synergies (transport and energy) call in 2016; and  
the preparation as of 2016 of an innovative CEF Transport Blending call for EUR 1,35 
billion, launched in 2017.  

INEA  has  developed  strong  internal  control  standards,  including  on  the  management  of 
financial  and  human  resources.  The  Agency  established  overarching  control  strategy  with  a 
large number of control and reporting mechanisms that allow progress against objectives to be 
closely  monitored  and  risks  to  its  operations  to  be  prevented  and  mitigated  in  a  timely  and 
effective  way.  INEA  has  in  particular  established  a  coherent  anti-fraud  strategy  which  it 
pursues effectively. This is reflected in assessments undertaken by the Court of Auditors and 
by the Commission’s Internal Audit Service. 

9 

 
 
 
 
 
 
 
 
In  2014-2016  INEA  achieved  almost  all  its  targets  in  terms  of  key  performance  indicators 
(KPI).  Only  the  target  for  the  net  time-to-pay  for  the  TEN-T  programme  2007-2013  was 
slightly exceeded in 2015 by 1 day (31 days instead of 30). The prompt conclusion of grant 
agreements  (measured  in  ‘Time-To-Grant’  –  TTG)  varied,  for  Horizon  2020,  between  a 
minimum of 216 days in 2015 and a maximum of 225 days in 2014 for Energy, and between 
232 days in 2015 and 237 days in 2014 for Transport (still below the Horizon 2020 target of 
245  days).  Concerning  the  CEF  programme,  the  longest  sectorial  TTG  in  2015-2016  (no 
grants were awarded and signed in 2014) were 264 days for CEF Transport, 249 days for CEF 
Energy and 253 days for CEF Telecoms, all below the ceiling of 276 days. The share of grants 
concluded within TTG targets, for both Horizon 2020 and CEF, reached nearly 100% for the 
2014-2016 calls.  

Regarding  payments  of  grants,  the  average  Time-to-Pay  (TTP)  stood  well  below  the 
contractual thresholds for all types of payments (pre-financing, interim and final payments) in 
2014-2016  both  for  Horizon  2020  and  CEF.  Concerning  the  share  of  payments  within 
contractual limits, nearly 100% of all pre-financing, interim and final payments in 2014-2016 
were executed on time.  

INEA  achieved  full  execution  of  its  operational  budget  during  2014-2016,  both  in 
commitment and payment appropriations; with the exception of the execution of commitment 
appropriations for the TEN-T17 and Marco Polo II programmes in 2014, both of which were 
well below the target of 100% , although this target was subsequently met in 2015 and 2016. 

With regard to the legality and regularity of transactions, the estimated residual multi-annual 
error  rate  remained  well  below  the  threshold  of  2%  for  the  TEN-T  and  Marco  Polo  II 
programmes  in  2014-2016.  Concerning  CEF  and  Horizon  2020,  this  KPI  was  not  yet 
applicable and there were not enough audits in this period to draw any conclusions; however, 
it is estimated that, by the end of the programme implementation, the error rate will be of 2% 
cumulative for CEF while for Horizon 2020 it is established within the range of 2% - 5%. 

17 For TEN-T, this was largely due to a lower than expected response to the 2013 Call for proposals. 

10 

 
 
 
 
 
 
 
                                                           
Summary of the KPIs over the period 2014-16 

Source: INEA activity reports 2014, 2015, 2016 

Despite  some  shortcomings  in  call  coordination  between  the  parent  DGs  concerning  the 
evaluation  period,  INEA  adapted  well  to  peak  periods  of  very  high  work-load.  Such  heavy 
work-load  was  especially  felt  in  2015/2016  on  the  occasion  of  the  2014  and  2015  CEF 
Transport  calls  for  proposals,  when  the  Agency  was  not  yet  fully  staffed.  Regular  meetings 
between  the  parent  DGs  and  INEA  resulted  overtime  in  more  efficient  planning  and 
coordination of the calls, though continuous efforts on call coordination are necessary in the 
run up to the last calls of the present MFF.  

While  providing  recommendations  for  addressing  minor  shortcomings,  the  reports  of  the 
independent observers accompanying the evaluations, for both CEF and Horizon 2020, were 
consistently  complimentary  on  the  transparency  and  efficiency  of  the  evaluation  process. 
Concerning  Horizon  2020,  INEA’s  efforts  to  improve  the  quality  of  the  evaluation  process 
were  reflected  in  a  very  small  share  of  evaluation  review/redress  cases  filed  and  (fully  or 
partially) upheld compared to the number of proposals evaluated18.  

The survey of INEA’s applicants and beneficiaries indicates satisfaction in their dealings with 
the  Agency,  with  a  great  majority  of  them  considering  that  INEA  has  contributed  to  a 
significant  improvement  of  the  management  of  the  delegated  programmes  and  helped  to 
develop  better  services.  90%  of  respondents  stated  that  INEA  managed  the  procedures 
(applications,  evaluation,  contract  negotiations,  project  monitoring,  etc.)  in  relation  to  their 
project very or quite efficiently. Only 8% of respondents indicated that during the application 
process  INEA  was  unresponsive  to  the  queries  or  problems  applicants  raised  and  only  9% 
indicated that the decision on the application was not taken within the acceptable timeframe. 

It  should  be  noted  however,  that  applicants’  recommendations  included  a  number  of  areas 
where improvement may be necessary, for example:  

  establishing  a  “hotline”  for  National  Contact  Points/Committee  members  to  obtain 

instant information and advice on application procedures;  
improving the feedback to applicants on the rejected proposals;  

 
  organising local information meetings on calls for proposals and other implementation 

matters and making them accessible online;  

  more  flexibility  in  reporting  on  project  progress,  both  regarding  how  to  submit  and 

save documentation (paper vs digital) and more use of electronic signatures; 

18 In 2015 the redress case rate was 2.4%: 749 proposals evaluated; 18 redress cases submitted; 0 re-evaluations 
(all evaluation results were confirmed).  In 2016 the redress case rate was 2.6% and the re-evaluation case rate 
was 0.14%: 734 proposals evaluated; 19 cases submitted; 1 re-evaluation done.  

11 

KPI’sTargetsTen-TMarco PoloCEFHorizon 2020Ten-TMarco PoloCEFHorizon 2020Ten-TMarco PoloCEFHorizon 2020 (1) Rate of execution of commitment appropriations100%85%60%100%100%N/AN/A100%100%N/AN/A100%100%(2) Rate of execution of payment appropriations100%100%100%N/AN/A100%100%100%100%100%100%100%100%(3) Time to grant (days)276274271N/A231N/AN/A242224N/AN/A254228(4) Net time to pay (Pre-financing) (days)3016.926.6N/AN/A3126.19.514N/AN/A1111.6(5) Error rate of payments at ex-post control< 2%0.80%1.1 - 1.8%N/AN/A0.78%0.97%N/AN/A0.80%0.37%N/AN/A201420152016 
 
 
 
 
 
 
                                                           
 

improving  the  participant  portal  and  more  workshops  to  update  on  progress  of  the 
programmes19; and  

  guidance on how to better manage project risks. 

Overall,  external  experts  showed  high  levels  of  satisfaction  with  INEA’s  selection  and 
contracting  process,  whether  they  worked  on  Horizon  2020,  on  CEF,  or  both.  58.8%  of  all 
experts  were  very  satisfied  and  an  additional  32.4%  were  quite  satisfied.  Overall  less  than 
10%  were  quite  or  very  dissatisfied,  or  had  no  response.  Experts  working  on  Horizon  2020 
alone  had  the  highest  level  of  satisfaction,  with  only  4.1%  dissatisfied  in  any  way  (an 
additional  4.5%  did  not  respond  or  were  unable  to  comment).  By  contrast,  9.4%  of  those 
working on both CEF and Horizon 2020 were dissatisfied, as were 11.8% of those working on 
CEF  alone.  Experts  working  on  both  projects  were  also  least  likely  to  be  very  satisfied 
(46.9%) compared to those working on Horizon 2020 (61.5%) or CEF (60.3%) alone. 

Experts overwhelmingly stated that evaluation / review consensus meetings are run efficiently 
by  INEA.  While  only  4.5%  of  the  experts  who  replied  to  the  survey  said  they  were 
dissatisfied,  the  proportion  of  experts  who  were  very  satisfied  was  51.6%.  Concerning 
perceptions  on  the  quality  of  evaluation/review  activities,  just  2%  felt  that  they  were  low 
quality, with 56.3% believing they were of high quality across the board.  

The extension of the Agency’s mandate and change of name (from TEN-T EA to  INEA) in 
2014 did not have an impact in respect of proximity to stakeholders and the visibility of the 
EU as a promoter of the programmes delegated to it. In terms of proximity to the addressees, 
INEA acted as a direct contact point between programme applicants and beneficiaries of EU 
funding, as  well as other stakeholders, and the Commission.  The most  regular  contact  takes 
place between project beneficiaries and the respective project officers at INEA. Separation of 
policy making and implementation benefits the beneficiaries insofar as they obtain specialised 
advice  and  service.  During  the  evaluation  period,  INEA  made  sustained  efforts  to  boost 
awareness of new funding opportunities for project promoters (both under CEF and Horizon 
2020) and to consolidate a service-oriented communication. It also supported parent DGs by 
giving visibility to the delegated programmes via success stories. 

Concerning  visibility  of  the  EU  as  a  promoter  of  the  programmes  entrusted  to  INEA,  the 
survey suggests that there is awareness amongst stakeholders that the Agency is acting under 
powers  delegated  by  the  Commission.  The  Agency’s  contributions  to  enhanced  visibility  of 
the programmes included Info-days, project management workshops, pro-active use of social 
media tools and good quality graphic and communication materials. Additionally, the website 
of the Agency includes a significant amount of material related to the delegated programmes: 
project  information,  brochures,  maps  and  statistics.  The  many  channels  of  communication 
used by the Agency have resulted in the enhanced visibility of the programmes. 

5.2 Efficiency  

This  section  considers  the  relationship  between  the  resources  used  by  the  Agency  and  the 
output.  The  analysis,  among  other  factors,  also  includes  analysis  of  administrative  and 
regulatory burden and looks at aspects of simplification. 

19  Applicants  were  also  suggesting  developing  an  online  platform  where  INEA  could  showcase  important 
developments  across  different  programme  areas.  INEA’s  website  is  already  a  dissemination  platform  for  the 
managed portfolio of projects. 

12 

 
 
 
 
 
 
 
                                                           
During  2014-2016  INEA  proved  to  be  an  efficient  and  cost-effective  structure  for  the 
management  of  the  delegated  programmes.  Its  administrative  budget  decreased  from  2%  of 
the operational budget based on payment appropriations in 2014 to 0.9% in 2016. 

INEA’s  actual  operational  budget  during  2014-2016  (in  commitment  appropriations) 
constituted  EUR  11,788  billion,  of  which  EUR  11,679  billion  from  EU  budget  and  EUR 
109,2  million  related  to  EEA/EFTA  and  third  countries’  contributions  (not  considered  in 
INEA’s  SFS).  INEA’s  administrative  budget  (including  such  contributions)  increased  from 
EUR 13.1 million in 2014 to EUR 18.2 million in 2015 and EUR 21.5 million in 2016. 

A  detailed  analysis  of  the  INEA  cost-efficiency  and  results  of  the  Cost-benefit  analysis  is 
presented under point 5.4. 

Programme  implementation  cost  (ratio  between  the  administrative  and  operational 
budget, percentage), payment appropriations 

10,00

8,00

6,00

4,00

2,00

0,00

7,97 

2,05 

2014

5,27 

0,89 

2015

4,72 

0,89 

2016

Source: CSES, PPMI and CSIL, based on INEA Annual Activity Reports. 

INEA

Average of the EAs

The number of proposals managed by INEA in 2014-2016 was 3490, very close to the CBA 
forecast of 3576. The number of total running projects increased from 496 in 2014 to 1383 in 
2016. During the evaluation period, the actual workload of INEA was higher than estimated 
in  the  2013  CBA  due  to  factors  beyond  the  Agency’s  control  that  influenced  its  workload, 
such  as  the  average  grant  size,  the  number  of  proposals  and  grants,  and  the  re-injection  of 
budget amounts. The number of running projects ‘per operational head’ increased from 4.4 in 
2014 to 7.6 in 2016 as a result of the first Calls of the 2014-2020 MFF. The ratio of budget 
‘per  operational  head’  did  not  fluctuate  much  in  terms  of  commitments  (from  EUR  31.8 
million to EUR 27.8 million) but increased significantly in terms of payments (from EUR 5.4 
million to EUR 13.5 million) in 2014-2016.  

INEA Budget, staff and workload indicators, 2014-2016 (in € million, n° or %) 

2014 

2015 

2016 

Operational budget, commitments 
Operational budget, payments 
Administrative budget, commitments 
Administrative budget, payments 
Actual number of staff (at the end of the year) 
Actual number of operational staff  
Programme management cost* (commitments) 
Programme management cost (payments) 

3590 
606 
13.1 
12.4 
151 
113 
0.4% 
2.0% 

13 

3251 
2021 
18.2 
17.9 
186 
145 
0.6% 
0.9% 

5057 
2448 
21.5 
21.9 
225 
182 
0.4% 
0.9% 

 
 
 
 
 
 
 
 
  
Budget ‘per operational head’ (commitments)  
Budget ‘per operational head’ (payments) 
Proposals received 
Total running projects  
Running projects ‘per operational head’  
* Defined as the ratio between the administrative and operational budget. 

31.8 
5.4 
479 
496 
4.4 

22.4 
13.9 
1548 
1026 
7.1 

27.8 
13.5 
1463 
1383 
7.6 

Human Resource Management 
In  order  to  optimise  allocation  of  human  resources  across  its  delegated  Programmes,  INEA 
has developed a methodology to calculate its operational workload and staffing needs which 
was  approved  by  its  Steering  Committee  in  February  2018  and  which  has  been  kept 
continuously updated. The methodology is based on three main workload drivers: proposals to 
be evaluated, grant agreements to be signed, ongoing projects to be managed. While current 
budgetary  rules  do  not  allow  shifting  Agency  staff  between  Programmes,  the  work-load 
assessment  methodology  proved  useful  to  allocate  staff  more  efficiently  within  Programme 
sectors, in agreement with the parent DGs. 

INEA  had  to  cope  with  an  increase  of  its  staff  turnover  rate  from  7.5%  in  2015  to  close  to 
10% in 2016 (the target level being 3%). INEA has actively addressed the underlying issues 
and  has  elaborated  a  new  staff  retention  policy  with  elements  such  as  encouraging  internal 
staff  mobility,  announcing  vacant  positions  internally  and  organising  job  shadowing.  As  a 
consequence, INEA is now being more effective in retaining staff and filling vacancies, and at 
the end of 2018 the vacancy rate had dropped to 5.4%. 

The staff opinion survey carried out in 2016 revealed an increase in INEA’s staff satisfaction 
compared to 2014. The staff engagement index is a composite indicator20 that helps measure 
how connected the staff is to the Agency and how committed to help it achieve its goals. In 
2016, INEA had a staff engagement index result of 68%, above average of the other executive 
agencies  (66%)  and  the  Commission  (64%).  With  an  improvement  of  5%  from  2014,  it 
clearly  shows  that  good  progress  has  been  done  in  this  area.  The  overall  INEA  staff  job 
satisfaction  in  2016  was  within  the  average  of  the  Commission  departments  (71%  in  INEA 
compared to 70% in the Commission) and the other executive agencies.  

Several other indicators are evidences of the good results achieved by INEA in terms of HR 
management. 62% of INEA staff felt the Agency was a modern and attractive workplace, 6% 
higher than the Commission average (56%). 95% of employees at INEA said they had a clear 
understanding  of  the  Agency’s  purpose  in  2016,  a  9%  increase  compared  with  2014.  Other 
examples  are:  the  managers’  approach  to  the  staff  (staff  has  a  much  better  perception  of 
management,  especially  middle  management,  as  compared  to  the  2014  survey,  while  it  is 
important  to  note  that  a  new  management  structure  was  just  evolving  in  2014)  and  training 
and opportunities for staff to advance their professional development.  

I have the appropriate and timely information to do my work well;  

20 The staff engagement index is composed of the following seven individual questions: 
• 
•  My colleagues are committed to doing quality work; 
• 
• 
• 
•  My manager seems to care about me as a person; 
•  My line manager helps me to identify my training and development needs. 

I have a clear understanding of what is expected from me at work; 
I have recently received recognition or praise for good work; 
I feel that my opinion is valued;  

14 

 
 
 
 
 
 
 
                                                           
At the same time, INEA staff perceived less positively the offer of training and development 
activities and the opportunities for their mobility and career development. 

To follow up these staff survey results, as mentioned under section 3, the Agency adopted in 
April 2017 a comprehensive multi-annual HR strategy followed by a staff retention action 
plan  in  September  2018.  The  evolution  of  key  indicators  for  staff  satisfaction  –  after  the 
evaluated period – is encouraging.  

In the latest  staff survey,  which was carried out in the 4th quarter of 2018 (thus outside of 
the  evaluation  period),  INEA  managed  to  increase  for  the  third  consecutive  time  the  staff 
engagement index - from 61% in 2014 to 68% in 2016 and 70% in 2018. Also, the overall 
job  satisfaction,  even  if  it  is  remains  lower  than  the  Commission  average  (77%),  has  also 
increased  to  75%.  The  results  of  this  new  staff  opinion  survey  also  show  points  for 
attention:  decreasing  staff  satisfaction  with  workload  levels,  work  recognition  and 
performance  management  as  well  as  lower  satisfaction  rates  concerning  the  Agency’s 
middle management. These matters will have to be addressed by the Agency together with 
its parent DGs in the most appropriate way and will feed into a new follow-up action plan. 

Overall, the size and structure of INEA, and the level of resources in each of its departments, 
were  appropriate  to  its  mandate  and  delegated  tasks  during  the  evaluation  period.  The 
organisational structure of INEA was, as of the start of its mandate in 2014, based on separate 
departments for each of the delegated programmes – CEF (Department C) and Horizon 2020 
(Department H) – as well as for Programme support and resources (Department R). Compared 
with the previous TEN-T EA, a new hierarchical level was therefore introduced in the form of 
heads of department function, which led to the need for staff to adapt to new internal reporting 
and approval circuits. Department C has 3 units for CEF Transport, reflecting the substantial 
respective  grant  budget  delegated  to  INEA,  split  by  funding  priority  and  geographical 
coverage, as well as 1 unit each for CEF energy and CEF telecoms (formerly one single unit). 
Department  H  has  2  units  respectively  for  transport  and  energy  research.  Department  R 
comprises  4  units  responsible  for  the  Agency’s  horizontal  tasks  such  as  planning  and 
coordination  of  the  calls  and  communication,  administration  and  finance,  HR  and  IT  and 
logistics,  as  well  as  the  internal  control  function.  INEA’s  organisational  structure  was 
reviewed and effectively aligned to cope with changes, such as the evolution of its mandate, 
notably  with  the  creation  of  a  dedicated  unit  for  CEF  Telecoms  on  the  occasion  of  the 
delegation of the WiFi4EU programme to INEA. 

During 2014-2016, INEA, in cooperation with the Commission, continued the optimisation of 
its  procedures  and  programme  management  functions  and  introduced  a  number  of 
simplifications.  Concerning  Horizon  2020,  the  improvements  and  simplifications  concerned 
wider  use  of  remote  evaluation  of  proposals,  the  introduction  of  a  single  set  of  rules, 
electronic  signature  of  grant  agreements,  the  Participant  Portal  as  the  one-stop-shop  for 
interactions with participants, single reimbursement rate, flat rate for indirect costs, etc. With 
regard  to  CEF,  these  concern  the  streamlining  of  submission  of  progress  reports  by  the 
beneficiaries  (Action  Status  Reports),  e-submission  and  evaluation  modules  in  the  TEN-tec 
portal,  harmonisation  of  Grant  Agreements  etc.  Following  its  successful  testing  end  2015, 
Speedwell  has  become  as  of  1st  January  2016,  the  standard  tool  to  process  administrative 
payment paperless. 

In  July  2014  the  Agency  revised  its  Manual  of  Procedures.  Furthermore,  an  exercise  for 
streamlining and updating the internal procedures and guidelines for the new programmes was 

15 

 
 
 
 
 
 
 
initiated.  For  document  management,  the  use  of  paperless  workflows  was  extended.  A 
specific  Agency  Task  Force  was  created  which  led  to  a  concrete  action  plan  to  increase 
paperless procedures.  

5.3 Coherence  

Coherence  looks  at  any  overlaps  and  complementarities  within  the  programme  portfolio 
managed by the Agency or delimitation of responsibilities between the Agency and its parent 
DGs.  

In the 2014-2016 period, INEA demonstrated flexibility in implementing a diverse thematic 
programming portfolio and being answerable to four different parent DGs. The evaluation did 
not  find  any  evidence  of  overlaps,  duplications,  gaps  or  inconsistencies  within  the  INEA 
programme  portfolio.  The  Memorandum  of  Understanding  as  last  amended  in  July  2018 
provides for a clearer and appropriate delimitation of responsibilities and tasks between INEA 
and its parent DGs. However, due consideration should be given to the fact that supervision 
tasks  (e.g.  financial,  auditing,  planning,  reporting,  coordination  etc)  have  increased 
significantly as of the start of the present MFF. Hence, the necessary supervision capacity in 
the parent DGs (with benchmarking among DGs) needs to be ensured. 

Since  INEA  was  established  around  the  concept  of  network  industries  and  related  research 
and innovation, there is a strong coherence between the CEF and Horizon 2020 programmes 
delegated  to  INEA  notably  in  respect  of  the  energy  and  transport  sectors.  Concerning  the 
transport  sector  in  particular,  the  Commission  benefitted  from  the  continuity  of  support  to 
TEN-T infrastructure actions within the same Agency since 2006, as well as the taking over 
by INEA of the Marco Polo programme from EACI (now EASME).  

The coherence in the programmes managed by INEA also helped to exploit synergies to the 
extent  possible.  The  mid-term  evaluation  of  the  CEF  programme21  noted  that,  on  the  one 
hand,  CEF  synergies  have  been  effective  at  programme  level,  notably  through  common 
implementation  by  INEA  with  the  same  business  processes,  3-sector  coordination,  common 
CEF  coordination  committee  of  Member  States,  and  common  Work  Programmes  for  CEF 
Financial  Instruments,  all  of  which  generate  economies  of  scale  and  simplification.  On  the 
other hand, the legal basis of the CEF programme is not sufficiently flexible to allow funding 
of  large-scale  infrastructure  projects  combining  2  or  3  sectors;  hence  the  limited  success  of 
the  CEF  synergies  (Transport  and  Energy)  call  for  proposals  of  2016.  The  Common 
Understanding on CEF II provides for facilitation of synergies within the programme which 
would benefit from a coherent set-up of the programmes managed by INEA.  

With regard to synergies between Horizon 2020 and CEF, despite some intrinsic difficulties 
linked with the legal bases of both programmes, different business processes and distinct type 
of  support  projects  (research  vs  deployment  and  /  or  infrastructure  building),  INEA  and  the 
parent DGs endeavoured to take initiatives in this field. INEA launched a reflection exercise 
concerning  the  transport  sector  which  led  to  a  report  being  presented  to  the  Steering 
Committee in 2017. Several actions were taken at project level including technical meetings 
with the parent DGs, TRIMIS collaboration, and a synergies webpage in INEA website. As a 
result of this exercise INEA Horizon 2020 projects had a greater presence at the TEN-T days 

21 COM(2018) 66 final of 14.2.2018, Report from the Commission to the European Parliament, the Council, the 
European Economic and Social Committee and the Committee of the Regions on the mid-term evaluation of the 
Connecting Europe Facility (CEF). 

16 

 
 
 
 
 
 
 
                                                           
organised  by  DG  MOVE  (as  of  the  2017  edition).  In  addition,  there  is  a  higher  degree  of 
coordination between Horizon 2020 and CEF Work Programmes. Several proposals applying 
for  CEF  Transport  funding  have  referred  to  results  from  Horizon  2020  projects.  A  similar 
exercise is currently being carried out in the Energy sector.  

The  evaluation  however  stressed  that  INEA  would  gain  from  a  closer  alignment  of 
procedures,  as  appropriate,  between  the  infrastructure  programme  (CEF)  and  the  research 
programme (Horizon Europe) in terms of call launching, submission of proposals, evaluation 
and selection process and grant awarding for the next MFF. Work is being developed in this 
respect in the context of the corporate model grant agreement and e-grants process. 
INEA  has  a  very  good  working  relationship  with  parent  DGs  which  are  in  general  satisfied 
with the levels of formal and informal communication taking place in a regular and structured 
manner. The fact that some of the Agency’s personnel, mainly at management level, is either 
seconded  from  one  of  the  parent  DGs  (in  2018,  19  of  277  staff)  or  has  previously  worked 
therein is an important asset in this respect.  

Despite  the  inherent  difficulties  linked  with  reporting  on  programme  implementation  (grant 
funding)  on  the  variety  of  delegated  programmes  and  sub-programmes,  INEA  has  been 
proactive  and  constructive  in  providing  policy  feedback  to  parent  DGs.  This  is  the  case 
notably  on  lessons-learned  from  project  management  and  project  funding.  However,  the 
evaluation found that, as the projects supported at the start of the programmes are unfolding in 
the  second  half  of  the  present  MFF,  INEA  could  use  its  knowledge  of  programme 
implementation to further help Commission policy-making notably on the preparation of the 
Work Programmes. INEA has also been proactive at proposing a framework for the provision 
of KPIs on CEF implementation, as well as preparing Country and Corridor reports (notably 
on  CEF  Transport)  and  brochures  on  achievements  and  successful  stories.  While 
improvements were found necessary on the provision of real-time information on programme 
and  project  progress,  this  is  being  addressed  and  improved  constantly  through  the  close 
cooperation between the parent DGs and INEA.  

5.4 Retrospective Cost-Benefit Analysis 

The retrospective Cost–Benefit Analysis for the period 2014-2016 was carried out  based on 
the  results  of  the  2013  ex-ante  CBA,  the  assumptions  laid  down  in  the  Specific  Financial 
Statement (SFS) and the actual costs of INEA. 

The 2013 ex-ante CBA estimated that the delegation of tasks to INEA would result in savings 
of  around  EUR  54  million  over  the  2014-2020  Multi-Annual  Financial  Framework  when 
compared to the management by the Commission services (in-house scenario). For the period 
2014-2016 such savings were estimated at around EUR 16.1 million. The savings estimate in 
the INEA SFS for the period 2014-2016 was in the order of EUR 22.8 million22. 

The  retrospective  CBA  revealed  that  the  actual  costs  of  the  executive  agency  approach  in 
2014-2016 were much lower than the costs of the in-house scenario estimated both in the ex-
ante CBA and in the SFS. The actual cost savings constituted EUR 24.4 million (or 29% of 
the estimated costs under the in-house scenario)23. 

22The retrospective CBA is based on the SFS version of November 2017 – see also footnote 9. 
23Including costs of coordination and monitoring by the Commission and costs of INEA covered from EFTA and 
third country contributions.  

17 

 
 
 
 
 
 
 
 
                                                           
Estimated costs and savings of the executive agency scenario (INEA) in 2014-2016, EUR 

As  forecasted  in  the  SFS  and  the  ex-ante  CBA,  savings  of  the  executive  agency  scenario 
primarily resulted from a higher share of lower-cost contract staff (CAs) employed within the 
Agency and the lower overall number of staff. 

Significant  cost  savings  occurred  in  Title  II  “Infrastructure  and  operating  expenditure”  and 
Title III “Programme Support Expenditure” of the administrative budget. Title I “Staff related 
expenditure”  was  higher  than  estimated  in  the  SFS,  which  related  to  higher  average  staff 
costs. It should be noted in this respect that while in the SFS the average staff cost estimations 
remain constant during 2014-2020 period, the actual average staff costs might raise further in 
the run up to the end of the period due to salary indexation, promotions and/or increasing staff 
seniority.  

The workload analysis in the retrospective CBA revealed, that  while the operational  budget 
actually executed by INEA was lower than initially estimated in the ex-ante CBA/SFS (92% 
of  the  estimations),  the  actual  number  of  projects  managed  by  INEA,  which  constitutes  the 
main  workload  driver  for  the  Agency,  in  2015-2016  largely  corresponded  to  the  CBA 
estimates  (number  of  running  projects  at  the  end  of  2016:  94%).  The  number  of  proposals 
evaluated was about 98% of the estimations.  

At the same time, many parameters of the delegated programmes significantly deviated from 
the  initial  CBA/SFS  estimations,  e.g.  higher  than  estimated  operational  budget  of  the  CEF-
Telecoms programme and much lower average project size, reflux of funds for CEF Transport 
following  cancellation  of  projects  and  cost  savings  into  new  calls24,  lower  than  anticipated 
number of projects for studies under CEF Energy, reallocation of CEF/Horizon 2020 funds to 
EFSI25, as well as operational budget changes resulting from the MFF review exercise and the 
annual  budget  review  exercise.  As  a  consequence  of  the  different  changes,  the  number  of 

24 In 2016, on request of the beneficiaries, INEA in coordination  with DG MOVE amended a number of CEF 
Transport grant agreements in order to free the corresponding budgetary commitments (approximately EUR 600 
million), which were then re-injected into the 2016 calls. 
25 Following the establishment of the European Fund for Strategic Investments (EFSI), 8.4% of CEF funds and 
2.9% of Horizon 2020 funds (3.5% of Horizon 2020 funds for Energy and Transport) were transferred to EFSI. 
See Annex I of the EFSI Regulation (EU) No 2015/1017 of the European Parliament and of the Council.   

18 

 
 
 
 
 
 
 
                                                           
authorised staff in INEA was reduced in 2016 (from 262 estimated in the SFS to 247)26. The 
actual execution of payment appropriations was slower than anticipated in the SFS and CBA, 
which  indicates  that  increase  in  the  projects’  management  workload  will  unfold  later  than 
estimated.  

6.  CONCLUSIONS 

The results of the evaluation of INEA operations in 2014-2016 confirmed that INEA achieved 
effective  management  of  the  delegated  programmes  while  operating  according  to  the  legal 
framework governing it. For each part of the CEF and Horizon 2020 programmes delegated to 
INEA, as well as the legacy programmes, the Agency has proven to be responsive and flexible 
to  all  challenges  and  evolving  policy  context  and  has  timely  and  efficiently  accommodated 
these.  Its  objectives  in  terms  of key performance indicators  (Time-to-grant,  Time-to-inform, 
Time-to-pay, residual error rate etc.) were almost continuously attained if not out-performed.  

Moreover, applicants, beneficiaries and experts showed a large satisfaction with the operation 
of  INEA.  In  particular,  the  Agency  achieved  its  objectives  in  terms  of  proximity  to  the 
addressees  and  of  visibility  of  EU  programmes.  During  the  2014-2016  period,  INEA 
improved its processes by adopting various simplification measures and IT tools. 

The retrospective CBA confirmed that the delegation of programme implementation to INEA 
remained  considerably  more  cost  effective  than  the  in-house  scenario.  During  this  period 
INEA  continued  to  implement  the  delegated  programmes  in  an  effective,  efficient  and 
coherent  manner,  generating  important  cost-savings  to  the  EU  budget  and  high  value  for 
money. The actual cost savings constituted EUR 24.4 million, or 29% of the estimated costs 
under the in-house scenario. 

The  staff  survey  of  2016  revealed  a  very  reasonable  level  of  staff  engagement  and  staff 
satisfaction for INEA. The Agency has actively addressed identified shortcomings in its 2017-
2020 multi-annual HR strategy and dedicated action plan, including through its staff retention 
policy.  In  particular  the  turn-over  levels  and  vacancy  rate  have  been  significantly  reduced. 
The  new  staff  survey  results  of  2018  show  that  staff  perception  of  key  areas  of  work  has 
considerably improved. 

The Memorandum of understanding between INEA and its parent DGs set a clear definition 
of tasks between the Agency and the Commission, which was observed  over the evaluation 
period.  

The  coordination  and  communication  mechanisms  that  are  in  place  facilitate  information 
flows between INEA and the Commission services, in particular with regard to the content of 
projects  supported  and  their  results.  No  evidence  of  overlaps,  duplication,  gaps  and 
inconsistencies within the programme portfolio managed by INEA were identified. 

The current evaluation identifies areas where further improvements are necessary from both 
the  Commission  and  INEA  and  which  aim  to  further  streamline  or  define  the  working 
arrangements between both sides.  

26 4 CAs financed from EEA/EFTA are not included, total number of authorized INEA’s staff in 2016 was 251.  

19 

 
 
 
 
 
 
 
 
 
 
                                                           
These relate to:  

  Coordination of the calendar for the calls for proposals;  
 

Improve synergies between the delegated programmes and between sectors within the 
same programme; 

  Further improve policy feedback from INEA through appropriate mechanisms; 
  Align  to  the  extent  possible  programme  implementation  business  processes  between 
the  infrastructure  (CEF)  and  the  research  and  innovation  (Horizon)  programmes  for 
the new MFF, notably in terms of model grant agreement and e-grants. 

The  parent  DGs  and  the  Agency  are  preparing  one  action  plan  that  will  address  the 
shortcomings  identified  in  this  evaluation.  This  action  plan  will  also  address  the 
recommendations outlined in the performance audit report of the European Court of Auditors 
(Special  Report  No  19/2019  “INEA:  benefits  delivered  but  CEF  shortcomings  to  be 
addressed”)  (mentioned  in  section  4.2  above).  This  action  plan  aims  to  further  increase  the 
Agency’s  performance  and  contributes  to  an  efficient  implementation  of  the  delegated 
programmes. 

20 

 
 
 
 
 
 
7.  ANNEX: EVALUATION QUESTIONS 

Effectiveness: to what extent have the objectives of delegating programme management 
tasks to INEA been achieved? 

  To  what  extent  has  INEA  been  operating  according  to  the  legal  framework 

establishing it? 

  To  what  extent  has  INEA  achieved  its  objectives  with  special  focus  to  (a)  the 
implementation  of  the  delegated  programmes  CEF,  Horizon  2020  and  the  legacy 
programmes  (TEN-T  and  Marco  Polo  II),  (b)  the  implementation  of  the  internal 
control principles27, notably sound financial and human resource management? What, 
if  anything,  could  be  done  to  render  INEA  more  effective  in  achieving  these 
objectives? 

  To  what  extent  has  INEA  contributed  to  an  improved  management  of  the  delegated 
programmes  and  better  services  to  the  stakeholders  and  addressees  in  terms  of  the 
elements  assessed  in  the  2013  CBA  and  as  compared  to  the  alternative  options 
mentioned in that CBA? 

  To what extent has INEA contributed to improved management of the programmes in 

terms of: 
−  Proximity  to  addressees  (e.g.  role  played  by  the  Agency  as  focal  point  for 
applicants and beneficiaries of the programmes; instruments and mechanisms put 
in place for communicating with the applicants and beneficiaries). 

−  Effective implementation of the programmes, taking into account  the interests of 

the addressees and those of the EU:  
  rate of execution of commitment appropriations  
  rate of execution of payment appropriations 
  time to grant 
  net time to pay 
  residual multi-annual error rate identified at ex-post control. 

−  Visibility of the EU as promoter of the programmes entrusted to the Agency (e.g. 
compliance  with  the  Commission’s  guidelines  on  information  and  visibility  of 
programmes28,  instruments  put  in  place  to  ensure  the  visibility  of  the  EU  as 
promoter of the programmes). 

Efficiency  -  to  what  extent  has  INEA  carried  out  its  tasks  efficiently  in  line  with 
estimations? 

  To what extent have the actual costs (including cost of coordination  and monitoring) 
of INEA corresponded to the estimates of the 2013 CBA? If not, what are the reasons 
behind? 

  To  what  extent  have  the  actual  benefits  corresponded  to  the  estimates  of  the  2013 

CBA? If not, what are the reasons behind? 

  To what extent has the management and execution of the programmes by INEA been 

cost-effective as compared to the alternative options29? 

27 Replacing, as of 01.01.2018, the internal Control Standards – see C(2017)2373 final. 
28 https://ec.europa.eu/info/resources-partners/european-commission-visual-identity_en  
29 Examples of alternatives to the Executive Agency: (i) management of the programme(s) by the Commission, 
(ii)  mixed Agency  - Commission management, (iii) partial management by the Commission while outsourcing 
some activities to the extent legally possible.  

21 

 
 
 
 
 
                                                           
 
  To what extent has INEA contributed to improved management of the programmes in 
terms  of  simplification  of  the  procedures  and  flexibility  in  the  implementation  of 
delegated tasks (e.g. capacity to adapt to periods of high workload)? 

  Which further scope for simplification exists? 
  Which  aspects/means/actors  or  processes  render  INEA  more  or  less  efficient?  What 

could be improved? 

  To what extent has the establishment of INEA resulted in savings to the EU budget as 
compared to the alternative options (e.g. difference in costs between the Commission 
option and the Agency option)? 

  To what extent have INEA’s internal organisation and procedures been conducive to 

its efficiency?  

  To what extent is INEA’s internal organisation capable and flexible to rapidly respond 

to resource needs due to uncertainties related to volumes of work? 

  To  what  extent  does  INEA’s  human  resources  management  contribute  to  the 

achievement of the Agency’s objectives?  

To this end, and as a minimum, the contractor shall analyse the following: 
 
 

Is the size and structure of the organisation and its separate entities appropriate? 
Is  the  balance  between  operational  staff  and  administrative  support  staff 
appropriate? 
Is the staff turnover and vacancy rate well managed? 

 
  How  does  the  Agency  follow  up  on  the  findings  of  the  two  latest  staff  surveys 

(2014; 2016)? 

The Commission will provide the relevant statistical data to the successful tenderer after the 
signature of the contract. 

The  management  performance  indicators  have  to  be  compared,  where  applicable,  with  the 
indicators used in the previous evaluation study (2011-2013). 

Coherence - to what extent there is consistency between the tasks managed by INEA and 
those managed by the parent DGs and other executive agencies, and has the Commission 
been  able  to  maintain  an  adequate  level  of  know-how  in  relation  to  the  programmes 
entrusted to the Agency? 

The establishment of INEA has enabled the Commission (within the parent-DG's and across 
Commission services via the central redeployment pool) to allocate staff to institutional tasks 
as  compared  to  the  alternative  options.  As  a  minimum,  the  following  questions  shall  be 
addressed: 

  To what extent have there been overlaps/ gaps/ inconsistencies within the programme 

 

portfolio managed by INEA? 
Is  there  a  clear  and  appropriate  delimitation  of  responsibilities  and  tasks  between 
INEA and the Parent DGs? Are there overlaps or gaps? 

  To what extent has INEA enabled the Commission to better focus on its policy related 

tasks? 
−  Are  there  any  governance  (financial  and  policy)  issues  in  relation  to  the 
implementation of the tasks that are delegated to the executive agencies/ issues as 
regards  the  Commission’s  services  being  able  to  steer  EU  policy  or  budget 
implementation?  

22 

 
 
 
 
 
 
 
−  Are  appropriate  mechanisms  and  instruments  in  place  to  ensure  an  adequate 
coordination and information flow between INEA and the Commission services? 
−  Does  INEA  provide  useful  information  in  support  of  the  policy  process  (e.g. 

information required for the annual Management Plan of the Parent DGs)? 
  To  what  extent  have  the  activities  of  INEA  resulted  in  unintended  effects  (both 

desirable and undesirable)? 

  To what extent has the Commission, in the presence of INEA, been able to maintain 
an  adequate  level  of  know-how  in  relation  to  the  programmes  entrusted  to  the 
Agency?  How  has  this  been  achieved?  What  are  the  feedback  channels,  means  and 
methods used for this purpose? What are areas for improvement, if any? 

  Have  the  monitoring,  reporting  and  supervision  arrangements  in  place  enabled  the 
Commission  to  benefit,  in  the  short  and  medium  term,  from  the  know-how  created 
within INEA? 

  How effective is the flow of information and communication between INEA and  the 

Commission services (in particular Parent DGs)? 

  To what extent would the closing down of INEA result in losing significant know-how 

in relation to the management of the programmes entrusted to INEA? 

23