COMMISSION STAFF WORKING DOCUMENT Evaluation of the the Executive Agency for Small and Medium-Size Enterprises (EASME) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency
Date: 2020-04-29
Institutions: European Commission, Secretariat-General
Subjects: EU aid, EU financing, EU programme, executive agency, financial control, performance audit, small and medium-sized enterprises

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 29.4.2020  SWD(2020) 76 final  COMMISSION STAFF WORKING DOCUMENT Evaluation of the the Executive Agency for Small and Medium-Size Enterprises (EASME) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency {COM(2020) 184 final} - {SWD(2020) 73 final} - {SWD(2020) 74 final} - {SWD(2020) 75 final} - {SWD(2020) 77 final} - {SWD(2020) 78 final}  
 
 
TABLE OF CONTENTS 

1. 

INTRODUCTION .................................................................................................................... 2 

2.  BACKGROUND ...................................................................................................................... 3 

2.1.  Description of the intervention and its objectives ....................................................... 3 

2.2.  Baseline and points of comparison .............................................................................. 5 

3. 

IMPLEMENTATION/STATE OF PLAY ....................................................................................... 5 

4.  METHOD .............................................................................................................................. 6 

4.1.  Short description of methodology ............................................................................... 6 

4.2.  Limitations and robustness of findings ........................................................................ 8 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ................................................. 8 

5.1.  Effectiveness ................................................................................................................ 8 

5.2.  Efficiency ................................................................................................................... 12 

5.3.  Coherence .................................................................................................................. 18 

5.4.  Relevance ................................................................................................................... 20 

5.5.  Retrospective Cost-benefit analysis ........................................................................... 20 

6.  CONCLUSIONS .................................................................................................................... 21 

ANNEX - EVALUATION QUESTIONS ......................................................................................... 24 

1 

 
 
 
 
 
1. 

INTRODUCTION 

The purpose of this evaluation is the periodical (triennial) evaluation of the operation of the 
Executive  Agency  for  Small  and  Medium-Size  Enterprises  (EASME).  EASME  is  governed 
by  its  Act  of  Establishment1,  which  sets  out  the  mandate  of  the  Agency  and  the  Act  of 
Delegation2, which specifies the tasks to  be carried out  by, and the powers delegated to  the 
Agency to perform its mandate. These tasks relate to the implementation of dedicated parts of 
certain  EU  financial  programmes.  Furthermore,  the  Agency  is  governed  by  the  Framework 
Regulation  for  executive  agencies3  and  the  decision  establishing  the  EASME  Steering 
Committee.  

In line with the Commission’s Better Regulation principles4, the evaluation applies different 
evaluation criteria to guide the analysis of the performance of the Agency. It assesses whether 
the Agency has fulfilled its tasks in an efficient and effective way, whether there are overlaps/ 
gaps/  inconsistencies  or  synergies  within  the  management  of  the  programme  portfolio 
executed by the Agency; and whether there is  a  clear delineation of tasks between  EASME 
and the parent DGs or other executive agencies (coherence) and whether objectives and tasks 
responded to past needs and still respond to current needs (relevance).  

The  evaluation  assesses  whether  the  functioning  of  the  Agency  has  yielded  the  expected 
positive results as estimated in the cost-benefit analysis for delegating task to the Agency5 and 
identifies  potential  areas  of  improvement.  To  this  end  the  estimations  of  the  cost-benefit 
analysis of 2013 have been tested to provide evidence on the validity of the assumptions made 
in  the  ex  ante  scenario  by  considering  the  actual  costs  and  benefits  of  programme 
implementation  by the  Agency in  a structured  way. The aspects  covered by the cost-benefit 
analysis  are  specified  in  Article  3(1)  of  the  Framework  Regulation6  and  the  Guidelines  on 
establishing and operating executive agencies7.  

The  evaluation  does  not  cover  the  achievements  of  the  programmes  managed  by  EASME, 
which are subject to separate mid-term and ex post evaluations. The evaluation of EASME is, 
nevertheless, providing useful input for these programme evaluations, as the performance of 
the Agency affects the efficiency and effectiveness of the programmes it manages.  
The evaluation  examines the efficient  use of resources and the  effective achievement of the 
tasks entrusted to the Agency. In particular it looks at whether the alignment of more coherent 

1 2013/771/EU: Commission Implementing Decision of 17 December 2013 establishing the ‘Executive Agency 
for Small and Medium-sized Enterprises’ and repealing Decisions 2004/20/EC and 2007/372/EC. 
2 There is an act of delegation for each of the programmes delegated to EASME. 
3 Commission Regulation (EC) No 1653/2004 of 21 September 2004 on a standard financial regulation for the 
executive  agencies  pursuant  to  Council  Regulation  (EC)  No  58/2003  laying  down  the  statute  for  executive 
agencies to be entrusted with certain tasks in the management of Community programmes and modification by 
the Commission Regulation (EC) No 651/2008 of 9 July 2008. 
4  Commission’s  Better  Regulation  Communication  COM  (2015) 215  final  and  its  accompanying  SWD  (2015) 
111 final and SWD (2015) 110 final. 
5  Cost-benefit  analysis  for  the  delegation  of  certain  tasks  regarding  the  implementation  of  Union  Programmes 
2014-2020 to the executive agencies - Final report for the Commission of 19 August 2013. 
6  The  cost-benefit  analysis  shall  take  into  account  a  number  of  factors  such  as  identification  of  the  tasks 
justifying outsourcing, a cost-benefit analysis which includes the costs of coordination and checks, the impact on 
human resources, possible savings  within the general budgetary framework of the European Union, efficiency 
and  flexibility  in  the    implementation  of  outsourced  tasks,  simplification  of  the  procedures  used,  proximity  of 
outsourced activities to final beneficiaries, visibility of the EU as promoter of the EU programme concerned and 
the need to maintain an adequate level of know-how inside the Commission.  
7  Appendix  II  of  the  Guidelines  for  the  establishment  and  operation  of  executive  agencies  financed  from  the 
Union budget (C (2014) 9109 from 2 December 2014; pp. 64-72). 

2 

 
 
 
 
 
                                                           
programme  portfolios  with  the  Agency’s  core  competences  and  its  brand  identity  delivered 
the estimated qualitative benefits; whether the assembly of the management  of different  EU 
programmes  delivered  the  estimated  synergies,  simplification  and  economies  of  scale  and 
whether  there  is  scope  for  simplification  and  further  efficiency  gains.  Also  the  continued 
relevance  of  all  objectives  and  tasks  entrusted  to  the  Agency  were  assessed  as  well  as  the 
coherence  of  its  programme  portfolio  and  the  delineation  of  tasks  between  the  Agency  and 
other  executive  agencies.  Furthermore,  the  evaluation  examined  whether  the  outsourcing 
allowed the parent DGs to focus on core tasks. 

The evaluation covers all the tasks carried out by the Agency during the years 2014-2016. The 
evaluation  is  supported  by  a  study  carried  out  by  an  external  contractor.  The  results  of  this 
study  are  summarised  hereafter  and  this  evaluation  will  be  presented  to  the  European 
Parliament, to the Council and to the Court of Auditors in accordance with Article 25(1) of 
the Framework Regulation.  
The  results  of  the  evaluation  will  feed  into  the  reflection  to  assess  the  opportunity  of 
expanding/modifying the Agency’s mandate in view of the delegation of the implementation 
of 2021-2027 EU programmes. 

2.  BACKGROUND  

2.1.  Description of the intervention and its objectives 

According  to  the  Framework  Regulation  and  EASME’s  Act  of  Establishment8,  the 
outsourcing of certain management tasks to the Agency intends:  

  To allow the Commission to focus on its institutional tasks; i.e. tasks assigned to the 
institutions  by  the  Treaty  which  require  discretionary  powers  in  translating  political 
choices into action. Such institutional tasks should not be outsourced.  

  To enable achieving the goals of EU programmes more effectively.  

According to a Cost-Benefit Analysis carried out in 2013 delegating tasks to the Agency was 
estimated  to  be  more  cost-efficient  than  an  in-house  scenario.  The  Act  of  Establishment 
estimated that the alignment of more coherent programme portfolios with the Agency’s core 
competences and its brand identity would bring qualitative benefits.  In addition, it projected 
that  assembling  the  management  of  different  EU  programmes  would  bring  synergies, 
simplification and economies of scale. Furthermore, the pooling of all aspects of the Horizon 
2020 ‘SME instrument’ would provide a single point of access to potential beneficiaries and 
guarantee consistent service delivery.  

The  Agency  implements  delegated  tasks  in  close  cooperation  with  its  seven  parent 
Directorates General: (1) DG for Internal Market, Industry, Entrepreneurship and SMEs (DG 
GROW),  (2)  DG  for  Research  and  Innovation  (DG  RTD),  (3)  DG  for  Communications 
Networks, Content and Technology (DG CNECT), (4) DG for Climate Action (DG CLIMA), 
(5) DG for Energy (DG ENER), (6) DG for Environment (DG ENV) and (7) DG for Maritime 
Affaires and Fisheries (DG MARE).  

The  Agency  is  entrusted  to  implement  SME-related  parts  of  several  EU  programmes  under 
the 
the  2014-2020  multiannual 

the  Programme 

framework 

financial 

(MFF): 

for 

8 2013/771/EU: Commission Implementing Decision of 17 December 2013 establishing the ‘Executive Agency 
for Small and Medium-sized Enterprises’ and repealing Decisions 2004/20/EC and 2007/372/EC. 

3 

 
 
 
 
 
 
 
                                                           
Competitiveness  of  Enterprises  and  small  and  medium-sized  Enterprises  (COSME),  the 
Framework Programme for Research and Innovation (Horizon 2020), the Programme for the 
Environment and Climate Action (LIFE), the European Maritime and Fisheries Fund (EMFF), 
the  legacy  of  certain  actions  under  the  Competitiveness  and  Innovation  Programme    2007-
2013 (CIP). 

The  Agency  implements  the  COSME  programme  excluding  the  financial  instruments  and 
some other actions which are managed by DG GROW. Under the Horizon 2020 programme 
the following parts are delegated: innovation in SMEs of “Part II Industrial Leadership”, SME 
Instrument,  fast  track  to  innovation  pilot  scheme,  societal  challenge  ‘Secure  and  efficient 
energy’,  societal  challenge  ‘Climate  action,  environment,  resource  efficiency  and  raw 
materials’. Under the LIFE programme the Agency manages the calls for action grants under 
the Environment sub-programme9 and the Climate Action sub-programme, as well as the call 
for  operating  grants  to  NGOs  that  help  to  implement  and  shape  EU  policies  on  the 
environment  and  climate  change.  Concerning  the  European  Maritime  and  Fisheries  Fund,  a 
large  part  of  the  direct  management  of  policy  support  actions  in  the  area  of  Integrated 
Maritime  Policy,  Control  and  Scientific  Advice  and  Knowledge  is  delegated.  The  Agency 
works closely with other partners such as the Common Support Centre for Horizon 2020 (now 
named  the  Common  Implementation  Centre),  the  Research  Executive  Agency  (REA)  for 
some logistic and administrative support services and an external contractor for certain tasks 
under the LIFE programme. 

The main tasks of the Agency are to: 

  Manage some or  all stages of programme implementation and some or  all phases of 

specific projects;  

  Adopt  the  instruments  of  budget  execution  and  carry  out  all  the  relevant  operations; 

and 

  Provide support in programme implementation. 

Specific tasks delegated to the Agency: 

9 Except for integrated projects and technical assistance projects, which have been managed by DG ENV until 
2017. 

4 

 
 
 
 
 
 
                                                           
2.2.  Baseline and points of comparison  

The current evaluation of EASME operations during 2014-2016 assesses the actual costs and 
benefits  of  programme  implementation  by  EASME  (executive  agency  scenario)  when 
compared with the alternative scenario of management by the Commission services (in-house 
scenario). 

Accordingly, the reference point for the present EASME evaluation is the 2013 ex ante Cost-
benefit analysis, and the specific financial statement10 (SFS) of EASME. 

Whenever possible, the analysis of the performance of EASME during the period of reference 
assesses the progress compared to the previous evaluations, covering the Agency’s operations 
during 2006-2013.  

According  to  the  SFS  of  EASME  prepared  in  2013  estimated  savings  are  EUR  9.9  million 
over  the  period  2014-201611.  The  total  number  of  full  time  equivalents  (FTEs)  required  to 
manage  the  relevant  programmes  in  year  2020,  the  peak  programming  year  in  terms  of 
workload,  was  estimated  at  502  FTEs  for  EASME.  The  total  initial  operational  budget 
entrusted  to  EASME  in  2020  was  estimated  at  around  EUR  11.100  million  in  commitment 
appropriations  and  EUR  11.316  million  in  payment  appropriations.  Considerable  efficiency 
gains  were  expected  compared  to  the  in-house  scenario  over  the  period,  along  with  non-
quantifiable  benefits  such  as  improved  quality  of  programme  management  and  service 
delivery, improved visibility of the EU programmes and proximity to beneficiaries. 

3. 

IMPLEMENTATION/STATE OF PLAY 

Initially  founded  as  the  Intelligent  Energy  Executive  Agency  (IEEA)12  to  implement 
renewable  energies  and  energy  efficiency  programmes,  the  Agency  was  transformed  in 
December  200713  into  the  Executive  Agency  for  Competitiveness  and  Innovation  (EACI) 
extending  its  mandate  to  the  Competitiveness  and  Innovation  Framework  Programme  (CIP) 
and  the  second  Marco  Polo  programme.  EASME  was  created  in  201314  by  expanding 
competences previously entrusted to EACI with the delegation of operational tasks regarding 
the  implementation  of  parts  of  the  Framework  Programme  for  Research  and  Innovation 
(Horizon  2020),  the  Programme  for  the  Competitiveness  of  Enterprises  and  small  medium 
size enterprises (COSME), the Programme for Environment and Climate Action (LIFE) and 
the European Maritime and Fisheries Fund (EMFF). The Agency is based in Brussels. 

The new mandate also brought changes in the governance of the Agency as new parent DGs 
joined  the  Steering  Committee,  and  subsequently  signed  a  Memorandum  of  Understanding 

10 SFS C(2014)6944 and C(2014)4636. 
11 The estimated efficiency gains over the period 2014-2024, covering full completion of the projects according 
to the SFS of 2013, were of EUR 158 million. 
12 Commission Decision (EC) 2004/20 setting up the Intelligent Energy Executive Agency (IEEA). 
13 Commission Decision 2007/372/EC of 31 May 2007 amending Decision 2004/20/EC in order to transform the 
Intelligent Energy Executive Agency into the Executive Agency for Competitiveness and Innovation. 
14 Commission implementing decision of 17 December 2013 establishing the ‘Executive Agency for Small and 
medium-sized Enterprises’ and repealing Decisions 2004/20/EC and 2007/372/EC (2013/771/EU). 

5 

 
 
 
 
 
 
 
                                                           
(MoU)  with  EASME15.  The  new  MoU  defined  the  modalities  and  procedures  of  interaction 
between  EASME  and  its  parent  DGs.  In  addition,  it  brought  a  clearer  delimitation  of  the 
administrative  activities  between  the  Agency  and  its  parent  DGs.  This  evaluation  assesses 
whether the distribution of tasks has been respected for the 2014-2016 period, and the effects 
of the intervention on the coherence and the efficiency of EASME’s action.  

EACI counted 161 staff in  December 2013.  In 2014, to  cope with  the increase of delegated 
activities, 140 candidates were recruited (18 left the Agency),  corresponding to  a 76% staff 
growth.  The  staff  progressively  grew16  to  283  in  2014,  373  in  2015,  and  417  in  2016.  The 
operational  budget  for  commitments  delegated  to  EASME  increased  from  EUR  1056.5 
million in 2014 to EUR 1208.5 million in 2015 and to EUR 1321.65 million in 2016. 

The  Agency’s  administrative  budget  increased  as  well  from  EUR  20.26  million  in  2014,  to 
EUR  30.17  million  in  2015  and  to  EUR  31.71  million  in  2016.  The  Agency  faced  teething 
problems  in  2014  with  the  massive  recruitment  and  the  substantial  increase  in  delegated 
activities.  Some delays occurred in implementation and activities were carried over to 2015.  
In 2015, not only did the Agency catch up with the backlog but it reached cruising speed.  The 
budget executed by the Agency grew from EUR 123.1 million to EUR 618.23 million in 2015 
and to EUR 1013.41 million in 2016. 

A  series  of  organisational  and  procedural  changes  started  with  the  implementation  of  the 
programmes under the multiannual financial framework (MFF) of 2014-2020: 

  Delegation of the management of additional programmes, new funding instruments and 

as a consequence new tasks and activities for the Agency. 

  Further  simplification  and 

improvement  of  grant  management  processes  and 
procedures, notably through the electronic submission of proposals and the adoption of 
the Commissions IT tools and systems (i.e. Compass-SyGMa).  

  The adoption of new business processes, Horizon 202017 IT tools and modes of service 

delivery. 

  The adoption of the multiannual human resource (HR) strategy for 2017-2020 created 
the conditions that will foster staff engagement and satisfaction at work. Based on this 
new  strategy  and  in  combination  with  a  yearly  action  plan,  the  Agency  implemented 
specific  measures  related  to  creating  an  attractive  workplace,  building  modern 
leadership and developing professional growth. 

4.  METHOD 

4.1. 

Short description of methodology 

The  evaluation  was  supported  by  a  study  carried  out  by  an  external  contractor18.  The 
methodology of EASME evaluation was consistent with the approaches employed for similar 
parallel evaluations of ERCEA, EACEA, REA, INEA and CHAFEA. 

15 Memorandum of Understanding between the Executive Agency for Small and medium-sized enterprises and 
DG Enterprise and Industry, DG Research and Innovation, DG Energy, DG Environment, DG Climate Action, 
DG Communication Networks, Content and Technology & DG Maritime Affairs and Fisheries of 5 June 2014. 
16 See table 16, Study supporting the evaluation of the operation of EASME, Final Report. 
17 Horizon 2020 is the European Union’s Framework Programme for Research and Innovation. 
18  Evaluation  of  the  Executive  Agency  for  Small  and  Medium-sized  Enterprises  (EASME),  Valdani  Vicari  & 
Associati, Deloitte, April – 2019. 

6 

 
 
 
 
 
 
                                                           
The study was structured around a series of evaluation questions available in Annex. 

The  supporting  study  was  performed  by  using  the  following  data  collection  methods  and 
specific methodological approaches were used to ensure data triangulation (confirmation from 
different sources): 

  Document review and desk research;  
  Semi-structured interviews;  
  Online survey and statistical analysis of survey data;  
  Retrospective cost-benefit analysis; and  
  Open public consultation.  

The document review and desk research covered EU-level reports and documents, previous 
evaluations and audit reports of EASME and its programmes, other relevant  action plans or 
reports.  Through  the  document  review,  the  study  team  gathered  contextual  information, 
indicators for assessment and data for comparison of yearly developments and other executive 
agencies. The evaluation also relied on the results of the 2016 staff opinion survey. 

The consultation strategy relies principally on targeted consultations and interviews to collect 
data on the implementation of the programmes and on the perception of EASME by the main 
stakeholders  of  the  Agency.  The  public  consultation  was  organised  as  an  exercise  of 
transparency and to give the opportunity to a wider public to express opinions. 

124 interviews were conducted with Commission staff (DG GROW, DG ENER, DG ENV, 
DG RTD, DG HR, DG MARE, DG CLIMA, and DG CNECT), EASME staff, Member States 
National  Contact  Points,  experts,  beneficiaries,  tenderers  and  other  stakeholders  (e.g. 
companies).  The  interviews  were  semi-structured  and  adapted  to  the  relevant  stakeholder 
group. Most of the interviews were conducted face to face.  

The online targeted survey ran for four weeks (15 November 2018 to 15 December 2018). 
1,864 stakeholders involved in the operation of EASME at different levels participated in the 
targeted survey.  

Stakeholder participation in the targeted survey 

Stakeholder replies 

Complete 

Partial 

Total 

Beneficiaries 

300 

Member state officials 

36 

Tenderers 

Experts 

Total 

Source: targeted survey 

16 

868 

1,220 

628 

98 

83 

1,055 

1,864 

328 

62 

67 

187 

644 

7 

 
 
 
 
 
 
 
 
A  cost-benefit  analysis  (CBA)  was  performed  providing  a  comparative  analysis  of  various 
actions and possible alternatives to the existing organisational structure of EASME in terms of 
its costs and benefits.  

The  online  Public  Consultation  (OPC)  was  published  on  the  website  of  the  European 
Commission19 for 12 weeks between 14 December 2018 and 8 March 2019. 69 respondents 
expressed  their  opinion.  More  than  26%  were  companies/business  associations  and  around 
20% EU citizens. Respondents are mostly from UK (around 19%) followed by Spain (around 
16%),  Netherlands  (7%)  and  Germany  (7%).  The  majority  of  respondents  were  projects 
coordinators (around 48%) or project applicants (19%) and most of them (67%) participated 
to Commission actions managed by EASME.  

4.2.  Limitations and robustness of findings 

Most  relevant  stakeholder  groups  were  consulted  (beneficiaries,  intermediaries,  external 
experts).  However,  compared  to  the  overall  population  reached,  unsuccessful  applicants 
participated less actively in the survey than programme beneficiaries.  

No  sampling  bias  was  observed  as  the  profile  of  the  respondents  to  the  surveys  was  very 
similar to the overall population, guaranteeing statistical representativeness. The non-response 
bias  (not  all  characteristics  of  the  group  that  did  not  reply  had  been  captured  in  full)  was 
mitigated  through  triangulation  with  the  results  of  follow  up  interviews.  The  triangulation 
approach,  using  multi-level  and  multi-stakeholder  dimension  in  the  data  collection,  ensured 
the robustness and reliability of the data and information used to draw up conclusions in the 
supporting study.  

Some views from Commission and Agency staff gathered from the interviews and mentioned 
in the study illustrate personal experience, although they may not be representative of overall 
relations between the Commission and the Agency. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

5.1.  Effectiveness 

For the purpose of this evaluation, effectiveness relates to how successful the Agency has 
been in achieving or progressing towards its objectives. 

Legal Framework  
During  2014-2016,  EASME  operated  according  to  the  legal  framework  that  established  it. 
EASME’s  activities,  as  planned  in  the  annual  work  programmes  and  reported  in  the  annual 
activity reports, corresponded to the tasks set out in the Commission Decisions and the Act of 
Delegation. 

The modalities and procedures of interaction between the Agency and its parent DGs are set 
out in the Memorandum of Understanding (MoU) between EASME and its parent DGs. The 
modalities and procedures of interactions prescribed by the MoU with the parent DGs clearly 
define the spheres of responsibilities between the various entities.  

19 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5558220/public-consultation_en 

8 

 
 
 
 
 
 
 
 
 
 
                                                           
How did the Agency perform in 2014-2016? 
The key performance indicators related to the operational efficiency of the project’s lifecycle 
were positive in the period 2014 - 2016 and the beneficiaries’ satisfaction ranked high (as per 
results of the online survey). Nevertheless, there is scope for improvement. 

During  the  time  period  of  the  evaluation,  the  Agency  fully  implemented  the  operational 
budget.  The  implementation  of  the  administrative  budget  was  committed  for  99%  and  the 
payment  implementation  increased  from  76  to  87%  over  the  three  years.  The  payment 
implementation was affected in 2014 by the slower than forecast recruitment rate (thus lower 
salaries and allowances), and also the cancellation of the forecasted office move.  Corrective 
measures  were  taken  leading  to  an  overall  improvement  in  administrative  payment 
implementation by the end of 2016. 

Budget execution, 2014-2016 

2014 

2015 

2016 

Commitments 

Payments 

Commitments 

Payments 

Commitments 

Payments 

100% 

100% 

100% 

100% 

100% 

100% 

91% 

76% 

99% 

84% 

99% 

87% 

% budget 
execution 

Operational 
budget 

Administrative 
budget 

Source: EASME 

In the period 2014 - 2016 the multiannual residual error rate was below the target of 2% for 
the CIP EEN legacy programme. On the other hand, the multiannual residual error rate for the 
IEE II (as of 2015) and CIP Eco Innovation (as of 2016) programmes were above 2% and the 
Agency had to issue reservations in its Annual Activity Report20. In 2017, for IEE II it was in 
a range of 2.7 to 2.8% and for the CIP Eco Innovation in a range of 2.5 to 7.5%. In 2017, the 
first year available, the residual error rate for Horizon 2020 was 2.2% with a target as close as 
possible to 2% (within the range of 2 to 5%) (as per Horizon 2020 audit strategy). Given the 
fact  that  the  majority  of  projects  of  the  newly  delegated  programmes  to  EASME  had  not 
reached the appropriate level of maturity (interim/ final payment), the ex-post control audits 
focused exclusively on the legacy programmes of the Agency.  

However,  in  2018,  EASME  had  also  to  issue  a  temporary  reservation  for  COSME  as  the 
residual  error  rate  was  temporarily  above  2%.  This  error  rate  was  a  preliminary  indication 
calculated on a small and non-representative sample of COSME grants (representing 1.12% of 
the COSME payments). The sample did not include EEN grants that represent more than half 
of the COSME grants and have an error rate below 2% in the 2007-2013 period. The Agency 
has put in place corrective measures that improved the situation and as of 2019 the reservation 
was lifted. 

There  are two main factors influencing the  error rates  and the results of  the ex-ante/ex-post 
controls: 

1.  Preventive and educative measures; and 
2.  Beneficiaries’ willingness to comply with the grant agreement rules. 

20  Due  to  the  small  proportion  of  the  financial  impact  on  the  overall  financial  portfolio  of  the  Agency,  these 
reservations were lifted by the Agency as of its 2019 Annual Activity Report. 

9 

 
 
 
 
 
 
 
 
                                                           
By analysing the open questions in the online survey, some project beneficiaries commented 
that  the  criteria  for  the  eligibility  of  the  costs  have  changed  and  become  more  flexible.  In 
some cases, this has proved to be problematic, as “less clear” rules made it difficult to assess 
the right eligible costs. 

The time to grant targets were mostly met during the evaluation period. The time to grant for 
Horizon 2020 Challenges 3 and 5, COSME and EMFF was clearly  within the target  of 274 
days in both 2015 and 2016. In 2016 the average time to grant for all programmes managed 
by the Agency (except the Fast Track Innovation - FTI) was within target. The average time-
to-inform results demonstrate a faster response to the applicants than the originally scheduled 
target21. 

Time to grant, 2014-2016 

Source: EASME Annual Activity Reports 2014, 2015, 2016 

The  Agency  received  a  very  low  number  of  evaluation  review  requests  ranging  from  0  to 
2.4%  of  the  number  of  proposals  evaluated,  all  below  the  target  of  3%,  which  is  a  good 
indication of the robustness of the grant award process and assurance on the effectiveness of 
the internal control system22. 

The survey also shows that EASME’s beneficiaries were largely satisfied with the evaluation 
process. However, other applicants mentioned their wish for more detailed feedback. 

The  vast  majority  of  the  number  of  payments  (pre-financing,  interim  and  final  payments) 
were  executed  within  contractual  limits  in  2014  –  2016.  As  regards  the  time  to  pay,  in  the 
period  2014-2016  the  percentage  of  payments  respecting  the  legal  time  limits  set  up  by  the 

21 EASME AAR 2016, https://ec.europa.eu/easme/en/about-easme 
22 EASME AAR 2015, https://ec.europa.eu/easme/en/about-easme 

10 

 
 
 
 
 
 
 
                                                           
Financial Regulation was close to the target of 100%. In 2016, 96% of all payments respect 
the  legal  time  limits,  despite  a  32%  increase  of  the  total  number  of  financial  transactions 
executed in 2016 compared to 2015. 

Average time to pay - % of payments within the legal deadlines 

2014 

CIP eco-innovation  100% 

CIP EEN  97% 

CIP IEE  96% 

COSME  98% 

EMFF 

H2020 Environment 
and resources 

67% 

H2020- 
ENERGY 

100% 

H2020 
SME 

98% 

2015 

100% 

// 

95% 

95% 

100% 

100% 

97% 

99% 

H2020 INNO-SUP  100% 

// 

LIFE 

//  100% 

Source: EASME AAR 2014, 2015 & 2016 

2016 

93% 

94% 

98% 

99% 

100% 

100% 

99% 

95% 

99% 

100% 

EASME  was  also  very  successful  in  timely  evaluation  and  selection  of  proposals.  The 
relevant  time-to-inform  (TTI)  targets  were  reached  and  the  satisfaction  of  project  applicants 
and contractors about the time taken to evaluate the proposals is high.  

75%  of  beneficiaries  consulted23  were  positive  about  how  supportive  EASME’s  staff  was 
during the grant application process. The evaluation of applications is generally well received 
(65,7% positive evaluation). During targeted interviews, some applicants suggested that they 
would appreciate a more transparent grading system and more constructive feedback in order 
to improve future applications. EASME’s beneficiaries, are largely satisfied with the way the 
Agency responded to their needs during the contractual phase. Across programmes, over 75% 
of respondents had a positive opinion about EASME’s service delivery. 

23 Customer satisfaction report, 2.2.10.  

11 

 
 
 
 
 
 
 
                                                           
To what extent has EASME considered the beneficiaries’ needs in the services it offers? 

EMFF

H2020

33% 

33% 

COSME

32% 

33% 

33% 

47% 

51% 

11% 

6% 

5% 

10% 

LIFE

24% 

45% 

14% 

13% 

All programmes

31% 

45% 

12% 

10% 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

To a large extent

To a certain extent

To a limited extent

Not at all

Don't know / no opinion

Source: targeted online survey 

In the period 2014 – 2016 the Agency has improved the management of the programmes in 
terms of proximity to beneficiaries (through the organisation of European information days, 
events  in  the  Member  States  or  kick-off  meetings).  The  majority  of  grant  applicants  and 
tenderers  responding  to  the  survey  were  satisfied  with  the  EASME  staff’s  availability 
(72.6%;)  and  its  promptness  in  responding  to  the  applicants’  requests  (grant  applicants: 
73.3%;  tenderers:  82.4%).  Through  concerted  actions  and  making  use  of  synergies  between 
programmes,  EASME  succeeded  in  raising  the  visibility  of  the  EU  as  promoter  of  the 
programmes concerned. 

5.2.  Efficiency 

This  section  considers  the  relationship  between  the  resources  used  by  the  Agency  and  the 
output.  The  analysis,  among  other  factors,  also  includes  analysis  of  administrative  and 
regulatory burden and looks at aspects of simplification. 

Cost-effectiveness 
In  the  period  2014-2016,  EASME  was  delegated  a  lower  amount  of  operational  budget  for 
Horizon 2020 and EMFF compared to the estimates of the SFS. Whereas, the budget allocated 
for COSME and LIFE is higher than the SFS estimates.  

12 

 
 
 
 
 
Operational budget delegated to the Agency, actual vs foreseen (in the 2013 CBA) 

EUR million  

COSME 

H2020  

LIFE 

2013 CBA 

76,9 

SFS 

80,388 

AAR 

94,95 

Ratio AAR - CBA 

123% 

Ratio AAR - SFS 

118% 

819,3 

785,904 

697,04 

85% 

89% 

262,34 

226,84 

229,89 

88% 

101% 

2013 CBA 

82 

914,9 

285,67 

SFS 

85,635 

1004,763 

240,444 

AAR 

102,19 

830,79 

251,86 

Ratio AAR - CBA 

125% 

Ratio AAR - SFS 

119% 

91% 

83% 

88% 

105% 

2013 CBA 

92,5 

916,7 

303,48 

SFS 

96,129 

1006,579 

258,638 

AAR 

102,71 

934,08 

270,90 

Ratio AAR - CBA 

111% 

Ratio AAR - SFS 

107% 

102% 

93% 

89% 

105% 

Source: VVA based on 2013 CBA, EASME AAR for the years 2014, 2015, 2016 and the SFS 2014 

70,7 

70,668 

34,59 

49% 

49% 

78,2 

78,164 

40,25 

51% 

51% 

86,1 

86,142 

35,88 

42% 

42% 

EMFF 

2014 

2015 

2016 

The  actual  administrative  costs  were  lower  than  forecasted  in  the  SFS  and  2013  CBA  for 
COSME  and  the  EMFF.  For  Horizon  2020,  the  administrative  costs  of  managing  the 
programme  were  (approx.)  aligned  with  the  ex-ante  estimations,  whereas  the  administrative 
costs of LIFE were substantially higher. 

By  expressing  the  cost-efficiency  as  the  ratio  between  the  administrative  budget  and  the 
operational budget, the predictions of the 2013 CBA and of the SFS differ from the “actual” 
results achieved by the Agency in the period 2014-2016. 

13 

 
 
 
 
 
 
Administrative over Operational budget (%) in 2014-2016, per programme delegated 

COSME 

H2020 

LIFE 

EMFF 

2013 CBA 

SFS 

AAR 

2013 CBA 

SFS 

AAR 

2013 CBA 

SFS 

AAR 

2013 CBA 

SFS 

AAR 

2014 

9,9% 

8,4% 

7,0% 

1,6% 

1,0% 

2,0% 

0,2% 

0,2% 

1,4% 

2,7% 

2,3% 

2,3% 

2015 

10,3% 

9,6% 

7,9% 

1,9% 

1,7% 

1,7% 

0,4% 

0,4% 

1,2% 

3,0% 

3,4% 

1,9% 

2016 

9,5% 

8,8% 

6,8% 

2,0% 

2,0% 

2,3% 

0,6% 

0,5% 

1,7% 

3,1% 

3,4% 

7,2% 

Source: VVA based on 2013 CBA, SFS, EASME Annual Activity Report for the years 2014, 2015, 2016 and the SFS 2014 

This  resulted  from  the  fact  that  both  variables  changed:  overall,  less  operational  budget  has 
been  delegated  to  EASME,  (mainly  in  relation  to  Horizon  2020)  and  at  the  same  time,  the 
actual administrative costs of the Agency were lower. The CBA and the SFS underestimated 
the difficulty of the massive recruitment that had to take place in 2014.  There were delays in 
the recruitment, but at the same time staff costs were lower.  The cancellation of the initially 
envisaged move of the Agency to new premises generated also savings. 

The cost of monitoring the Agency were reviewed together with the parent DGs.  They were 
not deemed to differ from the CBA and SFS estimates.  

More information on the EASME cost-efficiency and results of the cost-benefit analysis are 
presented under section 5.4. 

Workload 
The  Agency’s  performance  was  assessed  as  effective  overall.  But  the  fact  that  EASME 
manages a diverse portfolio of programmes and coordinates its activities with a large number 
of parent DGs made it more complex to manage the launch of new calls and the subsequent 
activities in 2014. 

The  actual  workload  related  to  the  management  of  the  programmes  significantly  deviated 
from  the  initial  CBA  estimates.  Staffing  levels  between  the  different  programmes  was 
imbalanced, in particular for the LIFE programme. Since 2014 the staff number increased and 
EASME counted 417 staff by the end of 2016. 

Although it is recognised that EASME managed to cope with the available resources and took 
actions  to  deal  with  unexpected  workload,  the  Agency’s  resource  allocation  could  be  more 
flexible.  As  reported  by  the  supporting  study,  many  factors  have  to  be  considered  when 
assessing  the  implementation  workload.  The  programmes  closely  linked  to  policy  (e.g. 

14 

 
 
 
 
 
 
 
 
 
Horizon 2020 energy efficiency) require more involvement of the staff of the Agency in the 
policy related issues and extensive work on policy feedback.  

In  2016,  the  Agency  performed  a  detailed  workload  assessment.  The  focus  of  this  exercise 
was  to  compare  the  assumptions  made  in  the  CBA  in  2013  with  the  actual  situation24. 
According  to  the  data  collected  in  this  exercise,  most  of  the  workload  is  situated  in 
operational activities, which account for 63% of all FTEs in the Agency. The second largest 
category  of  activities  (25%)  is  finance  and  control  activities.  Finally,  local  activities  (e.g. 
HRM, IT and logistics) represent 12% of all FTEs in the Agency25. 

In  addition,  the  study  makes  three  general  observations  on  the  differences  between 
programmes:  

  The  stability  in  delegated  actions  is  an  important  factor  determining  workload.  Both 
the  COSME  and  EMFF  units  were  confronted  with  frequent  revisions  of  the  Work 
Programme leading to changes in the delegated actions. Contrary to the Horizon 2020 
units, both programmes have a mix of calls for proposals and calls for tender of which 
the proportion also changes during the years, leading to extra instability in workload. 
Half  of  the  budget  delegated  for  COSME  is  implemented  through  the  EEN,  the 
remaining  is  allocated  to  a  number  of  small  actions.  While  smaller  actions  are 
necessary to test new ideas, a high number reduces the potential for cost efficiency in 
the programme implementation. 

  Centrally provided support services, and in particular IT-tools for the management of 

the programme, are vital in moderating the workload. 

While acknowledging challenges in terms of workload, further flexibility to redeploy staff had 
been allowed by the Steering Committee on a temporary and limited basis as of 2017. 

Human Resources Management 
Staffing  and  administrative  budget  are  in  line  with  the  establishment  plan  for  the  planned 
number of posts. The key challenges of human resource management are identified each year 
in the Agency’s work programme. During the evaluation period, the main challenges for the 
Agency concerned recruitment, integration of newcomers, progressive building of core teams 
and skills development26. Over the period 2014 - 2016, EASME’s staff numbers increased in 
line with the 2013 Communication on delegation of programmes to executive agencies27.  

It was challenging during the start-up phase of EASME in 2014 to recruit the high number of 
additional staff scheduled in the budgets. The largest recruitment occurred in 2014, when 140 
candidates  were  recruited,  corresponding  to  a  76%  staff  growth  in  that  year  alone.  This 
massive  recruitment  operation  allowed  EASME  to  cover  the  technical  and  financial  skills 
required  for  delivering  effective  programme  management.  Some  limited  delays  and  budget 
carried forward from 2014 were successfully absorbed in 2015. 

EASME  has  been  able to recruit suitable profiles, skills  and qualifications and has received 
positive  feedback.  Most  of  the  interviewees  (EASME  employees,  parent  DGs,  experts) 
perceive  EASME’s  staff  as  dedicated  and  competent.  The  general  observation  among  the 

24 Please refer to the Cost Benefit Analysis attached to the Evaluation Report from the external supporting study. 
25 The categorisation of activities in operational, neutral and local activities follows DG HR’s typology used in 
their screening exercise of executive agencies’ staff screenings. 
26 Annual Activity Reports EASME 2014, 2015 and 2016.  
27  Communication  to  the  Commission  on  the  delegation  of  the  management  of  the  2014-2020  programmes  to 
executive agencies SEC(2013)493. 

15 

 
 
 
 
 
 
 
 
                                                           
parent DGs is that EASME staff have a high level of expertise in managing the projects and a 
good understanding of the policy context.  

During  the  assessed  period  recruitment  process  faced  three  other  major  challenges: 
understaffing  for  certain  programmes,  delays  in  filling  the  posts,  and  challenges  in  filling 
certain  posts  (notably  Heads  of  Sector).   The  Agency  proposed  a  flexibility  mechanism 
approved by the Steering Committee, allowing a temporary and limited reallocation of human 
resources.  Despite  the  difficulties  to  fill  the  vacancies,  by  31  December  2016  EASME 
managed to  reach its recruitment  target  as foreseen in  the  Agency’s Work Programme  (415 
staff members).  

The  staff  turnover  rate  recorded  in  EASME  remained  stable  around  7%,  lower  than  the 
benchmark rate of 10% for contract agent staff. According to the 2016 Staff Opinion Survey, 
EASME staff positively evaluated the initiatives put in place by the EASME management to 
improve  the  turnover  rate,  such  as  the  adoption  of  a  new  management  charter  and  the 
introduction  of  trainings.  According  to  the  same  survey,  66%  of  EASME  staff  consider  the 
Agency  as  a  modern  and  attractive  place  to  work.  This  corresponds  to  an  increase  of  6% 
compared to the 2014 Staff Satisfaction Survey, although still below ERCEA (78%) and REA 
(76%).  

EASME  managed  to  improve  all  indicators  under  the  section  of  wellbeing  of  the  Staff 
Opinion Survey in 2016. Compared to other executive agencies, EASME scored highest in the 
work-life dimension. The increase in the wellbeing dimension can be seen as a success of the 
Agency’s actions for promoting staff’s work-life  balance through concrete initiatives during 
the assessed period. 

Only 21% of the 2016 staff survey respondents said that there were reasonable opportunities 
to move to another job that matches their current skills and competences. In terms of current 
job, 76% of EASME staff members reported to be satisfied with their job content and tasks, 
which corresponds to an increase of 6% compared to 2014 staff survey results.  

The  evolution  of  key  indicators  for  staff  satisfaction  –  after  the  evaluated  period  –  is 
encouraging: In the latest staff survey (4th quarter of 2018), EASME managed to increase its 
result for staff feeling that the Agency cares about their well-being to 72%, up from 54% in 
2016. Staff engagement rose to 71% (from 70% in 2016). A less positive issue is related to the 
limited  career  opportunities  within  the  Agency28  and  the  professional  future  developments. 
This trend is also confirmed by the 2016 staff survey and it is common to the other executive 
agencies. The main issues identified in  the survey  will have to  be addressed by the Agency 
together with its parent DGs in the most appropriate way and will feed into a follow-up action 
plan. 

Organisational structure 
The organisational structure of EASME has been assessed as functional:  

  Under the 2014-2020 MFF, the number of parent  DGs increased from four to  seven 
(GROW,  ENER,  ENV,  CLIMA,  RTD,  CNECT  and  MARE),  which  added  more 
complexity to the Agency’s governance mechanisms.  

  The Agency was delegated new programmes29 and as a consequence expanded from 
six units (in the predecessor agency) to three departments in EASME and eight units. 

28 Results of interviews conducted with EASME staff.  
29 Annual Activity Report, 2014, p.34. 

16 

 
 
 
 
 
 
 
 
                                                           
Two departments deal with the various new programmes (one is in charge of COSME, 
Horizon 2020 SME and  EMFF, the other  LIFE,  Horizon 2020 Energy, Environment 
and Resources), the third one deals with finances and administration30. 

This  allocation  of  tasks  in  the  different  departments  allows  synergies  (for  example  between 
COSME and Horizon 2020)31.  

The  structure  of  the  Agency  was  also  streamlined  through  the  regrouping  of  the  financial 
tasks  into  one  unit  to  increase  economies  of  scale  and  the  effectiveness  and  efficiency  of 
financial control32. In addition, a procurement cell was created to centralise all administrative 
tasks related to the launch and evaluation of calls for tenders33. 

Interviewees  (parent  DGs  and  EASME  staff)  mentioned  that  a  further  streamlining  of 
EASME’s organisational structure  could  contribute to  generate more synergies in  the future 
between the different programmes. In this regard, there are specific actions to create further 
links, such as the initiative PROBE that aimed at linking different results of operational units 
by analysing the diversity of data from all programmes. 

Simplification 
To increase its efficiency, the Agency took measures to simplify procedures. For example, in 
2014, following an internal audit, the Agency took several cost-saving initiatives in relation to 
missions. In 2015, following a thorough screening of business processes, a more generalised 
introduction of e-signatories was introduced and approval workflows were rationalised. 

The feedback gathered from stakeholders indicate there is room for further improvement.  In 
particular applicants and beneficiaries would appreciate better planning of calls.  This would 
help them better manage the timing of submission of proposals, preparation and conclusion of 
grants.  Some  parent  DGs  also  considered  that  the  preparation  of  procurement  and  grant 
procedures and evaluation of ensuing proposals and tenders could better take into account the 
situations of low risk to improve efficiency without reducing compliance. 

The IT infrastructure is considered satisfactory and effective by the interviewees (in particular 
EASME  staff).  EASME  receives  the  IT  services  and  tools  from  the  Commission  and  is  in 
charge  of  maintaining  the  infrastructure.  In  2014,  new  IT  tools  for  the  management  of 
Horizon  2020  programmes  were  introduced  in  EASME  and  a  sector  for  horizontal  IT  and 
logistic  topics  established.  COSME  started  using  adapted  Horizon  2020  IT  tools  for  the 
project  management.  For  the  LIFE  programme,  the  IT  tools  available  in  DG  ENV  and  DG 
CLIMA were used. The EMFF programme was implemented without any specific tools.  

However,  the  implementation  of  IT  tools  differs  across  programmes:  the  tools  for  Horizon 
2020  are  more  developed  and  have  been  used  as  a  basis  for  other  programmes  with 
adaptation. The introduction of these IT tools initially brought difficulties both for staff and 
applicants but did not impact the projects negatively.  Furthermore, the different programmes 
have very specific needs. 

30 https://ec.europa.eu/easme/sites/easme-site/files/easme-organigramme_en.pdf  
31 According to the majority of EASME employees and parent DGs, information from interviews. 
32 Annual Activity Report, 2014, p. 29. 
33 Annual Activity Report, 2014. 

17 

 
 
 
 
 
 
 
 
 
 
                                                           
5.3.  Coherence 

The  evaluation  did  not  find  any  evidence  of  overlaps,  duplications,  gaps  or  inconsistencies 
within  the  EASME  programme  portfolio.  In  the  2014-2016  period  EASME  demonstrated 
flexibility in implementing a diverse thematic programming portfolio and being answerable to 
seven different parent DGs. 

Portfolio coherence 
There  are  no  particular  inconsistencies  between  the  tasks  performed  by  EASME  in 
implementing the different programmes. The supporting study mentions that “the delegation 
of operational tasks regarding the implementation of the Framework Programme for Research 
and  Innovation  (Horizon  2020),  the  Programme  for  the  Competitiveness  of  Enterprises  and 
small  medium  size  enterprises  (COSME)  and  the  Programme  for  Environment  and  Climate 
Action (LIFE) had generated synergies, simplification and economies of scale.” However, the 
study  also  mentions  that  there  is  potential  to  further  exploiting  synergies  between  different 
programmes. 

The main difficulty identified is the tendency to work in silos given that the programmes have 
different  objectives  and  procedures.  Synergies  could  have  been  created  by  grouping  some 
activities  (e.g.  workshops,  presentations,  promotional  activities)  based  on  the  content  (e.g. 
energy  or  environment,  support  to  SMEs).  This  could  promote  knowledge  spill-overs  and 
learning curve effects. 

According  to  the  interim  evaluation  of  Horizon  202034,  there  is  a  good  level  of  coherence 
between the SME Instrument and the COSME programme, with synergies created by making 
use  of  the  COSME  Enterprise  Europe  Network  to  ensure  access  to  the  Horizon  2020  SME 
Instrument business innovation coaching and mentoring services. EASME staff also reported 
benefits  from  the  collaboration  between  the  units  working  on  the  Horizon  2020  SME 
instrument and COSME, such as using the same structure to identify coaches, dialogue with 
investors  and  offer  support  during  proposal  preparation.  In  addition,  some  COSME  EEN 
members  are  involved  in  a  selection  of  INNOSUP  calls  of  Horizon  202035,  which  offers 
highly  specialised  support  services  that  aim  to  complement  existing  national  and  regional 
support  initiatives36.  Synergies  were  also  created  between  LIFE,  Horizon  2020  Energy, 
Environment and Resources actions. 

Synergies can be exploited also in external communication. The networks and multipliers of 
the  Agency  could  be  used  across  programmes.  For  example,  during  the  EU  Sustainable 
Energy  Week,  SME  Instrument  companies  in  the  field  of  Energy  Efficiency  pitched  their 
projects and products37. 

Relations with parent DGs and policy feedback 
The rationale behind the creation of executive agencies is partly to allow the Commission to 
focus  on  institutional  tasks,  mainly  on  its  core  activities  and  functions  that  cannot  be 
delegated,  such  as  policy-making,  implementation  and  monitoring  of  the  application  of  the 

34  Commission  Staff  Working  Document  -  In-Depth  Interim  Evaluation  of  Horizon  2020  (2017).  SWD(2017) 
220 final. 
35 The Horizon 2020 INNOSUP programme aims to test new approaches for better innovation support through 
funding opportunities for innovation actors across Europe.  
https://ec.europa.eu/easme/en/innosup 
36 Technopolis (2017). Interim evaluation of the COSME programme. 
37 EASME 2016 Annual Activity Report. 

18 

 
 
 
 
 
 
 
                                                           
EU law, and strategic management. Almost  all interviewees from  the parent  DGs expressed 
the  opinion  that  the  delegation  of  the  implementation  of  their  programmes  to  EASME  has 
enabled them to better focus on other tasks, mainly their policy-related tasks.  

When  examining  any  potential  overlaps  and  inconsistencies  between  the  tasks  and 
responsibilities of the Agency and the parent DGs, the supporting study found that there is a 
clear  and  appropriate  delimitation  of  responsibilities  and  tasks  between  the  Agency  and  the 
parent DGs within the current legal framework. This is particularly important given that the 
Agency has seven parent DGs and implements a diverse set of programmes.  

The division of tasks is clearly outlined in the legal framework. A list of the tasks outsourced 
to EASME under each programme is provided in the Annexes to the Act of Delegation.  The 
general  principles  from  the  Act  of  Delegation  are  further  set  out  in  a  Memorandum  of 
Understanding  between  EASME  and  all  the  parent  DGs.  The  detailed  modalities  and 
procedures  of  interaction  between  the  Agency  and  its  parent  DGs  are  defined  in  four 
programme-specific Memoranda of Understanding between the  Agency  and the parent DGs 
ultimately responsible for the programme. The sharing of responsibilities for the execution of 
tasks  between  the  Commission  and  EASME  varies  according  to  the  scope  and  the  level  of 
political sensitivity of each action and is detailed in each MoU. These tasks are even further 
concretised in the Annual Work Programme of the Agency and the Multiannual and Annual 
Work  Programmes  of  the  different  programmes.  These  documents  present  the  key  actions, 
targets, Key Performance Indicators (KPIs) and budget lines of all the programmes managed 
by  the  Agency,  as  well  as  the  responsibility  for  each  action  at  the  level  of  units  within  the 
Agency.  

However,  it  appears  from  the  interviews  that  this  legal  framework  might  not  be  clear  or 
known  to  all  Commission  and  EASME  staff.  For  example,  in  the  area  of  policy  feedback, 
DGs  have  different  views  of  the  level  of  the  Agency’s  involvement  and  the  level  of 
information to be shared with the DGs. 

The  creation  and  further  reinforcement  of  the  Agency  has  allowed  a  retention  of  talent. 
While  the  Commission  was  reducing  its  resources,  a  number  of  contractual  agents  moved 
from the DGs to the Agency. In this way, the Agency has incorporated a good understanding 
of how the Commission works, facilitating the collaboration between the two entities. On the 
other hand, for Commission officers seconded to the Agency, working in EASME is a great 
opportunity to acquire management skills. 

The  exchange  of  information  between  the  Agency  and  the  parent  DGs  works  well  on 
administrative and financial issues, however according to some staff interviewed from a few 
parent  DGs,  the  information  sharing  related  to  the  implementation  of  projects  or  useful  for 
policy developments could be improved.  EASME staff thinks that a better definition of the 
needs of the parent DGs would help them in better addressing those needs. 

The evaluation shows a mixed result for the collaboration and know-how sharing. There is 
a  fruitful  collaboration  between  units  in  parent  DGs  and  units/sectors  in  EASME  but  some 
policy officers in the parent DGs feel a loss of know-how and contact with stakeholders.  
Policy  feedback  on  project  implementation  is  an  essential  element  for  the  Commission  to 
play its role of policy-maker. EASME provides regular and detailed factual reports to the DGs 
containing  information  and  data  on  the  implementation  of  projects  under  the  various 
programmes. The supporting study reported on positive examples, in particular as regards the 
policy-feedback mechanisms set  up between EASME and DG ENER. The supporting study 

19 

 
 
 
 
 
 
 
 
identified  as  a  key  factor  the  high  degree  of  involvement  of  the  Agency  in  policy  and 
programme shaping issues. Nevertheless, in other cases and according to some interviews, the 
policy feedback produced by the Agency do not always respond to the DGs’ needs which in 
turns must be clearly defined.  

Relations with other executive agencies 
There  is  no  overlap  between  the  tasks  of  EASME  and  of  other  executive  agencies.  Most 
collaboration takes place with  REA and  INEA in the implementation  of Horizon 2020.  The 
evaluation  reports  smooth  coordination  and  communication  with  INEA  and  REA.  Some 
stakeholders  mentioned  that  collaboration  with  other  executive  agencies  could  be  further 
strengthened on cross-cutting focus themes to learn from each other and increase synergies on 
work procedures. 

5.4.  Relevance 

The performance of the Agency has been relevant.  It has responded to the needs for which it 
has been created. It has allowed the Commission to focus on policy-making and has allowed 
the  creation  of  synergies,  has  contributed  to  simplification  and  streamlining  as  well  as 
economies of scale. 

Overall,  it  is  perceived  by  all  parent  DGs,  experts,  applicants  and  beneficiaries  alike,  that 
EASME has delivered good services. The client-oriented approach has benefited beneficiaries 
and  applicants  in  the  first  place.  The  expertise  of  EASME  is  highly  valued  by  some  parent 
DGs. The economies of scale the Agency benefits from in handling project applications (e.g. 
for  LIFE)  could  not  have  been  matched  by  the  Commission.  It  is  particularly  noted  that 
EASME succeeded in effectively creating a community of SMEs, which results in synergies 
across all programmes. 

5.5.  Retrospective Cost-benefit analysis 

The 2013 CBA examined four possible scenarios (full internalisation, full externalisation and 
two  alternative  scenarios  with  different  options  for  delegation).  The  results  of  the  ex-ante 
CBA were then adapted in the Specific Financial Statement (SFS).  

The supporting study compared two scenarios, full externalisation and in-house scenario (or 
full  internalisation),  by  using  actual  data.  The  actual  costs  of  the  executive  agency  scenario 
were  lower  than  forecast  in  the  SFS  and  in  the  2013  CBA.  The  actual  savings  were  higher 
than the initial  SFS estimations (EUR 33.9 million  compared to  EUR 9.89 million). This  is 
due  to  the  assumptions  used  in  the  calculations  of  the  “in-house  scenario”  and  the  lower 
number of FTEs who worked in EASME.  Savings of the executive agency scenario (in the 
2013 CBA, in the SFS and in the “actual” scenario) primarily resulted from the higher share 
of CAs employed within the Agency.  

20 

 
 
 
 
 
 
 
Planned  versus  actual  total  costs  under  the  in-house  and  externalisation  scenarios  in 
2014-2016  

 €160.000.000,00  

 €140.000.000,00  

 €136.319.320,50  

 €120.000.000,00  

 €109.456.197,75  

 €112.957.000,00  

 €103.061.000,00  

 €121.220.083,57  

 €87.227.360,00  

 €100.000.000,00  

 €80.000.000,00  

 €60.000.000,00  

 €40.000.000,00  

 €20.000.000,00  

 €-    

 €26.863.122,75  

 €9.896.000,00  

 €33.992.723,57  

CBA 2013

SFS

Actual

Commission scenario (In-house scenario)

Executive agency scenario (externalisation scenario)

Savings

Source: Own elaboration based SFS, EASME financial accounts for the years 2014, 2016 & 2016  

To perform the delegated tasks, the Agency’s resources were calculated on the basis of a cost-
benefit analysis (CBA) and fixed at the end of 2013 in the Communication chapeau and the 
Specific  Financial  Statement  accompanying  the  Agency’s  establishment  decision.  In 
December 2015, the Agency assessed the workload with an internal study. The focus of this 
study was to compare the assumptions made in the CBA in 2013 with the actual situation38. In 
2016, the Agency had 363 FTEs at its disposal (EASME, 2018). This number is clearly lower 
than the number of posts (“heads”) foreseen for the Agency in that same year, i.e. 437 posts 
(of which 417 were filled at the end of 2016). The reason is that not all staff worked full-time 
for  the  whole  year  (e.g.  newcomers  whose  contract  started  later  in  the  year,  staff  with  part 
time working arrangements, sick leaves, long-term leaves or time credits). 

6.  CONCLUSIONS 

Effectiveness 
EASME  has  been  effective  in  allowing  the  Commission  to  focus  on  institutional  tasks  by 
implementing the programmes that have been delegated.  EASME manages substantial parts 
of  four  different  delegated  programmes  covering  a  diverse  range  of  activities  and  funding 
instruments responding to seven parent DGs.  

Efficiency 
In  terms  of  cost-efficiency,  the  actual  figures  for  the  period  2014-2016  show  that  both 
administrative costs and total budget delegated were lower than envisaged in the 2013 CBA 
or in the Specific Financial Statement.  

38 The exercise covered data for the year 2015 and was completed in July 2016 for all operational units.  

21 

 
 
 
 
 
 
                                                           
The costs of the options set out in 2013 were recalculated in the light of the actual costs of the 
implementation.    This  shows  that  the  savings  were  more  important  than  envisaged.  This  is 
mainly  due  to  the  staff  structure  of  the  Agency  that  is  in  majority  composed  of  contractual 
agents. 

The level of staff well-being in EASME is good, and constant efforts are made to improve it.  
Several initiatives have been taken by EASME management and are still continuing after the 
time scope for this evaluation.  Nevertheless, according to the staff survey report, staff has the 
feeling to be limited in career development and this is mainly due to the contract type used.  
This remains a challenge for the future of executive agencies.  

Difficulties in recruiting new staff during the first year generated delays in the programmes’ 
implementation.  The Agency managed to overcome the difficulties and succeeded in building 
up  an  organisational  structure  that  delivered,  based  on  the  diversity  of  the  delegated 
programmes. 

Coherence 
EASME  activities  have  been  found  to  be  coherent  with  the  Commission  activities  and  with 
the  portfolio  delegated  to  the  other  executive  agencies.    EASME  has  been  able  to  produce 
synergies  at  different  levels  by  organising  events  that  were  beneficial  to  stakeholders  of 
various  programmes,  by  exploiting  the  synergies  that  exists  between  the  Enterprise  Europe 
Network and Horizon 2020 actions or by pooling expertise (in the field of financial resource 
for example) in specialised teams or units to the service of the various DGs. 

Nevertheless,  further  synergies  could  be  exploited.    The  structural  organisation  of  EASME 
around thematic departments is convenient from the point of view of everyday work but it can 
also  contribute  to  creating  silos.    It  is  clear  from  interviews  and  surveys  with  Commission 
staff  and  economic  operators  that  participated  in  the  different  programmes  that  the  IT  tools 
and procedures could be further simplified. 

The legal framework and the division of tasks between the Agency and the Commission are 
clear on paper. However, in practice the boundaries between policy development and project 
implementation  are difficult to  draw.  Policy feedback is  key for the Commission to  play its 
role of policy-maker. In some areas (energy) EASME provides relevant feedback and tangible 
contribution  to  policy  implementation,  while  in  other  areas  there  is  scope  for  improvement. 
To this end, the Commission services should provide a more precise definition of their needs. 
Coordination  and  cooperation  between  Commission  staff  and  EASME  staff  are  reported  to 
work and to produce results. 

Relevance and future role of the Agency 
The action of the Agency is still relevant, it has allowed the Commission to pursue its political 
objectives,  while  allowing  economies  of  scale  and  savings,  producing  synergies  and 
improving its performances as the measurement of key performance indicators demonstrate. 

Margins  for  improvement  can  be  found  in  the  fields  of  policy  feedback,  faster  procedures, 
synergies  between  programmes,  simplification  and  career  development  for  the  staff  of  the 
Agency.  To address  the  issue of policy  feedback, the Agency has  already  organised several 
workshops to share the good practices of each delegated programme with all parent DGs. 

22 

 
 
 
 
 
 
 
 
 
 
The  parent  DGs  and  the  Agency  will  prepare  an  action  plan  to  address  the  shortcomings 
identified in this evaluation. This action plan will aim to increase the Agency’s performance 
and contribution to an efficient implementation of the delegated programmes. 

23 

 
 
 
 
ANNEX - EVALUATION QUESTIONS 

Efficiency 
Q1: To what extent was the Agency operating according to the legal framework establishing 
it? 

Q2:  To  what  extent  has  the  Agency  achieved  its  objectives? What  could  be  done  to  render 
EASME more effective in achieving these objectives? 

Q3:  To  what  extent  has  the  Agency  contributed  to  improved  management  of  the 
programme(s) in terms of the elements assessed in the 2013 Cost-Benefit Analysis? 

Q4:  To  what  extent  have  the  activities  of  the  Agency  resulted  in  unintended  effects  (both 
desirable and undesirable)? 

Effectiveness 
Q5:  Did  the  actual  costs  (including  cost  of  coordination  and  monitoring)  of  the  Agency 
correspond to the estimates of the 2013 Cost-Benefit Analysis which was carried out in view 
of its establishment? If not, what are the reasons behind it? 

Q6: Did the actual benefits correspond to the estimates of the 2013 Cost-Benefit Analysis? If 
not, what are the reasons behind it? 

Q7: Was the management and execution of the programme(s) by the Agency cost-efficient? 

Q8: To what extent has the Agency contributed to improved management of the programmes 
in  terms  of  simplification  of  the  procedures  and  flexibility  in  the  implementation  of 
outsourced tasks (e.g. capacity to adapt to periods of high workload)? Where is there further 
scope for simplification? 

Q9: What aspects/means/actors or processes render the Agency more or less efficient? What 
could be improved? 

Q10:  To  what  extent  does  EASME’s  human  resources  management  contribute  to  the 
achievement of the Agency’s objectives? 

Coherence 
Q11: Are there overlaps/gaps/inconsistencies within the programme portfolio managed by the 
Agency? 

Q12:  Is  there  a  clear  and  appropriate  delimitation  of  responsibilities  and  tasks  between  the 
Agency and the parent DGs (no overlaps or gaps)? 

Q13:  To  what  extent  are  there  overlaps/inconsistencies  between  the  tasks  carried  out  by 
EASME and other EU executive agencies? 

Relevance 
Q14: Have objectives and tasks responded to past needs and still respond to current needs? 

24