Evaluation of the Executive Agency for Small and Medium-Size Enterprises (EASME)
Date: 2017-11-14
Institutions: Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs, European Commission
Subjects: EU investment



IMMC.Ares%282017%295558220.ENG.xhtml.PART-2017-522293V1.docx











EVALUATION ROADMAP  






Title of the evaluation




Evaluation of the Executive Agency for Small and Medium-Size Enterprises (EASME)






Lead DG – responsible unit 




Directorate General for  Internal Market, Industry, Entrepreneurship and SMEs - .unit H.1, COSME Programme, SME Envoys and Relations with EASME






Indicative Planning 


(planned start date and completion date)




Q4 2017


Q4/2018






Additional Information




-






The Roadmap is provided for information purposes only. It does not prejudge the final decision of the Commission on whether this initiative will be pursued or on its final content. All elements of the initiative described by the document, including its timing, are subject to change.









A. Context, purpose and scope of the evaluation






Context 






The evaluation of the Executive Agency for Small and Medium-Size Enterprises (EASME) is a legal obligation stemming from Article 25(1) of 'the Framework Regulation for executive agencies'
1
, requiring the Commission to carry out an external evaluation of the operation of each executive agency every three years.


EASME is the successor of Intelligent Energy Executive Agency (IEEA), set up in 2003. It was extended to implement new programmes and transformed into the European Agency for Competitiveness and Innovation (EACI) in 2007. In 2013, the EACI was replaced and succeeded by the EASME to implement all Union programmes in the field of energy, environment, climate action, competitiveness and SMEs, research and innovation and ICT.


According to the Framework Regulation and EASME's Establishing Act
2
, the outsourcing of certain management tasks to the executive agency intends:


-To allow the Commission to focus on its institutional tasks; i.e. tasks assigned to the institutions by the Treaty which require discretionary powers in translating political choices into action. Such institutional tasks should not be outsourced.


-To enable achieving the goals of Community programmes more effectively. According to a Cost-Benefit Analysis carried out in 2013 delegating tasks to the executive agency was estimated to be more cost-efficient than an in-house scenario. The Establishing Act estimated that the alignment of more coherent programme portfolios with the Agency’s core competences and its brand identity would bring qualitative benefits. In addition, it projected that assembling the management of different EU programmes would bring synergies, simplification and economies of scale. Furthermore, the pooling of all aspects of the Horizon 2020 ‘SME instrument’ would provide a single point of access to potential beneficiaries and guarantee consistent service delivery. 


The Agency implements delegated tasks in close cooperation with its seven parent Directorates General: (1) DG for Internal Market, Industry, Entrepreneurship and SMEs (DG GROW), (2) DG for Research and Innovation (DG RTD), (3) DG for Communications Networks, Content and Technology (DG CNECT), (4) DG for Climate Action (DG CLIMA), (5) DG for Energy (DG ENER), (6) DG for Environment (DG ENV) and (7) DG for Maritime Affaires and Fisheries (DG MARE).


The Agency is entrusted to implement SME-related parts of several EU programmes under the 2014-2020 Multi-Annual Financial Framework (MFF): COSME the Programme for the Competitiveness of Enterprises and small and medium-sized Enterprises, HORIZON 2020 the Framework Programme for Research and Innovation, LIFE the Programme for the Environment and Climate Action, EMFF the European Maritime and Fisheries Fund, the legacy of certain actions under CIP the Competitiveness and Innovation Programme  2007-2013.


The Agency implements the COSME programme excluding the financial instruments and some other actions which are managed by GROW.


Under the HORIZON programme the following parts are delegated: innovation in SMEs of "Part II Industrial Leadership", SME Instrument, fast track to innovation pilot scheme, societal challenge 'secure and efficient energy', societal challenge 'Climate action, environment, resource efficiency and raw materials.


Under the LIFE programme the Agency manages the calls for action grants under the Environment sub-programme
                           3
 and the Climate Action sub-programme, as well as the call for operating grants to NGOs that help to implement and shape EU policies on the environment and climate change.


Concerning the European Maritime and Fisheries Fund, a large part of the direct management of policy support actions in the area of Integrated Maritime Policy, Control and Scientific Advice and Knowledge is delegated.


The Agency works closely with other partners such as the Common Support Centre for Horizon 2020, the Research Executive Agency (REA) for some logistic and administrative support services and an external contractor for certain tasks under the LIFE programme.






Purpose and scope






The purpose of this evaluation is the periodical (tri-annual) evaluation of the operation of the executive agency for Small and Medium-Size Enterprises (EASME). EASME is governed by its Establishing Act'
4
, which sets out the tasks to be performed by the Agency and the Act of Delegation
5
 which delegates powers to the agency to perform its tasks. These tasks relate to the implementation of dedicated parts of certain EU financial programmes. Furthermore, the agency is governed by the Framework Regulation for executive agencies
6
. 


In line with the Commission's better regulation principles
7
, the evaluation applies different evaluation criteria to guide the analysis of the performance of the agency. It assesses whether the agency has fulfilled its tasks in an efficient and effective way, whether there are overlaps/ gaps/ inconsistencies or synergies within the management of the programme portfolio executed by the agency; between the delineation of tasks between EASME and the parent DGs or other executive agencies (coherence) and whether objectives and tasks responded to past needs and still respond to current needs (relevance)
8
.



The evaluation questions below indicate possible issues this evaluation will address.


EFFECTIVENESS – to what extent has the objective of delegating programme management tasks to EASME been achieved?


·To what extent was the EASME operating according to the legal framework establishing it?


·To what extent has the EASME achieved its objectives? What, if anything, could be done to render the EASME more effective in achieving these objectives?


·To what extent has the EASME contributed to improved management of the programme(s) in terms of the elements assessed in the 2013 Cost-Benefit Analysis.


EFFICIENCY – to what extent has EASME carried out its tasks efficiently in line with estimations?



·Did the actual costs (including cost of coordination and monitoring) of the EASME correspond to the estimates of the 2013 Cost-Benefit Analysis which was carried out in view of its establishment? If not, what are the reasons behind?


·Did the actual benefits correspond to the estimates of the 2013 Cost-Benefit Analysis? If not, what are the reasons behind?


·Was the management and execution of the programme(s) by the Agency cost-effective as compared to the alternative options assessed? 


·To what extent has the EASME contributed to improved management of the programme(s) in terms of simplification of the procedures and flexibility in the implementation of outsourced tasks (e.g. capacity to adapt to periods of high workload)?


·Which further scope for simplification exists?


·What aspects/means/actors or processes render the EASME more or less efficient? What could be improved?


·To what extent has EASME's internal organisation and procedures been conducive to its efficiency? 


·Is the size of the agency adequate? Is the balance between operational staff and administrative support appropriate?


COHERENCE


·Are there overlaps/ gaps/ inconsistencies within the programme portfolio managed by the agency?


·Are there inconsistencies between the tasks carried out by the different executive agencies?


·Is there a clear and appropriate delimitation of responsibilities and tasks between the EASME and the parent-DGs? Are there overlaps or gaps?


·To what extent has the EASME enabled the Commission to better focus on its policy related tasks?


·Are there any governance (financial and policy) issues in relation to the implementation of the tasks that are delegated to the executive agencies/ issues as regards the Commission's services able to steer EU policy or budget implementation? 


·Are appropriate mechanisms and instruments put in place to ensure an adequate coordination and information flow between the EASME and the Commission services?


·Does EASME provide useful information in support of the policy process?


RELEVANCE


·Do objectives and tasks responded to past needs and still respond to current needs?



The evaluation covers all the tasks carried out by the agency during the years 2014-2016 and will be supported by a study to be carried out by an external contractor. 


The results of this evaluation will be summarised in an evaluation Report to be established by the Commission and presented to the European Parliament, to the Council and to the Court of Auditors in accordance with Article 25(1) of the Framework Regulation. 


The evaluation will help the Commission services to assess whether the functioning of the agency has yielded the expected positive results as estimated in the Cost-Benefit Analysis for delegating task to the executive agency
9
 and identify potential areas of improvement.  The results will feed into a future Cost-Benefit Analysis to assess the opportunity of expanding/modifying the executive agency's mandate after the expiry of the current mandate. The aspects to be covered by the Cost-Benefit Analysis are specified in Article 3(1) of the Framework Regulation
10
 and the Guidelines on establishing and operating executive agencies
                           11
.


Following the evaluation, the executive agency and the Commission will take all appropriate steps to resolve any problems identified
12
. If, further to an evaluation, the Commission finds that the very existence of an executive agency is no longer justified with a view to sound financial management, the Commission shall decide to wind up the agency
13
.


The evaluation shall not cover the achievements of the programmes managed by EASME, which are subject to separate mid-term and ex-post evaluations. The evaluation of EASME will, nevertheless, provide useful input for these programme evaluations, as the performance of the agency affects the efficiency and effectiveness of the programmes it manages.


The evaluation will examine the efficient use of resources and the effective achievement of the tasks entrusted to it. It will in particular look at whether the alignment of more coherent programme portfolios with the Agency’s core competences and its brand identity brought the estimated qualitative benefits; whether the assembly of the management of different EU programmes brought the estimated synergies, simplification and economies of scale; whether the pooling of all aspects of the Horizon 2020 ‘SME instrument’ guaranteed consistent service delivery and whether there is scope for simplification and further efficiency gains.


Also the continued relevance of all objectives and tasks entrusted to the agency will be assessed as well as the coherence of its programme portfolio and the delineation of tasks between the agency and other executive agencies. Furthermore it will examine whether the outsourcing allowed the parent DGs to focus on core tasks.







B. Better regulation






Consultation of citizens and stakeholders 






The Commission envisages a consultation of the EASME's stakeholders as part of this evaluation for the 4th quarter of 2017.


In addition to SMEs, there are three types of main stakeholders: EU bodies and national authorities in charge of policy making and supervision of the activity in the area of each programme, various bodies (they may be public entities) playing the role of intermediaries or facilitators for projects, and last but not least the operators having obtained support for projects or having been rejected.  


The targeted consultation will collect views and data, from the identified stakeholder groups on the modus operandi of the Agency in each programme, with particular focus on those aspects of the agency's efficiency that are accessible to the stakeholders (e.g. access to information, quality of it, time and quality of response to communications, time and quality of decisions, time and quality of reporting)


A 12 week online open public consultation will take place in the first quarter of 2018. The open public consultation will be carried out in English while replies can be made in any of the 24 official EU languages. The open public consultation can be accessed via the Commission's central public consultation page.


A synopsis report in English, summarising the results of all consultation activities will be published on the public consultation page and on EASMEs website once all consultation activities are closed.






Data collection and methodology 






The following data will provide a key input for the evaluation:
                              


-agency establishment act and act of delegation


-strategic plans and annual management plans of the parent DGs


-annual work programmes of the Agency


-annual work programmes of the programmes COSME, HORIZON 2020, LIFE, EMFF


-list of projects financed by the Agency


-manual of procedures of the Agency


-annual activity reports of the parent DGs and programme statements of the relevant programmes


-annual activity reports of the Agency


-agendas and minutes of the steering committee meetings


-audit reports by any competent  body


as well as other relevant evaluations


2013 Cost Benefit Analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the Executive Agencies


2015 Evaluation of EACI (2011-2013).


2017 Interim Evaluation of Horizon 2020 (2014-2016).


2017 Ongoing Interim Evaluation of COSME (2014-2016).


2017 Interim Evaluation of LIFE (2014-2016).


2017 Ongoing Interim Evaluation of EMFF (2014-2016).










(1)


 Council Regulation (EC) No 58/2003 of 19 December 2002 laying down the statute for executive agencies to be entrusted with certain tasks in the management of Community programmes




(2)
 2013/771/EU: Commission Implementing Decision of 17 December 2013 establishing the ‘Executive Agency for Small and Medium-sized Enterprises’ and repealing Decisions 2004/20/EC and 2007/372/EC



(3)

   Except for integrated projects and technical assistance projects which will be managed by DG ENV until 2017



(4)
 
                        2013/771/EU: Commission Implementing Decision of 17 December 2013 establishing the ‘Executive Agency for Small and Medium-sized Enterprises’ and repealing Decisions 2004/20/EC and 2007/372/EC



(5)
 
                        There is an act of delegation for each of the programmes delegated to EASME.



(6)
3 
                        Commission Regulation (EC) No 1653/2004 of 21 September 2004 on a standard financial regulation for the executive agencies pursuant to Council Regulation (EC) No 58/2003 laying down the statute for executive agencies to be entrusted with certain tasks in the management of Community programmes and modification by the Commission Regulation (EC) No 651/2008 of 9 July 2008 



(7)
4
   
Commission's Better Regulation Communication COM (2015) 215 final
 and its accompanying 
SWD (2015) 111 final
 and 
SWD (2015) 110 final



(8)


5
   The assessment of 'EU added value', why the EU should act, is not perceived to be a relevant criterion, as EASME executes tasks which the European Commission has transferred to it. The EU added value of the programmes that EASME manages is assessed as part of the programme evaluation.




(9)

   Cost-Benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the executive agencies - Final report for the Commission of 19/08/2013



(10)


   Identification of the tasks justifying outsourcing, a cost-benefit analysis which includes the costs of coordination and checks, the impact on human resources, possible savings within the general budgetary framework of the European Union, efficiency and flexibility in the  implementation of outsourced tasks, simplification of the procedures used, proximity of outsourced activities to final beneficiaries, visibility of the Community as promoter of the Community programme concerned and the need to maintain an adequate level of know-how inside the Commission. 




(11)


   Appendix II of the Guidelines for the establishment and operation of executive agencies financed from the Union budget (C (2014) 9109 from 2/12/2014; page 64-72).




(12)

   Article 25(3) of Regulation 58/2003



(13)

   Article 25(4) of Regulation 58/2003