Source: EURLEX
Language: en
Format: md

|  |  |  |  |
| --- | --- | --- | --- |
| EVALUATION ROADMAP | | | |
| title of the evaluation/fc | Triennial (16 July 2015 – 15 July 2018) evaluation of the Research Executive Agency (REA). | | |
| lead dg – responsible unit | DG RTD – R4 "New management modes" | date of this roadmap | 03.2018 |
| type of evaluation | Evaluation | planned start date  planned completion date | Q1/ 2018    Q2/ 2019 |
|  |  | Planning calendar | <http://ec.europa.eu/smart-regulation/evaluation/index_en.htm>  (to be published) |
| This indicative roadmap is provided for information purposes only and is subject to change. | | | |

|  |  |
| --- | --- |
| A. Purpose | |
| (A.1) Purpose | |
| The purpose of this periodical (every 3 years) evaluation of the operation of the Research Executive Agency (REA) is to assess REA’s implementation of the parts of the EU funding programmes which have been entrusted to it and the provision of administrative and logistical support services.    REA is governed by the  [Framework Regulation for executive agencies](http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1518533098110&uri=CELEX:32003R0058) , its  [Establishing Act](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32013D0778%2801%29) , which sets out its mandate; the  [Act of Delegation](http://ec.europa.eu/transparency/regdoc/rep/3/2013/EN/3-2013-9418-EN-F1-1.PDF)  which specifies the tasks to be carried out by, and the powers delegated to the Agency in order to perform its mandate; and the decision establishing the REA Steering Committee.    In line with the Commission's Better Regulation principles [1](#footnote1) , the evaluation applies several standard evaluation criteria. The evaluation will assess whether the Agency has fulfilled its tasks in an effective and efficient way, whether there are overlaps / gaps / inconsistencies in the management of the programme portfolio by the Agency, and whether there is a clear delineation of tasks between REA and the parent DGs or other executive agencies (coherence) [2](#footnote2) .    The results of this evaluation will be summarised in an Evaluation Report to be established by the Commission and presented to the REA Steering Committee, the European Parliament, the Council and the Court of Auditors in accordance with Article 25(1) of the Framework Regulation.    The evaluation will help the Commission services to assess whether the functioning of the Agency has yielded the expected positive results as estimated in the ex-ante Cost-Benefit Analysis for the delegation of tasks to the executive agencies [3](#footnote3)  and identify potential areas of improvement.    The results will feed into a future Cost-Benefit Analysis to assess the opportunity of expanding / modifying the executive agency's mandate under the post-2020 Research and Innovation Framework Programme or after the expiry of the current mandate (31/12/2024).    Following the evaluation, the Commission and the Agency will take all appropriate steps to resolve any problems identified [4](#footnote4) . If, further to the evaluation, the Commission finds that the very existence of the Agency is no longer justified with a view to sound financial management, the Commission shall decide to wind up the Agency [5](#footnote5) . | |
| (A.2) Justification | |
| The tri-annual evaluation of REA is a legal obligation stemming from Article 25(1) of the  [Framework Regulation for executive agencies](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32003R0058&qid=1503926775681) , requiring the Commission to carry out an external evaluation of the operation of each Executive Agency every three years. | |
|  | |
| B. Content and subject of the evaluation | |
| (B.1) Subject area | |
| The subject area is the functioning of REA between 16 July 2015 and 15 July 2018.    REA was set up by Commission  [Decision 2008/46/EC](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32008D0046)  of 14 December 2007. The REA’s mandate was formally re-established by Commission  [Decision 2013/778/EU](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32013D0778%2801%29) , repealing the previous act of establishment.    REA is responsible, under the supervision of the Commission, for the implementation of the following EU programmes in parts or fully:     Horizon 2020 research projects. The REA manages scientific projects funded under the following parts of the H2020 framework programme (grouped by pillar):  Excellent Science  - [Marie Skłodowska-Curie Actions](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/marie-sklodowska-curie-actions)    - [Future and Emerging Technologies](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/future-and-emerging-technologies)  Industrial leadership:  - [Space Research](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/space)  Societal challenges:  - [Food Security, Sustainable Agriculture and Forestry, Marine, Maritime and Inland Water Research and the Bioeconomy](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/food-security-sustainable-agriculture-and-forestry-marine-maritime-and-inland-water)  - [Innovative, Inclusive and Reflective Societies](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/europe-changing-world-inclusive-innovative-and-reflective-societies)  - [Secure Societies](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/secure-societies-%e2%80%93-protecting-freedom-and-security-europe-and-its-citizens)  Specific objectives:  - [Spreading Excellence and Widening Participation](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/spreading-excellence-and-widening-participation)  - [Science with and for Society](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/science-and-society)    As of 2018, REA’s mandate is extended to the processing of projects that may generate EU classified information (EUCI) under the specific objective ‘’Secure Societies – Protecting freedom and security of Europe and its citizens’’ of part III Societal Challenges.    FP7 research projects    The Agency continues to manage ongoing FP7 research projects in the following areas:    Certain programme areas of the specific Programme  [People](http://cordis.europa.eu/fp7/people/)  of the 7th Framework Programme)  The actions “ [Research for SMEs](http://ec.europa.eu/research/sme-techweb/index_en.cfm) ” and “Research for SME associations” of the Specific Programme  [Capacities](http://cordis.europa.eu/fp7/capacities/home_en.html)  of the 7th Framework Programme.  The theme ‘ [’Space’’ of the Specific Programme Cooperation of the Seventh Framework programme](https://ec.europa.eu/growth/sectors/space/research/fp7_en)   [The theme ‘’Security’’](http://ec.europa.eu/research/fp7/index_en.cfm?pg=security)  of the Specific Programme Cooperation of the Seventh Framework programme    In addition, REA was entrusted with administrative and logistical support services:    The Agency runs the support for independent experts evaluating project proposals, including the management of on-site evaluation facilities for most of H2020 activities and provides services for the legal and financial validation of participants in H2020 and certain other programmes.  As of 2018, in the context of the Commission's initiatives for synergies and efficiencies between programmes, REA plays a major role in the development and roll-out of the SEDIA (Single Electronic Data Interchange Area) project, building on REA's participant validation activities supported by the Participant Data Management (PDM) tool.    The Agency works under the supervision of six parent Directorate-Generals:     [Research and Innovation (RTD) – Lead parent DG;](http://intranet-rtd.rtd.cec.eu.int/rtd_en.htm)   [Internal Market, Industry, Entrepreneurship and SMEs (GROW)](https://myintracomm.ec.europa.eu/dg/GROW/Pages/homepage.aspx) ;   [Education and Culture (EAC)](https://myintracomm.ec.europa.eu/dg/eac/Pages/index.aspx) ;   [Communications Networks, Content and Technology (CNECT)](https://connected.cnect.cec.eu.int/welcome) ;   [Agriculture and Rural Development (AGRI);](https://myintracomm.ec.europa.eu/dg/agri/Pages/Homepage.aspx)   [Migration and Home Affairs (HOME)](https://webgate.ec.europa.eu/fpfis/wikis/display/NMM/Research+Executive+Agency) .    The Agency works closely with the Common Support Centre for Horizon 2020. | |
| (B.2) Original objectives of the intervention | |
| The outsourcing of certain management tasks to the executive agency according to the Framework Regulation and REA's Establishing Act intends:    To allow the Commission to focus on its institutional tasks, i.e. tasks assigned to the institutions by the Treaty which require discretionary powers in translating political choices into action. Such institutional tasks should not be outsourced.    To enable the Commission to achieve the objectives of the delegated EU programmes more effectively and efficiently. According to the Cost-Benefit Analysis carried out in 2013, delegating tasks to REA was estimated to be more cost-efficient than an in-house scenario. The Establishing Act estimated that the alignment of more coherent programme portfolios with the Agency’s core competences and its brand identity would bring qualitative benefits. In addition, it projected that the assembly of the management of different EU programmes would bring synergies, simplification and economies of scale. | |
| (B.3) How the objectives were to be achieved | |
| To allow for the objectives to be achieved, the Agency was made responsible for the implementation of parts of the Horizon 2020 Specific Programme and FP7 legacy. It was also entrusted with the provision of administrative and logistical support services as outlined in section B.1 above.    In regards to the tasks delegated in the framework of Horizon 2020 and FP7 legacy, REA performs the following programme implementation tasks:    Manages some or all of the stages in the lifetime of projects of the delegated programmes. In this connection, it is responsible for monitoring the projects, making the necessary checks and recovery procedures, and for performing budget implementation tasks covering revenue and expenditure within the meaning of Regulation (EU, Euratom) No 966/2012.  Performs all the operations required to launch contests and award prizes in accordance with the Regulation (EU, Euratom) No 966/2012.  Concludes public procurement procedures and manages the ensuing contracts, including the operations required to launch and conclude public procurement procedures.  Provides support in programme implementation.    These tasks are further detailed in ANNEX I and II to the Act of Delegation.      In regards to the tasks delegated within the framework of the administrative and logistical support services, REA provides administrative and logistical services for an increasing number of programmes. The list of programs and tasks entrusted are listed in the Annex III to the Act of Delegation. | |
|  | |
| C. Scope of the evaluation | |
| (C.1) Topics covered | |
| A main topic will be to evaluate to which extent the original objectives of the delegation of Commission tasks to REA have been met (see section B2 above). To this end the estimations of the Cost-Benefit Analysis of 2013 will be tested to provide evidence on the validity of the assumptions made in the ex-ante scenario by considering the actual costs and benefits of programme implementation by the Executive Agency in a structured way.    The aspects to be covered by the Cost-Benefit Analysis are specified in Article 3(1) of the Framework Regulation7 and the Commission Guidelines for the establishment and operation of executive agencies8.    The evaluation shall not cover the achievements of the programmes managed by REA, which are subject to mid-term and ex-post evaluations themselves. The evaluation will nevertheless provide useful input for the preparation of the successor programmes post-2020, considering that the performance of the agency affects the efficiency and effectiveness of the programmes it manages.    It will in particular look at whether the alignment of more coherent programme portfolios with the Agency’s core competences and the development of its brand identity brought the estimated qualitative benefits; whether the assembly of the management of different EU programmes brought the expected synergies, simplification and economies of scale; whether the pooling of instruments guaranteed consistent service delivery and whether there is scope for simplification and further efficiency gains.    The evaluation will cover the operation of REA between 16 July 2015 and 15 July 2018. | |
| (C.2) Issues to be examined | |
| The below questions indicate possible issues this evaluation will address:  Effectiveness – to what extent have the objectives of delegating programme management tasks to REA been achieved?  To what extent was REA operating according to the legal framework establishing it?  To what extent has REA achieved its objectives? What, if anything, could be done to render REA more effective in achieving these objectives?  To what extent has REA contributed to improved management of the programmes, in terms of the elements assessed in the 2013 Cost-Benefit Analysis (see B2 above)?    Efficiency – to what extent has REA carried out its tasks efficiently in line with estimations?  Did the actual costs (including cost of coordination and monitoring) of REA correspond to the estimates of the 2013 CBA which was carried out in view of its establishment? If not, what are the reasons behind?  Did the actual benefits correspond to the estimates of the 2013 CBA? If not, what are the reasons behind?  Was the management and execution of the programmes by REA cost-effective as compared to the alternative options?  To what extent has the REA contributed to improved management of the programmes in terms of simplification of the procedures and flexibility in the implementation of outsourced tasks (e.g. capacity to adapt to periods of high workload)?  Which further scope for simplification exists?  What aspects/means/actors or processes render REA more or less efficient? What could be improved?  To what extent has REA’s internal organisation and procedures been conducive to its efficiency?  Is the size of the REA adequate? Is the balance between operational staff and administrative support appropriate?      Coherence – to what extent there is consistency between the tasks managed by REA and those managed by the parent DGs and other executive agencies, and has the Commission been able to maintain an adequate level of know-how in relation to the programmes entrusted to the Agency?    Are there overlaps/ gaps/ inconsistencies within the programme portfolio managed by REA?  Is there a clear and appropriate delimitation of responsibilities and tasks between REA and the Parent-DGs? Are there overlaps or gaps?  To what extent has REA enabled the Commission to better focus on its policy related tasks?  -Are there any governance (financial and policy) issues in relation to the implementation of the tasks that are delegated to the executive agencies/ issues as regards the Commission's services ability to steer EU policy or budget implementation?  -Are appropriate mechanisms and instruments put in place to ensure an adequate coordination and information flow between REA and the Commission services?  -Does REA provide useful information in support of the policy process? | |
| (C.3) Other tasks | |
| N.A. | |
| D. Evidence base | |
| (D.1) Evidence from monitoring | |
| The following data will provide a key input for the evaluation:    - REA's establishment act and act of delegation  - Memorandum of Understanding  - strategic plans and annual management plans of the parent DGs  -  [annual work programmes of the Agency](http://intranet-rea.rea.cec.eu.int/sites/rea/about/activities/strategic_planning/Pages/Strategic-Planning-and-Programming.aspx)  - annual work programmes of the delegated programmes  - manual of procedures of the Agency  - annual activity reports of the parent DGs and programme statements of the relevant programmes  - annual activity reports of the Agency  - minutes of REA's management meetings  - interim reports to the parent DGs (bi-annual)  - agendas and minutes of the steering committee meetings  - audit reports (Court of auditors)  - Corda data on proposals and grant agreements  - Human resources related data | |
| (D.2) Previous evaluations and other reports | |
| Cost Benefit Analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the Executive Agencies, 19 August 2013, GFK in cooperation with Technopolis;   [Mid-term evaluation of Horizon 2020 programme](https://ec.europa.eu/research/evaluations/index_en.cfm?pg=h2020evaluation)  and its annexes   [Evaluation of the Operation of the REA (2012-2015)](https://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/rea_evaluation_report.pdf) , PPMI (December 2016). This evaluation also includes an update of the CBA of 2013. | |
| (D.3) Evidence from assessing the implementation and application of legislation (complaints, infringement procedures) | |
| N.A. | |
| (D.4) Consultation | |
| The Commission envisages a consultation of REA's relevant stakeholders as part of this evaluation in the 2nd part of 2018.    The main stakeholders of the Agency are the Commission Services, the Member States and the beneficiaries /applicants of the different programmes managed by the Agency.    The targeted surveys and interviews to be performed in the context of the evaluation will collect views and data from the identified stakeholder categories on the modus operandi of the Agency in each programme, with particular focus on those aspects of the Agency's effectiveness, efficiency and coherence that are accessible to the stakeholders (access to and quality of information, time and quality of response to communications, time and quality of decisions, time and quality of reporting etc.)    [Open public consultations](https://ec.europa.eu/research/consultations/interim_h2020_2016/consultation_en.htm)  have already been carried out for the finalised and ongoing evaluations of part of the programmes managed by the Agency. Relevant results for the Agency will be considered in this evaluation.    A synopsis report in English, summarising the results of all consultation activities will be published on the public consultation page once all consultation activities are closed. | |
| (D.5) Further evidence to be gathered | |
| N.A. | |
|  | |
| E. Other relevant information/ remarks |
| An external independent contractor will be involved whose tasks will among others focus on the collection of data (including targeted surveys and interviews) and other factual information and their analysis for the purposes of this evaluation. |

  

 For internal use only – not for publication

|  |  |
| --- | --- |
| Planning | |
| Key milestones (indicative) | |
| Steering Group set up | Q1 2018 |
| Public Consultation | N/A |
| Other Stakeholder Consultation | Q3 2018 |
| Submission to Regulatory Scrutiny Board | If applicable |
| Final Report | December 2018 |
| ISC launch | 1Q 2019 |
| Deadline for Report to Council and European Parliament | Q1 2019 |
| Dissemination Plan | N/A |
| Action Plan | Q2 2019 |

:   [(1)](#footnoteref1)
     
       Commission's Better Regulation Communication COM (2015) 215 final and its accompanying SWD (2015) 111 final and SWD (2015) 110 final.
:   [(2)](#footnoteref2)
     
       The assessment of the evaluation criterion 'EU added-value', i.e. why the EU should act, is not perceived to be a relevant criterion for the evaluation of REA as REA carries out tasks which the European Commission has transferred to it. The EU added-value of the programmes that REA manages is assessed in the context of the programmes evaluations. The previous needs which REA were meant to address and whether they still are pertinent today (relevance) will be presented in the context of the evaluation.
:   [(3)](#footnoteref3)
     
       Cost-Benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the Executive Agencies - Final report for the Commission of 19/08/2013, GFK in cooperation with Technopolis.
:   [(4)](#footnoteref4)
     
       Article 25(3) of Regulation 58/2003.
:   [(5)](#footnoteref5)
     
       Article 25(4) of Regulation 58/2003.

[Top](#document1)