Interim evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA)
Date: 2018-03-15
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: cultural policy, education policy, young person, youth policy



IMMC.Ares%282018%291436575.ENG.xhtml.PART-2018-155884V1.docx











EVALUATION ROADMAP  






Roadmaps aim to inform citizens and stakeholders about the Commission's work to allow them to provide feedback and to participate effectively in future consultation activities. Citizens and stakeholders are in particular invited to provide views on the Commission's understanding of the problem and possible solutions and to share any relevant information that they may have.






Title of the evaluation




Interim evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA).






Lead DG – responsible unit 




EAC R2






Indicative Planning 


(planned start date and completion date)




Planned start date: Q1/ 2018


Planned completion date: Q4/ 2018






Additional Information




– 






The Roadmap is provided for information purposes only. It does not prejudge the final decision of the Commission on whether this initiative will be pursued or on its final content. All elements of the initiative described by the document, including its timing, are subject to change.









A. Context, purpose and scope of the evaluation






Context 






As required by Article 25(1) of the Framework Regulation for executive agencies of the Commission, an evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA) is performed with a 3-year frequency to assess EACEA’s implementation of the parts of the EU funding programmes which have been entrusted to it. The evaluations of all Executive Agencies is coordinated at a horizontal level.


EACEA is governed by the Framework Regulation for executive agencies; EACEA Establishment Act, which sets out its mandate; EACEA Delegation Act and its successive amendments, specifying the tasks to be carried out by, and the powers delegated to, the Agency in order to perform its mandate; and the decision establishing the EACEA Steering Committee.


EACEA is responsible for the management of parts of the programmes Erasmus+, Creative Europe, Europe for Citizens, and EU Aid volunteers, under direct management via grants and other forms of funding.






Purpose and scope 






In line with the Commission's better regulation principles
1
, the evaluation applies several standard evaluation criteria. The evaluation will assess whether the Agency has fulfilled its tasks in an effective and efficient way, whether there are overlaps / gaps / inconsistencies in the management of the programme portfolio by the Agency, and whether there is a clear delineation of tasks between EACEA and the parent DGs or other executive agencies (coherence)
2
.


The evaluation covers all the tasks carried out by the agency during the years 2015-2017 and will be supported by a study to be carried out by an external contractor. 


The results of this evaluation will be summarised in an evaluation Report to be established by the Commission and presented to EACEA Steering Committee, the European Parliament, the Council and the Court of Auditors in accordance with Article 25(1) of the Framework Regulation.


The evaluation will help the Commission services assess whether the functioning of the Agency has yielded the expected positive results as estimated in the ex-ante Cost-Benefit Analysis for the delegation of tasks to the executive Agencies
3
 and identify potential areas of improvement. 


The results will feed into a future Costs-Benefit Analysis to assess the opportunity of expanding / modifying the mandate of executive Agencies at the start of the next Multi-annual Financial Framework (MFF). Results may also be used in the context of the potential transfer of new activities from parent DGs to EACEA. 


Following the evaluation, the Commission and the Agency will take all appropriate steps to resolve any problems identified
4
. If, further to the evaluation, the Commission finds that the very existence of the Agency is no longer justified with a view to sound financial management, the Commission shall decide to wind up the Agency
5
.  






B. Better regulation






Consultation of citizens and stakeholders 






The Commission envisages a consultation of the EACEA's stakeholders as part of this evaluation around mid-2018. 


As a part of the methodology for providing answers to the evaluation questions, a survey in combination with interviews will be conducted among the beneficiaries and applicants for grant funding to provide a basis for the assessment of the quality of services delivered by the EACEA to the beneficiaries and applicants. 


The survey will be representative for the different programmes managed by EACEA, and for the individual programme strands in terms of their characteristics (annual/multi-annual and small/large projects, type of beneficiaries). Furthermore, the evaluation approach and survey will ensure comparability of the results with the previously carried out satisfaction surveys, while also adding useful additional aspects given the Agency's current situation. To the extent possible, the approach will emphasise the lessons to be learned that are applicable to future programme implementation, compared to activities that are in the process of being phased out. 


As an open public consultation has been carried out for the evaluation of the underlying programmes, this covers issues which are relevant to the agency, and will therefore be complemented by targeted stakeholder consultations rather than a second full open public consultation.








Data collection and methodology 






The following data will provide a key input for the evaluation:


- EACEA's establishment act and act of delegation 


- Strategic plans and annual management plans of the parent DGs


- Annual activity reports of the parent DGs and programme statements of the relevant programmes


- Minutes of EACEA's management meetings


- Interim reports to the parent DGs (bi-annual)


- Agendas and minutes of the steering committee meetings 


- Human resources related data


Publicly available at the EACEA document register:



- The EACEA's Annual Work Programmes for 2015-17, as well as the Annual work programmes for the underlying programmes; 


- The EACEA's annual activity reports for 2015-2017 (when available);


- Manual of procedures of the Agency


- The final reports of the 1st, 2nd and 3rd evaluations of the EACEA;


- Recent evaluation reports on EAC programmes: Erasmus+, Creative Europe, EU Aid volunteers and Europe for Citizens
6
;


- Special Report No 13/2009 of the European Court of Auditors - Delegating implementing tasks to the Executive Agencies: a successful option?










(1)


 Guidelines SWD: 
https://ec.europa.eu/info/files/better-regulation-guidelines_en


Toolbox: 
https://ec.europa.eu/info/better-regulation-toolbox_en




(2)
 
                        When evaluating executive agencies under the Framework Regulation, the Commission's better regulation guidelines can be applied in a proportionate way. In particular, there is no need to assess the criterion of 'EU value added' and relevance as the Agencies carry out tasks which the Commission has transferred to them. The evaluations should therefore focus on the criteria of effectiveness, efficiency and coherence.



(3)
 
                        Cost-Benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the executive agencies - Final report for the Commission of 19/08/2013, GFK in cooperation with Technopolis.



(4)
 
                        Article 25(3) of Regulation 58/2003.



(5)
 
                        Article 25(4) of Regulation 58/2003.