EUROPEAN COMMISSION Brussels,30.1.2018 COM(2018) 48 final REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the implementation of the Common Monitoring and Evaluation System for the European Maritime and Fisheries Fund



Acronyms: AIR: Annual Implementation Report CFP: Common Fisheries Policy CI: Context Indicator CMES: Common Monitoring and Evaluation System CPR: Common Provision Regulation EFF: European Fisheries Fund EMFF: European Maritime and Fisheries Fund EP: Evaluation Plan FAME: Fisheries and Aquaculture Monitoring and Evaluation FI: Financial Indicator FIFG: Financial Instrument for Fisheries Guidance IMP: Integrated Maritime Policy MA: Managing Authority MS: Member State OI: Output Indicator OP: Operational Programme RI: Result Indicator

SFC: System for Fund Management in the European Union



Table of Contents 1Introduction 2The seven components of the Common Monitoring and Evaluation System 2.1EMFF Intervention Logic 2.2Common Indicators 2.3Cumulative Data on Operations (Infosys) 2.4Annual Implementation Report (AIR) 2.5Evaluation Plan 2.6EMFF OPs Evaluations 2.7Performance Framework and Review 3Findings and recommendations 3.1EMFF Intervention Logic 3.2Common Indicators 3.3Cumulative Data on Operations selected for Funding (Infosys) 3.4Annual Implementation Report (AIR) 3.5Evaluation Plan 3.6Evaluations 3.7Performance Framework

4Conclusion: are the objectives of the CMES going to be achieved?



1Introduction The EMFF Regulation requires the establishment of a Common Monitoring and Evaluation System (CMES) for the operations co-financed by the EMFF under shared management. The purpose of this document is to report to the European Parliament and the Council on the implementation of the CMES. This report is required every four years, the first time at the end of 2017 . It provides an opportunity to take stock on the strengths and weaknesses of the system, and draw lessons for the future. The objectives of the CMES are spelled out in Article 108 of the EMFF Regulation as follows: a)to demonstrate the progress and achievements of the CFP and the IMP, to consider the general impact and to assess the effectiveness, efficiency and relevance of EMFF operations; b)to contribute to better targeted support for the CFP and the IMP; c)to support a common learning process related to monitoring and evaluation; d)to provide robust, evidence-based evaluations of EMFF operations that feed into the decision-making process. To achieve these objectives, the Common Monitoring and Evaluation System has been defined in the Regulation, and consists of seven elements: 1)An intervention logic showing the interactions between priorities, specific objectives and measures. 2)A set of common indicators. 3)Cumulative data on operations selected for funding. 4)The Annual Implementation Report of the Operational Programme. 5)The Evaluation Plan. 6)The ex-ante and ex-post evaluations and all other evaluation activities linked to the EMFF programme. 7)The performance review. Each of these elements has been discussed with the Member States (MS) in expert group meetings and adopted in the framework of delegated and implementing regulations. This participative approach was continued after the adoption of the legislative framework and the involvement of the Managing Authorities (MAs) in the further development of the CMES helped to fine-tune the CMES components whilst limiting the associated administrative burden. This ensured the acceptance of the system but also supported the development of monitoring and evaluation capacity in the MAs and the Commission.

To reach its objectives, the CMES must be applied consistently across the MS, in order to produce comparable data that can be aggregated and evaluated at European Union (EU) level. To ensure consistency and further develop the CMES, the Commission established a technical assistance facility FAME (Fisheries and Aquaculture Monitoring and Evaluation) supported by an external contractor over the seven years of the programming period. 2The seven components of the Common Monitoring and Evaluation System 2.1EMFF Intervention Logic The intervention logic is supporting the introduction of a reinforced result-orientation in the ESIF for the 2014-2020 period. The EMFF intervention logic describes the links between different levels of the programme: Union priorities (UPs), Specific Objectives and measures. A table showing these links was distributed to the MS in an EMFF expert group meeting and this logic was integrated in the 'Operational Programme' module of SFC 2014. 2.2Common Indicators A set of common indicators has been established, aiming at representing the main characteristics of the fisheries and aquaculture sector, reflecting the goals of relevant EU policies (CFP, IMP) and providing a connection with the EU 2020 strategy. These also ensure consistent and comparable monitoring and reporting at the EU level. The set of common indicators used by the EMFF was extensively discussed with the MS in expert group meetings prior to their adoption. Four types of common indicators are used in the CMES, addressing different levels of objectives and serving different purposes: §Context Indicators (CIs) are linked to the overall objectives of the EMFF and establish the baseline at the beginning of the OP implementation. The EMFF uses 25 common CIs. §Output indicators (OIs) are the direct product of the implemented activities. In the case of the EMFF, they are mostly expressed as a number of operations. The EMFF CMES defines 28 OIs covering the 50 measures. §Result Indicators (RIs) measure the gross effect of the EMFF. The CMES comprises a set of 28 RIs, some of which are linked to only one EMFF Specific Objective, others to more than one. MS were free to add specific indicators if they wished. §Financial indicators (FI) are common to all ESIF and are set at UP level. A working paper has been published in which common indicators were defined and made more operational by the provision of judgment criteria and the identification of possible sources of information. These definitions were extensively discussed with the MS in two workshops held in March and September 2016. 2.3Cumulative Data on Operations (Infosys) The mid-term and ex-post evaluations of the European Fisheries Fund (EFF, 2007-2013) were hampered by the lack of robust monitoring data, making the collection of data challenging. A reporting system was established to address this shortcoming and provide the Commission with detailed information at the level of every single operation. The aim of this reporting is to create a database of information that can be used to follow the implementation of the OPs, carry out specific analysis required by policy-making and ensure that robust data is available for evaluation. Art. 97 of the EMFF Regulation requires that, at the end of March each year, MAs provide the Commission with cumulative data on operations selected for funding up to the end of the previous calendar year. Two Implementing Regulations provide the list and structure of the operation implementation data required on each operation, including key characteristics of the beneficiary and the operation. A working paper has been developed with support from FAME to help the MS prepare the Infosys reports. The first set of reports were analysed and a feed-back to the MS helped them identify the errors made.

The Infosys reporting describes the progress and achievements at operation level by capturing what is happening “on the ground”. It provides key information about the beneficiary (e.g. size of enterprise, gender) and the operation (type of activities, budget, number of fishermen concerned, total area concerned, etc.). In order to limit the administrative burden on MAs and on beneficiaries, most of the data requested by the Infosys is already stored in national databases set up to comply with the minimum data to be recorded and stored for each operation. 2.4Annual Implementation Report (AIR) The monitoring of OPs implementation involves the drawing up of an AIR by the MS, to be submitted by 31 May each year, covering the period up to the end of the previous calendar year (hence the same period as the Infosys report). The AIR provides an overview of the implementation of programmes, including in relation to important issues, such as support for small-scale coastal fisheries or climate change. It is used by the Commission and the MS to review achievements and make adjustments where necessary. It also contains indicative information on finances planned at measure level, which is not included in the OPs. The AIR is divided into three parts: Part A (to be reported every year between 2016 and 2023) provides for, inter alia: ·an overview of implementation at programme and UP levels; ·cumulative data on the 28 Result Indicators (RIs) at Specific Objective level; ·cumulative data relating to the 28 Output Indicators (OI) for each of the 50 EMFF measures; ·financial data on the OP implementation at measure level; ·description of issues affecting the performance of the programme; and ·activities in relation to the Evaluation Plan, including a synthesis of the evaluations finalised in the year. Under Part B (only included in the comprehensive reports due in 2017 and 2019), MS are to report additionally on horizontal aspects of the implementation (gender, sustainable development, etc.), and on the support for climate change objectives. Part C (only included in the comprehensive reports due in 2019) should contain further information in relation to the OP’s contribution to the achievement of the objectives of the Union strategy for smart, sustainable and inclusive growth. 2.5Evaluation Plan In order to strengthen the contribution of evaluations towards effective and efficient programmes, the Common Provisions Regulation (CPR) makes it compulsory for MAs to design Evaluation Plans at the beginning of the programming period. This is a new feature for the EMFF MAs, who had to reflect on their evaluation needs over the implementation period and the way these were going to be addressed. The Evaluation Plan includes seven sections: purpose, objectives, activities, governance provisions, data sources, indicative timeline and resources available. Short guidelines on these sections were provided as part of the EMFF OP template. Unlike the other ESI Funds, the evaluation plan is part of the EMFF OPs, as this allowed the Commission to assess these plans as part of the OP negotiation process.

2.6EMFF OPs Evaluations Evaluations are necessary to assess the effectiveness, efficiency and impact of the funding in order to improve the implementation of programmes and to determine their impact. The CPR lays down the general rules for evaluation in all the ESI funds, whilst the EMFF Regulation specifies that the MS have to conduct an ex-ante evaluation and further evaluations during the implementation period, whilst the Commission has to conduct an ex-post evaluation, to be completed by the end of 2024. The ex-ante evaluation is the only evaluation undertaken so far by the MS. Going beyond a simple photography of the situation before the start of the programme, the ex-ante evaluation is seen as a process, including regular interactions between the MA and the evaluators during the different development stages of the OP. This process was documented in a guidance document discussed with the MS in expert group meetings. 2.7Performance Framework and Review The Performance Framework is one of the tools used across all ESIF to foster result-orientation. In the EMFF, it is based on a set of Output and Financial indicators, for which milestones and targets are defined for each Union Priority. Performance Frameworks indicators are reported in the AIRs. The assessment of the milestones in 2019 will form the basis for the allocation of the performance reserve, whilst the achievement of targets will be assessed in 2024. The Commission issued a guidance document to provide MAs with further details on the way this new feature will be implemented. In the EMFF, the Performance Framework is composed of financial indicators (payment certified) which provide a clear and classical indication of performance of the implementation through the level of spending. To reflect the result-orientation given to the ESIF, financial indicators are complemented by output indicators. Both sets of indicators must show a good level of progress to allow the release of the Performance Reserve. 3Findings and recommendations This report has been drafted with the help of the FAME Support Unit, using feedback from the EMFF MAs collected through workshops held in March and September 2016, a specific stakeholder meeting with the participation of 21 Member States in September 2017, but also questions addressed by the MS to FAME and the Commission during the implementation of the Operational Programmes. The section below summarises the findings in relation to the implementation of each element of the CMES and, where relevant, proposes recommendations to improve the functioning of the CMES. 3.1EMFF Intervention Logic The intervention logic of the EMFF is well suited to the structure of the Fund (Union Priorities - Specific Objectives - Measures), but the late inclusion of the EMFF under the umbrella of the CPR required supplementary reporting at the level of Thematic Objectives common to all ESIF. This added an extra layer of complexity to the intervention logic and made reporting more difficult for the MS. Recommendations: ·The intervention logic is a structuring element in the CMES which should be maintained in the future, although it should be discussed more extensively with the MS in expert group meetings. ·It should be simpler and more result oriented, showing clearly which measure or action contributes to which objectives and how is this measured. 3.2Common Indicators The following issues appeared during this assessment of the functioning of the CMES: ·The working paper providing precise definition of the common indicators and information on the way these should be used was only made available fairly late during the programming process. This led to misunderstandings for certain MS and some OPs were prepared using different methods for setting targets on the results they were planning to achieve.

·Result indicators are linked to the specific objectives of the EMFF. The MS had the possibility to declare that certain result indicators under a given specific objective were “non-applicable” to a specific objective. This possibility was used extensively and not as an exception. However since every EMFF specific objective is covered by more than one common result indicator, it is still possible to measure the contribution of the OPs to the EMFF specific objective. ·The MS also had the possibility to use their own indicators instead of selecting them from the common list, leading to a situation where some results cannot be taken properly into account in reporting or aggregated at EU level. ·Some measures were poorly covered by common result indicator (e.g. measures related to innovation) and the time needed for an effect to materialise was not always properly taken into account (e.g. increase in net profit due to an innovation). This implies that some important effects of the EMFF might not be taken into account. ·Some indicators were expressed in ratios, their calculation is difficult at operational level, and their aggregation is not meaningful. Recommendations: ·Result indicators should always be expressed in absolute numbers in the monitoring including targets and baselines. Since EMFF result indicators rely on data from the EMFF beneficiaries before and after the operation, ratios, percentages and arithmetic sign ambiguities should be avoided since they create errors. For example net profit increase should be expressed as 'before' and 'after' values and not as an increase in percentage. ·Before starting the programming exercise, MS should receive methodological framing from the Commission on the definitions of the indicators and on the calculation methods to be used for setting the targets. ·In order to provide an overall picture of the use of the funds and to allow for meaningful aggregation at EU level, common indicators should cover all investments. However, the MS should have the flexibility to use supplementary indicators to be used at the national level. 3.3Cumulative Data on Operations selected for Funding (Infosys) The Infosys is a valuable tool for the MAs as it can be used as a "dashboard" for checking the progress of the implementation. It is useful for the Commission as it provides information on EMFF achievements long before the first RI values can be reported meaningfully. It will be an especially useful tool for the MS when undertaking evaluations from 2018 onwards as it will ensure that a basic but robust set of data is available for the evaluators. Issues with the Infosys include the following: ·In the Infosys reports, it is not possible to differentiate fields left intentionally void from negligent omissions, since the structure of Infosys did not allow indicating “non-applicable” as an answer to certain fields. ·A baseline is the value of an indicator before the start of an operation (e.g. number of existing Full Time Equivalents in a SME) and forms a reference point against which progress can be measured. Baselines are not available in the Infosys reports, despite the fact that they are always requested from applicants by the MAs to be able to assess the applications for funding and are a compulsory element of the national monitoring system. ·The Infosys requires those applying for funding to provide an estimation of the result they think they will achieve with their operation. This estimate is then verified by the MAs after completion of the project. Methodological support will be provided to MAs for this second step in order to ensure the robustness of the validation. Recommendations: ·The Infosys should be continued in the future as it provides very valuable information whilst generating only a limited administrative input from Member State administrations. ·Baselines should be included in the Infosys reports. ·Users should have the possibility to use "non-applicable" when appropriate adding an explanatory note.

3.4Annual Implementation Report (AIR) Unlike the Infosys, AIRs are submitted through SFC using a structured workflow and have to be accepted by the Commission. Different types of issues may arise during the assessment of these reports by the Commission, which can lead to substantial delay. In particular, there should not be any discrepancies between the disaggregated information provided in the Infosys and the aggregated values included in the AIR since both reports cover the same period. For all the ESI Funds, key information coming from the AIR is made available to the general public on the Open Data Portal provided by the Commission to ensure transparency on the use of the Funds. Financial data on the implementation of the EMFF is only available annually in the AIR, whilst the Commission receives information on the ERDF, ESF and EAFRD three times per year. The information published on the ODP is therefore more up-to-date for these Funds than for the EMFF. Recommendations: ·Guidance should be made available to help the MS with their reporting. ·Better complementarity should be ensured between the Infosys and the AIR. ·Reporting on financial implementation should be aligned between the Funds. 3.5Evaluation Plan The Evaluation Plans vary in length, detail and quality, and often present only the minimum information required. The MAs gave priority to other sections of the OP and the Evaluation Plans were sometimes overlooked and drafted rapidly. The budget earmarked for evaluations varies substantially, reflecting proportionality of treatment: MS with numerous regions have a significantly higher budget for evaluation purposes than MS with only a small EMFF OP and smaller or no regional divisions. Recommendations: ·The Evaluation Plan included in the OP should only emphasise the governance of the evaluation process, provide basic time planning and indicative resource allocation. ·To maintain flexibility and avoid numerous OP modifications, more detailed planning should be left for a later stage, using updated plans adopted by the Monitoring Committee as it is the case for the other ESIF. 3.6Evaluations In April 2017, the majority of the MS had yet to start evaluations, as they considered that the level of implementation was too low to provide a basis for substantial evaluation results. It is therefore too early to draw conclusions on this component of the CMES.

The process of the ex-ante evaluation is presented in the OPs but in a very synthetic way which might need to be improved. The ex-ante evaluation teams have generally not been involved in the negotiation phase of the OPs or their finalisation and were not able to provide an assessment of the final version of the OP. A specific issue with evaluation of the EMFF is the difficulty to find evaluators with the right expertise in both fisheries and evaluation methodologies: this can only be found in a team of individuals with complementary skills. Guidance is also needed to support the MS when undertaking effectiveness or impact evaluations. This will be especially important when the Commission will undertake the ex-post evaluation which should be based on a set of robust national evaluations. Recommendations: ·Guidance on the ex-ante evaluation should be more detailed and include a definition of the role of the ex-ante evaluators in the finalisation of the OP. ·MS should seek to combine the different types of expertise required in an ad-hoc team of evaluators ·Methodological framing on evaluation should be made available rapidly. 3.7Performance Framework Output indicators in the Performance Framework have to be wisely defined in order to avoid misunderstandings: output indicators are mainly the number of operations selected under certain measures, a number which can be influenced by external factors. The calculation method used for the financial indicators is defined in the CPR, but raised many questions from the MS. This is probably reflecting the different terminology and reporting methods applied by the different Funds. Recommendations: ·The financial indicators should be similar and comparable throughout all ESIF. ·The use of output indicators should be better assessed ·It might be useful to complement the Performance Framework by explanations. 4Conclusion: are the objectives of the CMES going to be achieved? The present stock-taking report indicates that progress is being made towards the four objectives of the CMES but that further work is still required to achieve them fully.