REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

on the evaluation of the Union's finances based on the results achieved

1. Introduction

Article 318 TFEU provides that the Commission shall submit to the European Parliament and the Council an evaluation report on the Union's finances based on the results achieved, in particular in relation to the indications given by the European Parliament and the Council pursuant to Article 319. This report is the second report and relates to the financial year 2011.[1]

In the context of the budgetary discharge procedure for the financial year 2010, the European Parliament encouraged the Commission "to review its systems for evaluating the effectiveness of expenditure programmes to assess whether they are adding value, delivering value for money and achieving the objectives for which they were established".[2] The European Parliament sought the opinion of the Court of Auditors which suggested that the scope, purpose and content of the report be reviewed and this should be the starting point for a discussion between the Commission, the Council and the Parliament on how the report can be made useful to the discharge authority.[3]

Based on the indications given by the Parliament and taking into account the Court's opinion, the Commission is examining how to design and structure this report to ensure maximum value to the discharge authority. This includes a method to report consistently and reliably on performance on a yearly basis. For the near term, the Report will need to rely on performance information drawn from the current monitoring and evaluation frameworks under the Multi-annual Financial Framework (MFF) and the Commission's annual reporting instruments.[4] As indicated in last year's report,[5] the preparation of the next MFF provides the opportunity to make adjustments to improve the Report in the future, particularly as concerns the production of improved performance information.

Various issues of purpose, scope and timing are being taken into account in designing the future reports:

· meaningful annual assessment of the effectiveness and efficiency of expenditure to the budgetary authority. The Report should provide an indication if the financial programmes are on track or if there needs to be adjustment;

· great variety in the size and content of the programmes To be manageable, this requires agreement on a common framework– a set of key objectives and main indicators and a framework for monitoring and reporting on results relative to programme objectives and criteria such as effectiveness, efficiency and EU added-value. This framework should be coherent with and drawn from both the Commission's annual reporting instruments and its evaluation work;

· timely reporting on evaluations, taking into account the Parliament's need to have the report available in the context of the annual budgetary discharge procedure.

Article 318 links this evaluation Report to the Discharge process. There is clearly a need to further clarify with the discharge authority how to draft the evaluation report in the light of the discharge resolutions . On timing, during the early years of financial programmes to which the discharge would refer, the main focus is on the initiation of the actions being financed. Conclusions on impacts would only be possible at a later stage. This is because evaluations usually refer to funds committed at least three to five years previously. Meeting the request of the Court of Auditors to receive the Commission's next report on the n-2 budget year significantly earlier in the year in order to take it into account in their annual discharge report will further shorten the time available to report on results.

The link between the preparation and timing of the production of this report and the Annual Activity Reports also needs to be reviewed. Finding the right balance between early reporting and useful assessments of performance will be important.

The conceptual work has started and an agreed framework should underlie the future Evaluation Reports.

Taking into account the indications given by the Parliament, calling for a more comprehensive approach,[6] and the Court of Auditors[7] this report provides performance-related information which became available in 2011 for funding under all of the main budget headings.

The report identifies:

· key issues of EU value added, effectiveness and efficiency, indicating when relevant, the limitations inherent to the interim character of many of the available evaluation reports;

· information contributing to modifications in the management of the programmes which could contribute to improve their final impacts;

· examples of evaluation results having contributed to the design of the Commission proposals for the MFF 2014 – 2020. These examples show how evaluations have contributed to simplification of the programmes, better monitoring and evaluation frameworks and how they have influenced policy making; a full set of evaluation summaries for each budget heading[8] and links to evaluation information from the Annual Activity Reports (AAR)[9] and Activity Statements[10] are provided in the Staff Working Paper in annex.

2. Overview of results achieved

This Report uses the evaluations completed in 2011 (118 in total) as the main source of performance related information. It summarises evaluation results on the financial programmes for each budget heading, highlighting EU value added, focus and coherence, effectiveness and efficiency as well as organisational issues. The Report indicates if evaluations have made recommendations for improvements. This section follows the structure of the 2007 – 2013 Financial Framework.

Heading 1a - Competitiveness for Growth and Employment

Enterprise

The Competitiveness and Innovation Programme (CIP) includes the Entrepreneurship and Innovation Programme, the ICT Policy Support Programme and the Intelligent Energy Europe Programme. [11]

The interim evaluation[12] based on all three components, examined impacts and EU added-value. It concludes that the overall programme and its constituent elements are all performing well and on track to meet objectives. [13] The programme was assessed as highly relevant, appropriately structured, effective and efficient. The evaluation identified room for greater synergies between the component programmes and a better strategic steer. In terms of results, the CIP has become a major vehicle for promoting innovation with over 186,000 jobs created or maintained largely due to its interventions. It supported 134 eco-innovation projects in recycling, green business, food and drinks sector and the building sector. The Growth and Innovative SME Facility (GIF) and SMEG loan and micro credit facility were confirmed as relevant to the needs of small and medium-sized businesses (SMEs) and, in terms of EU added-value, fulfil a demand for finance which otherwise would not have been met. Between 2007 and 2011, the financial instruments of the programme have assisted more than 155,000 companies financing EUR 11.4 billion under guarantees and investment volumes up to EUR 2.2 billion under venture capital. More than 2 million SMEs each year have used the Enterprise Europe Network, half of them to access new markets or new product development.

The interim evaluation[14] of the Erasmus for Young Entrepreneurs Programme[15] identified the significant job creation potential of the programme. Half of the new entrepreneurs who took part in the programme commented that they would be likely to hire a new employee as a consequence of the placement despite the risk of those persons becoming competitors. Costs of EUR 7,500 to EUR 8,500 per job created, suggest cost effectiveness. Participation is increasing, with an improvement in processing applications resulting in a higher number of exchanges per month.

Economic and Financial Affairs

The ex-post evaluation of the Euratom Loan Facility (ELF)[16] showed that the aims of the ELF remain valid and aligned with its energy security and supply objectives. The ELF co-financed 21% of total investment in new construction over the period 1977-2003 and created 6000 skilled jobs. Without this investment, the EU would have had to import an additional 10m tons of energy annually. The EFL directly contributed to safety enhancements in Bulgaria and Romania, helping to bring their nuclear installations in line with internationally recognised standards. Euratom lending stimulated wider reform, including creation of decommissioning funds, reform of electricity tariffs etc. The financial performance of the EFL was good with all loans being fully repaid and no recourse to the EU budget guarantee.

Mobility and Transport

The Mid-term evaluation[17] of the The TEN-T programme[18] showed that while the programme played an important role in structuring the EU transport network and meeting mobility needs, progress in implementation was behind schedule with only a few priority projects completed and key parts - such as cross-border sections - missing. It indicated that the TEN-T network is an assembly of largely national sections, often poorly interlinked, rather than a full interoperable network. Most Priority Projects focused on rail,[19] without achieving a complete and coherent network linking all modes of transport. In spite of the focus on rail, a Single European Railway Area still does not exist, with continuing bottlenecks and significant obstacles to interoperability. The Priority Projects were not delivering the expected effects.

Based on this evaluation, the Commission proposed new Guidelines for TEN-T projects for the period 2014-2020 in the context of the Connecting Europe Facility. These draft guidelines set out standards to ensure efficiency, interoperability and better project planning. The aim is to achieve by 2030 of a core network of European infrastructure and by 2050 a comprehensive network. The focus until 2020 will be on those elements which provide the highest European added-value: cross border missing links, key bottlenecks and multi-modal modes.

The Evaluation[20] of the Marco Polo Programme 2003-2010[21] showed that while the programme delivers environmental benefits quantified in hundreds of million EUR[22], the modal shift targets expressed in tonne-kilometres of freight shifted from road to short sea shipping, rail and inland waterway transport have not been fully achieved. Being a market-driven programme, Marco Polo was particularly sensitive to the market situation and the economic crisis. It has a unique design where funding is largely performance related. However, at the same time, it is considered by the market as a complex instrument. And 42 per cent of beneficiaries stated that their projects would definitely have gone ahead without Marco Polo funds. However, it was also concluded that it is too early fully to assess the results of Marco Polo II as many projects are still being implemented.

As a result of these findings taking into account the results delivered by the programme and the evolution of the transport policy, the Commission has proposed that the initiative be continued in the framework of the new Connecting Europe Facility and the new TEN-T programme.

Information Society and Media

The interim evaluation[23] of the Safer Internet Programme[24] concluded that the programme adds value and responds well to changing societal issues, including aspects of social media and cyber-bullying. The programme supported projects that would not otherwise have been funded. It influenced national and international activities, with the research and results widely used and quoted. The programme could benefit from a longer term funding perspective and a more strategic vision. In its proposal for the programme for the next MFF (The Connecting Europe Facility) a more coherent and transparent approach to EU funding is provided which will offer certainty and should thus attract more private sector financing.

The final evaluation[25] of the eContent Plus Programme[26] found that the programme has contributed to making digital content in Europe more accessible, usable and exploitable, facilitating the creation and diffusion of information in areas of public interest. Concerning the importance of this contribution, given increasing demand for quality digital content in Europe from citizens, students, researchers, SMEs and other businesses, the evaluation recommended continued support at European level to increase the availability of European digital content. It was suggested to target funding at areas where progress is likely to be slow and access to content is limited by language and cultural barriers to maximise EU added-value.

Energy

The Final Evaluation[27] of the Intelligent Energy-Europe (IEE) II programme[28] found that the programme was well aligned with policy priorities and contributing to EU objectives by promoting energy efficiency and the utilisation of renewable energy. It has focused on the development of best practices and cross-border activities.

The evaluation found that the programme addressed the needs and problems related to sustainable energy but also that non-technological barriers continue to limit the attainment of EU energy goals. To better address the financial barriers, hindering the uptake of sustainable energy, the evaluation recommended increased funding of activities promoting innovative techniques, processes and products. This finding was corroborated by the ex-ante evaluation of the successor to the IEE II programme, which found that the most cost-effective policy option would be to continue the programme with an increased budget and to give more emphasis to actions supporting policy implementation, capacity-building and mobilisation of investment. The impact of the IEE II Programme could also be higher if more actions were targeted at small and medium players in the energy sector and if it was better linked to the Structural Funds. This issue is addressed by the Commission's proposal for the next programme. .Finally the evaluation showed that the efforts made by the Executive Agency for Competitiveness and Innovation (EACI) to simplify the management process might increase the effectiveness of the projects, and that the EACI was overall perceived as efficient.

The mid-term evaluation[29] of the European Energy Programme for Recovery (EERP)[30] found that the funding had not always been used as rapidly as envisaged. The overall objectives and the resources of the programme were relevant to economic recovery, and energy policy, with short-term employment generation and strong potential for long-term employment and competitiveness benefits. Some completed projects were already contributing to security of supply, competitiveness, greenhouse gas reduction and completion of the single market. In some cases, projects took longer to get off the ground than had been planned partially due to financing and permitting difficulties inherent in large infrastructure projects.

The Commission has responded to the evaluation findings in its proposal for energy infrastructure guidelines which aims to introduce major improvements in the speed of implementation of the programme. The proposal contains provisions to accelerate permit granting procedures by the creation of single national authorities to manage this part of the process, establishing a three year time limit for the permit granting decision and increasing transparency and public participation.

Research

Under the Seventh Framework Programme (FP7) Framework Programme on Research[31], in 2011, 40% of participants in retained proposals came from the Higher and Secondary Education sector, so mainly from universities, 26% from industry, and 23% from research organisations. SMEs received 16.3%[32] of the funding for the Cooperation Specific Programme compared to the initial aim of at least 15% SME funding. The strong international character of the FP is illustrated by the 169 countries involved. Up to February 2012, over 14,000 grant agreements have been signed and over 1,000 projects completed. Based on the final reports of the completed projects, on average each project produced eight publications, four being 'open access'[33]. On average, FP7 projects generated twenty-two direct full-time equivalent (FTE) jobs[34].

Evaluations conducted in 2011[35] underline that the Framework Programmes have succeeded in involving Europe's best researchers and institutes, picking up emerging fields of science or maintaining EU competitiveness in setting research agendas. The evaluation studies[36] show that the FP7 significantly contributes to helping European-wide partnerships to grow and strengthen. Some of the evaluation studies provide valuable evidence about the contribution of the Framework Programmes to developing innovative products, processes and services which have an impact on competitiveness and job creation.

A study into the 'Longer term impacts of the Framework Programmes' showed the diversity of impacts according to the different fields of research analysed, allowing, for example, the EU to become competitive with the other world R&D leaders in the area of Quantum Information Processing and Computing or helping the European research community to become a world leader in the area of stratospheric ozone research. The study further highlights the importance of more incidental impacts such as community-building, the creation of new disciplines and staff exchange between academia and industry.

It was also concluded that Framework Programme evaluations improved over time, but still suffer from the predominance of input indicators and a scarcity of output and impact indicators. The Commission's recent proposal for Horizon 2020 provides a more developed analytical framework. It presents a coherent intervention logic for the programme as a whole and its different components, including a hierarchy of programme objectives and a list of key indicators to measure the outcomes and impact of Horizon 2020. In addition, the proposal has been designed to introduce simplifications to ease the administrative burden for participants, streamline the applicable rules and procedures, ensure consistency between instruments and apply a new risk/trust balance. Particular attention is being paid to ensuring the participation of SMEs.

Education and Culture

The Interim Evaluation 2007-2013 of the Lifelong Learning Programme (LLP)[37] covered the four sectoral programmes[38] and the Transversal and the Jean Monnet programmes, supporting transnational mobility, partnerships and other cooperation projects in all educational sectors covered by the LLP.

It confirmed that the more integrated approach increased the efficiency and effectiveness of the programmes. Important synergies were exploited, particularly in administration and promotion actions. However, the programme was found to be highly complex with around 70 actions and too many objectives, often with missing links between different levels.

The LLP has progressed successfully towards its objectives. Up to 96-100% of the earmarked funds were used in the initial years of implementation. Institutional beneficiaries pointed to the benefits of improved content and practice in education and training. The evaluation noted significant European added-value: enhanced policy cooperation and interchange between the Participating countries and an increased European dimension. The evaluation identified nonetheless the risk that some of the programme's quantified targets (such as on mobility and the Grundtvig) may not be reached due to insufficient funding and that results needed to be better mainstreamed into policy to increase the effectiveness of the programme.

Based on these findings, the Commission proposals for 2014-2020 include further integration of the LLP with the Youth in Action and various international co-operation programmes. This is intended to create one simplified and streamlined system, helping to further develop a coherent approach to lifelong learning, thereby improving efficiency and cost-effectiveness. It is also planned to exploit similarities between the previously different programmes to the full and ensure that activities in certain areas gain the critical mass necessary to provide long lasting impact. By improving flexibility and increasing the incentives available, the budget allocation between actions, beneficiaries and countries should better reflect performance and potential impact.

The evaluation of the European Institute of Innovation and Technology [39] covered the creation of the Institute, its entry into operation and initial actions up to 2010. It concluded that EIT implementation remained on track and was even ahead of schedule for the selection of the first Knowledge and Innovation Communities (KICs). The budget was considered to be sufficient while leverage rates within the KICs had been marginally higher than anticipated, with the KICs drawing on additional sources of finance including partner’s own resources (49%), national and regional funds (21%), and other EU sources (7%). It was concluded that the EIT offers strong EU added-value.

Heading 1b - Cohesion for Growth and Employment

Employment and Social Affairs

The Evaluation[40] of the European Social Fund support for gender equality found that Member State implementation started more slowly for gender-equality specific actions than for other parts of the ESF Operational Programmes (OPs). For some Member States, this was the result of the economic crisis which shifted attention and resources away from the gender-equality objective towards labour market policies and reducing unemployment. Nevertheless, all Member States take account of gender equality although to varying degrees within their ESF OPs. In some cases the ESF provides the only source of funding for the actions concerned. The evaluation found that while ESF interventions focused on essential issues, there is a tendency for Member States to focus on the supply-side— i.e. women's labour market participation and their capacities to compete in the labour market’— more than on crucial aspects and causes of gender inequality (e.g. educational segregation, work organisation, cultural and social stereotypes). The evaluation concluded that the programmes provided added-value in capacity-building and supported the uptake of gender equality on the policy agenda of most Member States .

The evaluation of European Social Fund Support for Enhancing Access to the Labour Market and the Social Inclusion of Migrants and Ethnic Minorities (2000-2006) confirmed that the ESF was making a positive contribution in helping people with a migrant and minority background to overcome barriers to labour market access, to improve language skills and strengthen basic skills for employability. Regarding systems and structures, the capacity of public institutions to promote the integration of people with a minority background had been strengthened as a result.

In this policy area the ESF filled an important funding gap. It had brought additional targeted funding support that would not otherwise have been available. The ESF, and the EQUAL[41] programme in particular, supported wider target groups than those normally covered by national budgets. Participation by migrants and minorities was significantly higher in 2007-2013 than in 2000-2006 (1.19 m people, or approximately 8.7% of all ESF beneficiaries.)

The Commission proposal for the next generation of Structural Funds should contribute to simplification of procedures and improvement of the synergies with the European Regional and Development Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the future European Maritime and Fisheries fund.

The ex-ante evaluation of the future PROGRESS programme (the "European Union Programme Social Change and Innovation") highlighted the need to increase the coherence of EU action in the employment and social areas by bringing together and developing the combined implementation of the PROGRESS and EURES programmes and the European Progress Microfinance facility. In addition, the new Programme was found to provide an opportunity to simplify implementation through common provisions covering, inter alia, common general objectives, common typology of actions and rationalisation of reporting and evaluation.

Regional Policy

Regional Policy is delivered through shared management with Member States and regions responsible for implementing programmes combining EU and national and regional resources aiming to achieve economic, social and territorial cohesion. Evaluation is also a shared responsibility. Evaluation evidence is used whenever it becomes available and is most likely to feed into changes in policy when it accumulates across a number of evaluations. Evaluation evidence from the 2000-2006 period (programmes were in operation until 2009) informed the proposals for the future policy, 2014-2020, in the 5th Cohesion Report in 2010 as well as the MFF proposals in 2011. Evidence obtained in previous years, as well as that gathered in 2011 on the previous and current programming period has also fed into the European Commission's position papers on future Cohesion Policy in each Member State and will be discussed in the negotiations on the new programmes – a process which will continue until the end of 2013.

In 2011 the ex post evaluations of the 2000 – 2006 programming period were still being completed. Evaluation findings concerning the on-going implementation of the 2007-2013 period were also obtained. The evaluation results confirm the direction of the Commission's proposals for the 2014-2020 programming period. Detailed policy insights will be discussed with Member States so that they can be reflected in the new programmes which will be developed in the coming year.

As part of the ex-post evaluation of the Cohesion Fund 2000 – 2006, three work packages were completed in 2011 with the final synthesis to be available in 2012. Two studies,[42] on transport and environment, used ex post cost benefit analysis to evaluate impact. Although the evaluation concluded that it was too early to fully assess impacts, the method was found to be appropriate and will be used further in the future. All the transport projects delivered value for money and the Cohesion Fund contribution was necessary to unlock their economic benefits. The analysis of the environmental projects showed that many environment infrastructure projects were carried out to meet the legal requirements. Costs were higher than the benefits that could be quantified but the projects generated important effects in terms of environmental awareness and institutional learning. Important policy lessons from these evaluations and from those completed in earlier years on transport and environment projects co-financed by the ERDF are being discussed with the Member States and will feed into the design of the future policy.

Evaluations of Regional Policy enterprise and innovation support[43] provided further evidence to confirm the correctness of the Commission's proposal to exclude large enterprises from general ERDF support schemes in the future. The impact of enterprise support was found to be greatest when targeted on smaller firms. The study of innovation support in Germany[44] showed that the R&D grants impacted on a wide range of measures, from innovation in existing products and processes to the development of new ones. More research is needed to explore if smaller grants might be more effective than big grants over time and to further explore best use of loans rather than grants.

The Expert Evaluation Network, which reports each year on regional policy performance in the current programming period, concluded that delays in implementing programmes were increasing the risk of priority being given to absorption rather than effectiveness. Nevertheless, there are signs of accelerated implementation, increased evidence of achievements and overall improvement in the quality of evaluations undertaken by the Member States. Policy Papers on Renewable Energy and Energy Efficiency, undertaken by the network, showed that the EU support is marginal relative to national funding, raising the question of added-value. Investments in the sector need careful analysis on a case by case basis to ensure good value for money.

Overall, the evidence generated from evaluations in 2011 confirmed the necessity to strengthen the results-focus of current and future policy and the necessity for Member States to undertake more evaluations of the impacts of policy. These elements are at the heart of the Commission's proposals for the future.

Heading 2 - Preservation and Management of Natural Resources

Agriculture and Rural Development

The Evaluation of income effects of direct support[45] confirmed that direct payments have contributed to enhancing the income of farmers and played an important role in generating farm income. This was particularly true for grazing livestock farms (beef, sheep and goats), field crops, mixed farms and dairy farms. Direct payments have contributed to reducing the gap between the average farm income per labour unit of small and large farms. The efficiency of direct payments providing income support to farmers was quite high, although with notable differences across regions.

The Evaluation of CAP measures applied to the sugar sector[46] indicated that the 2006 sugar reform significantly accelerated the restructuring process of the sector. Improvements in yields accelerated as did the concentration of production in larger farms. Although the overall objectives of the reform[47] have been achieved, the management of the system by quota and reference price remain complex.

The evaluation of CAP reform measures applied to the dairy sector,[48] showed dairy policy contributed to a falling structural excess of supply from 2004 onwards. The total cost of the dairy support policy declined whilst the market balance. Producer income levels were maintained during the time covered by the evaluation data and the interests of producers in production flexibility and expansion were satisfied. The reform entailed more administrative burden and exposure to price risk. The evaluation recommended the introduction of more flexible instruments to respond to world market developments.

The evaluation of CAP reform measures applied to the sheep and goat sector showed that the reform facilitated an increase in productivity, although the overall competitiveness of the sector was not improved. Payments granted in Italy and Greece in the framework of assistance to sectors with special problems were low and had little impact. In the sheep meat sector, support offered through the rural development programmes is considered critical to maintaining the sector. In addition, coupled support provides environmental and social benefits in some regions and is necessary for maintaining production in sensitive regions.

The evaluation of promotion and information actions for agricultural products[49] reports that multi-country activities or campaigns in third countries would be less likely without EU funding. While stakeholders are enthusiastic about information and promotion programmes which have contributed to improving the image and raising awareness of the quality and the specific production methods of EU agricultural products, there is a need to better identify direct economic impact of these programmes.

The evaluation of the exceptional market support measures in the poultry and egg sector indicated that the common EU approach to tackle strong market disturbances (such as plummeting demand and prices caused by a lack of consumer confidence due to Avian Flu) provided an adequate response. Although the measures could not stabilize the market in the short term due to their long preparation time, they ensured that a substantial part of the severe income losses could be compensated within a reasonable time. The evaluation concluded that the Commission should develop instruments to deal with such crises in consumer confidence that can be used swiftly to stabilize the market.

Evaluation results have fed into the Commission's October 2011 CAP reform proposals, for example in relation to the proposals for further decoupling and in the design of the rural development policy framework. In future, monitoring and evaluation is planned to be brought under a common framework to measure the performance of the CAP as a whole within Europe 2020. To this end, a process has started to develop a common set of indicators linked to the policy objectives.

Maritime Affairs and Fisheries

The Interim evaluation (study) of the European Fisheries Fund (2007-2013)[50] concluded that the management and control systems for the Fund had met with start-up problems, but are now working satisfactorily. The shift towards one single operational programme, while beneficial at EU level, has increased the management costs for most Member States. Better audit and control methods have lowered financial risks, but the administrative burden remains too high for small Member States particularly the new Member States. Monitoring and evaluation frameworks focused on output rather than impacts and lacked a common definition of the units to be measured. Furthermore, indicators were not used for reporting nor verified systematically.

These findings were taken into account in the proposal for the new Fund for the EU maritime and fisheries policies (2014-2020). Main changes include a major simplification and alignment of different administrative rules and procedures on financial decisions, reporting, monitoring and evaluation; stronger links with the key objectives of the Common Fisheries Policy (CFP) reform through the introduction of conditionality rules, closer co-ordination with other funds as well as an improved monitoring and evaluation system based on common indicators across the Member States.

Heading 3a - Freedom, Security and Justice

Home

The Report of the Commission and the Mid-Term Evaluation of the External Borders Fund[51] found that, despite start-up difficulties, the Fund was delivering in line with its objectives to channel EU aid to boost investments at borders and in consulates in the interest of the Schengen area as a whole. Investments in national communication systems for border control and in the Schengen Information System II improved the safety of EU external borders. In addition, investments for the set–up of the Visa Information System improved the visa handling process Despite the overall contribution of the Fund to a better border management and use of new technologies, Member states have questioned the effectiveness and efficiency of some of the projects.

The Report of the Commission and the Mid-Term Evaluation of the European Return Fund (2008-2013) showed that the Fund has contributed to developing and consolidating return management practices in many Member States. For example, it contributed to: shortening the length of stay in reception and/or detention centres and increased numbers of third-country nationals opting for voluntary return. As a result, this population benefited from better treatment than they would have encountered had they been forcibly removed.

The Report of the Commission and the Mid-Term Evaluation of the European Fund for the Integration of third-country nationals (2008-2013) found that, in spite of start-up difficulties, the majority of Member States expected to complete the planned actions and meet the objectives set. By targeting specific measures or groups not normally within the scope of mainstream funding instruments, the Fund demonstrated genuine added value. In several Member States, the Fund supported the design of a comprehensive policy framework and provided resources for its implementation. However, the administrative workload associated with the programmes should be reduced.

The Mid-Term Evaluation of the Framework Programme Security and Safeguarding Liberties (2007-2013)[52] concluded that the Programme is effective in its pan-European dimension and in supporting a wide range of activities. Rather heavy administrative application procedures may have had a negative impact on project implementation. The added value of the EU was nevertheless assessed as high and the overall implementation of the programme was considered reasonably successful.For the years 2014-2020, the Commission has proposed that the Home Affairs funds and programmes should be reduced from six to two: The Asylum and Migration Fund and The Internal Security Fund. Other evaluation results were taken into account in these Commission proposals.

Justice

The interim evaluation[53] of the Civil Justice Programme[54] showed that the objectives were highly relevant in supporting European justice policy. The activities financed help civil society organisations and Member States to make an effective contribution to the formulation and smooth implementation of EU law. In addition, the programme was considered to be efficiently managed, despite limited human resources. The interim evaluation suggested a merge of the Civil Justice and the Criminal Justice Programme in the new Multiannual Financial Framework.

The interim evaluation of the Criminal Justice Programme [55]confirmed that the supported actions tackle important problems and complement national measures. However, it also identified the need to increase the effectiveness of funding through better priority setting, targeting increased European added value, improving publicity and simplifying implementation.

The interim evaluation[56] of the Daphne III Programme[57] concluded that the programme is highly relevant to the problems it was designed to address, although links between the programme and policy development could be strengthened. There is room for improvement to promote EU-wide partnerships and to ensure a balanced geographic spread of lead organisations. The financial resources to implement actions were assessed as allowing high quality projects to be funded, while the ratio of success in project selection was ensuring significant competition among projects. The dissemination of the results could be improved. The interim evaluation of the Fundamental Rights and Citizenship Programme[58] indicated that, while it is too early to draw conclusions on impacts due to the limited number of finalised projects, the type of projects funded were all well-suited to the objectives and priorities of the programme. A wider European dimension balanced participation of organisations from EU Member States and dissemination of results are areas for improvement.

The interim evaluation[59] of the Drug Prevention and Information Programme (DPIP)[60] could not draw conclusions on effectiveness as few projects were completed. Difficulties were being encountered involving vulnerable groups and in ensuring long-term value added of projects. The need for additional financial resources was identified as was the need to strengthen the synergies between the various financial instruments supporting the objectives of the EU Drugs Strategy. The evaluation recommended improved cooperation with other programmes, the Public Health Programme in particular, to avoid duplication and to allow potential beneficiaries to target their applications more efficiently.

Heading 3b - Citizenship

Communication

The interim evaluation of Euranet[61] showed that the existence of the network has led to an increased coverage of EU affairs and enriched editorial value. Increased production and broadcasting of EU news and programmes is unlikely to be sustainable without this funding. The evaluation on the other hand pointed to issues, such as the uneven extent of coverage of EU affairs as a share of overall production, the uneven representation of different language groups in the participating partners and the slow development of the network, especially in terms of expanding language coverage. In the follow-up on these issues seven new radio stations have added four languages across five countries and a stronger production focus on EU issues has been introduced. The design of the next phase of the project has been adapted according to the main conclusions of the evaluation.

Health and Consumer Protection

The ex-post evaluation of the Public Health programme for 2003-2008[62] concluded that even if, as the first programme in the field of public health at EU level, it may have been desirable to fund a broad spectrum of activities, a more targeted effort in selected areas was of crucial importance for the next generation of the programme. The evaluation called for a strengthened focus on European added-value, both through choice of priority areas in annual work programmes and in application decisions.

Similarly the mid-term evaluation of the successor programme[63] concluded that it should be much more focused on priorities and concentrate financial support to bring the largest EU added-value. In terms of organisation and management, it was confirmed that a process is in place to determine priorities in the Annual Work Programmes and to ensure their alignment with the overall objectives of the PHP. However, this process is not considered as particularly clear or consistent. PHP actions generally correspond to the objectives of the programme, but it is still too early to assess results and impacts. The majority of actions are seen to have had EU wide effects and most actions would not have taken place, or would have been undertaken with a less ambitious scope, in the absence of PHP funding. The case study assessment showed that EU added-value appeared to feature most prominently in the areas of “promotion of best practices” and “networking”. On efficiency, economies of scale were expected in most actions, although the ways to measure this have not been clearly set out. The outsourcing of management to the EAHC was identified as having resulted in a significant improvement in delivery.

The new Programme on Health aims to focus on fewer actions, of proven EU added-value, that deliver concrete results, and respond to identified needs or gaps. The new programme also seeks to improve the way Member States cooperate in the area of health and to provide leverage for reform of national health policies.

Taxation and Customs Union

The Mid-term Evaluation of the Fiscalis Programme[64] concluded that the programme had made a positive contribution to improving the functioning of taxation systems in the internal market and is relevant to the challenges faced by national authorities. It concluded that the programme offers high EU added-value: many activities necessary to achieve progress in taxation cooperation would not have happened at all, or would have only happened much later or at a higher cost and with less optimal results, if the cooperation framework of the programme had not existed. The overall conclusion was that Fiscalis offers good value for money.

The evaluation nonetheless indicated that the programme's effectiveness was more limited in terms of increasing understanding of EU law and in the fight against fraud. It also recommended that a results-based monitoring and evaluation system be set-up. Results of the programme should be better disseminated and programme activities could benefit from the involvement of a larger community of stakeholders.

The Mid-term Evaluation of the Customs Programme highlighted the programme's effective contribution to the functioning of the internal market. The programme demonstrated EU added-value supporting activities that would not have taken place, or would have been significantly more resource or time-intensive, without EU coordination and financial support. It facilitated increased co-operation and coordination between national customs administrations, thus enhancing exchange of information and good practices.

The efficiency of the Joint actions under the Programme were rated as high while there has been a much lower level of co-operation with other public and private bodies to improve security and fight against fraud. The contribution to trade facilitation with third countries was not highly rated. The evaluation pointed out that further efficiency gains could have been achieved by better disseminating results and broadening the involvement of and support to the candidate and potential candidate countries.

Education and Culture

The Interim Evaluation [65] of the Youth in Action Programme (YiA)[66] concluded that the programme is successfully achieving its objectives thus far, actively targeting young people with fewer opportunities. YiA contributes to the achievement of EU Youth policy objectives and has fostered continued youth participation in events after YiA, work for NGOs and increased mobility. The activities that YiA supports would likely not find funding elsewhere. YiA has a high absorption rate. The management costs appeared relatively high, but justified by the nature of the Programme. The allocation of the budget was appropriate. Nevertheless, youth organisations and National Agencies both continued to experience a relatively high administrative burden even though management tools had been considerably improved. There was high participant satisfaction, comparing positively with youth programmes organised in Member States, and the implementation structures were considered efficient.

Heading 4 - EU as a Global Player

Development and External relations

The two pilot evaluations on budget support in Tunisia[67] and Mali[68] concluded that Budget Support is an effective tool in countries where the government is committed to robust development policies. In Tunisia, Budget Support contributed to significant economic growth, private sector development but should have better addressed poverty reduction. In Mali, it contributed to some strong positive outcomes in education and health. Budget Support is complementary to other forms of aid, especially for capacity-building. It cannot determine major policy changes but can provide additional support for policy implementation. Its impact is greater when linked to wider political and economic partnerships.

The thematic evaluation[69] EU support for employment and social inclusion in partner countries (ESI)[70] concluded that, although ESI is a key priority in development policy, there was little mainstreaming into development programmes and there was a lack of ESI-related indicators in programme design. Few interventions outside vocational education and training focused explicitly on employment creation. The new instrument for development cooperation (2014-20) responds to the need to target more human development through its Global Public Goods and Challenges thematic programme, including health, education, gender equality, employment, skills, social protection and social inclusion as well as economic development-related aspects such as growth, jobs, trade and private sector engagement.

The evaluation[71] of support to Conflict Prevention and Peace-building (CPPB)[72] concluded that the EU has significantly increased its focus, strengthened its policy framework and developed a wide range of financial and non-financial instruments. The Commission provided value added in terms of neutrality, reliability (continued presence), critical financial mass, ability to draw on a wide array of instruments, long-term experience and credibility in terms of promoting democracy, peace and human rights. However, the conceptual orientations at policy level have not always been appropriate at operational level. The EU should consolidate and further develop its support for CPPB and ensure that its financial support is sufficiently complemented and leveraged by non-financial support.

The evaluation[73] of EC Cooperation with overseas countries and territories (OCTs)[74] concluded that EU support has been consistent with the EU policy objectives of promoting the economic and social development and bringing them economically closer to the EU. Regarding trade, the EU preference regime did not sufficiently counterbalance the constraints on economic diversification. The involvement of the OCTs in regional programmes was minimised by isolation from regional networking, high participation costs and sometimes by language barriers. Few results have been achieved on climate change and disaster crisis management.

The evaluation of the crisis response and preparedness components of the European Union’s Instrument for Stability (IfS)[75] concluded that the IfS has significantly contributed to enhancing the overall relevance, effectiveness and efficiency of EU crisis response and preparedness actions and that the IfS makes also a significant contribution to the coherence of the EU peace, security and development architecture – and to global peace and stability. Critical to its contributions is the IfS demonstrated capacity to provide quick and timely responses in situations of crisis.

Humanitarian Aid and Civil Protection

The evaluations concluded in 2011 showed that the Commission's activities: addressed major health risks, lowering mortality rates improving disease control for Burmese refugees in Cambodia; provided a relevant and effective response to the cholera epidemic in Zimbabwe; provided relevant and appropriate multi-sectoral programmes in Uganda; and contributed actively to the overall European response to the Haitian earthquake. .The evaluations in Haiti and Zimbabwe highlighted the need to define earlier in the interventions the necessary links between relief with rehabilitation and development (LRRD). This finding confirms the current Commission proposals in Humanitarian Aid, by consistently mainstreaming and strengthening LRRD in humanitarian interventions.

Finally, the Evaluation of the Civil Protection Mechanism and the CP Financial Instrument 2007-2009[76] has shown that both components have substantially contributed to developing further European cooperation and coordination.

Enlargement

An evaluation to support the preparation of pre-accession financial instruments (IPA)[77] beyond 2013 at sector, country and regional level, confirmed the overall relevance ensured by IPA assistance. The evaluation also concluded that the economic and wider benefits of enlargement would more than offset the cost of a new financial instrument and that the EU instrument would enable the realisation of European added value, linking financial support to progress with political criteria, enabling the EU to act as a catalyst for supranational and interregional cooperation, also acting as a broker to bring in the expertise of different administrations and agencies within the EU (e.g. twinning).

The evaluation also confirms the existence of a strong rationale for the future pre-accession financial instrument, which would usefully benefit from improved coherence between policy dialogue and financial cooperation, a programming process focused on sectoral approaches, multi-annual programming, greater involvement of beneficiaries and stakeholders in all stages as well as the introduction of performance incentives

3. Conclusions

Evaluation results in this Report generally indicate that objectives of various financial programmes have been met and EU value added demonstrated. However, the Report draws mainly on interim evaluations which have a strong operational focus and are not conclusive concerning final results or impacts. Lessons have been drawn from experience and indications are given of possible areas for improvement. Where possible this report has identified recommended remedial action that has influenced follow-on programmes. Generally the report points to the need for a stronger focus, linking results to objectives and for increased attention to coherence and consistency across actions.

The Commission wishes to define more clearly how to improve this report in discussion with the Discharge authority. The Commission also notes the request of the Court of Auditors for the report to be produced earlier in the year.

Looking to the future, the Report will be designed to show whether EU programmes are on track and to identify results and impacts as they become available. In order to achieve this goal, a strong and consistent framework needs to be established to ensure monitoring of progress throughout the life of the programmes.

[1] The first evaluation report on the financial year 2010 was adopted by the Commission on 17 Feb. 2012 (ref. COM(2012)40).

[2] EP discharge decision of 10.05.2011 on discharge in respect of the implementation of the EU budget for 2009; section III; paragraphs 71 and 72

[3] Court of Auditor's opinion 4/2012, received by the Commission on 7 June

[4] That is: the Commission's Strategic Planning and Programming Cycle – the Annual Activity Reports, Annual Budget Statements, etc.

[5] Conclusions section, page 16.

[6] EP Discharge 2010 (P7_TA(2012)0153)_Part I).

[7] Court of Auditor's opinion 4/2012, received by the Commission on 7 June

[8] Evaluative studies may contain reviews of specific aspects of the performance of activities and/or focus on monitoring and implementation issues.

[9] Standing instructions to 2011 AARs - Article 60(7) of the Financial Regulation,

[10] Art 33 FR

[11] Approximate multiannual budget of EUR 3,6 billion.

[12] The general approach in this report is to identify Interim evaluations. Where there is no indication of the interim or ex-post nature of the evaluation, it contains elements of both.

[13] Even if the programme covers the period 2007-2013, it is expected that the last results will be achieved around 2017.

[14] http://ec.europa.eu/enterprise/dg/files/evaluation/eye_final_report_en.pdf

[15] Approximate budget of EUR 34 million

[16] EU budget only used for guarantees

[17] The TEN-T programme is directed to ensuring the cohesion, interconnection and interoperability of the trans-European transport network and access to it.

[18] Approximate budget of EUR 8 billion

[19] Eighteen address rail and two address inland waterways

[20] The indication interim or ex post is not specified for evaluations which concern predecessor and ongoing programmes

[21] http://ec.europa.eu/transport/evaluations/doc/2011_marco-polo-programme-2003-2010.pdf. Partly an ex-post evaluation of the last programme and partly an interim evaluation of the current programme

[22] Approximate budget of EUR 417 million only for Marco Polo I

[23] http://ec.europa.eu/information_society/activities/sip/policy/programme/current_prog/index_en.htm

[24] Approximate budget of EUR 55 million

[25] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0548:FIN:En:PDF

[26] Approximate budget of EUR 149 million

[27] http://ec.europa.eu/energy/evaluations/doc/2011_iee2_programme.pdf

[28] Approximate budget of EUR 730 million

[29] http://ec.europa.eu/energy/evaluations/doc/2011_eepr_mid_term_evaluation.pdf

[30] Approximate budget of EUR 4 billion

[31] Approximate budget of EUR 55 billion (Euratom included)

[32] Situation on 30.09.2012

[33] ‘Open access’, defined as free access over the internet, aims to improve and promote the dissemination of knowledge, thereby improving the efficiency of scientific discovery and maximising return on investment in R&D by public research funding bodies.

http://ec.europa.eu/research/science-society/index.cfm?fuseaction=public.topic&id=1294&lang=1

[34] Fifth FP7 Monitoring Report - Monitoring Report 2011 – http://ec.europa.eu/research/evaluations/index_en.cfm?pg=fp7-monitoring

[35] http://ec.europa.eu/research/evaluations/index_en.cfm

[36] The evaluation of the FP7 Transport Research programme, the survey carried out in the Impact Assessment of the Regions of Knowledge Programme, the evaluation study of the FP6-NMP programme

[37] Approximate budget of EUR 7 billion

[38] http://ec.europa.eu/dgs/education_culture/evalreports/education/2011/llpreport_en.pdf

[39] Approximate budget of EUR 309 million

[40] http://ec.europa.eu/social/keyDocuments.jsp?type=0&policyArea=0&subCategory=0&country=0&year=0&advSearchKey=evaluationesf&mode=advancedSubmit&langId=en

[41] Addressing discrimination in employment and the jobs market

[42] http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpb_en.htm

[43] http://ec.europa.eu/regional_policy/information/evaluations/impact_evaluation_en.cfm#1

[44] http://ec.europa.eu/regional_policy/information/evaluations/impact_evaluation_en.cfm#2

[45] http://ec.europa.eu/agriculture/eval/reports/income/index_en.htm

[46] http://ec.europa.eu/agriculture/eval/reports/sugar-2011/index_en.htm

[47] Restructuring of sugar sector, market stabilisation and enhancing market orientation, availability of sugar supplies.

[48] http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/dairy-sector-2011_en.htm

[49] http://ec.europa.eu/agriculture/eval/reports/promotion/index_en.htm

[50] http://ec.europa.eu/fisheries/documentation/studies/eff_interim_evaluation_en.pdf Approximate budget of EUR 4.3 billion

[51] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52011DC0857:EN:NOT

[52] Composed of the programmes "Prevention and Fight against Crime (ISEC)” and “Prevention,

Preparedness and Consequence Management of Terrorism and other Security Related Risks (CIPS)".

[53] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0351:FIN:EN:PDF

[54] Approximate budget of EUR 109 million

[55] Approximate budget of EUR 199 million

[56] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0254:FIN:EN:PDF

[57] Approximate budget of EUR 116 million

[58] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0249:FIN:EN:PDF. Approximate budget of EUR 93 million

[59] http://eur-lex.europa.eu/Result.do?T1=V5&T2=2011&T3=246&RechType=RECH_naturel&Submit=Search

[60] Approximate budget of EUR 21 million

[61] The European radio network

[62] Approximate budget of EUR 312 million

http://ec.europa.eu/health/programme/docs/ex_post_evaluation_en.pdf

[63] http://ec.europa.eu/health/programme/docs/mthp_final_report_oct2011_en.pdf

[64] Approximate budget of EUR 157 million

http://ec.europa.eu/taxation_customs/resources/documents/common/publications/studies/fiscalis2013_mid_term_report_en.pdf

[65] http://ec.europa.eu/dgs/education_culture/evalreports/youth/2011/interimreport_en.pdf

[66] Approximate budget of EUR 885 million

[67] http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1286_docs_en.htm

[68] http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1290_docs_en.htm

[69] http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1296_docs_en.htm

[70] Approximate budget of EUR 7,9 billion

[71] http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1291_docs_en.htm

[72] Approximate budget of EUR 7,7 billion over 2001-2010

[73] http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1294_docs_en.htm

[74] Approximate budget of EUR 286 million over 1999-2009

[75] http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do

[76] Approximate budget of EUR 190 million for the 2007-2013 period.

[77] Approximate budget of EUR 11,5 billion over the 2007-2013 period