Table of contents Glossary Acronyms and abbreviations 1.Introduction 2.Background to the intervention Description of the intervention and its objectives Points of comparison 3.Implementation / state of play 4.Method Limitations and robustness of findings 5.Analysis and answers to the evaluation questions 5.1 Effectiveness 5.2 Efficiency 5.3 Relevance 5.4 Coherence 5.5. EU added value 6.Conclusions and lessons learnt Lessons learnt

Annex 1: Procedural information Annex 2: Stakeholder consultation Annex 3: Methods and analytical models Annex 4: Results of the calls for proposals Annex 5: Focus on third countries Annex 6: List of actions on the initiative of the Commission Annex 7: Supplementary information Glossary Agricultural/agri-food products refer to (agricultural/food) products produced agriculturally as opposed to through hunting, gathering, and so on. Annual work programmes are an administrative tool used by the Commission to set the annual strategic priorities for EU spending programmes. In this case, for the promotion programme, the work programmes are adopted by the European Commission in line with Article 8 of Regulation (EU) 1144/2014 of the European Parliament and of the Council of 22 October 2014 on information provision and promotion measures concerning agricultural products implemented in the internal market and in third countries. The Consumers, Health, Agriculture and Food Executive Agency (CHAFEA) is an executive agency for the management, technical and financial implementation of four spending programmes, including the multi programmes supported by the EU agricultural promotion policy. A competent national authority is the service designated by the government of the Member State responsible for managing a particular spending programme or legislative requirement. For the agricultural promotion policy, the competent national authority is the service charged with implementing simple programmes. High-level missions are a Commission initiative run under the EU agricultural promotion policy. They constitute missions for EU officials and business delegates to engage with target audiences outside of the EU (e.g. business representatives) and governments with the aim of opening up international markets, facilitating international partnerships and supporting exports by providing information on EU products. Implementing bodies are selected by proposing organisations and are responsible for the day-to-day implementation of the programmes. Key performance indicators are measurable programme milestones for key actions used to track the level of progress of an initiative or measure, often against a target. In the context of the EU agricultural promotion policy, a multi programme is a programme submitted by at least two proposing organisations from at least two Member States or by at least one organisation recognised at the European Union level, which is managed by CHAFEA. Protected designation of origin is a label given to products that are produced, processed and prepared in a specific area, using a particular, usually traditional, method. Protected geographical indications identify a product as originating in a particular area, which may be a whole country; the product's given quality, reputation or other characteristic must be attributable to this area and at least one of the production steps must take place in the defined area. In the context of the EU agricultural promotion policy, proposing organisations are both applicants and beneficiaries of simple and multi programmes, depending on the stage of the procedure. In case the proposing organisation is a body governed by public law within the meaning of Directive 2014/24/EU (of the European Parliament and of the Council of 26 February 2014 on public procurement), it must select bodies responsible for implementing programmes under the national legislation transposing this Directive.

In the context of the EU agricultural promotion policy, a simple programme is a promotion programme submitted by one or more proposing organisations from the same Member State and managed by national authorities together with the Directorate-General for Agriculture and Rural Development. Traditional Speciality Guaranteed is a label highlighting the traditional aspects of that product, such as the way it is made or its composition, without being linked to a specific geographical area. The name of a product being registered as a ‘traditional speciality guaranteed’ protects it against falsification and misuse. Acronyms and abbreviations AWP Annual Work Programme CAP Common Agricultural Policy CHAFEA Consumers, Health, Agriculture and Food Executive Agency DG Directorate-General DG AGRI Directorate-General for Agriculture and Rural Development EU European Union GVA Gross Value Added PDO Protected Designation of Origin PGI Protected Geographical Indications SPS Sanitary and Phytosanitary Standard TSG Traditional Speciality Guaranteed 1.Introduction Purpose and scope The European Union’s agricultural promotion policy is designed to support the EU’s agricultural sector by promoting EU products on the internal market and in non-EU countries. This is achieved by co-financing promotion projects and organising information campaigns and high-level missions to raise awareness about the quality of the EU’s agricultural products. Providing information and promoting agricultural products are the key methods used under the common agricultural policy to increase the competitiveness and consumption of EU agricultural products and to increase their market share. By finding new markets, consolidating existing markets and raising awareness among consumers inside and outside the EU of the high standards and high quality of EU agricultural products, the promotion policy contributes to the Commission’s objectives of ensuring jobs and growth in rural areas and of tackling increased pressure on the EU agricultural sector. In addition, by explaining to consumers and importers the high standards and the quality of EU agri-food products, EU promotion measures can support EU producers in increasingly competitive markets. The policy is laid down in Regulation (EU) No 1144/2014, Commission Delegated Regulation (EU) 2015/1829 and Commission Implementing Regulation (EU) 2015/1831. The evaluation covers all three regulations. Under Article 34 of the Financial Regulation applicable to the general budget of the European Union, the EU agricultural promotion policy is subject to periodical evaluations, like all EU programmes and activities that entail significant spending.

Article 26(2) of Regulation (EU) No 1144/2014 states that ‘[b]y 31 December 2020, the Commission shall submit to the European Parliament and to the Council a report on the application of this Regulation together with any appropriate proposals’. The evaluation provides input to the report issued by the European Commission to the European Parliament and the European Council. The evaluation will also feed into the upcoming review of the EU promotion policy for agricultural and food products with a view to stepping up its contribution to sustainable production and consumption, as set out under the European Commission’s farm to fork strategy. The evaluation period begins with implementation of Regulation (EU) No 1144/2014, and covers the information and promotion projects carried out since their start in 2016 by the European Commission and the organisations that ran the promotion programmes from all EU Member States and the United Kingdom (i.e. the geographical scope of the evaluation is the EU-28). The evaluation examines the EU agricultural promotion policy measures targeting EU and non-EU markets over the period 2016-2019. It focuses on assessing the effectiveness, efficiency, relevance, policy coherence and EU value added of specific changes to the implementation and scope of the policy since the previous legislation on promotion policy under Council Regulation (EC) No 3/2008. Specifically, the changes were an increased budget and higher co-financing rates; a broadened scope (measures supported, eligible beneficiaries and products); simplified administrative processes, and more focus on non-EU country markets. The evaluation is underpinned by the support study on the EU agricultural promotion policy targeting both EU and non-EU markets, published on 14 October 2020 (the ‘support study’) and takes into account additional data and information. 2.Background to the intervention Description of the intervention and its objectives The measures specified under the common agricultural policy (CAP) are information provision and the promotion of agricultural products. The general objective is to enhance the competitiveness of the EU agricultural sector and thereby make a direct contribution to the general objective of the CAP in terms of viable food production. The specific objectives of these information and promotion measures are to:

a)increase awareness of the merits of EU agricultural products and of the high standards applicable to the production methods in the EU; b)increase the competitiveness and consumption of EU agricultural products and certain food products and to raise their profile both inside and outside the EU; c)increase the awareness and recognition of EU quality schemes; d)increase the market share of EU agricultural products and certain food products, specifically focusing on those markets in non-EU countries that have the highest growth potential; e)restore normal market conditions in the event of serious market disturbance, loss of consumer confidence or other specific problems. Information and promotion measures take the form of: a)information and promotion programmes (hereafter ‘programmes’), which can be ‘simple programmes’ submitted by one or more proposing organisations from the same Member State, or ‘multi programmes’ submitted by at least two organisations from at least two Member States, or by one or more EU-level organisations; and b)measures taken at the Commission’s initiative, including developing technical support services for the sector. The EU helps the agri-food sector to finance information and promotion campaigns. By explaining to consumers and importers the high standards and the quality of EU agri-food products, EU promotion programmes can support European producers on an increasingly competitive market. Using the generic signature ‘Enjoy, it´s from Europe’, the policy aims to encourage opening up of new markets, to diversify trading partners and to raise consumer awareness of the high quality methods used by European farmers. Below the main changes brought in by the 2014 reform of the promotion policy. ·The new promotion policy was given a more substantial budget, increasing from EUR 60 million in 2015 to EUR 200 million as of 2019. ·Although national co-financing by Member States was withdrawn, the EU co-financing rates were significantly increased to reach 70% for simple programmes in the internal market, 80% for multi programmes and simple/multi programmes targeting non-EU countries and 85% for programmes to restore normal market conditions in the event of serious market disturbance, loss of consumer confidence or other specific problems. There is also a 5-percentage point top-up for beneficiaries from Member States under financial assistance. ·The promotion policy is now based on a genuine strategy framed at EU level, with the adoption of an annual work programme. The strategic priorities and their corresponding budgets for promotion policy in terms of population groups, products, schemes or markets to be targeted and the nature of the information and promotion messages to be disseminated must be specified annually in the work programme. It also provides for specific temporary arrangements in response to a serious market disturbance or loss of consumer confidence. It sets out the criteria for evaluating the proposals and specifies that the annual work programme is implemented by means of published calls for proposals. ·The scope of eligible beneficiaries was extended to include producer organisations and EU-level organisations . The list of eligible products was extended to include more processed products (e.g. chocolate). To generate a higher return on investment, restricted mentions of origin and displays of brands are now allowed, without jeopardising the generic nature of the promotion programmes. ·The promotion programmes should demonstrate a European dimension, in terms of impact and content, with a main EU-level message, complementing the initiatives run by the Member States and the private sector. ·Follow-up output, result and impact indicators were defined to better evaluate the performance of promotion measures and be consistent with the common monitoring and evaluation framework of the common agricultural policy. ·The Commission provides a technical support function to help businesses participate in co-financed programmes, prepare efficient campaigns and develop activities on export markets. ·Another important aspect of the reform was to simply the administrative procedures: -some implementation tasks were transferred from the Commission to the Consumers, Health, Agriculture and Food Executive Agency (CHAFEA) , for cost efficiency reasons; -the administrative procedures were simplified by abolishing the two-step selection procedure (which previously included a pre-selection step);

-the workflow was also streamlined and modernised. In response to the calls, programme proposals are submitted electronically and evaluated by CHAFEA with the contribution of external expert evaluators specialised in the agri-food business and marketing; -the Commission’s selection of the proposals is based on the evaluation carried out by external evaluators and strictly follows the ranking of the proposals drawn up by the evaluators; -multi programmes are managed by CHAFEA under direct management, while simple programmes continue to be managed by national authorities under shared management. The Agency’s mandate was extended to cover coordination of measures taken at the Commission’s initiative, such as communication campaigns, participation in trade fairs and high-level missions.



Other major world players in the agri-food sector also use similar policies. The U.S. Department of Agriculture, for instance, issued funding amounting to USD 300 million in 2019 alone under the agricultural trade promotion programme. Over the past few years, Russia has seen exceptional growth in the agriculture sector, and both the Russian Export Centre and the regional authorities offer a wide range of services to promote Russian goods in cross-border markets. The scale of governmental funds spent on agricultural promotion should also be put in context alongside private-sector initiatives on food advertising (estimated to amount, for example, to EUR 2 billion for Unilever in 2019). The European Commission adopts the budget for implementing the EU agricultural promotion policy as part of the annual work programmes for information and promotion measures. The budget allocated to information and promotion measures in the annual work programmes almost doubled between 2016 and 2019, from EUR 113 million to EUR 201 million. In particular, the annual budget for multi programmes increased from EUR 16.8 million in 2016 to EUR 91.6 million in 2019. Since 2017, the annual budget allocated to Commission own initiatives remained stable at EUR 9.5 million. The annual work programmes set the strategic priorities of a given year, according to the needs of the sector. Priorities can be set on more than one basis (i.e. by product type or to prioritise EU or non-EU markets). Then calls for proposals are launched. The first calls for proposals for simple and multi programmes under the new policy on information provision and promotion measures for agricultural products were launched in 2016 (with the selection in November 2016, of the first programmes awarded funding ). After that, two calls for proposals – one for simple and one for multi programmes – are launched each year. Further information on implementing the policy is given in chapter 3. Points of comparison When making policy comparisons, note that the period covered by this evaluation spans 2016 to 2019 for Commission own initiatives, but only 2016-2017 for simple and multi programmes (that started implementation in 2017). The period before Regulation (EU) No 1144/2014 was implemented should serve as the baseline for comparison in the context of this evaluation. Under the common agricultural policy, Commission funding for promotion and information on agricultural products has evolved significantly since the early 1980s. Since the outset, the policy has addressed the need to increase awareness of EU quality products, increase competitiveness and increase sales, in particular in non-EU countries. However, before Regulation (EU) No 1144/2014, it was unclear whether the policy met these needs effectively and efficiently as a suitable monitoring and evaluation framework and it largely lacked quantitative outcome indicators. This meant that it was difficult to measure the success of the promotion policy over that period. Under Regulation (EC) No 3/2008, the scheme was based on co-financing, usually up to a maximum contribution of 50% from the Commission , with at least 20% from the private sector and the remainder financed by the Member States concerned. Promotion projects had to be presented in the form of a programme submitted by a professional trade or branch organisation in response to national calls for proposals. The programmes were pre-selected by the Member State and the Commission carried out the final selection. Prior to the 2014 reform, a high number of the programmes submitted (up to 57%) were rejected by the European Commission’s Directorate-General for Agriculture and Rural Development (DG AGRI), mainly due to eligibility and quality reasons. To clarify requirements and expectations, the Commission issued several guidelines since 2006, incorporating lessons learnt from previous programmes and from framework contract evaluations. The yearly overall Commission expenditure for promotion measures increased from minor amounts in 2002 to relatively stable amounts since 2007, between EUR 45 million and EUR 50 million. Approximately three quarters of this funding was spent on promotion in the EU and the rest on promotion to non-EU countries.

Six products and themes accounted for 78% of all Commission expenditure from 2002 to 2010: fruit and vegetables (26%); dairy (15%); meat (11%); EU quality schemes (9%); wine (8%); and organic products (8%). As for the breakdown by country, 56% of Commission expenditure went on programmes proposed by five countries: Italy (19%), France (14%), Spain (8%), Germany (8%) and Greece (7%). A further 16% was allocated to promote multi-country programmes. Out of a total 458 programmes signed between 2001 and 2010, most were single proposer programmes targeting national markets. Multi-country programmes, although a priority in the selection process, accounted for less than 10% of total spending, with 41 programmes running between 2001-2010, but for 16% of budget, indicating that they were often of higher financial value. 3.Implementation / state of play The annual work programme The annual work programme introduced by Regulation (EU) No 1144/2014 is adopted through an implementing decision taken by the Commission that sets annual strategic priorities for the promotion measures. When deciding on the substance of the work programme, the Commission takes into account contributions from the Member States (via the Committee for the Common Organisation of the Agricultural Markets), from stakeholders from the agri-food sector and the socio-professional sector (via the Civil Dialogue Group), and internal macroeconomic analysis. 128 promotion programmes were funded under the annual work programmes over 2016-2017: 112 simple programmes and 16 multi programmes. Market priorities are based on the objectives set out in the Regulation, notably to increase the number of activities aimed at non-EU countries where there is the highest potential of growth. Within the EU, the main objective is to inform consumers about the high standards of EU products, the EU quality schemes and the associated symbols, namely organic production, protected geographical indication, protected designation of origin, traditional speciality guaranteed, ‘mountain product’ and the logo for products from outermost regions. For non-EU countries, the Commission conducted a macroeconomic analysis on the projected increase in imports, and a policy review of free trade agreements and expected removal of barriers to trading health and plant-health products. Indeed, promotion programmes can help lay the groundwork for potential exports to non-EU markets before concluding or applying a free trade agreement. The aim of annual work programmes is to provide a relevant tool to achieve a dynamic and targeted promotion policy, by aligning each year with the evolving needs and opportunities of the sector, and identifying priorities with an indicative, specific budget. In addition, the promotion policy is responsive and includes a facility to allocate ad hoc, specific budget to fund programmes supporting sectors that face difficulties, such as the dairy and pigmeat sectors in 2016 and 2017, beef and sheep/goat meat in 2017, table olives in 2018, rice in 2019, and the beef/veal sector in 2020. In parallel, annual work programmes have contributed to aligning promotion policy with other EU policy objectives. Notably, they ring-fenced a dedicated budget to co-finance programmes on sustainable agriculture (sustainable agriculture and the role of the agri-food sector for climate action and the environment in 2017, sustainable aspects of sheep and goat meat production in 2018, sustainable rice production in 2019). This supported Commission work to contribute to the 2030 Sustainable Development Goals and the Paris Climate Agreement. In its 2021 annual work programme, the Commission proposes ring-fencing 50% of the budget to promote organic products, sustainable agriculture, and increased consumption of fruit and vegetables in the context of balanced and proper diets. All of these topics contribute to the farm to fork and biodiversity strategies adopted by the Commission in 2020. Annual work programmes also reserved dedicated budget to co-finance programmes that encourage healthy eating and dietary practices, in line with the Commission's strategy on nutrition, overweight, obesity and other related health issues (to finance programmes in the EU on fresh fruit and vegetables as part of a balanced and proper diet in 2018, 2019 and 2020). Finally, annual work programmes also plan budget for action to respond to unexpected serious market disturbance, loss of consumer confidence or other specific problems, in line with exceptional measures taken under Part V, Chapter 1 of Regulation (EU) No 1308/2013. These budget lines were used for the first time in 2020 as a response to the COVID-19 crisis on the agri-food sector. Additional calls for proposals were launched to address serious market disturbance, loss of consumer confidence or other specific problems under section 1.2.2 of the 2020 annual work programme, on 7 July 2020, to fund action complementary to the COVID-19 crisis package adopted on 30 April 2020. The promotion programmes Scope A promotion programme is a coherent set of operations with the aim of informing the target audience or promoting a product, sector or scheme. It is implemented over a period of between one and three years. It can include advertising campaigns in the press, on television, on radio or online; point-of-sale promotions; public relations campaigns; participation in exhibitions and fairs, social media and a range of other activities. It can be a business-to-business campaign and/or business-to-consumer campaign. Overview of the calls for proposals The high number of proposals (between 144 and 226 every year) received after publishing the calls for proposals from 2016 to 2019 indicates a genuine interest in the scheme. Between 2011 and 2014 the yearly average number of proposals received was 73. Each proposal is evaluated by three independent experts against the criteria in the calls. All eligible proposals are ranked according to the total number of points awarded. The highest ranked proposal or proposals are awarded co-financing, subject to budget availability. Neither the annual work programmes nor the calls for proposals include a breakdown of the budget between Member States.

The selection of proposals is based on the quality of the proposals, i.e. how well they meet the criteria set out in the calls. Another important factor is the level of competition in the topic within which they were submitted. Statistics on the type of participants show that not only well-known trade or inter-trade organisations participate, but also a high number of producer organisations or associations of producer organisations, groups of producers and operators active in geographical indications, etc. The details on selected programmes (names of beneficiaries, budget, product sector, targeted countries, messages and statistics) can be found on CHAFEA’s website . It also gives examples of campaigns, including visuals, pictures of the implemented projects, first results, etc. Statistics on the calls for proposals CHAFEA’s website publishes yearly detailed statistics on the calls. Annex 4 provides a description of the results of the calls from 2016 (the first year of implementing the reformed promotion policy) to 2019. Focus on non-EU countries The reformed policy gives priority to programmes targeting non-EU countries to take advantage of their growth potential. As described below, both the number of programmes and the budget of programmes targeting non-EU countries increased in this programming period (2016-2019). Annex 5 provides more details. In terms of the number of programmes, only a third of the proposals submitted covered non-EU countries under the 2008 Regulation, increasing over the period 2016-2019 to around 60% of simple and multi programmes. In the period 2016-2018, the proportion of programmes targeting non-EU countries was higher among simple programmes than for multi programmes (63% against 35% in multi programmes). In 2019, this trend continued, with 77% of simple programmes and 48% of multi programmes targeting non-EU countries. When compared by year, the number of programmes targeting non-EU countries was at its highest in 2019 (55 programmes), and lowest in 2017 (32 programmes). Between 2016-2018, 53% of the budget was allocated to programmes targeting non-EU markets. Regarding the targeted non-EU countries, over 50% of the funds in each annual budget from 2016 to 2019 were earmarked for China, Japan, South Korea, Taiwan together with the rest of the South East Asia region and India, Colombia and North American markets. Indicators The annex to Regulation (EU) No 2015/1831 presents a list of indicators that can be used by the proposing organisations to assess the impact of the programmes. For simple programmes, Member States send the Commission electronic notification of the impact indicators, under Article 21 of Regulation (EU) No 2015/1831. For multi programmes, the beneficiaries notify directly the indicators via an online platform. These communications are stored in a joint database. Programme managers at the Commission and CHAFEA compare the indicators received with those included in the programme proposals for follow-up of the promotion programmes. Impact indicators include, for example, sales figures for the sector in the year following the promotion campaign, the trend in average sales price of the exported product, change in the level of recognition of the logos of EU quality schemes, or an increase in awareness of intrinsic values of EU agricultural products. To illustrate the impact indicator outcome, for multi programmes, there are two representative examples (covering both the internal market and non-EU countries) that show how in general the multi programmes increased awareness, market share and sale price of the promoted products: -In the internal market: a campaign to promote cheeses with a protected designation of origin or protected geographical indication registered a 28% awareness of these EU quality schemes with an important change in the level of recognition, considering that, according to special Eurobarometer 504, only 14% of European consumers recognised the logos of products carrying a protected designation of origin and 20% for a protected geographical indication;

-In non-EU countries: a campaign to boost the competitiveness of EU quality wines in the US and Canada achieved by the end of the programme a 17% increase in market share and 8% in average sales price of the promoted products. For simple programmes, the following examples are illustrative of their success in increasing awareness, market share and recognition of EU quality scheme logos: -A programme promoting pigmeat in China registered an 18% increase in awareness of intrinsic values of EU agricultural products and a 36% increase in the level of recognition of EU quality scheme logos; -A programme promoting cheese in Colombia and Mexico achieved a 16% and 11% increase in market share of EU products respectively, and an increase in the level of recognition of EU quality scheme logos of 10% in Colombia and 9% in Mexico. Measures taken at the Commission’s initiative These measures include high-level missions, participation in trade fairs and exhibitions of international importance by representation at stands, or action to boost the image of EU products, including communication campaigns. They also include developing technical support services, including market handbooks. Annex 6 provides a detailed list of measures taken at the Commission’s initiative from 2016 to 2019. Technical support CHAFEA has developed a portal to help potential applicants. It includes webinars that have, according to feedback from the surveys, enabled a better understanding of promotion policy, different market opportunities and new rules for applications. High-level missions High-level missions, about two per year (see Annexes 6-7) aim to facilitate market access for European agri-food products. They typically include high-level political meetings involving members of the Commission and an intensive business-related programme for leading representatives of EU exporters and the agri-food sector. The political agenda generally includes discussions on trade agreements, trade and market access issues, health, plant-health and technical barriers to trade issues, discussions on organic production, geographical indications, etc. Business activities typically concentrate on improving market intelligence through targeted seminars and workshops, retail visits and site visits, complemented by business-to-business meetings or business fora. Participants in these missions are selected via calls for applications and on the basis of several criteria, particularly geographical and sectoral balance, balance between small, medium and larger companies; national and European umbrella organisations. After each mission, the Commission conducts at least two surveys of the participants to assess the results and impact. The average satisfaction rate among business delegates is generally above 90%. Most business delegates confirmed that the contacts made during the mission facilitated their business and contributed to advancing their business goals. EU pavilions (stands) at major international agriculture food, beverage and hospitality fairs Trade fairs, which make up 17% of the budget, proved particularly successful. They are an effective communication channel to help raise awareness of the merits and quality features of EU agricultural products (e.g. quality standards and geographical indications) with positive feedback reported in all cases (90%). The Commission runs two to three EU stands each year at selected major international agriculture, food, beverages and hospitality fairs to showcase the excellence of European agri-food products. The stands focus on storytelling about European products, developing online content, advertising and media relations, as well as putting on tasting sessions and cookery masterclasses for business audiences. The aim of the events is to convey the strength of EU agricultural products strength in food safety, quality, traceability, authenticity, tradition and nutritional value, with a special focus on EU quality labels. Other activities Seminars on EU standards for agri-food products These seminars are typically held with target groups of government officials, major importers, retailers, distributors and specialised journalists in non-EU countries. The events are designed to explain and convey information on EU sanitary, plant health (SPS) and food quality standards, with a view to boosting agri-food exports. These regulatory seminars may be accompanied by product displays and culinary demonstrations to showcase both the EU’s high production standards and its rich culinary heritage.

Communication campaigns Communication campaigns are run primarily to follow up on promotion activities already carried out in a given target country (most notably high-level missions) or to engage with target audiences in countries before an EU-led business delegation visits. They are designed to promote EU agri-food products and are focused on the safety, authenticity (e.g. guarantee of origin) and quality of EU products. They aim to raise awareness of the excellence of EU agri-food products; encourage business audiences to import EU products; encourage consumers to buy EU products; and increase awareness of the benefits of economic partnership agreements (e.g. with Canada and Japan). The target audience of these campaigns is typically consumers (usually segmented in each target market, focusing on those with the highest purchasing power or interest in foreign foods), business people (importers, retailers, distributor, hospitality sector), media and key opinion leaders. Technical support services Technical support consists of providing information on the dedicated portal and organising events and other communication activities. The dedicated portal has been created with the aim of helping potential applicants understand the promotion policy and the process of submitting proposals. It also provides best practice examples and legal guidance, and information on all co-funded programmes since 2014. The portal contains a platform helping potential applicants seek partners for multi-applicant proposals. The second aim of the portal is to raise awareness on different markets by providing tailor-made market intelligence reports and information on key target countries listed in the annual work programmes, such as information on market access, sanitary and phytosanitary (SPS) regulations, and protecting intellectual property rights. During the call submission period, from January to April each year, CHAFEA’s helpdesk replies to questions by email on the conditions in the call for applications. CHAFEA actively contributes to communications on the promotion policy by organising an annual Info Day in Brussels . The Commission (DG AGRI and CHAFEA) also participated in over 50 national info days organised by the Member States, each attended by 20-90 participants. 4.Method The evaluation is underpinned by an external support study and complemented by data and analysis carried out by the Commission (DG AGRI and CHAFEA). It also draws on the responses to the consultation on the roadmap conducted from 18 July 2018 to 15 August 2018 and the public consultation carried out between 8 May and 11 September 2020 on the Commission’s web portal ‘Have Your Say’. A mixed-method approach has been used. It draws on a range of tools to compile and analyse quantitative as well as qualitative data and information to provide a robust, nuanced assessment not only of the effects of the EU’s agricultural promotion policy, but also of how and why these effects have been generated. It looks at the main features and changes brought in by the new legal framework. Based on the above methodological considerations, the mixed methods of data collection are listed below. The first three methods collected data on the policy as a whole, while the case studies yielded a more in-depth assessment of the effect of a few measures. ·Document and data review (EU-level). The review was carried out in English to collect qualitative and quantitative data on different aspects of the intervention logic. ·EU-level interviews with: 1)the European institutions and agencies involved in programming and implementing the EU promotion policy; 2)external evaluators hired by CHAFEA for the selection of applications; 3)those involved in other EU policies in the field of trade, development, environment, climate change and health (e.g. Directorates-General for Trade, International Cooperation and Development, Health and Food Safety, Environment and Maritime Affairs and Fisheries); 4)pan-European agricultural producer and trade organisations and associations and

5)a non-governmental organisation. ·Two online surveys. To collect the views of a wide range of stakeholders involved in the policy: one survey targeting the competent national authorities, and the other targeting the proposing organisations (and the teams implementing the measures), including successful and unsuccessful applicants. ·Case studies. Data for the case studies was collected through: -Case study document and data review: this includes programme implementation or evaluation reports (or if available, evaluations at Member State level aggregating the results of all programmes funded), as well as secondary data to assess the effects of the programmes. -Targeted interviews: with at least one competent national authority and one proposing organisation per programme. For programmes targeting non-EU countries, the interviews includes a member of the EU Delegation in that country, a producer or trade association, where feasible, business delegates in the country, etc. ·Public consultation. A public consultation was held as part of this evaluation. It generated 144 replies, of which 38 (i.e. 27% of the respondents) were from non-governmental, consumer and environmental organisations, 64 (44%) from business organisations / associations or trade unions, 12 (8%) from public authorities, 17 (12%) from EU citizens and 13 (9%) from academic/research institutions or ‘others’. The analytical tools used in this evaluation included programme statistics (such as number of visitors at fairs and participants in non-EU delegations) and indicators (such as impact indicators measuring the impact of the policy on change in awareness). The review of the programme indicators included all available output, result and impact indicators used for reporting on simple and multi programmes funded in 2016 and 2017. The indicators were provided by CHAFEA and DG AGRI. For the support study, complete indicator data was available for 74 out of 112simple programmes and all multi programmes. Limitations and robustness of findings The evidence collected served as the basis for a robust and nuanced assessment of the effects of the EU agricultural promotion policy, in addition to how and why these effects have been generated. However, the study faced a number of inherent challenges and resultant limitations. Impact indicators could often not be measured during the reporting, e.g. when data were not yet available or their measurement appeared difficult. The analysis could therefore take into account output and result indicators only. Some difficulties arose when evaluating the use of indicators as well. First, none of the promotion programmes had ended, so final impact indicators were not available. In theory, the types of impact indicators set and compiled by independent evaluators would provide evidence that the policy’s specific objectives are met. However, these are reported once a programme has been completed and, therefore, these were not available for this evaluation . By the end of 2020, output, result and impact indicators are expected to be available only for the multi programmes that ended in the first part of 2020 (three-year programmes selected in 2016 and two-year programmes selected in 2017). Second, the range of indicators used in the different programmes made it impossible to use them to assess the promotion policy overall. For example, it was impossible to aggregate impact indicators for different products promoted in different target countries, and it was inconsistent to merge samples of multi programmes with simple programmes. Third, many of the result indicators did not allow for meaningful conclusions to be drawn on the effectiveness of the promotion policy, because in many cases the intended results would not be sufficient to support the generation of impact assessments on the promotion policy’s specific objectives. Fourth, the causality between the promotion policy and changes in market share and awareness is difficult to measure with the present monitoring and evaluation framework. As far as the quantitative impact of the promotion programmes on the market is concerned, although some trade data are available, it is thus not possible to equate the activities of the campaigns to increases in consumption or market share of the specific products and conclude there is a causal link. This is due in part to the size and diversity of the market, which makes it difficult to attribute broader trends to the specific activities of any individual programme. The speed of change in the food and drink sectors can also make it difficult to attribute observed changes to any specific promotion programmes. As pointed out, the timing of the evaluation meant that no programmes were yet complete, which hampered the ability to fully assess the effectiveness of measures at this time. Second, the range of different programmes and Commission own initiatives meant that a ‘one-size-fits all’ approach could not be applied to analyse all programmes and initiatives. Instead, it was necessary to make a purposeful selection of case studies to provide an in-depth analysis indicative of the bigger picture though not representative of all programmes. Third, estimating the costs and benefits of the promotion policy in quantitative terms proved unfeasible due to lack of comprehensive data. As a result, the assessment of links between operational and administrative aspects and efficient achievement of results had to be based primarily on the qualitative data collected. The simplification of procedures and processes in the reformed promotion policy did not produce more quantitative data for a cost benefit analysis. It is important to keep these aspects in mind when analysing and interpreting the results of the study. A further challenge relates to the difficulty to isolate specific effects of the promotion policy measures from the influence of contextual factors, which was addressed by analysing the promotion policy within a wider context. Nevertheless this is a residual limitation of the evaluation. The 144 replies received in the public consultation result in a low statistical significance.

5.Analysis and answers to the evaluation questions The evaluation gives an assessment of the effectiveness, efficiency, coherence, relevance, and EU value added of the EU agricultural promotion policy on the basis of six evaluation questions. 5.1 Effectiveness Two evaluation questions aim at assessing the extent to which the EU agricultural promotion policy has been effective in achieving its specific objectives and the general objective to enhance the competitiveness of EU agricultural products, and to assess how the policy design has contributed to its effectiveness. The assessment is based on data available from the document review, online survey, interviews and evidence from case studies conducted in the support study. 5.1.1 Effectiveness in achieving the specific objectives and the general objective to enhance competitiveness Increased awareness of merits and high standards Given the multiple limitations of the analysis (explained in chapter 4), it is difficult to quantify the extent to which the promotion policy is effective in contributing to increasing the awareness of the merits of EU agricultural products and the high standards applicable to EU production methods, or the exact size and profile of the target group reached in internal and external markets. The evaluation study found that there is stakeholder consensus that EU policy and programmes contributed to raising awareness and reaching the target groups, as least to some extent. It also found evidence to confirm the promotion of messages on the merits of EU agricultural products. The stakeholder survey responses showed that most stakeholders (proposing organisations, implementing bodies, national competent authorities) agreed that the EU agricultural promotion policy had reached the target audiences of the programmes (both EU and non-EU target audiences). A comparative analysis of the online survey responses across the different objectives shows that: ·proposing organisations and implementing bodies are mostly positive about the effects of the EU agricultural promotion policy. Notably it has raised awareness of the merits of EU products and the high standards applicable (51% (59) strongly agreed), it increased awareness and recognition of EU quality schemes (42% (49) strongly agreed) and it has contributed to enhancing the overall competitiveness of the EU agricultural sector (42% (49) strongly agreed); ·national competent authorities were most positive about the promotion policy’s effectiveness in achieving the policy’s general objective; ·both the national competent authorities and the proposing organisations surveyed had slightly less confident views on the policy’s effectiveness in achieving specific objectives relating to increasing consumption and market share. Source: support study (adaptation) According to Eurobarometer No 504 released in October 2020, 81% of respondents are aware of the merits of EU agriculture in terms of high standards. The reform brought in the mandatory use of the signature ‘Enjoy! It’s from Europe’ in the visuals of the promotion measures. The support study showed different levels of appreciation of that signature by target audiences in the EU and in non-EU countries. While reaching the target audience appears to be easier within the EU, the message stressing quality standards of products was more relevant in non-EU countries, where it was associated with other values such as safety and tradition. The signature was generally well received with some minor concerns raised, such as the generic nature and limited use of brands and specific country origin. Target audiences in non-EU countries may have limited knowledge of individual countries in Europe, but they do understand where Europe is and can usually easily associate the signature with Europe. Most stakeholders agreed that, to the extent of their knowledge, the signature is starting to gain widespread recognition in non-EU countries. About a quarter of the stakeholders noted that signature identification is still not as high as hoped for. However, they agreed that the level of awareness is slowly increasing and gathering momentum and that keeping the same “”signature can help boost recognition. Increased competitiveness, consumption and raised profile The exact effect that the promotional policy had on the level of consumption and competitiveness of EU agricultural products is very difficult to calculate, notably due to the timing of the evaluation and the many external factors (political, legal and regulatory changes, broader economic developments, etc.) that influence sales and consumption. Although case studies of promotion programmes for selected sectors (wine, beer and vinegar, fruit and vegetables and meat) proved inconclusive in terms of assessing the impact on competitiveness, the stakeholders consulted in the support study expressed a positive perception of the policy’s effects on consumption and competitiveness, with respondents from the dairy sector being the most positive regarding the effects on overall competitiveness and respondents from the meat sector the least positive.





Source: support study (adaptation) This positive perception is echoed in the replies to the public consultation. Most respondents agreed that the EU's promotion measures both on the EU market (71% (102) of replies) and outside the EU (62% (89) met the objective of increasing competitiveness. The case studies provided some evidence, e.g. for EU fruit and vegetables marketed in the United Arab Emirates, that the selected programmes contributed to increased sales and consumption in the target markets, with programmes targeting non-EU markets having a higher potential than programmes targeting the EU market. The case study interviews also highlighted that activities focusing on direct contact with consumers, networking at trade fairs and events, product promotion by influencers and using social media instead of journalists in promotional campaigns were found to be particularly effective in increasing competitiveness and consumption. Awareness and recognition of quality schemes Special Eurobarometer No 473 provides a useful picture of consumer perception, highlighting the relative importance of EU quality labels on purchase decisions (albeit EU consumers lack awareness of the schemes ). In 2018, one third of EU respondents were not aware of any of the EU quality labels, but 63% of respondents were aware of at least one of the logos. According to the latest EU-wide survey of public opinion on agriculture and the CAP (Eurobarometer 504), in 2020, 57% was aware of the organic label, 14% of the ‘protected designation of origin’ label and 20% of the ‘protected geographical indication’ label. The survey found a significant increase in the EU (+30 percentage points) in recognition of the EU organic label and a consistent increase of the fair trade and protected geographical indication labels since 2017, recognition of the other EU quality labels remaining stable over the reference period. It is difficult to assess the effects of the promotion of EU quality schemes on consumer perception, given the lack of data on recognition in non-EU markets. The in-depth interviews at EU level indicate potential of the programmes targeting EU and non-EU markets with awareness-raising activity on quality schemes, rather than evidence of this in practice. However, since the promotion policy included organic produce, the awareness of the organic label increased with 30 percentage points between 2018 and 2020 (Eurobarometer 504), which can be partly attributed to the impact of the promotion policy. There was also a slight increase in the level of awareness of the ‘protected geographical indication’ label while awareness of the other EU labels remained the same. Source: Eurobarometer No 504, October 2020 The stakeholders interviewed or who responded to the online survey were confident that the programmes had contributed to increasing awareness and recognition of the quality schemes. Over 95% of stakeholders considered that the agricultural promotion policy had increased awareness of the EU quality schemes among target audiences. The case studies confirmed that differences in the level of awareness of quality schemes can have an effect on success (with lower initial levels of awareness making it harder for programmes to achieve an increase). The case studies covering non-EU markets showed the promotion of quality labels did have some traction (for example on the US market for wines and spirits) but also challenges, such as perceived consumer indifference to the labels (in the United Arab Emirates). Source: support study (adaptation) Influencing market share

Desk research conducted as part of the case studies showed an increase in both sales and exports of several EU agricultural products promoted by the programmes. However the extent to which the promotion programmes (200 contracts signed over the period 2016-2018 excluding abandoned programmes) contributed to these results is impossible to assess, given the large number of factors affecting the market. Stakeholders agree the promotion policy has had an effect on influencing the market share of the EU agricultural products, in particular by providing support to help EU producers enter certain new markets. This is particularly effective for non-EU markets though reports on programme-level results confirm that, at this point, there is limited concrete evidence to substantiate this perception. Regarding promotion campaigns for EU wine, beer, spirits and vinegar in the USA, all coordinators, consortium partners and implementing agencies interviewed for this case study reported anecdotal evidence of increased sales on the US market. There has been an increase in the sales value of EU exports to the US over the evaluation period for wine, brandy and gin/genever, suggesting that EU trade competitiveness increased in general over this period. Interviewees stopped short of directly linking their increased sales to the awareness campaigns, but they did report significant investments by individual brands in sales and direct marketing, which is likely to have had a more direct and/or significant impact on sales. They perceived the campaigns as a ‘springboard’ or ‘starting point’ to educate influencers, opinion setters and ultimately consumers about the specific products promoted. Source: Eurostat (chart by Ipsos) As illustrated by Figure 7 , different stakeholders have very different views on the extent to which the promotion policy increases market share. Only 54% of responding national competent authorities thought that the agricultural policy increased the market share of EU agricultural products and certain food products, against 91% of successful proposing organisations, 64% of unsuccessful proposing organisations and 92% for implementing bodies. Source: support study (adaptation). Case studies indicated that in most markets targeted by programmes, positive trends can be seen for the products promoted (in terms of increased exports or sales on the target market). Stakeholder interviews from case studies back up the perception that the promotion policy met its goal to target markets in non-EU countries with high-growth potential. Figure 8 illustrates the example of the increase in EU exports of dairy products to China, which was the goal of several promotion programmes (see Annex 2). Source: CLAL (based on Eurostat and IHS) (2019-2019). Dataset EU-28: Dairy sector. Though it is impossible to attribute fully these trends to the programmes funded, stakeholders suggest that the programmes make a positive contribution to these results. Restoring normal market conditions The effectiveness of the programme in addressing the specific objective of restoring normal market conditions cannot be assessed due to the lack of measures implemented over the evaluation period. The evaluation did not identify situations in which such measures should have been used in the evaluation period. Additional calls for proposals were launched for the first time on 30 June 2020 in response to the Covid-19 crisis. The general objective of the EU promotion policy Overall, the evaluation finds positive signs that the programmes funded are in line with the policy’s objective to enhance the competitiveness of EU agricultural products and are making progress towards achieving the policy’s specific objectives. However, a full assessment of the policy’s effectives is premature. It should be carried out once the programmes have come to an end and the impact can be more fully assessed.

5.1.2 Contribution of the policy design to its effectiveness Budget, co-financing rates and eligibility criteria Feedback from the case studies and in-depth interviews indicated that, although the current level of co-financing (generally perceived to be relatively high) is considered appropriate to attract small-scale organisations, the successful organisations are usually the better resourced ones (and they are often able to seek support from organisations that would help implement the programmes if successful, i.e. advertising agencies). Smaller organisations are likely to be weaker not only in terms of their financial resources but other resources too, e.g. human resources or expertise in marketing and product promotion, which in turn can have a negative impact on the extent to which programme measures are implemented effectively. Proposing organisations and implementing bodies praised the levels of co-financing and budgets in terms of attracting proposals of satisfactory quality and ensuring effective implementation of the proposed measures. The fact that the percentage of ineligible proposals decreased over the period 2016-2019 for both simple and multi programmes (except for multi programmes in 2017, where ineligibility went up temporarily) implies that organisations gradually became familiar with the new requirements and ineligible organisations ceased applying. However, the decrease may also be attributable to the action taken to better inform and guide applicants, via info days, frequently asked questions and other information published online (such as webinars and the eligibility checker). Results from the online survey (covering both successful and unsuccessful organisations) combined with stakeholder views collected by means of in-depth interviews and case studies show that most stakeholders (survey respondents and interviewees) reported having a positive impression of the fairness of the rules (e.g. over half of all survey respondents agreed or strongly agreed that the eligibility criteria were fair). A significant share of producers requested a larger budget for the promotion policy. In particular, most fresh fruit and vegetable producers favoured support for promotion campaigns in the EU market. Several respondents in the public consultation requested a higher contribution from the European Commission to the co-financing rates, and a two-step selection procedure that would reduce the submission costs. Some respondents asked for more flexibility on the financial arrangements such as more advance payments and interim instead of annual payments. Some underlined the need to digitalise reporting from proposing organisations to the Member States. They also emphasised the need to simplify the requirements during the selection of implementing bodies. Some suggested that two calls for proposals per year would be more adequate. Some Member States highlighted difficulties faced by smaller organisations in their countries who are not eligible, or do not have sufficient financial capacity to submit applications given the current rate of co-funding required. Extended scope Overall, the qualitative feedback on the scope of the eligibility rules was positive. No concerns were raised by those assessing the proposals; nor by those managing the process (i.e. CHAFEA and the national authorities). Proposing organisations described the extended scope as providing greater ‘flexibility’ and ‘freedom’ and thereby facilitating the effectiveness of the programmes by allowing organisations that may not have been previously eligible to be involved. EU-wide agricultural producer organisations also appreciated that the list of types of eligible products was extended. They specifically mentioned the inclusion of food-industry products, instead of only primary-agricultural products under the previous regime. There was no evidence/examples of issues or problems caused by extending the scope of eligible proposing organisations and/or products such as ambiguity of rules, exclusion of relevant organisations to date. This is confirmed by all sources of input, including the online survey, case studies and in-depth interviews. Most of the respondents to the public consultation had a positive (38%) or very positive (27%) opinion on the impact of extending the eligibility of proposing organisations and the list of products for funding under promotion, with only 13% expressing a negative opinion. Shared and direct management Overall, respondents indicated a higher level of stakeholder satisfaction with management by CHAFEA than with management by national authorities. This relates partly to the fact that CHAFEA is better placed to administer the programmes, given its deeper understanding of the programmes and their underlying aims, as well as having the dedicated systems and experienced staff. Having sufficient management resources is more challenging for some national authorities than for others, which may have given rise to the other issues highlighted: ·national authorities’ inconsistent interpretation of the guidelines and application of procedures and rules, which implies a need for greater harmonisation of rules to reduce confusion, ensure a level playing field for applicants and create more opportunities to exchange good practice; ·national authorities’ difficulties in accessing the information they need and proposing organisations’ difficulties in knowing which managing authority to contact for information (i.e. CHAFEA or national authorities). This remains an issue, even though the CHAFEA website provides exhaustive guidance and addresses many replies in the frequently asked questions; ·national authorities reported a lack of confidence in their approval and implementation of adjustments; ·proposing organisations perceived a lack of flexibility with budgetary/contractual or other adjustments in simple programmes, which means it is difficult to make the measures responsive to contextual changes. Provisions on origin and brands Approximately 40% of national and EU-wide organisations who responded to the online survey faced issues with restrictions on mentions of origin and the visibility of brands. The in-depth interviews and case studies revealed concerns about the extent to which the provisions on origin and brands were in line with the needs of proposing organisations. The most common concerns raised on the requirement to have the EU message as the primary focus are related to the: ·dissemination of the message that the product promoted is of ‘European’ origin, as it was suggested in the survey that there is typically significantly higher consumer awareness of a specific country or region than of ‘Europe’;

·disclaimer accompanying the EU message, stating that the EU is not responsible for the content of the message, which is confusing and self-defeating; ·restrictions on visibility that discourage involvement from stakeholders who are not willing to invest resources in a campaign that restricts to a large extent the mention of brands and origins; ·perception that multiple messages, brand-related outside the programmes and not brand-related inside the programmes, dilute the effectiveness of the measures; ·restrictions in mentioning the country of origin, which limit the effectiveness of the programmes as the national brand may sometimes be stronger and serve the campaign better than the fairly generic reference to ‘European products’. At the same time, the evaluation highlights the importance of examining the policy against the principle of non-discrimination and the fact that the policy is inherently an EU policy and not a national or private policy. In this context, given the complementary nature of EU action in this area, ensuring that the EU message is the main message of the campaigns and the consequent restrictions in the provisions on origin and brands appear to support their intended purpose. Moreover, this approach safeguards related EU agricultural policies (such as on quality logos) and fulfils the principle of non-discrimination, which prohibits giving favourable treatment on the grounds of nationality. All in all, when taking the above into consideration and in the absence of a counter-factual measurement which would indicate how different rules impact the effectiveness of the measures, it is challenging to establish a link between the concerns expressed by some stakeholders and the potential impact of these issues on policy effectiveness. In addition, the 144 respondents to the public consultation had different, albeit mostly positive opinions regarding the new possibility to help promote EU agricultural and food products by allowing a restricted mention of the origin and display of brands on the visuals created: 16% consider it effective ‘to a large extent’, 35% ‘to a fairly good extent’, 27% ‘to some extent only’, only 4% replying ‘not at all’ and 19% ‘don’t know’. Monitoring and evaluation system of the simple and multi programmes According to available data, there is significant variation in the way that proposing organisations report on promotion programmes indicators (see table below on output indicators ), and full compliance with the Commission’s requirements is not achieved. Indicator data was missing for 36 out of 110 simple programmes implemented in 2016 and 2017. Information collected from the online survey, the in-depth interviews and the case studies lay bare a number of limitations with the current system: ·Most national authorities who participated in the online survey agreed that the monitoring and reporting system is clear, useful and proportionate. Most successful proposing organisations (78%) also agreed that reporting requirements are easy to understand. However, this perception is not confirmed by the findings from the case studies, where national authorities and proposing organisations reported difficulties to understand the content of simple programmes. ·National authorities in charge of simple programmes also reported difficulties in understanding the concept of the key performance indicators and how they should be used in the reporting process. ·Information collected from case studies and in-depth interviews suggests that proposing organisations and implementing organisations view the monitoring and evaluation system as work-intensive and demanding, especially when specific promotion services/activities are subcontracted. Reporting on indicators comes on top of time-consuming programmes that must be delivered on time. This seems also to relate to the issue of how well stakeholders understand the reporting process, as limited understanding makes the process more time-consuming, and restricts the scope to learn the monitoring process. Burdensome reporting can also potentially have an impact on the effectiveness of the measures, as it limits the time available for design and implementation of programme activities. 2016 and 2017 Simple Programmes (N=74) Multi programmes (N=16) Indicator N= % of total N= % of total O1 Number of stands at trade fairs organised 60 81% 10 63% O2 Number of seminars/workshops organised 48 65% 14 88% O3 Number of press conferences organised 34 46% 10 63% O4 Number of tastings days at points of sales organised 40 54% 12 75% O6 Number of study visits organised 35 47% 8 50% O8 Number of spots aired on TV 20 27% 9 56% O9 Number of spots aired on radio 24 32% 8 50% O10 Number of published print adds 42 57% 13 81% O11 Number of published advertorials 26 35% 8 50% O12 Number of published online adds 43 58% 9 56% O13 Number of press releases 50 68% 15 94% O14 Number of email newsletters sent out 27 36% 10 63% Source: support study (based on data from DG AGRI and CHAFEA) Additional observations can be made about the indicators: ·The current indicator framework described in the Regulation is incomplete: the three levels of indicators are not sufficient to capture the effects of the simple and multi programmes. In a performance chain, each step of the chain should, at least in theory, generate the next level of effects. However, there is a missing step between the stated ‘results’ and the intended ‘impacts’. For example, it is difficult to ascertain that there was a direct link between the number of website visits or event participants and an increase in consumption. ·The allocation of indicators to the headings outputs, results and impacts are less than clear and usable. Some impact indicators currently listed can be beyond the direct scope of impact of simple and multi programmes, even if the programmes may have some influence on these aspects. Impact indicators such as change in market share may mainly result from agricultural market dynamics: world prices, quantities harvested, etc. Promotion and commercial activities cannot change these dynamics on their own and should be complemented with other tools. ·At the results level, there is a focus on quantitative indicators (on the number of target audience reached, number of press coverings, value of press clippings and number of subscribers to a newsletter). As they currently stand, indicators at this level require mostly quantitative data that attempt to provide a straightforward numerical description of the issues/success of the programmes and steer the policy response in a specific direction, and to easily measure the success or failure of a programme. This design of the indicators, especially those tracking results and impact, lacks recognition that data on the promotion policy continue to be subject to contextual and other limitations, which would require systematic reporting of qualitative information.

Overall, the evaluation finds that there is greater satisfaction with the system as rolled out for multi programmes than for simple programmes due to CHAFEA providing more support for multi programmes than national authorities do for simple programmes and because multi programmes with their international scope are more powerful in achieving results. Once complete, it will be difficult to assess programme impact because relevant indicators have been difficult to calculate, and therefore the focus of monitoring and evaluation is on outputs and results and not impact. There are difficulties and costs in capturing robust evidence on impacts due to the likely weak causal effects of information and promotion measures on, for example, impacts such as market share. This situation is exacerbated by the difficulties experienced by proposing organisations in understanding how to set appropriate indicators and systems to capture the required data, and by the administrative burden in the reporting process, which some stakeholders found excessive. Overall, a general lack of focus on the quantitative effects makes it difficult to assess at the aggregate level how target audiences have responded to the information they received and whether improvements could be made across the programmes. 5.1.3 Commission own initiatives The assessment of the extent to which Commission own initiatives contributed to the effectiveness of the promotion policy is based on available Commission data, desk research and interviews with key stakeholders. The limited evidence gathered on technical support, which account for a small proportion of the budget (4%) and 19% of the contracts signed for allocated to Commission own initiatives, suggests it does provide cross-cutting support to EU agricultural organisations on the type of support available and how to access it. The portal created to help potential applicants and the webinars have, according to feedback from the surveys, improved understanding of the promotion policy, of market opportunities and of the new rules for applications. The high-level missions (accounting for 25% of awarded contracts under own initiatives and 13% of the budget), are part of the EU’s economic diplomacy activities. They appear to be successful at facilitating market access for EU agri-food products by successfully raising awareness. An overwhelming majority of attendees reported a level of satisfaction of over 90%, as well as new contacts being concluded as a result of these missions. Trade fairs, which account for 17% of the budget and 22% of contracts awarded, proved to be particularly successful and an effective communication channel to support awareness raising of the merits and quality schemes of EU agricultural products (e.g. quality standards and geographical indications). Positive feedback was reported in all cases (90%). The assessment found a high level of satisfaction with EU product quality/SPS (sanitary and phytosanitary standards) seminars, which account for 9% of the budget and 19% of overall contracts awarded, involve government officials and business representatives from the sector providing information on EU products and standards (over 90%). Most participants reported an increased knowledge of EU standards and of the merits of EU agricultural products. Communication campaigns, which accounted for the largest share of budget (56%) aim to follow up on other own initiatives and support EU-led business delegations engaging for the first time with target audiences in non-EU countries. The data available indicate that such campaigns have successfully increased the target audience’s understanding of EU products and their quality. As above, the assessment found that action under Commission own initiatives: ·contributed to increasing awareness among stakeholders in non-EU markets, which helps create a favourable environment to promote EU agri-products and helps boost the competitiveness of EU products; ·contribute, to some extent, to the objective of increasing the awareness and recognition of the merits and high standards of EU products, and of quality labels; ·increase consumer awareness of the merits of EU agricultural products, notably the presence at international trade fairs and visits of EU stakeholders; ·consistent targeting of certain countries since 2017 improved the effectiveness of measures (as shown by the increasing success of initiatives in China). Source: public consultation 2020, 144 respondents Most respondents to the public consultation expressed positive views about the effectiveness of EU promotion measures: communication campaigns (71%), participation in fairs (67%) and high-level missions (60%), production of market entry handbooks (59%). Only 47% gave a positive assessment on the effectiveness of seminars.

Monitoring and evaluation system of Commission own initiatives Given that some initiatives were still ongoing or closed recently, only a limited assessment of the quality of reporting was possible. The assessment reveals that key performance indicators give a sense of the results and outcomes, but provide limited information on impact (it is difficult to draw a link between individual initiatives and changes in market conditions). They mainly track the perceptions of participants, giving rather limited evidence of quantifiable impact (e.g. number of contracts signed, increase in sales). There was a lack of standardisation at EU level of project-specific indicators for all projects. Overall, the usefulness of reporting could be improved by having clearer definitions of specific expected results (e.g. in terms of targets), of performance measures and systems to collect data. CHAFEA and DG AGRI have already remedied this weakness by streamlining the set of indicators and clarifying the methodology for measurement. 5.2 Efficiency The evaluation found that the promotion policy is broadly efficient but efficiency can be improved, for example by better quantifying the costs and benefits. Due to limitations of the monitoring and evaluation system and the lack of completed projects, the costs and benefits of most promotion measures could not be measured. Where impact could be measured precisely e.g. the pig meat programme in China that increased awareness of intrinsic values of the Union agricultural products with 18 percentage points and the cheese programme in Columbia and Mexico generating a market share increase of 16 and 11 percentage points respectively, the attribution of the impact to the policy is difficult as also other factor can have played a role. The same applies to the positive outcome of promotion programmes shown in the six case studies described in Annex 2 of the evaluation support study: 1.Wine, beer, spirits and vinegar in the USA 2.Dairy products in China 3.Fruit and vegetables in the United Arab Emirates 4.Products recognised under European quality schemes in the EU market 5.European meat on the EU market 6.Fruit and vegetables in the EU market Another challenge to make a quantitative assessment on the efficiency of the EU agricultural policy is the question: What are reasonable costs to achieve benefits in terms of awareness, consumption and market share? The evaluation support study shows that the marketing budgets of private companies can be two digit amounts in billion euro whereas competitors like USA and Australia spend relatively higher amounts on their promotion policy. Calculations regarding the costs for the output of Commission own initiatives show that these strongly differ per output unit for different initiatives. Despite the change brought about by the reform of 2014 (that entered into force in 2015), as regards the system of application, application submission, evaluation, selection, management of promotion programmes and technical support provided to the proposing organisations, one third of respondents (36%) still found that the new system for EU agricultural and food products promotion is not very efficient. By contrast, 30% of respondents had a positive opinion and 10% were very positive. Administrative burden Overall, respondents found that the system placed a substantial administrative burden on applicants in preparing an application, though this lessens with repeated applications as applicants learn. This means that applicants with the capacity to apply more regularly, i.e. those who are financially stronger and have more resources, have an advantage to some extent. The case study interviews provide some evidence that the level of support provided by national authorities correlates with the level of participation in agricultural policy promotion programmes. The difference in support has knock-on effects in terms of the workload for applicants and leads to the perception of an uneven playing field between applicants from different countries. As of 2016, over 50% of selected proposals were submitted by applicants who had never received promotion policy funding before, evidence that the revised eligibility conditions concerning proposing organisations had helped widen the base of potential applicants. Regarding the workload of national authorities is concerned, case study evidence highlighted differences in national authorities’ knowledge of the policy, rules and procedures. The support study found that, at times, many national authorities were unable to answer questions raised by applicants and had to ask the Commission for support. Proposing organisations and implementing bodies consulted in the case studies also emphasised their frustration with the lack of information available and an inability to get specific responses; they spent a substantial amount of time trying to understand the responses provided by national control authorities and some even asked directly the Commission and CHAFEA for support. These differences in knowledge of the policy, rules and procedures had an effect on how much time and effort national authorities were able or willing to spend assisting proposing organisations in applying for and managing programmes and how strictly they applied the rules set under the Regulation. Although 45% of national control authorities that responded to the support study survey felt that there was no need to change the current management systems of simple programmes, 36% said it should be changed. Eight national control authorities made some specific suggestions for changes, with four noting the need for more guidance on the interpretation and application of rules (to ensure equal treatment across all Member States).

Costs and benefits Due to the difficulty in quantifying benefits, a comparison of costs and benefits cannot be established for simple and multi programmes. This is largely due to lack of quantitative data on the benefits such as the change in the competitive position after the intervention. Data on the costs and result indicators have enabled a partial quantification of the costs and benefits of Commission own initiatives such as audiences reached and the number of visitors to fairs as well as the costs involved (Annex 7). In summary, the evidence indicates that there are some areas for improvement in terms of efficient implementation of promotion measures and technical support. Clearer communication between the contracting authority and sub-contractor on the expectations of speakers and any preferences may help prevent future issues in delivering technical and promotional support (such as the need to cancel webinars). The evidence from high-level missions and trade fair reports indicates that these visits have been implemented efficiently, with clear examples of contractors using their initiative to overcome barriers, particularly last-minute schedule changes which are (to some extent) par for the course with this type of event. Nonetheless, delays to approval processes were highlighted as one aspect to improve and give more time for organisation and implementation. There is limited information, in particular on impact indicators, to assess all aspects of efficiency related to the Commission initiatives. Different methods are used to calculate the number of participants and the use of different metrics such as ‘PR value’ and ‘number of articles’ to measure the value of media coverage, which means that there is no comparable cost data for high-level missions and trade fair events. Table 9 in Annex 7 illustrates the cost and benefits of high-level missions and trade fairs and provides an overview of the budget, estimated cost per participant and estimated value of media coverage for each event. Most respondents to the public consultation reported that the EU budget for promotion programmes yielded the expected results either fully (20%) or partially (63%), with 4% reporting that the available budget did not yield expected result at all, 2% stating ‘on the contrary’ and 10% had no opinion. 5.2.1 Efficiency of direct management of multi programmes compared to shared management of simple programmes The assessment is based on the information collected through desk research, an online survey, interviews and case study interviews. It looks at the rationale for and perceived advantages of direct management of multi programmes, the differences between feedback from proposing organisations of multi and simple programmes and their management, and issues of the management modes in the context of specific measures. There appears to be an emerging consensus amongst all stakeholders that the direct management of multi programmes by CHAFEA was more efficient than the shared management of simple programmes. This can be traced to a number of main factors: ·stakeholders perceived the use of digital reporting methods as much more efficient than paper methods. This has implications for the simple programmes, where the first step in the reporting process (for proposing organisation coordinators to send to national competent authorities) is still paper-based; ·the standard approach taken by CHAFEA to apply the rules governing applications, grant management and reporting, compared with a range of procedures (and, by implication, differences in terms of flexibility and rigour in applying certain rules) between Member States; ·CHAFEA is credited with providing more useful information in response to queries raised by proposing organisation coordinators on project management, which leads to longer term efficiency in implementing multi programmes (due to a higher level of certainty of the rules) compared to simple programmes, where there is less flexibility and more fear of strict and opaque financial audits. Source: support study These findings are backed up by the replies to the public consultation, where a clear majority of the respondents (60%) considered direct management to be more efficient than shared management, with 32% considering it ‘far more efficient’ and 28% ‘more efficient’. Only 10% of respondents replied ‘less efficient’, 5% of respondents found ‘no difference’ and 25% had no opinion. 5.2.2 Efficiency of the application system (requirements, submission, evaluation, selection, management and technical support) The assessment is based on information collected through desk research, the online survey, interviews and case study interviews. It looks at the length and cost of the application process, the extent to which applicants, programme managers and evaluators perceive the application process, requirements and support available to be proportionate, and issues of the application system in the context of specific measures. The evidence collected indicates that the application process is time-consuming and work-intensive for applicants (with some organisations hiring external consultants to help them prepare and submit the proposals). However, applicants are clear on the eligibility criteria and deemed it quite efficient. The number of ineligible proposals decreased over the period 2016-2019 for both simple and multi programmes, and interviewees who responded to the survey conducted during the support study reported that the eligibility criteria were clear and that the time required to complete the application process does decrease after applying for funding several times. However, although it has decreased, the ineligibility rate remains at 10%.

On the award criteria, stakeholders requested more clarity and transparency on how proposals are awarded. However, they reported that applying for funding and drafting proposals that respond to the award criteria becomes easier after having applied a couple of times, implying that increased exposure to and experience with the application procedure increases familiarity with the process. However, there are significant differences in the level of guidance and support provided to proposing organisations by national competent authorities during the application process. The public consultation yielded mixed views regarding the efficiency of the new EU system for the promotion of EU agricultural and food products, with 40% of respondents providing a positive opinion, 38% an unfavourable opinion and 23% no opinion. 5.2.3 Efficiency of the rules applied by Member States governing the procedure for selecting implementing bodies The assessment is based on information collected through the online survey, interviews and case study interviews. It looks at the differences in how the rules are interpreted and applied by different Member States, and the reasons, the extent to which proposing organisations find the procedure for selection of implementing bodies efficient, and the issues, problems, effects of the application system in the context of specific campaigns and measures. The evidence indicates that the procedure for selecting implementing bodies was very work-intensive and therefore expensive, although the process seems transparent and fair, as measured by the final selection of implementing bodies. Feedback from interviews and case study research shows that proposing organisation coordinators invest significant time in understanding and complying with the rules for selecting implementing bodies, particularly for simple programmes. However, there are differences in the rules applied by Member States (both according to their national laws, but also in the level of strictness with which certain requirements are implemented), which directly affects the efficiency of the selection process. The rules governing the selection of implementing bodies appear to be applied more strictly for simple programmes than for multi programmes. Given the difference in rules applied by Member States under their national laws and the interpretation of rules by national authorities (for simple programmes) and by CHAFEA (for multi programmes), this increased efficiency over time only works if the proposing organisation-coordinators and implementing bodies work on consecutive programmes, and therefore have continued exposure to the programmes and are able to gain familiarity with the rules and requirements. This could therefore also be perceived as giving an unfair advantage to ‘old hands’. There was no clear conclusion from the public consultation on the efficiency of rules applied by the Member States concerning the procedure for selecting implementing bodies, 43% of replies gave a positive opinion on the efficiency, 25% a negative opinion and 32% were undecided. 5.2.4 Efficiency of Commission own initiatives Due to the limited information from the online survey, interviews and case study interviews, the assessment relies primarily on a review of the reports provided by the organisations contracted to implement services. It looks at the unit cost of Commission missions, pavilions, seminars per participant/visitor/article etc., the extent to which stakeholders perceive additional value from the Commission own initiatives, as well as the issues, problems and effects of specific Commission initiatives and the way they were carried out. The findings indicate that there are some areas for improvement in terms of the efficiency of implementing promotional activities and technical support. Clearer communication between the contracting authority and sub-contractor on the expectations of speakers and any preferences may help prevent future issues in delivering technical and promotional support (such as the need to cancel webinars). High-level missions and trade fair report evidence indicate that these initiatives have been implemented efficiently, with clear examples of contractors using their initiative to overcome barriers, particularly last-minute schedule changes (which are to some extent usual for this type of event). Nonetheless, delays to approval processes were identified as an aspect to improve to help provide more time to organise and run the initiatives. Figure 8 presents the results of the public consultation. Most of the replies gave a positive opinion on the efficiency of EU promotion measures: participation in fairs (65%), communication campaigns (61%), high-level missions (57%), and the production of market entry handbooks (52%), with the exception of seminars (49%). Note: The differences in aggregate numbers presented in the text are due to rounding up. Source: public consultation. Limited information is available to assess all aspects of efficiency related to Commission initiatives. There are different methods for calculating the number of participants and the use of different metrics such as ‘public relations value’ and ‘number of articles’ to measure the value of media coverage, which means that there is no comparable cost data for high-level missions and trade fair events. 5.2.5 Efficiency of the facility for organisations to submit several programmes in several topics

The assessment is based on the information collected from document and data reviews, the online survey and interviews. It looks at the take-up of the facility for organisations to submit several applications, the extent to which proposing organisations value the ability to submit applications for several programmes and how this has contributed to the efficiency of the application and management process, as well as other aspects related to multiple submissions. The evidence indicates that stakeholders welcomed the scope to apply for several programmes as it increased their familiarity with the application process, and, for previously unsuccessful applicants, allowed them to learn from feedback and improve their knowledge of the application process. Proposing organisation coordinators and implementing agencies managing more than one programme believed that efficiency gains can be made in implementing initiatives by applying learning across multiple programmes. The survey responses from evaluators highlighted the potential for resourcing difficulties if proposing organisations and implementing bodies receive funding for several projects at the same time. However, this concern about resource constraints was not borne out by feedback from proposing organisation coordinators. Most of the 133 replies to the specific question in the public consultation were either very positive (32%) or moderately positive (35%) that the EU promotion policy was efficient in offering the scope for proposing organisations to submit several programmes under several topics when submitting applications to the European Commission. Only 4% of respondents gave a negative reply; 6% of respondents saw ‘no difference’ and 23% had no opinion. 5.3 Relevance The assessment of the relevance of the promotion policy analyses the extent to which the promotion policy’s general and specific objectives are in line with the most pressing needs and problems facing the EU agricultural sector today. The analysis is based on desk research complemented by data collected during interviews, online surveys, and case studies carried out in the support study. 5.3.1 Relevance of the general and specific objectives of the policy Overall, the evidence collected from stakeholders indicates that the EU agricultural promotion policy’s general and specific objectives are relevant and positively addressing stakeholder needs as well as market challenges. In terms of stakeholders needs, the duration of funding is found to be linked to the ability to achieve meaningful change, although for certain products or markets, the possibility to run programmes for more than 36 months (which is the limit set by legislation) would seem adequate. Source: support study (adaptation) On the basis of replies on the most common reasons for applying to the promotion programmes, the most relevant policy objectives addressing stakeholder needs were to: ·increase awareness of the merits of EU agricultural products and the high standards of production methods (27%); ·increase the competitiveness and consumption of European agricultural products within the EU (27%); ·increase the market share of European agricultural products outside of the EU (23%). Source: support study

The promotion policy proved relevant in light of recent market and policy developments such as the Russian import ban on EU products, African swine fever, US sanctions, etc. which impacted EU trade of agricultural products, requiring policy intervention. Moreover, given the recent focus in the Green Deal and the farm to fork strategy on environmental and climate aspects, the policy could have a significant role in the transition towards a green and sustainable EU agricultural sector, provided that annual work programmes are tailored to the objectives of these initiatives. The public consultation included a question on the relevance of the strategic targets in the EU promotion policy in terms of audience, products and regions. An overwhelming majority of the replies considered the targets either ‘very relevant’ or ‘moderately relevant’ in terms of audience (87% of 104 replies), products (85% of 115 replies) and regions (83% of 111 replies). In each case, only a small minority (3-7%), considered the targets irrelevant. 5.3.2 Relevance of the annual work programmes The evaluation indicated that annual work programmes provide clear strategic priorities that are both relevant to the promotion policy and aligned with evolving market conditions and policy objectives e.g. they point out which third countries are particularly targeted by the promotion programmes and align the promotion policy to some extent with EU environmental policies. Annual work programmes set the scene for programmes aimed to increase awareness of the high standards of EU goods, boost the consumption and competitiveness of these goods internally and outside of the EU and increase market share, especially in non-EU countries. Annual work programmes set out targets based on sound information and current market needs and challenges. Despite being relevant to the annual work programme strategic priorities, these objectives sometimes appear to be more similar to overarching aims than measurable goals, as objectives have no quantifiable thresholds to evaluate whether it achieved the intended result. In sum, the evaluation found that annual work programmes appear to be for the most part relevant to the challenges and pressures faced in recent times by the European agricultural sector. Therefore the annual programmes can be considered generally relevant to the needs of the stakeholders. Some urgent and extraordinary problems of the latter were resolved adequately, for example in reacting to the economic impact of Russian sanctions, while other concerns related to environmental and climate change could be improved in the priorities of future annual work programmes. 5.3.3 Relevance of Commission own initiatives Commission own initiatives seem to respond to current challenges and market needs. The in-depth interviews of the support study revealed that the most relevant aspects for stakeholders were technical support and information seminars, although it depends on the stakeholder groups. For example, national competent authorities appear to particularly appreciate high-level missions, regarding them as highly relevant to promote EU agricultural products in non-EU countries. Successful proposing organisations particularly valued technical support services for the information provided. Implementing bodies especially rated trade fairs and international exhibitions at EU pavilions, which they viewed as valuable fora to carry out their promotion measures. Source: support study (adaptation) Overall, Commission own initiatives made a positive contribution to the general strategic vision for the EU’s agricultural sector, though unsuccessful proposing organisations agreed less with this statement than other groups of respondents. A clear majority of the respondents to the public consultation gave a positive assessment of the relevance of EU promotion measures, from participation in fairs (82% of 133 replies), communication campaigns (82% of 134 replies), high-level missions (71% of 135 replies), production of market entry handbooks (67% of 133 replies) and seminars (66% of 132 replies). Source: public consultation 5.3.4 Relevance of the signature ‘Enjoy it’s from Europe!’ The support study findings indicate that, overall, the signature ‘Enjoy! It’s from Europe’ was relevant. This is backed up by the replies to the public consultation, to which respondents gave an overall positive reply, with 25% (from 139 replies) considering the signature to be relevant ‘to a large extent’, 31% ‘to a fairly good extent’ and 24% ‘to some extent only’. Only 7% replied ‘not at all’ and 13% had no opinion. However, the support study also found that opinions on the relevance of the signature vary according to whether it is used in the EU market or in non-EU countries. Stakeholders tended to agree that the signature is more relevant in non-EU countries than in the internal market, particularly when the products are new to the market. On the one hand, the signature is gaining widespread recognition in non-EU countries, and consumers positivel