Table of contents 1.Introduction Purpose and scope of the evaluation/fitness check 2.What was the expected outcome of the intervention? 2.1 Description of the intervention and its objectives 2.2 Point(s) of comparison 3.How has the situation evolved over the evaluation period? Current state of play 4.Evaluation findings (analytical part) 4.1.To what extent was the intervention successful and why? 4.2.How did the EU intervention make a difference? 4.3.Is the intervention still relevant? 5.What are the conclusions and lessons learned? 5.1.Conclusions 5.2.Lessons learned Annex I: Procedural Information Annex II. Methodology and Analytical models used Annex III. Evaluation matrix and, where relevant, Details on answers to the evaluation questions (by criterion)

Annex IV. Overview of benefits and costs [and, where relevant, Table on simplification and burden reduction] Annex V. Stakeholders consultation



ABBREVIATIONS AND ACRONYMS AWP Annual Work Programme CBCDG Central Bank Counterfeit Deterrence Group CfP Call(s) for proposals CNA Competent National Authority DG ECFIN Directorate General for Economic and Financial Affairs DG HOME Directorate General for Migration and Home Affairs DG JUST Directorate General for Justice and Consumers DG NEAR Directorate General for Neighbourhood and Enlargement Negotiations ECB European Central Bank ECEG Euro Counterfeiting Experts Group EQ EU Evaluation Question(s) European Union Eurojust European Union's Judicial Cooperation Unit Europol European Police Office Interpol International Criminal Police Organization ISG Inter-Service Group MS Member State(s) OLAF European Anti-fraud Office OPC Open Public Consultation SWD Staff Working Document TAIEX Technical Assistance and Information Exchange TOR Terms of Reference 1.Introduction Purpose and scope of the evaluation/fitness check Pericles 2020 was a European Commission exchange, assistance and training programme aiming at promoting actions for the protection and safeguarding of the euro against counterfeiting. The budget for the Programme's implementation was approximately EUR 7.3 million. The Pericles 2020 programme ran for the period 2014-2020 and replaced the Pericles programme, which was established in 2001 for a period of four years (1 January 2002 to 31 December 2005) and was then extended until 31 December 2013. The Programme was a multiannual programme to promote actions for the protection and safeguarding of the euro against counterfeiting, thus enhancing the competitiveness of the Union's economy and securing the sustainability of public finances. It was an exchange, assistance and training programme aiming at preventing and combating counterfeiting and related fraud, thus enhancing the competitiveness of the European Union's (EU) economy and securing the sustainability of public finances. The Programme actively encouraged and entailed an increase in transnational cooperation for the protection of the euro against counterfeiting both inside and outside the EU. Particular attention was paid to those Member States or third countries that have the highest rates of euro counterfeiting. More specifically, the Programme, supported and supplemented the activities undertaken by the Member States and assisted the competent national and European authorities in order to develop among themselves and the Commission a close and regular cooperation and an exchange of best practices, where appropriate including third countries and international organisations.

The actions in order to achieve the above mentioned objectives, as defined in Article 8 of the Regulation, were: a. the exchange and dissemination of information, through the organisation of workshops, meetings and seminars, trainings, targeted placements and staff exchanges of competent national authorities; b. technical, scientific and operational assistance including relevant studies with a multidisciplinary and transnational dimension; c. grants to finance the purchase of equipment to be used by specialised anti-counterfeiting authorities for protecting the euro against counterfeiting. Projects financed under the Programme were implemented either directly by the Commission’s Directorate General for Economic and Financial Affairs (DG ECFIN) or in the form of grants awarded to national competent authorities in the EU (both in the euro area and non-euro area Member States). The common co-financing rate for grants awarded under the Programme was 75% of the total eligible costs. These projects took place both inside and outside the EU. Pursuant to Article 13(6) of the Regulation, by 31 December 2021 the Commission shall present to the European Parliament and to the Council a final evaluation report on the achievement of the objectives of the Programme. In this respect, an evaluation study was carried out by an external contractor and assessed the Programme’s overall implementation in the period from 1 January 2014 to 31 December 2020. The study covered all types of actions under the Pericles 2020 Programme (both within EU Member States and third countries) and considered the progress on the findings, conclusions, and recommendations of the mid-term evaluation of the Programme. The study ran from April 2021 to March 2022. The cut-off date for the data/information used was 31 December 2020. However, as agreed during the Inception stage, because of implementation delays to some actions due to the COVID-19 pandemic the study also considered, where possible, actions financed by Pericles 2020 that were implemented in 2021. 2.What was the expected outcome of the intervention? 2.1 Description of the intervention and its objectives Since the introduction of the euro as a single currency, there has been a need to protect it at an EU level against counterfeiting and to have a specific programme dedicated for this purpose. The new single currency entailed a higher risk of counterfeiting which in turn had the potential to undermine trust in the euro. Therefore, to address this issue at the EU level, the Pericles programme was established in 2001 to precede the introduction of the single currency coins and banknotes in January 2002. Pericles was the predecessor of the Pericles 2020 programme. The legal basis of the Programme is Article 133 of the TFEU. The Pericles 2020 programme built on the work of the previous Pericles programmes, which operated from 2001 to 2013. Pericles 2020 was a training for trainers programme and can broadly be qualified as a capacity building, information dissemination and networking initiative. Its activities were aimed at: (i) raising awareness of the threat posed by euro counterfeiting; (ii) promoting closer and more regular coordination and cooperation among relevant institutions; (iii) enhancing the operational capabilities of staff; and (iv) developing improved tools and methods in the areas of euro counterfeit prevention, detection and repression. The Programme was also actively involved in supporting the improvement of the legal and institutional framework for euro protection, namely in connection with the general framework established by the 1929 Geneva Convention on Currency Counterfeiting and the necessary measures for the protection of the euro against counterfeiting established by Regulations 1338/2001 and 1339/2001 . The Programme was inspired by three key principles, namely: (i) transnationality, which entails the requirement that all Programme activities see the participation of at least two countries; (ii) multidisciplinarity, intended to facilitate the adoption of a common approach to euro protection by the target groups mentioned hereafter; and (iii) complementarity, with Programme activities intended to supplement and not to replace other euro protection initiatives implemented by Member States or by EU/international institutions.

In line with its multidisciplinary approach, the Programme sought to benefit a wide range of target groups, including: (i) law enforcement authorities (police, customs, ministries of interior, intelligence personnel); (ii) monetary authorities (national central banks, mints); (iii) judiciary authorities (ministries of justice, judges, prosecutors); (iv) commercial banks and other financial sector operators (money exchange or transport companies, etc.); and (v) other private sector organisations (bankers’ associations, etc). The Programme's strategy during the programming period focussed on four priority areas, namely: (i) supporting EU Member States particularly affected by euro counterfeiting; (ii) fostering cooperation with third countries where there is evidence for or suspicion of counterfeit euro production; (iii) maintaining an efficient framework for the protection of the euro in South Eastern Europe; and (iv) addressing new developments and challenges. The Programme's priorities areas were updated on an annual basis and endorsed by the Euro Counterfeiting Experts Group (ECEG), a group of national counterfeit experts from all 27 Member States as well as representatives from the Commission, European Central Bank (ECB), and Europol and then incorporated into the Annual Work Programme (AWP). Figure 1: Pericles 2020 Programme’s Intervention Logic for the period 2014-2020 The ‘intervention logic’ is a logical model describing in diagrammatic form the linkages between the various elements comprising the intervention (Figure 1). Pericles 2020’s intervention logic can be described with reference to six main elements, namely: ·The first element refers to the needs addressed by the Programme, i.e. the growing international dimension and changing nature of euro counterfeiting; ·The second consists of the inputs available to address these needs, which include the Programme’s financial, human and managerial/administrative resources as well as the resources of the other entities involved; ·The third concerns the actual deployment of available resources through the implementation of Pericles 2020 actions, i.e. the organisation of conferences and workshops, staff exchanges, etc.; ·The fourth refers to the outputs delivered by the Programme actions, which can be described in terms of staff trained, best practices disseminated, etc.; ·The fifth relates to the ‘transformation’ of the outputs into outcomes, i.e. the discrete changes in euro protection capabilities that the Programme is expected to generate (in terms of increased operational capability, closer institutional cooperation, etc.); and ·Finally, the sixth element refers to the Programme’s impact, i.e. its contribution to the general improvement in euro protection, which in turn is expected to contribute to the end goal of improving general economic conditions. 2.2Point(s) of comparison The Pericles 2020 programme built on the work of the previous Pericles programmes, which operated from 2001 to 2013, aiming at protecting the euro as the single currency against the ongoing threat of counterfeiting at EU level. It also operated alongside the additional measures and actions related to euro counterfeiting taken at national level. The Programme aimed at identifying both emerging and recurrent threats and priorities outlined in the annual strategy paper aiming - in this way - to continue to respond to these threats and priorities in a flexible and effective way. The distribution of counterfeits and high quality components on the internet as well as high quality euro coin counterfeits were identified as threats at the beginning of the programming period and remained among the priorities of the Programme through the entire implementation period. Specific attention was given to high-risk third countries whose criminality represents a threat for euro counterfeiting. Europol and ECB contributed to the annual strategy paper for implementing Pericles 2020 and participated in the vast majority of the Programme’s actions.

The Programme used a set of four specific quantitative performance indicators as laid down in the basic legal act: the number of (i) counterfeits detected, (ii) illegal workshops (print shops and mints) dismantled, (iii) individuals arrested, and (iv) penalties imposed. The Programme’s performance overview illustrates that progress in achieving the targets for the performance indicators was assessed positively and kept within the range of the reference band. Despite the fact that the numbers for these indicators approached their targets, the primary challenge with some of the performance indicators is that the linkage with Pericles 2020 actions appears weak; in many instances other external factors can influence the indicators and they primarily measure the repression (and not the prevention) aspect of the phenomenon. The Programme showed a high degree of consistency and complementarity with other relevant programmes and actions at Union level because it is dedicated exclusively to and focused on protecting the euro from a specific form of crime, namely euro counterfeiting. This complementariry appears clear with respect to DG NEAR’s Technical Assistance and Information Exchange (TAIEX) instrument, that is most often used in support of accession negotiations, and DG HOME's Internal Security Fund - Police, which covers the prevention and combating of crime in general. 3.How has the situation evolved over the evaluation period? Current state of play Budget and Management The Programme had a total budget of € 7.344 million for the 2014 – 2020 period, i.e. around € 1 million per year. Funds were used for: (i) the provision of grants to Competent National Authorities (CNA) interested in implementing actions (‘CNA-implemented actions’); and (ii) the financing of actions implemented directly by DG ECFIN (‘direct actions’). The annual distributions varied, but around 70% of the annual budget was commonly allocated to CNA-implemented actions and 30% to direct actions. Until the end of 2014, the Programme was managed by the European Anti-fraud Office (OLAF), which was the entity responsible for managing the previous Pericles Programme. From January 2015 to the end of the programme, Pericles 2020 was managed by DG ECFIN. Activities were undertaken on the basis of a yearly updated rolling strategy and Annual Work Programmes (AWP). Advice on the strategic orientations of and planning for the Programme was provided by the ECEG. Operating Modalities Pericles 2020 actions were implemented through the provision of grants to CNAs or directly by DG ECFIN, through so called ‘direct actions’. Grants were provided on the basis of Calls for Proposals (CfPs), while direct actions were implemented via procurement contracts. CNA-implemented actions benefited from a financial contribution in the form of a grant. CfPs were launched annually with two deadlines, one in spring and one in autumn in order to guarantee a prompt and flexible reply to upcoming unforeseen threats. The overall coordination, management, and administrative structures have been assessed positively by the stakeholders interviewed for the evaluation (see Annex V for details of the stakeholders’ consultation). These stakeholders pointed out the importance of the Pericles 2020 programme, which provided training and capacity building activities that respond to law enforcement needs and fill gaps that would not be covered otherwise by a national or a European instrument. Each CfP indicated the value of funding available and the types of actions eligible for financing. Applications were assessed against a set of pre-defined criteria, reflecting the Programme's basic principles (transnationality, multidisciplinarity and complementarity) and taking into account the quality of the proposals and cost-effectiveness considerations. In case of an award, subsequent steps followed the Commission’s standard procedure for grant-based initiatives, with the signing of a Grant Agreement and, upon completion of the action, the submission of a Technical Report and a Financial Report. As specified in Article 10(4) of the Regulation, the Programme’s maximum financial contribution was normally set at 75% of eligible costs, but in certain duly justified cases, it could be increased up to 90%.

Although managed by DG ECFIN, direct actions were always implemented in collaboration with national authorities in EU Member States or third countries. From an administrative point of view, direct actions were implemented through procurement contracts. Upon completion, results were summarised in a report similar to the Technical Reports required for CNA-implemented actions. Actions Implemented – Overview Over the period between January 2014 to December 2020, commitments were made for a total of 80 actions. These included 21 actions implemented directly by DG ECFIN and 59 actions implemented by 18 CNAs. Table 1: Overview of implemented actions based on the delivery mechanism Year Grants Projects initiated by the Commission 2020 7 1 2019 10 3 2018 8 3 2017 10 3 2016 9 4 2015 9 3 2014 6 4 Total 59 21 The 80 actions had an initial total budget of about € 8.2 million, with an EU financing contribution of about € 6.9 million. The value of the actions varies considerably, the average value being € 80,000. The Programme's actions were generally evenly distributed over time, with 10 actions approved in 2014, 12 in 2015, 13 in 2016, 13 in 2017, 11 in 2018, 13 in 2019 and 8 in 2020. Nearly all actions were completed by the end of 2020, with five postponed to 2021 and one cancelled, because owing to restrictions liked to the Covid-19 pandemic. The CNAs involved in implementation originated from ten Member States (Croatia, France, Germany, Italy, Spain, Romania, Bulgaria, Portugal, the Netherlands, and Austria) and included monetary authorities (central banks and national mints) and law enforcement authorities. The Member States involved in the implementation of Pericles 2020 actions are listed in the table below. Table 2: Distribution of Pericles actions financed per country (by year of commitment) Year Total EC ES IT BG PT HR FR NL RO AU DE EE BE PL HU LV 2006-2013 - - 11 25 4 1 4 1 3 10 2 5 3 1 1 2014 10 4 1 3 2 2015 12 3 2 4 1 1 1 2016 13 4 3 4 1 1 2017 13 3 3 3 1 1 1 1 2018 11 3 3 5 2019 13 3 2 3 2 1 1 1 2020 8 1 3 1 1 1 1 2014-2020 80 21 17 23 3 2 3 6 1 2 1 1 0 0 0 0 0 As presented (see table 2), DG ECFIN and the Italian and Spanish CNAs were the most active implementers, cumulatively accounting for over 75% of actions and the related budget. Technical training, conferences, and staff exchanges are the most common types of actions, each accounting for about one quarter of all actions. Overview of Actions Implemented Table 3 – Distribution per type of Pericles action financed (by year of commitment) Year Total Staff exchanges Seminars / conferences Technical training Purchase of equipment E-grants Study 2006-2013 113 30 32 41 7 2014 10 2 3 3 2 2015 12 4 7 1 2016 13 3 5 3 1 1 2017 13 4 6 2 1 2018 11 4 3 2 2 2019 13 2 7 4 2020 8 2 2 2 1 1 2014-2020 80 21 33 17 4 1 4

It is to be noted that the Pericles 2020 programme funded 80 actions whilst the previous programme funded 113, This is due to the reduction of the available budget and, more markedly, due to the increase of the average cost of the co-financed actions. CNAs were requested through the ECEG to provide data related to law enforcement activities on an annual basis. These data are used to monitor the key performance indicators (KPIs) as specified in Article 4 of the Regulation; namely, the number of counterfeits detected, illegal workshops dismantled, individuals arrested and penalties imposed. Due to reporting limitations by CNAs, data on penalties imposed are not available. Table 4: KPIs state of play KPIs 2014 2015 2016 2017 2018 2019 2020 Counterfeit notes detected 838,000 899,000 684,000 694,000 563,000 559,000 460,000 Counterfeit coins detected 192,195 146,899 150,258 160,914 179,353 187,602 201,565 Workshops dismantled 37 32 29 22 23 22 Individuals arrested 7946 8879 8724 7494 5932 5994 NA Penalties imposed NA NA NA NA NA NA NA Data from stakeholders (e.g.: CNAs, implementers and participants) provided a major source of evidence to inform the evaluation. 4.Evaluation findings (analytical part) Evaluation Framework The evaluation study covered all the five evaluation criteria typically used in the assessment of EU programmes, namely: (i) relevance; (ii) effectiveness; (iii) efficiency; (iv) EU added value; and (v) coherence. In addition the evaluation also considered a sixth criteria: (vi) sustainability, in line with the evaluation requirements of Regulation (EU) No 331/2014. In particular: ·With respect to relevance, the exercise reviewed the rationale of the Programme with regard to the objectives, priorities, target groups, and actions; ·The assessment of effectiveness focused on the Programme’s ability to achieve the intended results in terms of outputs, outcomes, and impact; ·The analysis of efficiency involved a review of the management and administrative arrangements put in place for Programme implementation; ·With regard to EU added value, the evaluation focused primarily on the Programme’s ability to promote transnational cooperation; and ·The analysis of coherence considered the complementarity of Pericles 2020 activities with other initiatives implemented at the national and EU/international levels.

Since this was a final evaluation of the Programme, it also considered the impact of the Programme actions (as a part of the effectiveness assessment), and the sustainability of the Programme’s actions in protecting the euro against counterfeiting. 4.1.To what extent was the intervention successful and why? The Programme throughout its entire period covered all the envisioned types of activities, which led to the achievement of the operational objective. According to the Annual Reports, almost all the financed actions under the programme were completed on time (with a few exceptions because of the pandemic), indicating the Programme’s effectiveness in delivering the diverse types of activities, envisioned. The Programme was overall effective in reaching the targeted number of participants in actions. The analysis of a selected number of Pericles 2020 actions by the external contractor show that even though the number of actual participants in these actions was lower than the planned participants, the divergence was marginal. Even though the external study did not identify direct links between the size of the divergence and the number of participants, however it was noted that the larger the event, the bigger the risk for not meeting the target of expected participants. The external study also concluded that the effectiveness of larger events is smaller as concerns the achievement of the target on the number of participants. Evidence in the study from the external contractor also suggests that Pericles 2020 has been effective in taking into account the transnational aspect of the counterfeiting phenomenon, as well as the multidisciplinary aspects of the fight against counterfeiting, in line with the provisions of Regulation (EU) No 331/2014. The Programme participants are more diverse when compared with the preceding programme, and it has been effective in taking into account the multidisciplinary aspects of the fight against counterfeiting. Moreover, the survey results from participants of actions implemented under the Programme, show that activities undertaken were effective in terms of both establishing contacts with other people involved in the protection of the euro and in disseminating best practices. The establishment of contacts between people and institutions has been praised as one of the most important outputs of the Programme by the interviewees. Figure 2: Assessment of the Programme effectiveness in delivering qualitative outputs The study concludes that the specific objective of the Programme has been achieved, as Pericles 2020 contributed strongly to the enhancement of the capacity to protect the euro. The available evidence resulting from the ‘deep dives’ performed and the stakeholders’ consultation shows that the specific objectives and outcomes have been delivered. The Programme achieved strengthened networking amongst participants and greater awareness on the counterfeiting phenomenon, increased the cooperation and coordination between Member States, third countries and institutions, enhanced the operational capacities of staff involved and the use of new and improved tools and methods against euro counterfeiting. In addition, the results from the deep dives pointed to some key success factors of the programme and challenges. The evaluation study also concurs with the finding of the Mid-Term evaluation that the available data on the performance indicators were broadly on track to achieve the intended targets. At the same time it underlines that these indicators, mostly linked to the results of operational counterfeit repression activities, are not fully adequate to measure the performance of the Programme. Quantifying the impact of a capacity building initiative in terms of operational results is an exercise fraught with difficulties due to the influence of a host of intervening factors. This is particularly the case with initiatives intended to protect against criminal activities, the magnitude of which is almost by definition unknown. Subject to this major caveat, there is evidence that the Programme outcomes contributed to detecting counterfeit euro and dismantling workshops. The study concludes that the EU framework for the protection of the euro, including the Pericles programme, has a bearing on the competitiveness of the economy and on the stability of public finances. In addition and as shown by the statistics, the EU framework , has been successfully keeping euro counterfeiting in check. This, not only is avoiding any significant direct adverse effects on the public finances but, mainly, increases the trust of the citizens and businesses in their currency and, thereby, in the economy in turn, this enhances the conditions for the economy’s competitiveness and increases the sustainability of public finances. It is indicative that the overall coordination, management, and administrative structures have been assessed positively by the stakeholders. The communication with DG ECFIN has also been positively assessed by interviewees, and the support provided by the EC experts has been highly rated and described as ‘prompt’. Promptness of the Programme management has been confirmed by its relevant timelines. Partner institutions such as the ECB and Europol also provided a very positive assessment of their cooperation with DG ECFIN regarding the Pericles Programme. To a large extent, the application procedure was considered demanding by applicants and non-applicants alike, an issue which is expected to be minimised by the new digital grant system of Pericles IV. No significant efficiency-related issues were reported in relation to the implementation process. The external study compared the management costs of the Programme with the ones of Fiscalis 2020 and Customs 2020 and concluded that these costs are higher for the Pericles 2020 programme as a proportion of its spending. However, the external study considers that the high management costs of the programme are well justified since they are linked to the programme’s specificity, which can only be fully maintained if the programme remains stand-alone and can offer tailor-made actions for specific objectives. The study showed adequate programming and allocation of the available resources, evidenced by the very high percentage of allocation as compared to the reference budget, achieved by Pericles 2020 (higher than Pericles 2006-2013). The committed budgets proved to be higher than the actual final payments that needed to be made. Whilst the outputs of the actions were largely delivered, they were delivered at a lower cost than initially planned, which suggests the Programme is broadly efficient; at the same time, the deep dives analysis made by the external contractor showed that the difference on average between the planned and actual grants is -19%. However, the overestimated grant budgets requested by applicants clearly have a negative impact on the pipeline of Pericles actions as unused budgets are effectively blocked until project completion, an issue for which solutions need to be found. Other indicators of efficiency are the fact that the costs for trainers were within the statutory staff costs of the respective countries, and that the average unit costs for seminars/conferences and training sessions can be considered reasonable. Whilst comparisons with other programmes - such as Fiscalis 2020 and Customs 2020 programmes - are challenging due to different types of activities, geography, and measurement units, the comparison of the number of seminars/conferences of the programmes demonstrates that the costs for Pericles 2020 are broadly in line with those of other programmes, albeit slightly higher.

Data collected by the study illustrated that most actions are performed at the co-financing threshold level, with many actions requiring a higher co-financing rate for well-justified cases, and only a few requiring a co-financing of the Programme below 70%, thereby showing how crucial the financing of the Programme is for the implementation of the actions. The co-financing setup of the Programme is generally regarded as appropriate by interviewees, yet some interviewees pointed out that an increase in the threshold of the EU contribution to 90% would be preferable. However, a higher co-financing rate would mean a decrease in the number of actions financed by the Pericles budget, which would result in less actions financed by the Pericles programme. For this reason, an increase in the contribution rate would be undesirable if not accompanied by an increase in the overall Programme budget. Based on all the considerations above, it can be concluded that the current co-financing setup (up to 75% standard rate, and 90% in duly justified cases) is appropriate. Moreover, the consistency and complementarity of Pericles 2020 with other relevant EU programmes has been broadly confirmed by stakeholders in combination with the results of the desk research. No significant overlaps, or risks for potential overlaps have been identified. The added value of the Programme and its overall complementarity with other EU initiatives carried out at EU level have been highlighted. In particular, as regards existing actions implemented by the ECB and the Europol, the results of both the survey and interview feedback confirm the complementarity of Europol’s work to the Pericles programme, with the largest share of respondents agreeing on complementarity. As regards national initiatives, the study confirmed the findings of the Mid-Term Evaluation, namely that Member States’ initiatives against euro counterfeiting were of a relatively basic nature, distant from Pericles 2020’s objectives and working methods, and therefore indicating the Programme’s full complementarity with these initiatives. The results of the stakeholders’ consultations and of the deep dives, carried out in the context of the evaluation study clearly revealed the complementarity between the Programme’s actions with the national initiatives, without any overlaps, with the Programme effectively filling a gap. 4.2.How did the EU intervention make a difference? The protection of the European single currency as a public good has a clear transnational dimension, and therefore euro protection goes beyond the interest and the responsibility of individual EU Member States, and even extends beyond the EU area. Thus, there is the need to sustain cross border collaboration. The Programme adds value in this area by deepening relations between Member States. The value of transnational actions organised by Member States generally increased, with actions unlikely to have materialised without the Programme. Pericles broadened and deepened the common actions across different countries and regions and was seen to be the driving force behind transnational knowledge-generating activities, such as conferences, which would not take place without it. Figure 3: Adopted measures and practices The study concluded that the delivered outputs and results achieved through the Programme are likely to be sustained over time, as available evidence suggests that Programme participants have adopted measures or practices to ensure the sustainability of delivered outputs and progress towards results. The study revealed a high involvement of participants in activities disseminating the knowledge and skills gained during the Programme. A significant number of people have benefitted from training workshops and presentations carried out after Pericles 2020 events. Regulatory changes have been introduced in third countries that lay the ground for using methods and practices acquired through the Programme’s actions. The majority of participants in the Programme's activities saw an evolvement in their role after participating in the Programme and an increase in their involvement in euro protection activities. In addition, staff involved in the Programme tend to continue to work for the same institution, with this degree of job stability suggesting longer term sustainability. The Commission shares the contractor's opinion that these points indicate a high level of strategic and institutional commitment to the continuation of euro protection activities linked to Programme activities. 4.3.Is the intervention still relevant? Whilst the ‘dark’ figure of counterfeit (the amount of unreported or undiscovered counterfeits) could, to a certain extent, complicate the ability precisely to determine the actual extent and nature of the current counterfeit problem, most stakeholders consider counterfeiting to be a problem in their country with an EU and international dimension as well. The relevance of preventing and combatting counterfeiting and related fraud is not questioned, and stakeholders confirmed the importance of continuing to tackle counterfeiting, and the continuing need for closer and more regular institutional cooperation and coordination. The role of criminal elements in third countries is perceived to be increasingly a threat as a lack in these countries of effective measures against counterfeiting of foreign currencies (such as the euro) could result in more circulation in the EU economy. The Union’s free movement of people and goods makes the EU an ‘attractive’ place for organised crime, including counterfeiting. The innovative forms of production and distribution of counterfeit euros (for example through the darknet) and the developments related to the digital euro that could open up new forms of “e-counterfeiting” constitute new emerging threats. In particular, experts from Italy, Spain and the Netherlands mentioned the importance of considering the possibility to extend the scope of the Pericles programme in order to protect the future digital currencies, such as the digital euro (if adopted), against counterfeiting. Depending on the architecture of these currencies, criminal groups could be interested in counterfeiting tokens used to pay with digital currencies or the digital euro units themselves. This is also confirmed by the ECB in its report on a digital euro in October 2020. Figure 4: Overall perception on counterfeiting as a problem

Figure 5: WordCloud of emerging threats/topics identified by survey respondents When asked about the relevance of contributing to the enhancement of the capacity to protect the euro among relevant institutions in EU Member States and third countries, stakeholders above all confirm the continuing need for closer and more regular institutional cooperation and coordination. The Programme’s relevance lies in the fact that Pericles brings stakeholders from different institutions and countries together, and in this way drives cooperation in tackling the problem. The need to enhance operational capacities and improve institutional and legal frameworks are seen to be most relevant in light of the cooperation with third countries. Stakeholders have emphasised that cross-border actions are needed in order to build trust and foster collaboration. Key participants in the Programme confirm that the Programme adequately addresses their needs. It is recognised that regarding the typologies of actions, the Programmes's set of instruments is well suited to the purpose. Conferences, workshops, trainings and staff exchanges serve different but complementary purposes and are generally highly appreciated by stakeholders. Studies are also considered useful, especially to inform on new developments, and the possibility provided by the Programme to finance studies in the field is unique and can drive innovations. The purchase of equipment for third country authorities is not well-known to all stakeholders, and while some expressed that their procedures for purchasing equipment are particularly difficult nevertheless it represents an important tool to strengthen the protection of the euro in high risk third countries by reinforcing the equipment at disposal of competent authorities. As regards the Programme’s target groups, key participants to the Programme indeed confirm that the right groups are targeted. 5.What are the conclusions and lessons learned? 5.1.Conclusions All findings converge towards an unambiguously positive overall assessment. From the evaluation study it is clear that Pericles 2020 has achieved both its general and specific objectives. It has received remarkable praise from stakeholders, and it is seen as the only programme that supports, on an EU and global level, the enhancement of the operational capacity of stakeholders involved in the protection of the euro, dissemination of best practices regarding the fight against counterfeiting, and essentially building trust between institutions across countries and regions. For all these reasons and due to the ever-evolving threats to the euro that counterfeiting poses, there is a continued need for Pericles actions. Overall, stakeholders interviewed and surveyed perceive euro counterfeiting as a problem within their countries and as a phenomenon that crosses Member State and EU external borders. In particular, counterfeiting activities in third countries are increasingly perceived to be a threat. The lack of effective measures in third countries against counterfeiting of foreign currencies (such as the euro) could result in more circulation of counterfeits in the EU economy. Secondly, the Union’s free movement of people and goods makes the EU an ‘attractive’ place for cross-border crime, such as counterfeiting. Moreover, the use of the dark web, movie money and digital currencies are highlighted as actual or potential threats. A new emerging threat area identified evolves around the possible development of the digital euro and the related risk of e-counterfeiting for example the replication of tokens possibly used for a digital currency. Finally, this evaluation finds that there is a continuing need for closer and more regular institutional cooperation and coordination. In addition, the groups targeted and actions implemented under the Programme align with the expectations of stakeholders. The Programme has been effective, as both its specific and general objective have been achieved. The Programme has been successful in delivering all the envisaged diverse types of activities with respect to the protection of the euro, whilst there is still some room for improvement in strengthening the efforts to establish contacts between EU and third country officials involved. It has also been effective in reaching the envisaged number of participants in actions. The Programme participants are more diverse when compared with the previous programme, and it has been effective in taking into account the multidisciplinary aspects of the fight against counterfeiting. Moreover, it became evident that the face-to-face aspect of the actions implemented under the Programme are vital for its effectiveness and it constitutes one of its main benefits. The overall coordination, management, and administrative structures have been positively assessed by the stakeholders and also the Programme’s beneficiaries did not report any significant efficiency-related issues in the implementation stage. The evaluation study indicates that the application procedure was considered demanding by applicants and non-applicants alike, but this issue is expected to be reduced by the digital grant system of Pericles IV. Moreover, the study considers that the Programme’s management costs are relatively high but they are well-justified. The external study also points outs that the Programme is very specific, which can only be fully maintained if it remains stand-alone and can offer tailor-made actions for specific objectives. Furthermore, the Programme has achieved a very high percentage of allocation as compared to the reference budgets. The outputs of the actions were largely delivered at a lower cost than what was initially envisaged. At the same time, the difference on average between the planned and actual grants creates challenges for the financial planning of the Programme and its pipeline of actions. The current co-financing setup is deemed appropriate.

The study also concludes that the Programme adds value, in particular in sustaining cross border collaboration by providing training and establishing networks. In addition, the Programme is seen to be the driving force behind transnational knowledge-generating activities, such as conferences, which would not take place without Pericles. Problems that Member States need to tackle in relation to counterfeiting cannot be addressed only at the national level. While operational work against counterfeiting would likely take place regardless of Pericles, the Programme is seen to broaden and deepen the common actions across different countries and regions. Lastly, the results achieved through Pericles 2020 actions and the improvements in institutional capacity resulting from these actions are likely to be sustained over time. Measures or practices to ensure the sustainability of delivered outputs and progress towards results have been adopted by participants. New methods used and the strengthening of operational capabilities were applied the most after participation in the Programme, with a high level of participants indicating involvement in dissemination activities. A significant number of people have benefitted from training workshops and presentations carried out after Pericles 2020 events. Overall, the majority of participants in Pericles 2020 actions saw a positive evolution in their role since participating in the Programme, with an increase in involvement in euro protection activities. Most of the staff participating or organising Pericles 2020 actions still work for the same institution, showing a high retention rate and thereby contributing to institutional capacity building. 5.2.Lessons learned Based on the findings and conclusions of the evaluation, key lessons learned have emerged. To begin with, as the problem of counterfeiting evolves with new tools and methods, it is important to carry out a continuous strategic and operational needs assessment, to ensure the continued relevance of the Programme. This could take the form of dedicated sessions on needs assessment during the ECEG meetings and dedicating a specific section on needs in the annual updates of the strategy. There are indications of the expansion of the scope of action of anti-counterfeiting authorities to cover also digital currencies, including the digital euro if introduced. Thus, there is a growing need to follow closely the developments related to the digital euro project, and to continuously assess the need to potentially expand the scope of future Pericles actions to include counterfeiting issues related to the digital euro. Based on the stakeholders’ feedback, it can be concluded that the face-to-face aspect of the Pericles actions is a crucial factor for the success of the Programme. Moreover, evidence shows that the larger the event, the bigger the risk of not meeting the target of expected participants. Hence, extra efforts are needed by Programme applicants to reach the targeted number of participants in large events. Furthermore, it can be observed that the prudent budgetary approach of Pericles applicants allows the over-commitment of the available budget by at least 15% more than the budget envisaged per annual programme, and that there is a very high diversity in the costs for Pericles events beyond the EU. The higher co-financing rate was useful in extending the geography of participation, but leads to the reduction of the number of Pericles actions. For this reason, it is advisable to keep the higher co-financing rate for limited cases. In addition, it is important to maintain a regular coordination with relevant DGs and other institutions as a way of ensuring complementarity and avoiding overlaps on counterfeit-related projects. Lastly, it can be concluded that keeping the focus on increased cooperation with third countries continues to be a valid objective of the Programme. The Pericles Programme has been continued through the Pericles IV Programme, for the period covering January 2021 – December 2027. The agreed programme budget is EUR 6.2 million. Its main novelty being the revised Key Performance Indicators (KPIs) that were modified to better reflect the results of the actions implemented under the Programme. More specifically, two of the KPIs were removed, namely the number of individuals arrested and the penalties imposed. The reason for their removal was that the linkage with Pericles 2020 actions appeared weak, as other external factors could influence the indicators. In addition these indicators primarily measured the repression (and not the prevention) aspect of the phenomenon and were difficult to measure. Finally, one quantitative indicator has been added, namely the number of competent authorities applying to the Programme. Two qualitative indicators have also been added to the KPIs for Pericles IV, namely the satisfaction rate of participants in the actions financed by the Programme and the feedback of participants that have already taken part in previous Pericles actions on the impact of the Programme on their activities in protecting the euro against counterfeiting.

Annex I: Procedural Information The final evaluation study of the Pericles 2020 Programme was led by DG ECFIN (PLAN/2020/8652). DG ECFIN has lead responsibility for the oversight, implementation and evaluation of the Pericles 2020 Programme. An Interservice Group (ISG) was established to steer the final evaluation study. The ISG was composed of representatives from DG ECFIN, DG JUST and SG. The ISG was responsible for finalising the evaluation roadmap, establishing the terms of reference for the appointment of an external contract and the quality assessment of each stage of the reporting process. In addition, the ISG undertook a significant portion of work by written procedure throughout the evaluation process. The ISG met on 2 occasions between May 2020 –March 2021. The chronology of the evaluation was as follows: Date Task 04/05/2020 Appointment of the Interservice Steering Group (ISG) including representatives of ECFIN, SG and JUST 15/07/2020 Evaluation and Fitness Check Roadmap 15/07/2020 Completion of the Consultation Strategy 19/01/2021 Service request sent to the contratcors under framework contracts TAXUD/2019/CC/148-149-150 11/02/2021 Offers received from contractors in responce to the ToR 24/03/2021 Evaluation committee report 30/03/2021 Award Decision by the AOSD 26/04/2021 Signature of the procurement contract 15/07/2021 ISG Kick-off meeting 29/07/2021 Inception Report 03/09/2021 Interim Report 01/12/2021 Draft Final Report 17/12/2021 ISG Meeting 08/03/2022 Final Report of the evaluation study Evidence used to inform the evaluation The study covered all types of actions under the Pericles 2020 Programme and considered the progress on the findings, conclusions, and recommendations of the Mid-term evaluation of the Programme. A wide range of data sources were used to gather evidence to carry out the evaluation study whilst the great majority of stakeholders were consulted through a targeted consultation strategy during the evaluation phase, since the Programme was addressed to very specialist stakeholders. The initial fact finding work consisted of different components. Each component gathered different forms of information. Desk Research The Desk Review involved the analysis of the following types of documentary sources. Analysis of Programme and Action Documents. Legal texts, AWPs, Working Programme Statements, ECEG meeting reports, Action Technical Reports, Annual Implementation Reports, Technical and Financial Reports, ToRs, Grant Agreements

Analysis of Other Documents. This group of documents include: ·Documentation (e.g. annual reports) on capacity building initiatives of EC, ECB, Interpol ·Documentation on operational activities of Europol, Eurojust, Interpol ·ECEG reports ·statistics published by ECB, ETSC, Europol reports (OCTA/SOCTA) Interviews Interviews were carried out with four categories of intervieweess, namely: Member State competent authorities, applicants and beneficiaries of the Programme, participants in the actions financed under the Programme, EU Institutions and international partners. All interviews were carried out on the basis of structured questionnaires developed during the Inception Phase and agreed upon with DG ECFIN. There were several categories for interviwees, who received different questionaires prepared for each specific category. Input from above 40 interviewees was received. Online Survey The Survey of Participants was targeted at the individuals who had taken part in Pericles 2020 actions committed for the Programme’s entire period (2014-2020). The survey was primarily aimed at collecting information on the effectiveness of Pericles 2020 actions. The main themes investigated included: (i)overall assesment of the programme; (ii)extent to which counterfeiting is considered a problem; (iii)Results of Programme participation, (iv) dissemination of results. The Survey of Participants was conducted through an online questionnaire, implemented via the web, using EU Survey. In order to incentivise participation, the questionnaire was relatively short, mostly consisting of closed questions and was made available in four languages; English, French, Italian and Spanish. The survey was carried out over the September-october period. A total of 148 individuals participated in the survey.

Deep Dives The desk research and the interviews also involved deep dives into the type of actions implemented under the Pericles 2020 programme for two selected countries, namely Italy and Spain as the biggest beneficiaries in terms of budget allocated and the number of actions implemented, while keeping in mind a balanced distribution of the type of actions implemented. Notably, these are among Member States most affected by euro counterfeiting, and the selection was deemed appropriate by the interviewees in the Inception phase. The deep dives covered a set of 10 implemented activities (4 for Italy, 3 for Spain, and 3 actions implemented by the Commission) under the three types of actions eligible for financing under Pericles 2020. The objective of the deep dives was to provide key information for several criteria, in particular, the evaluation of efficiency, coherence and EU added value. The information was gathered by a review of the following four main types of documents: (i) the grant applications submitted by CNA (inclusive of the proposed budgets); (ii) the grant agreements with CNA and the procurement contracts for the direct actions; (iii) the Technical Reports summarising the results achieved, (iv) the Financial Reports summarising the costs incurred; and (v) evaluation forms (where applicable). External expertise An external contractor was engaged to undertake the supporting work of the evaluation study; the work of the contractor included the development of a comprehensive evaluation framework that sought to gather and analyse a wide range of data from desk research and a detailed and extensive consultation strategy.

Annex II. Methodology and Analytical models used An Inter-Service Group (ISG) was established and a third party contractor was engaged to undertake the work of the evaluation. The specific contract for the evaluation study became effective in April 2021 and expired in March 2022. The study was carried out in line with the principles commonly applied for the evaluation of EU initiatives, as enshrined in the Better Regulation Guidelines. The study was performed in four stages: 1) structuring; 2) data collection; 3) analysis; and 4) drafting and reporting phase. The first stage, structuring, represents the Inception phase, which resulted in the Inception report. Data collection focused on collecting quantitative and qualitative data through the use of different tools. Analysis refers to the iterative steps used to process data. This means verifying, organising, integrating and extracting data to assess and describe facts, detect patterns and develop explanations. Reporting concerns the final stage in the evaluation approach, where answers to the evaluation questions and lessons learned are provided. The most substantial part of the work was carried out during the data collection and analysis phase. The methodological approach was discussed at the kick-off meeting with the external contractor and it was endorsed by the members of the ISG. Data collection (Field work) Data collection consisted of five components, namely: desk research, interviews, online survey, deep dives, and expert group meetings I)Desk Review

It involved the analysis of different types of documentary sources, namely: (i) various Programme and Action Documents ((ii) other relevant documentation and documentary resources from other EU programmes and (iii) ECEG reports. II)Stakeholder Consultations and their results Stakeholder consultations were used to ensure efficient data collection, and to ensure a representative sample of consulted stakeholders to gather evidence to answer the evaluation questions. A combination of data collection and validation activities were used throughout the process guaranteeing regular updates and buy-in from the main relevant stakeholders. The stakeholder consultation tools included: interviews, an online survey, and consideration for the results of the feedback on the evaluation roadmap. III)Semi- structured interviews Interviews in a semi-structured format were used as a means to add to the knowledge gathered through desk research. They were useful to validate findings in a one-on-one setting. A wide spectrum of stakeholder groups was included, with different stakeholders groups having been interviewed, namely i) EU institutions and international organisations, ii) Implementers of the Pericles programme, iii) Unsuccessful applicants, iv) Non-applicants, and v) Supported authorities in third countries. For the different stakeholder groups different questionnaires were prepared, and in line with the research design, overall, input from above 40 interviewees was received (including in writing via filled-in questionnaires). There was a strong consensus among different stakeholders on the merits of the Programme. In this sense, the input received was largely homogenous, and the findings based on the interview input are robust. IV)Online Survey The online survey was carried out during the period September-October 2021, implemented by the web, using EU Survey. It took the form of short and straightforward questionnaires, to ensure a decent response rate. The online survey targeted participants of activities implemented under Pericles 2020 in the period 2014 – 2020. A list of recipients was prepared based on documentation from the Pericles programme, which was confirmed by the Commission. The survey was available in different languages, namely i) English, ii) Italian, iii) Spanish, and iv) French to suit all target groups. A total number of 148 responses was received, with a distribution that is overall aligned to the Programme participants: law enforcement authorities – 52%, judiciary authorities – 14.2%, monetary authorities – 21.6%, private entities and banks – 10.8%. Similarly to the interview input, the online survey results also delivered homogenous and positive feedback on the Programme merits. Moreover, both stakeholder consultation tools (interviews and online survey) yield similar results on the key topics. V)Deep dives The desk research and the interviews also involved deep dives into a set of implemented activities under the three type of actions eligible for financing and implemented under the Pericles 2020 programme for two selected countries, namely Italy and Spain because they are the biggest beneficiaries in terms of budget allocated and the number of actions implemented. Notably, these are also among Member States most affected by euro counterfeiting, and the selection was deemed appropriate by the interviewees in the Inception phase. The deep dives aimed to provide key information for several criteria, in particular, the evaluation of efficiency, coherence and EU added value. The information was gathered by a review of the following four main types of documents: (i) the grant applications submitted by CNA; (ii) the grant agreements with CNA and the procurement contracts for the direct actions; (iii) the Technical Reports summarising the results achieved, (iv) the Financial Reports summarising the costs incurred; and (v) evaluation forms.

VI)Expert group meetings The contractor attended and used information during the data collection phase by also attending one meeting of the Euro Counterfeiting Expert Group meeting and two of the Programme’s actions. Limitations, considerations for future evaluations The evaluation process followed the envisaged steps; the interviews, deep dives, and survey were conducted as planned; and the data gathering and analysis proceeded in line with the agreed intervention logic and evaluation matrix. No major limitations to the study were identified. Some considerations emerged that could potentially limit the study during the evaluation process, namely, that the evaluation also considered actions performed in 2021 (actions the performance of which had to be postponed due to the COVID-19 pandemic), the number of responses to the online survey was lower than the responses to the survey used in the Mid-term evaluation, the absence of targets of the indicators for outputs, the fact that drawing a causal link between the Programme and the number of counterfeits detected and illegal workshops dismantled was challenging due to the lack of evidence, and the fact that comparisons of unit costs with other programmes was challenging because of the diversity of Pericles actions. However, all these considerations were addressed and did not affect in a negative way the robustness and reliability of the conclusions. External reviewers Two independent experts assured the quality of the evaluation study, as foreseen by the ToR. The reviewers provided a critical review of the methodology and approach, the scale and scope of the activities to be undertaken as well as they provided a critical review of the draft final report, the validity, relevance and adequacy of the findings, conclusions and lessons learnt. The comments made by the external reviewers during the different phases of the evaluation study were taken into consideration by the external contractor. Annex III. Evaluation matrix and, where relevant, Details on answers to the evaluation questions (by criterion)

The methodology for the evaluation study conducted by the external contractor was based on two key elements: the evaluation matrix and the fieldwork tools. The evaluation matrix as shown below, included judgement criteria, indicators and sources of information for each respective evaluation question, grouped per evaluation criterion. Evaluation Matrix: Relevance (EQ #1 – 4) Evaluation Questions 1. To what extent is there a need to protect the euro against counterfeiting and related fraud? 2. To what extent is the Programme relevant to meet this need and any possible evolution of this need? 3. To what extent are the specific objectives of the Programme (i.e. enhance institutional capacity) relevant to achieve its overall objective (i.e. euro protection)? 4. To what extent are the Programme actions and target groups relevant to achieve its overall and specific objectives? What do we want to measure? The analysis of relevance of the Pericles 2020 Programme assesses the rationale of the programme in relation to its objectives (EQ #3), the defined actions and, target groups (EQ #4) and the problems to be addressed (both initial needs and actual needs or ‘new threats’) (EQ #1, EQ #2), as defined in the Regulation 331/2014 and other relevant programming documents (e.g., Commission Decisions for the AWPs, Pericles 2020 Strategy, etc). The analysis can be divided into two main lines of inquiry. The first line of inquiry will assess: (1) the actual extent and nature of the current euro counterfeit problem and how it has evolved since the Programme was launched (i.e. Is the initial need still relevant today?) (EQ #1); and (2) whether the programme’s objectives (specific and general) remain relevant to address actual needs, including ‘new threats’ (EQ #2). The aim is to determine whether the rationale underlying the Programme in general and its specific objectives and priorities in particular, are still appropriate and are expected to remain appropriate. The second line of inquiry will examine the logical design of the Programme and its strategic orientations (EQ# 3, EQ#3). This analysis can be cascaded in two steps. The first step will assess the alignment between the specific objectives of the programme (i.e. enhanced institutional capacity to protect the euro within relevant institutions) and the general objective of the programme (i.e. euro protection. The second step will assess the alignment and relevance of the various types of actions (Article 8) and target groups (Article 7) with the general and specific objectives. Judgement criteria Indicators JC1: The risk of euro counterfeiting and related fraud remains a threat §Nature and severity of actual euro counterfeiting problem across countries (baseline indicators described in section 3.2.2) §Counterfeit euro coins detected in circulation (ETSC reports on euro counterfeiting) §Nature and severity of ‘new threats’ resulting from innovative forms of counterfeiting production and/or distribution (e.g. distribution on internet) §Illegal mints discovered (ETSC report) §Stakeholders’ perceptions of threat severity JC2: The objectives of the Programme are covering the identified risks of euro counterfeiting and related fraud remains a threat §Match between Programme objectives (and their scope) and the existing/emerging threats §Stakeholders assessments of the extent to which the Programme is relevant to meet the current/emerging needs JC3: There is a continued need to increase the institutional capacity to protect the euro against counterfeiting and related fraud §State of the legal and institutional framework in the countries considered ‘at risk’ §Status of operational capabilities to protect the euro against ‘new threats’ (production and/or distribution) §Stakeholders’ assessment of capacity building needs §Stakeholder motivations to participate in the Programme (e.g., improved understanding of euro counterfeiting issues, establish contacts in EU and non-EU countries, learn best practices, acquire practical skills, etc) JC4: The various types of actions and stakeholder groups correspond to actual and perceived needs and Programme objectives §Strength of the logical links between needs, actions, stakeholder groups, and objectives §Stakeholders perceptions regarding the relevance of the target groups §Stakeholders perceptions regarding the usefulness of different types of actions to addressed needs (initial needs and actual needs / new threats) §Stakeholder views regarding appropriateness of actions targeting weak legal and institutional frameworks in ‘at risk’ countries Methods and approach The analysis will rely primarily on a qualitative content analysis of the relevant programming documents, complemented with consultations with CNA’s, third-country authorities and programme participants. Perceptions of EU and international stakeholders will be collected to triangulate the results. The analysis will sketch the actual and perceived current needs, as well as the evolution of needs over the period of implementation according to, inter alia, objectives set in the relevant programme and action documents, other documentary evidence and stakeholder consultations. The analysis will focus in particular on the adequacy of institutional and operational capacities to protect the euro, taking into account the differing levels of capabilities and needs across countries and the degree to which there is a collective or targeted need for continued capacity-building support. Four all four evaluation questions, we will conduct a mapping exercise to identify if there are: (i) gaps between initial and actual needs (i.e. the continued relevance of initial needs), (ii) gaps between actual needs / new threats and objectives set in the corresponding programming documents (i.e. continued relevance of the Programme); (iii) gaps between actions / target groups and objectives; and (iv) gaps between specific and general objectives. Sources §Programme and Action Documents (e.g. legal texts, AWPs, Working Programme Statements, ECEG meeting reports, etc.) §Other documentary sources (e.g. statistics published by ECB, ETSC, Europol reports (OCTA/SOCTA), etc.) §Interviews with CNAs (Implementers, Unsuccessful applicants, Non-applicants) and supported third country authorities §Interviews with EU and other international institutions §Survey to programme participants §Deep dives §ECEG meetings Evaluation Matrix: Effectiveness (formerly EQ # 8 – 10) Evaluation Questions 5. To what extent are the operational objectives (exchange of best practices/training, provision of studies, and purchase of equipment) of the Programme achieved? 6. To what extent is the specific objective (enhance the capacity to protect the euro among relevant institutions in EU MS and third countries) of the Programme achieved? 7. To what extent is the general objective (prevent and combat euro counterfeiting and related fraud, contributing to enhancing trust in the euro, and sustainable public finances) of the Programme achieved? What do we want to measure? The assessment of effectiveness looks at the extent to which the Programme’s actions delivered the intended outputs, results, and impacts. It explores evidence of the expected and achieved contribution of the implemented actions to the operational, specific, and general objectives of the Programme in terms of improved institutional capabilities and impact on euro protection operations. Thus, the evaluation questions are structured along the different levels of objectives of the Programme and cover issues like enhanced operational capacity, increased awareness, and improved cooperation. Judgement criteria Indicators JC5: The outputs of the Programme were delivered §Number of staff trained §Best practices/information on emerging threats was disseminated §Number of conferences and workshops §Number of studies performed §Inter-institutional contacts were established §Purchased equipment JC6.1: The results of the Programme were achieved JC6.2: Stakeholders agree that the Programme contributed to the achievement of the results §Enhanced operational capacities of staff §Improved tools and methods §Improved institutional and legal frameworks (e.g. centres established, legislation introduced/ratified) §Greater awareness of the threat of euro counterfeiting §Closer and more regular institutional cooperation and coordination (with third countries) §Stakeholders’ assessment on contribution of Pericles actions to enhancing the institutional and operational capacity of relevant authorities in both EU Member States and third countries §Stakeholders’ perceptions on the role and impact of other external factors in contributing to enhanced capacities §Concrete examples of Programme actions contributing to improvements in the above indicators JC7.1: The expected impacts of the Programme were achieved JC7.2: Stakeholders agree that the Programme contributed to the achievement of the impacts §Number of counterfeits detected and their evolution over time §Illegal workshops dismantled and their evolution over time §Stakeholders’ assessment on contribution of Pericles actions to enhancing the institutional and operational capacity of relevant authorities in both EU Member States and third countries §Stakeholders’ perceptions on the role and impact of other external factors in contributing to enhanced capacities §Concrete examples of Programme actions contributing to improvements JC7.3: Contribution to support smart, sustainable and inclusive growth and to improve the efficient functioning of the Monetary Union §Stakeholders’ perceptions on the possible contribution to ‘broad economic’ effects §Stakeholders’ perceptions on the role and impact of other external factors. Methods and approach In answering the evaluation questions listed in the Tender specifications, the study will apply a standard, contribution analysis approach that draws primarily on qualitative evidence, complemented with quantitative data as much as possible. Relevant documents to be consulted include, inter alia: (i) awarded grant applications submitted by CNAs and the specifications prepared by DG ECFIN for the direct actions; (ii) Technical Reports summarising results achieved; (iii) the Programme’s strategic and implementation documents, including annual implementation reports; and (iv) previous evaluations and impact assessment documents. The first category will be used to refine our understanding of the action-specific theories of change in terms of how the selected activities were expected to lead to the desired outputs and ultimately contribute to the intended outcomes. The document analysis will be complemented with stakeholder consultations (interviews with CNA’s and third-country authorities and the survey to programme participants) to provide detailed information on the achieved outputs/results/impacts from specific actions. In particular, questions will be formulated to collect factual information and insights regarding the concrete and tangible ways in which these outputs (i.e. contacts that were developed and/or the knowledge, skills and information acquired through participation in the Programme’s actions) have been put into practice at both the personal and institutional level and how these changes have contributed to enhanced institutional capacity to protect the euro against counterfeiting (e.g. through seizure of illegal mints, strengthening of procedures in commercial banks, identification of smugglers of counterfeited euros, etc). To assess the relationship between the delivery of capacity building outputs and the effectiveness (or results) of operational counterfeit repression activities, the analysis of key performance indicators will be supplemented with qualitative inputs and analysis derived from both documentary sources (e.g. SOCTA) and primary sources (interviews and survey). The data on performance indicators will be used to formulate targeted interview and survey questions around identifying whether particular operational activities in selected situations can be directly or indirectly linked to improvements in institutional capacity resulting from participation in specific Programme actions (i.e. contribution analysis). External enabling and/or hindering factors will also be explored during the interviews. Sources §Action and Programme Documents (Action Technical Reports, Annual Implementation Reports) §Interviews with CNAs, third-country authorities, EU / international institutions §Participants survey §National statistics / EU reports (e.g., Europol) §ECEG reports §Crime statistics (national and EU level) §Deep dives §ECEG meetings Evaluation Matrix: Efficiency (formerly EQ # 5 – 7) Evaluation Questions 8. To what extent do the coordination (with MS, ECB, Europol and other stakeholders), management and administrative structures currently in place ensure efficient use of resources in the achievement of the Programme outputs, results and impacts? 9. To what extent are the actions and outputs of the Programme delivered at a reasonable cost? 10. To what extent is the co-financing rate appropriate? What do we want to measure? The analysis of efficiency examines the extent to which the established coordination, management and administrative structures enable the Programme to deliver the intended outputs and contribute to outcomes in an efficient manner (EQ #8); the appropriateness of the co-financing rate for actions (EQ #10); and the overall cost-effectiveness of the implemented actions and delivered outputs (EQ #9). The management and coordination structures comprise the programming documents (e.g. Pericles 2020 Strategy, AWPs, etc) and ECEG meetings. The study will assess the extent to which the established structures and administrative procedures have facilitated or otherwise hindered the implementation of high-quality and priority actions. The analysis will focus on the extent to which the established structures enable the Programme to deliver the planned outputs, thereby contributing to the intended outcomes in an efficient manner. Judgement criteria Indicators JC8.1: The existing management and coordination structures and procedures ensure efficient use of resources §Stakeholders’ assessment of relevance and usefulness of information communicated in programming documents §Support provided during the various stages of implementation by the unit managing the programme §Costs associated with programme management (Pericles, similar programmes) JC8.2: The established administrative procedures ensured efficient use of resources §CNAs’ perceptions regarding clarity of procedures and quality of assistance provided by ECFIN §Administrative burdens borne by CNAs (proposal preparation, reporting) §Administrative burdens borne by beneficiaries in other, similar EU-funded programmes (to the extent available/comparable) §Stakeholders’ motivations for applying or not applying for funding (linked to management / administrative procedures) §Proportionality of administrative costs to delivered outputs (burdens relative to action budgets and/or achieved outputs) §CNAs’ assessment of cost-benefit ratio, areas for improvement in reducing / simplifying procedures §CNAs’ views regarding areas for improvement in the administrative and financial arrangements JC9: The Programme outputs are achieved in a cost-effective manner §Unit cost of Pericles actions per participant for specific outputs §Unit cost of EU contribution per participant for specific outputs §Unit costs (total and EU contribution) for specific outputs / outcomes in other, similar EU-funded programmes (to the extent available / comparable) §Difference between budgeted and actual costs §Personnel costs of CNAs as implementers §Cost-benefit ratio for sample selection of actions’ outputs / outcomes JC10.1: The mobilization of co-financing does not discourage participation JC10.2: Different co-financing rates could broaden (or shrink) the pool of interested applicants §Co-financing rates applied to actions §Number, quality and focus of applications for CNA-implemented actions at different EU contribution rates §Stakeholders’ views on the ability to mobilise co-financing and incentivization of top priority actions §Data on co-financing rates in other, similar EU-funded programmes Methods and approach The assessment of the efficiency of established coordination, management and administrative structures will rely primarily on sources collected via interviews with CNAs, complemented with a review of financial reports estimate the quantitative costs associated with programme implementation. Costs associated with the management of the Pericles programme will be quantified and monetised on the basis of DG ECFIN Annual Reports, according to the reported Full-Time Equivalent (FTE) staff requirement for programme implementation and management. The overall programme management costs will be compared against similar programmes to establish the relative cost-efficiency of the Pericles Programme. Focused interview questions will be formulated around the allocation of CNA staff resources (staff time spent in FTE) for the preparation of Pericles actions (proposal preparation) and compliance with reporting requirements, as well as perceptions of the cost-benefit ratio. Staff costs will be quantified and monetised by multiplying the amount of time spent (FTE) in preparing proposals and complying with reporting requirement by the average daily labour cost as indicated in available the Financial Reports summarising costs incurred. The proportionality of administrative burdens borne by CNAs will be analysed by comparing the administrative cost estimates with the average budgets of the different types of actions implemented. The findings will be complemented with stakeholder perceptions on the cost-benefit ratio and documentary evidence from Technical Reports summarising the results achieved. Information on the budgetary outlays to fund Pericles actions will be collected from the relevant Action Documents (i.e., grant applications and/or Financial Reports summarising costs incurred). Stakeholder perceptions regarding the cost-benefit ratio of funded actions will be collected via interviews and the survey to programme participants, supplemented with concrete examples of tangible results (quantitative and qualitative) that can be linked to participation in Programme actions. The results will additionally be compared against the corresponding values of other, comparable EU-funded programmes that support public administrations using similar implementing modalities. The assessment of the co-financing rate will rely on a combination of interviews with CNAs (Implementers, Unsuccessful Applicants and Non-Applicants) and documentary sources. The latter will include a review of the grant applications submitted by CNAs (including proposed budgets) and corresponding Tender specifications prepared by DG ECFIN in order to identify potential variations in the number and types of applicants, the quality of applications submitted and the focus on top priorities / themes based on the co-financing rate applied. The results will additionally be compared against data on co-financing rates in other, similar EU-funded programmes. Sources §Action documents (Technical and Financial Reports, ToRs, Grant Agreements, etc.,) §Documentation from other EU programmes (Implementation / Financial reports, Evaluations, etc.,) §Interviews with CNAs (Implementers, Unsuccessful applicants, Non-applicants) and supported third country authorities, Interviews with EU and other international institutions §Survey to programme participants and Deep dives, and ECEG meetings Evaluation Matrix: EU Added Value (EQ # 11) Evaluation Questions 11. To what extent does the Programme provide EU added value, within the meaning of Article 2 of the Regulation? What do we want to measure? The assessment of EU Added Value looks at the benefits of the Programme in terms of promoting transnational and multidisciplinary cooperation. As laid down in Article 2 of the Regulation, such cooperation shall contribute to strengthening euro protection through the exchange of best practice, common standards and joint specialised trainings. In addressing this evaluation question, we will investigate EU added value of the Pericles Programme from three perspectives: §The added value resulting from the intervention(s) compared to what could be achieved by the Member States at national level. The study will focus in particular on the programme’s ability to support collective forms of international cooperation that are beyond reach of individual national authorities. §The analysis will investigate the feasibility for MS to perform the same types of transnational activities on a comparable scale in a hypothetical scenario without the Programme. We will focus in particular on the role and importance of Pericles 2020 financial support and the importance of an EU programme in providing visibility for the action and attracting participants from different countries. §The extent to which the needs addressed by the intervention continue to require action at EU level. This line of investigation will draw from the findings on the continued relevance of the programme (EQ #1, EQ #2) with particular focus on the role and relevance of the programme in creating and fostering a transnational network and promoting cross-border cooperation. Judgement criteria Indicators JC11.1: The EU Member States would not be able to implement transnational actions without Pericles §Financial resources dedicated to comparable euro protection activities at Member State / third country level §Stakeholders’ views regarding possible developments, if the Programme was not available §Stakeholder perceptions / assessment of the ability to implement similar transnational actions without Pericles (role / impact of EU financial support and visibility of EU programme) JC11.2: The programme increased the ability of MS to engage with partners outside of the EU on issues related to counterfeit prevention and combating §Stakeholders’ views on the importance of the programme in triggering transnational actions (and feasibility of such actions in the absence of financial support) §Stakeholders’ assessment on the importance of a transnational approach to enhance euro protection Evaluation Matrix: EU Added Value (EQ # 11) Methods and approach The approach will mostly rely on interviews with CNAs, third-country authorities and EU institutions, as well as the survey to participants. Interviews will provide critical insights on the more qualitative benefits of transnational cooperation activities supported by Programme in relation to strengthening euro protection through the creation of a transnational network, promoting cross-border cooperation and fostering relationships with more challenging third countries (e.g. China, South America). Interviews will also provide expert judgement on how the programme compares to what could be achieved by Member States alone. Information collected from interviews will be complemented by a comparative assessment of the financial resources dedicated to comparable euro protection activities at Member State level. Sources §Answers to the previous evaluation questions §Interviews with CNAs and third-country authorities §Participants survey §Institutional websites / Financial reports of other programmes at national / EU level §Deep dives §ECEG meetings Evaluation Matrix: Coherence (EQ # 12 – 13) Evaluation Questions 12. To what extent have the coordination and cooperation mechanisms in place for the Programme ensured consistency and complementarity with other relevant EU programmes and activities? 13. To what extent have the coordination and cooperation mechanisms in place for the Programme ensured consistency and complementarity with existing actions implemented by MS, the ECB and Europol, with a view to achieving the overall objective of protecting the euro against counterfeiting? What do we want to measure? The evaluation of coherence looks at the consistency and complementarity of the Programme and the implemented actions with other relevant EU programmes and initiatives (EQ #12) and/or existing actions implemented at a national level or by the ECB and Europol (EQ #13) seeking to protect the euro against counterfeiting and related fraud. At the national level, related initiatives are likely to include generalised trainings on the authentication of banknotes. At the EU and international level, the assessment will consider, inter alia, §capacity building initiatives supported by the Commission, such as ISF-P (DG HOME), TAIEX and Twinning instruments (DG NEAR); §analytical and technical assistance support provided by entities such as the ETSC, ECB and CBCDG; and §operational and tactical assistance provided to law enforcement authorities by EU and international entities such as Europol, Interpol and Eurojust. A final topic to be investigated is the degree to which the established coordination and cooperation mechanisms have either helped to ensure, or alternatively hindered consistency and complementarity between the Pericles Programme and other relevant programmes and initiatives at national, EU and international levels. The analysis will focus on the role and effectiveness of coordination mechanisms in reducing and/or avoiding overlaps with other initiatives at the programming stage, during the selection of the actions to be implemented as well as implementation of specific actions. Judgement criteria Indicators JC12.1: Degree of complementarity / overlap with other EU / international euro protection initiatives JC12.2: Contribution of established coordination and cooperation mechanisms to achieved consistency / complementarity with EU / international initiatives §Nature, scope of capacity building, analytical and/or technical support initiatives implemented by EC, ECB, Interpol §Nature, scope of operational activities supported by Europol, Eurojust and Interpol §Stakeholders’ views on complementarity of Programme actions and other EU / international programmes and initiatives §Stakeholders’ assessment on the role and contribution of coordination and cooperation mechanisms (e.g. ECEG meetings) JC13.1: Degree of complementarity / overlap with national initiatives and operations JC13.2: Contribution of established coordination and cooperation mechanisms to achieved consistency / complementarity with national initiatives §Nature, scope of national initiatives and operations (e.g. training on authentication methods implemented by Central Banks or banking associations) §Stakeholders’ views on complementarity of Programme actions and initiatives implemented at national level §Stakeholders’ assessment on the role and contribution of coordination and cooperation mechanisms (e.g. ECEG meetings) Methods and approach The assessment of coherence will be based on a qualitative review of various documentary sources, including institutional websites, websites of other programmes and initiatives, such as ISF-P and the database of TAIEX events, as well as information gathered via stakeholders consultations (interviews with CNAs, third-country authorities, EU and international entities). CNA interviews will cover Implementers, Unsuccessful Applicants and Non-Applicants. Interviews with non-applicants will serve to mitigate potential selection bias among responses. Specifically, targeted questions will be formulated to assess the degree to which a lack of coherence and complementarity between Pericles actions and those at the national level was a factor behind CNAs’ decisions not to apply. Sources §Documentation (e.g. annual reports) on capacity building initiatives of EC, ECB, Interpol §Documentation on operational activities of Europol, Eurojust, Interpol §ECEG reports §Interviews with EU / International organisations §Interviews with CNAs, third-country authorities §Participants survey §Deep dives §ECEG meetings Evaluation Matrix: Sustainability (EQ # 14) Evaluation Questions 14. To what extent are the delivered outputs and results achieved (likely to be) sustainable? What do we want to measure? The evaluation of sustainability will assess the extent to which the outputs and progress towards the achievement of intended outcomes are likely to endure over time. Specifically, the assessment will investigate the measures and practices adopted by participating Member States’ and supported third countries’ CNAs to ensure that delivered outputs of the supported actions are implemented or otherwise institutionalised after support ends, and how such measures have been implemented and performed in practice. Measures to ensure sustainability might include: §formal or informal distribution of materials received through the programme action; §preparation of reports, memos, etc on the action; §delivery of a presentation at internal team meetings, workshops, etc; §delivery of a formal training course; §sharing of acquired contacts, information, knowledge, skills, etc. with colleagues through informal mechanisms or regular operational activities; or §or other forms of dissemination. Typical challenges to sustainability include: §insufficient mobilisation of national resources (financial support); §staffing shortages and/or high staff turnover; §political and/or legislative support; §stakeholder commitment; or §cultural barriers, such as language barriers. The assessment of both the current sustainability (based on existing established practices) and the likely future sustainability of delivered results will take into account these and other relevant barriers as part of the analysis. Judgement criteria Indicators JC14.1: Programme participants have adopted measures or practices to ensure sustainability of delivered outputs / progress towards results JC14.2: Adopted measures or practices to ensure sustainability of delivered outputs / progress towards results have been implemented in practice §Measures adopted to ensure sustainability of delivered outputs / progress towards outcomes (i.e. delivery of internal trainings, preparation of memos or reports, sharing of contacts / knowledge, delivery of presentations, etc) §Actual utilization of contacts developed and/or information / knowledge / skills acquired in practice §Stakeholders’ assessment on the role and impact of external factors in contributing to (or hindering) sustainability of results JC14.3: Involvement of participating organisations in euro protection activities has evolved (increased) §Intensity and quality of transnational coordination / cooperation activities §Stakeholders’ assessment on the role and impact of external factors in contributing to (or hindering) increased involvement in euro protection activities JC14.4: Level of strategic and institutional commitment to the continuation of euro protection activities linked to programme activities §Stakeholders’ experiences regarding the availability and allocation of resources (financial, human) to sustain the outputs / results achieved to date. §Stakeholders’ perceptions regarding prospects for future sustainability in terms of allocation of resources and strategic commitment, external enabling / hindering factors, etc. Methods and approach The assessment will rely primarily on evidence gathered through the interview consultations with CNAs and third country authorities and the survey of Programme participants, complemented with a review of Action documents to identify concrete measures that were envisioned by the actions to ensure sustainability of delivered outputs as needed. Building on the analysis of effectiveness, the approach takes as the starting point the collection of factual information and insights regarding the concrete and tangible ways in which the delivered outputs (i.e. contacts that were developed and/or the knowledge, skills and information acquired through participation in the Programme’s actions) have been put into practice at both the personal and institutional level and stakeholder perceptions regarding the likely sustainability of such practices in the mid- to long-term. The consultations will examine a mix of actions that have been implemented over the Programme duration, though with particular emphasis on a representative sample of actions that were implemented during the first half of the programme in order to understand how outputs and results have been sustained (i.e. continued to be utilised) over a longer timeframe. Targeted interview and survey questions will be formulated around identifying (i) the specific types of measures adopted by programme participants to utilise and further disseminate the delivered outputs, and (ii) how these measures and practices have been implemented in practice, and (iii) the degree to which there is evidence of the programme having contributed to increased involvement in euro protection activities among the Programme participants. Sources §Interviews with CNAs, third country authorities, EU institutions §Participants survey §Deep dives §ECEG meetings §Action Documents (proposals, Technical reports)