EUROPEAN COMMISSION Brussels, 6.4.2022 SWD(2022) 103 final COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Evaluation of the European Union Agency for Law Enforcement Training



based on Article 32 of the CEPOL Regulation (EU) 2015/2219 of the European Parliament and of the Council of 25 November 2015 on the European Union Agency for Law Enforcement Training (CEPOL) and replacing and repealing Council Decision 2005/681/JHA































































{COM(2022) 153 final}



Table of contents 1.Introduction 2.Background to the intervention 3.State of Play 4.Method 5.Analysis and answers to the evaluation questions 6.Conclusions

List of Abbreviations AFRIPOL African Police Cooperation Organisation APEC Association of European Police Colleges AWP Annual Work Programme CA Contract Agents CAAR Consolidated Annual Activity Report CARIN Camden Asset Recovery Inter-Agency Network CC Common curriculum CCA CEPOL Cybercrime Academy CCWP Customs Cooperation Working Party CEP CEPOL Exchange Programme CEPOL European Union Agency for Law Enforcement Training CKC CEPOL Knowledge Centre CNUs CEPOL National Units COSI Standing Committee on Operational Cooperation on Internal Security CSDP Common Security and Defence Policy EASO European Asylum Support Office ECTEG European Cybercrime Training and Education Group EEAS European External Action Service EJMP European Joint Masters Programme EJTN European Judicial Training Network EMCDDA European Monitoring Centre for Drugs and Drug Addiction EMPACT European Multidisciplinary Platform Against Criminal Threats ENP European Neighbourhood Policy ENTRi Europe’s New Training Initiative for Civilian Crisis Management EPEP European Police Exchange Programme ESDC European Security and Defence College EU European Union eu-LISA European Union Agency for the Operational Management of Large Scale IT System EUPST European Union Police Services Training Eurojust European Union Agency for Criminal Justice Cooperation EUROPOLL European Union Agency for Law Enforcement EU-STNA EU Strategic Training Needs Assessment EY Ernst & Young FRA Fundamental Rights Agency FRONTEX European Border and Coast Guard Agency FWP Framework Partner HENU Head of Europol National Unit HWP Horizontal Working Party on Cyber Issues IPA Instrument for Pre-Accession JHA Justice and Home Affairs JIT Joint Investigation Team ICU International Cooperation Unit KPI Key Performance Indicators LEEd Law enforcement education LETS Law Enforcement Training Scheme LEWP Law Enforcement Working Party MB Management Board MASP Multi Annual Strategic Plan (EMPACT) NCP National contact point OAP Operational Action Plan (EMPACT) OSCE Organisation for Security and Cooperation in Europe OTNA Operational Training Needs Analysis SPD Single Programming Document SNE Seconded National Experts TA Temporary agents WP Work Programme Member State Abbreviations AT Austria LT Lithuania BE Belgium LU Luxembourg BG Bulgaria LV Latvia CY Cyprus MT Malta CZ Czechia NL Netherlands DE Germany PL Poland DK Denmark PT Portugal EE Estonia RO Romania EL Greece SE Sweden ES Spain SI Slovenia FI Finland SK Slovakia FR France HR Croatia HU Hungary IE Ireland IT Italy Other UK United Kingdom 1. Introduction The European Union Agency for Law Enforcement Training (CEPOL) was founded in 2000 and its current legal mandate entered into force on 1 July 2016, resulting from Regulation (EU) 2015/2219 (hereinafter: the CEPOL Regulation). As per Article 3 of the Regulation, CEPOL’s mandate includes supporting, developing, implementing and coordinating training for law enforcement officials. The external study supporting this evaluation was conducted by a consortium of RAND Europe and Ernst & Young (EY). Purpose and scope As foreseen in Article 32 of the CEPOL Regulation and based on the Terms of Reference for the external evaluation, the contractors assessed in the evaluation study:

1. The effectiveness and efficiency, relevance, coherence and EU added value of CEPOL and of its working practices since 2015. 2. Lessons learned and recommendations to better respond to the challenges posed by the constantly changing environment, against the background of the current authorised financial and human resources for CEPOL. The evaluation also had to verify to what extent the provisions establishing the Agency are aligned with the Common Approach on Decentralised Agencies, looking in particular into the Agency's governance, internal structures and procedures and applicable administrative and financial rules. Another intention of the evaluation was to provide solid grounds for any possible decision concerning the Agency’s future mandate. The results of the evaluation are expected to be used by CEPOL, its Management Board, the Commission, the Council, and the European Parliament and other Justice and Home Affairs agencies in the future decision-making process regarding CEPOL’s functions and operations. The scope of the evaluation in terms of subject, timeframe, stakeholders consulted, and territory was: • Subject: The evaluation addresses the objectives of the Agency as established in Article 3 of the CEPOL Regulation 2015/2219 of 25 November 2015. In doing so, it covers: the working practices of CEPOL; the provisions establishing CEPOL; CEPOL’s governance, internal structures, procedures and administrative and financial rules; CEPOL’s internal (within the EU) and external (with third countries) activities; and how the Agency cooperates with other EU agencies and international organisations. • Timeframe: The evaluation covers the period 2015 to 2020. • Stakeholders: The following categories of stakeholders have been consulted for the evaluation: CEPOL staff; the CEPOL Management Board; CEPOL National Units (CNUs) and National Contact Points (NCPs); EU institutions; Justice and Home Affairs (JHA) agencies; European level organisations; International organisations; third countries; Framework Partners (FWPs); Training participants; and the general public. A full list of stakeholders consulted for the evaluation is provided in Annex II. • Territory: The evaluation covers CEPOL’s activities with all EU Member States (except Denmark ), third countries with which CEPOL has working arrangements, cooperation agreements and similar arrangements, and international organisations and EU agencies. 2. Background to the intervention Established as European Police College by Council Decision 2000/820/JHA, CEPOL became operational on 1st January 2001 and later an Agency in 2005. The revised CEPOL Regulation 2015/2219 of 25 November 2015 entered into application on 1st July 2016. As of that date, CEPOL became the European Union Agency for Law Enforcement Training (the acronym remained unchanged). The general objective of the new Regulation is to align CEPOL with the Lisbon Treaty and to give it the appropriate legal mandate to implement the Commission Communication on “Establishing a Law Enforcement Training Scheme” (LETS), as well as to improve the governance of CEPOL by seeking increased efficiency. The new legal framework has modified CEPOL’s mandate. It now includes a wider target group (all law enforcement agents as opposed to only senior police officers), a more active role in the delivery of training activities (the Agency only used to coordinate national academies), the task to identify EU strategic training needs, and a more prominent activity in the external dimension. CEPOL's mission is to provide training and learning opportunities to law enforcement officers on issues vital to the European police cooperation. Training covers topics ranging from leadership to law enforcement techniques and from EU cooperation to economic crime. Activities are designed to facilitate the sharing of knowledge and best practices and to contribute to the development of a common European law enforcement culture. CEPOL contributes to a safer Europe by facilitating cooperation and knowledge sharing among law enforcement officials of the EU Member States and to some extent, from third countries, on issues stemming from EU priorities in the field of security, notably from the EU Policy Cycle on serious and organised crime, now commonly know as EMPACT (European Multidisciplinary Platform Against Criminal Threats). Moreover, CEPOL assesses training needs to address EU security priorities. The agency’s annual work programme is built with input from its network and all concerned stakeholders, resulting in topical and focused activities designed to meet the needs of Member States in the priority areas of the EU internal security strategy. The Agency also supports the EU Neighbourhood Partners through a few capacity building projects, which cover in different formations the Eastern Partnership Region, the MENA countries, Turkey and the Western Balkans. CEPOL’s headquarters were relocated to Budapest (Hungary) as of 1st October 2014, at the initiative of the Member States, which rejected the European Commission's proposal of merging CEPOL and Europol in The Hague. The relocation from London to Budapest happened despite the European Commission's negative opinion. Description of the intervention and its objectives In accordance with the Better Regulation guidelines for evaluations of the European Commission, the evaluation examines the effectiveness, efficiency, relevance and coherence of the actions of CEPOL, as well as at the achieved EU added value of these actions. Table 1 below sets out the definition of each of these components. Table 1: Overview of cross-cutting evaluation components Evaluation criterion General description based on Better Regulation Guidelines Effectiveness This aim of the effectiveness criterion is to assess the extent to which the objectives of CEPOL (as described in the intervention logic, see Annex IV) have been achieved. The Better Regulation Guidelines specify that as part of this criterion the evaluation should examine why an objective has or has not been met as well as factors driving or hindering progress. Efficiency This criterion demands an assessment of the relationship between the cost of operating CEPOL and the changes it has led to (or contributed to) because of its activities. For this evaluation, the benefits of CEPOL’s activities cannot be quantified so the focus is on the costs of CEPOL’s activities and the efficiency of CEPOL’s operations. Efficiency is a measure of how resources/inputs (funds, human resources, expertise, time etc.) are converted into results. Relevance The relevance criterion examines the extent to which CEPOL’s activities are in line with the needs of Member States, the EU and its external stakeholders. It examines the relationship between the CEPOL’s work programs and the needs and problems that existed in relation to the need for coordination in law enforcement training. Coherence For CEPOL, this criterion assesses the extent to which activities and objectives of the Agency are: (i) consistent, logical and not overlapping internally, and (ii) seek to achieve common objectives with those of external stakeholders (i.e. Member States, the Commission, EU agencies and non-EU agencies). EU added value This criterion aims to identify the extent to which CEPOL has brought benefits in comparison with what exists at EU and national levels. The criterion also assesses the extent to which CEPOL has helped to improve the coordination of police training.

Source: Based on Tool #42: Identifying the evaluation criteria and questions . Intervention logic To aid the analysis, an intervention logic was created for this evaluation (see Annex IV) . This is the main tool to understand causal and logical links among the objectives of the Agency and the relevance, coherence and effects expected from work and activities. The intervention logic highlights CEPOL’s overarching objective, which is to support, develop, implement, and coordinate training for law enforcement officials. This is a crucial role that is intended to ensure law enforcement officials are equipped to protect human rights, prevent and fight serious crime and terrorism, and maintain public order. The specific objectives of CEPOL are to: ·Support Member States in the provision of training to law enforcement officials, in particular on elements of cross-border and European law enforcement cooperation; ·Support Member States in developing bilateral and regional cooperation through the delivery of multilateral training; ·Develop, implement and coordinate training on specific thematic areas; ·Develop, implement and coordinate the training of law enforcement officials when preparing for participation in EU missions and in capacity building activities in non-EU countries; ·Provide training to law enforcement trainers and supporting the development and exchange of best practice; ·Develop, upgrade and evaluate learning tools and methodologies that support the learning and development of law enforcement officials. Evaluation framework The methodological approach was derived from a careful analysis of the 33 evaluation questions set out in the terms of reference and presented in Annex V, and the production of evaluation grids separated by each of the five evaluation criteria. Building on the evaluation questions, the evaluation grids include: ·Judgment criteria: statements that need to be confirmed or not confirmed by the analysis. ·Analytical approach to be used to answer the evaluation question. The type of analysis proposed informed the type of information collected. ·Indicators: quantitative and qualitative measures that support the analysis. ·Data sources: quantitative and qualitative sources to feed the analysis and indicators. In this report, the evaluation criteria are addressed in the order of the Better Regulation Guidelines. The evaluation questions are provided in Annex V. Baseline and points of comparison This is the first evaluation carried out by the Commission; the previous evaluation was commissioned by CEPOL and covered the preceding five years. 3. State of Play

Description of the current situation The new legal basis resulting from the 2015 Regulation (EU) 2015/2219 established CEPOL as a European Agency. The Regulation was implemented following the European Parliament’s request for the development of a strong EU framework for judicial and police training and built on the work of the Stockholm Programme. The Regulation outlined a range of new tasks that included capacity building in third countries, promoting the mutual recognition of training between Member States and the assessment of EU initiatives. Moreover, the new legal basis broadened CEPOL’s remit in terms of external relations and target audience , and led to changes in its governance structure. The current objectives as outlined in the 2015 Regulation (Article 3) are supporting, developing, implementing and coordinating training for law enforcement officials. More specifically: ØSupport Member States in the provision of training to law enforcement officials, in particular on elements of cross-border and European-level policing. ØSupport Member States in developing bilateral and regional cooperation through the delivery of multi-lateral training. ØDevelop, implement and coordinate training on specific thematic areas. ØDevelop, implement and coordinate the training of law enforcement officials when preparing for participation in EU missions and in capacity building activity in non-EU countries. ØProvide training to law enforcement trainers and support the development and exchange of best practice. ØDevelop, upgrade and evaluate learning tools and methodologies that support the learning and development of law enforcement officials . CEPOL aligns its activities with relevant EU strategies and policies, such as the European Agenda on Security, focusing its learning and training outputs on threat areas such as terrorism, serious and organised crime, cybercrime and the smuggling of migrants . CEPOL has identified cybercrime as the key priority for the coming years, demonstrated by the launch in 2019 of the CEPOL Cybercrime Academy (CCA). These strategic areas also address the priorities outlined in the EMPACT . Another key task of CEPOL is to conduct the EU Strategic Training Needs Assessment (EU-STNA). This task emerged from an acknowledgment in 2013, following the establishment of the Law Enforcement Training Scheme (LETS), of the need to address the lack of a systematic process for identifying the evolving training needs for law enforcement in the EU. The Regulation identified that this was necessary to avoid duplication and ensure better coordination of training activities. The first EU-STNA was launched in 2017. This led to a consolidated list of 184 EU-level training needs across 21 thematic categories that were distributed among the Member States, with a request to prioritise and rank the training needs. CEPOL prepared a report based on these findings, which was later endorsed by the European Parliament. In addition to the EU-STNA, CEPOL also undertakes an Operational Training Needs Analyses (OTNA) on the key priority topics outlined by the EU-STNA. The aim of this exercise is to gain a detailed understanding of the profile of the officials to be trained, as well as the proficiency and urgency of the training to be delivered. In 2020, OTNAs were carried out in five areas: illegal immigration; trafficking in human beings; criminal finances; money laundering and asset recovery; and drug trafficking. Multiannual training portfolios are informed by the outcome of OTNAs. CEPOL is governed through a Management Board, made up of representatives from 26 Member States and the European Commission. The Chair of the Management Board is a representative of one of the three Member States that have jointly prepared the Council of the European Union’s 18-month programme. The board meets twice per year and works to adopt CEPOL’s annual and multi-annual work programming and its annual budget. Programming documents set out a number of ‘goals’ along with key performance indicators (KPIs) used to monitor future progress in consultation with the Member States. The Management Board also provides suggestions to improve CEPOL’s internal processes and compiles the Consolidated Annual Activity Report (CAAR) for each financial year, which assesses the delivery of activities against requisite work programmes and KPIs. CEPOL is headed by an Executive Director who is accountable to the Management Board. The responsibilities of the Executive Director include, but are not limited to, the day-to-day administration of CEPOL; making proposals to the Management Board as regards the establishment of CEPOL's internal structures; implementing decisions adopted by the Management Board and preparing the draft multi-annual programming and annual work programmes and submitting them to the Management Board. Between 2016 and 2018 CEPOL’s operations services were reorganised, with a dedicated International Cooperation Unit (ICU) now in place. In addition, e-learning provision and research and analysis sectors were added to an updated Training and Research Unit and legal functions became incorporated into the Director’s Office, rather than under corporate services. The Director’s office also combines external relations, communications, and quality management functions, among others (see Figure 1 below). CEPOL has dedicated CNUs in every Member State to provide information and assistance to law enforcement officials wanting to participate in CEPOL’s activities. CNUs also support CEPOL’s operations and influence the annual work programming. Figure 1: CEPOL’s Management and Governance Structure Source: CEPOL Single Programming Document 2020-2022. The aforementioned CEPOL Management Board oversees the Agency’s governance, risk management and internal control practices. This process takes place through the use of tools such as progress reports (including performance indicators and audit recommendations), annual activity reports and internal/external audits. CEPOL’s internal organisational structure is composed of two departments, which are coordinated by the Head of Operations and Head of Corporate Services, as well as three units under the supervision of a head of unit (Head of Training and Research Unit, Head of International Cooperation, Head of Finance, Procurement and Travel). At the level of units, all managers are accountable for the achievement of objectives and operational and internal control and comply with the reporting arrangements within the Agency. The main responsibility of the CEPOL units is to manage risk and controls on a daily basis. CEPOL also uses an Internal Control Framework (ICF) to support its decision making to achieve objectives and sustain operational and financial performance, respecting rules and regulations. CEPOL does not have a dedicated managerial position in charge of risk management and internal control; this is a shared management responsibility. In addition to the ICF, CEPOL has followed the Quality Management System Standard ISO 9001:2015, since February 2017, when the certification was obtained. Staffing In 2020, CEPOL’s staff numbered 84 in total (see Table 2). According to CEPOL’s 2020 Establishment Plan, the Agency aimed to have 33 Temporary Agent (TA) positions in 2020, complimented by 19 Contract Agent (CA) positions and 7 Seconded National Expert (SNE) positions. Fluctuations in staffing levels can be attributed to varying needs for the delivery of EU projects funded from grant agreements, such as the Western Balkan Financial Investigations project concluded in March 2020. Table2: Number of CEPOL staff, 2015 – 2020

Number of staff 2015 2016 2017 2018 2019 2020 Temporary agents (TA) 26 25 30 32 30 31 Contract agents (CA) 9 18 17 30 32 46 Seconded national expert (SNE) positions 10 6 6 5 7 7 TOTAL 45 49 53 67 69 84 Source: CEPOL Consolidated Annual Activity Reports (CAAR) (2015 – 2020). Notes: Staff numbers are those at the end of each year. The most recent CAAR, published in 2021, noted that the staff turnover at CEPOL continued to be relatively high. Consequently, the Agency used a number of interim staff and SNEs to fill in for staff absences, until recruitment of statutory staff was completed and continued the reclassification of staff and functions, as well as maintaining teleworking and flexitime arrangements. Budget and finances CEPOL’s activities are funded through an annual subsidy from the European Commission, as well as grant agreements specifically for activities delivered through CEPOL’s ICU . As shown in the table 3 below, CEPOL’s overall revenue increased gradually over the period 2015 to 2020, mainly due to a steady increase in the revenue from grant agreements. There was no significant increase in CEPOL’s EU annual subsidy in the period before and after the new legal basis came into effect. That is, in 2012, 2013 and 2014 the EU annual subsidy was €8.45 million, €8.45 million, and €8.35 million respectively. At the time of the new legal basis, in 2015 and 2016, the revenue from the EU subsidy remained stable. The subsidy increased to over €10 million in 2018, decreased again to €9 million in 2019 and increased again to over €10 million in 2020. Table 3: CEPOL’s annual budget (‘000 Euro) 2015 2016 2017 2018 2019 2020 CEPOL's voted budget Operational expenditure (‘000 Euro) 4,429 4,812 5,173 5,897 4,620 5,618 Operational expenditure (% of total) 52% 56% 56% 57% 50% 54% Buildings and equipment (‘000 Euro) 495 481 514 596 423 453 Buildings and equipment (% of total) 6% 5% 5% 6% 4% 4% Staffing (‘000 Euro) 3,548 3,347 3,593 3,923 4,265 4,368 Staffing (% of total) 42% 37% 39% 38% 46% 42% SUB-TOTAL 8,471 8,641 9,280 10,417 9,308 10,439 Grant agreements CT MENA 300 1,244 907 1,816 - - CT2 - - - 1,946 2,381 1,370 FI - - - 1,217 1,165 1,178 CT inflow - - - - 3,750 3,750 Euromed Police - - - - - 1,504 SUB-TOTAL 300 1,244 906 4,979 7,296 7,802 GRAND TOTAL 8,771 9,885 10,187 15,396 16,604 18,241 Proportion of total comprised of grant funding 3% 13% 9% 32% 44% 43% Source: CEPOL’s CAARs (2015 - 2020). CEPOL’s activities are organised as part of its annual work programme (AWP), which is established at the outset of each year and agreed upon with the Member States and the Commission. The programme contains detailed objectives, expected results and performance indicators. For example, CEPOL routinely monitors the number of participants for each activity and measures how this has changed from year to year, against its annual targets. The AWP also features a description of the actions to be financed and an indication of the financial and human resources allocated to each action, in accordance with the principles of activity-based budgeting and management. CEPOL offers a range of activities for the learning and development of law enforcement officials in the Member States and third countries. Table 4 provides data on the number of training activities delivered by the Agency over the period 2015 to 2020 by type of activity and year. While CEPOL does itself deliver some activities, including residential and online learning, CEPOL implements most of its activities through the allocation of grants to Framework Partners identified through restricted Call for Proposals that are published every four years. Table 4: Number of training activities by type and year 2015 2016 2017 2018 2019 2020 Residential 85 87 105 103 100 4 Exchanges 389 492 598 526 468 47 Webinars 66 87 95 98 98 169 Online Courses 1 1 8 4 9 22 Source: CEPOL CAARs (2015 - 2020). Residential training activities involve participants from across the EU coming together in one location (often not at CEPOL’s headquarters in Hungary) for several days of knowledge, expertise and skill transference. The Activity Organiser is responsible for preparing the content, budget planning, implementation and logistics. The CEPOL Exchange Programme (CEP), formerly the European Police Exchange Programme (EPEP) allows law enforcement officers to spend a week in another country with a counterpart. The CEP, which is modelled on the Erasmus programme, offers participants an exchange of knowledge and good practice, with the opportunity to network and form long lasting professional relationships. All of CEPOL’s activities are supported by the Agency’s online training platform, Law Enforcement Education (LEEd), which is open to all EU law enforcement officers and is free to use. LEEd replaced CEPOL’s old e-learning platform, ‘e-Net’, in 2020. CEPOL’s online learning offer comprises webinars, online courses and online modules. The CEPOL European Joint Master Programme (EJMP) was an EU academic programme developed in 2015 with the objective of addressing common challenges of police cooperation in the area of internal security. The EJMP targeted senior law enforcement officers or experts in general management and specialist positions. From 2020 the continuation of the EJMP was put on hold after the Internal Audit Service identified non-compliant practices around the setup of the programme. A more recent activity was the 2019 inauguration of the Cybercrime Academy (CCA). Built to train up to 100 participants simultaneously, the establishment of the academy reflects CEPOL’s prioritisation of cybercrime as a growing threat to European economies and societies. Until April 2021, the CCA had delivered 38 training activities. In 2017 two CEPOL Centres of Knowledge (CKCs) were established in the areas of counterterrorism and CSDP Missions to pilot a new business model of implementing courses through multiannual direct awards granted to Framework Partners. CEPOL’s CKC model intended to bring together subject experts from Member States and relevant EU Agencies to design multi-annual training portfolios (including residential and online activities) on given topic areas. CEPOL would be responsible for designing and implementing courses within the CKC, in collaboration with hosting training institutions in Member States. The pilot aimed to increase efficiency by encouraging Framework Partners’ ownership of projects, reducing the administrative burden on applicants generated by the annual grants’ procedure as well as addressing a consistent underspending of resources and limited capacity of Framework Partners to implement CEPOL activities. Further to an external evaluation, CEPOL designed and implemented a hybrid business model in 2020, with a clearer division of responsibilities. In this hybrid model, CEPOL began to enlist the support of CKCs, composed of representatives from Member States, as well as from relevant EU JHA agencies and bodies (e.g. Europol, Frontex, EASO), with expertise on a specific topic listed in the call for experts. The CKC groups are tasked with helping CEPOL to prepare its multiannual training portfolio for each thematic area, based on strategic and operational training needs assessments. In 2020, one new CKC was piloted in line with this new business model, with a thematic focus on counterterrorism, and one additional pilot is being set up in 2021 focusing on law enforcement cooperation, information exchange and interoperability. One of CEPOL’s multiannual objectives is to contribute to and encourage the development of research relevant for training activities and to disseminate research findings. To fulfil this objective, CEPOL aims to integrate developments in research and promote the creation of stronger partnerships between universities and law enforcement training institutes in the Member States. In terms of research outputs, since 2009 CEPOL has published several bulletins on an annual basis on its website. From 2018 onwards, these bulletins were published as the ‘European Law Enforcement Research Bulletin’. There were on average two research bulletins published each year between 2015 and 2020.

Described as the Agency’s ‘flagship event’ the CEPOL Annual Research and Science Conference aims to provide insight into the future challenges of modern European policing, although no conference has been held in 2017-20. The conference celebrated its 15th anniversary in 2016 and brought together researchers and practitioners from across the EU to explore the challenges of law enforcement training and education . In 2017 the conference was organised in collaboration with the Hungarian National University of Public Services and focused on ‘Innovations in Law Enforcement – implications for practice, education and civil society. CEPOL is also an occasional host of workshops. Additionally, CEPOL has an e-library and registered users of CEPOL’s LEEd platform have online access to more than 300 European and international titles of e-journals covering areas such as police science, law enforcement and criminal justice. CEPOL is required by its mandate to design and implement its activities in accordance with the challenges and needs of the EU. The capacity building in third countries has worked towards main policy documents, including the European Agenda on Migration, the European Agenda on Security and the Council Conclusions on EU External Action on Counter-terrorism. The EU/Middle East and North Africa (henceforth ‘MENA’), the Counter-Terrorism Training Partnership (1 and 2) and Financial Investigation in-service training Western Balkans are projects, which have aimed to enhance the Agency’s role in strengthening the internal security of the immediate neighbourhood of the EU. The activities in this area include residential stays, regional workshops, evaluation visits and round tables. 4. Method Short description of methodology Limitations and robustness of findings There are a number of limitations of the data collected to inform the evaluation, which should be borne in mind when interpreting the findings. Below we summarise the limitations and mitigation measures taken to address them. 1.Measuring effectiveness: Measurement of the Agency’s effectiveness is limited by lack of statistical, financial, and administrative data to test, challenge and validate the expert judgements and stakeholder opinions collected through fieldwork activities. For example, there is an absence of data to assess whether the skills and knowledge of CEPOL training participants have actually increased as a result of the training provided. 2.Relying on self-reported data: Data collected from stakeholders through interviews and surveys is self-reported, may be subject to recall bias and may reflect the specific agenda of those consulted. Mitigation measure to points 1 and 2: Triangulation of self-reported data with data analysed from documentary sources including factual information on the functioning of CEPOL, information on CEPOL’s activities, previous evaluation documents, and documents related to CEPOL’s partnerships. 3.Attributing outcomes: In the absence of a counterfactual, there are challenges in unequivocally attributing outcomes to the activities of CEPOL. The aim of this evaluation was to assess the strength of evidence for CEPOL’s progress against the intended intervention logic, rather than assess direct, verifiable causality based on strong empirical evidence. Mitigation measure: The evaluation is a contribution analysis, rather than an impact evaluation assessing causality. 4.Quantifying benefits: The inability to quantify the benefits of CEPOL means that this analysis is unable to make any concrete conclusions about the cost-effectiveness of the Agency. Mitigation measure: The efficiency analysis focuses on the costs of CEPOL’s activities and the efficiency of the Agency in light of the resources available. 5.Representativeness of stakeholder views: There may be selection bias because those interviewed and surveyed were selected by the research team following a stakeholder mapping exercise (involving the Commission and CEPOL) and may not be representative of all stakeholders of CEPOL. Mitigation measure: Selection of stakeholders across a range of groups including those at the EU, Member State, and international levels. 5. Analysis and answers to the evaluation questions This section includes a summary of the analysis carried out during the evaluation. 5.1 Effectiveness Key findings: ·CEPOL key programming documents consider and reflect the objectives of its mandate and the priorities of the EU Agenda on Security and related policy documents. ·CEPOL and its stakeholders recognise the importance of the peer-to-peer transfer of knowledge (‘cascading’) to reach a broader audience. ·CEPOL’s EU Strategic Training Needs Assessment (EU-STNA) methodology aligns with the priorities set at EU level (EMPACT), while taking into account the input provided by national stakeholders in the consultation process. ·CEPOL’s relationship with each relevant JHA agency and actor is distinctive, signalling that CEPOL can adapt its working approach to maximise potential synergies. ·CEPOL is widely recognised for building trust and facilitating the development of cooperation among law enforcement services to an extent that would not be achieved by the Member States without EU intervention. ·CEPOL has a solid quality assurance mechanism in place to guaranty the high quality of content and deliver of training activities when CEPOL has full oversight of the process. Following the Better Regulation Guidelines, this criterion examines why an objective has or has not been met, as well as factors driving or hindering progress. The evaluation looked at the effectiveness in relation to the general and specific objectives outlined in the intervention logic (Annex IV). Given the diverse potential beneficiaries of CEPOL’s work, the assessment of effectiveness attempts to provide sufficient details so as to explore what is effective, when and for whom.

The analysis of available evidence shows that CEPOL’s AWPs reflect both the Agency’s objectives and relevant EU priorities. Most CNUs/NCPs indicated that, in their opinion, almost all of CEPOL’s objectives have been reflected ‘to a great extent’ in the Agency’s actions and activities over the past five. Multiple interviewees noted that CEPOL struggles to address the enlarged target audience introduced by the 2015 Regulation. Compared to the total number of law enforcement personnel in the EU, CEPOL directly reaches only a minimal proportion of its target audience. It should be noted that CEPOL is not expected to reach a specific numeric target of law enforcement practitioners by mandate, as the Stockholm Programme tried to set a quantifiable target for law enforcement training activities, yet the proposition did neither find its way in the LETS Communication, nor in Regulation 2015/2219. Given CEPOL’s relatively small size (69 staff members) and annual budget, it is difficult for CEPOL to directly reach a sizeable proportion of the target audience. Police officers alone in the 26 Member States covered by CEPOL amount to around 1.5 officers , a figure that does not account for the extended targeted audience introduced by the 2015 Regulation . CEPOL activities in 2019, its best year on record to date in terms of outreach, reached a total of 34,615 participations ( Table 5 ). The available data show that CEPOL training participants usually take part in more than one activity, and with the current system CEPOL is unable to distinguish between unique participants (whereby individuals would be counted only once, even if they attended multiple CEPOL courses) and participations (total attendance to CEPOL activities, counting individuals multiple times for each activity they attended), leading to an inaccurate representation of its outreach activities in the Annual Reports. Even admitting that CEPOL managed to substantially enlarge its audience, CEPOL must therefore rely on indirect means to extend its reach and address the broadest target audience possible, most notably through the ‘cascading mechanism’ and its digitalisation strategy. Table 5: Number of participants by training type and CEPOL outreach capacity assuming it only targeted police officers 2015 2016 2017 2018 2019 2020 Residential 3,073 2,946 3,383 3,062 3,057 108 Exchanges 428 492 598 526 468 47 External project - 281 507 527 1,226 572 E-learning webinars 6,731 10,889 13,748 15,415 29,864 38,764 E-learning modules 2,709 3,329 5,301 8,570 Online course 51 72 230 141 Total online 9,491 14,290 19,279 24,126 Total reported participations 12,992 18,009 23,494 29,003 34,615 39,401 Total excluding external projects 12,992 17,728 22,987 28,476 33,381 38,829 Police officers in EU-26 1,489,896 1,493,486 1,498,322 1,502,053 1, 481,297 n/a % of EU-26 police officers CEPOL could have reached 0.87% 1.19% 1.53% 1.90% 2.25% n/a Source: CEPOL CAARs (2015-2020) (participants), Eurostat (police officers). Given the above, CEPOL should aim to broaden the reach of its target audience through the cascading mechanism resulting from the training of trainers and peer-to-peer transfer of knowledge. The evaluation found that the extent of cascading is inconsistent across Member States and largely out of CEPOL’s control. As a matter of fact, Regulation 2015/2219 does not include the explicit promotion of the peer-to-peer transfer of knowledge in the remit of CEPOL, although Article 3 includes the objective to “train trainers and assist in improving and exchanging best learning practices” and cascading seems the logical means to ensure a broader outreach, given the Agency’s current capacity. Information collected shows that CEPOL, while not actively promoting cascading, recognises its importance and attempts to capture its impact, although with mixed results. CEPOL uses its customised version of Kirkpatrick’s methodology for training evaluation to collect information from training participants on the extent to which they transfer knowledge acquired through CEPOL. However, CEPOL’s reporting on cascading is inconsistent – using slightly different metrics in different CAARs, precluding meaningful comparison (see Table 6). Table 6: Accounts of the cascading effect reported by CEPOL Accounts of the cascading effect reported by CEPOL CAAR 2015 ‘The cascading of gained knowledge is an on-going practice in all Member States.’ CAAR 2016 ‘94% of participants declared that they successfully transferred the obtained knowledge in their professional environment. 96% of line managers confirmed the practical use of the knowledge gained in CEPOL trainings.’ CAAR 2017 ‘Last year overwhelming majority of the course attendants declared that they had shared their acquired skills and knowledge in their national environment.’ CAAR 2018 ‘[The] overwhelming majority of course attendees declared that they had shared their acquired skills/knowledge in their national environment. They shared the knowledge primarily with their fellow colleagues (88 %), with their line managers (46%) or with their staff (36%).’ CAAR 2019 ‘Cascading the knowledge gained in CEPOL trainings is strong, participants shared the knowledge primarily with their fellow colleagues, with their line managers and/or with their staff.’ CAAR 2020 No reference to cascading Sources: CEPOL CAARs (2015-2020). Nonetheless, most of the surveyed CNUs/NCPs and training participants affirm that the knowledge and skills acquired through CEPOL’s training programmes are being transferred by training participants to their peers. Survey respondents were also of the view that cascading is not in CEPOL’s hands, however, some offered suggestions as to how CEPOL could better support cascading by: (i) actively including the requirement to envisage cascading activities for the participants as part of the training developed by CEPOL and by its FWPs, and to report on those; (ii) developing a process to collect data to evidence the cascading; (iii) developing operational guidance on how cascading could be ensured at the national level; and (iv) actively reminding participants of the importance of cascading their knowledge to their peers; and (v) sharing best practices on cascading. All information collected by stakeholders point to the conclusion that capturing the impact of the cascading effect is indeed challenging, however, given (i) the small number of EU law enforcement officials trained per year as compared to the entire population; and (ii) the recognised importance attributed by CEPOL to cascading for multiplying the impact of training activities, there is a strong argument that CEPOL should do more to actively promote cascading. CEPOL appears committed to fulfil its mandate when it comes to cross-border and European-level policing and invests its resources accordingly, with appreciable results. Regulation 2015/2219 details three specific aspects of cross-border cooperation and EU-level policing that CEPOL’s activities should focus on: (1) training on the implementation and use of international and Union instruments for law enforcement cooperation; (2) training for Union bodies and JHA agencies; (3) training on judicial cooperation and practical use of information exchange channels. The evidence collected shows that all three aspects are addressed, as CEPOL dedicates almost one-third of the total number of its yearly training activities to police cooperation and information exchange (see Table 7). Table 7: Number of CEPOL activities dedicated to cross-border cooperation and EU-level policing between 2016 and 2019 2016 2017 2018 2019 2020 Cross-border cooperation and European-level policing Corresponding objective in CEPOL CAAR N/A 1.C 1.E 2.C 2.3 2.6 2.7 Total residential activities 24 42 32 27 1 Total webinars and online courses 33 31 21 34 35 Participants’ satisfaction rate 95% 92% 94% 96% 93% % of total yearly activities dedicated to police cooperation and information exchange 33% 35% 25% 25% 14% Information exchange and EU police and judicial cooperation Residential activities 13 12 10 9 0 Webinars and online courses 19 14 11 25 13 Common practices for maintenance of law and order and public security Residential activities 1 5 1 2 0 Webinars and online courses 2 2 1 1 8 Special law enforcement techniques Residential activities 10 25 20 16 0 Webinars and online courses 12 15 11 0 28 Sources: CEPOL CAARs (2016-2020). ‘Strand 2’ of the LETS Communication indicated the need to raise awareness and to reinforce the use of law enforcement cooperation mechanisms and instruments to promote effective bilateral and multilateral cooperation among Member States. The analysis of SPDs and CAARs found that CEPOL covers a wide range of cooperation mechanisms and instruments through dedicated residential and online training activities. Furthermore, CEPOL exported these activities beyond the Union’s border and introduced collaborative mechanisms (such as the courses on JITs delivered in collaboration with the EJTN) to EU neighbours in the Western Balkans, effectively extending its support to bilateral and regional cooperation in law enforcement to the EU neighbourhood. To ensure that the appropriate thematic areas are covered in its training offer, CEPOL developed and launched the EU-STNA to identify the evolving training needs of CEPOL’s target audience. The evaluation found that the introduction of the EU-STNA has strongly improved the prioritisation of training needs. Consistent with this, the overwhelming opinion across all consulted stakeholder groups is that CEPOL provides great added value with regard to training on specific crime areas compared to what is offered at the national level. Some of the interviewees indicated that CEPOL’s training offer underserved certain thematic areas and categories (e.g. forensics, customs, judiciary), and that the results of the EU-STNA overemphasise the priorities set at EU level by the EMPACT at the cost of priorities indicated by stakeholders at the national level. CEPOL progressively increased the number of activities dedicated to EMPACT priorities - except in the year of COVID - and their relative weight within CEPOL activities increased accordingly ( Table 8). The interest from training participants in EMPACT priorities has grown accordingly, registering a four-fold increase between 2015 and 2019. Table 8: EMPACT priorities: activities and participation over 2015-2020 EU Policy Cycle/EMPACT priorities 2015 2016 2017 2018 2019 2020 Activities 46 61 79 107 125 96 % of activities on total 30% 35% 38% 51% 51% 30% Participants 3,629 5,661 10,314 16,453 16,031 19,543 % of participants on total 28% 32% 45% 58% 48% 49% Sources: CEPOL CAARS (2015-2019).

The number of CSDP-related activities has increased over time (see Table 9), although it seems to have reached the limits of CEPOL’s capacity evidenced by the fact that CSDP-related training ranks last among the 21 priorities identified by the EU-STNA, and most of CEPOL’s capacity is therefore allocated to top ranking priorities. One interviewed NCP confirmed that the interest in CSDP-related activities within CEPOL’s Management Board was low. As it emerged in case study 2, CEPOL can leverage a comparative advantage in the provision of highly specialised training for civilian CSDP missions, but probably should leave generic pre-deployment training to other providers, as there is little added value to gain from CEPOL’s involvement. Table 9: Quantitative indicators on CEPOL's CSDP related training 2015 2016 2017 2018 2019 2020 Activities 1 1 7 12 12 4 Participants n/a 28 245 588 717 300 Satisfaction n/a n/a 96% 95% 97% 92% Sources: CEPOL CAARs (2015-2020). Unlike CSPD-related training, CEPOL started providing capacity-building activities in third countries immediately following the introduction of the new legal basis. Regulation 2015/2219 enabled CEPOL to receive direct grants from the European Commission to implement projects in the EU neighbourhood. CEPOL has progressively received more resources through direct grants from the European Commission to expand its activity in third countries in the Western Balkans, in the Southern Neighbourhood and, most recently, to the Eastern Partnership countries (see Table 4 ). Representatives from partner third countries are particularly satisfied with the service-oriented approach of CEPOL, which distinguishes the Agency from other training providers in a crowded market (also see case study 2). Table 10: Quantitative indicators on CEPOL's activities in third countries 2015 2016 2017 2018 2019 2020 Activities n/a 16 15 24 42 62 Exchanges/visits n/a n/a 34 77 187 18 Participants n/a 281 507 451 1334 572 Satisfaction n/a 92% 96% 96% 97% 100% Sources: CEPOL CAARs (2015-2020). Training of law enforcement trainers was intended by the LETS Communication as a way to reduce training gaps across the EU, attributed in part to a lack of qualified trainers. CEPOL responded by developing residential and online activities dedicated to law enforcement trainers and training methodologies (see Table 1 1 ). The majority of surveyed stakeholders found that CEPOL achieved the objective of providing training to law enforcement trainers. More relevant to the stated objective of the LETS is the contribution that CEPOL provides in closing the gap between Member States. The majority of survey respondents within all stakeholder categories believe that CEPOL provides great added value compared to training for trainers provided at the national level. It is worth noting that in interviews ‘training the trainers’ emerged as a key enabler of the cascading of knowledge from CEPOL’s activities to Member States. Yet, case study 3 found that CEPOL’s train the trainers offer is comparatively smaller than that of other EU JHA agencies. Consideration should be given to expanding it. Table 11: Number of training activities dedicated to train the trainers CEPOL Activities 2015 2016 2017 2018 2019 2020 Residential (R)/e-learning (E) R E R E R E R E R E R E Train the trainers 4 0 2 7 2 6 2 4 2 3 0* 0 Source: CEPOL training catalogues (2015-2020). *In 2020, two residential courses were planned but not implemented due to Covid-19 restrictions. Knowledge-sharing and exchange of best practices, as seen across the CAARs of 2015-2020, was pursued through the creation of a cluster of services to develop CEPOL “into a European law enforcement knowledge base”. As described above, these services include CEPOL’s e-Library, the Lecturers, Trainers and Researchers Database (LTRdb), the publication of the Bulletin (CEPOL’s own scientific periodical), and the Research and Science Conference. The majority of stakeholders from all surveyed categories believed that CEPOL ‘fully achieved’ the objective of supporting the development and exchange of best practice. The review of CEPOL’s AWPs between 2015 and 2020 showed that CEPOL actively seeks to develop, evaluate and upgrade its training digital delivery tools. CEPOL’s objective to upgrade and evaluate new learning methodologies has been addressed both in-house by CEPOL’s Training and Research Unit, and through the network of national training providers. The results are different ‘blended training’ methodologies, which entail a mix of training tools, including event workshops, residential training modules and coaching support, with different degrees of interactivity. According to interviewees, there is not one good way of blending the different training tools, and results depend on how individual participants respond to different kinds of training. Yet, from the evaluation it emerged that law enforcement officials respond better to practical cases and operational training, including elements such as tabletop exercises and simulations, suggesting that CEPOL could further tailor its training methodologies to its target audience. Throughout the 2015-2020 period, CEPOL reached and exceeded most of its targets and it actively revised the set of Key Performance Indicators (KPIs) to include new challenges and address emerging issues (see table 45 at page 134 of the Final Report in the external Evaluation for the full list of KPIs). The few exceptions align with issues identified by this evaluation and presented in this report. CEPOL occasionally missed targets on: (i) reducing the flow of statutory staff turnover, (ii) meeting its budgetary commitments/payments, and (iii) the implementation of activities under Framework Agreements. Overall, the review of the number of activities (see Table 1 2 ) implemented by CEPOL over the 2015-2020 period indicates that the Agency has effectively delivered the activities planned, often adding additional courses or increasing exchange opportunities to meet an increase in demand. As expected, the degree of implementation for residential activities and exchanges took a massive hit in 2020 due to the COVID-19 outbreak, which was only partially compensated by the increase of online activities. CEPOL’s training activities are considered of good quality by most of the stakeholders consulted through interviews and surveys and are deemed of poor quality only by a very small proportion (1%) of the surveyed training participants. Table 12: CEPOL planned vs implemented activities 2015 2016 2017 2018 2019 2020 Planned (P) / Implemented (I) P I P I P I P I P I P I Residential 80 85 80 87 103 105 102 103 98 100 81 4 Webinars 62 66 79 87 88 95 86 98 128 142 145 169 Online courses 1 1 1 1 10 8 4 4 Exchanges - 389 - 492 - 598 374 526 330 468 535 47 Sources: CEPOL CAARs (2015-2020), CEPOL Annual Report 2016. Within the surveyed stakeholder groups, CEPOL’s residential training activities receive the strongest consensus regarding their quality, followed by seminars and conferences, and online training. The great majority of training participants found that CEPOL training meets their expectations and training needs under all aspects investigated by this evaluation (see Figure 2 below). The training offer is appreciated for its relevance and breath of topics covered, and for filling gaps and adding value to the training provided at the national level. The trainers and training materials are also considered of good quality by the surveyed training participants. Beyond the immediate learning benefits, CEPOL’s training opportunities are also widely appreciated for facilitating networking opportunities and strengthening cooperation with European counterparts and effectively building trust.

Several training participants satisfied with the quality of CEPOL’s training indicated that they would like to see CEPOL further increase the volume of activities and some would welcome a CEPOL certification system, as proof of the knowledge acquired. The latter appears as an objective in CEPOL’s programming documents, but has not yet been implemented. Practical training and experience sharing are typically used in CEPOL exchange programmes, which also received an overwhelmingly positive judgement from all surveyed stakeholder groups and interviewees. They stressed how operational collaboration is the most effective way for law enforcement officials to assimilate and internalise new practices. The available information during the evaluation on the CEPOL European Joint Master Programme (EJMP) is too limited to provide a decisive judgement, however, the general perception from the few sources available is positive, except for one account, which lamented the absence of exams for assessing and certificating the knowledge acquired, which are, by comparison, available in the Frontex programme. The EJMP was discontinued in 2019 after its second edition due to issues in the selection procedures of the providers. Figure 2: To what extent were you satisfied with the training opportunities offered by CEPOL between 2015 and 2020 in terms of: CEPOL’s investment in upgrading and expanding its online training offer emerged as a positive strategy for extending its outreach and enabling a smooth transition to the delivery of training in pandemic times. However, the commitment to its own strategy was discontinuous over the years and by CEPOL’s own admission its online delivery tools fell behind those of other EU agencies. CEPOL tried to reinforce its digital offer through the new proprietary online training platform for Law Enforcement Education – LEEd, which went online in April 2020. All of CEPOL’s online activities are now supported by LEEd, which is open to all EU law enforcement officers and is free to use. The platform – which is now compatible with mobile devices – presents CEPOL’s training activities clustered according to thematic areas, course type, and topical tags. According to CEPOL’s Digitalisation Strategy , LEEd will eventually address some of the requests of its target audience, including the categorisation of courses by level of difficulty, and a progression and certification system for the learner. In addition to the training area, LEEd provides a repository of articles from the International Police Science Journal (e-Journals) with browsing and filter capabilities by categories, a forum for the users to communicate, publish or modify content and a space for conference management. LEEd appears to satisfy the majority of training participants, and some explicitly say it represents an improvement from CEPOL’s previous platform ‘e-Net’ , although opinions on LEEd are mixed. Some find it fit for purpose, user-friendly, and even eye-catching, while others say it is harder to use than its predecessor or find the module-based approach unintuitive and expressed dissatisfaction with content availability. Moreover, a non-negligible proportion of surveyed training participants indicated that they were unsatisfied with LEEd’s lack of simplicity and clarity of use, and the ineffectiveness of its search engine. LEEd’s deployment encountered some challenges. It was delayed by four months and the migration of content from the previous platform suffered from further delays; some stakeholders also reported bugs and compatibility issues on some operating systems. The hosting infrastructure of the LEEd platform, managed by a German private company, suffered a security breach on 17 November 2020. While the overall balance of opinions depicts a positive user experience, LEEd’s cumulative issues have been an obstacle to the smooth delivery of online courses at a time when all of CEPOL activities moved online due to COVID-19. Other issues identified related to the development and delivery of training activities outsourced to CEPOL’s FWPs, namely the short span of the yearly training development cycle and a blind spot in the quality assurance mechanism managed by CEPOL. As for the training development cycle, it was pointed out that once the thematic areas for training have been approved by the Management Board, the time at the disposal of FWPs to undergo grant procedures and develop course content is too short, effectively discouraging FWPs from applying and favouring those with off-the-shelf courses that happen to fit the chosen topics. As a matter of fact, the number of grant applications from FWPs does not meet the expectations. Outsourced training activities also suffer from uneven quality of delivery from FWPs, which CEPOL’s quality assurance mechanisms is currently unable to prevent. Finally, the selection of the right participants for residential training is crucial to ensure that law enforcement officials gain the most out of CEPOL activities, in terms of exchange and relationship-building. However, stakeholders interviewed highlighted that participants selected for residential training do not always have a suitable profile in terms of the level of expertise and seniority. Activity Managers within CEPOL have the power to vet and decline participants’ applications, leaving the spot open for more suited candidates, but the evaluation finds that this power is hardly ever used, as it is an established practice to accept the first candidate presented by Member States, regardless of his/her profile. The LETS Communication identified a number of training gaps that affected the effectiveness of cross-border cooperation across the EU: (i) a lack of qualified trainers and consistent learning materials on cross-border investigations; (ii) insufficient coverage in Member States’ training programmes of EU instruments and the EU dimension of day-to-day policing; and (iii) inadequate English language skills, considered a crucial competence for all law enforcement officials involved in cross-border cooperation. Evidence shows that CEPOL addresses all three gaps with dedicated training activities (see Table 1 3 ) and the development of ‘common curricula’, putting particular emphasis on training on EU instruments and EU policing. Table 13: CEPOL activities to address training gaps identified by the LETS Communication CEPOL Activities 2015 2016 2017 2018 2019 2020 Residential (R) / e-learning (E) R E R E R E R E R E E Train the trainers 4 0 2 7 2 6 2 4 2 3 n/a EU instruments and EU policing 20 9 24 33 42 21 32 21 27 34 35 Police English Language 1 2 1 3 2 2 2 2 2 3 n/a Source: CEPOL CAARs (2015-2019), CEPOL training catalogues (2015-2020). The majority of respondents to the surveys were of the view that CEPOL effectively provides training to law enforcement trainers, and training participants have an overwhelmingly positive opinion of the trainers selected by CEPOL, suggesting that an increase of CEPOL’s offer in this respect would be positively received. Similarly, the majority of stakeholders believe that CEPOL has played a key role in shaping the tools and materials for law enforcement training, effectively filling gaps in training provided at the national level. Despite all of this, the majority of survey respondents within each group believed that training gaps between Member States have been reduced only ‘to some extent’. However, it should be noted that representatives from national police may not be able to offer a fully objective view of this issue, perhaps being inclined to overestimate the strengths of their national practices compared to alternative approaches, especially when it comes to national key areas of expertise. The analysis of available evidence shows that CEPOL is convincingly covering the right bases and is moving in the right direction to further reduce training gaps, but there is more distance to be covered to even out the field across the EU. The 2015 CEPOL Regulation expanded CEPOL’s mandate with the responsibility to further coordinate training between EU agencies in the JHA area. Stakeholders interviewed recognised that CEPOL became more proactive in seeking to coordinate activities with agencies and actors in this policy sphere, organising annual meetings with both DG HOME and JHA agencies (including EASO, EIGE, EMCDDA, eu-LISA, Eurojust, Europol, FRA and Frontex) to synchronise training activities. However, a lack of sustained active collaboration within the JHA sphere (e.g. actors contributing to the EU JHA landscape that are not JHA agencies) was noted by some stakeholders, for instance with DG TAXUD, which coordinates training for customs and tax authorities in the EU. Furthermore, it was also suggested within the JHA area that CEPOL should reach out to other relevant groups that host law enforcement and prosecutorial networks, such as the Camden Asset Recovery Interagency Network (CARIN), although, progress was noted in forging cooperation with the European Network for Investigation and Prosecution of Genocide, Crimes against Humanity and War Crimes. Interviewees from JHA agencies unanimously considered CEPOL’s role to be complementary in achieving their objectives. Synergies and working relationships varied, centring on a number of key themes. Some JHA agencies provide training content to CEPOL, either by sharing their topic expertise to inform CEPOL modules in the absence of CEPOL in-house specialism, or by supplying experts to deliver training or speak at CEPOL organised webinars. Interviewees articulated that there is often a clear division of labour, with the JHA agencies responsible for the technical content of training and CEPOL providing administrative and organisational functions to disseminate and structure modules, including through the use of the LEEd platform. CEPOL provides a nexus and forum through which relevant JHA agency training can be disseminated to audiences beyond their traditional scope. Infrequently, CEPOL may take the lead in creating the content for a training course alongside JHA agencies. The effectiveness of CEPOL’s tools to pursue its coordination function is less certain. In undertaking this coordination function, CEPOL’s interactions with JHA agencies varies according to identified synergies and training needs of the relevant organisation. CEPOL previously made use of the JHA Training Matrix to identify training synergies. This methodology required other JHA agencies to list their specific activities, number of participants in attendance, and the target audience of them in a formatted Excel file, which had to be sent to CEPOL. When in use, the matrix was found to be an effective tool to map training activities, improve their coordinated delivery, help to reduce duplication of training activities. Figure 3: In 2014 CEPOL introduced the JHA Training Matrix as a coordinated planning tool for training activities. To what extent do you agree with the following statements: (CNU/NCPs) Source: Survey of CNUs/NCP (n=33). In 2019, CEPOL decided to stop using the JHA Training Matrix, on the basis that it was not systematically completed by agencies, resulting in the collated information being incomplete. In an Audit on Needs Assessment conducted in 2016 it was reported that over half of the training activities recorded by the relevant agencies were uploaded after training had taken place, limiting the utility of the Training Matrix for planning purposes. The reasons for agencies not to complete the matrix may be linked to perceptions and experiences of the JHA Training Matrix being cumbersome to complete and ineffective. No structured methodology has replaced the JHA Training Matrix; instead, CEPOL undertakes coordination through a peer-to-peer approach. Some stakeholders considered this shift to ad hoc meetings as increasing the risk of future gaps and overlaps in training provision. According to stakeholders, initial requests from CEPOL for agencies to contribute content or speakers to training courses were sometimes received at short notice, limiting the Agency’s ability to fulfil such requests. As mentioned above, the working relationship and synergies between agencies varies. Whilst the majority of stakeholders agreed that CEPOL’s training activities were consistent and complementary with their own activities, the JHA Training Matrix may not have been the mechanism through which this coherence was achieved. Stakeholders reported that CEPOL is an active participant in the JHA Agencies Network. Ultimately, some stakeholders questioned whether CEPOL’s updated coordination role, and therefore identity, was made clear to all agencies and organisations, as they linked it to the perceived absence of strategic direction or guidance from EU legislative bodies in assisting CEPOL to fulfil this role, as CEPOL lacks authority to require agencies to cooperate. Nonetheless, this represents a concern that may undermine CEPOL’s external coherence in the long term. Some EU-level stakeholders and groups generally accorded with this view of CEPOL being an active cooperating partner when coordinating training at the EU level, although the frequency and purpose of CEPOL’s communications varied across organisations. The expansion of CEPOL’s activities to train, in collaboration with other entities, relevant cross-professional audiences – such as, for instance, prosecutors who authorise police use of investigative tools – was not perceived to have impacted upon training coherence in practice. Some stakeholder reported an increase in CEPOL’s courses produced with the Network of Public Prosecutors and the EJTN and the ENFSI, although the provision of topics relevant to judges is said to be limited. Stakeholders identified that prosecutors and judges may be more readily served by other bodies, such as the EJTN and training identification assisted by the European Training Platform. The LETS Communication emphasised the importance of exchange programmes as a key instrument to increase cooperation, share good practice and build trust among EU law enforcement officials. CEPOL responded with a progressive increase of exchange opportunities over the evaluation period both by intensifying exchanges, and by extending exchange opportunities to other stakeholder categories and partner third countries within the scope of CEPOL’s projects in the EU neighbourhood. Stakeholders’ opinion on the CEP is overwhelmingly positive. The exchange programmes emerged as clear favourites among the interviewees as well, which stressed how operational police work conducted during exchanges is the most effective way to build a trusting relationship between law enforcement services. Exchange programmes are not the only activity conductive to cooperation and trust-building. The majority of all consulted stakeholders agree that CEPOL activities as a whole effectively led to an increase in mutual trust among law enforcement organisations. Moreover, stakeholders believed that CEPOL’s activities facilitated an increase in the number of opportunities for cooperation and in the quality of cooperation between national law enforcement services at EU level as well as, to a smaller extent, on a bilateral and regional basis. The majority of training participants agree that CEPOL’s training activities provide concrete opportunities to establish collaborations with their European counterparts and that CEPOL adds unique value to develop cooperation that would not have otherwise been achieved through national means. One interviewee underlined that, because of the nature of their work, law enforcement officials are naturally reluctant to trust and share information with people they do not know, but meeting their counterparts from other Member States during CEPOL’s activities contributes to reduce the barrier of mistrust. Several stakeholders also stressed that moving training activities online due to COVID-19 (which was a positive and responsive step by CEPOL given the circumstances) meant losing one of the most impactful aspects of CEPOL’s activity: networking and relationship building, deemed crucial for effective cooperation. Stakeholders with direct knowledge of CEPOL’s external activities consider that CEPOL’s activities are an important tool for improving relationships between the EU and third countries, in particular, to harmonise investigation and prosecution practices with EU standards and to reinforce cooperation among partner countries with positive effects on regional stability and enhanced dialogue between law enforcement institutions in the EU neighbourhood.

The evaluation has considered to what extent the current governance structure of CEPOL has brought positive effects to the overall functioning of the agency. Case study 1 addresses aspects of CEPOL’s governance and internal workings in more detail. Evidence suggests that the new governance structure had positive effects on the Agency’s ability to fulfil its activities and objectives. The 2015 evaluation of CEPOL noted that the relationship between the Management Board (formerly known as the Governing Board) and the Agency was disproportionately skewed towards the former, as Board Members were entirely responsible for developing CEPOL’s strategic direction and the Agency only had an administrative and supporting role. The current governance, which reinforced the role of the Executive Director, strikes a good balance of responsibilities between the two, in the opinions of the overwhelming majority of stakeholders. Thus the Executive Office is fully dedicated to the Agency and can partly compensate for some of the issues affecting the Board identified in the course of the evaluation, namely the turnover within the Board and the low level of engagement of a non-negligible part of Member State representatives. It emerged that the Board consistently suffers from low levels of participation from several Member States, an issue already present in the pre-2015 era and which according to these stakeholders remained unchanged by the introduction of the 2015 Regulation. The Executive Director emerges prominently as the driving force within the Board, on one hand compensating for less engaged Board Members, but also further disincentivising participation. There are also issues around representation on the Board, with Member States sometimes appointing persons that are unqualified or unable to commit to the job, and the composition of the Board does not currently represent CEPOL’s target audience; furthermore, the Board also consistently lacks the balance in gender representation required by Article 8 of Regulation 2015/2219, as only 35% of NCPs are women (9 women out of 26), and the unbalance is only slightly reduced to 38% of women when accounting for alternate voting members (19 women out of 50). The agenda of Board Meetings being cramped with administrative items is also recognised by some interviewed as an issue affecting the room for strategic discussion, although CEPOL has already tried to tackle this issue by regularly using the written procedure for administrative decisions and by encouraging an increased participation of Board Members, also through Strategic Discussion Groups. Thus the great majority of stakeolders consider CEPOL decision-making procedures to be effective, despite the issues identified. Communication between the different components of CEPOL’s governance appears effective. The introduction of CNUs improved CEPOL’s ability to collect the necessary information from dedicated operational interlocutors at the national level. However, some NCPs feel they are overloaded with information, which they are able to digest only through the filter of CNUs and pointed out a bottleneck in communication at the national level. This is due to the fact that CNUs are organised differently in each Member State (as permitted by the 2015 Regulation), and in some national setups the NCP and the CNU have been selected from different institutions with hardly any contact, affecting the awareness and preparation of some NCP during Board Meetings. High staff turnover among CNUs and CEPOL has led to the loss of consolidated relationships between CEPOL staff and Member States representatives and a general difficulty in forming new ones, leading to stiffer and somewhat less efficient communication. The analysis has also found that most of the provisions of Regulation 2015/2219 are aligned to the requirements of the Common Approach , and it appears clear that the Common Approach informed the revision of CEPOL’s Regulation in 2015, as illustrated in case study 1, with the notable exception of the Executive Board, which was left out of the 2015/2219 Regulation. An analysis of the compliance of the main provisions of CEPOL’s legal basis defining the Agency’s internal structures, governance, administrative and financial rules is presented in Table 1 4 below. Table 14: Matrix of correspondence between CEPOL's regulation and working practices and the requirements of the Common Approach Common approach Element of the Common Approach Compliance CEPOL’s legal basis Internal structure and procedures Part II - “Structure and governance of agencies” The internal structure includes a Management Board (MB), an Executive Director (ED) and may also include other internal bodies (Scientific Committee for training and other advisory bodies) The Agency is compliant since it includes a Management Board and an Executive Director, and its legal basis foresees the possibility to include additional internal advisory bodies. However, the Scientific Committee for Training is not implemented. Art. 7 Administrative and management structure of CEPOL Par. 10 The MB is composed of one representative from each Member State and two representatives from the Commission. Only one EC representative is included in the MB Art 8 Composition of the Management Board The MB members are appointed for a four-year renewable term taking into account their knowledge in the field of training for law enforcement officials and their relevant managerial, administrative and budgetary skills In order o streamline the decision-making process […] a two-level governance structure should be introduced […] small-sized Executive Board should operate and be more closely involved in the monitoring of the agency’s activities. Regulation 2015/2219 did not introduce an Executive Board Par. 11 The MB adopts the Agency’s internal rules for the prevention and management of conflicts of interest Art. 9 Functions of the Management Board Par. 12 The MB is the staff appointing authority Par. 27-32 The Agency drafts annual and multi-annual work programming, which are adopted by the MB Par. 14 The ED is appointed by the MB, it is accountable to the MB and reports directly to the European Parliament Art. 14 Responsibilities of the Executive Director Par. 15 The ED is responsible for the Agency’s overall management and relationship with other EU institutions Par. 19 The ED may be removed from office by a decision by 2/3 of the Management Board Art 23 Executive Director Par. 13 By general rule, the MB takes decisions by absolute majority Art 13 Voting rules of the Management Board The MB uses 2/3 majority as a voting rule for the appointment and dismissal of the ED, appointment of the MB chairperson, adoption of the annual budget and work programme Art 9 Functions of the Management Board Working procedures Par. 10 When this is deemed as more efficient, a two-level governance structure is introduced, including a MB and a small-sized Executive Board The legal basis does not include this possibility – even in case it is deemed to be more efficient / EPRS, 2018, EU Agencies, Common Approach and Parliamentary Scrutiny Where appropriate, other members of the MB may include members designated by the European Parliament and other stakeholders The legal basis does not include this possibility – even if deemed to be appropriate / Par. 65 When relevant stakeholders are not represented in the MB, they should be involved in agencies’ internal bodies/advisory bodies/working groups The legal basis does not include the points of involving stakeholders– even if deemed to be appropriate / Par. 29 The Commission should be consulted on the adoption of both annual and multi-annual strategic documents Art. 10 (1) Multi-annual programming and AWPs The EP should be consulted on the adoption of the multiannual work programme and informed on the annual programme Par. 49 A Single Annual report should be submitted to the Court of Auditors, the EP, the Commission and the Council Art.9 (1, c) Functions of the Management Board Parr. 56-59 The adoption of the budget is carried out through the discharge procedure, it is overseen by the EP and audited by the Court of Auditors Art. 20 Presentation of accounts and discharge Administrative and financial rules Par. 40 The allocation of financial and human resources to the actions undertaken complies with the principles of activity-based budgeting and management Art. 10 Multi-annual programming and AWPs Par. 56 The Agency sends the provisional accounts for the financial year to the Commission and to the Court of Auditors Art. 20 Presentation of accounts and discharge Notes: Areas shaded in green indicate compliance, while areas marked in amber indicate non-compliance.

Source: Research team’s construction from available evidence. CEPOL has a quality assurance mechanism in place to control its training activities from development to delivery. The evaluation found that CEPOL’s quality control is effective in guaranteeing high-quality content and trainers when CEPOL is in full control of this process. However, training activities outsourced to FWPs, while still appreciated by participants, suffer from instances of variable quality that escape CEPOL’s quality assurance mechanism. CEPOL recognises the issue and tried to address it through a new centralised business model which has not found consensus within the governance. CEPOL opted then to adopt the current “hybrid” model which relies on both outsourcing training activities to FWPs, and centralising training development through the CKCs. In the 2015-2020 period, CEPOL invested part of its resources in assessing the quality of its training activities and the performance of trainers, as required by the LETS and recommended in the 2015 evaluation of CEPOL. Documentation on evaluated activities provided by CEPOL confirms that all of its activities – either delivered within the EU or in third countries – do, at a minimum, undergo an evaluation of participants’ satisfaction based on Kirkpatrick’s model for training evaluation. However, the Kirkpatrick’s model appears to be more of a tool for categorising the levels of training knowledge rather than a practicable method to evaluate them. In order to robustly measure the effectiveness of CEPOL’s training offer in improving the capabilities of law enforcement in the EU (in line with the objectives of the Agency), methodologies should ideally be experimental (e.g. pre-post design comparing outcomes among a group of law enforcement officials that are trained by CEPOL and those who are not). However, the research team acknowledge that measuring the impact of training beyond satisfaction and knowledge retention is resource-dependent and methodologically challenging, as also pointed out in interviews. As for internal control systems, CEPOL has been using a custom version of the European Commission Internal Control Framework (ICF), adapted to the CEPOL context and which complies with 13 ICF standards and CEPOL is certified in accordance with the Quality Management System Standard ISO 9001:2015 since 2017. The only external factor influencing CEPOL’s effectiveness that emerged prominently from the analysis is the multiplicity of training providers that compete for the attention of the law enforcement target audience. At the EU level, as discussed in case study 3 below, CEPOL shares segments of its audience with several other EU agencies that run their own training units and have developed separate training platforms. While there is evident room for improvement, the EU space is one that incentivises coordination, and all JHA agencies interviewed considered there to be good synergies between their own training offering and that of CEPOL. Still, the time law enforcement officials can dedicate for professional development activities is a finite resource, and CEPOL’s outreach numbers are inevitably influenced by the availability of alternative options provided by other JHA agencies. In the EU neighbourhood, as discussed in case study 2 below, the level of competition is much more pronounced. Not only does CEPOL have to compete for its target audience with international organisations and providers from other third countries with political interests in the partner countries, but with the bilateral initiatives of Member States as well. While all consulted stakeholders believe that coordination among different providers has much improved over the years, turning some of the duplication into synergies, the ‘training market’ in third countries appears significantly oversaturated and there is room for improvement, at least between CEPOL’s activities and the Member States’ initiatives. Some issues and areas for improvement emerged from the analysis: o The results of CEPOL in certain areas suffer from under-exposure or low recognition within CEPOL’s own governance (i.e. support to bilateral/regional cooperation, and support to capacity-building in third countries). o The Operational Training Needs Assessment (OTNA), which builds upon the EU-STNA, is suffering from a low response rate, resulting in less relevant operationalisation of selected training topics. o CEPOL’s training methodologies could be further tailored to its target audience as law enforcement officials respond better to hands-on practical training (joint exercises, simulations, table-top exercises) rather than (virtual) classroom theoretical training. o The newly implemented e-learning platform LEEd was delayed, albeit the user experience appears generally positive. o The possibility to manage Union External Assistance funds introduced in Regulation 2015/2219 proves crucial as CEPOL must rely on external funding from Commission programmes to fulfil its objectives in support of the EU external action. oAlthough CEPOL has been effective at addressing the gaps in law enforcement training, there is room for bringing the Member States’ training systems closer together, as stakeholders still recognise disparities in the level of maturity of national training systems. oNo structured methodology has replaced the JHA Training Matrix; instead, CEPOL undertakes coordination through a peer-to-peer approach. oAccounts of cooperation resulting from CEPOL’s activities are anecdotal since national law enforcement services do not systematically record or measure their occurrence or impact. CEPOL does not measure these outcomes either: while CEPOL’s mandate prescribes the agency to evaluate its training, the agency has focused on measuring participants’ satisfaction and learning outcomes. oThe Management Board suffers representation issues in terms of engagement across Member States, gender balance, and CEPOL’s enlarged target audience. • CEPOL struggles to attract and retain suitably qualified personnel as it is not a strong competitor on the jobs market; the Agency considers that this is mainly due to the low salary correction coefficient.

• CEPOL’s current business model to outsource part of the training activities to Framework Partners in the Member States weakens the quality assurance mechanism. • CEPOL competes with several training providers (other EU agencies, international organisations, foreign governments, Member States’ bilateral initiatives) for the attention of its target audience both within the EU and in third countries. 5.2. Efficiency Key findings: ·The number of CEPOL staff increased steadily over the period 2015 to 2020 (from 45 to 84), although there has been high staff turnover (in part as a result of the Agency’s relocation to Budapest). ·Overall, the consulted stakeholders agreed that the benefits stemming from participating in CEPOL’s activities outweighed the costs for Member States. Whilst the costs borne by Member States (including for hosting CEPOL activities and coordinating the CNU) vary from one country to another, these are relatively limited. ·CEPOL’s resources were managed well and used in the most efficient way possible for achieving the Agencies’ objectives. The analysis undertaken by the evaluation team suggests that there has been a downward trend in the cost per participant, though these results should be interpreted with caution because these are estimates only and rely on some assumptions. ·CEPOL’s internal structure appears to operate efficiently. The Agency’s internal efficiency has reportedly been boosted by improved governance, human resource management system, and internal working methods and procedures. However, some stakeholders noted inefficiencies related to the Management Board’s decision-making process, CEPOL’s planning processes and monitoring procedures. This efficiency criterion demands an assessment of the relationship between the cost of operating CEPOL and the changes it has led to (or contributed to) because of its activities. For this evaluation, the benefits of CEPOL’s activities cannot be quantified so the focus is on the costs of CEPOL’s activities and the efficiency of CEPOL’s operations. Efficiency is a measure of how resources/inputs (funds, human resources, expertise, time etc.) are converted into results. This evaluation has considered the costs incurred by CEPOL and its broader stakeholders in the delivery of its activities, as well as any potential benefits generated. For the purpose of the evaluation, the costs have been aggregated at a high-level (broken down by operations, staffing and building and equipment to be consistent with CEPOL documentation and reporting, rather than by activities). The inability to quantify the benefits means that this analysis is unable to make any concrete conclusions about the cost effectiveness of the Agency. As shown in Table 3 above, CEPOL’s over the 2015 to 2020 period, operating costs comprised on average 54% of the budget, staffing costs comprised 41% and other costs comprised 5%. The overall number of CEPOL staff has increased steadily over the period 2015 to 2020 from 45 to 84 (see Table 2 ). CEPOL funds Member States’ representatives to participate in CEPOL activities through annual allocation of grants to Framework Partners selected via restricted calls for proposals every four years. These grants typically cover travel, accommodation and catering for Member State representatives to attend CEPOL activities. On average it sems that €5,000 in additional costs are incurred for hosting each CEPOL activity. This variation between Member States can be explained bythe fact that Member States’ available resources and infrastructure to host CEPOL activities can vary. Member States fund the cost of staff within the CNU who act as a liaison point for law enforcement officials wanting to participate in CEPOL’s activities. There may be other ‘soft’ costs associated with Member States’ participation in CEPOL activities, but it is difficult to quantify these costs and they likely vary to a great degree across the EU. Most of the CNU/NCPs, FWPs representatives, training participants and third country representatives that responded to the surveys agreed that the benefits stemming from participating in CEPOL’s activities outweigh the costs for Member. These benefits of CEPOL are difficult to quantify, though the evaluation found that the overall outputs of the Agency are valued by stakeholders at the EU/national/international levels. The evaluation considers whether CEPOL could have achieved greater benefits with the same level of human and financial resources, and whether CEPOL’s governance structure, organisational set-up, management systems and working methods have contributed to the Agency’s efficiency, or whether there have been any delays or bottlenecks in these internal structures that may have negatively influenced efficiency. The stakeholders generally perceived that CEPOL’s resources were managed well and used in the most efficient way possible for achieving the Agency’ objectives, with only a minority of respondents indicating a partial efficiency in the use of resources. As one measure of efficiency, the research team calculated a high-level estimate of the cost per training participant over the period 2015 to 2020, to interpreted with caution as they rely on crude assumptions. There appears to be a downward trend in the cost per participant, which is consistent with the trend observed in the previous evaluation commissioned by CEPOL and the views of stakeholders. Table 15: Estimated cost per training participant 2015 2016 2017 2018 2019 2020 A CEPOL’s operating costs (‘000 €) 4,429 4,812 5,173 5,897 4,620 5,618 B Number of participants in training and learning activities (n) 12,992 17,728 23,494 29,003 34,723 39,401 C Average cost per participant (€) 341 271 220 203 133 143 Source: Operating costs were from CEPOLs CAARs (2015 – 2020) (as presented in Table 3 ); and participant information is taken from CEPOL CAARs (2015, 2016, 2017, 2018, 2019, 2020) Notes: Average cost per participant is the research team’s calculation, where C = A / B. When calculating estimated cost per training participant from CEPOL’s total costs (operating, staff and administrative) the trend is consistent (i.e. 2015: 1,912, 2016: 487, 2017: 394, 2018: 359, 2019: 268, 2020: 264). Representatives of the Management Board, complimented CEPOL’s internal functions for its efficient communication between staff and strong leadership from the Executive Director, while an EU-level representative reflected on the overall professionalism of CEPOL. Nevertheless, results from the 2019 CEPOL Staff Engagement survey highlight that some CEPOL staff shared less positive views about the Agency’s internal communication (see case study 1). Despite clear internal structures, there was more disagreement observed amongst survey respondents in relation to the efficiency of the CEPOL Management Board decision-making, as well as CEPOL’s planning processes and monitoring procedures. Notably, two stakeholders highlighted that the Management Board’s deliberations and decision-making process was often slowed down by some representatives’ lack of awareness of the EU law enforcement landscape and variance in terms of Member States’ participation in the meetings. According to CEPOL staff, CEPOL HQ’s internal structure appears to operate efficiently. The Agency’s move towards greater digitalisation of procedures (starting prior to the 2015 Regulation) was seen as a positive development. Nevertheless, CEPOL’s staff turnover might have affected the Agency’s overall efficiency during the 2015 to 2020 period. CEPOL has experienced a high staff turnover in the past five years, which the Agency attributes to the relocation from the UK to Hungary in 2015 and the low country coefficient applied to staff salaries. According to CEPOL’s CAARs, this relocation has not assisted CEPOL in attracting staff from the broadest possible geographical basis among the Member States (see also case study 1). In 2020, Hungarian citizens represented 40% of all staff employed at the Agency. As of 31 December 2020, CEPOL does not employ staff from Bulgaria, Cyprus, Czechia, Denmark, France, Ireland, Luxembourg, Malta and Slovenia, which reflects the lack of applications from suitable candidates in these countries. This point was raised also by the European Parliament Committee on Budgetary Control. Despite some mitigating actions being implemented, the Committee reported concerns on the possible impact on business continuity and the Agency’s ability to implement activities. Moreover, the disproportionate number of Hungarian staff should be interpreted in the context of the EU’s emphasis on attracting a diverse workforce and fostering diversion in selection and recruitment procedures. These findings are consistent with concerns raised as part of the previous evaluation of CEPOL (see case study 1). One CEPOL representative stressed that CEPOL’s internal decision-making process was sometimes hampered by the Agency’s complex management structure, which required multiple parties to approve decisions. The administrative burden on Member States was highlighted as an issue by stakeholders in the previous evaluation. CEPOL now operates under a hybrid business model, which came about further to the Member States’ rejection of a totally new business model, meant to streamline and simplify the grants management. While the majority of stakeholders agreed that the hybrid model improved the quality of training activities and helped produce a comprehensive multiannual portfolio, when asked questions relating to internal efficiency, the opinion was less positive. A relatively small share of respondents strongly agreed that the model’s capacity reduced the administrative burden on Member States and increased the efficiency of CEPOL in organising and assigning training. Overall, survey data showed that CEPOL’s Framework Partner representatives were satisfied about their working relationship with CEPOL. The majority of respondents strongly agreed that i) the selection process for Framework Partners was clear and streamlined; ii) the roles and responsibilities of Framework Partners is clearly defined and allocated; and iii) CEPOL’s procedures and decision-making process for awarding grants to Framework Partners is sufficiently fast and transparent. Yet, most Framework Partner respondents stated that CEPOL’s monitoring and reporting requirements could be streamlined and simplified. The survey data also highlighted that some of the roles and responsibilities of different national actors interacting with CEPOL (including Framework Partners, National Units and NCPs) might be somewhat unclear. The evaluation considers whether political, economic, social and cultural, or technical factors at the EU or Member State level may have affected the efficiency of CEPOL. The only external factors mentioned by stakeholders as having the potential to impact on CEPOL’s efficiency in the last five years and also representing an ongoing threat was the COVID-19 pandemic. This evaluation has considered whether the resources allocated to CEPOL have been adequate for the Agency to carry out its mandate since 2015. There was no significant increase in CEPOL’s EU annual subsidy in the period before and after the new legal basis came into effect in 2016, despite the fact that CEPOL’s mandate was expanded. The Internal Audit Service (IAS) Audit Report from 2020 highlighted that an inherent risk for CEPOL is the “lack of agreement between the Agency and the European Commission regarding the resources necessary to fulfil its mandate.” Consistent with this, a number of Member State-level, JHA agency and EU-level representatives raised these concerns, similarly to the previous evaluation of CEPOL. Many of the respondents did not consider CEPOL’s current resources to be sufficient in light of emerging and new challenges facing the Agency. The evaluation identified two areas where there was compelling evidence from both desk and field research that additional resource could have further enhanced CEPOL’s ability to meet its objectives during the period from 2015 to 2020: CEPOL’s ability to reach a larger target audience and CEPOL’s information and communication technology (ICT). The analysis of the sufficiency of CEPOL’s resources also takes into account CEPOL’s budget implementation. As shown in Table 16, it can be seen that CEPOL was capable of absorbing the funding provided to the agency by the EU.

· Budget implementation 2015 2016 2017 2018 2019 2020 Budget (EU contribution) (excl. external assigned revenue) € 8,471,309 8,641,000 9,280,216 10,431,827 9,308,000 10,423,904 Budget cancelled by MB (amending budget) € 0 0 0 0 0 2,594,654 Total authorised agency EU contribution (closing) € 8,471,309 8,641,000 9,280,216 10,431,827 9,308,000 7,829,250 Committed € 8,090,826 8,290,875 9,009,906 10,205,621 9,298,748 7,179,123 Cancellation of commitment appropriations of the year (budgeted less committed) € 380,174 350,125 270,094 211,099 9,252 650,127 % 4 4 3 2 0 8 Paid € 6,692,232 6,813,587 7,796,649 7,970,203 8,314,318 5,806,184 Automatic carry forward of appropriations of the year into the subsequent year (committed less paid) € 1,398,594 1,477,288 1,213,257 2,235,418 984,430 1,370,471 % 17 17 13 21 11 18 Cancellation of payment appropriations carried over from the preceding year € 180,662 0 0 0 0 0 % 2 0 0 0 0 0 Excess balance economic outturn account of the year (recovered in the year after on final agency annual accounts and recycled into the budget of year n+2) € 559,216 567,666 460,918 339,479 212,578 772,772 as % of total authorised EU contribution % 7 7 5 3 2 10 SCEPOL’s CAARs 2015 – 2020, CEPOL Report on budgetary and financial management for 2020 (art. 9), the evaluation team notes that the summarised data was kindly provided by a representative from DG HOME.



5.3. Relevance

Key findings: • CEPOL’s objectives are overall relevant to the EU’s needs and challenges. ·CEPOL’s objectives are