EUROPEAN COMMISSION Brussels, 27.6.2019 SWD(2019) 282 final COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament and the Council Evaluation of the 2015-2019 action plan on firearms trafficking between the EU and the south-east Europe region {COM(2019) 293 final}



Evaluation of the 2015-2019 action plan on firearms trafficking between the EU and the south-east Europe region Contents Table of acronyms 1. Introduction 2. Background to the intervention 2.1. Objectives of the action plan 2.2. Legal nature of the action plan and governance 2.3. Intervention logic 2.4. Baseline 2.5. Current policy and other developments 3. State of Play 4. Methodology 4.1. Sources and methods used 4.2. Limitations and robustness of findings 5. Analysis and Answers to the Evaluation Questions 5.1. Relevance 5.2. EU Added value and sustainability 5.3. Effectiveness

5.4. Efficiency 5.5. Consistency 6. Conclusions Annexes Annex 1. Procedural information Annex 2. Stakeholder consultation General overview Results of the questionnaire Meetings of the Joint Committee between European Union Firearms Experts and South East Europe Partners Experts Annex 3. Methods and analytical model Annex 4. Description of activities under the Firearms Platform Annex 5. Description of activities carried out by the South Eastern and Eastern Europe Clearinghouse for the Control of Small Arms and Light Weapons Annex 6. Training activities Annex 7. Conclusions of the meetings of the Joint Committee between European Union Firearms Experts and South East Europe Countries Experts Annex 8. List of National action plans adopted by Western Balkan Partners Annex 9. General overview of activities carried out



Table of acronyms

CEPOL EU Agency for Law Enforcement Training CFSP Common Foreign and Security Policy EFE European Firearms Expert Group – EU Experts EMPACT European Platform Against Criminal Threats - the Multidisciplinary Platform Europol EU Law Enforcement Agency EBCGA EU Border and Coast-Guard Agency ("Frontex") NATO North Atlantic Treaty Organisation OSCE Organisation for Security and Cooperation in Europe SALW Small arms and Light Weapons – small arms SEEFEN South East Europe Firearms Expert Network - also known as the Western Balkan Experts SEESAC South Eastern and Eastern Europe Clearinghouse for the Control of Small Arms and Light Weapons –also known as the Clearinghouse SELEC Southeast European Law Enforcement Centre SIENA Secure Information Exchange Network Application UNODC United Nations Office on Drugs and Crime UNDP United Nations Development Programme



1.Introduction At the end of 2014, the Council of the European Union and the Ministers of Interior of the Western Balkans partners agreed on an action plan on firearms trafficking between the EU and the south-east Europe region. In its Communication of 6 February 2018, "A credible enlargement perspective for and enhanced EU engagement with the Western Balkans", the Commission stressed the need to strengthen its efforts against firearms trafficking. During their meeting in Sofia at the EU-Western Balkan Summit, on 17 May 2018, heads of states and governments set as a priority to "prepare a renewed action plan for cooperation on firearms to address more effectively illicit firearms and the large stockpile of weapons". On 26 January 2018, the second meeting of the Joint Committee between EU firearms experts and experts from partners in south-east Europe partners experts was held. At this meeting, the Commission and representatives from both the EU member states and the Western Balkan partners underlined the need to assess the different initiatives that had been carried out to the implement the action plan, to inspire future policy developments. This evaluation aims to inform policy makers about the functioning of the action plan and possible future policy needs. Many players cooperate with the Western Balkans in the fight against firearms trafficking and this evaluation provides an overview of what has been achieved between January 2015 and December 2018 in the region. The evaluation assesses the functioning of the action plan, based on five evaluation criteria: the relevance, effectiveness, efficiency, consistency and EU added value of the action plan. The evaluation also aimed to identify needs and policy gaps that may need being addressed to respond to the threat of firearms trafficking in the Western Balkans and between the Western Balkans and the European Union. This evaluation takes into account the EU member states, EU agencies and international organisations that cooperate with the Western Balkans in the fight against firearms trafficking. It analyses the implementation of the action plan and focuses in particular on a series of activities such as: ·activities of the South Eastern and Eastern Europe Clearinghouse for the Control of Small Arms and Light Weapons (SEESAC – hereinafter 'the Clearinghouse'); ·activities of the European Platform Against Criminal Threats (EMPACT) Firearms (hereinafter 'the Firearms Platform'); ·actions by EU agencies such as the EU Law Enforcement Agency (Europol), the EU Border and Coast-Guard Agency (EBCGA) or the EU Agency for Law Enforcement Training (CEPOL); ·the bilateral actions of member states and Western Balkan partners; ·activities of other partners such as the North Atlantic Treaty Organisation (NATO), the United Nations Office on Drugs and Crime (UNODC) or the Organisation for Security and Cooperation in Europe (OSCE). For the purpose of this evaluation, the term 'small arms and light weapons' (hereinafter 'small arms') is used to describe military-grade weapons and the term 'firearms' is used solely to describe civilian firearms.

The evaluation covers the Western Balkan region and in particular Albania, Bosnia and Herzegovina, Kosovo *, Montenegro, Serbia and North Macedonia. Although the action plan formally concerns the broader south-east Europe, cooperation has in practice focused on the Western Balkans. The assessment therefore does not include the Republic of Moldova and Ukraine. The reasons for this are explained in Section 5.1 on relevance. 2.Background to the intervention 2.1.Objectives of the action plan On 5-6 November 2012, ministers from the south-east Europe region met at the EU-Western Balkans ministerial forum on Justice and Home Affairs in Tirana. At this meeting, the ministers acknowledged the need to increase efforts to counteract the illicit trafficking and accumulation of firearms, their parts and essential components and ammunition in the south-east Europe region. They also stressed their desire to work together with EU partners towards a joint solution for the whole region. The ministers identified the following needs: ·To improve the exchange of criminal information and intelligence at regional level and with EU member states, involving Europol, on the production, stockpiling and trafficking of firearms and ammunition; ·To develop more effective investigative and intelligence standards; ·To improve operational law enforcement co-operation at regional level and with EU member states and Europol. This should focus on controlling the production, stockpiling and trafficking of firearms and ammunition; ·To improve the collection and exchange of statistics on the production, stockpiling and trafficking of firearms and ammunition; ·To promote networking at all levels, exchange of best practices and joint training among experts in the south-east Europe region; ·To harmonise national legislation on firearms with EU and international standards. To meet these needs, the action plan was set up. It contains a set of general objectives (goals) and specific measures (actions) to reduce and possibly halt the illicit flow of firearms to the EU in the period 2015-2019. Goal 1 focused on the modernisation of law enforcement agencies. Action 1 required every partner in the south-east Europe region to create a firearms focal point by the end of 2019. This focal point must use all available tools for tracing firearms, including the weapon tracking database iTRACE, Interpol's iARMS database, which lists seized, stolen and lost weapons, and the Europol Information System. These focal points are required to coordinate all firearms-related actions and work on a similar basis, aligned through the best practice guidance for the creation of national firearms focal points. Action 2 provided for a pilot project on data collection. The Joint Committee committed to provide comprehensive data on seizures of weapons as part of the Global Firearms Study of the United Nations Office on Drugs and Crime, funded by the European Commission. Action 3 required a study to be conducted on systems for the exchange of information in order to improve the strategic and operational analysis of information. The Joint Committee stressed the need to improve the exchange of information at regional level and with member states. It also stressed the importance of involving different organisations in this information exchange, such as Europol. It said that the information exchanged should include ballistic identification and it was important to make better use of the Secure Information Exchange Network Application (SIENA) Goal 2 was dedicated to enhancing mutual trust. Action 1 called for joint meetings of firearms experts. The Joint Committee went a step further by recommending that representatives of southeast Europe should be invited to meetings of the European firearms experts group. Action 2 was dedicated to organising joint operations (including yearly joint actions days), and to involving Western Balkan partners the European Multidisciplinary Platform against Criminal Threats – Firearms. The Joint Committee also stressed the need to provide a consistent framework for cooperation between the EU and the south-east Europe region in the fight against firearms trafficking. Goal 3 focused on capacity building.

Action was designed to focus on organising training activities on firearms trafficking. The Joint Committee also requested a feasibility study on: (i) the continuation of voluntary surrenders of unregistered weapons; (ii) continuous support to destruction of surplus firearms; and (iii) improving the security and safety of stockpiles. 2.2.Legal nature of the action plan and governance The action plan was endorsed by the Council and included in a joint declaration of the EU – Western Balkans Ministerial Forum on Justice and Home Affairs of 12 December 2014 in Belgrade. This action plan is a political and programmatic document that is not based on a formal Commission proposal. It does not include provisions related to its implementation, except that "senior officials of the south-east Europe partners will regularly report to the European Commission on implementation of the above mentioned actions so as to allow the EU member states make the best use of this information in the context of the framework of the EU Policy Cycle." It therefore makes a link between the European Commission and the EU policy cycle, which is a policy process led by member states. The action plan also refers to the need to "call for regular joint meetings of firearms experts from the established groups and networks in the region to assess ways to exchange information, evaluate joint initiatives aiming at improving cooperation and reducing the risk of overlaps among different actions in this field" (Goal 2, Action 1). However, the action plan does not mention the respective responsibilities of the various players (for instance on pilot projects, studies, organisation of meetings, coordination of activities, joint actions or training activities). These responsibilities can be inferred from the introductory references to the Firearms Platform, to the European Firearms Expert Group and to the Commission. For this reason, the governance and implementation of the action plan mostly developed in a pragmatic way, with the Commission and various stakeholders taking individual initiatives. Because of this (see section 5.5 on consistency), the Commission has attempted to coordinate the various streams of work, mostly by playing an active role in the Firearms Platform, attending all firearms expert meetings in the Western Balkans and organising the joint meetings of firearms experts from the EU member states and Western Balkans partners. 2.3.Intervention logic No structured intervention logic was decided upon before the drafting of the action plan. The aim of the action plan is to reduce all risks linked to the illegal use of firearms and more in general the fight against illicit trafficking in firearms. Reconstructing the intervention logic requires to set out the link with these risks and conversely, what it should be the impact of the action plan on these risks. It is difficult to assess the achievements of the action plan. It is also difficult to assess if any progress in the fight against firearms trafficking was due to the implementation of the action plan or different factors. The following diagram is an attempt to reconstruct this approach in hindsight. It takes into account the conclusions of the meetings of the Joint Committee of EU firearms experts and firearms experts from south-east Europe partners. 2.4.Baseline Illicit trafficking of firearms is the lifeblood of organised crime and terrorism in Europe. In particular, it facilitates drug trafficking through intimidation and coercion, fuels turf conflicts between urban gangs, and empowers terrorists. Firearms pose a serious threat to the EU’s security. It is only by recognising this threat that we will be able to: (i) prevent and fight serious and organised crime; (ii) tackle and prevent terrorism; and (iii) strengthen border security. The EU and countries in south-east Europe have always had a shared interest in increasing their cooperation to address the threats posed by illicit trafficking in firearms. This interest is underpinned by the prospect of EU accession for the countries of south-east Europe. However, before the adoption of the action plan, EU law enforcement agencies had no regular exchanges of best practices and intelligence with law-enforcement authorities in the Western Balkans. Regional cooperation within the Western Balkans was only starting, and there was no long-standing body of Western Balkans firearms experts. Legislation in the Western Balkans was not aligned with EU law, and this opened up opportunities for criminals to avail of legislative discrepancies and easily buy and sell firearms. Since the early 1990s, the firearms trafficked illicitly in the EU have originated from three main sources. Firstly, countries in the former Soviet Union and Warsaw Pact bloc were a source of illicit firearms due to the fall of the iron curtain. The ongoing conflict in Ukraine could increase the risk of firearms being diverted into the EU (especially following a possible peace process). Secondly, as a consequence of the wars of Yugoslav succession, the Western Balkans became and are still considered a significant source of illicit firearms. More recently, in the aftermath of the "Arab spring", ongoing conflicts and weak states present an increased risk of diversion from North Africa and the Middle East possibly following some of the main drug trafficking routes into the EU. Several reports from Europol confirmed the threat of firearms and mentioned the need to increase the flow of information and intelligence between the EU and the Western Balkans. In 2014, the first operational action plan on crime was produced as part of the EU Policy Cycle 2014-2018 under the section on "Firearms". The action plan mentioned the need to increase the flow of information and intelligence between the EU and the Western Balkans. The 2015 terror attacks in Paris, Copenhagen and on a Thalys train, underlined the link between organised crime and terrorism. This link was especially clear in the issue of firearms trafficking. These attacks were clear evidence of how this multidimensional terror threat involved different areas of crime. These events showed why there was a need to strengthen further the fight against trafficking of firearms, through a coordinated and consistent approach. As the SAFTE research programme on firearms acquisitions by terrorists in Europe concluded, "today the main source countries for firearms smuggled into the EU are generally situated in the Western Balkans". SAFTE stressed the importance of increasing firearms checks in the EU neighbourhood area to strengthen EU internal security and prevent terrorism.

To compensate for the absence of structured law enforcement cooperation against firearms trafficking, the EU has for a long time been involved in various forms of technical support as part of the Common Foreign & Security Policy and the revised European Neighbourhood Policy. This technical support has aimed at blocking trafficking routes, improving the management of firearms stocks, and preventing the diversion of firearms from the legal market, especially in the Western Balkan region. The action plan tries to improve the situation before its adoption. Limited data were available at the start of that action plan’s implementation to enable an analysis of the evolutions. According to available figures for 2015, 2500 weapons were confiscated and 4800 destroyed throughout the Western Balkans in 2015. In terms of operational cooperation, in 2015, 75 SIENA messages linked to the Western Balkans were exchanged with Europol, which is without proportion to the threat of firearms coming from the region identified by the agency. Only a negligible undetermined number of firearms were seized during the first joint targeted checks carried out in 2015. As mentioned above, on 5-6 November 2012, the ministers of the south-east Europe region acknowledged the need to increase efforts to counteract the illicit firearms trafficking. This acknowledgment was reiterated on 19-20 December 2013, in Budva, Montenegro, where the ministers decided to set up a network of experts in firearms trafficking. Other initiatives were also taken place, which are referred to in Section 5.5 Consistency. These initiatives included the 2013 Commission Communication on "Firearms and the internal security of the EU." This Communication stated, "large amounts of powerful military grade weapons have since the mid-1990s reached the EU from the Western Balkans". The Communication also referred to initial attempts to structure law enforcement cooperation before the establishment of the EU policy cycle and of the Firearms Platform. It said that these initial attempts sought to create an "overall plan for coherent operational action" including "police control operations to tackle the principal sources and routes of illegal firearms, including the Western Balkans." In 2002, the risk of proliferation of weapons led the EU to begin providing its support to the Clearinghouse. This support has continued since 2002 and is given, under the joint mandate of the United Nations Development Programme (UNDP) and the Regional Cooperation Council. Funded since 2002 by the European Union, it has been working to: (i) strengthen the capacities of national and regional stakeholders; (ii) control and reduce the proliferation and misuse of small arms and light weapons; and (iii) facilitates coordination on small arms issues among governments and other stakeholders. At policy level, The Clearinghouse facilitates regional processes through meetings of small arms commissions and of the south-east Europe firearms expert network (SEEFEN – hereafter Western Balkan Experts), involving representatives of police, customs, prosecutors and ballistics experts. In addition, a "Memorandum for cooperation and for establishing an expert group for measures against illegal trading of firearms in south-eastern Europe and promotion of the joint cooperation" (SEEFEG) was signed in October 2015 in North Macedonia, by high police officials from North Macedonia, Serbia, Bosnia and Herzegovina, Montenegro, Albania and Kosovo*. This group was embedded as a subgroup of SEEFEN and it was composed by police representatives only. 2.5.Current policy and other developments Even though Europe has some of the toughest rules on firearms in the world, criminals always find new ways to circumvent the rules, which led to continued misuse of weapons, with direct and indirect impacts on the security of citizens. This had led the Commission to adopt in 2013 a Communication on Firearms and the European Agenda on Security on 28 April 2015 . This integrated policy aimed at addressing this threat, through legislation, operational action, training and EU funding. This policy is built around four priorities: ·Safeguarding the licit market for civilian firearms through new European Union (EU) standards on which firearms can be sold for civilian use, how firearms should be marked, and how to licence persons who wish to possess and to use firearms. This led to the revision of the Directive N. 91/477/EEC on control of the acquisition and possession of weapons in 2017 ; ·Reducing diversion of firearms into criminal hands through the development of effective standards on safe storage of civilian firearms and on how to deactivate civilian and military firearms, and greater efforts to reduce illicit trafficking of firearms (whether civilian or military) from outside the EU ; ·Increasing pressure on criminal markets through better cross-border cooperation between police, customs and border guards and by assessing the need for common EU rules on which offences linked to firearms should be criminalised and what level of criminal sanctions should be imposed by member states; ·Building better intelligence by gathering and sharing more information on firearms crimes, and by targeted training of law enforcement officers. Since 2015, the Commission has organised four joint meetings of firearms experts of the EU member states and Western Balkans partners that adopted operational conclusions to implement the action plan. On 10 July 2018, Western Balkan partners subscribed to a Regional Roadmap (hereafter the Roadmap) for a sustainable solution to the illegal possession, misuse and trafficking of Small Arms and Light Weapons and their ammunition in the Western Balkans by 2024. This Roadmap was drafted and developed at the initiative of the French and German Ministries of Foreign Affairs in the context of the "Berlin Process", an intergovernmental work stream aimed at stepping up regional cooperation in the Western Balkans and aiding the integration of the region into the European Union with the involvement of selected EU member states. This Roadmap essentially amounts to a needs assessment and a set of general objectives to be achieved by 2024, with the identification of 14 Key Performance Indicators. On 19 November 2018, the Council adopted a Decision endorsing the Roadmap and entrusting the South Eastern and Eastern Europe Clearinghouse for the Control of Small Arms and Light Weapons to support its implementation until December 2019. This action is funded under the Common Foreign and Security Policy (CFSP) budget. On 18 June 2018, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy adopted a Joint Communication on Elements towards an EU Strategy against illicit Firearms, Small Arms & Light Weapons and their Ammunition. The Council adopted this Strategy on 19 November 2018. According to it, the EU continues to support the Clearinghouse "to maintain cooperation with relevant regional organisations in the field of Small Arms control and take into account regional initiatives such as the Roadmap addressing illicit Small Arms in the Western Balkans." 3.State of Play Stakeholders have always expressed their commitment to implement the different goals of the action plan. Nevertheless, on a number of occasions, political commitments were not always translated in operational actions. Certain challenges of information, involvement of Western Balkan partners in the EU policy cycle and the development of national focal points had to be stressed repeatedly during joint meetings before they could be addressed. However, most of the activities announced in the action plan (see Section 2.1) were implemented during these years. Goal 1 - related to the modernisation of the law enforcement agencies. Under Action 1, countries of the region worked to set up national firearms focal points. The Clearinghouse has provided assistance using the financial and political support channeled to the region through the EU. The Clearinghouse provided targeted equipment, training and advice. In parallel, the EU adopted best practice guidance for the creation of national firearms focal points. As of 31 December 2018, 4 focal points were established in the Western Balkans (the focal point in Kosovo* was fully operational, and the focal points in Albania, the Republic of North Macedonia, Serbia had been set up but were not yet operational). Under Action 2, the Clearinghouse developed a regional small arms survey methodology for a survey. The aim of this was to promote: (i) capacity building of law enforcement agencies, (ii) and better data flow for coordinated assessment. The methodology sought to ensure that consistent and harmonised data that were disaggregated by gender and age would be provided on possession (legal and illegal), legal distribution, stockpiles and surplus reduction, exports and on the impact of firearms misuse. All six Western Balkan partners covered by this evaluation already provided input to the survey and they are now using this methodology for the threat assessment. In the meantime, the United Nations Office on Drugs and Crime (UNODC) started tan EU financed project for collection of data on firearms seizures involving Western Balkans partners. Under Action 3, the Clearinghouse developed a feasibility study on linking small arms & light weapons registration systems in south-east Europe. The study sought to improve the strategic and operational analysis of information of legal firearms and prevent diversion.

Goal 2 – related to enhancing mutual trust Under Action 1, the Commission organised four meetings of the Joint Committee. Twice a year the Clearinghouse has organised regional meetings for strategic and policy level decision makers (regional meetings of the small arms commissions) and meetings of tactical-operational level representatives of Western Balkan law enforcement and judiciary authorities (SEFEN). The meetings was an opportunity for the Commission to introduce and explain the EU legislative framework on the control of acquisition, possession, transfers, exports, imports and transit authorizations for firearms, and for EU member states to explain how those provisions were transposed into national legislation. In the conclusions of these meetings, EU and Western Balkan partners said that it was important to more systematically exchange information, carry out dedicated checks and make full use of available tools for tracing firearms, such as iTRACE, iARMS and the Europol's databases. Intelligence sharing between EU and Western Balkans remains an issue, as well as the implementation of more effective investigative and intelligence standards. Penal policy in the Western Balkans should be strengthened and better enforced. Works must be done to improve stockpile management and prevent theft. Finally, the collection of statistics on the illicit production, and trafficking of firearms remains a challenge. Representatives of the European Commission, the European Firearms Expert Group, the Firearms Platform, Europol, and the European Border and Coast Guard Agency (the EBCGA) are regularly invited. Similarly, Western Balkan experts are now invited to the annual meeting of European Firearms Experts Group; and all Western Balkan Partners have been invited as full participants in all Firearms Platform Activities. The participation of Western Balkan partners in these meetings is fully funded from the budget of the Firearms Platform. Under Action 2, the core of operational cooperation was carried out as part of the Firearms Platform. The action leaders were Sweden in 2015 and 2016, the UK in 2017 and Spain since 2018. Joint coordinated checks operations were organized every year, mostly at selected border crossing points (checks on international coaches) and registered firearms dealers (see annex 4 for more details). These checks were heavily supported by Europol's Analysis Project "Weapons and Explosives", and in particular by Europol’s team of analysts, its joint operation centre and its Europol mobile office. The European Border and Coast Guard Agency (EBCGA) also provided substantial support to strengthen border checks in and around the region, since firearms trafficking falls under the definition of "cross-border crime" for which it is responsible. The EBCGA has notably identified the Western Balkans as an important operational area. It notably supported the coordinated data collection on firearms trafficking (linked to Action 2) by working together with the Western Balkan Risk Analysis Network set up by the Agency to elaborate a booklet containing a set of common risk indicators. To facilitate its use, the booklet was translated into three regional and local languages. Exchanges of information were enabled by the signature of operational cooperation agreements between Europol and Serbia, Albania, Bosnia and Herzegovina, North Macedonia and Montenegro. These agreements provide the opportunity to exchange information and intelligence and to cooperate on operation. For example, this cooperation could take place within joint investigation team with third countries outside the EU. Goal 3 – related to capacity building The Clearinghouse developed specialised training course modules for operational analysts, criminal investigators, criminal intelligence analysts, and forensic investigators. It assisted government institutions in coordinating the small arms control activities, promoting EU standards in small arms control. The Clearinghouse also assisted government institutions in the implementation of the national strategic master plans and action plans covering illicit trafficking in firearms and ammunition. It also assisted Western Balkan partners to adapt their national legal framework and bring it into line with EU and international standards. The EU Agency for law enforcement training - CEPOL developed working arrangements with all Western Balkan countries, except North Macedonia, for which negotiations are ongoing. The Agency organised residential activities, webinars, online modules and an exchange programme. Training and legal assistance was also provided by the EU "Partner to Partner Export Control Programme" for arms managed by the German Federal Office for Economic Affairs and Export Control BAFA. This training and legal assistance was funded from the Common Foreign and Security Policy budget. Consulted stakeholders recognised that all these activities contributed in a general movement of legislative upgrades throughout the region. Work has continued on voluntary surrenders of unregistered weapons owned by civilian across the region. The goal of this work is to reduce the number of illicit firearms by encouraging firearms owners to return their weapons without fear of prosecution. Surrendered weapons are either legalised and duly registered, subject to authorisation, destroyed, or returned to police or army stockpiles. In 2016, the Commission funded a study on voluntary surrender/legalisation of firearms programmes throughout the Western Balkans. Several voluntary surrenders campaigns have taken place since 2015 (Serbia from March to June 2015, Montenegro from June 2015 to June 2016, Albania from the end 2016 to March 2017, and in Kosovo* from December 2018 to June 2019; the results of these campaigns were not made public). Nevertheless, stockpiles of weapons remain significant. Finally, EU member states and Western Balkan partners continued to develop their existing bilateral cooperation, notably through the network of security liaison officers in embassies. Although this bilateral cooperation is not part of the implementation of the action plan, it is worth mentioning because it will inform relationship between the action plan and other initiatives. Sweden, for instance, signed strategic bilateral cooperation agreements on joint combating of cross-border criminality notably with Kosovo* and Bosnia and Herzegovina. In September 2016, France and Serbia signed a protocol creating a permanent Serbian-French criminal intelligence unit to combat international trafficking of arms and explosives. Germany provided substantive support by giving technical equipment, legal training and capacity building to Bosnia and Herzegovina in the project "Countering Illicit Arms Trafficking in Bosnia and Herzegovina" (implemented by the Clearinghouse). The UK set up a regional "Western Balkans firearms capacity building programme" which involved the British National Ballistic Intelligence Service. Methodology Sources and methods used This evaluation relies primarily on an extensive consultation of member states’ enforcement authorities, Western Balkan partners, EU agencies, and the international organisations involved on the ground. The Commission, due to its easy access to restricted information and its participation in all expert meetings, carried it out. The evaluation takes into account the content and outcome of all the meetings that have taken place with these stakeholders between 2015 and late 2018. These meetings include joint meetings of EU and Western Balkans firearms experts, the meetings of the Western Balkan Experts, the meetings of the European Firearms Experts Group (EU Experts), the Firearms Platform meetings (including the debriefing meetings following joint actions days), and the meetings of Small arms and Light Weapons commissions in the Western Balkans. The desk research relied on several types of sources. First, the Commission took into consideration a number of external studies conducted by independent organisations. Second, it used all the data included in the presentations made by national authorities at each meeting of firearms experts mentioned above (the Firearms Platform, Western Balkan Experts, EU Experts, Small Arms commissions). Documents produced by the Clearinghouse also proved particularly valuable. On operational cooperation, a major source of information was the debriefing of joint coordinated targeted controls and the reports of the action leaders. The meeting documents and presentations made during meetings of the Western Balkan Experts also contributed greatly to building a comprehensive picture of the achievements of countries and stakeholders. However, due to a lack of organised steering in the implementation of the plan, available information remains patchy. The Commission relied heavily on the continuous feedback from law enforcement authorities themselves, who made presentations and adopted conclusions during the meetings of the Joint Committee (see Annex 7 for the detail of the conclusions of each meeting). The Commission also took as a valuable source of information the needs assessment for the region, and the analysis of current challenges developed during the preparation of the regional roadmap for a sustainable solution to the illegal possession, misuse and trafficking of small arms and light weapons. The elaboration of the Roadmap was carried out between 1st February 2018 and 29 May 2018 at the initiative of the ministries of Foreign Affairs of France and Germany, coordinated by the Clearinghouse and to which the Commission, the Firearms Platform, the EBCGA and Europol were closely associated. For each of its Goals, the Roadmap includes a short description of the baseline in May-June 2018, which effectively describes the current situation; i.e. the needs analysis. This exercise complemented the information gathered through a targeted consultation of stakeholders. This consultation took the form of a questionnaire. On 26 January 2018, the Joint Committee between European Union Firearms Experts and South East Europe Partners Experts had requested that the Commission circulate a questionnaire to all EU and Western Balkan competent administrations to collect information about the initiatives carried out by the countries according to the action plan, the Firearms Platform activities and the conclusions of all relevant meetings. On that basis, the questionnaire was sent to members of the European Firearms Experts (EFE) Group, of the South East Europe Firearms Expert Group (SEEFEG), to European Agencies and other stakeholders (such as the Clearinghouse) on 27 July 2018. No specific public consultation took place, since the action plan was addressed to public authorities only, and all information on the actions implemented under the action plan (trainings, coordination meetings, exchange of information between law enforcement agencies, investigations etc.) was available only to the specific stakeholders involved in the implementation of the action plan, and to which the general public does not have access. Therefore, a public consultation would not have provided any additional data for the evaluation of the action plan compared to the targeted consultation. Limitations and robustness of findings The Commission relied mostly on qualitative reporting. This qualitative reporting was made more necessary because of the absence of a defined intervention logic at the time of adoption led to a lack of performance indicators and of a scoreboard in the action plan. This was reinforced by an uneven detail of reporting. The analysis was also hampered by a lack of data at the EU level and limitations in the comparability of information. First, the absence of a clear baseline, with reliable data, made it difficult to compare any evolution. Second, access by the Commission to operational information, including for operations where Europol was involved, was limited. Sometimes, the Commission also received contradicting data from different sources which made it difficult to evaluate the involvement of countries. Statistics on seizures, voluntary surrenders and destructions or weapons are not sufficiently detailed to allow a trend analysis and a comparison between jurisdictions. Even data collected during joint coordinated targeted checks over the years were often too dissimilar from one year to another to enable comparison, even though all such checks were coordinated within the Firearms Platform and with the involvement of Europol. This has made it difficult to gather a comprehensive picture during the research.

The assessment made by stakeholders consulted in the targeted questionnaire sometimes refers to the overall European framework for cooperation against firearms trafficking and was not always limited to cooperation in and with Western Balkan partners. Responses to the questionnaire sent were of varying quality and length, which sometimes made it necessary to interpret replies or infer answers to ensure comparability between them. Details on the limitations can be found in Annex 3. The assessment of the various firearms-related workshops financed by the Technical Assistance and Information Exchange (TAIEX) instrument of the European Commission was made impossible. This was because applicants were not required to submit reports on the workshops until 2018. It also proved a challenge to evaluate changes in the funding of the Firearms Platform for activities in or with the Western Balkans over the years (for joint coordinated targeted checks, other operational meetings, equipment and informants), since the management team does not have this information for the years 2015 to 2017. Taking into account all the limitations, the evidence was based on testimonies and qualitative assessments rather than figures. However, when available, figures did confirm the opinions of stakeholders. Besides, the convergence of these contributions guarantees the reliability of the conclusions and replies to evaluation questions. Analysis and Answers to the Evaluation Questions Relevance To assess the relevance of the action plan, the evaluation examined whether the action plan was adapted to the past and current security needs, in terms of geographical scope and content. The evaluation also examined whether it is still adapted to current security considerations. As regards the geographical coverage, although the action plan formally concerns south-east Europe, cooperation has in practice been focusing on the Western Balkans. Firearms trafficked to, in and from the Western Balkan were considered a threat for the whole continent at the inception of the action plan, as explained in section 2.4. The threat remains acute. This is confirmed by law-enforcement agencies. It is also demonstrated by regular terrorist or criminal attacks, and by seizures in which firearms (converted or not) can be tracked down to this region. The ongoing threat was confirmed by several studies, the most recent of which stresses that weapons used in recent terrorist attacks "were acquired through local illicit firearm markets and were found to originate from the Balkans". This study also said that "the Balkans were identified as the main supplier of firearms in Europe" and spoke of "the link between the Balkans and small arms used in EU member states in various types of criminal activity." In addition the study said that, "the comparison of ballistic information from Swedish and Serbian police investigation showed that the firearms found in Sweden were used first in Serbia before being smuggled into Sweden to commit other crimes." Representatives from law enforcement authorities in the EU and Western Balkans point to a decrease in the number of illicit firearms owned in the Western Balkans. However, new trends are also emerging in the region. These new trends include changes in the conversion techniques for turning blank-firing or gas and alarm weapons into firearms, concealment methods, and distribution patterns which have resulted in a shift from exports of traditional conflict weapons to imports of new weapons. The Western Balkans region appears to be the most relevant for fighting firearms trafficking in the medium term and for the continent's stability as a whole. It also has a European perspective, recognised most recently by the Commission Communication of 6 February 2018, which encouraged approximation of national laws with EU standards. Besides, due to their shared history and common culture, strong links remain between the partners, which also facilitated cooperation. However, according to several stakeholders’ reports during meetings, the Western Balkan source of illicit weapons is starting to decrease in importance thanks to the combined efforts of law enforcement authorities. Therefore, Ukraine or Moldova, where control of firearms is not as rigorous as in the EU or in the Western Balkans, is likely to become a new area of interest. Therefore, cooperation with Belarus, the Republic of Moldova and Ukraine needs developing, and expansion of the activities of the Clearinghouse point in this direction. The Firearms Platform is also reflecting on the expansion of the current Operational Action "cooperation with the Western Balkan" to those two countries. In the consultation, stakeholders confirmed this overall assessment, and considered that an action plan is needed, but said that the current action plan was not fully adapted to current security challenges and emerging threats and trends. This was also confirmed by the needs assessment and the discussion taking place between all players under the adoption of the regional roadmap. Chart 1.Relevance of the action plan for current needs The perception of the relevance of the action plan was substantially different between Western Balkan partners and the EU member states. The first are generally more positive about it than EU member states, many of which seem often not concerned by EU-Western Balkans co-operation: a large number of EU member states either did not reply to the questionnaires (10 countries) or did not reply to specific questions, while all Western Balkans returned the questionnaire. This indicates the fact that not all EU member states are equally affected by firearms trafficking from or to the Western Balkans (e.g. Cyprus, Latvia, Ireland, Malta and Portugal). It may also reflect the fact that a lack of staff may have prevented some EU member states from engaging further in operational cooperation at a multilateral level (e.g. Austria, Germany, Greece and France). To assess whether the action plan is still relevant and adapted to current security risks, it is necessary to compare initial needs and current needs. As mentioned in Section 2.1, the starting point that led to the adoption of the action plan was the need to: (i) improve the exchange of criminal information and intelligence; (ii)improve operational co-operation; (iii) improve the collection and exchange of statistics concerning the production, stockpiling and trafficking of firearms and ammunition; (iv) promote networking , the exchange of best practices and training and (v) bring national legislation on firearms closer to EU and international standards. These overarching needs have not changed. As will be discussed in the following sections, there is still room for improvement in operational cooperation, exchange of information, data collection, training and approximation of legislation. This is confirmed by the work, which led to the adoption of the roadmap, by the answers received during the targeted stakeholder consultation and by the conclusions of the joint meetings of EU and Western Balkans firearms experts. However, the evaluation also pointed at gaps in the current action plan. The action plan focused on structures, law enforcement cooperation and technical assistance and training. However, it neglected aspects such as the strengthening of criminal sanctions, criminal procedures and criminal policy. These three issues appeared to be significant: according to law enforcement officials, available figures on convictions, criminal sanctions in the Western Balkans often do not adequately address trafficking of firearms as a separate crime. Trafficking in firearms is rarely prosecuted. Similarly, the action plan did not focus on awareness-raising activities regarding stockpile management, surplus reduction or the gender perspective (such as the underrepresentation of women, awareness of gender-based violence etc.). Key findings ·The action plan focused the main issues of the fight against firearms trafficking by considering common administrative structures, agreed way of working and common training needs; ·The action plan promoted the collection of data for threat assessment in a coordinated way and identified common threats. ·It promoted joint coordinated targeted controls supported by EU Agencies. ·The action plan gave insufficient attention to criminal policy, the reinforcement of criminal sanctions and criminal procedures. · The action plan does not address new emerging threats and trends of firearms trafficking. ·The action plan does not sufficiently tackle future possible threat linked to trafficking to or from Ukraine and Moldova. ·Not all EU member states feel the need to increase multilateral cooperation with the Western Balkans, due to more pressing priorities or lack of staff. ·The action plan does not sufficiently take into consideration the issue of criminal sanctions and policies, and of stockpile management and destruction of surplus arms and ammunition.

EU Added value and sustainability To assess the EU added value of the action plan, the evaluation examined whether the actions envisaged under the action plan were better achieved than through bilateral cooperation or intergovernmental initiatives. It also examined whether the results were sustainable. Finally, it considered whether the termination of the action plan would have negative impacts on the objectives pursued. The quantitative analysis of the EU added value could not be carried out. As explained later (Section 5.4), the composition of potential indicators such as firearms seizures, number of voluntary surrenders of weapons, size of stockpiles, etc., still vary across the region and over time. Statistics are either not systematically available or still insufficiently harmonised, especially in countries that still rely on paper archives rather than centralised analytical tools. For instance, central firearms registers are being developed, but the backlog of data will take several years to process. In addition, registers of firearms traders are not yet computerised and interconnected with centralised registers. There is a similar lack of harmonisation in national infringement databases, where firearms seized and the context of the seizure (both in the Western Balkans and in the EU) are still not systematically centrally registered. Instead, this information often remains in the local police station or on individual files. Therefore, no direct causal link can be drawn between the activities mentioned in the action plan and the trends in firearms trafficking, which are influenced by a variety of other factors. The analysis therefore relies on the qualitative assessment made by stakeholders themselves, and on perception of the evolution of the trends, with respect to the various needs identified at the time of inception of the action plan. With these limitations, it appears that the action plan provided a global framework for EU-Western Balkan cooperation to address the challenge of firearms trafficking. Without the action plan, there would not have been any such a framework cooperation. The vast majority of stakeholders involved in the consultation recognised the added value of this action plan, in particular because it created a forum for law enforcement agencies to meet, develop connections and improve cooperation. To some extent, EU involvement was instrumental in providing a forum for dialogue and cooperation between law enforcement authorities of the Western Balkans. This in itself can be considered a major achievement in a region where the scars of the wars remain visible and political tensions remain significant. Chart 2.Main areas of added value of the action plan Without the action plan, individual member states would have continued carrying out un coordinated bilateral activities for operational cooperation with Western Balkan partners. Although such activities did take place despite the Action plan, it provided an overall strategic framework and a forum for dialogue and multilateral cooperation. Similarly, individual international organisations, possibly financed by the European Union, would have carried out their own activities. Besides, there would not have been a strong incentive to establish firearms national focal points, to bring national laws closer to EU legislation and standards or participate in EU-provided trainings. On operational cooperation, the organisation of joint coordinated checks operations has been mostly praised by stakeholders for its ability to identify bottlenecks facilitate dialogue, test processes of information sharing. Outside of the action plan, these would have been a one-off process, not linked to the establishment of national structures and the implementation of EU law in the Western Balkans. Most Western Balkans partners recognized Europol’s support for the operational cooperation through the involvement of its operation centre and its 24/7 intelligence reporting, which could not have been provided without Europol’s involvement. Half of Western Balkan partners however criticised the Agency for not providing timely feedback and useful intelligence and analysis, because a very few intelligence cases were related to the Western Balkans (5.4% in 2017, 7.6% in 2018). This could be addressed if the Analysis Project Weapons and Explosives received regular information directly through SIENA both on actual seizures and on follow-up investigations. The contribution of other structures was more limited. In particular, no joint investigation team on firearms trafficking was organised by Eurojust in the Western Balkans. Besides, unlike the joint coordinated targeted checks or individual police operations (to which the EBCGA participated as well), there was no broad appliance of the Eurosur Fusion Services and not many successful stories with major seizures of firearms, due to a lack of border detections. The meetings (both the Joint Committee and other meetings such as of the Western Balkan Experts, the EU Experts and the Firearms Platform) were a great step towards building trust, confidence and greater understanding of the opportunities and obstacles for the closer cooperation between the EU and the Western Balkans. Meeting of the Western Balkan experts helped the establishment of national focal points in the Western Balkans. Since 2018, all Western Balkan countries have been invited to be full participants in the Firearms Platform. For the Firearms Platform2019 operational action plan 5 Western Balkan countries already signed up to participate in 16 out of the 19 operational actions. The approaches to training varied between EU member states and Western Balkan countries. Only 30% of member states, promoted joint training courses compared to 67% of Western Balkan partners. Western Balkan partners therefore seem to have a higher awareness of and need for firearms-related capacity building. On the sustainability of the outcomes achieved under the action plan, it is unclear whether its termination would have a negative impact. The answer to this question very much depends on whether and how the action plan will continue in future (is considered as a static or as a living document). Its mere extension, without taking into account the new realities and needs identified in Section 5.1, would probably not have any discernible effect. The main current challenges and needs have now been properly identified in the regional roadmap; and Western Balkan partners are now fully involved in all relevant activities of the Firearms Platform. These can continue to be rolled-out, as they have started to, outside of the strict implementation of the action plan. Its mere extension would simply enable the continuation of the current situation. However, the action plan has been a dynamic document, in the sense that it provided general objectives and targets, but also enabled the Joint Committee between EU firearms experts and south-east Europe partners experts to build on it and set out specific objectives and activities to implement it. The simple non-renewal of the action plan and the absence of a fully modernised action plan would put to a halt the synergies and coordination that have developed so far. This would also fail to address the main weaknesses of overall governance at regional level, as identified later in this document. If the action plan was not renewed the EU would mainly be considered as a donor, with no contribution to ensuring proper consideration of common challenges or steering of overall strategy. Moreover, if the action plan was not renewed, individual member states and organisations would turn to the EU only for financial support, but would revert back to separate and most likely overlapping initiatives. Key findings ·The Action plan provided an overall framework for cooperation by ensuring a forum for law enforcement agencies to meet, develop connections and improve cooperation. ·Peer review and encouragement to set up National Focal Points and to receive joint training courses. ·Full involvement, on an equal footing, with EU member states in EMPACT Firearms. ·Modest involvement of EU Agencies (Europol, Eurojust and EBCGA) to the activities of the action plan. ·Unsatisfactory exchanges of information between Europol and Western Balkan partners. ·Low added value of other available tools, either underused (Joint Investigation Teams), or with low operational results (Eurosur Fusion services and EBCGA activities). Effectiveness Assessing the progress made towards achieving the objectives of the action plan proved a challenge in the course of this evaluation. This was notably the case for two reasons. First, the action plan did not set key performance indicators against which to evaluate progress towards achieving its objectives. Second, the action plan was developed in a context where several other initiatives (national, regional, European and international) took place in parallel, and where the overall impact on the reduction of firearms trafficking would have been too indirect or hypothetical to draw any conclusion. Nevertheless, it is impossible to draw initial conclusions based on the patchy quantitative information available and on the qualitative contributions made by stakeholders during the evaluation and over the duration of the action plan's implementation. Consulted stakeholders expressed mixed feelings about the effectiveness of the action plan. Activities related to the modernisation of law enforcement agencies (Goal 1) were generally considered more effective than those related to mutual trust and actual cooperation (Goal 2). Such a mixed feeling was also expressed for activities on enhancing training and capacity building (Goal 3).

Chart 3.Effectiveness of Actions under Goal 1 Under Goal 1, harmonised data collection on firearms seizures remains one the main stumbling blocks both in the EU and in the Western Balkans. Only 16 EU member states and four Western Balkan partners stated that they participated in the UNODC's Global Firearms Study and this limited the effectiveness of this study. The reason for this low level of participation is that only one of the four focal points in the Western Balkans is fully operational. Similarly, in the EU, only a few focal points have sufficient access to the information required to provide the data required. A template with minimum standard intelligence requirements was not developed, because different players disagreed over what include on the template. Enforcement agencies often considered that this limited the usefulness of the information exchanged. Finally, while all countries of the region are connected to Interpol's iARMS database, which lists seized, stolen and lost weapons, consulted stakeholders did not confirm whether this database is systematically populated. Cases of seizures and joint operations presented very rarely provided information on tracing. This made it difficult to assess the trafficking routes and modus operandi of traffickers. Taking those limitations into consideration, it was nevertheless possible to compile overall figures of confiscated weapons in the Western Balkans (including in purely internal situations), but the data shows no specific trend and which does not make it possible to draw a conclusion as to the effectiveness of the action plan. Since 2014, circa 25 000 small arms and light weapons were destroyed, 6 000 parts and components and 250 000 rounds of ammunition in Bosnia and Herzegovina, Montenegro, Kosovo* and Serbia only. However, statistics are not sufficiently detailed to assess to what extent these activities derive directly from the action plan. Chart 4.Number of confiscated firearms It was one of the objectives of the action plan to ensure that legislation in the Western Balkans was in concordance with EU law. This concordance is well under way. Each meeting organised was an occasion for benchmarking between various countries, and for the Commission to clarify aspects linked to the legislation. The Clearinghouse also provided continuous support to help countries adapt their national legislative framework (by providing training, technical and legal advice). Of course, the latest changes to EU legislation are yet to be fully reflected in national law, as recognised by most stakeholders during expert meetings. However, more work remains to be done with legislation on criminal sanctions and criminal policy, or with bylaws on deactivation or marking of firearms. This need for more work to be done was mentioned by many stakeholders both in response to the questionnaire and during firearms experts meetings. Respondents often pointed to the lack of a sufficiently stringent criminal policy, with traffickers being often more easily prosecuted for illicit possession than for trafficking. On deactivation and marking standards, although the basic principles of the legislation in the Western Balkan partners are generally in line with EU law, according to experts from the Clearinghouse and national firearms experts during meetings, not all countries of the region have adopted administrative rules to fully implement of those principles. This is partly due to the fact that some changes are very recent in EU law (deactivation standards of December 2015 were amended in March 2018 and marking requirements were harmonised in January 2019 ). Chart 5.Effectiveness of Actions under Goal 2 Under Goal 2, stakeholders were generally satisfied with the effectiveness of joint meetings. According to the assessment of the Commission, which chaired those meetings, and confirmed by stakeholders during the consultation exercise, the effectiveness of these meetings was sometimes hampered by the fact that participants were mid-ranking investigators who required authorisations and instructions to address international meetings. Moreover, they were not always aware of broader developments within their respective institutions. Overall, however, stakeholders appeared satisfied with their usefulness of these meetings, especially in terms of enabling dialogue between firearms experts. Chart 6.Effectiveness joints EU-WB meetings Daily cooperation and exchanges of information were made difficult by a high turnover of staff in the national law-enforcement agencies of Western Balkans partners. Nevertheless, available data demonstrated the ability of participants to learn from the difficulties encountered during joint coordinated targeted checks. This is illustrated by the number of SIENA messages exchanged within the joint checks operations, which substantially increased (before, during and after the action). The number of seized firearms (during the joint action days and in the follow-up investigations) has also been steadily increasing. Both figures demonstrate the effectiveness of this activity in improving the exchange of criminal information and intelligence and operational law enforcement co-operation. Chart 7. Chart 8. On the ground, border checks across the region have demonstrated their own inadequacy. Although there is evidence that firearms travel across the Western Balkans, from and to the EU, as recognised by all stakeholders and intelligence assessments, seizures of firearms rarely take place at borders, but mostly inland, not in relation to international trafficking. This is confirmed by law enforcement agencies across the region, as well as the EBCGA, who rarely ever report border seizures. According to several stakeholders, this is because border guards and customs officers (i) are not fully aware of problem of firearms trafficking and the ways in which firearms are trafficked, (ii) do not sufficiently rely on risk profiling and prior intelligence, (iii) are not specifically trained to detect firearms, (iv) are not adequately equipped (they do not use mobile scanners, and have no firearms-trained sniffer dogs), (v) and focus on border crossing points with no clear strategy to control the other land borders used by smugglers. The operational cooperation between the EU and the Western Balkans revealed several weaknesses in criminal procedures in both areas. One of the weakness is that special investigative techniques (phone tapping, under-cover operations) differ widely across countries and this can create difficulties when setting up cross-border operations. Such special investigative techniques can sometimes only be used in particularly serious cases of organised crime, of which weapons trafficking is not always considered to belong. According to discussions held during firearms expert meetings, checks on suspected deliveries of weapons (both in the Western Balkans and in the EU) are limited by a general reluctance to authorise those operations in the transit countries. This is sometimes because (i) the national legal framework makes it compulsory to seize any illicit weapon on the national territory (e.g. Austria); (ii) prosecutors do not trust other countries to carry out the investigations (a recurring complaint both from law enforcement and several prosecutors attending firearms expert meetings); or (iii) such operations are time and staff-consuming, with no direct impact of the trafficking for the country of transit. This has affected the effectiveness of the cross-border cooperation and made it harder to dismantle international firearms trafficking rings. Stakeholders, during meetings and in the replies to the consultation, unanimously regretted the lack of prosecutors aware of the challenges of firearms trafficking. They also point out constraints put on their work by legislation, which hampers exchanges of information for intelligence purposes outside of specific investigations. Similarly, during the joint checks operations, Preparation and debriefing of Joint Action Days (i.e. joint control operations) under the Firearms Platform showed that member states with strong connections to the Western Balkans did not actively help law enforcement agencies to detect traders were trafficking gas and alarm weapons, which were not considered as firearms in their national legislation. As another illustration of the limits in operational cooperation, within the Firearms Platform, Operation Bosphorus was dedicated to the threat of gas and alarm weapons but did not include customs checks or the participation of border guards. This limited the effectiveness of the operation. On exchanges of information through SIENA, several stakeholders complained that the system was not sufficiently user-friendly to replace traditional emails. Significant challenges remain in forensics (notably ballistics). Technologies vary widely across the region and this impedes exchanges of information and cross-comparison beyond national borders. According to the works carried out in this field in the Firearms Platform and in the discussions during expert meetings, in most EU member states, ballistic analysis and international comparisons are carried out on a case-by-case basis, and are neither systematic nor fully automatised. In the Western Balkans, only one partner (Kosovo*) performs a gun crime analysis report and has forensic laboratory which achieved the ISO 17025 accreditation for its ballistic investigation and other related processes. All partners face a large backlog of bullets and casings, which have never been checked for ballistic information even though they could enable connections between crimes across Europe and movements of illicit firearms. Under Goal 3, the greatest challenge was assessing the effectiveness of activities to improve training and capacity-building. This is because the effectiveness of these activities cannot be evaluated simply by counting the numbers of trainees or workshops that were held. However, the participation of Western Balkan partners in trainings provided by CEPOL substantially increased from 2015 and 2018. Figures show uneven participation in training by EU member states, with no specific identifiable trend, the rate of participation is stable and low. Two countries stand out for their participation: Greece and Portugal. Both these countries had a substantially above-average number of trainees attending webinars. On average, Greece had 66 webinar trainees per year and Portugal had 15.

The Clearinghouse provided useful practical training for law enforcement officers across the region , including by developing for all agencies of the region a weapon identification platform to help identify seized weapons, ammunition and explosives, and generate reports for further criminal investigation. The Clearinghouse (as well as individual member states and international organisation such as UNODC, but not in the context of this action plan) also provided capacity-building support to raise firearms-control standards of Western Balkan countries. According to information provided by the Clearinghouse and law enforcement representatives, all Western Balkan partners now have in place functioning registers of legally owned firearms, databases on external trade of weapons and databases on denials to grant authorisations. However, the lack of sufficiently practical training was raised by several stakeholders, who identified needs in the establishment of Joint investigation teams and on specific issues such as convertible firearms, as well as specific awareness-raising activities for customs and border guards. The lack of implementation of the United Nations Firearms Protocol, for instance with respect to marking or the definition of firearms trafficking, calls for trainings on criminal codes. Key findings · Activities carried-out under the action plan for modernization of law enforcement agencies compared to other actions. ·Approximation of legislation in the Western Balkans towards EU standards is well on track. · The organization of joint meetings for creating a unique space for dialogue and exchange of best practices between firearms experts. ·Steady improvement in the exchange of information and operational cooperation during joint coordinated targeted controls. ·Positive responsiveness of Western Balkan countries to training and capacity-building opportunities. ·Lack of key performance indicators that would have enabled a comprehensive assessment of progress towards achieving the objectives. ·National Focal Points still not established in all countries, and as a consequence: ostill no harmonized data collection firearms seizures. oiArms database not systematically populated ono systematic tracing of weapons seized ono systematic or automatized ballistics analysis and cross-border comparisons ·No noticeable reduction of stockpiles. ·Inefficiency of border checks, which do not yield results in comparison with inland checks. ·Insufficiently stringent criminal sanction and criminal policies. ·Difficult use of criminal proceedings during cross-border operations. Efficiency Conducting a cost-benefits analysis of the action plan was limited by the fact that figures and hard data are almost non-existent, especially with respect to financial costs. Similarly, neither the Firearms Platform nor Europol have provided a financial analysis of their operations. As far as EU financial programmes are concerned, notably the Instrument for Pre-Accession Assistance (IPA II), supports actions against firearms trafficking. Under the current budget structure, it is therefore not possible to identify exactly how much funds are used specifically for the fight against firearms trafficking. In terms of outcomes (such as firearms seizures, number of voluntary surrenders of weapons, size of stockpiles etc.), figures are still insufficiently harmonised or nor systematically available to allow a quantitative analysis. Besides, where specific figures could be identified, as explained in section 5.3 on effectiveness, the link between the activities mentioned in the action plan and the outcome would be too indirect to enable a reliable cost-benefit analysis. This means that it is not possible to draw a direct causal link between a headline and the addition of individual seizures, operations and investigations against firearms trafficking. The Firearms Platform can in principle provide substantive support to EU member states and Western Balkan partners for the costs of participation in action days (even bilateral ones), information rewards, funding of special equipment, rental of premises for covert operations etc. However, a dedicated meeting for local EU liaison officers in Belgrade on 19 March 2018 demonstrated a general lack of knowledge about this facility and about the Firearms Platform in general and a widespread reliance on national funding and purely bilateral cooperation. The country in charge of coordinating the Firearms Platform is also aware of many ongoing joint investigations that could benefit from the Firearms Platform support and do not rely on it. Information linked to the Firearms Platform is not properly transmitted to those who might need it. The criticism of several Western Balkan partners against the added value of exchanges of information with Europol (see section 5.2), is unsurprisingly reflected the overall decrease of secure SIENA messages which Western Balkan partners exchange with Europol and EU member states (outside of joint control operations, where an increasing trend can be observed). Their SIENA contributions amounted in 2018 to 29% of all contributions, down from 67% in 2017 notably due to a strong increase of direct contributions from EU member states (from 67 to 111). The same trend can be observed for cases contributed to Europol: while the number increased between 2017 and 2018 for EU member states (26 to 29), it decreased for Western Balkan Partners (4 to 2). Discussions during meetings of firearms experts have shown that the reason for this decrease was a lack of added value of such exchanges through Europol, as identified in section 5.2. On ballistic analysis, respondents stated that ballistics analysis was not efficiently carried out. Respondents says that it was due to a lack of harmonised standard procedures and lack of detail about the type of information needed by investigators. These problems were identified in several meetings of firearms experts, in particular meetings of the Western Balkan Experts that are regularly attended by ballistics analysts. The Western Balkan Experts raised the issue of a lack of dialogue between forensics analysts (of which ballistics is part) and investigators: the former do not know what type of information to provide to investigators, and the latter rarely provide feedback about how the information is used. Another problem is that techniques for ballistics analysis vary from country to country making difficult to exchange ballistics information across borders and conduct transnational investigations. On trainings, CEPOL decided to cover the expenses of participants from Western Balkan partners in its activities (6 in 2016 and 2017 and 4 in 2018), and they sent twice as many participants than they received funding for. This indicates a greater commitment and interest from Western Balkan partners than EU member states in transnational training opportunities, and hence a high value for money. The Clearinghouse also provided important practical training for law enforcement officers across the region, including by developing for all agencies a weapon identification platform to help identify seized weapons, ammunition and explosives, and generate reports for further criminal investigations. However, the efficiency of trainings is difficult to assess, since no analysis of cascading trainings (i.e. further trainings at national level provided by participants in regional training courses) was provided by the organisers. Most of the consulted trainees did not express any opinion about trainings they received (either on efficiency or in general). In two countries (Albania and Bosnia and Herzegovina), the efficiency of voluntary surrenders campaigns was limited by the need to adopt new legislation for each new programme of surrenders and legalisation of weapons, but an overall assessment was impossible by the unavailability of data. Key findings ·Useful support of EMPACT firearms to operational cooperation. ·High interest and participation rate of Western Balkan partners to trainings provided by CEPOL and SEESAC. ·Not enough data to assess the cost of activities. No detailed budgeting or financial reporting. ·No harmonized data on concrete outcomes. ·Lack of awareness of the financial support provided by EMPACT or by other EU tools. ·Exchanges of information between Europol and Western countries not providing satisfactory results to all parties concerned. ·Cross-border exchanges of ballistic information is not considered worth the effort because of complex legal procedures and the fact that ballistics information systems are not interoperable. ·Not sufficiently practical trainings and no information on cascading trainings. ·No data to assess the efficiency of voluntary surrender campaigns. Consistency Under this heading, the evaluation first focused on internal consistency, i.e. how the various components of the action plan operate together to achieve its objectives. The evaluation also sought to assess the action plan’s consistency with other interventions that have similar objectives, its relations with international law and strategies, with wider EU policy. On internal consistency, the action plan did not establish a direct link between the objectives pursued and the goals and actions. The objectives, goals and actions often intertwined and overlapping, thus increasing the complexity of the assessment. For example, the establishment of national firearms focal points envisaged in Action 1 of Goal 1, in practice was carried out by capacity building and training activities, which falls under Goal 3. Similarly, the work on exchange of information was directly linked to the actual conditions of operational cooperation, and the newly created firearms focal points were meant to enhance operational cooperation. Some of the needs were reflected in several actions (exchange of criminal information and intelligence, improving cooperation in operational law enforcement; and networking and joint training), while the collection and exchange of statistics was mentioned in one action only. Although ministers mentioned expressly "harmonising national legislation on firearms with EU and international standards" as one of the six measures they identified and committed themselves to implement, this was not specifically translated into one of the Goals and Actions of the action plan. It was rather referred to at the end of the action plan as "continue helping in bringing legislation up to EU standards and implementing it", among the many activities not specifically related to one of the three main goals. On the external consistency, the connection between the action plan and EU legislation related to the control of weapons was analysed. The perspective of European Union membership has acted as a major incentive for partners of the region to implement this part of the action plan and bring their national legislation in line with EU standards. On other aspects of external consistency, the action plan itself already mentioned that cooperation would be based on: ·respect for international law, human rights and fundamental freedoms; ·the guidelines set in the EU firearms strategy and the EU firearms operational action plans; ·the existing stabilisation and association agreements between the EU and the south-east Europe region;

·an integrated and balanced approach based on partnership; ·EU support to the south-east Europe partners in meeting the commitments they have made to combat firearms trafficking; ·Council Decision of 15 February 2007 to combat in an effective way organised forms of criminality through Europol with Albania, Bosnia and Herzegovina, Croatia, former Yugoslavia Republic of Macedonia, Serbia and Montenegro and the establishment of a Firearms Focal Point in Europol for facilitating the exchange of information on firearms." Those commitments were reaffirmed through several initiatives, mentioned in Section 2.5, such as the Joint Communication “Elements towards an EU strategy against illicit Firearms, Small Arms & Light Weapons and their Ammunition”. The Communication also refers to a number of international agreements, which have been duly taken into account by all players when implementing the action plan. Some of international agreements that have been taken into account include the UN Programme of Action on Small Arms and Light Weapons, the International Tracing Instrument, the Arms Trade Treaty, and the Protocol against the illicit manufacturing of and trafficking in firearms, their parts and components and ammunition, supplementing the UN Convention against transnational organised crime. The action plan did not provide for any set structure to implement governance. Therefore, the European Commission took it upon itself to steer the implementation of the action plan and support a consistent coordination between the players involved in this implementation. The European Union has therefore worked to ensure that the actions of all players are complementarity. In particular, it has worked to integrate the activities of the Firearms Platform and the Clearinghouse into the Integrative Internal Security Governance (IISG) in the Western Balkans and its three pillars on counter-terrorism, fight against serious and organised crime and border security. The Clearinghouse coordinates a small arms control activities in the region and all activities aimed at combatting illicit trafficking of firearms in the region. This work by the Clearinghouse was mandated by the Regional Implementation Plan on Combatting Proliferation of Small Arms and Light Weapons in South East Europe. The Clearinghouse is now in charge of implementing the regional roadmap for a sustainable solution to the illegal possession, misuse and trafficking of small arms and light weapons. On the roadmap, Section 4.1 on methodology explains how the Commission used the Roadmap as a valuable source of information for the needs assessment for the region. During the drafting of the roadmap, the Commission ensured that the roadmap would be fully consistent with the content and objectives of the ongoing action plan, even if its structure might differ in parts. The fact that the Clearinghouse has been in charge of coordinating the implementation of the roadmap ensures a strong convergence between the various work streams. Firearms trafficking was also included in the joint action plan on counter terrorism for the Western Balkans of 5 October 2018. In practice, while the Commission has attempted to ensure the comprehensiveness of the activities and provide general oversight over them, the Firearms Platform has focussed on operational cooperation and the Clearinghouse on providing capacity training and development of national strategies. Nevertheless, the multiplicity of bodies of cooperation can be considered a downside of the overall management of the EU’s fight against illicit firearms trafficking. Various groups and bodies often overlap. International players like the Small Arms Survey, the Southeast European Law Enforcement Centre (SELEC), North Atlantic Treaty Organisation, the United Nations Office on Drugs and Crime or the Organisation for Security and Co-operation in Europe are also active in the region. However, there is unclear coordination of these groups’ activities with the Clearinghouse, the Firearms Platform, or within the Integrative Internal Security Governance. These international players were not directly involved in the implementation of the action plan. As an illustration of the overlapping nature of the work done by different organisations, activities of operational cooperation (joint investigations, exchange of information) in the region can be supported by the Firearms Platform, Europol, Eurojust and the Southeast European Law Enforcement Centre or within the Multi-Beneficiary Regional IPA 2017 "Countering Serious Crime in the Western Balkans". Trainings for border detection are provided by the Clearinghouse, the EBCGA and UNODC. Similarly, training and legal assistance provided by the EU Partner to Partner Export Control Programme for Arms managed by the German Federal Office for Economic Affairs and Export Control BAFA took place outside the coordination framework of the Clearinghouse / the Firearms Platform /Commission. EU funding can be sought – and obtained – through various channels (Common Foreign and Security Policy funding, the Instrument for Pre-accession Assistance including the Technical Assistance and Information Exchange instrument (TAIEX) modality, the Internal Security Fund) without a central coordinating authority. This initially created inconsistencies and lack of coordination between several Commission departments, which have been addressed and overcome through systematic exchange of information between them, sharing of reports, joint participation to external meetings, regular internal meetings, and joint and coordinated drafting of notes and briefings. However, the Commission is only informed and consulted on funding received through Council Decisions in the Common Foreign and Security Policy framework. This means that funding (for instance for the Clearinghouse or the EU "Partner to Partner" Export Control Programme for Arms) can be granted without a clear overview of other similar or overlapping activities in the region. The lack of dialogue and coordination has led to operations taking place at the same time. Many stakeholders have noted an unnecessary duplication of meetings, with similar participants having the same discussions. Conversely, in other cases, representatives attend different meetings with no prior knowledge of similar discussions or presentations having already taken place in similar formats. This is due to a lack of internal coordination and information sharing. Key findings ·strong consistency with other overarching strategies, including EU strategy on small arms and light weapons and the Regional Roadmap; ·close coordination ensured with legislative developments inside the EU on firearms control to ensure approximation of Western Balkan laws with EU standards; ·consistency ensured with international instruments. ·weak internal coherence between activities; many overlaps; ·lack of integ ·rated steering structure; ·lack of integrated budgetary approach. Conclusions The European Union and Western Balkan partners are facing common challenges. Changing realities mean that there is still a need for an action plan but the current action plan has lost some of its relevance as it is not longer fully adapted to developing needs. According to the consultation of stakeholders and drawing lessons from the development of the regional Roadmap, what appears to be lacking is a set of operational objectives, detailed performance indicators and a clear steering structure. On geographical scope, the activities have correctly been focusing on the Western Balkan, but future challenges may justify covering all of south-east Europe, including the Republic of Moldova and Ukraine, to develop future-proof solutions in the EU's Eastern neighbourhood. The action plan mostly provided added value by improving networking and the exchange of information and by putting in place the building blocks of future cooperation. It encouraged multilateral cooperation and a coordinated strategic approach which purely bilateral relations and individual initiatives would not have enabled. Other tools have not been made use of, such as joint investigation teams. The effectiveness of the action plan has been uneven, depending on its various objectives. Data collection of firearms seizures is not sufficiently widespread or harmonised. This made difficult to conduct a proper and reliable assessment of trafficking trends, and produce evidence-based policies. The setting up of national focal points (and in the Western Balkans the setting up of small arms commissions) is well under way, but not yet completed. This still prevents proper enforcement strategies, coordination, intelligence and exchanges of information, both nationally and at European level. Efficiency is also hampered. Taking into account the lack of available data that would have enabled a proper assessment, it is nevertheless possible to note that efficiency has been limited. Under the current budget structure, it is therefore not possible to identify exactly how much funds are used specifically for the fight against firearms trafficking. In terms of outcomes (such as firearms seizures, number of voluntary surrenders of weapons, size of stockpiles etc.), figures are still insufficiently harmonised or nor systematically available to allow a quantitative analysis. Besides, where specific figures could be identified, as explained in Section 5.3 on effectiveness, the link between the activities mentioned in the action plan and the outcome would be too indirect to enable a reliable cost-benefit analysis. This means that it is not possible to draw a direct causal link between a headline and the addition of individual seizures, operations and investigations against firearms trafficking. Consistency and complementarity with other EU action has consistently improved, especially in the last years. However, several bilateral or intergovernmental initiatives created a challenge to avoid overlaps and conflicting activities. Closer co-ordination between the Commission, the Firearms Platform and the Clearinghouse is needed to ensure a better use and allocation of funds.





Annexes Annex 1.Procedural information The evaluation was led by the Directorate General for Migration and Home Affairs (DG HOME), in close association with the Directorate General European for Neighbourhood Policy and Enlargement Negotiations (DG NEAR) and the European External Action Service (EEAS). The evaluation process started during the Joint Committee between firearms experts from EU and south-east Europe partners Experts of 26 January 2018. The Joint Committee agreed that: "In order to assess the different initiatives carried out between EU member states and Western Balkans partners as defined in the EU/WB action plan 2015-2019, an in-depth evaluation will be carried out in 2018. The Commission will circulate a questionnaire by the end of February 2018 to all EU and WB competent administrations to collect information about the initiatives carried out by the partners according to the action plan, the Firearms Platform activities and the conclusions of all relevant meetings. The relevant partners will send the replies to the Commission by the end of June 2018. The Commission will draft a report to inspire the draft new action plan to be adopted in 2019." DG HOME consulted DG NEAR on the draft questionnaire on 22 June 2018. The questionnaire was then sent to members of the European Firearms Experts (EFE) Group, of the South East Europe Firearms Expert Group (SEEFEG), to European agencies and other stakeholders (such as the Clearinghouse) on 27 July 2018. An Inter-service Group composed of DG HOME, DG NEAR, DG DEVCO, the FPI, DG TRADE, DG TAXUD, the Commission’s legal service and the Secretariat General, was set up and met on 23 January 2019., The inter-service consultation on this staff working document and on the Commission report were then launched. No specific public consultation took place, since the Action plan was addressed to public authorities only, and all relevant information on the actions implemented under the action plan (trainings, coordination meetings, exchange of information between law enforcement agencies, investigations etc.) was available only to the specific stakeholders involved in the implementation of the action plan. The public does not have access to this action plan so a public consultation would not have provided any additional data for the evaluation of the action plan compared to the targeted consultation carried-out. An evaluation roadmap was published on the Commission's Better Regulation Portal on 29 January 2019, and was open for feedback from the public until 27 February 2019. One stakeholder from Mexico provided feedback, stressing the need to improve border controls and fight against corruption. During its meeting of 24 September 2018, the Joint Committee between firearms experts of the EU and of the south-east Europe partners discussed the outcome of this consultation. The meeting focused on the evaluation of the action plan and experts identified measures to help improving cooperation in the fight against firearms trafficking. In parallel, the preparation of the regional roadmap for a sustainable solution to the illegal possession, misuse and trafficking of Small Arms and Light Weapons was carried out between 1st February 2018 and 29 May 2018 at the initiative of the Ministries of Foreign Affairs of France and Germany, and coordinated by the Clearinghouse. Other organisations were also closely associated with the preparation of the analysis, including the Commission, the Firearms Platform, the EBCGA and Europol. The roadmap focussed on developing a needs assessment for the region, and therefore contributed greatly to the analysis of current challenges. The roadmap also complemented the information gathered through the questionnaire. DG HOME, DG NEAR and the External Action Service were in close and permanent contact to coordinate the position of the European Union in various external meetings. This conduct included (i) inviting all relevant Commission DGs and the External Action Service to the meetings of the Joint Committee; (ii) the work on the regional roadmap leading to the London Ministerial Summit of 10 July 2018; (iii), the drafting of the Joint Communication on Elements towards an EU Strategy against illicit Firearms, Small Arms & Light Weapons and their Ammunition; (iv) the continued financial support to the Clearinghouse and (v) the preparation for the high-level meeting of the French-German Co-ordination Initiative on Small Arms and Light Weapons in the Western Balkans of 11 December 2018, to which both DG HOME and DG NEAR took the floor.



Annex 2.Stakeholder consultation General overview The consultation strategy relied on three main sources:

·A stakeholder questionnaire; ·Meetings of the Joint Committee between firearms experts from the EU and south-east Europe partners; ·Meetings and documents for the elaboration of the regional roadmap for a sustainable solution to the illegal possession, misuse and trafficking of Small Arms and Light Weapons. In addition, an evaluation roadmap was published on the Commission's better regulation website on 29 January 2019, and was open for feedback from the public until 27 February 2019. One stakeholder from Mexico provided feedback, stressing the need to improve border checks and fight corruption. Participation: The questionnaire was sent to members of the European Firearms Experts (EFE) Group, of the South East Europe Firearms Expert Group (SEEFEG), to European Agencies and other stakeholders (such as the Clearinghouse) on 27 July 2018. The rate of response to the questionnaire was as follows: Replies Received EU/EEA 18 60% Western Balkans 6 100% Total 24 71% Method of interpretation of the questionnaire: Respondents to the stakeholder consultation often left blanks to certain questions, thus increasing the proportion of "don't know/no opinion". However, were information related to these member states was available through other channels (such as whether a National Focal Point had been established), this was nevertheless reflected in the analysis, in order to ensure accurate figures. Most replies went beyond the strict assessment of the EU-Western Balkan action plan, and cover the general EU activities against firearms trafficking (including the Firearms Platform). The quality and length of responses was uneven. It was often necessary to interpret replies or infer answers to ensure comparability between answers. Besides, most questions were drafted in qualitative terms, which meant that not all of them could be translated into a quantitative assessment. The qualitative assessment is reflected in the body of the Staff Working Document, under each evaluation question. Some replies did not enable a proper assessment, since they did not respond to the question. For instance, some replies to questions such as "how do you assess the efficiency of the activities under this goal" took the form of the provision of factual data (organisation, figures), but no judgement on the efficiency (high, low, negative etc.). Others replied to several questions in a single answer of a few lines, or simply ignored the question, which made the analysis difficult. The analysis of replies showed in several instances different approaches and point of views between the EU and Western Balkan Partners, which explains why the two groups are often presented separately. Results of the questionnaire The questionnaire only included open questions to provide as much freedom to respondents as possible and spur the reflection. The questionnaire included two parts, on the assessment of the action plan and on suggestions for the future. Although the second part was forward-looking, it facilitated the assessment of current needs and gaps. The first part included questions on each main goal of the Action plan, both with requests for factual information and request for a qualitative assessment. The second part included questions on the relevance and added value of the action plan, as well as requests for suggestions for specific aspects (multilateral cooperation, operational cooperation, exchange of information, training and exchanges of best practices). Summary of replies Strategic Goal 1: To modernise the law enforcement agencies and their operating methods by improving the strategic and operational analysis of information, with particular regard to risk analysis, investigative procedures, data collection, exchange, dissemination and use of information. a)Have you set up in your competent administration a National Firearms Focal Point to coordinate the fight against illicit firearms trafficking in your country and liaise with other international partners?

b)Have you participated or are you participating in United Nations on Drugs and Crime (UNODC) data collection project on the seizures of firearms (project financed by the European Commission)? EU/EEA % WB % Participation to UNODC Data collection exercise 16 53% 4 67% c)How do you assess the overall effectiveness and the impact of the activities under this goal? In order to accomplish Strategic Goal 2: "To enhance mutual trust among law enforcement agencies of the South East Europe region and the authorities in the other EU member states. […] To promote inter-regional cooperation between law enforcement agencies, including the exchange of information and operational cooperation." a)Have you regularly attended EU/Western Balkan meetings on the fight against firearms trafficking (Joint meetings of European Firearms Experts (EFE) and of South East Europea Firearms Experts (SEEFEN), EMPACT meetings) ? EU/EEA % WB % Participation to Joint EFE/SEEFEN meetings 18 60% 6 100% Participation to EMPACT meetings 17 57% 6 100% b)How do you assess the relevance and the outcomes of these meetings? c)How do you assess the participation of EU member states and Western Balkan partners in the work of EU Agencies (Europol) or their involvement with the EFE? What would you suggest to improve it? d)How do you assess the overall effectiveness and the impact of the activities under this goal? In order to accomplish Strategic Goal 3: "To enhance capacity building for the prevention of illicit trafficking in firearms by raising awareness, cooperation and sharing of b