EUROPEAN COMMISSION Brussels, 14.5.2019 SWD(2019) 174 final COMMISSION STAFF WORKING DOCUMENT Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 {COM(2019) 228 final}



TABLE OF CONTENTS GLOSSARY I. INTRODUCTION A. PURPOSE OF THE EVALUATION B. SCOPE OF THE EVALUATION II. BACKGROUND TO THE INITIATIVE A. DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES B. BASELINE AND INTERVENTION LOGIC III. IMPLEMENTATION / STATE OF PLAY IV. EVALUATION QUESTIONS V. METHOD A. SHORT DESCRIPTION OF METHODOLOGY B. LIMITATIONS AND ROBUSTNESS OF FINDINGS VI. ANSWERS TO THE EVALUATION QUESTIONS A. RELEVANCE B. EFFECTIVENESS C. EFFICIENCY D. COHERENCE

E. EU ADDED VALUE VII. CONCLUSIONS AND LESSONS LEARNED VIII. ANNEXES ANNEX I – PROCEDURAL INFORMATION ANNEX II – METHODS, SOURCES AND LIMITATIONS ANNEX III – STAKEHOLDER CONSULTATION ANNEX IV – INTERVENTION LOGIC ANNEX V – SUMMARY OF THE MAIN AREAS AND GOALS IN THE 2013-2015 AND 2016-2018 STRATEGIES AND WORK PROGRAMMES



GLOSSARY Abbreviation Definition AWP Annual Work Programme CEPOL European Union Agency for Law Enforcement Training DG GROW Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs DG HOME Directorate-General for Migration and Home Affairs DG SANTE Directorate-General for Health and Food Safety DG TAXUD Directorate-General for Taxation and Customs Union EASO European Asylum Support Office EBCGA European Border and Coast Guard Agency (also known as Frontex) EC European Commission ECDC European Centre for Disease Prevention and Control EMA European Medicines Agency EMCDDA European Monitoring Centre for Drugs and Drug Addiction EWS EU Early Warning System on new psychoactive substances ESPAD European School Survey Project on Alcohol and Other Drugs EU-OSHA European Agency for Safety and Health at Work Eurojust European Union Agency for judicial cooperation in criminal matters Europol European Union Agency for Law Enforcement Cooperation FRA Fundamental Rights Agency HDG Horizontal Working Party on Drugs HIV Human immunodeficiency virus JHA Justice and Home Affairs JRC Joint Research Centre MAOC(N) Maritime Analysis and Operation Centre – Narcotics MFF Multiannual Financial Framework NPS New psychoactive substances Reitox European Information Network on Drugs and Drug Addiction Network SWP (3-year) Strategy and Work Programmes UNODC United Nations Office on Drugs and Crime WHO World Health Organisation



I. INTRODUCTION A. PURPOSE OF THE EVALUATION

The European Monitoring Centre for Drugs and Drugs Addiction (EMCDDA; referred to in this document as “the Agency”) was established in 1993 with the mission to provide the EU and its Member States with factual, objective, reliable and comparable information at European level on drugs and drug addiction and their consequences. Its core tasks are to collect and analyse existing data, improve data comparison methods, disseminate data and to cooperate with European and international bodies and organisations as well as with third countries. Regulation (EC) 1920/2006 (referred to as “founding Regulation”), together with the Financial Framework Regulation, the Common Approach on Decentralised Agencies and the principles of sound and efficient management require that the Agency be evaluated on a regular basis. Article 23 of the founding Regulation specifically stipulates that the Commission should initiate an external evaluation every six years to coincide with the completion of two of the Agency’s three-year work programmes. The evaluation has to include the Reitox system (the European Information Network on Drugs and Drug Addiction). The evaluation report has to be transmitted to the European Parliament, the Council and the Agency’s Management Board. The previous external evaluation covered the period 2007-2012 and was completed in mid-2012 so that its findings would be available ahead of the following multiannual work programme of the Agency. The current six-year programming phase covers the last two multiannual work programmes, namely 2013-2015 and 2016-2018, and the findings of the evaluation will feed the Agency's future multiannual and annual work programmes and will contribute to the shaping of the future EU Drugs Strategy (the current one ends in 2020). The evaluation has two main objectives: -to assess the relevance, effectiveness, efficiency, coherence and EU added value of the Agency's performance since 2013 in terms of the implementation of the EU Drugs Strategy (2013-2020) and its Action Plans, of the European Agenda on Security and of its tasks laid down in its founding Regulation (Article 2); and -to propose concrete and useful recommendations to the Agency to better respond to the challenges posed by the constantly changing environment against the background of the current authorised financial and human resources for the Agency, bearing in mind the current and future EU budgetary constraints. The evaluation assesses also the compliance of the Agency with the objectives and provisions of the Common Approach on Decentralised Agencies. B. SCOPE OF THE EVALUATION As foreseen in Article 23 of the founding Regulation and based on the Terms of Reference for the external evaluation, the contractors assessed in the evaluation study: -the implementation of the last two three-year work programmes, more precisely from January 2013 until as late as possible in 2018; in this context, the evaluation examined in particular the way and extent to which the Agency effectively contributed to the implementation of the strategic documents referred to above and fulfilled its tasks laid down in its founding Regulation; -the European Information Network on Drugs and Drug Addiction (Reitox), whose members are national institutions or agencies responsible for data collection and reporting on drugs and drug addiction, in particular to what extent the network contributed to the overall performance of the Agency; and -whether the provisions and the scope of the founding Regulation are still adapted to current needs given the evolved context in which the Agency now operates and the new challenges it has to face. The evaluation also had to verify to what extent the provisions establishing the Agency are aligned with the Common Approach on Decentralised Agencies, looking in particular into the Agency's governance, internal structures and procedures and applicable administrative and financial rules. Another intention of the evaluation was to provide solid grounds for any possible decision concerning the Agency’s future mandate. Based on these principles, the external evaluation study assessed the five evaluation criteria (relevance, effectiveness, efficiency, coherence and EU added value) by taking into account the following scope: -Material scope: the evaluation covers the two pillars of the Agency’s work, i.e. health and security. It looks into its governance and administration, the organisational structure, operations, funding and resourcing, its information management and the work of the Reitox network, cooperation with other relevant EU agencies (such as the Justice and Home Affairs agencies, the European Medicines Agency (EMA), the European Centre for Disease Prevention and Control (ECDC)) and international organisations (such as the World Health Organisation (WHO), the United Nations Office on Drugs and Crime (UNODC), etc.), partnerships with third countries, and communication and dissemination of research; -Geographical scope: the evaluation covers the countries reporting data to the Agency, i.e. the EU Member States as well as Norway and Turkey, and third countries with which the Agency has closer relations due to the conclusion of working arrangements, cooperation agreements or similar; -Temporal scope: the evaluation covers the activities carried out from 1 January 2013 until 30 June 2018 to coincide with the completion of two consecutive 3-year strategies and work programmes of the Agency. The evaluation was the fourth evaluation of the Agency. The current evaluation built on the results of the previous external evaluation.

II. BACKGROUND TO THE INITIATIVE A. DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES The market for illicit drugs is the most dynamic criminal market in the EU, which is able to adapt rapidly in response to drug control measures. Its socio-economic, demographic and international context has evolved considerably over the last years. The human and social costs of drug addiction are very high and it generates costs for public health (drug prevention, healthcare and treatment), public safety, the environment and labour productivity. The drugs market value is estimated at EUR 24 billion and is facilitated by the emergence of new internet technologies. Online marketplaces have emerged and are spreading. The number of New Psychoactive Substances is on the rise. An estimate of at least 7929 overdose deaths happened in 2016. There has been a shift from drug use to poly-drug use (where people use two or more psychoactive drugs or alcohol in combination) and on-going debates and shifting policies across the world regarding medical and/or recreational use of cannabis. The issue of misuse of prescription drugs is becoming more and more important also in Europe. Purity and availability on the market of conventional drugs (such as cocaine and heroin) are increasing. In addition, national drug policies developed further over time and many Member States' new strategies go beyond only covering illicit drugs to cover also other addictions, such as alcohol, tobacco, gambling, etc. In order to be able to understand better the nature of Europe’s drug problem and to respond better to them, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) was established in 1993 as a decentralised agency to provide independent, science-based information on these issues. Its objective, pursuant to Article 1 of the founding Regulation, is "to provide in the areas referred to in Article 3, the Community and its Member States with factual, objective, reliable and comparable information at European level concerning drugs and drug addiction and their consequences". Annex I, to which Article 3 of the founding Regulation refers, reads as follows: "A. The work of the Centre shall be carried out with due regard to the respective powers of the Community and its Member States in the area of drugs, as those powers are defined by the Treaty. It shall cover the various facets of the drugs and drug addiction phenomenon, and the solutions applied. In doing so, the Centre shall be guided by the Drugs Strategies and Action Plans adopted by the European Union. The Centre shall focus on the following priority areas: 1) monitoring the state of the drugs problem, in particular using epidemiological or other indicators, and monitoring emerging trends, in particular those involving poly-drug use; 2) monitoring the solutions applied to drug-related problems; providing information on best practices in the Member States and facilitating the exchange of such practices among them; 3) assessing the risks of new psychoactive substances and maintaining a rapid information system with regard to their use and also regarding new methods of using existing psychoactive substances; 4) developing tools and instruments to help Member States to monitor and evaluate their national policies and the Commission to monitor and evaluate European Union policies. B. The Commission shall make available to the Centre, for dissemination, the information and statistical data which it possesses pursuant to its powers." Based on the objectives, tasks and priority areas of work set out in the founding Regulation, the multi-annual strategies and work programmes detail the main areas, goals and expected results of the work of the Agency further. For the period relevant for this evaluation, these are set out in particular in the “EMCDDA 2013-15 strategy”, the “Strategy and work programme 2016-18” and the “EMCDDA Strategy 2025”, with further details included in the annual work programmes. These documents formed and form the basis of the Agency’s operating framework over the evaluation period. B. BASELINE AND INTERVENTION LOGIC When the Agency was established, no detailed intervention logic was prepared. It was developed for this evaluation by the contractor, also taking into account the intervention logic prepared for the previous external evaluation. A high-level overview of the intervention logic for this evaluation is included as Annex IV in this Staff Working Document. The high-level principle of the Agency is the aim to contribute to a healthier and more secure Europe and to meet the information needs of policy-makers and practitioners at EU, national and international level (general objective). As the Agency is subject to a complex hierarchy of specific objectives, based on the founding Regulation and the different strategic documents, the intervention logic illustrates the core objectives and priorities stemming from these documents. The Agency’s inputs are its financial and human resources, while its core tasks are set out in the founding Regulation. The following three levels of expected results can be identified:

-Outputs are the immediate products and services. These include the different periodic reports, i.e. annual and multiannual reports, such as the European Drug Report (EDR), the EU Drugs Market Report (EDMR) and the European Drug Responses Report (EDRR); strategic and situational analysis and threat assessment analysis; statistics, Country Drug Reports, thematic and technical reports and other written outputs; training and capacity building, etc. -Outcomes are second-level intermediate results stemming from the outputs. These include facilitated information exchange, better informed policy-makers and practitioners, transferring know-how and improved cooperation and coherence with partners. -Impacts are third-level long-term results, which are influenced by factors outside the control of the Agency. Its activities ultimately aim to support the objectives of the EU drugs policy, including the EU Drugs Strategy and its Action Plans, in reducing both demand and supply of drugs and thus contribute to a reduction in drug use and minimising the threat of this phenomenon. The general baseline for the establishment of the Agency was that no factual, objective, reliable and comparable information at European level on drugs and drug addiction and their consequences was available. No concrete or detailed baseline was established at that time. The contractors tried to re-establish baselines by using – as much as possible – the results of the previous external evaluation. However, this was possible only to a limited extent as the baselines were lacking for most elements. III. IMPLEMENTATION / STATE OF PLAY The European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) was set up in 1993 as one of the EU’s decentralised agencies. It started working fully in 1995 and is located in Lisbon, Portugal. Structure and management The Agency’s organisation consists of a Directorate, two statutory bodies (the Management Board and the Scientific Committee) to advise and assist in the decision making process, and various working units (which comprises the majority of the staff) to deliver its output. The Agency has approximately 100 staff members (state of play June 2018) and is managed by a Director who is appointed for a 5-year period by the Management Board based on a proposal by the European Commission. The Director is ultimately responsible for meeting the strategic objectives and implementing the Work Programmes, budget frameworks and Management Board decisions. He is accountable for his activities to the Management Board. The internal organisation looks currently as follows: The Management Board is the main decision-making body. It meets at least once a year and consists of one representative from each EU Member State, Norway and Turkey, two representatives from the European Commission and two independent experts particularly knowledgeable in the field of drugs designated by the European Parliament. Several observers are invited to participate in the Management Board meetings. The Chairperson and the Vice-Chairperson of the Management Board are elected for a (once renewable) three-year period. An Executive Committee assists the Management Board. The Executive Committee consists of the Chairperson and the Vice-Chairperson of the Management Board, two other members of the Management Board representing the Member States and appointed by the Management Board and two Commission representatives. The Executive Committee prepares the Management Board meetings and decides on those matters provided for in the Financial Framework Regulation which are not reserved to the Management Board by the founding Regulation. Although not provided for in the founding Regulation, the Agency also set up a Budget Committee to ensure better monitoring and follow-up of financial issues. The Scientific Committee advises the Board and the Director on the scientific aspects of the work and contributes to the risk assessment of new psychoactive substances (NPS). The Scientific Committee consists of at most fifteen well-known scientists appointed in view of their scientific excellence and their independence by the Management Board, following the publication of a call for expression of interest in the Official Journal of the European Union. The members of the Scientific Committee are appointed in their personal capacity. Reitox is the European information network on drugs and drug addiction. Members of the Reitox network are designated national institutions or agencies responsible for data collection and reporting on drugs and drug addiction (‘national focal points’ or ‘national drug observatories’). The Reitox network links national drug information systems and is the main way in which the Agency exchanges data and methodological information on drugs and drug addiction in Europe and thereby directly contributes to the core task of collecting and reporting consistent, harmonised and standardised information. The three core functions of a national focal point are: data collection and monitoring; analysis and interpretation of data collected; and reporting and dissemination of the results at national level. Budget The annual budget over the evaluation period was around EUR 16 Mio. The main part of the budget comes from the European Union subsidy. Norway and Turkey, both members without voting rights, also contribute to its budget. Main areas of work The key areas of activity of the Agency are in accordance with Article 2 of its founding Regulation: collection and analysis of existing data; improvement of data comparison methods; dissemination of data; cooperation with European and international bodies and organisations as well as with third countries; information obligations; and – as added by the 2017 amendment to the founding Regulation – exchange of information on, early warning system for, and risk assessment of, new psychoactive substances.

During the evaluation period, the Agency published two 3-year strategies and work programmes: the “EMCDDA 2013-15 strategy” and the “Strategy and work programme 2016-18”. For the first time, a 10-year strategy was also published, the “EMCDDA Strategy 2025”. These strategic goals were transformed into more concreted (planned) outputs in the annual work programmes. They provided detailed information about the activities and outputs of the Agency. The coherence of the main areas of work and goals set out in these documents is presented below. The 2013-2015 Strategy identified 12 main areas, each with a goal and further specific objectives and ‘primary interventions’. The 2016-2018 Strategy and Work Programme was built around six strategic action areas, composed of three key action areas: communicating evidence and knowledge exchange; early warning and threat assessment; and situation, responses and trend analysis; three cross-cutting action areas: information collection and management; quality assurance; and cooperation with partners; and two corporate areas: governance, and administration and ICT. The main areas and goals are presented in more detail in a table in Annex V. The “EMCDDA Strategy 2025” attributed the activities and the tasks of the Agency to two main pillars, health and security. In addition, it defined four business drivers, i.e. institutional, partnership, scientific capacity and management. The strategic objectives of the two pillars reflect the priority areas set out by the founding Regulation. While the Agency is primarily European in focus, it also works with partners in other world regions, exchanging information and expertise. The Agency cooperates with candidate and potential candidate countries as part of their accession process to the EU. When it comes to neighbouring countries of the European Neighbourhood Policy and with other non-EU countries, cooperation takes place based on bilateral agreements or in the context of EU-funded projects. Cooperation ranges from implementation of technical assistance projects to ad-hoc training or consultative support. Collaboration with European and international organisations in the drugs field is central to its work as a means of enhancing understanding of the global drugs phenomenon. Main achievements The achievements of the Agency are set out in detail and on an annual basis in the General Report of Activities. Some key achievements of the Agency in relation to the priority areas set out in the founding Regulation are summarised below. This is only a small snapshot of the work of the Agency and more detailed outputs and achievements are presented throughout this Staff Working Document. The main achievement of the establishment of the Agency is the availability of factual, objective, reliable and comparable information at European level concerning drugs and drug addiction. This is crucial to allow the development of an integrated, balanced and evidence-based approach to drugs policy. Some examples of main achievements: When it comes to the monitoring of the state of the drugs problem, the solutions applied to drug-related problems as well as of European and national policies, the Agency has published almost 400 scientific and institutional reports from 2013 to 2017. Among its flagship publications are the annual European Drug Report, including the country reports, the EU Drug Markets Report, published in 2013 and 2016, and “Health and social responses to drug problems – A European Guide”, which was published for the first time in 2017 but will be updated every 3 years. The Agency facilitated the exchange of best practices and drug-related information through the participation of its staff in more than 1500 events (contributions to conferences, policy forums, etc.) and authoring more than 150 scientific articles during the same period. The Agency organised numerous meetings of its stakeholders, mainly in Lisbon, which also allowed for the sharing of best practice. When it comes to new psychoactive substances, the maintenance of the Early Warning System, which is available 24/7, is crucial for the assessment of their risks. The Agency carried out 26 joint reports and risk assessments from 2013 to 2017. IV. EVALUATION QUESTIONS In accordance with the Commission's Better Regulation Guidelines, the evaluation looked at the effectiveness, efficiency, relevance, coherence and the EU added value of the work of the European Monitoring Centre for Drugs and Drug Addiction. The following questions were addressed during the evaluation: Relevance 1.To what extent have the EMCDDA work programmes covering the 2013-2018 period addressed the objectives, tasks and priorities set out in the EMCDDA recast regulation as well as those of the EU Drugs Strategy and its Action Plans and those stemming from the European Agenda on Security? 2.To what extent have the outcomes of the EMCDDA work programmes covering the 2013-2018 period responded to the needs of its multiple stakeholders (policy-makers, scientific community, practitioners and the general public)? 3.To what extent are the EMCDDA activities contributing to the EU's priorities? 4.How well adapted is the EMCDDA to subsequent economic, technological, scientific, social, political or environmental advances? Effectiveness 5.To what extent has the EMCDDA met its core objective as required in its regulatory framework to provide the EU with factual, objective, reliable and comparable information? 6.To what extent has the EMCDDA achieved the objectives of its two three year work programmes 2013-2015 and 2016-2018?

7.To what extent have the Reitox national focal points delivered the data and information required to meet the objectives of the aforementioned EMCDDA work programmes? 8.To what extent have the changes in the EMCDDA governance structure resulting from the implementation of the EMCDDA Strategy 2025 and the recent internal re-organisation impacted on the effectiveness of the EMCDDA? 9.To what extent are the EMCDDA tools to monitor and review its outputs and results adequate for ensuring accountability and an appropriate assessment of performance? 10.To what extent are the internal and external mechanisms for monitoring, reporting and evaluating the agency adequate for ensuring accountability and appropriate assessment of the overall performance? 11.To what extent and how have external factors influenced the effectiveness of the agency? Efficiency 12.Which were the costs and the benefits of the EMCDDA activities? 13.To what extent has the EMCDDA efficiently spent its resources (human and financial) to achieve the objectives set out in its work programmes during the 2013-2018 period? To what extent are available resources adequate to these objectives? 14.Is the EMCDDA providing value for money? 15.To what extent have the EMCDDA governance, organisational set-up, management systems and working methods been conducive to the efficiency of its operations? 16.Is there scope for simplifying the administrative set-up and working methods in the context of the current administrative and financial regulations? 17.To what extent and how have external factors influenced the efficiency of the agency? Coherence 18.To what extent are the objectives and activities of the EMCDDA for the 2013-2018 period coherent with the objectives set out in its regulatory framework? 19.To what extent are the objectives and activities of the EMCDDA coherent with the EU policy developments as defined in the documents presented in "EU policy documents" section? 20.To what extent are the objectives and activities of the EMCDDA coherent with and complementary to the drugs-related objectives and activities of the Commission, of other EU agencies, namely Europol, European Centre for the Prevention of Disease Control and the European Medicines Agency, and of the Member States? What are the overlaps and potential synergies and the option of merging with other EU agencies carrying out similar or identical activities? EU Added Value 21.To what extent have the EMCDDA activities and outputs helped to improve the ability of Member States to monitor and respond to drug problems compared to what they could do at national level? 22.To what extent have the EMCDDA activities provided a European level information resource for informing the policy debate on drug issues?

23.To what extent has the EMCDDA been more effective in achieving its objectives in the 2013-2018 period compared to existing or alternative options of implementing drugs policy as set in the strategic documents referred to under point 2 (e.g. by the Member States, through the Commission services themselves, an executive agency, etc.)? 24.To what extent are the outcomes of the EMCDDA activities sustainable? 25.What would be the most likely consequences of the termination of the agency? Lessons learnt The evaluation also assessed how the EMCDDA could assist policymakers and practitioners even more in planning and delivering policies and programs that contribute to a healthier and more secure Europe, in an evolving political, socio-economic, legal, demographic international context. In particular, the following questions were addressed during the evaluation: 26.On the health aspects, to what extent can the scope of monitoring and identifying best practices be broadened in the field of prevention and care of addictive behaviours (illicit and licit substances including alcohol, but also addictions without substances, like gambling)? 27.In terms of synergies with other JHA agencies, to what extent and how can the agency be more to the forefront of the security issues and inform more on threats and trends concerning links between drugs trafficking and organised crime, drug-related crime, drugs markets, in particular the way drugs can be sold and bought through Internet, social networks and encryption software? To what extent and how synergies with other JHA agencies and in particular with Europol can be increased in this field? 28.How could the EMCDDA work better with International organisation, such as the World Health Organisation and the United Nations Office on Drugs and Crime (UNODC)? At UN level, as the EU strongly advocates for evidence-based policies and has the most performant regional data collection system, to what extent and how can EMCDDA promote its expertise in the revision process of the UN data collection system, including the possibility for the EMCDDA to respond for the EU to certain UN questionnaires and enquiries? 29.At international level, to what extent can the international activities of the EMCDDA be optimised in terms of impact? Are the activities carried out by EMCDDA outside the EU compatible with the EU priorities in the external action? Should EMCDDA do more or less in the international sphere? 30.In terms of communication/dissemination, to what extent and how can EMCDDA spread more widely its outputs, thus reaching out a higher number of policy-makers and professionals? 31.What conclusions and recommendations can be drawn from the evaluation of the EMCDDA and its work programmes relating to the 2013-2018 period, with a view to supporting the post-2020 EMCDDA programming cycles, future Commission's proposals for EU drugs policy initiatives and the possible revision of the EMCDDA mandate? V. METHOD A. SHORT DESCRIPTION OF METHODOLOGY The external evaluation study was carried out by ICF Consulting Services Limited (ICF) in cooperation with the Centre for the Study of Democracy (CSD) and Optimity Advisors, following a call for services under a DG HOME framework contract. The evaluation was conducted through a mixed methods approach and was informed by the triangulation of a variety of sources. For more details on the methods and the stakeholder consultation, see Annexes II and III of this Staff Working Document. The evaluation was informed by a thorough desk research of documentation being available online or provided directly (by e-mail and/or on a dedicated extranet site on the Agency’s website to which only the contractor and the responsible DG HOME desk officer had access). The external evaluators had interviews with 133 stakeholders, which were accompanied – where appropriate – by questionnaires. Targeted online surveys took place for the Agency staff, on the one hand, and the scientific community and civil society, on the other hand. 55 and 24 replies, respectively, were gathered through the two surveys. A panel of experts undertook a peer review of selected publications; the results of the peer review were integrated in the final report and informed the replies to the evaluation questions. Finally, the contractor untertook five case studies, which provided input to the evaluation and were presented as such in the final report.

The preliminary results and conclusions of the evaluation were discussed during a workshop on 6 September 2018 in Brussels. In addition to the targeted consultations, the Commission organised an internet-based public consultation. 147 contributions were submitted through the online questionnaire during the consultation period from 18 May to 30 August 2018. The contributions were published on the Commission website. An overview and analysis of the consultation activities are presented in Annex III. B. LIMITATIONS AND ROBUSTNESS OF FINDINGS The following limitations to the robustness of the findings have been identified by the contactor: -Available baseline: as already mentioned above, it was difficult for the contractor to define a baseline on all elements of the evaluation due to different approaches compared to and missing in the previous external evaluation. As far as possible, a baseline was identified nevertheless by the contractor. -Short duration of the evaluation: this was identified from the outset, i.e. in the offer of the contractor, as a potential risk for the external evaluation. The contractor took the necessary measures by spreading the work over different subcontractors. -Low response rate: this applies, on the one hand, to the targeted survey of the scientific community and the civil society, and, on the other hand, to certain questions in the questionnaires, in particular those that needed specific knowledge. For both issues the contractor took the necessary measures by carrying out additional desk research (e.g. as regards the efficiency criterion for which only the Agency staff could provide any information) and/or by using the outcomes of other methods to complement the missing information (e.g. civil society representatives also provided input through the public consultation). -Quality of the received data: in some cases, open-ended questions were used in the interviews and in the public consultation. This might have led to some respondents providing substantive answers whereas others did not answer the question or did not think about the same particular issue. Therefore, the evidence presented in quantitative terms should be treated with caution in this regard. In addition, there were slight variations in the formulation of the question regarding the potential for an expansion of the Agency’s competences. However, these slight variations should not have any impact on the overall results. -Performance measurement: due to the use of different key performance indicators (KPIs) over the evaluation period and the lack of an Activity Based Budget, the contractor faced some challenges in measuring the performance of the Agency. This was mitigated by using headline indicators instead. -Stakeholder bias: there might be a certain positive bias of the results as most of the stakeholder groups work closely with and/or benefit from the outputs of the Agency. However, due to the variety of the stakeholders consulted and the different data collection methods employed, this should be minor. In addition to these limitations, which apply to the results of the evaluation, the evaluation has to be seen in the context of the future multiannual financial framework. This provides a potential budgetary constraint for any considerations regarding any future development, being within its current mandate or through the deepening or expansion of its mandate. Despite the limitations set out above, the evaluation findings are valid and reliable. Mitigation measures were taken and a large body of qualitative evidence, which provides a solid basis for drawing conclusions, underpins the evaluation. VI. ANSWERS TO THE EVALUATION QUESTIONS The information provided in this section is a summary of the information included in the final report of the external evaluation, which will be made available online. More detailed information can be found, therefore, in the (main) final report as well as in Annex 3 of the final report of the external evaluation. A. RELEVANCE The relevance criterion considers the adaptability of the Agency to its stakeholder needs as well as to scientific, economic, political, social and technological changes. In addition, the question related to the potential expansion of the mandate (listed under “Lessons learned” in the Terms of Reference) was also addressed under the relevance criterion.

The stakeholder needs of the Agency are very diverse. Overall, the analysis showed that the Agency responded very well to these diverse needs. The Agency has an excellent international standing and is regarded as a credible and authoritative source. The participants in the public consultation consider the information provided by the Agency useful across all areas. The information found most useful related to the emergence and use of new drugs (90% found it either “very useful” (63%) or somewhat useful (27%)) and EU drug policy (90% found it either “very useful” (61%) or “somewhat useful” (29%)). The least useful area of information related to links between drug trafficking and organised crime with 15% finding it not useful, 38% somewhat useful and 26% very useful. The main stakeholders of the Agency are policy-makers on national and European level. Their primary need is information and evidence on which to base policy decisions. The evaluation showed that the Agency is more relevant to the needs of European than national policy-makers. As regards the national policy-makers, the relevance of the information provided by the Agency for policy decisions depends partly on the size of the Member State and the related level of development of drug information systems. The better developed the national drug information systems are, the lower the impact of the information provided by the Agency. However, stakeholders, in particular the interviewed members of the Horizontal Working Party on Drugs, recognised cross-EU comparative analysis across the board as highly relevant to their needs. Another reason for the lower impact on national level might be that national drug policies of several Member States go beyond illicit drugs and cover other addictions. As regards the scientific community and practitioners, their needs were partially addressed. The respondents from the scientific community indicated that they use the work of the Agency on a daily basis. However, there is scope for greater engagement of the Agency with the scientific community. The practitioners from the public health area appreciated the increased work on this pillar. The focus of the consulted stakeholders in this area is on harm reduction and prevention and the respondents from this field suggested increased involvement of the Agency. The Agency is increasingly engaged with law enforcement practitioners, through e.g. input into trainings of the European Union Agency for Law Enforcement Training (CEPOL) and closer cooperation with Europol. This is linked to the increased focus of the Agency on supply side issues. Finally, when it comes to the general public, the evaluation concluded that the visibility is not very high. It should however be noted that the general public is not defined as a core customer. Information provided on national issues is usually targeted at the national focal points, which are then disseminating the information as appropriate in the Member State. The evaluation confirmed that the Agency adapted overall well to the developments posed by the changes on the drug markets during the evaluation period. Examples are the use of new monitoring tools and methods, such as wastewater analysis, the use of open source data, trendspotting and similar; the reorganisation of the Agency to better reflect the political priorities by putting more emphasis on security and public health as the two pillars of the Agency’s work; or the integration of new digital solutions. In addition, the Agency adapted to the changing economic environment, i.e. budget constraints, by redeploying resources to (new/different) priority areas without an increase of its overall budget. At the same time, stakeholders suggested that the Agency could produce more forward-looking products and that the Agency could invest more in IT training for staff and upgrading their databases and data collection systems. Stakeholders from the scientific community underlined the need to improve the dialogue with the scientific and research communities to maintain and expand the scientific expertise of the staff. One of the issues where the previous external evaluation stated that further work should be done is the drug supply side. The Agency stepped up work on supply side issues compared to the previous evaluation period. This led, for example, to an intensified cooperation with Europol and other relevant interlocutors, the publication of several topical reports and the stronger involvement in the EU policy cycle. In addition, the Agency is collecting more information on supply side issues through the Reitox network. However, the current evaluation showed that even more could be done in this context and that the Agency’s work to monitor supply side issues should be further enhanced. This could include further strengthening of cooperation with Europol and other bodies working in the area. Several stakeholders mentioned during the consultation the supply side as one element where further information from the Agency would be welcome. However, it is important to stress that this would have to be matched with increased data provision by the national focal points on drug supply related issues. In order to be able to do so, the national focal points would have to collaborate closely with the law enforcement and other relevant agencies in their Member State. It should be noted that full achievement of this objective is dependent on the mandate and financial resources available to national focal points. Another element where some stakeholders suggested that the activities could be improved is harm reduction and public health. In this context, the Agency could consider focussing more on assessing the effectiveness of harm reduction and prevention interventions and social integration measures as well as on vulnerable populations. This is related closely to the fact that although outreach to stakeholders is well developed and appreciated, a stronger involvement of practitioners could help to make an even stronger impact on contributing to a healthier Europe. In addition to the better regulation related questions on relevance, the evaluation also looked under this criterion at the question on the potential broadening of the scope of monitoring and identifying best practice in the field of prevention and care of addictive behaviours. This question addressed the issue of whether the Agency should continue to focus on illicit substances only or whether its mandate/competences should be revised to also cover other addictions, including e.g. alcohol, tobacco, prescription medicines and addictions without substances such as gambling. The background to this question is that national drug strategies of several Member States already have a broader scope than illicit substances. The following table provides an overview of (national) drug strategies with a broader focus (state of play 2016). Arguments put forward during the consultation included that addiction is an overarching health problem and should be addressed holistically; the trend of Member States to move to broader addiction strategies; the arbitrary distinction between what substances are licit or illicit; and the capacity for collection of good quality data. It should be noted that some data related to other substances are already collected by the Agency due to the involvement in the European School Survey Project on Alcohol and Other Drugs (ESPAD) and the need to address poly-drug use as part of the existing mandate. Other stakeholders argued that such a broadening could lead to a loss of focus on current work and could decrease quality; a lack of expertise and resources in the national focal points to provide the additional data; and issues related to EU competences and potential creation of overlaps. The stakeholder consultation was inconclusive as to the way forward. Whereas national stakeholders and Agency staff were largely in favour of broadening the mandate, other stakeholder groups (including EU-level stakeholders) were critical vis-à-vis such a development. Among those speaking in favour of a potential future broadening of the mandate, alcohol was mentioned most frequently. Other substance-related addictions followed. National stakeholders expressed less support to include addictive behaviours (not related to substance use) as it would be difficult to set the limits for such a mandate. All stakeholders, whether in favour or against a broadening of the mandate, stressed that any change in the mandate would have to be accompanied by sufficient financial and human resources as there is no scope to do so within the currently available resources. B. EFFECTIVENESS The effectiveness criterion considers the extent to which the Agency’s activities and outputs met its objectives set out in the founding Regulation, but also in the EU Drugs Strategy (and its Actions Plans) and the EU Agenda on Security. In addition, the question related to the international activities (listed under “Lessons learned” in the Terms of Reference) was addressed under the effectiveness criterion. The main objective of the Agency is to provide factual, objective, reliable and comparable information on drugs, drug addictions and their consequences, as set out in Article 1(2) of the founding Regulation. The Agency has fully achieved this objective as it is highly regarded as a centre of excellence in providing such information, not only in Europe but internationally. The information provided is considered politically independent, is tailored to the particular stakeholder groups it wants to address and is based on an increasing use of different sources. As regards “monitoring the state of the drugs problem, in particular using epidemiological or other indicators, and monitoring emerging trends, in particular those involving poly-drug use”, the Agency performed overall quite well. The Agency’s provision of demand and supply data contributed to informing relevant authorities and practitioners, improving their ability to respond to drug trends. The Agency established mechanisms to carry out regular and sustained monitoring of developments in the drug field, aimed at identifying emerging risks. The monitoring task is complemented by publishing regular and up-to-date information on drug supply and demand developments, in particular through the European Drug Report, but also through its many other publications. However, the evaluation also concluded that more could be done as regards drug-supply indicators and that work on poly-drug use is largely lacking. As regards the latter, the external study stated that the term “poly-drug use” is not mentioned once in the Agency’s 3-year strategies and work programmes. As regards “monitoring the solutions applied to drug-related problems; providing information on best practices in the Member States and facilitating the exchange of such practices among them”, the conclusion of the external evaluation is similar. Stakeholders evaluated the performance of the Agency positively, in particular the civil society organisations participating in the public consultation as well as answering to the survey and to the targeted interviews. The Best Practice Portal is central in sharing best practice in the areas of prevention, treatment, harm reduction and social reintegration. In addition, the work of the Reitox network as well as holding expert meetings is crucial for sharing best practices across Member States.

As regards “assessing the risks of new psychoactive substances and maintaining a rapid information system with regard to their use and also regarding new methods of using existing psychoactive substances”, the Agency contributes through the implementation of the Early Warning System (EWS), which is available 24/7, and the risk assessments of new psychoactive substances. Over the last years, the number of new psychoactive substances detected for the first time in the EU and therefore notified to the Early Warning System decreased from a peak in 2014 (see the figure below). At the end of 2018, the Early Warning System monitored more than 700 new psychoactive substances, of which more or less half are available in any given year on the European market. The monitoring by the Agency enhances Member States’ capacity to tackle this growing phenomenon. However, the substances detected are becoming more dangerous and, therefore, the Agency finalised in 2017-18 eleven risk assessments, leading to Commission proposals for putting the substances under control. The role of the Agency in this process has been reinforced by the new legislation on new psychoactive substances. Finally, as regards “developing tools and instruments to help Member States to monitor and evaluate their national policies and the Commission to monitor and evaluate European Union policies”, specific support was provided upon request to some Member States to help them design or monitor national drug strategies and policies. This was the case in 2015 for Germany, Ireland and Luxembourg. Tools and instruments for the evaluation of drugs policies and monitoring of drug markets have been developed. The achievement of the objectives of the 3-year strategies and the related work programmes has further improved from an already good baseline in the previous external evaluation. This is partly due to a more focussed and realistic planning as consecutive strategies and work programmes reduced the number of planned outputs following a streamlining of the activities. Another element for improving the delivery of outputs is related to the three-level prioritisation system introduced. The evaluation showed that the Agency achieved the goals and objectives set out in its two 3-year strategies and work programmes as shown by the table below. Implementation of Annual WPs 2014 2015 2016 2017 Level 1 (target 100%) 88% 92% 97% 97% Level 2 (target 70%/80%) 78% 77% 83% 92% Level 3 (target 40%/50%) 66% 75% 60% 77% This conclusion is also confirmed by the outcome of the public consultation as shown in the figure below. As regards the question to what extent the national focal points delivered the data and information required for the Agency to meet its objectives, the overall conclusion of the evaluation is that the data and information was delivered effectively during the evaluation period. The Reitox network and its national focal points are considered a cornerstone of the Agency’s work and activity. The “Reitox Development Framework”, which was put in place in 2018, addresses several of the issues identified by the external evaluation and includes measures to build up further the capacity of the national focal points and to improve the quality of their data collection and analysis. New reporting packages, including detailed handbooks, guidelines and procedures, were adopted during the evaluation period in order to rationalise the data collection process. The provision of data by the national focal points by the deadline set by the Agency increased considerably. This is crucial for the Agency being able to provide the analysis of the data in due time for the publication of the annual European Drug Report. Despite progress made, further improvements are considered possible and necessary. This relates in particular to the comparability of the data provided and in some cases the data quality. The problems identified by the external evaluation as regards the data provision by the national focal points are related to the varied mandates and resources available to national focal points. The appointment, setting up, functioning and maintenance of a national focal point is the responsibility of the national authorities and is therefore 100 % ﬁnanced by the country concerned. However, the Agency co-ﬁnances national focal points in EU Member States to a certain extent by means of a grant agreement, which supports the implementation of speciﬁc activities in relation to national reporting functions and obligations in the Agency’s annual work programme. The mandate of the national focal points also depends on the national drug policy. For example, the mandate can go far beyond drugs to cover different tasks as regards all kinds of addiction or it can be very narrow and only cover certain data collection tasks. Governance The previous external evaluation already concluded that the governance structures and procedures worked well. The changes made to the structure and the internal reorganisation are quite recent as they happened in 2016 and 2017 following the adoption of the “EMCDDA Strategy 2025”. Therefore, their impacts are not fully visible yet. However, the findings of the current evaluation, in particular the input received from interviews and the survey of the Agency’s staff, suggest that the changes have been generally positive. The internal reorganisation helped to focus resources better on the two pillars of the “EMCDDA Strategy 2025”, i.e. security and public health. This reorganisation improved the internal coordination within the Agency, led to providing more robust data and addressing new issues, and enabled the Agency to react better to challenges and contextual shifts in the field of drugs. Monitoring The evaluation report stresses that due to the use of different key performance indicators (KPIs) over the evaluation period and the lack of an Activity Based Budget, performance monitoring by the contractor could be done only based on headline indicators. Further efforts to streamline the indicators as well as to introduce an Activity Based Budget should be undertaken by the Agency to mitigate this for the future. The tools available to monitor and review the Agency’s outputs and results are considered adequate for ensuring accountability and an appropriate assessment of its performance. Starting with the 2014 work programme, a specific key performance indicator (KPI) was put in place for the monitoring of the work programme delivery. A detailed Monitoring and Evaluation Plan has been developed as internal tool to follow-up. In addition, a Management and Implementation System, a performance monitoring IT system, is in place; however, only limited training of the staff happened so far. The evaluation concluded that the internal and external mechanisms in place for monitoring, reporting and evaluating work well and are adequate to ensure accountability and appropriate assessment of the overall performance. Several regular review exercises are in place to review progress and monitor the outputs and achievements of results. These measurements are regularly discussed and followed up in the appropriate meetings. A mid-term review of the “EMCDDA Strategy 2025” is planned for 2020. As regards external mechanisms, the Agency is subject to the scrutiny of the annual discharge procedure, the European Parliament, the European Court of Auditors and the Internal Audit Service. The General Report of Activities provides an annual follow-up to these monitoring mechanisms. External factors The evaluation identified four main external factors, which had an influence on the effectiveness of the Agency: -Changing European drug landscape: The significant increase of new psychoactive substances was considered the main challenge. The Agency stepped up the proactive monitoring of new psychoactive substances through its Early Warning System and developed innovative monitoring tools to identify the changes in drug consumption (e.g. trendspotting studies or wastewater analysis).

-Emergence of new marketplaces: New online marketplaces as well as changing trafficking routes are most notable. The Agency increased its collaboration with other EU agencies, such as Europol, to monitor better these new marketplaces and improved its online monitoring tools. This external factor will increase further over time and therefore adaptations of the work will be needed. -New security challenges: These challenges stem from the involvement of organised crime groups in several criminal activities (“poly-criminality”), such as drug trafficking, human trafficking, migrant smuggling, or financing terrorist activities. The links between these activities need to be addressed in the future and this will have an impact on the information to be provided to policy-makers. -Evolving national policies: With the change of national drug/addiction policies comes a change in the needs of national stakeholders. This had a significant impact on the Agency’s effectiveness, in particular with regard to the identification of best practices in health and social responses. Cooperation with international organisations and third countries The recast of the founding Regulation in 2006 gave the Agency a more explicit role as regards cooperation with international organisations and third countries. The previous external evaluation concluded that the “relationship with key partners at international level should be further developed to improve the capacity to monitor and analyse the drugs situation and responses to it”. The Agency took on board this recommendation through continued work on the “EMCDDA Strategy for International Cooperation”, which has largely been implemented, and consecutively, in 2018, the adoption of the “EMCDDA International Cooperation Framework 2018-25”. Its activities at international level are compatible with the EU priorities in external action. The Agency brings the EU’s evidence-based policy-making experience in drug monitoring to third countries, thereby helping to improve the global understanding of the drug phenomenon, including the impacts global developments have on the European drug market. The Agency is supporting the European Commission in several regional cooperation projects and the analysis of progress of candidate countries as regards adherence with the EU acquis in the drugs field. In addition, the Agency has concluded over time a number of working arrangements with third countries in order improve the capacity development in these third countries. Finally, the Agency cooperates closely with the United Nations Office on Drugs and Crime (UNODC) and the World Health Organisation (WHO), and to a lesser degree with other international organisations. As regards the cooperation with the United Nations Office on Drugs and Crime, several stakeholders raised the duplication of data collection as something that needs to be addressed, despite the need to overcome several legal and methodological difficulties. It was also commented during the consultation that the cooperation with countries of the Western Balkan and immediate neighbourhood of the EU might be a too limited approach and that the Agency, in general, could do more on the international sphere, although within the limits of the general resource constraints. The Agency should clearly define the benefit to its objectives and the understanding of the EU drug situation of cooperation with third countries. This could also include the review of the priority countries/ regions. International cooperation activities should focus on facilitating the obtaining of information in a timely and regular manner from third countries or international organisations. Based on these considerations, the countries for future cooperation could be chosen. In this context, the funding opportunities for the work with third countries should be looked at, in particular the possibility of the Agency to receive ad hoc funding to implement its activities outside the EU. C. EFFICIENCY The efficiency criterion considers whether the Agency achieved its objectives and delivered results in a cost-effective manner. More specifically, this section reviews the costs and benefits of activities; the adequacy of the human and financial resources to attain its objectives; value for money; the efficiency of its governance and organisational set-up and the scope for further simplification of its administrative structure and ways of working; and the external factors influencing its performance. In addition, the question related to the communication on and dissemination of outputs (listed under “Lessons learned” in the Terms of Reference) was addressed under the efficiency criterion. The annual budget over the evaluation period was around EUR 16 million. The three main types of expenditure are staff costs (on average 59% of the budget); operational activities (on average 27% of the budget) and support activities (on average 14% of the budget). The budget remained relatively stable over the evaluation period with a decrease in 2014 and a one-off increase in 2015. The budget distribution is comparable with other agencies of similar size. Distribution of budgets of selected EU agencies (2017) Budget EMCDDA FRA EU-OSHA EASO Staff 60% 53% 36% 21% Support/Administrative 10% 11% 8% 12% Operational 30% 36% 52% 67% The number of staff also remained stable with on average 105 full-time equivalents per year. The share of operational vs. administrative staff increased in favour of the operational staff whose share in the overall staff increased from 68% in 2012 to 71% in 2017. This lead to a consequential decrease in support staff, from 32% in 2012 to 29% in 2017. The benefits of the Agency include clear and thorough understanding of the drug situation in the EU, better informed debates on drug policies and strategy, more effective exchange of information among professionals (including through the Reitox network), a proactive approach to new psychoactive substances and emerging trends, and the promotion of scientific excellence. These benefits are difficult to quantify, but overall the outputs of the Agency are valued highly by all stakeholders at EU, national and international level. The evaluation concluded within the limits of the available data for the efficiency analysis, in particular that Activity Based Budgets were not available, that the Agency spent its resources, both human and financial ones, efficiently. Despite the budget staying relatively flat and the need to respond to new challenges and an increasing volume of tasks, the Agency managed to deliver without affecting the quality of the outputs by efficiently redeploying resources. Another indicator for the efficiency of the performance is the implementation of the annual work programmes and delivery of its outputs. Level 1-targets were implemented by 88% (2014) to 97% (2017). As the table below shows, the share of non-completed results/outputs declined from 21% in 2013 to 9% in 2017. At the same time, the number of specific objectives in the work programmes was streamlined without affecting the volume of its outputs and its outreach.

Status of results/outputs (2013-2017) Results/outputs 2013 2014 2015 2016 2017 Not implemented 1 0 2 2 0 Delayed 61 14 26 22 14 Cancelled 12 12 2 0 0 Partially implemented 15 54 14 2 0 Total not completed 89 80 44 26 14 Total planned per AWP 424 397 251 153 164 % not completed 21% 20% 18% 17% 9% The Agency consistently used all allocated funds to achieve its work programmes. The budget execution rate for core activities (excluding the special projects) was very high, over 99.6% in each of the evaluation years, reaching 100% for 2016 and 2017. This indicates that the allocated funds were spent efficiently and no waste occurred. Therefore, the evaluation concluded that the Agency provides value for money and that the resources available have been used efficiently. The evaluation also notes that two thirds of the stakeholders consulted considered the financial and human resources as adequate for the current priorities and tasks, but insufficient in case any new tasks would be added. Despite the overall very positive conclusion as regards the use of human and financial resources, further improvements are possible. Further synergies, in particular on administrative issues, should be considered with other agencies, in particular with the European Maritime Safety Agency (EMSA), with which the Agency already shares some resources and uses common services. The implementation of change management programmes and the further development of targeted use of secondments or short-term contracts with scientists or interns should be considered. The governance, organisational structures and procedures work in general well. The changes introduced following the adoption of the “EMCDDA Strategy 2025” were beneficial in terms of efficiency as it strengthened cross-unit working and the coordination of scientific activities. The increased efficiency is shown by the fact that the Agency produced more outputs while investing the same resources in absolute terms. Overall, the evaluation concluded that the structure and organisation is efficient and no major changes are needed. The following issues were mentioned as influencing efficiency: the decrease of the subsidy in 2014; the increase in numbers and in dangerousness of new psychoactive substances; the increase in poly-drug use; the decrease in national funding for national focal points; and the shift in policy focus to other (security) areas, such as terrorism. Overall, it should be noted that no additional financial (or human) resources were allocated to the Agency for additional work, except for the work related to the implementation of the new legislation on new psychoactive substances. As mentioned before, the Agency still managed to deliver results and outputs on these issues through the redeploying of resources. In terms of communication and dissemination, the feedback from stakeholders was very positive and it was considered as efficient. The evaluation concluded that the online outreach, in particular the social media outreach increased considerably, in particular taking into account the limited dedicated financial and human resources compared to other agencies. Improvements were made, following the recommendations of the previous external evaluation, as regards printed communication material, making in particular the European Drug Report timelier and including more visual content. However, these are still areas where there is potential for further improvement, i.e. making reports and information more interactive, providing more and better translation, and more generally improve the website. In addition, the outreach to the general public and national stakeholders (policy-makers and practitioners) should be further improved. Outreach at national level is done currently through the national focal points, which act as multipliers and disseminators of the Agency’s products. However, it could be worth considering addressing these target group also more directly. D. COHERENCE The coherence criterion considers the extent to which the activities and objectives of the Agency support the key EU policy developments, as well as are complementary to actions undertaken by other EU agencies and Member States, including being consistent with its regulatory framework. In addition, the questions related to synergies with other Justice and Home Affairs agencies on security issues and to cooperation with the United Nations level, in particular with the United Nations Office on Drugs and Crime, (listed under “Lessons learned” in the Terms of Reference) were also addressed under the coherence criterion. The external evaluation mapped the objectives, tasks and priority areas set out in the founding Regulation as the well as the EU priorities with the two 3-year strategies covering the evaluation period. The same was done for checking coherence more generally with EU policy developments. The desk research of the contractor and the mapping exercise of the founding Regulation and the EU strategies with the Agency’s 3-year strategies and work programmes and other relevant strategic documents of the Agency is the basis for the information presented below. The table below presents the number of specific objectives in the 3-year strategies and work programmes for each of the priority areas set out in Annex I of the founding Regulation. Priority area (as per Annex I of the founding Regulation) SWP 2013-2015 SWP 2016-2018 Number of specific objectives % of total no. of specific objectives Number of specific objectives % of total no. of specific objectives Priority 1 - Monitoring the state of the drugs problem, in particular using epidemiological or other indicators, and monitoring emerging trends, in particular those involving poly-drug use. 20 71% 25 76% Priority 2 - Monitoring the solutions applied to drug-related problems; providing information on best practices in the Member States and facilitating the exchange of such practices among them. 16 57% 21 64% Priority 3 - Assessing the risks of new psychoactive substances and maintaining a rapid information system with regard to their use and also regarding new methods of using existing psychoactive substances. 16 57% 25 76% Priority 4 - Developing tools and instruments to help Member States to monitor and evaluate their national policies and the Commission to monitor and evaluate European Union policies. 10 36% 15 45% The external evaluation shows that all priority areas set out in the founding Regulation have been addressed in both programming periods and that all objectives, tasks and priority areas are covered well by the work programmes. This confirms the conclusions of the previous external evaluation. One element of the founding Regulation where some gaps were identified during the evaluation is the monitoring of data on emerging trends in poly-drug use. There are only very few outputs in relation to the poly-drug use and the issue itself is as such not even mentioned in the Agency’s strategic objectives. Some work is done in the context of the European School Survey Project on Alcohol and Other Drugs (ESPAD), but the evaluation concluded that more could be done. Other areas where the evaluation concluded that more could be done are work on drug supply, in particular the related indicators, or further cooperation with the local level and with the scientific community. When it comes to the EU Drugs Strategy 2013-2020 and its Action Plans, the mapping showed a strong alignment between the Agency’s outputs and its results with key and overarching EU objectives. This is evidenced by the fact that the Agency is mentioned in 22 out of 55 actions of the EU Action Plan on Drugs 2017-2020 as responsible party. In addition, the Agency is required to report on 14 of the 15 indicators for both EU Action Plans. The mid-term assessment of the EU Drugs Strategy highlighted the significant contribution by the Agency and its Reitox network to the implementation of the EU Drugs Strategy and to a better understanding of all aspects of the drugs situation in the EU and trends in drug markets. The 3-year strategies and work programmes address and contribute to the objectives, principles and priorities of the European Agenda on Security. Given the limited remit of the Agency on the wider security issues, it cannot be expected that it comprehensively covers all aspects of the European Agenda on Security. However, through the growing focus on supply-side issues and the cooperation with other agencies working in the area (such as Europol, the Maritime Analysis and Operations Centre (MAOC(N)), or the European Union Agency for Law Enforcement Training (CEPOL)) on, for example, joint trainings for law enforcement agencies, joint publications or the stronger involvement of the Agency in the EU Policy Cycle, the Agency increasingly contributes to this priority.

Therefore, it can be concluded that the objectives and the activities set out in the two 3-year strategies and work programmes are highly coherent with the regulatory framework. All of the objectives set out in these documents align with and can be traced back to the founding Regulation. The impact case study indicated a moderate to strong causal link between the Agency’s activities and the impact of providing an evidence base for policy-making. Several activities, in particular in cooperation with other European and international agencies, provide evidence for the coherence of their work with EU policy developments. Coherence with the work of the most relevant EU agencies is facilitated by mutual review of the respective work programmes. Overall, the Agency is well aligned with the Common Approach for decentralised agencies. The tools and mechanisms in place work quite well and are adequate to ensure accountability and appropriate assessment of the overall performance. Monitoring of the implementation of the 3-year strategies and work programmes and the Management Plan takes place via a series of monitoring exercises. The development of clear key performance indicators is positive; however, these have changed during the evaluation period and therefore comparisons over time and with the previous evaluation were in parts not possible. In addition, the Agency has not yet implemented some Common Approach actions, in particular those relating to Activity Based Management and Activity Based Budgets. Other elements where improvements would be needed to achieve full compliance with the Common Approach are related to making the website as multilingual as possible, the Management Board considering the need for ex-ante evaluations of activities and programmes that have budget implications, and looking into the potential joint procurement with contracting authorities of the host Member States. All these actions are on-going within the Agency, but have not been completed for different reasons. When looking more closely into the coherence of the objectives and activities with the drug-related objectives of the European Commission and other EU agencies, in particular Europol, the European Centre for Disease Prevention and Control (ECDC) and the European Medicines Agency (EMA), the evaluation concluded that the objectives set out by the Agency in its 3-year strategies and work programmes and the “EMCDDA Strategy 2025” are complementing those of its partners on EU level. The analysis shows that there are hardly any overlaps with other EU agencies, but there could be considerable synergies. Many of these synergies are already exploited by the agencies. Examples include the sharing of information on new psychoactive substances among relevant agencies, in particular the European Medicines Agency; cooperation with Europol regarding the role of the internet and the darknet in selling drugs; the cooperation with the European Union Agency for Law Enforcement Training (CEPOL) on training activities; or the cooperation with Eurojust on issues related to judicial challenges in transnational prosecution. More concrete examples are provided in the following paragraphs. The merging with another agency was considered detrimental to the quality of the scientific output. In addition, the fact that the Agency’s activities are very specific is seen by consulted stakeholders as there not being a potential for a merge with another agency. The conclusion as regards coherence with Member States’ national drug policies is slightly less positive as some Member States apply a pan-addiction approach whereas the mandate of the Agency is limited to illicit substances. As the Agency cannot provide data on other addictions to the Member States to support their policies, some interviewed stakeholders, in particular from Member States, saw this as limiting the coherence of the work of the Agency with their work on national level. Synergies with the Commission exist for example in the work on drugs precursors and on health-related issues. As regards precursors, the Agency cooperates with DGs TAXUD and GROW. This cooperation takes place on a voluntary basis and therefore relies on the goodwill and dedication of the involved individuals. Therefore, there is a risk that these activities will be interrupted if staff or priorities change. This risk could be mitigated by a formal working arrangement with the Commission on drug precursors. Such a working arrangement already exists with DG SANTE. However, the evaluation concluded that a closer structural relationship could be developed. Cooperation also takes place with the Joint Research Centre (JRC), which is involved in the detection of drugs and precursors in Europe and provides data to the Agency. Cooperation with JRC is important, as the Agency does not have its own forensic laboratory facilities for scientific testing. The cooperation with Europol is very fruitful and is considered by Europol as optimal. Several joint publications have been published over the years, starting with the joint work on risk assessments of new psychoactive substances, to (up to now) two editions of the EU Drug Markets Report to specialised publications such as the report on drugs and the darknet. Both agencies provided their relevant expertise for the joint work, with Europol’s focus being on data on drug production, distribution, seizure and the role of organised crime. The cooperation with the European Medicines Agency is related mainly to work in the context of the Early Warning System and the assessment of new psychoactive substances. The two agencies also exchange and cross-reference each other’s data as part of pharmacovigilance efforts to prevent misuse of medicinal products and the use of new psychoactive substances. This leads to more accurate data for both agencies. The collaboration with the European Centre for Disease Prevention and Control focusses on areas where drug use constitutes a risk factor for the contraction of infectious diseases. The joint activities include gathering data from Member States and joint missions to specific Member States in the context of HIV. Again, the collaboration means that each agency can provide its scientific expertise to effectively conduct reviews and risk assessments. There is scope to update joint products more regularly and additional fields of work could be considered, assuming that resources are made available. Other relevant Justice and Home Affairs agencies with which the Agency already collaborates are the European Union Agency for Law Enforcement Training (CEPOL) and the European Union Agency for judicial cooperation in criminal matters (Eurojust). As regards the European Union Agency for Law Enforcement Training, the Agency contributes heavily to training activities aimed at enhancing awareness among EU law enforcement authorities working in the field of drugs. As regards Eurojust, a publication aimed at policy-makers and legal practitioners to find suitable responses to the phenomenon of new psychoactive substances was published. Given Eurojust’s increasing involvement in drug trafficking cases, the Agency is well placed to continue to inform and provide context on drug trafficking. No collaboration exists so far with the European Border and Coast Guard Agency. However, the evaluation recommended that joint activities with European Border and Coast Guard Agency should be explored, in particular as regards data exchange and the monitoring of potential links between drug trafficking, human trafficking and migration. The Agency also cooperates with the Maritime Analysis and Operation Centre – Narcotics (MAOC(N)), an operational platform of several Member States, third countries and agencies set up to principally tackle the transatlantic flow of cocaine from South America to Europe. The Maritime Analysis and Operation Centre – Narcotics is dependent on information from the Agency for its operations. Closer cooperation might be beneficial for both, on the one hand, for the Maritime Analysis and Operation Centre – Narcotics to further enhance its effectiveness due to more direct data access, on the other hand, for the Agency to be more involved in operational activities and thereby having a stronger impact on creating a more secure Europe. As regards the international level, the objectives were set out in the “2007 Strategy for international cooperation” which was replaced by the “EMCDDA International Cooperation Framework 2018-25” in 2017. When it comes to international organisations, the focus is the cooperation with the United Nations Office on Drugs and Crime and the World Health Organisation and in particular the exchange of data with these organisations. The Agency also participated as experts in several of their meetings and provided input to their publications. In addition, cooperation also takes place with the Pompidou Group and the American Drug Abuse Control Commission (CICAD). The external study shows that data exchange between the Agency and international organisations is largely unidirectional. An example is the work of the Expert Committee on Drug Dependence of the World Health Organisation on new psychoactive substances. The Agency is providing all data available in its database on the substances, which are critically reviewed by the Expert Committee. On the other hand, there is no or very limited provision of data from the United Nations Office on Drugs and Crime to the Agency, which then could be used by the Agency in its reports. Getting more information from international organisations would help the Agency to develop better its understanding of how the global drug markets affect the European drug market. The organisation with which most overlaps were identified was the United Nations Office on Drugs and Crime (UNODC). Several stakeholders, in particular the Reitox network, identified duplications in the data provision to the two agencies. The parallel reporting streams partly lead to discrepancies in data due to occasionally different individuals or departments reporting to the two organisations. It was suggested by some consulted stakeholders that the Agency could support the Member States in their reporting to the United Nations Office on Drugs and Crime; however, some practical and legal issues would have to be clarified beforehand. In any case, the two agencies should work towards further aligning the different reporting requirements and questionnaires. The Agency’s stakeholders also identified further scope for increased cooperation with international organisations on issues such as sharing best practices and adoption of common guidelines on data collection. E. EU ADDED VALUE The EU added value criterion considers the benefits that the Agency brings as compared to what could have been done at national level. More specifically, it seeks to assess the Agency’s capacity to improve the ability of Member States to monitor and respond to drug-related problems, and the extent to which it is a valuable source of information for its main ‘customers’. This section also explores the sustainability of the activities and identifies the advantage of attributing these tasks to an agency like the European Monitoring Centre for Drugs and Drug Addiction as compared to other alternatives. It concludes by reflecting on the consequences of a potential termination of the Agency. Its stakeholders regard the Agency as the main EU-level information source in the field of drugs. The provision of factual, objective, reliable and comparable information is the main objective of the Agency based on its founding Regulation. The evaluation concluded that the Agency has an excellent reputation as a source of scientific and comprehensive quality data, not only in Europe but also on an international level. The information, which is considered objective and not politically influenced, contributes highly to the policy-making on European and to a lesser extent on national level. The Agency contributes to informing the policy debate at EU level through the consolidation and dissemination of scientific and timely information, as well as through its participation in meetings and debates where drug policies are discussed (for example the meetings of the Council’s Horizontal Working Party on Drugs). The references made to the Agency and its outputs in various legislative and policy documents at EU level show its tangible impacts. This is especially visible with respect to new psychoactive substances, where the Agency has a central role in the risk assessment of new substances, and the action plans adopted in the framework of the EU Drugs Strategy, where the Agency has been assigned the role of responsible party in relation to several actions.

The information provided by the Agency is heavily relying on the provision of data from the national focal points. For this to happen, the Agency implemented a drug-related data collection system that requires the national focal points to collect and subsequently report to the Agency the same set of data using a common methodology for every reporting country. The data provided by the national focal points is then analysed by the Agency and provided in different forms, including reports, to its stakeholders. The EU-level overviews are used for the evidence-based EU-level policy-making, but are also valuable for national stakeholders. It allows Member States not only to design evidence-based policies but also to monitor the effectiveness of their strategies, policies and interventions as well as to increase their capacity to understand and to respond to drug-related threats. For the Agency to be in a position to provide sufficient information on emerging threats and trends in drug demand and supply, it would have to develop additional state-of-the-art methodologies to get the underlying data. This would make the Agency more responsive to emerging threats and the dynamic nature of the drug situation in the EU. One example of such additional state-of-the-art methodologies is wastewater analysis. This was funded originally by the Commission through project-based funding to research institutions with the involvement of the Agency. If such analysis should be continued, financial resources have to be made available to the Agency. Such new and innovative (state-of-the-art) data collection methods are needed to ensure that a complete picture of the drug phenomenon in Europe is available to policy-makers and that the Agency remains on top of the policy developments in the drugs field An important tool in this context is the Early Warning System (EWS) on new psychoactive substances. The Agency collects data provided by Member States, analyses the data and then shares the information with the Early Warning System network. This information is a key component of many national early warning systems. Other forms of support to the national level include specific support upon request of a Member State to help them design or monitor national drug strategies and policies; the identification and sharing of best practice as well as training which is related to capacity-building exercises; and financial support. When it comes to input towards national drug policies, the evaluation recommended that the Agency could further develop its tools to support Member States in evaluating their national drug policies and strategies and in assessing the effectiveness of drug-related measures. The feedback received from consulted stakeholders, in particular from the reporting countries, showed that in some countries, in particular those with a shorter tradition in doing so, the collection of drug-related data and the related monitoring might not survive without the Agency in place. Consulted stakeholders expressed clearly that without the common standards, guidance and methodological support provided by the Agency, it would not be possible to collect drug-related data with the same quality, accuracy and timeliness. A clear outcome of the evaluation was that without the help and support of the Agency such a comprehensive and comparable data collection would not be possible. Although national policy-makers often rely on the work of the Agency, e.g. as regards identifying best practices, evidence-based information on hot topics or input to their national drug policies, the evaluation concluded that the Agency did not always manage to inform the national debates to the same extent as at EU level. In particular those Member States with a robust national drug monitoring system do not necessarily use the Agency’s data and information as the primary source of information but might rely more on their national standalone reports. Several Member State representatives also claimed that the Agency did not succeed in triggering political shifts on a national level. Therefore, the Agency could enhance its engagement with national stakeholders and more proactively involve practitioners, including through training and exchange of best practices. As regards the sustainability of the activities, the main contributors to its sustainability are the nature of the data collected, i.e. the comprehensive nature and regular updating of the information provided together with the use of new analytical methods, which allow for long series of statistical data; the political impact of the outputs through the provision of input for evidence-based policy-making; the forward-looking vision of the EMCCDA; and the development of a cooperation network, in particular with the national focal points. Several other elements further support the sustainability such as the expertise available in the Agency, the development of methodological tools and indicators or the use of modern technologies. The stakeholders identified as the main challenge for sustainability going forward the availability of resources as limited resources will lead to the stopping of some of its activities. Another challenge is that keeping information relevant for future challenges requires significant efforts by the Agency to ensure that the data are up-to-date and that new information sources can be used. The question whether other existing or alternative options of implementing drug policy would have been more efficient in achieving the objectives set out in the EU strategic documents was assessed. The overall conclusion on this question from the external evaluation was that the Agency is the most effective option in achieving the objectives set out in the EU Drugs Strategy and its Action Plans. One alternative option would be to allocate the tasks to the Commission services in cooperation with other EU agencies already active in the area (such as Europol, the European Medicines Agency or the European Centre for Disease Prevention and Control). While this option would still ensure a regular EU-level data collection and analysis, it has two main disadvantages compared to the current set-up. Firstly, the Commission would have to outsource some of the activities, which would lead to a loss of institutional capacity and expertise in the longer term. Secondly, this loss of expertise might compromise the quality of the data analysis and thereby might lead to a loss of trust in the data and information provided. The second alternative option would see the responsibility for monitoring the drug phenomenon reverting to the Member States. The Member States would need to take steps to coordinate themselves and to ensure that the collection of existing data and descriptive analysis continues. However, an agreement on common methodologies, indicators and quality standards would be more difficult to reach, compromising the comparability of data used to produce EU-level overviews. In addition, the reliability and impartiality of the information might suffer as Member States with more expertise and resources would be able to influence the analysis and formulation of policies more than others. Finally, this option would also lead to losing the Agency as a major actor in implementing the EU Drugs Strategy and subsequently would make it more difficult for the EU to speak with one voice on the international level. The third alternative option would be to attribute the activities (or at least some of them) to international organisations active in the field, such as the United Nations Office on Drugs and Crime, the World Health Organisation, the Council of Europe or others. Although the collection of data would be maintained with a central organisation to coordinate, it would probably mean that the picture of the drugs phenomenon in Europe would be incomplete as less data on a regional level would be available. This might also lead to a loss of expertise and understanding of the influencing factors in Europe and to a diminished role of the EU on the international level. This links with the question about the most likely consequences of a potential termination of the Agency. All stakeholders agreed that the termination of the Agency would have negative impacts and should be avoided. While at the first instance it would be policy-makers who would be affected by such a termination, ultimately, the general public would also feel the impacts, albeit more indirectly. The most likely consequence would be a loss of EU-level information because of the interruption of data collection and analysis. As there would be no obligation to collect the relevant data and no more centralised support, some reporting countries would cease to collect the data. Even those who would continue collecting drug-related data would likely do so to a more limited extent and would agree on the least common denominator. This would result in incomplete and fragmented data. The exchange of best practice would be more difficult without a central organisation bringing the information and people together. The cessation of the Agency would undermine the possibility to design and implement drug policies and strategies based on objective and scientific evidence, both at EU and at national level. Finally, the termination would have negative consequences on the role of the EU in the international sphere. It would make it more difficult to have a common understanding of the drug phenomenon and therefore to speak with one voice in international fora. To conclude, the termination of the Agency would constitute an undesirable step backwards. VII. CONCLUSIONS AND LESSONS LEARNED The overall conclusion of the evaluation, supported by an external study carried out between March and November 2018 by ICF Consulting Services Limited (ICF) in collaboration with the Centre for the Study of Democracy (CSD) and Optimity Advisors, was that the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) is working well. The evaluation was positive as regards all five evaluation criteria, but further improvements are possible. The evaluation was carried out in accordance with the requirement set out in Article 23 of the founding Regulation, which obliges the Commission to initiate an external evaluation every six years to coincide with the completion of two of the Agency’s 3-year strategies and work programmes. The evaluation concluded that both pillars of the Agency, i.e. public health and security, are well covered by the work programmes and strategies. The Agency’s work programmes and strategies, but also the outputs and outcomes produced, are relevant in relation to the overall objectives of EU drugs policy. The Agency meets the needs of its many stakeholders and was able to adapt to changes in the drug markets as needed. The Agency’s work is coherent with the EU drug policy objectives. Most synergies available with the work of the EU institutions, other EU agencies and international organisations are already used. In general, only few overlaps of work exist due to the very specific work area of the Agency. As regards efficiency, within the limitations of the available data, the evaluation concluded that the Agency used the available human and financial resources efficiently to deliver the outputs, outcomes and impacts set out in its work programmes. The Agency is well recognised as a centre of excellence in providing information on the drugs phenomenon, not only in Europe but also internationally. The work of the Agency also has important impacts on the international level, including bringing the EU’s evidence-based policy-making in drug monitoring to third countries. The information produced is considered factual, objective, reliable and robust. The Reitox network is seen a cornerstone of the Agency’s work and activities as its members, the national focal points, are the providers of the relevant data.

There is scope for greater engagement with some other stakeholder groups, in particular the scientific community and practitioners, in order to better cover their needs in the future. As regards the general public, more direct engagement should be considered to improve the visibility, but also to ensure equal information across Europe, e.g. on new psychoactive substances. Outreach and communication should be improved in particular with these stakeholder groups. Synergies with the Commission could be reaped further by formalising the working arrangements regarding the work on precursors, developing a closer structural relationship when it comes to health-related issues and cooperating close with the Joint Research Centre. Despite the very positive recognition of the work of the Agency as data provider, the evaluation concluded that there is room for further improvement. During the interviews, stakeholders commented that more forward-looking information and products should be made available in order to better help EU preparedness and response in this fast changing policy area. Although many efforts have been made following the previous external evaluation on the comparability of data and considerable improvements have been implemented, further efforts are needed to ensure better comparability of data. This is true similarly for supply side information, where considerable efforts have been made during the evaluation period, but the Agency’s work on this should be further enhanced. However, more work on drug supply issues, including the further development of relevant indicators, would have to be matched by the availability of additional data, including through the provision of such data from the national focal points. Other areas, which could be addressed better and where the Agency’s input would provide added value, are poly-drug use and the support to Member States in evaluating their national drug policies. More targeted support, e.g. through Reitox Academies, is considered useful to develop further the knowledge among the staff of the national focal points. Several consulted stakeholders suggested the enhancement of the IT systems for data collection, in particular the FONTE-system. The use of visual aids in reports was acknowledged positively, but a few stakeholders suggested to further streamline the provision of information and to make it more user-friendly and interactive. The quality of the translation was put forward as a concern. It has to be acknowledged that translations are very costly and therefore a good balance in selecting the products for translation has to be found. However, whenever translations are made, they should be of good quality. The evaluation was inconclusive on the potential future broadening of the scope of the Agency to other licit and illicit substance and addictive behaviours. The Agency provides good value for money; however, there is room for improvement when it comes to simplification of the administrative set-up and the internal working methods. Despite the available budget remaining relatively flat over the evaluation period and the increased demands during the same time, the A