EUROPEAN COMMISSION Brussels, 14.5.2019 COM(2019) 228 final REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 {SWD(2019) 174 final}



I. CONTEXT The human and social costs of drug addiction are very high. Drugs and drug addiction affect the health and wellbeing of many people, in particular young ones, create security concerns and lead to premature deaths. Therefore, drug addiction generates costs for public health (drug prevention, healthcare and treatment), public safety, the environment and labour productivity. The market for illicit drugs is the most dynamic criminal market in the EU and is able to adapt rapidly in response to drug control measures. The European Monitoring Centre for Drugs and Drug Addiction (EMCDDA; referred to as “the Agency”) was established in 1993 as one of the EU’s decentralised agencies. Its objective, pursuant to Article 1 of Regulation (EC) 1920/2006 (referred to as “the founding Regulation”), is to provide the EU and its Member States with factual, objective, reliable and comparable information at European level on drugs, drug addiction and their consequences. The Agency’s core tasks are to collect and analyse existing data, improve data comparison methods, disseminate data and to cooperate with European and international bodies and organisations as well as with third countries. The Agency acts as a centre of excellence in providing information on the drugs phenomenon, not only in Europe but also internationally. Article 23 of the founding Regulation provides for "an external evaluation of the Centre every six years, to coincide with the completion of two of the Centre's three-year work programmes". The evaluation should also address the Reitox system (the European Information Network on Drugs and Drug Addiction). The previous evaluation was carried out in 2011/2012. Therefore, an evaluation was initiated by the Commission in late 2017 and carried out in 2018. The evaluation was conducted by a consortium lead by ICF Consulting Ltd., in collaboration with the Centre for the Study of Democracy (CSD) and Optimity Advisors, and took place between March and November 2018. The final report of the evaluation will be published online. This report will be submitted to the European Parliament and the Council as well as to the Management Board of the Agency. The Staff Working Document, which accompanies this Commission Report, provides a detailed analysis of the outcomes of the external evaluation. II. FINDINGS OF THE EVALUATION The evaluation had the following scope: -Material scope: the evaluation covered the two pillars of the Agency’s work, i.e. public health and security. It looked into its governance and administration, the organisational structure, operations, funding and resourcing, its information management and the work of Reitox, cooperation with other relevant EU agencies (such as the Justice and Home Affairs agencies, the European Medicines Agency (EMA), the European Centre for Disease Prevention and Control (ECDC)) and international organisations (such as the World Health Organisation (WHO), the United Nations Office on Drugs and Crime (UNODC), etc.), partnerships with third countries, and communication and dissemination of research; -Geographical scope: the evaluation covered the countries reporting data to the Agency, i.e. the EU Member States as well as Norway and Turkey, and third countries with which the Agency has closer relations due to the conclusion of working arrangements, cooperation agreements or similar; -Temporal scope: the evaluation covered the activities carried out from 1 January 2013 until 30 June 2018 to coincide with the completion of two consecutive 3-year strategies and work programmes of the Agency. The evaluation assessed the Agency, building on the findings of the previous evaluation, along the five standard evaluation criteria: effectiveness, efficiency, coherence, relevance and EU added value. The evaluation questions related to lessons learned were attributed to these five criteria. The conclusions of the evaluation are summarised below along those five criteria and in more detail in the accompanying Staff Working Document. The contractor identified a number of limitations when carrying out the external evaluation study, which have an impact on this report. The main limitation was the unavailability of a baseline for most elements, which made it difficult to analyse progress. Other issues, which might limit the robustness of the findings, include the short duration of the evaluation, the low response rate to some questions or from certain stakeholder groups, respectively, the quality of the received data as well as the use of different key performance indicators over the evaluation period and the unavailability of an Activity Based Budget. A. Relevance The relevance criterion examined the alignment of the Agency’s outputs with the needs of its multiple stakeholders. It assessed the Agency’s adaptability to scientific, economic, political, social and technological changes. Finally, the relevance criterion explored stakeholders’ views on any potential broadening of the scope of monitoring as well as the identification of best practice with regard to illicit and legal substances, and addictive behaviours not involving substances. The outcomes of the Agency's activities met the needs of its multiple stakeholders, by providing relevant and timely information. As shown through all means of the consultation activities, the Agency largely addressed the needs of policy-makers, especially at the EU level and to a more limited extent at the national level. There is scope for more engagement with the scientific community as well as increasing its visibility with practitioners and the general public. The Agency adapted well to various changes during the evaluation period. Drug demand and supply is a dynamic and changing landscape, and the Agency responded by increasing its focus on emerging threats and trends, developing new methodologies and using open-source information such as waste-water analysis or trendspotter studies. When it comes to publications, the Agency addresses different audiences by tailoring them to various types of stakeholders. Overall, the interviews showed a need for more forward-looking products identifying future trends and risks to better support EU preparedness and response in the ever-changing drugs landscape as well as communicating more directly with national stakeholders. The Agency carried out an internal reorganisation to align itself better with political and strategic objectives in drug policy, and addressed economic constraints by prioritising its activities and redeploying its resources. The evaluation considered also the question regarding the potential future broadening of the scope of the Agency. The outcome of the evaluation on this question was inconclusive. While the majority of Member State representatives and the Agency staff were in favour of an expansion to other licit and illicit substance, such as alcohol, tobacco or prescription medicines, and – to a lesser extent – addictive behaviours, such as gambling, EU-level stakeholders and international organisations presented divergent views.

B. Effectiveness The effectiveness criterion assessed how successful the Agency has been in achieving and progressing towards its objectives and the priorities set out in the founding Regulation. It examined the extent to which the changes in the structure resulting from the implementation of the “EMCDDA Strategy 2025” and the recent internal reorganisation impacted its effectiveness. Further aspects examined under this criterion included the effectiveness of the Reitox network, the use of internal monitoring tools, external factors affecting the impact of the Agency and its international activities, i.e. work with third countries and international organisations. Taking into account the overall impression from the stakeholder consultations, the evaluation showed that the Agency is well recognised and highly regarded by its stakeholder communities as a centre of excellence in providing information on the drugs phenomenon, not only in Europe but also internationally. The information produced is considered factual, objective, reliable and robust, as evidenced by the targeted survey for civil society and the scientific community as well as by the public consultation. The same sources showed that the main areas to be improved and strengthened include increasing the comparability of information, further use of visual aids and improving the quality of translation. The evaluation found that the majority of the objectives set out in the Agency’s 3-year strategies and work programmes were achieved. Regarding the monitoring of the state of the drugs problem, the Agency performed well. The Agency’s provision of demand and – and to a lesser extent – supply data contributed to informing relevant authorities and practitioners, improving their ability to respond to drug trends. The Agency established mechanisms to carry out regular and sustained monitoring of developments in the drug field, aimed at identifying emerging risks. The monitoring task is complemented by publishing regular and up-to-date information on drug supply and demand developments, in particular through the European Drug Report, but also through its many other publications. However, the evaluation also concluded that work on poly-drug use is largely lacking. In this context, the external study stated that the term “poly-drug use” is not mentioned once in the Agency’s 3-year strategies and work programmes. Similarly, the work of the Agency on the monitoring of solutions applied to drug-related problems was evaluated positively by stakeholders, in this case in particular by civil society organisations participating in the public consultation, answering to the survey and the few targeted interviews. The Best Practice Portal is central in sharing best practice in the areas of prevention, treatment, harm reduction and social reintegration. Through the implementation of the Early Warning System (EWS), which is available 24/7, the Agency contributed to the detection of new psychoactive substances (NPS) and subsequently enhanced Member States’ capacity to tackle this growing phenomenon. Over the last years, the number of new psychoactive substances detected for the first time in the EU and therefore notified to the Early Warning System decreased from a peak in 2014. At the end of 2018, the Early Warning System monitored more than 700 new psychoactive substances, of which more or less half are available in any given year on the European market. However, the substances detected are becoming more dangerous and, therefore, the Agency undertook in 2017-18 more risk assessment than ever before in a given year. The role of the Agency in this process has been reinforced by the new legislation on new psychoactive substances. Finally, when it comes to developing tools and instruments to support Member States with their national drug policies, specific support was provided upon request to some Member States to help them design or monitor national drug strategies and policies. This was the case in 2015 for Germany, Ireland and Luxembourg. Tools and instruments for the evaluation of drugs policies and monitoring of drug markets have been developed. The Reitox network effectively delivered the data and information needed to meet the objectives set out in the Agency’s 3-year strategies and work programmes during the evaluation period. The external study also concluded, based on the feedback from different stakeholders, including the EMCDDA staff, that the quality and timeliness of data provision varied between national focal points, although progress has been made since the last evaluation. The reason for these differences is mainly due to the human and financial resources available to the national focal points. Further improvements are possible in particular as regards the comparability of the data provided. The activities of the Agency at international level are compatible with the EU priorities in external action. The Agency brings the EU’s evidence-based policy-making experience in drug monitoring to third countries, thereby helping to improve the global understanding of the drugs phenomenon, which in turn translates into a more complete picture of the situation in the EU. International stakeholders and Agency staff generally agreed that more could be done at international level, i.e. working with other third countries and better cooperation with international organisations. However, this has to be in line with the Agency’s mission, linking back to a better understanding of the drugs phenomenon. The consulted statkeholders noted the existence of overlaps, such as the duplication of data collection with United Nations Office on Drugs and Crime, which should be addressed. Although the changes in the Agency’s structure resulting from the implementation of the “EMCDDA Strategy 2025” and the internal reorganisation are relatively recent and more time is needed for the impacts to materialise, they appear to have had a positive impact on the Agency’s effectiveness. The internal reorganisation has further supported the performance of the tasks set out in the founding Regulation and in achieving its objectives. For example, the restructuring of scientific resources in line with the two pillars of the “EMCDDA Strategy 2025” has helped the Agency to provide data that are more robust, adapt to new scientific and environmental developments, and react to challenges and contextual shifts in the field of drugs. External factors that have influenced the effectiveness of the Agency include the changing landscape of drug consumption, the emergence of new marketplaces, and new security challenges relating to drugs and evolving national policies. The Agency proactively reacted to these factors as set out above. The overall conclusion of the evaluation is that the numerous tools and mechanisms (both internal and external) used to monitor and review the Agency’s outputs and results seem to work well and be adequate to ensure accountability and an appropriate assessment of its overall performance. However, the evaluation report stresses that due to the use of different key performance indicators (KPIs) over the evaluation period and the lack of an Activity Based Budget, performance monitoring could be done only based on headline indicators. Further efforts to streamline the indicators as well as to introduce an Activity Based Budget should be undertaken to mitigate this for the future. C. Efficiency The efficiency criterion assessed the extent to which the Agency conducted its activities, achieved its objectives and delivered results at a reasonable cost, in terms of financial and human resources and administrative arrangements. Within the limitations of the available data, the evaluation could reasonably conclude that the Agency has used the available human and financial resources efficiently to deliver the outputs, outcomes and impacts set out in its 3-year strategies and work programmes. By implementing changes in its organisational and governance structure, and re-deploying available (human and financial) resources, the Agency increased its outputs and introduced new types of products and services without receiving any additional funding. The budget remained relatively flat during the evaluation period, with only some small fluctuations. Compared to 2012, the budget increased by less than 1%. Most of the budget (60% in 2017) is spent on staff costs. These costs include both operational and administrative/support functions. During the evaluation period, the share of operational staff gradually increased, from 68% in 2012 to 71% in 2017. The benefits of the Agency’s work include: a clear and thorough perspective on the drug situation in the EU; an informed debate on drug policies and strategies; the effective facilitation of a useful network of professionals in the field of drugs and drug addictions (Reitox), including the effective exchange of information and best practices; a proactive approach to new substances and emerging trends; and the promotion of scientific excellence. These benefits are difficult to quantify, but overall the outputs of the Agency are valued highly by all stakeholders at EU, national and international level. The Agency provides good value for money as the cost of producing and managing its outputs and activities compares favourably to the same measurements of other EU agencies which are similar in size and scope, as shown by the benchmarking case study. The benefits at European level could not be delivered by a single national entity, in particular the development of standard indicators applied by all members of the Reitox network, the testing of drug-related policies and measures, the early warning system for emerging threats, and the exchange of best practice. The achievement of key performance indicators and the high percentage of budget execution show that the decentralised management model adopted at the beginning of the evaluation period produced the expected results. They indicate that the current governance, organisational set-up, management systems and working methods are appropriate to its operations. There is some scope for simplifying the Agency’s administrative set-up and working methods, such as review and modernisation of information and communication technology tools, and better internal planning through the adoption of Activity Based Budgets.

The Agency has improved its online outreach, and stakeholders reported their satisfaction with the outreach and communication, considering it quite efficient. Further improvements could be made to reach national stakeholders, primarily policy-makers and practitioners. D. Coherence The coherence criterion examined the extent to which the activities and objectives of the Agency supported the key EU policy developments and complemented actions undertaken by other EU agencies and Member States, including consistency with the regulatory framework. It also considered the alignment with the common approach for decentralised agencies and the correspondence of the Agency’s 3-year strategies and work programmes with the objectives of the founding Regulation, the EU Drugs Strategy (2013-2020) and its Action Plans, and the European Agenda on Security. The desk research of the external evaluation found a high degree of coherence between the regulatory framework and the objectives and activities set out in the three-year strategies and work programmes. All of the objectives set out in the two strategy documents align with and can be traced back to the founding Regulation. The Agency’s activities contributed to a large extent to the wider EU priorities in the area of drugs policy by addressing both drug demand and drug supply issues, though to a more limited extent for the latter, as well as by providing technical support and expertise to EU institutions. However, while the founding Regulation specifically outlines in Annex I that data on emerging trends in poly-drug use should be monitored, this is not mentioned in its objectives set out in the strategy documents and remains underexplored. The objectives and activities of the Agency listed in the 3-year strategies were also found to be coherent with the EU Drugs Strategy and its Action Plans. The coherence with the European Agenda on Security is evidenced inter alia by extensive joint work with the European Union Agency for Law Enforcement Cooperation (Europol), including several joint publications such as the European Drug Market Reports and the most recent publication on “Drugs and the darknet”. While the Agency has made progress on addressing supply-side issues compared to the previous evaluation period, there is scope for further improvement in this area. The Agency developed supply-side indicators and cooperated closely with Europol on drug supply issues. However, data collection on drug supply issues should be further improved. In order to do so, additional data would have to be provided by the national focal points, the main data providers for the Agency. The Agency is well aligned with the Common Approach for decentralised agencies. The tools and mechanisms in place work well and are adequate to ensure accountability and appropriate assessment of the overall performance. Monitoring of the implementation of the 3-year strategies and work programmes and the Management Plan takes place via a series of monitoring exercises. The development of clear key performance indicators is positive; however, these have changed during the evaluation period and therefore comparisons over time and with the previous evaluation were in parts not possible. In addition, the Agency has not yet implemented some Common Approach actions, in particular those relating to Activity Based Management and Activity Based Budgets. Other missing elements to the full compliance with the Common Approach have been identified in the external evaluation and are set out in the accompanying Staff Working Document. The coherence with other EU agencies, including Europol, the European Medicines Agency (EMA), the European Centre for Disease Prevention and Control (ECDC) and the EU Agency for judicial cooperation in criminal matters (Eurojust), was facilitated by these agencies mutually reviewing each other’s work programmes. In addition, the objectives and activities of the Agency, including those set out in the “EMCDDA Strategy 2025”, complement those of the European Commission and other European agencies. Synergies exist with the European Commission in the area of drug precursors, with Europol and the European Medicines Agency in new psychoactive substances, with the European Centre for Disease Prevention and Control through joint publications and missions in the context of infectious diseases transmitted by drug use, and with the Consumers, Health, Agriculture and Food Production Executive Agency in the development of best practice in the area of public health. Synergies that could be exploited better include the relationship with the European Commission department in charge of health (DG SANTE). In general, few overlaps exist with other agencies because the Agency is unique in its position relating to the drug situation across the EU. As such, there is minimal duplication but a high likelihood of synergies. The potential to merge the Agency with another agency was not deemed desirable by stakeholders. With regard to the coherence with the drug-related policy objectives of Member States, a minority of respondents suggested that the Agency’s activities were somewhat incoherent. This was attributed to the fact that several Member States take a pan-addiction approach when setting objectives for their drug policies, while the Agency’s mandate does not cover legal substances and addictive behaviours. Therefore, the Agency cannot provide data on other addictions to the Member States to support their policies. The Agency conducts joint activities and has established synergies with Europol, the EU Agency for Law Enforcement Training (CEPOL) and Eurojust on security-related issues in the context of the EU drugs market. Its monitoring and scientific expertise has complemented Europol operations by providing the EU-wide context to drug security issues. The Agency could provide more support on security issues by developing supply side indicators and corresponding data sets. The “EMCDDA International Cooperation Framework 2018-25” aligns the activities of the Agency in this area with the “EMCDDA Strategy 2025”. However, the Agency could focus better on understanding how global drug issues and policy developments impact on the EU drug situation. The evaluation identified a significant degree of overlap for Member States in their annual reporting duties to the Agency and the United Nations Office on Drugs and Crime. It was suggested that the two agencies should work with each other to identify possible solutions to optimise and streamline Member States’ reporting processes and thus improve the quality of reported data. E. EU added value The EU added value criterion examined the benefits of the Agency compared to what could have been achieved solely at national level. More specifically, it assessed the Agency’s capacity to improve Member States’ monitoring and response to drug-related problems, and the extent to which it was a valuable source of information for its main ‘customers’. This criterion also explored the sustainability of the Agency’s activities and assessed whether, in a scenario in which the Agency was terminated, there would be viable alternatives to carry out its tasks. The Agency has an excellent reputation as a source of comprehensive, scientific and reliable drug-related information. It is seen as the main source of EU-level data and is used by national, EU and international stakeholders, especially policy-makers, researchers and practitioners working in the field. The Agency has contributed significantly to informing the policy debate at both EU and (to a lesser extent) national level. At EU level, the provision of scientific and timely information and the involvement in the design, implementation and monitoring of drug-related policies were considered crucial to ensuring that political debates and drug-related policies are evidence-based. At national level, the exchange of best practice and the production of evidence-based information on hot topics were deemed particularly relevant by national authorities. However, it appears that in some Member States the Agency is not the primary source of drug-related information, suggesting that the Agency could improve its engagement at national level and by being more proactive in involving practitioners. The evaluation found that the main added value of the Agency lies in the establishment of a drug-related data collection system, obliging its reporting countries to systematically monitor the drug phenomenon using a common methodology and common drug demand and supply indicators. This ensures a high level of uniformity in the quality and comparability of the data collected, allowing the Agency to analyse the information and produce a comprehensive overview of the drug situation across the EU. The EU-level overview is greatly valued by national authorities, which rely on it to design evidence-based policies, monitor the effectiveness of their drug-related policies and interventions and identify trends occurring elsewhere in Europe, thus improving their capacity to monitor and respond to the drug phenomenon. Other forms of support, such as the exchange of best practice, the methodological guidance provided, and its involvement in the EU Early Warning System on new psychoactive substances, are seen as adding value, compared to what Member States could do individually. The evaluation explored to what extent the activities allocated to the Agency would be less effective if carried out by other institutions at EU or national level. It is thus deemed the most effective option, as other alternatives (e.g. the Commission, Member States or international organisations) would compromise the quality of its analysis, the continuity of activities, and the common understanding of the phenomenon that allows the EU to speak with one voice in international forums.