COMMISSION STAFF WORKING DOCUMENT Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018
Date: 2019-05-14
Institutions: Directorate-General for Migration and Home Affairs, European Commission
Subjects: European Monitoring Centre for Drugs and Drug Addiction, drug addiction, operation of the Institutions, psychotropic substance

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 14.5.2019  SWD(2019) 174 final  COMMISSION STAFF WORKING DOCUMENT Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Evaluation of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 2018 {COM(2019) 228 final}  
 
 
TABLE OF CONTENTS 

GLOSSARY ............................................................................................................................... 2 

I.  

INTRODUCTION .............................................................................................................. 3 

A.  PURPOSE OF THE EVALUATION ............................................................................. 3 

B.  SCOPE OF THE EVALUATION ................................................................................... 4 

II.  BACKGROUND TO THE INITIATIVE ........................................................................... 5 

A.  DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES ........................ 5 

B.  BASELINE AND INTERVENTION LOGIC ................................................................ 7 

III. 

IMPLEMENTATION / STATE OF PLAY .................................................................... 8 

IV.  EVALUATION QUESTIONS ...................................................................................... 12 

V.   METHOD ...................................................................................................................... 15 

A.  SHORT DESCRIPTION OF METHODOLOGY ......................................................... 15 

B.  LIMITATIONS AND ROBUSTNESS OF FINDINGS ............................................... 16 

VI.   ANSWERS TO THE EVALUATION QUESTIONS .................................................. 17 

A.  RELEVANCE ............................................................................................................... 17 

B.  EFFECTIVENESS ........................................................................................................ 20 

C.  EFFICIENCY ................................................................................................................ 27 

D.  COHERENCE ............................................................................................................... 29 

E.  EU ADDED VALUE .................................................................................................... 35 

VII.  CONCLUSIONS AND LESSONS LEARNED ........................................................... 38 

VIII.   ANNEXES .................................................................................................................... 41 

ANNEX I – PROCEDURAL INFORMATION .................................................................. 41 

ANNEX II – METHODS, SOURCES AND LIMITATIONS ............................................. 43 

ANNEX III – STAKEHOLDER CONSULTATION .......................................................... 51 

ANNEX IV – INTERVENTION LOGIC ............................................................................ 56 

ANNEX V – SUMMARY OF THE MAIN AREAS AND GOALS IN THE 2013-2015 
AND 2016-2018 STRATEGIES AND WORK PROGRAMMES ...................................... 57 

1 

 
 
 
 
 
 
 
GLOSSARY 

Abbreviation 
AWP 
CEPOL 
DG GROW 

DG HOME 
DG SANTE 
DG TAXUD 
EASO 
EBCGA 
EC 
ECDC 
EMA 
EMCDDA 
EWS 
ESPAD 
EU-OSHA 
Eurojust 
Europol 
FRA 
HDG 
HIV 
JHA 
JRC 
MAOC(N) 
MFF 
NPS 
Reitox  
SWP 
UNODC  
WHO 

Definition 
Annual Work Programme 
European Union Agency for Law Enforcement Training 
Directorate-General for Internal Market, Industry, Entrepreneurship and 
SMEs 
Directorate-General for Migration and Home Affairs 
Directorate-General for Health and Food Safety 
Directorate-General for Taxation and Customs Union 
European Asylum Support Office 
European Border and Coast Guard Agency (also known as Frontex) 
European Commission 
European Centre for Disease Prevention and Control  
European Medicines Agency  
European Monitoring Centre for Drugs and Drug Addiction  
EU Early Warning System on new psychoactive substances  
European School Survey Project on Alcohol and Other Drugs 
European Agency for Safety and Health at Work 
European Union Agency for judicial cooperation in criminal matters 
European Union Agency for Law Enforcement Cooperation 
Fundamental Rights Agency 
Horizontal Working Party on Drugs  
Human immunodeficiency virus 
Justice and Home Affairs 
Joint Research Centre 
Maritime Analysis and Operation Centre – Narcotics  
Multiannual Financial Framework 
New psychoactive substances  
European Information Network on Drugs and Drug Addiction Network  
(3-year) Strategy and Work Programmes 
United Nations Office on Drugs and Crime 
World Health Organisation  

2 

 
 
 
 
 
 
 
 
I.  

A. 

INTRODUCTION 

PURPOSE OF THE EVALUATION 

The European Monitoring Centre for Drugs and Drugs Addiction (EMCDDA; referred to in 
this document as “the Agency”) was established in 1993 with the mission to provide the EU 
and  its  Member  States  with  factual,  objective,  reliable  and  comparable  information  at 
European  level  on  drugs  and  drug  addiction  and  their  consequences.  Its  core  tasks  are  to 
collect and analyse existing data, improve data comparison methods, disseminate data and to 
cooperate  with  European  and  international  bodies  and  organisations  as  well  as  with  third 
countries. 

Regulation  (EC)  1920/20061  (referred  to  as  “founding  Regulation”),  together  with  the 
Financial  Framework  Regulation2,  the  Common  Approach  on  Decentralised  Agencies3  and 
the principles of sound and efficient management require that the Agency be evaluated on a 
regular  basis.  Article 23  of  the  founding  Regulation  specifically  stipulates  that  the 
Commission  should  initiate  an  external  evaluation  every  six  years  to  coincide  with  the 
completion  of  two  of  the  Agency’s  three-year  work  programmes.  The  evaluation  has  to 
include  the  Reitox  system  (the  European  Information  Network  on  Drugs  and  Drug 
Addiction).4  The  evaluation  report  has  to  be  transmitted  to  the  European  Parliament,  the 
Council and the Agency’s Management Board. 

The previous external evaluation covered the period 2007-20125 and was completed in mid-
2012  so  that  its  findings  would  be  available  ahead  of  the  following  multiannual  work 
programme  of  the  Agency.  The  current  six-year  programming  phase  covers  the  last  two 
multiannual  work  programmes,  namely  2013-2015  and  2016-2018,  and  the  findings  of  the 
evaluation will feed the Agency's future multiannual and annual  work programmes and will 
contribute to the shaping of the future EU Drugs Strategy (the current one ends in 2020). 

The evaluation has two main objectives: 

 

to assess the relevance, effectiveness, efficiency, coherence and EU added value of the 
Agency's  performance  since  2013  in  terms  of  the  implementation  of  the  EU  Drugs 

1  

2  

3  

4  

5  

Regulation (EC) No 1920/2006 of the European Parliament and of  the Council of 12 December 2006, 
OJ  L  376,  27.12.2006,  p.  1. This  Regulation  repealed  Council  Regulation  (EEC)  No  302/93  and  was 
amended most recently by Regulation (EU) 2017/2101 of the European Parliament and of the Council 
of 15 November 2017 amending  Regulation (EC) No 1920/2006 as regards information exchange on, 
and  an  early  warning  system  and  risk  assessment  procedure  for,  new  psychoactive  substances,  OJ  L 
305, 21.11.2017, p.1. 
Commission  Delegated  Regulation  (EU)  No  1271/2013  of  30  September  2013  on  the  framework 
financial regulation for the bodies referred to in Article 208 of Regulation (EU, Euratom) No 966/2012 
of the European Parliament and of the Council, OJ L 328, 7.12.2013, p.42. 
For  further 
union/sites/europaeu/files/docs/body/joint_statement_and_common_approach_2012_en.pdf.  
See 
Regulation. 
the 
http://www.emcdda.europa.eu/about/partners/reitox-network.  
External  Evaluation  of  the  European  Monitoring  Centre  for  Drugs  and  Drugs  Addiction,  Centre  for 
Strategy 
report, 
final 
https://publications.europa.eu/en/publication-detail/-/publication/7d189e3b-f767-460b-b748-
acf44d2daf9a/language-en/format-PDF/source-74343049.  

the  "common  approach",  please  see  https://europa.eu/european-

information  on 

information 

Evaluation 

founding 

Article 5 

Services 

(CSES), 

2012, 

more 

June 

and 

For 

see 

11 

of 

3 

 
 
                                                           
Strategy (2013-2020)6 and its Action Plans7, of the European Agenda on Security8 and 
of its tasks laid down in its founding Regulation (Article 2); and 

 

to  propose  concrete  and  useful  recommendations  to  the  Agency  to  better  respond  to 
the challenges posed by the constantly changing environment against the background 
of  the  current  authorised  financial  and  human  resources  for  the  Agency,  bearing  in 
mind the current and future EU budgetary constraints. 

The evaluation assesses also the compliance of the Agency with the objectives and provisions 
of the Common Approach on Decentralised Agencies.9 

B. 

SCOPE OF THE EVALUATION 

As foreseen in Article 23 of the founding Regulation and based on the Terms of Reference for 
the external evaluation, the contractors assessed in the evaluation study:  

 

 

the implementation of the last two three-year work programmes, more precisely from 
January 2013 until as late as possible in 2018; in this context, the evaluation examined 
in  particular  the  way  and  extent  to  which  the  Agency  effectively  contributed  to  the 
implementation of the strategic documents referred to above and fulfilled its tasks laid 
down in its founding Regulation;  

the  European  Information  Network  on  Drugs  and  Drug  Addiction  (Reitox),  whose 
members  are  national  institutions  or  agencies  responsible  for  data  collection  and 
reporting  on  drugs  and  drug  addiction,  in  particular  to  what  extent  the  network 
contributed to the overall performance of the Agency; and  

  whether the provisions and the scope of the founding Regulation  are still adapted to 
current  needs  given  the  evolved  context  in  which  the  Agency  now  operates  and  the 
new challenges it has to face. 

The  evaluation  also  had  to  verify  to  what  extent the  provisions  establishing  the  Agency  are 
aligned with the Common Approach on Decentralised Agencies, looking in particular into the 
Agency's  governance,  internal  structures  and  procedures  and  applicable  administrative  and 
financial  rules.  Another  intention  of  the  evaluation  was  to  provide  solid  grounds  for  any 
possible decision concerning the Agency’s future mandate. 

Based on these principles,  the external  evaluation  study  assessed the five evaluation  criteria 
(relevance, effectiveness, efficiency, coherence and EU added value) by taking into account 
the following scope: 

6  
7  

8  
9  

OJ C  402, 29.12.2012, p.1. 
EU Action Plan on Drugs 2013-2016, OJ C 351, 30.11.2013, p.1; EU Action Plan on Drugs 2017-2020, 
OJ C 215, 5.7.2017, p.21. 
COM(2015) 185 final. 
The  Common  Approach  sets  out  agreed  principles  on  the  following  issues  related  to  decentralised 
agencies: role and position of agencies in the EU’s institutional landscape; structure and governance of 
agencies; operation of agencies; programming of activities and resources; accountability, controls and 
transparency; and relations with stakeholders. 

4 

 
 
 
                                                           
  Material scope: the evaluation covers the two pillars of the Agency’s work, i.e. health 
and  security.  It  looks  into  its  governance  and  administration,  the  organisational 
structure,  operations,  funding  and  resourcing,  its  information  management  and  the 
work of the Reitox network, cooperation with other relevant EU agencies (such as the 
Justice  and  Home  Affairs  agencies10,  the  European  Medicines  Agency  (EMA),  the 
European  Centre  for  Disease  Prevention  and  Control  (ECDC))  and  international 
organisations  (such  as  the  World  Health  Organisation  (WHO),  the  United  Nations 
Office  on  Drugs  and  Crime  (UNODC),  etc.),  partnerships  with  third  countries,  and 
communication and dissemination of research; 

  Geographical scope: the evaluation covers the countries reporting data to the Agency, 
i.e.  the  EU  Member  States  as  well  as  Norway  and  Turkey,  and  third  countries  with 
which the Agency has closer relations due to the conclusion of working arrangements, 
cooperation agreements or similar; 

  Temporal scope: the evaluation covers the activities carried out from 1 January 2013 
until  30  June  201811  to  coincide  with  the  completion  of  two  consecutive  3-year 
strategies and work programmes of the Agency. 

The evaluation was the fourth evaluation of the Agency. The current evaluation built on the 
results of the previous external evaluation. 

II. 

A. 

BACKGROUND TO THE INITIATIVE 

DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES 

The market for illicit drugs is the most dynamic criminal market in the EU, which is able to 
adapt  rapidly  in  response  to  drug  control  measures.  Its  socio-economic,  demographic  and 
international context has evolved considerably over the last years. The human and social costs 
of  drug  addiction  are  very  high  and  it  generates  costs  for  public  health  (drug  prevention, 
healthcare and treatment), public safety, the environment and labour productivity.  

The drugs market value is estimated at EUR 24 billion12 and is facilitated by the emergence of 
new internet technologies. Online marketplaces have emerged and are spreading. The number 
of New Psychoactive Substances is on the rise. An estimate of at least 7929 overdose deaths 
happened in 2016. There has been a shift from drug use to poly-drug use (where people use 
two or more psychoactive drugs or alcohol in combination) and on-going debates and shifting 
policies across the world regarding medical and/or recreational use of cannabis. The issue of 
misuse of prescription drugs is becoming more and more important also in Europe. Purity and 

10  

11  

12  

The  Justice  and  Home  Affairs  agencies  are:  CEPOL  (European  Union  Agency  for  Law  Enforcement 
Training), EASO (European Asylum Support Office), EIGE (European Institute for Gender Equality), 
European Monitoring Centre for Drugs and Drug Addiction (EMCDDA), eu-LISA (European Agency 
for the operational management of large-scale IT systems in the area of freedom, security and justice), 
Eurojust,  Europol  (European  Union  Agency  for  Law  Enforcement  Cooperation),  FRA  (Fundamental 
Rights Agency) and EBCGA (European Border and Coast Guard Agency (also known as Frontex)). 
The cut-off date for the work of the contractor was set for 30 June 2018, i.e. the date of submission of 
2017; 
Report 
the 
http://www.emcdda.europa.eu/publications/gra/2017_en.  
EU  Drug  Markets  Report  2016,  http://www.emcdda.europa.eu/start/2016/drug-markets.  An  update  of 
this estimate will be provided in the EU Drug Markets Report 2019, which will be published in 2019. 

Activities 

Agency’s 

General 

on 

5 

 
 
 
                                                           
availability on the market of conventional drugs (such as cocaine and heroin) are increasing.13 
In addition, national drug policies developed further over time and many Member States' new 
strategies go beyond only covering illicit drugs to cover also other addictions, such as alcohol, 
tobacco, gambling, etc.14 

In order to be able to understand better the nature of Europe’s drug problem and to respond 
better to them, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 
was  established  in  199315  as  a  decentralised  agency  to  provide  independent,  science-based 
information on these issues.  

Its  objective,  pursuant  to  Article  1  of  the  founding  Regulation,  is  "to  provide  in  the  areas 
referred to in Article 3, the Community and its Member States with factual, objective, reliable 
and  comparable  information  at  European  level  concerning  drugs  and  drug  addiction  and 
their consequences". Annex I, to which Article 3 of the founding Regulation refers, reads as 
follows: 

"A. 

The work of the Centre shall be carried out with due regard to the respective powers 
of  the  Community  and  its  Member  States  in  the  area  of  drugs,  as  those  powers  are 
defined by the Treaty. It shall cover the various facets of the drugs and drug addiction 
phenomenon, and the solutions applied. In doing so, the Centre shall be guided by the 
Drugs Strategies and Action Plans adopted by the European Union. 

The Centre shall focus on the following priority areas: 

1)  monitoring  the  state  of  the  drugs  problem,  in  particular  using  epidemiological 
or  other  indicators,  and  monitoring  emerging  trends,  in  particular  those 
involving poly-drug use; 

2)   monitoring 

the  solutions  applied 

to  drug-related  problems;  providing 
information on best practices in the Member States and facilitating the exchange 
of such practices among them; 

3) 

4) 

assessing  the  risks  of  new  psychoactive  substances  and  maintaining  a  rapid 
information system with regard to their use and also regarding new methods of 
using existing psychoactive substances; 

developing tools and instruments to help Member States to monitor and evaluate 
their  national  policies  and  the  Commission  to  monitor  and  evaluate  European 
Union policies. 

13  

14  

15  

For details on the evolving drug markets see in particular the annual European Drug Reports, which set 
out  trends  and  developments  and  thereby  present  a  top-level  overview  of  the  drug  phenomenon  in 
Europe, covering drug supply, use and public health problems as well as drug policy and responses. The 
most  recent  report  is 
the  one  for  2018:  http://www.emcdda.europa.eu/publications/edr/trends-
developments/2018_en.  
For  example,  France's  addiction  strategy  covers  illicit  drugs  and  NPS,  alcohol,  tobacco,  medicines, 
behavioural  addictions  (e.g.  gambling)  and  doping;  Austria's  addiction  strategy  covers  all  these 
addictions except doping; Portugal's addiction strategy covers illicit drugs and NPS, alcohol, medicines 
and behavioural addictions. An overview (state of play 2016) is available in the EMCDDA Papers: New 
developments 
Europe, 
http://www.emcdda.europa.eu/system/files/publications/6402/20175662_TDAU17002ENN_PDF.pdf; 
see also Section VI.A. below.  
Council  Regulation  (EEC)  No  302/93  of  8  February  1993  on  the  establishment  of  a  European 
Monitoring Centre for Drugs and Drug Addiction, OJ L 36, 12.2.1993, p. 1. 

strategies 

national 

drug 

in 

in 

6 

 
 
                                                           
B. 

The  Commission  shall  make  available  to  the  Centre,  for  dissemination,  the 
information and statistical data which it possesses pursuant to its powers." 

Based on the objectives, tasks and priority areas of work set out in the founding Regulation, 
the  multi-annual  strategies  and  work  programmes  detail  the  main  areas,  goals  and  expected 
results of the work of the Agency further. For the period relevant for this evaluation, these are 
set  out  in  particular  in  the  “EMCDDA  2013-15  strategy”16,  the  “Strategy  and  work 
programme 2016-18”17 and the “EMCDDA Strategy 2025”18, with further details included in 
the annual work programmes19. These documents formed and form the basis of the Agency’s 
operating framework over the evaluation period.  

B. 

BASELINE AND INTERVENTION LOGIC  

When  the  Agency  was  established,  no  detailed  intervention  logic  was  prepared.  It  was 
developed for this evaluation by the contractor, also taking into account the intervention logic 
prepared for the previous external evaluation. A high-level overview of the intervention logic 
for this evaluation is included as Annex IV in this Staff Working Document.  

The high-level principle of the Agency is the aim to contribute to a healthier and more secure 
Europe and to meet the information needs of policy-makers and practitioners at EU, national 
and international level (general objective). As the Agency is subject to a complex hierarchy of 
specific objectives, based on the founding Regulation and the different  strategic documents, 
the  intervention  logic  illustrates  the  core  objectives  and  priorities  stemming  from  these 
documents. 

The Agency’s inputs are its financial and human resources, while its core tasks are set out in 
the founding Regulation. 

The following three levels of expected results can be identified: 

  Outputs are the immediate products and services. These include the different periodic 
reports, i.e. annual and multiannual reports, such as the European Drug Report (EDR), 
the  EU  Drugs  Market  Report  (EDMR)  and  the  European  Drug  Responses  Report 
(EDRR);  strategic  and  situational  analysis  and  threat  assessment  analysis;  statistics, 
Country  Drug  Reports,  thematic  and  technical  reports  and  other  written  outputs; 
training and capacity building, etc. 

  Outcomes  are  second-level  intermediate  results  stemming  from  the  outputs.  These 
include  facilitated 
informed  policy-makers  and 
practitioners,  transferring  know-how  and  improved  cooperation  and  coherence  with 
partners. 

information  exchange,  better 

16  
17  
18 

19  

http://www.emcdda.europa.eu/system/files/publications/676/wp2013-15_393821.pdf.  
http://www.emcdda.europa.eu/system/files/publications/2095/TDAX16001ENN_.pdf.  

 http://www.emcdda.europa.eu/system/files/publications/4273/2017.1998_EMCDDA_STRATEGY_202
5_web-1.pdf.  
http://www.emcdda.europa.eu/publications-seriestype/work-programmes-and-strategies_en.  

7 

 
 
 
                                                           
 

Impacts are third-level long-term results, which are influenced by factors outside the 
control of the Agency. Its activities ultimately aim to support the objectives of the EU 
drugs policy, including the EU Drugs Strategy and its Action Plans, in reducing both 
demand  and  supply  of  drugs  and  thus  contribute  to  a  reduction  in  drug  use  and 
minimising the threat of this phenomenon. 

The  general  baseline  for  the  establishment  of  the  Agency  was  that  no  factual,  objective, 
reliable and comparable information at European level on drugs and drug addiction and their 
consequences  was  available.  No  concrete  or  detailed  baseline  was  established  at  that  time. 
The contractors tried to re-establish baselines by using – as much as possible – the results of 
the  previous  external  evaluation.  However,  this  was  possible  only  to  a  limited  extent  as  the 
baselines were lacking for most elements.20  

III. 

IMPLEMENTATION / STATE OF PLAY 

The  European  Monitoring  Centre  for  Drugs  and  Drug  Addiction  (EMCDDA)  was  set  up  in 
1993 as one of the EU’s decentralised agencies. It started working fully in 1995 and is located 
in Lisbon, Portugal.  

Structure and management 

The Agency’s organisation consists of  a Directorate, two statutory  bodies (the Management 
Board and the Scientific Committee) to advise and assist in the decision making process, and 
various working units (which comprises the majority of the staff) to deliver its output. 

The Agency has approximately 100 staff members (state of play June 2018) and is managed 
by  a  Director  who  is  appointed  for  a  5-year  period  by  the  Management  Board  based  on  a 
proposal by the European Commission.21 The Director is ultimately responsible for meeting 
the  strategic  objectives  and  implementing  the  Work  Programmes,  budget  frameworks  and 
Management Board decisions. He is accountable for his activities to the Management Board.  

The internal organisation looks currently as follows22: 

20  

21  
22  

For the limitations of  the  finding of the external evaluation, see Section V.B. and  Annex II(2) of this 
Staff Working Document. 
See Article 11 of the founding Regulation.. 
Source: EMCDDA. 

8 

 
 
 
                                                           
The Management Board is  the main decision-making body.23  It  meets  at  least  once a  year24 
and  consists  of  one  representative  from  each  EU  Member  State,  Norway  and  Turkey,  two 
representatives  from  the  European  Commission  and  two  independent  experts  particularly 
knowledgeable  in  the  field  of  drugs  designated  by  the  European  Parliament.25  Several 
observers are invited to participate in the Management Board meetings. The Chairperson and 
the Vice-Chairperson of the Management Board are elected for a (once renewable) three-year 
period. 

An Executive Committee assists the Management Board.26 The Executive Committee consists 
of the Chairperson and the Vice-Chairperson of the Management Board, two other members 
of the Management Board representing the Member States and appointed by the Management 
Board  and  two  Commission  representatives.  The  Executive  Committee  prepares  the 
Management  Board  meetings  and  decides  on  those  matters  provided  for  in  the  Financial 
Framework  Regulation  which  are  not  reserved  to  the  Management  Board  by  the  founding 
Regulation. Although not provided for in the founding Regulation, the Agency also set up a 
Budget Committee to ensure better monitoring and follow-up of financial issues. 

The Scientific Committee27 advises the Board and the Director on the scientific aspects of the 
work  and  contributes  to  the  risk  assessment  of  new  psychoactive  substances  (NPS).  The 
Scientific  Committee  consists  of  at  most  fifteen  well-known  scientists  appointed  in  view  of 
their  scientific  excellence  and  their  independence  by  the  Management  Board,  following  the 
publication of a call for expression of interest in the Official Journal of the European Union. 
The members of the Scientific Committee are appointed in their personal capacity. 

23  
24  

25  
26  
27  

See Article 9 of the founding Regulation. 
In practice, two meetings took place every year, one in June and one in December, during the evaluation 
period. 
For the constitution of the Management Board, see http://www.emcdda.europa.eu/about/mb.  
See Article 10 of the founding Regulation. 
See Article 13 of the founding Regulation. 

9 

 
 
 
                                                           
Reitox28 is the European information network on drugs and drug addiction. Members of the 
Reitox network are designated national institutions or agencies responsible for data collection 
and  reporting  on  drugs  and  drug  addiction  (‘national  focal  points’  or  ‘national  drug 
observatories’). The Reitox network links national drug information systems and is the main 
way in which the Agency exchanges data and methodological information on drugs and drug 
addiction  in  Europe  and  thereby  directly  contributes  to  the  core  task  of  collecting  and 
reporting consistent, harmonised and standardised information. The three core functions of a 
national  focal  point  are:  data  collection  and  monitoring;  analysis  and  interpretation  of  data 
collected; and reporting and dissemination of the results at national level. 

Budget 

The annual budget over the evaluation period was around EUR 16 Mio. The main part of the 
budget comes from the European Union subsidy. Norway and Turkey, both members without 
voting rights29, also contribute to its budget.  

Main areas of work 

The  key  areas  of  activity  of  the  Agency  are  in  accordance  with  Article 2  of  its  founding 
Regulation:  collection  and  analysis  of  existing  data;  improvement  of  data  comparison 
methods;  dissemination  of  data;  cooperation  with  European  and  international  bodies  and 
organisations as well as with third countries; information obligations; and  – as added by the 
2017 amendment to the founding Regulation30 – exchange of information on, early warning 
system for, and risk assessment of, new psychoactive substances. 

During  the  evaluation  period,  the  Agency  published  two  3-year  strategies  and  work 
programmes:  the  “EMCDDA  2013-15  strategy”  and  the  “Strategy  and  work  programme 
2016-18”. For the first time, a 10-year strategy was also published, the “EMCDDA Strategy 
2025”. These  strategic goals  were transformed into  more concreted  (planned) outputs in  the 
annual work programmes. They provided detailed information about the activities and outputs 
of the Agency. The coherence of the main areas of work and goals set out in these documents 
is presented below31. 

The  2013-2015  Strategy  identified  12  main  areas,  each  with  a  goal  and  further  specific 
objectives  and  ‘primary  interventions’.  The  2016-2018  Strategy  and  Work  Programme  was 
built  around  six  strategic  action  areas,  composed  of  three  key  action  areas:  communicating 
evidence  and  knowledge  exchange;  early  warning  and  threat  assessment;  and  situation, 
responses  and  trend  analysis;  three  cross-cutting  action  areas:  information  collection  and 
management;  quality  assurance;  and  cooperation  with  partners;  and  two  corporate  areas: 
governance,  and  administration  and  ICT.  The  main  areas  and  goals  are  presented  in  more 
detail in a table in Annex V.  

The “EMCDDA Strategy  2025” attributed the  activities and the tasks of the Agency to two 
main pillars, health and security. In addition, it defined four business drivers, i.e. institutional, 

28  

29  
30  

31  

See  Article  5  of  the  founding  Regulation.  The  abbreviation  ‘Reitox’  stands  for  the  French  ‘Réseau 
Européen d’Information sur les Drogues et les Toxicomanies’. 
See Article 21 of the founding Regulation. 
Regulation  (EU)  2017/2101  of  the  European  Parliament  and  of  the  Council  of  15  November  2017 
amending  Regulation  (EC)  No  1920/2006  as  regards  information  exchange  on,  and  an  early  warning 
system and risk assessment procedure for, new psychoactive substances, OJ L 305, 21.11.2017, p.1 
See Section VI.D. 

10 

 
 
                                                           
partnership,  scientific  capacity  and  management.  The  strategic  objectives  of  the  two  pillars 
reflect the priority areas set out by the founding Regulation. 

While the Agency is primarily European in focus, it also works with partners in other world 
regions,  exchanging  information  and  expertise.  The  Agency  cooperates  with  candidate  and 
potential candidate countries as part of their accession process to the EU. When it comes to 
neighbouring  countries  of  the  European  Neighbourhood  Policy  and  with  other  non-EU 
countries,  cooperation  takes  place  based  on  bilateral  agreements32  or  in  the  context  of  EU-
funded projects33. Cooperation ranges from implementation of technical assistance projects to 
ad-hoc  training  or  consultative  support.  Collaboration  with  European  and  international 
organisations in the drugs field is central to its work as a means of enhancing understanding 
of the global drugs phenomenon. 

Main achievements 

The achievements of the Agency are set out in detail and on an annual basis in the General 
Report of Activities.34 Some key achievements of the Agency in relation to the priority areas 
set out in the founding Regulation are summarised below. This is only a small snapshot of the 
work  of  the  Agency  and  more  detailed  outputs  and  achievements  are  presented  throughout 
this Staff Working Document.  

The  main  achievement  of  the  establishment  of  the  Agency  is  the  availability  of  factual, 
objective, reliable and comparable information at European level concerning drugs and drug 
addiction. This is crucial to allow the development of an integrated, balanced and evidence-
based approach to drugs policy.  

Some examples of main achievements: When it  comes to  the monitoring of the state of the 
drugs  problem,  the  solutions  applied  to  drug-related  problems  as  well  as  of  European  and 
national  policies,  the  Agency  has  published  almost  400  scientific  and  institutional  reports 
from  2013  to  2017.  Among  its  flagship  publications  are  the  annual  European  Drug  Report, 
including the country reports, the EU Drug Markets Report, published in 2013 and 2016, and 
“Health  and social responses to  drug problems  – A European Guide”, which was published 
for  the  first  time  in  2017  but  will  be  updated  every  3  years.35  The  Agency  facilitated  the 
exchange of best practices and drug-related information through the participation of its staff in 
more than 1500 events (contributions to conferences, policy forums, etc.) and authoring more 
than 150 scientific articles during the same period. The Agency organised numerous meetings 
of  its  stakeholders,  mainly  in  Lisbon,  which  also  allowed  for  the  sharing  of  best  practice. 
When  it  comes  to  new  psychoactive  substances,  the  maintenance  of  the  Early  Warning 
System,  which  is  available  24/7,  is  crucial  for  the  assessment  of  their  risks.  The  Agency 
carried out 26 joint reports and risk assessments from 2013 to 2017.  

32  

33  

34  
35  

Working  arrangements  are  currently  in  place  with  Russian  Federation,  Ukraine,  Moldova,  Israel, 
Armenia, Georgia, Switzerland and Albania. 
See for example the participation of the Agency in the Cooperation Programme between Latin America, 
the  Caribbean  and  the  EU  on  Drugs  Policies  (COPOLAD  II)  or  the  Central  Asia  Drug  Action 
Programme (CADAP). 
www.emcdda.europa.eu/publications-database_en?f%5B0%5D=field_series_type%3A599.  
All publications of the Agency can be found here: www.emcdda.europa.eu/publications_en.  

11 

 
 
 
                                                           
IV. 

EVALUATION QUESTIONS 

In accordance with the Commission's Better Regulation Guidelines, the evaluation looked at 
the effectiveness, efficiency, relevance, coherence and the EU added value of the work of the 
European Monitoring Centre for Drugs and Drug Addiction.  

The following questions were addressed during the evaluation:36 

Relevance 

1.  To what extent have the EMCDDA work programmes covering the 2013-2018 period 
addressed the objectives, tasks and priorities set out in the EMCDDA recast regulation 
as  well  as  those  of  the  EU  Drugs  Strategy  and  its  Action  Plans  and  those  stemming 
from the European Agenda on Security? 

2.  To  what  extent  have  the  outcomes  of  the  EMCDDA  work  programmes  covering  the 
2013-2018 period responded to the needs of its  multiple stakeholders (policy-makers, 
scientific community, practitioners and the general public)? 

3.  To what extent are the EMCDDA activities contributing to the EU's priorities? 

4.  How well adapted is the EMCDDA to subsequent economic, technological, scientific, 

social, political or environmental advances?   

Effectiveness 

5.  To what  extent has the EMCDDA met its core  objective as required  in  its  regulatory 
framework  to  provide  the  EU  with  factual,  objective,  reliable  and  comparable 
information? 

6.  To what  extent has the EMCDDA achieved the  objectives of its two three  year work 

programmes 2013-2015 and 2016-2018? 

7.  To what extent have the Reitox national focal points delivered the data and information 
required to meet the objectives of the aforementioned EMCDDA work programmes? 

8.  To what extent have the changes in the EMCDDA governance structure resulting from 
the  implementation  of  the  EMCDDA  Strategy  2025  and  the  recent  internal  re-
organisation impacted on the effectiveness of the EMCDDA? 

9.  To what  extent are the EMCDDA tools to monitor and review its outputs and results 
adequate for ensuring accountability and an appropriate assessment of performance? 

10. To what extent are the internal and external mechanisms for monitoring, reporting and 
evaluating the agency adequate for ensuring accountability and appropriate assessment 
of the overall performance? 

11. To  what  extent  and  how  have  external  factors  influenced  the  effectiveness  of  the 

agency? 

36  

The questions are copied from the Terms of Reference provided to the potential contractors. 

12 

 
 
 
                                                           
Efficiency 

12. Which were the costs and the benefits of the EMCDDA activities?  

13. To what extent has the EMCDDA efficiently spent its resources (human and financial) 
to achieve the objectives set out in its work programmes during the 2013-2018 period? 
To what extent are available resources adequate to these objectives? 

14. Is the EMCDDA providing value for money? 

15. To  what  extent  have  the  EMCDDA  governance,  organisational  set-up,  management 

systems and working methods been conducive to the efficiency of its operations? 

16. Is  there  scope  for  simplifying  the  administrative  set-up  and  working  methods  in  the 

context of the current administrative and financial regulations? 

17. To what extent and how have external factors influenced the efficiency of the agency? 

Coherence 

18. To  what  extent  are  the  objectives  and  activities  of  the  EMCDDA  for  the  2013-2018 

period coherent with the objectives set out in its regulatory framework? 

19. To what extent are the objectives and activities of the EMCDDA coherent with the EU 
policy developments as defined in the documents presented in "EU policy documents" 
section? 

20. To  what  extent  are  the  objectives  and  activities  of  the  EMCDDA  coherent  with  and 
complementary  to  the  drugs-related  objectives  and  activities  of  the  Commission,  of 
other  EU  agencies,  namely  Europol,  European  Centre  for  the  Prevention  of  Disease 
Control and the European Medicines Agency, and of the Member States? What are the 
overlaps  and  potential  synergies  and  the  option  of  merging  with  other  EU  agencies 
carrying out similar or identical activities? 

EU Added Value 

21. To what extent have the EMCDDA activities and outputs helped to improve the ability 
of  Member  States  to  monitor  and  respond  to  drug  problems  compared  to  what  they 
could do at national level? 

22. To  what  extent  have  the  EMCDDA  activities  provided  a  European  level  information 

resource for informing the policy debate on drug issues? 

23. To what extent has the EMCDDA been more effective in achieving its objectives in the 
2013-2018  period  compared  to  existing  or  alternative  options  of  implementing  drugs 
policy as set in the strategic documents referred to under point 2 (e.g. by the Member 
States, through the Commission services themselves, an executive agency, etc.)? 

24. To what extent are the outcomes of the EMCDDA activities sustainable?  

25. What would be the most likely consequences of the termination of the agency? 

13 

 
 
Lessons learnt37 

The evaluation also assessed how the EMCDDA could assist policymakers and practitioners 
even more in planning and delivering policies and programs that contribute to a healthier and 
legal,  demographic 
more  secure  Europe, 
international  context.  In  particular,  the  following  questions  were  addressed  during  the 
evaluation: 

in  an  evolving  political,  socio-economic, 

26. On the health aspects, to what extent can the scope of monitoring and identifying best 
practices  be  broadened  in  the  field  of  prevention  and  care  of  addictive  behaviours 
(illicit  and  licit  substances  including  alcohol,  but  also  addictions  without  substances, 
like gambling)?  

27. In terms of synergies with other JHA agencies, to what extent and how can the agency 
be more to  the forefront  of the security issues and inform  more on threats  and trends 
concerning  links  between  drugs  trafficking  and  organised  crime,  drug-related  crime, 
drugs  markets,  in  particular  the  way  drugs  can  be  sold  and  bought  through  Internet, 
social networks and encryption software? To what extent and how synergies with other 
JHA agencies and in particular with Europol can be increased in this field? 

28. How  could  the  EMCDDA  work  better  with  International  organisation,  such  as  the 
World  Health  Organisation  and  the  United  Nations  Office  on  Drugs  and  Crime 
(UNODC)? At UN level, as the EU strongly advocates for evidence-based policies and 
has the most performant regional data collection system, to  what  extent and how can 
EMCDDA  promote  its  expertise  in  the  revision  process  of  the  UN  data  collection 
system, including the possibility for the EMCDDA to respond for the EU to certain UN 
questionnaires and enquiries? 

29. At international level, to what extent can the international activities of the EMCDDA 
be  optimised  in  terms  of  impact?  Are  the  activities  carried  out  by  EMCDDA  outside 
the EU compatible with the EU priorities in the external action? Should EMCDDA do 
more or less in the international sphere?  

30. In  terms  of  communication/dissemination,  to  what  extent  and  how  can  EMCDDA 
spread  more  widely  its  outputs,  thus  reaching  out  a  higher  number  of  policy-makers 
and professionals? 

31. What  conclusions  and  recommendations  can  be  drawn  from  the  evaluation  of  the 
EMCDDA and its work programmes relating to the 2013-2018 period, with a view to 
supporting  the  post-2020  EMCDDA  programming  cycles,  future  Commission's 
proposals  for  EU  drugs  policy  initiatives  and  the  possible  revision  of  the  EMCDDA 
mandate? 

37  

The questions listed in this section were attributed by the contractor to the 5 better regulation principles, 
i.e.  to  the  criterion  where  the  question  suits  best.  Q26  was  addressed  under  “Relevance”;  Q29  under 
“Effectiveness”; Q30 under “Efficiency”; and Q27 and Q 28 under “Coherence”. Q31 was addressed by 
the key findings under each criterion and by the overall recommendations. It should also be noted that 
some  of  the  questions  were  slightly  modified,  in  agreement  with  the  Commission,  to  better  address 
some issues. 

14 

 
 
 
                                                           
V.   METHOD 

A. 

SHORT DESCRIPTION OF METHODOLOGY 

The external  evaluation  study was carried out  by  ICF Consulting Services  Limited (ICF) in 
cooperation  with  the  Centre  for  the  Study  of  Democracy  (CSD)  and  Optimity  Advisors, 
following  a  call  for  services  under  a  DG  HOME  framework  contract.  The  evaluation  was 
conducted  through  a  mixed  methods  approach  and  was  informed  by  the  triangulation  of  a 
variety  of  sources.  For  more  details  on  the  methods  and  the  stakeholder  consultation,  see 
Annexes II and III of this Staff Working Document. 

The evaluation was informed by a thorough desk research of documentation being available 
online  or  provided  directly  (by  e-mail  and/or  on  a  dedicated  extranet  site  on  the  Agency’s 
website  to  which  only  the  contractor  and  the  responsible  DG  HOME  desk  officer  had 
access).38  

The  external  evaluators  had  interviews  with  133  stakeholders39,  which  were  accompanied  – 
where appropriate – by questionnaires.  

Targeted online surveys took place for the Agency staff, on the one hand, and the scientific 
community and civil society, on the other hand. 55 and 24 replies, respectively, were gathered 
through the two surveys.  

A  panel  of  experts  undertook  a  peer  review  of  selected  publications;  the  results  of  the  peer 
review were integrated in the final report and informed the replies to the evaluation questions.  

Finally, the contractor untertook five case studies, which provided input to the evaluation and 
were presented as such in the final report.  

The preliminary results and conclusions of the evaluation were discussed during a workshop 
on 6 September 2018 in Brussels. 

In addition to the targeted consultations, the Commission organised an internet-based public 
consultation40. 147 contributions were submitted through the online questionnaire during the 
consultation period from 18 May to 30 August 2018. The contributions were published on the 
Commission website.41  

An overview and analysis of the consultation activities are presented in Annex III. 

38  

39  

40  

41  

The  documents  reviewed  for  the  evaluation  are  listed  in  Annex  2  of  the  final  report  of  the  external 
evaluation. 
The  actual  number  of  interviews  conducted  differs  as  some  of  the  Member  States’  representatives 
belong  to  various  groups  of  stakeholders  (i.e.  members  of  the  Horizontal  Working  Party  on  Drugs, 
National Focal Points and Management Board members), meaning that in some cases various groups of 
stakeholders  were  covered  by  a  single  interview.  For  an  overview  of  the  interviews  carried  out,  see 
Annex 1 of the final report of the external evaluation. 
https://ec.europa.eu/info/consultations/public-consultation-evaluation-european-monitoring-centre-
drugs-and-drug-addiction-emcdda_en.  
https://ec.europa.eu/home-affairs/what-is-new/work-in-progress/initiatives/evaluation-emcdda. 

15 

 
 
 
                                                           
B. 

LIMITATIONS AND ROBUSTNESS OF FINDINGS 

The  following  limitations  to  the  robustness  of  the  findings  have  been  identified  by  the 
contactor:42 

  Available baseline:  as already mentioned above, it was difficult  for the contractor to 
define  a  baseline  on  all  elements  of  the  evaluation  due  to  different  approaches 
compared  to  and  missing  in  the  previous  external  evaluation.  As  far  as  possible,  a 
baseline was identified nevertheless by the contractor. 

  Short duration of the evaluation: this was identified from the outset, i.e. in the offer of 
the contractor, as a potential risk for the external evaluation. The contractor took the 
necessary measures by spreading the work over different subcontractors. 

  Low  response  rate:  this  applies,  on  the  one  hand,  to  the  targeted  survey  of  the 
scientific community and the civil society, and, on the other hand, to certain questions 
in  the  questionnaires,  in  particular  those  that  needed  specific  knowledge.  For  both 
issues  the  contractor  took  the  necessary  measures  by  carrying  out  additional  desk 
research (e.g. as regards the efficiency criterion for which only the Agency staff could 
provide  any  information)  and/or  by  using  the  outcomes  of  other  methods  to 
complement  the  missing  information  (e.g.  civil  society  representatives  also  provided 
input through the public consultation). 

  Quality  of  the  received  data:  in  some  cases,  open-ended  questions  were  used  in  the 
interviews  and  in  the  public  consultation.  This  might  have  led  to  some  respondents 
providing substantive answers whereas others did not answer the question or did not 
think about the same particular issue. Therefore, the evidence presented in quantitative 
terms  should  be  treated  with  caution  in  this  regard.  In  addition,  there  were  slight 
variations in the formulation of the question regarding the potential for an expansion 
of  the  Agency’s  competences.  However,  these  slight  variations  should  not  have  any 
impact on the overall results. 

  Performance  measurement:  due  to  the  use  of  different  key  performance  indicators 
(KPIs)  over  the  evaluation  period  and  the  lack  of  an  Activity  Based  Budget,  the 
contractor  faced some challenges  in  measuring the performance of the Agency. This 
was mitigated by using headline indicators instead. 

  Stakeholder  bias:  there  might  be  a  certain  positive  bias  of  the  results  as  most  of  the 
stakeholder groups work closely with and/or benefit from the outputs of the Agency. 
However,  due  to  the  variety  of  the  stakeholders  consulted  and  the  different  data 
collection methods employed, this should be minor. 

In addition to these limitations, which apply to the results of the evaluation, the evaluation has 
to  be  seen  in  the  context  of  the  future  multiannual  financial  framework.  This  provides  a 
potential budgetary constraint for any considerations regarding any future development, being 
within its current mandate or through the deepening or expansion of its mandate. 

42  

More details are available in Annex II (2) of this Staff Working Document. 

16 

 
 
                                                           
Despite the limitations set out above, the evaluation findings are valid and reliable. Mitigation 
measures were taken and a large body of qualitative evidence, which provides a solid basis for 
drawing conclusions, underpins the evaluation.  

VI.   ANSWERS TO THE EVALUATION QUESTIONS 

The information provided in this section is a summary of the information included in the final 
report  of  the  external  evaluation,  which  will  be  made  available  online.  More  detailed 
information  can  be  found, therefore, in  the (main) final report as well  as in  Annex 3 of the 
final report of the external evaluation.  

A. 

RELEVANCE 

The  relevance  criterion  considers  the  adaptability  of  the  Agency  to  its  stakeholder  needs  as 
well  as  to  scientific,  economic,  political,  social  and  technological  changes.  In  addition,  the 
question related to the potential expansion of the mandate (listed under “Lessons learned” in 
the Terms of Reference) was also addressed under the relevance criterion.  

The stakeholder needs of the Agency are very diverse. Overall, the analysis showed that the 
Agency  responded  very  well  to  these  diverse  needs.43  The  Agency  has  an  excellent 
international standing and is regarded as a credible and authoritative source.44 The participants 
in the public consultation consider the information provided by the Agency useful across all 
areas. The information found most useful related to the emergence and use of new drugs (90% 
found  it  either  “very  useful”  (63%)  or  somewhat  useful  (27%))  and  EU  drug  policy  (90% 
found  it  either  “very  useful”  (61%)  or  “somewhat  useful”  (29%)).  The  least  useful  area  of 
information related to links between drug trafficking and organised crime with 15% finding it 
not useful, 38% somewhat useful and 26% very useful.45 

43  

44  

45  

76% of the respondents in the public consultation stated that the activities of the Agency responded to 
their needs to a large or to some extent. This result was also mirrored in the interviews undertaken. 
This  is  evidenced  by  the  overall  impression  from  the  stakeholder  consultations,  i.e.  the  answers  and 
feedback  received.  The  majority  of  the  stakeholders  from  international  organisations  (5  out  of  6 
interviewed) and from third countries (4 out of 5 interviewed) confirmed this view. 
More  detailed  information  is  shown  in  the  graph;  see  Figure  3  in  Annex  3  of  the  final  report  of  the 
external evaluation. 

17 

 
 
 
 
 
                                                           
The main stakeholders of the Agency are policy-makers on national and European level. Their 
primary need is information and evidence on which to base policy decisions. The evaluation 
showed  that  the  Agency  is  more  relevant  to  the  needs  of  European  than  national  policy-
makers.  

As  regards  the  national  policy-makers,  the  relevance  of  the  information  provided  by  the 
Agency for policy decisions depends partly on the size of the Member State and the related 
level  of  development  of  drug  information  systems.  The  better  developed  the  national  drug 
information  systems  are,  the  lower  the  impact  of  the  information  provided  by  the  Agency. 
However,  stakeholders,  in  particular  the  interviewed  members  of  the  Horizontal  Working 
Party on Drugs, recognised cross-EU comparative analysis across the board as highly relevant 
to  their needs.  Another reason for the lower impact  on national  level  might  be that national 
drug policies of several Member States go beyond illicit drugs and cover other addictions.  

As  regards  the  scientific  community  and  practitioners,  their  needs  were  partially  addressed. 
The  respondents  from  the  scientific  community  indicated  that  they  use  the  work  of  the 
Agency on a daily basis. However, there is scope for greater engagement of the Agency with 
the  scientific  community.  The  practitioners  from  the  public  health  area  appreciated  the 
increased work on this pillar. The focus of the consulted stakeholders in this area is on harm 
reduction and prevention and the respondents from this field suggested increased involvement 
of  the  Agency.  The  Agency  is  increasingly  engaged  with  law  enforcement  practitioners, 
through  e.g.  input  into  trainings  of  the  European  Union  Agency  for  Law  Enforcement 
Training (CEPOL) and closer cooperation with Europol. This is linked to the increased focus 
of the Agency on supply side issues. 

Finally, when it comes to the general public, the evaluation concluded that the visibility is not 
very  high.  It  should  however  be  noted  that  the  general  public  is  not  defined  as  a  core 
customer.  Information  provided  on  national  issues  is  usually  targeted  at  the  national  focal 
points, which are then disseminating the information as appropriate in the Member State.  

The evaluation confirmed that the Agency adapted overall well to the developments posed by 
the changes on the drug markets during the evaluation period.46 Examples are the use of new 
monitoring  tools  and  methods,  such  as  wastewater  analysis47,  the  use  of  open  source  data, 
trendspotting48  and  similar;  the  reorganisation  of  the  Agency  to  better  reflect  the  political 
priorities  by  putting  more  emphasis  on  security  and  public  health  as  the  two  pillars  of  the 
Agency’s work; or the integration of new digital solutions. In addition, the Agency adapted to 
the  changing  economic  environment,  i.e.  budget  constraints,  by  redeploying  resources  to 
(new/different)  priority  areas  without  an  increase  of  its  overall  budget.  At  the  same  time, 
stakeholders suggested that the Agency could produce more forward-looking products49 and 
that the Agency could invest more in IT training for staff and upgrading their databases and 
data  collection  systems.  Stakeholders  from  the  scientific  community  underlined  the  need  to 
improve the dialogue with the scientific and research communities to maintain and expand the 
scientific expertise of the staff.  

46  
47  
48  

49  

84% of stakeholders interviewed. 
http://www.emcdda.europa.eu/topics/pods/waste-water-analysis_en. 
Trendspotter  manual:  www.emcdda.europa.eu/publications/manuals/trendspotter-manual_en;  the  most 
cocaine  market: 
recent 
recent 
http://www.emcdda.europa.eu/system/files/publications/10225/2018-cocaine-trendspotter-rapid-
communication.pdf. 
See below in Section VI.B. for more detail. 

trendspotter 

concerned 

trends 

study 

the 

on 

18 

 
 
                                                           
One  of  the  issues  where  the  previous  external  evaluation  stated  that  further  work  should  be 
done is the drug supply side. The Agency stepped up work on supply side issues compared to 
the  previous  evaluation  period.  This  led,  for  example,  to  an  intensified  cooperation  with 
Europol  and  other  relevant  interlocutors,  the  publication  of  several  topical  reports50  and  the 
stronger  involvement  in  the  EU  policy  cycle51.  In  addition,  the  Agency  is  collecting  more 
information  on  supply  side  issues  through  the  Reitox  network.  However,  the  current 
evaluation showed that even more could be done in this context and that the Agency’s work to 
monitor  supply  side  issues  should  be  further  enhanced.  This  could  include  further 
strengthening  of  cooperation  with  Europol  and  other  bodies  working  in  the  area.  Several 
stakeholders mentioned during the consultation the supply side as one element where further 
information from the Agency would be welcome. However, it is important to stress that this 
would have to be matched with increased data provision by the national focal points on drug 
supply  related  issues.  In  order  to  be  able  to  do  so,  the  national  focal  points  would  have  to 
collaborate  closely  with  the  law  enforcement  and  other  relevant  agencies  in  their  Member 
State. It should be noted that full achievement of this objective is dependent on the mandate 
and financial resources available to national focal points. 

Another element where some stakeholders suggested that the activities could be improved is 
harm reduction and public health. In this context, the Agency could consider focussing more 
on  assessing  the  effectiveness  of  harm  reduction  and  prevention  interventions  and  social 
integration measures as well as on vulnerable populations. This is related closely to the fact 
that  although  outreach  to  stakeholders  is  well  developed  and  appreciated,  a  stronger 
involvement of practitioners could help to make an even stronger impact on contributing to a 
healthier Europe. 

In addition to the better regulation related questions on relevance, the evaluation also looked 
under this criterion at the question on the potential broadening of the scope of monitoring and 
identifying  best  practice  in  the  field  of  prevention  and  care  of  addictive  behaviours.  This 
question  addressed  the  issue  of  whether  the  Agency  should  continue  to  focus  on  illicit 
substances  only  or  whether  its  mandate/competences  should  be  revised  to  also  cover  other 
addictions,  including  e.g.  alcohol,  tobacco,  prescription  medicines  and  addictions  without 
substances such as gambling.  

The  background  to  this  question  is  that  national  drug  strategies  of  several  Member  States 
already have a broader scope than illicit substances. The following table provides an overview 
of (national) drug strategies with a broader focus (state of play 2016).52 

50  

51  

52  

the 

the 

e.g. 

report 

“Drugs 

darknet” 

(December 

See 
2017), 
and 
http://www.emcdda.europa.eu/darknet_en;  or  the  latest  version  of  the  EU  Drug  Market  Report  (April 
2016), http://www.emcdda.europa.eu/start/2016/drug-markets.  
The EU Policy Cycle for organised and serious international crime for the period 2018  – 2021 aims to 
tackle  the  most  important  threats  posed  by  organised  and  serious  international  crime  to  the  EU  in  a 
coherent  and  methodological  manner  through  improving  and  strengthening  co-operation  between  the 
relevant services of the Member States, EU institutions and EU agencies as well as third countries and 
organisations, including the private sector where relevant. Drug trafficking is one of the 10 priorities of 
the EU policy cycle. More information can be found at https://www.europol.europa.eu/empact.  
EMCDDA 
http://www.emcdda.europa.eu/system/files/publications/6402/20175662_TDAU17002ENN_PDF.pdf.  

Papers:  New 

developments 

strategies 

national 

Europe, 

drug 

in 

in 

19 

 
 
                                                           
Arguments  put  forward  during  the  consultation  included  that  addiction  is  an  overarching 
health problem and should be addressed holistically; the trend of Member States to move to 
broader  addiction  strategies;  the  arbitrary  distinction  between  what  substances  are  licit  or 
illicit; and the capacity for collection of good quality data. It should be noted that some data 
related to other substances are already collected by the Agency due to the involvement in the 
European  School  Survey  Project  on  Alcohol  and  Other  Drugs  (ESPAD)53  and  the  need  to 
address poly-drug use as part of the existing mandate. Other stakeholders argued that such a 
broadening could lead to a loss of focus on current work and could decrease quality; a lack of 
expertise and resources in the national focal points to provide the additional data; and issues 
related to EU competences and potential creation of overlaps. 

The  stakeholder  consultation  was  inconclusive  as  to  the  way  forward.  Whereas  national 
stakeholders  and  Agency  staff  were  largely  in  favour  of  broadening  the  mandate,  other 
stakeholder  groups  (including  EU-level  stakeholders)  were  critical  vis-à-vis  such  a 
development.  Among  those  speaking  in  favour  of  a  potential  future  broadening  of  the 
mandate,  alcohol  was  mentioned  most  frequently.  Other  substance-related  addictions 
followed.  National  stakeholders  expressed  less  support  to  include  addictive  behaviours  (not 
related  to  substance  use)  as  it  would  be  difficult  to  set  the  limits  for  such  a  mandate.54  All 
stakeholders,  whether  in  favour  or  against  a  broadening  of  the  mandate,  stressed  that  any 
change  in  the  mandate  would  have  to  be  accompanied  by  sufficient  financial  and  human 
resources as there is no scope to do so within the currently available resources.  

B. 

EFFECTIVENESS 

The effectiveness criterion considers the extent to which the Agency’s activities and outputs 
met its objectives set out in the founding Regulation, but also in the EU Drugs Strategy (and 
its  Actions  Plans)  and  the  EU  Agenda  on  Security.  In  addition,  the  question  related  to  the 
international  activities  (listed  under  “Lessons  learned”  in  the  Terms  of  Reference)  was 
addressed under the effectiveness criterion.  

53  
54  

http://www.espad.org/.  
For  a  detailed  analysis  of  the  arguments  in  favour  and  against  and  the  stakeholder  perceptions,  see 
Table 7 in Annex 3 of the final report of the external evaluation as well as the thematic case study in 
Annex 4 of the final report of the external evaluation. 

20 

 
 
 
 
                                                           
The  main  objective  of  the  Agency  is  to  provide  factual,  objective,  reliable  and  comparable 
information on drugs, drug addictions and their consequences, as set out in Article 1(2) of the 
founding Regulation. The Agency has fully achieved this objective as it is highly regarded as 
a centre of excellence in providing such information, not only in Europe but internationally. 
The  information  provided  is  considered  politically  independent,  is  tailored  to  the  particular 
stakeholder groups it wants to address and is based on an increasing use of different sources.  

As regards “monitoring the state of the drugs problem, in particular using epidemiological or 
other  indicators,  and  monitoring  emerging  trends,  in  particular  those  involving  poly-drug 
use”, the Agency performed overall quite well. The Agency’s provision of demand and supply 
data contributed to informing relevant authorities and practitioners, improving their ability to 
respond  to  drug  trends.  The  Agency  established  mechanisms  to  carry  out  regular  and 
sustained monitoring of developments in the drug field, aimed at identifying emerging risks. 
The  monitoring  task  is  complemented  by  publishing  regular  and  up-to-date  information  on 
drug supply and demand developments, in particular through the European Drug Report, but 
also  through its many other publications. However, the evaluation  also  concluded that more 
could  be  done  as  regards  drug-supply  indicators  and  that  work  on  poly-drug  use  is  largely 
lacking.  As  regards  the  latter,  the  external  study  stated  that  the  term  “poly-drug  use”  is  not 
mentioned once in the Agency’s 3-year strategies and work programmes.  

As regards “monitoring the solutions applied to drug-related problems; providing information 
on best practices in the Member States and facilitating the exchange of such practices among 
them”,  the  conclusion  of  the  external  evaluation  is  similar.  Stakeholders  evaluated  the 
performance  of  the  Agency  positively,  in  particular  the  civil  society  organisations 
participating in the public consultation as well as answering to the survey and to the targeted 
interviews.  The  Best  Practice  Portal55  is  central  in  sharing  best  practice  in  the  areas  of 
prevention,  treatment,  harm  reduction  and  social  reintegration.  In  addition,  the  work  of  the 
Reitox network as well as holding expert meetings is crucial for sharing best practices across 
Member States.  

As  regards  “assessing  the  risks  of  new  psychoactive  substances  and  maintaining  a  rapid 
information system with regard to their use and also regarding new methods of using existing 
psychoactive  substances”,  the  Agency  contributes  through  the  implementation  of  the  Early 
Warning  System  (EWS),  which  is  available  24/7,  and  the  risk  assessments  of  new 
psychoactive  substances.  Over  the  last  years,  the  number  of  new  psychoactive  substances 
detected  for  the  first  time  in  the  EU  and  therefore  notified  to  the  Early  Warning  System 
decreased from a peak in 2014 (see the figure below56). At the end of 2018, the Early Warning 
System monitored more than 700 new psychoactive substances, of which more or less half are 
available  in  any  given  year  on  the  European  market.57  The  monitoring  by  the  Agency 
enhances  Member  States’  capacity  to  tackle  this  growing  phenomenon.  However,  the 
substances  detected  are  becoming  more  dangerous  and,  therefore,  the  Agency  finalised  in 
2017-18 eleven risk assessments, leading to Commission proposals for putting the substances 
under  control.58  The  role  of  the  Agency  in  this  process  has  been  reinforced  by  the  new 
legislation on new psychoactive substances.59  

55  
56  
57  
58  

http://www.emcdda.europa.eu/best-practice_en.  
European Drug Report 2018, Figure 1.10, p. 32. 
Source: EMCDDA. 
The  substances  covered  by  these  risk  assessments  were  acryloylfentanyl,  furanylfentanyl,  AB-
4-
CHMINACA, 

CUMYL-4CN-BINACA, 

5F-MDMB-PINACA, 

ADB-CHMINACA, 

21 

 
 
                                                           
Finally, as regards “developing tools and instruments to help Member States to monitor and 
evaluate their national policies and the Commission to monitor and evaluate European Union 
policies”,  specific  support  was  provided  upon  request  to  some  Member  States  to  help  them 
design  or  monitor  national  drug  strategies  and  policies.  This  was  the  case  in  2015  for 
Germany, Ireland and Luxembourg. Tools and instruments for the evaluation of drugs policies 
and monitoring of drug markets have been developed.60   

The achievement of the objectives of the 3-year strategies and the related work programmes 
has further improved from an already good baseline in the previous external evaluation. This 
is  partly  due  to  a  more  focussed  and  realistic  planning  as  consecutive  strategies  and  work 
programmes reduced the number of planned outputs following a streamlining of the activities. 
Another  element  for  improving  the  delivery  of  outputs  is  related  to  the  three-level 
prioritisation system introduced.61 The evaluation showed that the Agency achieved the goals 

59  

60  

61  

and 

tetrahydrofuranylfentanyl, 

carfentanils,  methoxyacetylfentanyl 

fluoroisobutyrylfentanyl, 
cyclopropylfentanyl. 
Regulation  (EU)  2017/2101  of  the  European  Parliament  and  of  the  Council  of  15  November  2017 
amending  Regulation  (EC)  No  1920/2006  as  regards  information  exchange  on,  and  an  early  warning 
system  and  risk  assessment  procedure  for,  new  psychoactive  substances,  OJ  L  305,  21.11.2017,  p.1; 
Directive  (EU)  2017/2103  of  the  European  Parliament  and  of  the  Council  of  15  November  2017 
amending Council Framework Decision 2004/757/JHA in order to include new psychoactive substances 
in the definition of ‘drug’ and repealing Council Decision 2005/387/JHA, OJ L 305, 21.11.2017, p. 12. 
See  for  example  “Evaluating  drug  policy:  a  seven-step  guide  to  support  the  commissioning  and 
managing  of  evaluations”,  www.emcdda.europa.eu/system/files/publications/4680/td0417390enn1.pdf, 
and  the  related  website  (www.emcdda.europa.eu/publications/topic-overviews/policy-evaluation);  or 
“Trendspotter  manual:  a  handbook  for  the  rapid  assessment  of  emerging  drug-related  trends”, 
http://www.emcdda.europa.eu/system/files/publications/10233/2018-trendspotter-manual.pdf.  
Level  1:  must-do-tasks  with  a  target  of  100%  achievement;  Level  2:  supporting  tasks  to  key 
commitments  with a target of 80% achievement; and Level 3: supplementary  tasks that  can be scaled 
down without significantly impacting the achievement of the Agency’s objectives. 

22 

 
 
 
                                                                                                                                                                                     
and objectives set out in its two 3-year strategies and work programmes as shown by the table 
below.62  

Implementation of Annual WPs  2014 

2015 

2016 

2017 

Level 1 (target 100%) 

88% 

92% 

97% 

97% 

Level 2 (target 70%/80%) 

78% 

77% 

83% 

92% 

Level 3 (target 40%/50%) 

66% 

75% 

60% 

77% 

This conclusion is also confirmed by the outcome of the public consultation as shown in the 
figure below.63 

As  regards  the  question  to  what  extent  the  national  focal  points  delivered  the  data  and 
information  required  for  the  Agency  to  meet  its  objectives,  the  overall  conclusion  of  the 
evaluation  is  that  the  data  and  information  was  delivered  effectively  during  the  evaluation 
period. The Reitox network and its national focal points are considered a cornerstone of the 
Agency’s work and activity. The “Reitox Development Framework”64, which was put in place 
in  2018,  addresses  several  of  the  issues  identified  by  the  external  evaluation  and  includes 
measures  to  build  up  further  the  capacity  of  the  national  focal  points  and  to  improve  the 
quality  of  their  data  collection  and  analysis.  New  reporting  packages,  including  detailed 
handbooks, guidelines and procedures, were adopted during the evaluation period in order to 
rationalise the data collection process.  The provision of data by the national  focal  points by 
the deadline set by the Agency increased considerably. This is crucial for the Agency being 
able to provide the analysis of the data in due time for the publication of the annual European 
Drug Report.  

62  

63  
64  

Annex  3  of  the  final  report  of  the  external  evaluation,  p.  75;  based  on  the  annual  General  Report  of 
Activities of the Agency.  
Annex 3 of the final report of the external evaluation, p. 43. 
http://www.emcdda.europa.eu/system/files/publications/8137/TD0118374ENN.pdf.  

23 

 
 
 
 
 
                                                           
Despite  progress  made,  further  improvements  are  considered  possible  and  necessary.  This 
relates  in  particular  to  the  comparability  of  the  data  provided  and  in  some  cases  the  data 
quality.65  

The  problems  identified  by  the  external  evaluation  as  regards  the  data  provision  by  the 
national  focal  points  are  related  to  the  varied  mandates  and  resources  available  to  national 
focal  points.  The  appointment,  setting  up,  functioning  and  maintenance  of  a  national  focal 
point is  the responsibility  of the national  authorities and is  therefore 100 %  ﬁnanced by the 
country  concerned.  However,  the  Agency  co-ﬁnances  national  focal  points  in  EU  Member 
States to a certain extent by means of a grant agreement, which supports the implementation 
of speciﬁc activities in relation to national reporting functions and obligations in the Agency’s 
annual  work  programme.  The  mandate  of  the  national  focal  points  also  depends  on  the 
national  drug  policy.  For  example,  the  mandate  can  go  far  beyond  drugs  to  cover  different 
tasks  as  regards  all  kinds  of  addiction  or  it  can  be  very  narrow  and  only  cover  certain  data 
collection tasks.  

Governance 

The  previous  external  evaluation  already  concluded  that  the  governance  structures  and 
procedures worked well. The changes made to the structure and the internal reorganisation are 
quite  recent  as  they  happened  in  2016  and  2017  following  the  adoption  of  the  “EMCDDA 
Strategy 2025”.66 Therefore, their impacts are not fully visible yet. However, the findings of 
the current evaluation, in particular the input received from interviews and the survey of the 
Agency’s  staff,  suggest  that  the  changes  have  been  generally  positive.  The  internal 
reorganisation helped to focus resources better on the two pillars of the “EMCDDA Strategy 
2025”, i.e. security and public health. This reorganisation improved the internal coordination 
within the Agency, led to providing more robust data and addressing new issues, and enabled 
the Agency to react better to challenges and contextual shifts in the field of drugs. 

Monitoring 

The  evaluation  report  stresses  that  due  to  the  use  of  different  key  performance  indicators 
(KPIs)  over  the  evaluation  period  and  the  lack  of  an  Activity  Based  Budget,  performance 
monitoring by the contractor could be done only based on headline indicators. Further efforts 
to  streamline  the  indicators  as  well  as  to  introduce  an  Activity  Based  Budget  should  be 
undertaken by the Agency to mitigate this for the future. 

The tools67 available to monitor and review the Agency’s outputs and results are considered 
adequate  for  ensuring  accountability  and  an  appropriate  assessment  of  its  performance. 
Starting with the 2014 work programme, a specific key performance indicator (KPI) was put 
in  place  for  the  monitoring  of  the  work  programme  delivery.  A  detailed  Monitoring  and 
Evaluation Plan has been developed as internal tool to follow-up. In addition, a Management 
and Implementation System, a performance monitoring IT system, is in place; however, only 
limited training of the staff happened so far. 

65  
66  
67  

As regards drug supply data, see above in Section VI.A. 
See Section III above for the current organisational chart of the Agency. 
In the evaluation, ‘tool’ was defined as an IT system or platform and/or an indicator used to carry out a 
particular  monitoring  or  performance  measurement  function  (as  opposed  to  ‘mechanism’,  which  is  a 
process involving multiple stakeholders). 

24 

 
 
                                                           
The evaluation concluded that the internal and external mechanisms68 in place for monitoring, 
reporting and evaluating work well and are adequate to ensure accountability and appropriate 
assessment of the overall performance. Several regular review exercises are in place to review 
progress  and  monitor  the  outputs  and  achievements  of  results.  These  measurements  are 
regularly  discussed  and  followed  up  in  the  appropriate  meetings.  A  mid-term  review  of  the 
“EMCDDA Strategy 2025” is planned for 2020. As regards external mechanisms, the Agency 
is  subject  to  the  scrutiny  of  the  annual  discharge  procedure,  the  European  Parliament,  the 
European Court of Auditors and the Internal Audit Service. The General Report of Activities 
provides an annual follow-up to these monitoring mechanisms. 

External factors 

The  evaluation  identified  four  main  external  factors,  which  had  an  influence  on  the 
effectiveness of the Agency:69 

  Changing  European  drug  landscape:  The  significant  increase  of  new  psychoactive 
substances was considered the main challenge. The Agency stepped up the proactive 
monitoring  of  new  psychoactive  substances  through  its  Early  Warning  System  and 
developed  innovative  monitoring  tools  to  identify  the  changes  in  drug  consumption 
(e.g. trendspotting studies or wastewater analysis). 

  Emergence  of  new  marketplaces:  New  online  marketplaces  as  well  as  changing 
trafficking routes are most notable. The Agency increased its collaboration with other 
EU agencies, such as Europol, to monitor better these new marketplaces and improved 
its  online  monitoring  tools.  This  external  factor  will  increase  further  over  time  and 
therefore adaptations of the work will be needed. 

  New  security  challenges:  These  challenges  stem  from  the  involvement  of  organised 
crime  groups  in  several  criminal  activities  (“poly-criminality”),  such  as  drug 
trafficking, human trafficking, migrant smuggling, or financing terrorist activities. The 
links between these activities need to be addressed in the future and this will have an 
impact on the information to be provided to policy-makers. 

  Evolving national policies: With the change of national drug/addiction policies comes 
a  change  in  the  needs  of  national  stakeholders.  This  had  a  significant  impact  on  the 
Agency’s effectiveness, in particular with regard to the identification of best practices 
in health and social responses. 

Cooperation with international organisations and third countries70 

The recast of the founding Regulation in 2006 gave the Agency a more explicit role as regards 
cooperation  with  international  organisations  and  third  countries.  The  previous  external 

68  

69  

70  

In  the  evaluation,  ‘mechanism’  was  defined  as  a  process  involving  multiple  (internal  or  external) 
stakeholders,  used  to  monitor,  report  and  evaluate  the  Agency’s  overall  performance  and  or  ensure 
accountability. 
The contractors noted in the final report of the external evaluation that most stakeholders misinterpreted 
this  question.  Most  identified  budgetary  concerns,  data  collection  through  the  Reitox  network  or  the 
shifting European policies as  external factors that could influence the  Agency’s effectiveness, but the 
contractors  considered  those  to  be  internal  factors,  as  they  fall  within  the  control  of  the  European 
Commission, which is also the case for the Agency. 
For more details on the work of the Agency on international level, see the organisational case study in 
Annex 4 of the final report of the external evaluation. 

25 

 
 
                                                           
evaluation concluded that the “relationship with key partners at international level should be 
further  developed  to  improve  the  capacity  to  monitor  and  analyse  the  drugs  situation  and 
responses to it”.  

The Agency took on board this recommendation through continued work on the “EMCDDA 
Strategy  for  International  Cooperation”71,  which  has  largely  been  implemented,  and 
consecutively, in 2018, the adoption of the “EMCDDA International Cooperation Framework 
2018-25”72.  Its  activities  at  international  level  are  compatible  with  the  EU  priorities  in 
external  action.  The  Agency  brings  the  EU’s  evidence-based  policy-making  experience  in 
drug monitoring to third countries, thereby helping to improve the global understanding of the 
drug  phenomenon,  including  the  impacts  global  developments  have  on  the  European  drug 
market.  

The  Agency  is  supporting  the  European  Commission  in  several  regional  cooperation 
projects73  and  the  analysis  of  progress  of  candidate  countries  as  regards  adherence  with  the 
EU acquis in the drugs field74. In addition, the Agency has concluded over time a number of 
working  arrangements  with  third  countries  in  order  improve  the  capacity  development  in 
these third countries.75  

Finally, the Agency cooperates closely with  the United Nations Office on Drugs and Crime 
(UNODC)  and  the  World  Health  Organisation  (WHO),  and  to  a  lesser  degree  with  other 
international  organisations.  As  regards  the  cooperation  with  the  United  Nations  Office  on 
Drugs and Crime, several stakeholders raised the duplication of data collection as something 
that  needs  to  be  addressed,  despite  the  need  to  overcome  several  legal  and  methodological 
difficulties.  

It  was  also  commented  during  the  consultation  that  the  cooperation  with  countries  of  the 
Western Balkan and immediate neighbourhood of the EU might be a too limited approach and 
that  the  Agency,  in  general,  could  do  more  on  the  international  sphere,  although  within  the 
limits of the general resource constraints. The Agency should clearly define the benefit to its 
objectives and the understanding of the EU drug situation of cooperation with third countries. 
This could also include the review of the priority countries/ regions. International cooperation 
activities  should  focus  on  facilitating  the  obtaining  of  information  in  a  timely  and  regular 
manner from third countries or international organisations. Based on these considerations, the 
countries for future cooperation could be chosen. In this context, the funding opportunities for 
the work with third countries should be looked at, in particular the possibility of the Agency 
to receive ad hoc funding to implement its activities outside the EU. 

71 

72 

73  

74  

75  

 http://www.emcdda.europa.eu/attachements.cfm/att_255775_EN_EMCDDA_30_07_International_coo
peration_FINAL.doc.  

 http://www.emcdda.europa.eu/system/files/publications/9886/International%20Cooperation%20Frame
work.pdf.  
For example, the Cooperation Programme between Latin America, the Caribbean and the EU on Drugs 
Policies (COPOLAD II) or the Central Asia Drug Action Programme (CADAP). 
For  example,  the  Agency  was  part  of  assessment  missions  to  Serbia  and  Montenegro  to  assess  the 
advances in the Justice and Home Affairs area as part of the membership negotiations. 
Working  arrangements  are  currently  in  place  with  Russian  Federation,  Ukraine,  Moldova,  Israel, 
Armenia, Georgia, Switzerland and Albania. 

26 

 
 
 
                                                           
C. 

EFFICIENCY 

The efficiency criterion considers whether the Agency  achieved its objectives and delivered 
results  in  a  cost-effective  manner.  More  specifically,  this  section  reviews  the  costs  and 
benefits  of  activities;  the  adequacy  of  the  human  and  financial  resources  to  attain  its 
objectives; value for money; the efficiency of its governance and organisational set-up and the 
scope for further simplification of its administrative structure and ways of working;  and the 
external  factors  influencing  its  performance.  In  addition,  the  question  related  to  the 
communication on and dissemination of outputs (listed under “Lessons learned” in the Terms 
of Reference) was addressed under the efficiency criterion. 

The  annual  budget  over  the  evaluation  period  was  around  EUR  16  million.  The  three  main 
types of expenditure are staff costs (on average 59% of the budget); operational activities (on 
average 27% of the budget) and support activities (on average 14% of the budget). The budget 
remained  relatively  stable  over  the  evaluation  period  with  a  decrease  in  2014  and  a  one-off 
increase in 2015. The budget distribution is comparable with other agencies of similar size.  

Distribution of budgets of selected EU agencies (2017)76 

EMCDDA  FRA 

EU-OSHA  EASO 

Budget 

Staff 

60% 

Support/Administrative 

10% 

Operational 

30% 

53% 

11% 

36% 

36% 

8% 

52% 

21% 

12% 

67% 

The number of staff also remained stable with on average 105 full-time equivalents per year. 
The  share  of  operational  vs.  administrative  staff  increased  in  favour  of  the  operational  staff 
whose share in the overall staff increased from 68% in 2012 to 71% in 2017. This lead to a 
consequential decrease in support staff, from 32% in 2012 to 29% in 2017. 

The benefits of the Agency include clear and thorough understanding of the drug situation in 
the  EU,  better  informed  debates  on  drug  policies  and  strategy,  more  effective  exchange  of 
information  among  professionals  (including  through  the  Reitox  network),  a  proactive 
approach  to  new  psychoactive  substances  and  emerging  trends,  and  the  promotion  of 
scientific  excellence.  These  benefits  are  difficult  to  quantify,  but  overall  the  outputs  of  the 
Agency are valued highly by all stakeholders at EU, national and international level.  

The evaluation concluded within the limits of the available data for the efficiency analysis, in 
particular that Activity Based Budgets were not available, that the Agency spent its resources, 
both human and financial ones, efficiently. Despite the budget staying relatively flat and the 
need to respond to new challenges and an increasing volume of tasks, the Agency managed to 
deliver without affecting the quality of the outputs by efficiently redeploying resources.  

Another indicator for the efficiency  of the performance is  the implementation of the  annual 
work  programmes  and  delivery  of  its  outputs.  Level 1-targets  were  implemented  by  88% 

76  

Annex 3 of the final report of the external evaluation, Table 10, p. 74; based on the 2017 budgets of the 
respective agencies. 

27 

 
 
 
                                                           
(2014) to 97% (2017).77 As the table below shows, the share of non-completed results/outputs 
declined  from  21%  in  2013  to  9%  in  2017.  At  the  same  time,  the  number  of  specific 
objectives  in  the  work  programmes  was  streamlined  without  affecting  the  volume  of  its 
outputs and its outreach. 

Status of results/outputs (2013-2017)78 

Results/outputs 

2013 

2014 

2015 

2016 

2017 

Not implemented 

Delayed 

Cancelled 

Partially implemented 

Total not completed 

1 

61 

12 

15 

89 

0 

14 

12 

54 

80 

2 

26 

2 

14 

44 

2 

22 

0 

2 

26 

Total planned per AWP 

424 

397 

251 

153 

% not completed 

21% 

20% 

18% 

17% 

0 

14 

0 

0 

14 

164 

9% 

The Agency consistently used all allocated funds to achieve its work programmes. The budget 
execution rate for core activities (excluding the special projects) was very high, over 99.6% in 
each  of  the  evaluation  years,  reaching  100%  for  2016  and  2017.  This  indicates  that  the 
allocated funds were spent efficiently and no waste occurred. 

Therefore,  the  evaluation  concluded  that  the  Agency  provides  value  for  money  and  that  the 
resources available have been used efficiently. The evaluation also notes that two thirds of the 
stakeholders  consulted  considered  the  financial  and  human  resources  as  adequate  for  the 
current priorities and tasks, but insufficient in case any new tasks would be added.  

Despite  the  overall  very  positive  conclusion  as  regards  the  use  of  human  and  financial 
in  particular  on 
improvements  are  possible.  Further  synergies, 
resources,  further 
administrative  issues,  should  be  considered  with  other  agencies,  in  particular  with  the 
European  Maritime  Safety  Agency  (EMSA),  with  which  the  Agency  already  shares  some 
resources  and  uses  common  services.  The 
implementation  of  change  management 
programmes  and  the  further  development  of  targeted  use  of  secondments  or  short-term 
contracts with scientists or interns should be considered. 

The governance, organisational structures and procedures work in general well. The changes 
introduced following the adoption of the “EMCDDA Strategy 2025” were beneficial in terms 
of efficiency as it strengthened cross-unit working and the coordination of scientific activities. 

77  
78  

Level 2-targets by 78% (2014) to 92% (2017) and Level 3-targets by 66% (2014) to 77% (2017). 
Annex 3 of the final report of the external evaluation, Table 12, p. 75; based on the General Reports of 
Activities for the years 2013 to 2017.  

28 

 
 
 
 
 
 
 
 
 
                                                           
The increased efficiency is shown by the fact that the Agency produced more outputs while 
investing the same resources in absolute terms.  

Overall, the evaluation concluded that the structure and organisation is efficient and no major 
changes are needed.  

The following issues were mentioned as influencing efficiency: the decrease of the subsidy in 
2014;  the  increase  in  numbers  and  in  dangerousness  of  new  psychoactive  substances;  the 
increase in poly-drug use; the decrease in national funding for national focal points; and the 
shift  in  policy  focus  to  other  (security)  areas,  such  as  terrorism.  Overall,  it  should  be  noted 
that no additional financial (or human) resources were allocated to the Agency for additional 
work,  except  for  the  work  related  to  the  implementation  of  the  new  legislation  on  new 
psychoactive  substances.  As  mentioned  before,  the  Agency  still  managed  to  deliver  results 
and outputs on these issues through the redeploying of resources.  

In  terms  of  communication  and  dissemination,  the  feedback  from  stakeholders  was  very 
positive  and  it  was  considered  as  efficient.  The  evaluation  concluded  that  the  online 
outreach79,  in  particular  the  social  media  outreach  increased  considerably80,  in  particular 
taking  into  account  the  limited  dedicated  financial  and  human  resources  compared  to  other 
agencies. Improvements were made, following the recommendations of the previous external 
evaluation,  as  regards  printed  communication  material,  making  in  particular  the  European 
Drug Report timelier and including more visual content. However, these are still areas where 
there  is  potential  for  further  improvement,  i.e.  making  reports  and  information  more 
interactive, providing more and better translation, and more generally improve the website. In 
addition,  the  outreach  to  the  general  public  and  national  stakeholders  (policy-makers  and 
practitioners) should be further improved. Outreach at national level is done currently through 
the national focal points, which act as multipliers and disseminators of the Agency’s products. 
However, it could be worth considering addressing these target group also more directly. 

D. 

COHERENCE 

The  coherence  criterion  considers  the  extent  to  which  the  activities  and  objectives  of  the 
Agency  support  the  key  EU  policy  developments,  as  well  as  are  complementary  to  actions 
undertaken  by  other  EU  agencies  and  Member  States,  including  being  consistent  with  its 
regulatory  framework.  In  addition,  the  questions  related  to  synergies  with  other  Justice  and 
Home Affairs agencies on security issues and to cooperation with the United Nations level, in 
particular  with  the  United  Nations  Office  on  Drugs  and  Crime,  (listed  under  “Lessons 
learned” in the Terms of Reference) were also addressed under the coherence criterion. 

The external evaluation mapped the objectives, tasks and priority areas set out in the founding 
Regulation  as  the  well  as  the  EU  priorities  with  the  two  3-year  strategies  covering  the 
evaluation  period81.  The  same  was  done  for  checking  coherence  more  generally  with  EU 

79  

80  

81  

Figure  21  in  Annex  3  of  the  final  report  of  the  external  evaluation,  p.81,  shows  the  number  of 
downloads of the main products of the Agency. The main downloaded publication is by far the annual 
European Drug Report, which has between 66.000 and 118.000 downloads. 
At the end of 2017, the Agency had approximately 11450 Twitter followers (compared to 3400 at the 
end  of  2013),  almost  8000  Facebook  followers  (compared  to  below  2000  in  2013)  and  more  than 
190000 Youtube views (compared to around 12500 in 2013). 
Annex 3 of the final report of the external evaluation, p. 82. 

29 

 
 
 
                                                           
policy developments.82 The desk  research of the contractor and the mapping exercise of the 
founding  Regulation  and  the  EU  strategies  with  the  Agency’s  3-year  strategies  and  work 
programmes  and  other  relevant  strategic  documents  of  the  Agency  is  the  basis  for  the 
information presented below. 

The table below presents the number of specific objectives in the 3-year strategies and work 
programmes for each of the priority areas set out in Annex I of the founding Regulation.83 

Priority area (as per 
Annex I of the founding 
Regulation) 

SWP 2013-2015 

SWP 2016-2018 

Number of 
specific 
objectives 

% of total no. of 
specific 
objectives 

Number of 
specific 
objectives 

% of total no. of 
specific 
objectives 

20 

71% 

25 

76% 

16 

57% 

21 

64% 

16 

57% 

25 

76% 

10 

36% 

15 

45% 

Priority 1 - Monitoring the state 
in 
the  drugs  problem, 
of 
using 
particular 
epidemiological 
other 
indicators, 
and  monitoring 
emerging  trends,  in  particular 
those involving poly-drug use. 

or 

Priority  2  -  Monitoring  the 
to  drug-
solutions  applied 
related  problems;  providing 
information on best practices in 
the  Member 
and 
States 
the  exchange  of 
facilitating 
such practices among them. 

Priority  3  -  Assessing  the  risks 
of new psychoactive substances 
and  maintaining 
rapid 
information system with regard 
to  their  use  and  also  regarding 
new  methods  of  using  existing 
psychoactive substances. 

a 

to 

instruments 

Priority  4  -  Developing  tools 
help 
and 
Member  States  to  monitor  and 
evaluate  their  national  policies 
and the Commission to monitor 
and  evaluate  European  Union 
policies. 

The external evaluation shows that all priority areas set out in the founding Regulation have 
been addressed in both programming periods and that all objectives, tasks and priority areas 
are  covered  well  by  the  work  programmes.  This  confirms  the  conclusions  of  the  previous 

82  
83  

Annex 3 of the final report of the external evaluation, p. 84. 
Annex 3 of the final report of the external evaluation, p. 1. 

30 

 
 
 
                                                           
external evaluation. One element of the founding Regulation where some gaps were identified 
during the evaluation is the monitoring of data on emerging trends in poly-drug use. There are 
only very few outputs in relation to the poly-drug use and the issue itself is as such not even 
mentioned  in  the  Agency’s  strategic  objectives.  Some  work  is  done  in  the  context  of  the 
European  School  Survey  Project  on  Alcohol  and  Other  Drugs  (ESPAD),  but  the  evaluation 
concluded  that  more  could  be  done.  Other  areas  where  the  evaluation  concluded  that  more 
could  be  done  are  work  on  drug  supply,  in  particular  the  related  indicators84,  or  further 
cooperation with the local level and with the scientific community. 

When  it  comes  to  the  EU  Drugs  Strategy  2013-2020  and  its  Action  Plans,  the  mapping 
showed  a  strong  alignment  between  the  Agency’s  outputs  and  its  results  with  key  and 
overarching EU objectives. This is evidenced by the fact that the Agency is mentioned in 22 
out of 55 actions of the EU Action Plan on Drugs 2017-2020 as responsible party. In addition, 
the  Agency  is  required  to  report  on  14  of  the  15  indicators  for  both  EU  Action  Plans.  The 
mid-term assessment of the EU Drugs Strategy highlighted the significant contribution by the 
Agency  and  its  Reitox  network  to  the  implementation  of  the  EU  Drugs  Strategy  and  to  a 
better  understanding  of  all  aspects  of  the  drugs  situation  in  the  EU  and  trends  in  drug 
markets.85  

The  3-year  strategies  and  work  programmes  address  and  contribute  to  the  objectives, 
principles and priorities of the European Agenda on Security. Given the limited remit of the 
Agency on the wider security issues, it cannot be expected that it comprehensively covers all 
aspects of the European Agenda on Security. However, through the growing focus on supply-
side issues and the cooperation with other agencies working in the area (such as Europol, the 
Maritime  Analysis  and  Operations  Centre  (MAOC(N)),  or  the  European  Union  Agency  for 
Law  Enforcement  Training  (CEPOL))  on,  for  example,  joint  trainings  for  law  enforcement 
agencies,  joint  publications  or  the  stronger  involvement  of  the  Agency  in  the  EU  Policy 
Cycle, the Agency increasingly contributes to this priority. 

Therefore, it can be concluded that the objectives and the activities set out in the two 3-year 
strategies  and  work  programmes  are  highly  coherent  with  the  regulatory  framework.  All  of 
the objectives set out in  these documents align with and can be traced back to the founding 
Regulation.  

The  impact  case  study86  indicated  a  moderate  to  strong  causal  link  between  the  Agency’s 
activities and the impact of providing an evidence base for policy-making. Several activities, 
in particular in cooperation with other European and international agencies, provide evidence 
for the coherence of their work with EU policy developments. Coherence with the work of the 
most relevant EU agencies is facilitated by mutual review of the respective work programmes.  

84  

85  

86  

For a state of play, see the EMCDDA-Europol Joint Publication “Improved drug supply indicators for 
report”, 
progress 
Europe: 
www.emcdda.europa.eu/system/files/publications/10178/Improved%20drug%20supply%20indicators%
20for%20Europe_Joint%20publication.pdf. 
Mid-Term Assessment of the EU Drugs Strategy 2013-2020 and final evaluation of the Action Plan on 
https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-
Drugs 
do/policies/organized-crime-and-human-trafficking/drug-control/eu-response-to-
drugs/20161215_final_report_annexes_en.pdf.  
See Annex 4 of the final report of the external evaluation. 

2013-2016,  Annex 

J; 

31 

 
 
                                                           
Overall, the Agency is well aligned with the Common Approach for decentralised agencies.87 
The tools and mechanisms in place work quite well and are adequate to ensure accountability 
and appropriate assessment of the overall performance. Monitoring of the implementation of 
the 3-year strategies and work programmes and the Management Plan takes place via a series 
of  monitoring  exercises.  The  development  of  clear  key  performance  indicators  is  positive; 
however,  these  have  changed  during  the  evaluation  period  and  therefore  comparisons  over 
time and with the previous evaluation were in parts not possible. In addition, the Agency has 
not yet implemented some Common Approach actions, in particular those relating to Activity 
Based Management and Activity Based Budgets. Other elements where improvements would 
be needed to achieve full compliance with the Common Approach are related to making the 
website as multilingual as possible, the Management Board considering the need for ex-ante 
evaluations of activities and programmes that have budget implications, and looking into the 
potential joint procurement with contracting authorities of the host Member States. All these 
actions are on-going within the Agency, but have not been completed for different reasons.88 

When looking more closely into the coherence of the objectives and activities with the drug-
related objectives of the European Commission and other EU agencies, in particular Europol, 
the  European  Centre  for  Disease  Prevention  and  Control  (ECDC)  and  the  European 
Medicines Agency (EMA), the evaluation concluded that the objectives set out by the Agency 
in  its  3-year  strategies  and  work  programmes  and  the  “EMCDDA  Strategy  2025”  are 
complementing those of its partners on EU level. The analysis89 shows that there are hardly 
any  overlaps  with  other  EU  agencies,  but  there  could  be  considerable  synergies.  Many  of 
these  synergies  are  already  exploited  by  the  agencies.  Examples  include  the  sharing  of 
information  on  new  psychoactive  substances  among  relevant  agencies,  in  particular  the 
European Medicines Agency; cooperation with Europol regarding the role of the internet and 
the  darknet  in  selling  drugs;  the  cooperation  with  the  European  Union  Agency  for  Law 
Enforcement  Training  (CEPOL)  on  training  activities;  or  the  cooperation  with  Eurojust  on 
issues related to judicial challenges in transnational prosecution. More concrete examples are 
provided in the following paragraphs. 

The merging with another agency was considered detrimental to the quality of the scientific 
output. In addition, the fact that the Agency’s activities are very specific is seen by consulted 
stakeholders as there not being a potential for a merge with another agency.90   

The  conclusion  as  regards  coherence  with  Member  States’  national  drug  policies  is  slightly 
less positive as some Member States apply a pan-addiction approach whereas the mandate of 
the  Agency  is  limited  to  illicit  substances.  As  the  Agency  cannot  provide  data  on  other 
addictions to the Member States to support their policies, some interviewed stakeholders, in 
particular from Member States, saw this as limiting the coherence of the work of the Agency 
with their work on national level. 

Synergies  with  the  Commission  exist  for  example  in  the  work  on  drugs  precursors  and  on 
health-related  issues.  As  regards  precursors,  the  Agency  cooperates  with  DGs  TAXUD  and 

87  

88  
89  

90  

The analysis of the compliance with the Common Approach  on Decentralised Agencies is included in 
Annex 6 of the final report of the external evaluation. 
For more details see Annex 6 of the final report of the external evaluation. 
See analysis of questions EQ23 and EQ24, Annex 3 of the final report of the  external evaluation, as of 
p. 88. 
54%  of  all  interviewed  stakeholders  expressed  this  view.  In  addition,  almost  all  participants  in  the 
public  consultation  provided  a  clear  answer  to  the  potential  closing  or  merging  of  the  Agency  with 
another body. 

32 

 
 
                                                           
GROW. This cooperation takes place on a voluntary basis and therefore relies on the goodwill 
and dedication of the involved individuals. Therefore, there is a risk that these activities will 
be interrupted if staff or priorities change. This risk could be mitigated by a formal working 
arrangement with the Commission on drug precursors. Such a working arrangement already 
exists  with  DG  SANTE.  However,  the  evaluation  concluded  that  a  closer  structural 
relationship could be developed. Cooperation also takes place with the Joint Research Centre 
(JRC), which is involved in the detection of drugs and precursors in Europe and provides data 
to  the  Agency.  Cooperation  with  JRC  is  important,  as  the  Agency  does  not  have  its  own 
forensic laboratory facilities for scientific testing. 

The  cooperation  with  Europol91  is  very  fruitful  and  is  considered  by  Europol  as  optimal. 
Several joint publications have been published over the years, starting with the joint work on 
risk assessments of new psychoactive substances, to (up to now) two editions of the EU Drug 
Markets  Report92  to  specialised  publications  such  as  the  report  on  drugs  and  the  darknet93. 
Both agencies provided their relevant expertise for the joint work, with Europol’s focus being 
on data on drug production, distribution, seizure and the role of organised crime.  

The  cooperation  with  the  European  Medicines  Agency94  is  related  mainly  to  work  in  the 
context of the Early Warning System and the assessment of new psychoactive substances. The 
two  agencies  also  exchange  and  cross-reference  each  other’s  data  as  part  of 
pharmacovigilance  efforts  to  prevent  misuse  of  medicinal  products  and  the  use  of  new 
psychoactive substances. This leads to more accurate data for both agencies.  

The collaboration with the European Centre for Disease Prevention and Control95 focusses on 
areas where drug use constitutes a risk factor for the contraction of infectious diseases. The 
joint  activities  include  gathering  data  from  Member  States  and  joint  missions  to  specific 
Member States in  the  context  of HIV. Again,  the collaboration means that  each  agency  can 
provide its scientific expertise to  effectively  conduct  reviews and risk assessments.  There is 
scope  to  update  joint  products  more  regularly  and  additional  fields  of  work  could  be 
considered, assuming that resources are made available. 

Other relevant Justice and Home Affairs agencies with which the Agency already collaborates 
are the European Union Agency for Law Enforcement Training (CEPOL) and the  European 
Union  Agency  for  judicial  cooperation  in  criminal  matters  (Eurojust).  As  regards  the 
European Union Agency for Law Enforcement Training96, the Agency contributes heavily to 
training  activities  aimed  at  enhancing  awareness  among  EU  law  enforcement  authorities 
working in the field of drugs. As regards Eurojust97, a publication98 aimed at policy-makers 
and  legal  practitioners  to  find  suitable  responses  to  the  phenomenon  of  new  psychoactive 
substances was published. Given Eurojust’s increasing involvement in drug trafficking cases, 
the Agency is well placed to continue to inform and provide context on drug trafficking. No 
collaboration exists so far with the European Border and Coast Guard Agency. However, the 
evaluation recommended that joint activities with European Border and Coast Guard Agency 

91  
92  

93  
94  
95  
96  
97  
98  

http://www.emcdda.europa.eu/about/partners/europol.  
2013:  http://www.emcdda.europa.eu/system/files/publications/741/TD3112366ENC_406102.pdf;  2016: 
http://www.emcdda.europa.eu/start/2016/drug-markets.  
http://www.emcdda.europa.eu/darknet.  
http://www.emcdda.europa.eu/about/partners/ema.  
http://www.emcdda.europa.eu/about/partners/ecdc.  
http://www.emcdda.europa.eu/about/partners/cepol.  
http://www.emcdda.europa.eu/about/partners/eurojust.  
http://www.emcdda.europa.eu/system/files/publications/3353/TD0416736ENN.pdf.  

33 

 
 
                                                           
should  be  explored,  in  particular  as  regards  data  exchange  and  the  monitoring  of  potential 
links between drug trafficking, human trafficking and migration. 

The  Agency  also  cooperates  with  the  Maritime  Analysis  and  Operation  Centre  –  Narcotics 
(MAOC(N))99, an operational platform of several Member States, third countries and agencies 
set up to principally tackle the transatlantic flow of cocaine from South America to Europe. 
The Maritime Analysis and Operation Centre  –  Narcotics is  dependent  on information  from 
the  Agency  for  its  operations.  Closer  cooperation  might  be  beneficial  for  both,  on  the  one 
hand,  for  the  Maritime  Analysis  and  Operation  Centre  –  Narcotics  to  further  enhance  its 
effectiveness  due  to  more  direct  data  access,  on  the  other  hand,  for  the  Agency  to  be  more 
involved  in  operational  activities  and  thereby  having  a  stronger  impact  on  creating  a  more 
secure Europe. 

As  regards  the  international  level,  the  objectives  were  set  out  in  the  “2007  Strategy  for 
international  cooperation” which was replaced by  the “EMCDDA  International Cooperation 
Framework 2018-25” in 2017. When it comes to international organisations, the focus is the 
cooperation  with  the  United  Nations  Office  on  Drugs  and  Crime100  and  the  World  Health 
Organisation101 and in particular the exchange of data with these organisations. The Agency 
also  participated  as  experts  in  several  of  their  meetings  and  provided  input  to  their 
publications.  In  addition,  cooperation  also  takes  place  with  the  Pompidou  Group102  and  the 
American Drug Abuse Control Commission (CICAD)103.  

The  external  study  shows  that  data  exchange  between  the  Agency  and  international 
organisations  is  largely  unidirectional.  An example is  the work of the Expert Committee on 
Drug  Dependence  of  the  World  Health  Organisation  on  new  psychoactive  substances.  The 
Agency is providing all data available in its database on the substances, which are critically 
reviewed by the Expert Committee. On the other hand, there is no or very limited provision of 
data from the United Nations Office on Drugs and Crime to the Agency, which then could be 
used by the Agency in its reports. Getting more information from international organisations 
would  help  the  Agency  to  develop  better  its  understanding  of  how  the  global  drug  markets 
affect the European drug market.  

The organisation with which most overlaps were identified was the United Nations Office on 
Drugs and Crime (UNODC). Several stakeholders, in particular the Reitox network, identified 
duplications  in  the  data  provision  to  the  two  agencies.  The  parallel  reporting  streams  partly 
lead to discrepancies in data due to occasionally different individuals or departments reporting 
to  the  two  organisations.  It  was  suggested  by  some  consulted  stakeholders  that  the  Agency 
could support the Member States in their reporting to the United Nations Office on Drugs and 
Crime; however, some practical and legal issues would have to be clarified beforehand. In any 
case,  the  two  agencies  should  work  towards  further  aligning  the  different  reporting 
requirements and questionnaires. The Agency’s stakeholders also identified further scope for 
increased cooperation with international organisations on issues such as sharing best practices 
and adoption of common guidelines on data collection.  

99  
100  
101  
102  
103  

http://www.emcdda.europa.eu/about/partners/maoc.  
http://www.emcdda.europa.eu/about/partners/unodc.  
http://www.emcdda.europa.eu/about/partners/who.  
http://www.emcdda.europa.eu/about/partners/pompidou.  
http://www.emcdda.europa.eu/about/partners/cicad.  

34 

 
 
 
                                                           
E. 

EU ADDED VALUE 

The EU  added value  criterion  considers the benefits  that the Agency  brings as compared to 
what could have been done at national level. More specifically, it seeks to assess the Agency’s 
capacity  to  improve  the  ability  of  Member  States  to  monitor  and  respond  to  drug-related 
problems,  and  the  extent  to  which  it  is  a  valuable  source  of  information  for  its  main 
‘customers’.  This  section  also  explores  the  sustainability  of  the  activities  and  identifies  the 
advantage  of  attributing  these  tasks  to  an  agency  like  the  European  Monitoring  Centre  for 
Drugs and Drug Addiction as compared to other alternatives. It concludes by reflecting on the 
consequences of a potential termination of the Agency. 

Its  stakeholders  regard  the  Agency  as  the  main  EU-level  information  source  in  the  field  of 
drugs.  The  provision  of  factual,  objective,  reliable  and  comparable  information  is  the  main 
objective of the Agency based on its founding Regulation. The evaluation concluded that the 
Agency  has  an  excellent  reputation  as  a  source  of  scientific  and  comprehensive  quality 
data104,  not  only  in  Europe  but  also  on  an  international  level.  The  information,  which  is 
considered  objective  and  not  politically  influenced,  contributes  highly  to  the  policy-making 
on European and to a lesser extent on national level.  

The Agency contributes to informing the policy debate at EU level through the consolidation 
and dissemination of scientific and timely information, as well as through its participation in 
meetings  and  debates  where  drug  policies  are  discussed  (for  example  the  meetings  of  the 
Council’s  Horizontal Working  Party  on  Drugs).  The  references  made  to  the  Agency  and  its 
outputs  in  various  legislative  and  policy  documents  at  EU  level  show  its  tangible  impacts. 
This is especially visible with respect to new psychoactive substances, where the Agency has 
a  central  role  in  the  risk  assessment  of  new  substances,  and  the  action  plans  adopted  in  the 
framework  of  the  EU  Drugs  Strategy,  where  the  Agency  has  been  assigned  the  role  of 
responsible party in relation to several actions.  

The information provided by the Agency is heavily relying on the provision of data from the 
national  focal  points.  For  this  to  happen,  the  Agency  implemented  a  drug-related  data 
collection system that requires the national focal points to collect and subsequently report to 
the Agency the same set  of data using a common methodology for every reporting country. 
The data provided by the national focal points is then analysed by the Agency and provided in 
different forms, including reports, to its stakeholders. The EU-level overviews are used for the 
evidence-based  EU-level  policy-making,  but  are  also  valuable  for  national  stakeholders.  It 
allows  Member  States  not  only  to  design  evidence-based  policies  but  also  to  monitor  the 
effectiveness of their strategies, policies and interventions as well as to increase their capacity 
to  understand  and  to  respond  to  drug-related  threats.  For  the  Agency  to  be  in  a  position  to 
provide sufficient information on emerging threats and trends in drug demand and supply, it 
would  have  to  develop  additional  state-of-the-art  methodologies  to  get  the  underlying  data. 
This would make the Agency more responsive to emerging threats and the dynamic nature of 
the drug situation in the EU. 

One  example  of  such  additional  state-of-the-art  methodologies  is  wastewater  analysis.  This 
was  funded  originally  by  the  Commission  through  project-based  funding  to  research 
institutions  with  the  involvement  of  the  Agency.  If  such  analysis  should  be  continued, 
financial resources have to be made available to the Agency. Such new and innovative (state-

104  

The peer review of selected publication of the Agency resulted in an average score across the selected 
publications of 4.2 (1 being lowest/5 highest). 

35 

 
 
                                                           
of-the-art)  data  collection  methods  are  needed  to  ensure  that  a  complete  picture  of  the  drug 
phenomenon in Europe is available to policy-makers and that the Agency remains on top of 
the policy developments in the drugs field 

An important tool in this context is the Early Warning System (EWS) on new psychoactive 
substances. The Agency collects data provided by Member States, analyses the data and then 
shares  the  information  with  the  Early  Warning  System  network.  This  information  is  a  key 
component of many national early warning systems. 

Other  forms  of  support  to  the  national  level  include  specific  support  upon  request  of  a 
Member  State  to  help  them  design  or  monitor  national  drug  strategies  and  policies105;  the 
identification and sharing of best practice106 as well as training which is related to capacity-
building  exercises;  and  financial  support107.  When  it  comes  to  input  towards  national  drug 
policies,  the  evaluation  recommended  that  the  Agency  could  further  develop  its  tools  to 
support  Member  States  in  evaluating  their  national  drug  policies  and  strategies  and  in 
assessing the effectiveness of drug-related measures. 

The feedback received from consulted stakeholders, in particular from the reporting countries, 
showed  that  in  some  countries,  in  particular  those  with  a  shorter  tradition  in  doing  so,  the 
collection  of  drug-related  data  and  the  related  monitoring  might  not  survive  without  the 
Agency  in  place.  Consulted  stakeholders  expressed  clearly  that  without  the  common 
standards,  guidance  and  methodological  support  provided  by  the  Agency,  it  would  not  be 
possible to collect drug-related data with the same quality, accuracy and timeliness. A clear 
outcome  of  the  evaluation  was  that  without  the  help  and  support  of  the  Agency  such  a 
comprehensive and comparable data collection would not be possible.108 

Although  national  policy-makers  often  rely  on  the  work  of  the  Agency,  e.g.  as  regards 
identifying best practices, evidence-based information on hot topics or input to their national 
drug policies, the evaluation concluded that the Agency did not always manage to inform the 
national debates to the same extent as at EU level. In particular those Member States with a 
robust  national  drug  monitoring  system  do  not  necessarily  use  the  Agency’s  data  and 
information  as  the  primary  source  of  information  but  might  rely  more  on  their  national 
standalone  reports.  Several  Member  State  representatives  also  claimed  that  the  Agency  did 
not  succeed  in  triggering  political  shifts  on  a  national  level.  Therefore,  the  Agency  could 
enhance its engagement with national stakeholders and more proactively involve practitioners, 
including through training and exchange of best practices.  

As regards the sustainability of the activities, the main contributors to its sustainability are the 
nature  of  the  data  collected,  i.e.  the  comprehensive  nature  and  regular  updating  of  the 
information provided together with the use of new analytical methods, which allow for long 
series of statistical data; the political impact of the outputs through the provision of input for 
evidence-based  policy-making;  the  forward-looking  vision  of  the  EMCCDA;  and  the 
development  of  a  cooperation  network,  in  particular  with  the  national  focal  points.  Several 

105  
106  

107  

108  

In 2015, the Agency provided support to Germany, Ireland and Luxembourg. 
This  is  done  in  particular  through  the  Best  Practice  Portal  (http://www.emcdda.europa.eu/best-
practice_en),  but  also  through  gathering  the  relevant  experts  in  meetings  and  sharing  information  at 
such occasions.  
The  national  focal  points  are  co-financed  up  to  a  certain  amount  by  the  Agency  through  grant 
agreements. 
See  above  in  Section  VI.B.  (Effectiveness)  where  the  need  to  make  further  efforts  towards  a  better 
comparability of data was stressed. 

36 

 
 
                                                           
other elements further support the sustainability such as the expertise available in the Agency, 
the  development  of  methodological  tools  and  indicators  or  the  use  of  modern  technologies. 
The  stakeholders  identified  as  the  main  challenge  for  sustainability  going  forward  the 
availability of resources as limited resources will lead to the stopping of some of its activities. 
Another  challenge  is  that  keeping  information  relevant  for  future  challenges  requires 
significant  efforts  by  the  Agency  to  ensure  that  the  data  are  up-to-date  and  that  new 
information sources can be used.  

The question whether other existing or alternative options of implementing drug policy would 
have  been  more  efficient  in  achieving  the  objectives  set  out  in  the  EU  strategic  documents 
was assessed. The overall conclusion on this  question from  the external  evaluation  was that 
the Agency is the most effective option in achieving the objectives set out in the EU Drugs 
Strategy and its Action Plans.  

One  alternative  option  would  be  to  allocate  the  tasks  to  the  Commission  services  in 
cooperation with other EU agencies already active in the area (such as Europol, the European 
Medicines  Agency  or  the  European  Centre  for  Disease  Prevention  and  Control).  While  this 
option  would  still  ensure  a  regular  EU-level  data  collection  and  analysis,  it  has  two  main 
disadvantages  compared  to  the  current  set-up.  Firstly,  the  Commission  would  have  to 
outsource  some  of  the  activities,  which  would  lead  to  a  loss  of  institutional  capacity  and 
expertise in the longer term. Secondly, this loss of expertise might compromise the quality of 
the  data  analysis  and  thereby  might  lead  to  a  loss  of  trust  in  the  data  and  information 
provided. 

The  second  alternative  option  would  see  the  responsibility  for  monitoring  the  drug 
phenomenon reverting to the Member States. The Member States would need to take steps to 
coordinate  themselves  and  to  ensure  that  the  collection  of  existing  data  and  descriptive 
analysis continues. However, an agreement on common methodologies, indicators and quality 
standards would be more difficult to reach, compromising the comparability of data used to 
produce  EU-level  overviews.  In  addition,  the  reliability  and  impartiality  of  the  information 
might suffer as Member States with more expertise and resources would be able to influence 
the analysis and formulation of policies more than others. Finally, this option would also lead 
to  losing  the  Agency  as  a  major  actor  in  implementing  the  EU  Drugs  Strategy  and 
subsequently  would  make  it  more  difficult  for  the  EU  to  speak  with  one  voice  on  the 
international level. 

The third alternative option would be to attribute the activities (or at least some of them) to 
international organisations active in the field, such as the United Nations Office on Drugs and 
Crime,  the  World  Health  Organisation,  the  Council  of  Europe  or  others.  Although  the 
collection  of  data  would  be  maintained  with  a  central  organisation  to  coordinate,  it  would 
probably mean that the picture of the drugs phenomenon in Europe would be incomplete as 
less data on a regional level would be available. This might also lead to a loss of expertise and 
understanding of the influencing factors in Europe and to a diminished role of the EU on the 
international level. 

This links with the question about the most likely consequences of a potential termination of 
the  Agency.109  All  stakeholders  agreed  that  the  termination  of  the  Agency  would  have 
negative impacts and should be avoided. While at the first instance it would be policy-makers 

109  

For  more  information  on  the  impacts  of  a  potential  termination  of  the  Agency,  see  also  the 
counterfactual case study in Annex 4 of the final report of the external evaluation. 

37 

 
 
                                                           
who would be affected by such a termination, ultimately, the general public would also feel 
the impacts, albeit more indirectly.  

The  most  likely  consequence  would  be  a  loss  of  EU-level  information  because  of  the 
interruption  of  data  collection  and  analysis.  As  there  would  be  no  obligation  to  collect  the 
relevant data and no more centralised support, some reporting countries would cease to collect 
the data. Even those who would continue collecting drug-related data would likely do so to a 
more limited extent and would agree on the least common denominator. This would result in 
incomplete  and  fragmented  data.  The  exchange  of  best  practice  would  be  more  difficult 
without a central organisation bringing the information and people together. The cessation of 
the  Agency  would  undermine  the  possibility  to  design  and  implement  drug  policies  and 
strategies based on objective and scientific evidence, both at EU and at national level. Finally, 
the termination would have negative consequences on the role of the EU in the international 
sphere. It would make it more difficult to have a common understanding of the drug  

phenomenon  and  therefore  to  speak  with  one  voice  in  international  fora.  To  conclude,  the 
termination of the Agency would constitute an undesirable step backwards. 

VII.  CONCLUSIONS AND LESSONS LEARNED 

The overall conclusion of the evaluation, supported by an external study carried out between 
March and November 2018 by ICF Consulting Services Limited (ICF) in collaboration with 
the Centre for the Study of Democracy (CSD) and Optimity Advisors, was that the European 
Monitoring  Centre  for  Drugs  and  Drug  Addiction  (EMCDDA)  is  working  well.  The 
evaluation  was  positive  as  regards  all  five  evaluation  criteria,  but  further  improvements  are 
possible.  

The evaluation was carried out in accordance with the requirement set out in Article 23 of the 
founding Regulation,  which obliges the Commission  to  initiate an external  evaluation  every 
six years to coincide with the completion of two of the Agency’s 3-year strategies and work 
programmes.  

The evaluation concluded that both pillars of the Agency, i.e. public health and security, are 
well  covered  by  the  work  programmes  and  strategies.  The  Agency’s  work  programmes  and 
strategies, but also the outputs and outcomes produced, are relevant in relation to the overall 
objectives of EU drugs policy. The Agency meets the needs of its many stakeholders and was 
able to adapt to changes in the drug markets as needed. The Agency’s work is coherent with 
the EU drug policy objectives. Most synergies available with the work of the EU institutions, 
other  EU  agencies  and  international  organisations  are  already  used.  In  general,  only  few 
overlaps  of  work  exist  due  to  the  very  specific  work  area  of  the  Agency.  As  regards 
efficiency,  within  the  limitations  of  the  available  data,  the  evaluation  concluded  that  the 
Agency  used  the  available  human  and  financial  resources  efficiently  to  deliver  the  outputs, 
outcomes and impacts set out in its work programmes. 

The Agency is well recognised as a centre of excellence in providing information on the drugs 
phenomenon,  not  only  in  Europe  but  also  internationally.  The  work  of  the  Agency  also  has 
important  impacts  on  the  international  level,  including  bringing  the  EU’s  evidence-based 
policy-making in drug monitoring to third countries. The information produced is considered 
factual,  objective,  reliable  and  robust.  The  Reitox  network  is  seen  a  cornerstone  of  the 

38 

 
 
 
Agency’s work and activities as its members, the national focal points, are the providers of the 
relevant data. 

There is scope for greater engagement with some other stakeholder groups, in particular the 
scientific community and practitioners, in  order  to  better cover their needs in  the future. As 
regards  the  general  public,  more  direct  engagement  should  be  considered  to  improve  the 
visibility,  but  also  to  ensure  equal  information  across  Europe,  e.g.  on  new  psychoactive 
substances.  Outreach  and  communication  should  be  improved  in  particular  with  these 
stakeholder  groups.  Synergies  with  the  Commission  could  be  reaped  further  by  formalising 
the  working  arrangements  regarding  the  work  on  precursors,  developing  a  closer  structural 
relationship  when  it  comes  to  health-related  issues  and  cooperating  close  with  the  Joint 
Research Centre.  

Despite  the  very  positive  recognition  of  the  work  of  the  Agency  as  data  provider,  the 
evaluation  concluded  that  there  is  room  for  further  improvement.  During  the  interviews, 
stakeholders commented that more forward-looking information and products should be made 
available  in  order  to  better  help  EU  preparedness  and  response  in  this  fast  changing  policy 
area.  Although  many  efforts  have  been  made  following  the  previous  external  evaluation  on 
the comparability of data and considerable improvements have been implemented110, further 
efforts are needed to ensure better comparability of data. This is true similarly for supply side 
information, where considerable efforts have been made during the evaluation period, but the 
Agency’s  work  on  this  should  be  further  enhanced.  However,  more  work  on  drug  supply 
issues, including the further development of relevant indicators, would have to be matched by 
the  availability  of  additional  data,  including  through  the  provision  of  such  data  from  the 
national  focal  points.  Other areas, which  could  be addressed better and  where the Agency’s 
input  would  provide  added  value,  are  poly-drug  use  and  the  support  to  Member  States  in 
evaluating their national drug policies.  

More targeted support, e.g. through Reitox Academies, is considered useful to develop further 
the  knowledge  among  the  staff  of  the  national  focal  points.  Several  consulted  stakeholders 
suggested  the  enhancement  of  the  IT  systems  for  data  collection,  in  particular  the  FONTE-
system. 

The  use  of  visual  aids  in  reports  was  acknowledged  positively,  but  a  few  stakeholders 
suggested to further streamline the provision of information and to make it more user-friendly 
and  interactive.  The  quality  of  the  translation  was  put  forward  as  a  concern.  It  has  to  be 
acknowledged that translations are very costly and therefore a good balance in selecting the 
products  for  translation  has  to  be  found.  However,  whenever  translations  are  made,  they 
should be of good quality. 

The  evaluation  was  inconclusive  on  the  potential  future  broadening  of  the  scope  of  the 
Agency to other licit and illicit substance and addictive behaviours. 

The Agency provides good value for money; however, there is room for improvement when it 
comes  to  simplification  of  the  administrative  set-up  and  the  internal  working  methods. 
Despite  the  available  budget  remaining  relatively  flat  over  the  evaluation  period  and  the 
increased demands during the same time, the Agency continued to  deliver excellent  outputs 
due  to  a  reprioritisation  of  its  work  and  the  related  redeployment  of  resources.  The  key 

110  

The Agency provided handbooks, guidelines and procedures to Member States in an effort to harmonise 
data  collection  and  reporting.  All  Member  States  use  the  same  protocols  for  the  collection  of  key 
indicators.  

39 

 
 
                                                           
bottlenecks as regards the organisation were identified as the administrative burden which is 
considered (by the staff) to be too heavy for an agency of the size of the Agency; the need to 
review  and  modernise  information  and  communication  technology  (ICT)  tools;  the  need  to 
introduce  Activity  Based  Management  and  related  budgeting;  and  better  internal  planning. 
The key performance indicators should be streamlined and developed further. 

The EU added value of the work of the Agency is very high. The evaluation clearly concluded 
that the Agency is instrumental for continued collection of good scientific, comprehensive and 
(largely) comparable quality data. No other institution, on international, EU or national level, 
or  other  EU  agency  could  take  over  the  work  of  the  Agency  with  the  same  results.  All 
stakeholders considered the possibility of a termination of the Agency as very negative.  

As regards international cooperation, overlaps have been identified with the data provision to 
the Agency and the United Nations Office on Drugs and Crime, which should be addressed, 
together with the questions regarding some legal and methodological difficulties. The Agency 
should  also  address  the  fact  that  data  provision  to  international  organisations  is  currently 
mainly  unidirectional.  Overall,  the  consultation  suggested  that  more  could  be  done  on  the 
international sphere. Therefore, the Agency should (re-)consider the priority setting as regards 
its  cooperation  with  third  countries,  taking  into  account  the  EU  policies  in  the  area  and  the 
understanding  the  Agency  might  gain  from  working  with  a  third  country  on  the  EU  drug 
markets.  

Some of the issues set out above are already addressed by the “EMCDDA Strategy 2025” or 
are considered for implementation in the Agency’s most recent work programme as they have 
been acknowledged by the Agency already as areas for improvement. However, as the budget 
of  the  Agency  is  constraint,  it  would  not  be  possible  for  the  Agency  to  do  more  with  the 
currently  attributed  human  and  financial  resources111.  It  should  be  clearly  noted  that  this 
applies to activities and tasks covered by the current mandate. 

111  

budget; 

For the EU contributions to the budget of the Agency, see the Financial Statements which are part of the 
EU 
of 
https://ec.europa.eu/info/sites/info/files/about_the_european_commission/eu_budget/draft-budget-2019-
legal-personality-public-private-ownership-com-2018-600_2018_en.pdf.  

335-342 

budget 

annual 

2019 

page 

see 

the 

for 

40 

 
 
  
 
                                                           
VIII.   ANNEXES 

ANNEX I – PROCEDURAL INFORMATION 

The  external  evaluation  of  the  European  Monitoring  Centre  for  Drugs  and  Drug  Addiction 
(EMCDDA)  has  been  carried  out  in  the  period  April  to  November  2018.  Preparatory  steps, 
such as the roadmap of the initiative112 and the Terms of Reference for hiring a contractor113, 
were  initiated  in  summer  2017.  The  Management  Board  was  informed  about  the  upcoming 
evaluation  at  its  meeting  of  December  2017.  The  agenda  planning  reference  for  this 
evaluation is PLAN/2017/1680.  

An  inter-service  steering  group  was  set  up  for  the  evaluation.  The  following  Directorates-
General  (DGs)  and  services  followed  the  invitation  to  nominate  a  participant  in  the  inter-
service  steering  group:  the  Secretariat-General  of  the  Commission  (SG),  the  Legal  Service 
(LS),  DG  Human  Resources  and  Security  (HR),  DG  Budget  (BUDG),  DG  Taxation  and 
Customs Union (TAXUD), DG Neighbourhood and Enlargement Negotiations (NEAR), DG 
Health and Food Safety (SANTE), DG Internal Market, Industry, Entrepreneurship and SMEs 
(GROW),  DG  Mobility  and  Transport  (MOVE),  DG  Research  and  Innovation  (RTD),  the 
Joint Research Centre (JRC), Eurostat and the European External Action Service (EEAS).  

In addition, an ad hoc group was set up to allow an inclusive evaluation process. The ad hoc 
group consisted of the Chair and the Vice-Chair of the Agency’s Management Board and two 
staff members of the Agency.  

DG  Migration  and  Home  Affairs  (DG  HOME)  chaired  the  meetings  of  the  inter-service 
steering group and of the ad hoc group. Both groups were systemically consulted during the 
evaluation process taking into account the crosscutting nature of drug policies in the EU. 

The  inter-service  steering  group  was  consulted  by  e-mail  on  the  Terms  of  Reference  in 
December 2017 with all comments having been taken into account. The members of the inter-
service  steering  group  were  invited  to  the  kick-off  meeting  with  the  contractor,  which  took 
place on 28 March 2018 (DGs NEAR, RTD and the EEAS participated in the meeting). The 
questionnaire for the public consultation, for which the contractor prepared a first draft, was 
sent to the inter-service steering group for input in April 2018.  

Following the submission of the inception report, back-to-back meetings with the contractors 
and the ad hoc group as well as the inter-service steering group took place on 25 April 2018. 
The  draft  interim  report  was  discussed  with  the  ad  hoc  group  on  6  July  and  with  the  inter-
service group on 10 July 2018. The meetings on the draft final report took place back-to-back 
on 28 September 2018. All meetings took place in the presence of the contractor. 

Both,  the  ad  hoc  group  as  well  as  the  inter-service  steering  group,  had  the  opportunity  to 
provide  their  comments  on  the  different  draft  reports,  ask  for  clarifications  and  submit 

112  

113  

https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5812828_en.  The 
roadmap  was 
published  on  28  November  2017  for  the  4-week  feedback  period.  No  feedback  was  provided  by 
stakeholders. 
The services were requested within framework contract HOME/2015/EVAL/02. 2 possible contractors 
applied  for  carrying  out  the  evaluation.  The  chosen  contractor  was  ICF  Consulting  Services  Limited 
(ICF)  who collaborated with the Centre for the Study of Democracy (CSD) and Optimity Advisors in 
carrying out the external evaluation. 

41 

 
 
 
                                                           
comments on the developments of the external evaluation. They received the draft and revised 
inception  report,  the  draft  and  revised  interim  report  as  well  as  the  draft  final  report  for 
comments and review, before the reports could be deemed approved.  

The final report of the external evaluation was submitted to the Commission in the beginning 
of November 2018. 

Further  details  on  the  evidence  used  in  the  evaluation  as  well  as  the  discussion  of  its 
limitations  are  detailed  in  Annex  II  on  "Methods  and  Sources"  and  in  Annex  III  on 
"Stakeholder consultation". 

42 

 
 
 
 
 
ANNEX II – METHODS, SOURCES AND LIMITATIONS 

(1)  Methods and sources 

In  order  to  gather  the  necessary  information  for  the  external  evaluation,  the  contractor 
undertook an extensive desk research, carried out interviews, undertook two targeted surveys, 
peer-reviewed  several  Agency  reports  and  undertook  case  studies.  The  external  evaluation 
was informed also by the results of the public consultation and a workshop. 

(a) 

Desk research 

The contractor reviewed all programming and reporting documents available on the Agency’s 
website,  together  with  those  uploaded  on  the  online  portal  specifically  set  up  for  the 
evaluation. The reviewed documents are listed in Annex 2 of the final report of the external 
evaluation. Annex 3 of the final report of the external evaluation provides in detail the results 
of the desk research, in particular the mapping tables. 

(b) 

Interviews, surveys and public consultation 

See below in Annex III for details on the stakeholder consultation process. 

(c) 

Peer review 

Experts reviewed the effectiveness, relevance and EU added value of selected publications114, 
based on the agreed methodology outlined in the inception report. For each output, the experts 
completed  a  template  to  assess  each  criterion.  More  specifically,  they  provided  a  score  and 
short written assessment of the key questions under each  criterion.  The contractor  reviewed 
the  templates  and  the  outcome  of  the  peer  review  exercise  was  incorporated  into  the  final 
report. 

The peer review of the selected publication of the Agency resulted in an average score across 
the  selected  publications  of  4.2  (1  being  lowest/5  highest).115  It  showed  that  –  across  the 
documents reviewed – there was a clear orientation of the information to the respective target 
group. The experts rated the scientific quality of the information provided by EMCDDA very 
highly for robustness, traceability (referencing) and evidence base. For some documents, the 

114  

115  

1.  Flagship  products:  EU  Drug  Market  Report  (jointly  produced  with  Europol),  Health  and  Social 
Responses Guide. 2. Routine reporting activities: Rapid communication on the MDMA market, Rapid 
Communication on Drug-related infectious diseases, Assessing illicit drugs in wastewater, The Internet 
and  Drug  Markets  (2016),  Drugs  and  the  Darknet:  perspectives  for  enforcement,  research  and  policy 
(2017),  Drug  impaired  driving,  EMCDDA  7-step  guide  to  aid  drug  policy  evaluation,  Rapid  Policy 
review: Ireland, Take-home naloxone programmes. 3. Products from the work of the Early Warning and 
Risk  Assessment  Activities: Substance:  Acryloylfentanyl a) EMCDDA Formal Notification to the EU 
EWS  Network  b)  EMCDDA  Risk  Communication  c)  EMCDDA-Europol  Joint  Report  d)  EMCDDA 
Risk  Assessment  Report;  Substance:  Carfentanil  a)  EMCDDA  Risk  Communication,  b)  EMCDDA-
Europol Joint Report, c) EMCDDA Risk Assessment Report. 
Other  key  scores  across  the  peer-reviewed  publications:  user-friendliness  and  readability:  4.2;  factual 
and  objective  information:  4.4;  accurate  and  credible  information  based  on  robust  evidence:  4.3; 
comparability: 3.6; traceability of information (e.g. referencing): 4; clearly outlined methodology: 4.3; 
accessibility of outputs: 4.3. 

43 

 
 
                                                           
experts concluded that they were particularly important to introduce policy-makers to general 
drug policy principles or to close the gaps in terms of reliable data at national level.116 

(d) 

Case studies 

The  contractor  developed  five  case  studies  (thematic,  organisational,  benchmarking, 
counterfactual  and  impact)  by  conducting  additional  interviews,  carrying  out  desk  research, 
and taking into account the outcomes of the peer review exercise and the conclusions from the 
workshop.  The  case  studies  are  included  as  Annex  4  in  the  final  report  of  the  external 
evaluation. 

Thematic case study117 

The objective of the thematic case study was to explore the themes that could be relevant for 
the Agency if any broadening of its scope was to  be considered in  the future to  enhance its 
preventative  action  and  achieve  its  strategic  goals.  The  case  study  summarised  the  method 
used  to  produce  the  report  and  discussed  the  factors  influencing  any  expansion  of  scope.  It 
also  presented  four  main  themes  identified  as  potential  areas  for  expansion  (alcohol 
consumption, tobacco consumption, prescription medicines and gambling). 

The  thematic  case  study  concluded  that  given  its  existing  remit  and  responsibilities,  the 
Agency  might  be  able  to  play  a  role  in  assessment,  monitoring  and  intervention  of  other 
addictive behaviours. The complexity of the drug phenomenon is  illustrated by the fact  that 
many of the factors that are linked to drug use and/or exacerbate drug problems are broader 
social  issues,  including  those  associated  with  other  addictions  (i.e.  alcohol  or  prescription 
medicines) or addictive behaviours such as gambling.118  

As  the  “EMCDDA  Strategy  2025”  highlights,  the  Agency  should  have  a  more  strategic, 
situational and holistic understanding of the European drug situation and its implications if it 
is to achieve its vision of a healthier and more secure Europe. The EU Drugs Strategy 0 sets 
out the need for prevention measures, early detection and intervention, promotion of healthy 
lifestyles,  and  targeted  prevention  directed  at  families  and  communities.  Poly-drug  use  is 
already  mentioned  as  a  key  priority  for  the  Agency  in  its  founding  Regulation  and  this 
provides  an  appropriate  framework  to  address  some  of  the  four  themes  discussed  in  the 
thematic case study.119  

If  a proposal  to  extend the mandate would be made in  the future, further considerations are 
needed on what would be of most benefit for the Agency and its stakeholders, and what would 
be  possible  within  the  resources  available.  Part  of  this  will  include  understanding  possible 

116  

117  

118  
119  

The peer-reviewed publications scored on average 4.15 on relevance. The topics of the peer-reviewed 
documents reflect key stakeholder priorities with only two of 18 peer-reviewed outputs scoring below 4. 
The “Internet and the Drug Markets” report and the joint reports between the Agency and Europol on 
two new psychoactive substances were rated as most relevant (received the maximum score of 5 on all 
peer  review  questions  pertaining  to  relevance).  The  lowest  score  on  relevance  was  given  to  “Drug 
Impaired Driving” (3.5/5), mostly because it was not clear which target group the report is aimed at and 
whether it reflects key stakeholder priorities. 
As a summary of the arguments put forward, see Table 7 in Annex 3 of the final report of the external 
evaluation. 
See also Recital 3 of the founding Regulation. 
For example, it is very clear that alcohol consumption is intrinsically linked to many drug issues, which 
has led to efforts to encourage and undertake joint analysis within the European School Survey Project 
on Alcohol and Other Drugs (ESPAD). 

44 

 
 
                                                           
synergies  with  partners  working  in  the  same  area  and  ways  in  which  the  Agency  can 
contribute  to  existing  work,  e.g.  through  exchange  of  information  compared  to  full 
participation or improving data collection systems. 

Organisational case study 

The  objective  of  the  organisational  case  study  was  to  assess  how  cooperation  with 
international  stakeholders  has  contributed  to  the  core  activities  and  the  implementation  of 
various relevant  strategies where international  cooperation is  an explicit objective. The case 
study  specifically  assessed  how  the  Agency’s  organisation  can  be  adapted  to  provide  a 
comprehensive and up-to-date understanding of the drugs market in Europe, by analysing the 
impact of developments in the international drugs markets on the European drugs market. 

The  organisational  case  study  concluded  that  the  “EMCDDA  Strategy  for  International 
Cooperation”  has  been  largely  implemented.  The  Agency  has  worked  towards  the 
achievement of all three objectives, i.e. (1) better assessment of the global drug situation, (2) 
improving the understanding of the drug situation in  third countries,  and (3) supporting  EU 
policies and initiatives in the drugs field. The Agency has become internationally recognised 
for  the  quality  and  scientific  rigour  of  its  work  and  is  recognised  as  a  key  partner  for 
international  organisations,  non-governmental  organisations  and  third  country  government 
agencies  working  in  the  field  of  drugs.  In  many  cases,  Member  States  continue  to  work 
directly  with  international  organisations,  whose  activities  often  overlap  with  those  of  the 
Agency.  

The case study also found that the “EMCDDA Strategy for International Cooperation” did not 
prioritise  the  needs  of  the  Agency  with  regard  to  the  benefit  of  engaging  in  international 
cooperation activities, especially with third countries, nor did it clearly outline the principles 
on which to base the priorities for cooperation, in view of its limited resources. The objectives 
and key principles of the “EMCDDA International Cooperation Framework 2018-25” provide 
a much more aligned and focused approach to international cooperation.  

The assessment of cooperation activities between 2013 and 2017 shows that: 

  International  cooperation  remains  a  valuable  means  for  the  Agency  to  spread  its 
methodological and scientific standards, as well as to understand better the global drug 
situation, which shapes the EU’s drug issues. 

 

 Cooperation  with  international  organisations  focused  primarily  on  working  with  the 
United Nations Office on Drugs and Crime and the World Health Organisation, and to 
a much lesser degree with other organisations.  

  International  cooperation  activities  directly  contributed  to  a  significant  extent  to 
certain core activities, including: a) the improved understanding of global issues linked 
to  third  countries  where  the  Agency  does  not  have  cooperation  agreements  (e.g.  on 
heroin  issues);  and  b)  receiving  support  from  international  organisations  in 
implementing activities in enlargement countries.  

  All  stakeholders  noted  the  duplication  of  data  collection  with  the  United  Nations 

Office on Drugs and Crime as something that needs to be addressed. 

45 

 
 
  The  cooperation  with  third  countries  beyond  candidate  and  neighbourhood  countries 
was,  and  will  remain,  limited.  There  is  scope  for  better  alignment  with  the  EU’s 
Global  Strategy,  increasing  cooperation  with  priority  regions  (such  as  Central  and 
South Asia, Middle East, and Africa) where issues of drugs, migration and terrorism 
intertwine.  

The  strategic  decision,  outlined  in  the  “EMCDDA  International  Cooperation  Framework 
2018-25”, to rely primarily on additional or project-based funding limits the sustainability of 
its  activities  and  the  extent  to  which  the  Agency  can  build  and  maintain  relationships  with 
third countries and international organisations. Future international cooperation activities may 
need  to  demonstrate  more  direct  evidence  to  satisfy  the  underpinning  principles  of  the 
Agency’s cooperation with international stakeholders. 

Benchmarking case study 

The objective of the benchmarking case study was to draw conclusions on the efficiency and 
effectiveness  of  the  Agency  by  comparison  with  another  EU  agency.  The  benchmarking 
provided  a  comparative  understanding  of  key  aspects  of  the  Agency’s  operations  and 
structure.  It  also  helped  to  identify  best  practices  and  recommendations  on  potential 
improvements  to  areas  where  inefficiencies  are  identified.  The  benchmarking  case  study 
compared the Agency with the Fundamental Rights Agency (FRA). 

The  benchmarking  case  study  concluded  that  the  two  agencies  have  similar  financial 
structures and are facing similar challenges: increased demand for their products and services 
while  available  financial  resources  remain  relatively  flat.  Both  agencies  have  been  able  to 
meet these challenges within available budgets.120  

Both  agencies  appear  to  be  facilitating  the  networks  they  support  in  an  efficient  way, 
considering their existing needs and working methods. The analysis shows certain  room for 
improvement  in  mechanisms  ensuring  the  quality  of  data  and  analysis  the  Agency  receives 
from Reitox. 

Despite appearing more burdensome and bureaucratic compared to the organisational set-up 
of the Fundamental Rights Agency, the Agency’s governance structure is more conducive to a 
system  of  checks  and  balances,  where  there  are  a  number  of  bodies  on  its  several 
organisational layers evaluating performance and assuring quality of outputs.  

Both agencies have developed communication strategies and boast high satisfaction rates for 
their communication and outreach activities.121 The Fundamental Rights Agency has a wider 
online  and  social  media  reach.122  It  is  focused  more  on  national  stakeholders,  whereas  this 
category  is  less  well  reached  by  the  Agency.  The  Fundamental  Rights  Agency  facilitates 
actors  at  national  level  to  consistently  communicate  and  implement  fundamental  rights 
initiatives.  

120  

121  

122  

It should be noted that the staff numbers of the Fundamental Rights Agency have grown by 15% from 
2013  to  2017,  while  the  Agency’s  remained  unchanged.  Cost  for  staff  (per  full-time  equivalent)  is 
higher  at  the  Fundamental  Rights  Agency,  but  this  may  be  due  to  the  differences  in  the  correction 
coefficients  applied  to  remuneration  of  expatriate  officials  in  the  Member  States  hosting  the  two 
organisations. 
In  terms  of  dedicated  resources,  the  Fundamental  Rights  Agency  has  a  much  larger  communications 
budget (two-three times higher than that of the Agency). 
However, when it comes to YouTube followers, the Agency has a better outreach than the Fundamental 
Rights Agency. 

46 

 
 
                                                           
The  content  of  the  two  agencies  is  highly  valued  by  their  stakeholders.  The  Fundamental 
Rights Agency produces many more diversified outputs, repurposing content to reach a wider 
array  of  stakeholders.123  The  two  organisations  have  different  strengths  in  this  regard:  the 
specialisation  of  the  Agency  is  conducive  to  the  production  of  highly  specialised  scientific 
content and larger, in-depth publications, while the Fundamental Rights Agency’s wider array 
of  topics  is  conducive  to  the  production  of  many  shorter  products,  which  are  less  technical 
and can be targeted towards a larger audience. 

Impact case study 

The objective of the impact case study assessed the contribution of the Agency’s activities to 
delivering  intended  impacts.  It  focused  on  two  specific  impacts  based  on  the  overarching 
objectives,  which  cover  both  the  drug  demand  and  drug  supply  aspects  of  the  Agency’s 
activities, namely: development of national-level drug demand/ harm reduction strategies and 
policies; and improved EU capacity to act on drug supply reduction measures. 

As  regards  the  contribution  of  the  Agency  to  demand  side  policy  developments  at  national 
level,  the  impact  case  study  concluded  that  national  assessments,  evaluations,  general  and 
thematic  reporting  and  the  Best  Practice  Portal  are  all  viable  routes  for  providing  national 
level  stakeholders  with  information  and  delivering  results.  The  strength  of  this  relationship 
appears  to  diminish  as  the  number  of  harm  reduction  responses  increases.  There  are  many 
Member  States  with  fewer  harm  reduction  responses,  which  develop  their  strategies  and 
policies  based  on  information  provided  by  the  Agency,  whereas  other  Member  States  rely 
more heavily on data collected and reports published at national level. Overall, there appears 
to be a causal link between the activities, outputs, results and impact on policy development, 
to which the Agency contributes moderately to strongly.   

The contribution of the Agency to impact policy development depends greatly on the national 
political  environment  and  the  coherence  of  the  attitudes  and  strategies  across  national 
institutions  operating  in  the  field  of  drug  demand/  harm  reduction.  The  Agency  has  limited 
scope to translate the evidence considered by policy makers directly into tangible policy.  

As  regards  the  capacity  of  the  EU  to  take  action  on  drug  supply  reduction,  the  impact  case 
study concluded that the measures to enhance this capacity were the provision of an overview 
of  drug  precursors,  the  EU  Drug  Markets  Report  and  the  risk  assessments  on  new 
psychoactive substances.   

Combatting drug precursor diversion and trafficking is an objective of the EU Drugs Strategy 
and its Action Plans. The information on drug precursors is presented in the EU Drug Markets 
Report. The impact case study suggests that the Agency’s presentation of information on drug 
precursors  in  a  wider  EU  context  makes  a  moderate  contribution  to  precursor  scheduling 
decisions.  The  establishment  of  formal  working  arrangements  with  DG  TAXUD  and  DG 
GROW, together with continued and comprehensive reporting of precursors in the context of 
the  EU  drug  situation  as  supporting  evidence  in  scheduling  considerations,  could  provide 
further impact as supply reduction measures. 

The EU Drug Markets Report provides a holistic overview of the drug market across Europe. 
The  impact  case  study  found  that  outputs  provide  key  stakeholders  with  information  on 
emerging  trends,  although  this  tends  to  be  somewhat  historical.  There  is  evidence  that  the 

123  

Due to its larger amount of publications, the Fundamental Rights Agency has a lower cost per output. 

47 

 
 
                                                           
information  from  the EU Drug Markets  Report  fed directly into the development  of the EU 
drugs  policy  and  in  Europol’s  Serious  and  Organised  Crime  Threat  Assessment  (SOCTA).  
The collaboration with Europol and the direct input into the EU Policy Cycle means that the 
Agency  is  well  placed  to  strongly  contribute  to  EU-level  drug  supply  reduction  measures, 
including via the EU Drug Markets Report. 

The  Agency  examines  all  available  data  to  produce  risk  assessments  of  new  psychoactive 
substances  as  part  of  the  Early  Warning  System.  There  is  a  direct  causal  link  between  the 
production of the risk assessments and its consequent submission to control measures at EU-
level. Thus, the Agency’s role in conducting risk assessments of new psychoactive substances 
has a significant impact on supply reduction measures. 

Counterfactual case study 

The  objective  of  the  counterfactual  case  study  was  to  provide  an  in-depth  response  to  the 
question  ‘What  would  happen  if  the  Agency  did  not  exist?’  This  case  study  aimed  to 
contribute to the analysis of the EU added value evaluation criteria. It identified and described 
the  possible  consequences  and  impacts  of  the  absence  of  the  Agency’s  functions  and 
activities, at both EU and Member State level.  

The  analysis  and  data  presented  in  the  counterfactual  case  study  on  the  Member  States’ 
capacity  to  monitor  the  drug  situation  and  produce  and  share  early  warning  information  on 
new psychoactive substances shows that the Agency adds value and without its support most 
Member States and the EU as whole would achieve less in drugs policy.  

As regards the Member States’ capacity to monitor the drug situation, the sample of countries 
reviewed124 suggests that: 

 

 Some  Member  States  produce  detailed  and  high-quality  information  on  the  drug 
situation, that meets their individual needs. 

  EU-level  information  and  the  ability  to  compare  data  with  other  Member  States  is 
crucial in helping countries to understand how the domestic drugs situation compares 
to other Member State. This, in turn, helps Member States to determine whether or not 
their policies need to be further improved.  

  The Agency has a significantly greater analytical and expert capacity and competence 
than many Member States and can thus more easily produce high-quality analysis or 
develop methods and standards for harmonisation.  

With respect to Member States’ capacity to maintain a national early warning system, it can 
be concluded from the counterfactual case study that: 

  National early warning systems in most Member States would be much less effective 

if EU-level data were absent.  

  An EU-level mechanism would function far less effectively without the Agency’s role 
in collecting and analysing the information before sharing it with all Member States. 
The value of simply exchanging national signals or information between national early 

124  

Belgium,  Bulgaria,  Germany,  France,  Ireland,  Netherlands,  Poland,  Spain,  Sweden  and  the  United 
Kingdom. 

48 

 
 
                                                           
warning systems is much lower than the value of the current Early Warning System, 
which is based on EU-level analysis.  

  Even  Member  States  with  pre-existing  or  well-developed  national  early  warning 
system see value in the EU-level aggregation and analysis of information from other 
Member States. 

(e)  Workshop 

A workshop took place on 6 September 2018, gathering the evaluation team of the contractor, 
the panel of four independent experts, who undertook the peer reviews, and the ad hoc group. 
The intention of the workshop was to have an open discussion on the preliminary results of 
the external evaluation and possible recommendations. The discussion took place based on a 
short background note.  

(2) 

Limitations and robustness of the findings 

(a) 

General limitations 

The  following  limitations  should  be  noted  with  respect  to  the  data  collection  activities 
undertaken by the contractors: 

  Quality  of  baseline  in  the  previous  external  evaluation:  Although  efforts  were  made 
by the contractor to compare the results to the previous evaluation period (2007-2012), 
some  aspects  examined  were  omitted  or  only  partially  examined  in  the  previous 
external  evaluation.  Given  the  length  of  time  that  has  elapsed  since  the  2012 
evaluation,  it  was  difficult  to  re-establish  the  baseline  where  it  was  found  to  be 
lacking.     

  Qualitative  data  interviews  presented  as  quantitative  (open-ended  questions):  While 
survey data shows the share of responses in quantitative terms, the interview evidence 
presented  in  quantitative  terms  (i.e.  number  of  respondents)  should  be  treated  with 
caution.  This  is  because  interviewees  were  asked  open-ended  questions  and  the  fact 
that  some  interviewees  pointed  to  a  specific  issue  does  not  mean  that  those 
respondents  who  did  not  give  the  same  response  disagree,  but,  rather,  may  not  have 
thought about the same issue during the interview.   

  Slight  variations  in  the  question  on  expanding  the  competencies:  There  were  slight 
variations in the formulation of the question related to the potential expansion of the 
competences across various data collection templates (the two  surveys as well as the 
interview  questionnaires  targeting  different  stakeholder  groups).  For  example,  the 
question  was  sometimes  phrased  as  ‘Can  the  EMCDDA  broaden  ...’,  and  in  other 
cases ‘Should the EMCDDA broaden ...’. Similarly, the addictive behaviour question 
was  asked  sometimes  as  a  separate  question  and  other  times  included  in  the  overall 
question.  

  Short duration of the evaluation: At the outset of the study, the short duration of the 
evaluation, including the summer holiday period, was highlighted as a potential risk. 

49 

 
 
 
However, 
programme in due time.  

the  contractor  successfully  completed 

the  stakeholder  consultation 

  Low  response  rate  to  the  external  survey:  Only  26  respondents  provided  answers  to 
the  external  survey  with  the  scientific  sector  and  civil  society.  This  was  due  to  the 
public  consultations  running  in  parallel  with  the  survey  and  being  promoted  more 
actively. However, as the same stakeholder group responded to the public consultation 
and  a  selected  number  of  respondents  were  interviewed,  the  contractor  collected 
sufficient evidence of the views of these stakeholder groups.  

  Low  response  rate  regarding  some  specific  questions:  Some  questions  that  required 
greater  familiarity  with  the  processes  of  the  Agency  were  difficult  for  certain 
stakeholders to answer. For instance, only the staff of the Agency could answer most 
of  the  questions  related  to  efficiency,  so  there  were  limited  stakeholder  views  as 
evidence on several questions. The contractor used desk research and literature review 
to mitigate this absence and close any gaps in evidence.  

  Stakeholder bias: The risk of bias among any particular group of stakeholders was low 
because  all  relevant  stakeholder  groups  were  covered  sufficiently  through  different 
data collection methods. However, there is a slight risk of positive bias, as many of the 
consulted  stakeholder  groups  work  closely  with  the  Agency  and/or  benefit  from  its 
outputs. 

(b) 

Budget limitations and constraints 

  Constraints  set  by  the  Multiannual  Financial  Framework  (MFF):  The  Commission 
proposals for the next Multiannual Financial Framework 2021-2027 were adopted on 
2  May  2018125,  including  an  indicative  envelope  for  the  expenditure  of  the 
decentralised agencies. This sets a budgetary constraint on the involvement in future 
initiatives. However, it was agreed at the inception meeting that these considerations 
should  not  limit  the  contractor’s  thinking,  but  that  the  budget  limitations  should  be 
included as contextualisation in the evaluation. 

  Cost-effectiveness  ratios  to  measure  the  performance:  Over  the  course  of  the 
evaluation  period,  the  Agency  used  different  key  performance  indicators  (KPIs)  and 
different  indicators,  making  like-for-like  comparisons  difficult.  The  lack  of  Activity 
Based Budgets also created challenges to conduct a cost-effectiveness analysis and/or 
to compare the Agency’s efficiency with those of other agencies. To mitigate this, the 
contractor focused on headline indicators (staff per function, budgeted expenditure per 
core activity, etc.). 

125  

COM(2018) 321; see for more details and all relevant documents regarding the Multiannual Financial 
Framework 2021-27 http://ec.europa.eu/budget/mff/index_en.cfm.  

50 

 
 
 
 
 
                                                           
ANNEX III – STAKEHOLDER CONSULTATION 

A broad stakeholder consultation accompanied the external evaluation. The aim of the process 
was to obtain views on the five evaluation criteria, on the functioning of the Agency and on 
possible future challenges  and developments  from  a range of stakeholders working with  the 
Agency or benefitting from the work of the Agency. The contractor carried out the targeted 
stakeholder  consultation.  The  Commission's  standards  for  Stakeholder  Consultations  were 
duly taken into account and met. 

As part of the stakeholder consultation, the Commission conducted a public consultation, with 
the  purpose  of  gathering  views  from  private  individuals,  organisations,  the  industry  and  the 
public administration. 

The replies to the stakeholder consultation addressed all evaluation questions. The outcomes 
of  the  interviews,  the  targeted  surveys  and  the  public  consultation  are  reflected  fully  in  the 
external  evaluation  study  and  in  this  Staff  Working  Document.  Summarising  them  in  this 
Annex would be a repetition of the conclusions of the external evaluation set out in the main 
text of this document. 

(1) 

Targeted stakeholder consultations (interviews and surveys) 

Targeted  consultations  took  place  within  the  framework  of  the  external  evaluation  and 
involved  different  types  of  stakeholders.  The  contractor  consulted  with  representatives  of 
Member  States,  the  Agency  staff,  EU  Institutions  and  agencies,  EU  funded  projects, 
international organisations, third countries, the scientific community and civil society. These 
targeted  consultations  were  conducted  mainly  via  interviews  and  partly  also  through  a 
targeted  survey  (the  civil  society  and  the  scientific  community,  on  the  one  hand,  and  the 
Agency  staff,  on  the  other  hand).  The  interviews  were  accompanied,  where  necessary,  by  a 
written questionnaire to collect quantitative data.  

133 stakeholders were interviewed.126 In 16 instances the interview could not be completed as 
the  envisaged  interviewee  either  did  not  reply  to  the  request  for  an  interview,  declined  the 
interview  or  it  was  not  possible  to  complete  the  interview  for  other  reasons.  A  list  of 
interviews carried out is available in Annex 1 of the final report of the external evaluation. 

The following list indicates the main targeted stakeholder consultations: 

(a)  Member States 

At Member State level, three different types of interviewees were consulted: members of the 
Agency’s  Management  Board,  Member  State  representatives  in  the  Council’s  Horizontal 
Working Party on Drugs (HDG) and the Reitox national focal points.  

Interviews carried out: 

a)  Members of the Horizontal Working Party on Drugs: 23 interviews 
b)  Reitox national focal points: 30 interviews 

126  

The actual number of interviews conducted differs as some of the Member State representatives belong 
to  various  groups  of  stakeholders  (i.e.  members  of  the  Horizontal  Working  Party  on  Drugs,  National 
Focal  Points  and  Management  Board  members),  meaning  that  in  some  cases  various  groups  of 
stakeholders were covered by a single interview. 

51 

 
 
                                                           
c)  Management Board members: 22 interviews 

(b) 

Staff 

Interviews  with  a  selected  number  of  staff  took  place  (as  group  interviews).  11  interviews 
were carried out in total with staff. 

In  addition,  the staff was consulted through a targeted survey. This  online survey  was open 
from  25  June  to  13  August  2018.  It  was  circulated  to  all  the  staff.  The  survey  received  55 
responses, representing 55% of the target group. 

(c)  

European institutions and bodies and EU-funded projects 

At EU level, different types of institutions and bodies were consulted. Interviews were carried 
out  with  the  European  Commission  (DG  HOME,  DG  NEAR,  OLAF,  DG  TAXUD,  DG 
GROW, JRC, DG SANTE and DG DEVCO), the European External Action Service (EEAS), 
the  Council  of  the  European  Union  (Secretariat),  Members  of  the  European  Parliament,  the 
relevant EU agencies (Europol, European Medicines Agency (EMA), the European Centre for 
Disease Prevention and Control (ECDC), the European Union Agency for Law Enforcement 
Training  (CEPOL),  Eurojust,  and  the  Consumers,  Health,  Agriculture  and  Food  Executive 
Agency (CHAFEA)). In addition, a representative of the EU-funded Cooperation Programme 
on Drugs Policies (COPOLAD) was interviewed. 

Interviews carried out: 

a)  EU agencies: 6 interviews 
b)  EU institutions and European External Action Service (EEAS): 11 interviews 
c)  EU-funded programme: 1 interview 

(d)  

International organisations and third countries 

Interviews were carried out with representatives of the  United Nations Office on Drugs and 
Crime, the World Health Organisation, the International Narcotics Control Board (INCB), the 
Pompidou  Group  of  the  Council  of  Europe,  the  Inter-American  Drug  Abuse  Control 
Commission  (CICAD),  and  the  Maritime  Analysis  and  Operation  Centre  –  Narcotics.  As 
regards third countries, representatives of Colombia, Albania, Montenegro and Moldova were 
interviewed.127 

Interviews carried out: 

a)  International organisations: 6 interviews 
b)  Third countries: 4 interviews 

(e) 

Civil society and scientific community 

The  contractor  consulted  the  Civil  Society  Forum  on  Drugs  through  a  targeted  survey  and 
interviewed the members of the Core Group of the Civil Society Forum on Drugs.  

127  

The  countries  were  chosen  in  relation  to  work  being  done  with  the  Agency  and/or  involvement  in 
projects. 

52 

 
 
                                                           
As  regards  the  scientific  community,  several  members  of  the  Scientific  Committee  were 
interviewed  as  well  as  some  other  representatives  of  universities,  think  tanks  and  research 
facilities. The scientific community was also the subject of a targeted survey.  

The online survey for the civil society and the scientific community was open from 9 July to 
15 August 2018. The survey received 24 responses.  

(2) 

Public consultation 

In addition to the targeted consultations, the Commission organised an internet-based public 
consultation.  

The  public  consultation  was  launched  on  18  May  2018  on  the  European  Commission's 
website128  and  was  open  for  14  weeks.129  The  public  consultation  was  available  in  seven 
languages, i.e. in English, German, French, Spanish, Italian, Polish and Portuguese, but could 
be answered in all official languages of the EU.  

The aim of this consultation was to gather views from private individuals, non-profit/private 
organisations,  and  national/regional/local  public  administrations  on  the  Agency.  The 
consultation  covered  the  majority  of  the  evaluation  criteria:  effectiveness,  relevance, 
coherence and EU value added.130 As such, the public consultation was intended to form part 
of the input for the external evaluation.  

(a) 

Profile of respondents 

147 contributions131 were submitted through the online questionnaire. Out of the total number 
of respondents, 68 consented to the publication of their full contribution, while 80 opted for 
publication in anonymous form.  

Respondents  were  invited  to  identify  themselves  as  one  of  the  following  categories:  as  "an 
individual  in  your  personal  capacity"  or  "on  behalf  of  an  organisation/  in  your  professional 
capacity".  More  than  half  of  the  submissions  (55%)  came  from  respondents  answering  as 
private  individuals  with  the  remainder  answering  on  behalf  of  an  organisation.  Those 
answering  on  behalf  of  an  organisation  can  be  broken  down  as  follows:  non-governmental 
organisations  (34%  of  respondents),  Member  State  governments  (28%  of  respondents)  and 
academic  institutions  (17%  of  respondents).  Only  two  respondents  answered  on  behalf  of 
private companies and one respondent answered on behalf of an EU-level institution. The last 
17% of respondents represented other categories, for example political groups. 

Only  11  out  of  65  respondents  answering  on  behalf  of  an  organisation  replied  that  their 
organisation is included in the Transparency Register. Over half of the respondents (52% of 
respondents)  indicated  that  this  option  is  not  applicable  to  their  organisation,  while  the 
remaining 31% of respondents answered negatively. 

128  

129  
130  

131  

https://ec.europa.eu/info/consultations/public-consultation-evaluation-european-monitoring-centre-
drugs-and-drug-addiction-emcdda_en.  
The consultation period was extended by two weeks due to the summer holiday period. 
No questions were included on efficiency as it was considered as too complicated to answer questions 
on this criterion in a public consultation. 
One additional reply was received after the end of the public consultation period. Therefore, some of the 
figures add up to 148 instead of 147. 

53 

 
 
 
                                                           
Regarding  the  geographical  distribution  of  responses,  respondents  were  asked  to  indicate 
which  country  (or  the  EU  as  a  whole)  their  responses  referred  to.  Most  of  the  respondents 
indicated to come from Portugal (14% of respondents), France and Italy (10% of respondents 
each), followed by Germany (8% of respondents), Spain (7% of respondents), Belgium (6% 
of respondents), the United Kingdom (6% of respondents), Netherlands (5% of respondents) 
and  the  Czech  Republic  (4%  of  respondents).  Respondents  from  Austria,  Estonia,  Finland, 
Greece, Hungary, Ireland, Luxembourg, Malta, Poland, Romania, Slovak Republic, Slovenia 
and  Sweden  also  replied  to  the  public  consultation.  Answers  from  non-EU  countries  came 
from  Norway,  Switzerland,  United  States,  Serbia,  Ivory  Coast,  Mali  and  Georgia.  The 
generally  low  number  of  respondents  per  country  does  not  allow  for  a  meaningful 
disaggregation of responses by country in the responses. 

82%  of  respondents  stated  that  they  had  an  affinity  with  the  field  of  drug  and  addiction 
policies.  Furthermore,  78%  of  respondents  stated  that  they  had  a  good  to  very  good 
understanding  of  the  field.  Only  21%  of  total  respondents  stated  that  their  knowledge  and 
understanding of EU drug policies was limited to very limited. One respondent (representing 
1%  of  responses)  stated  that  he/she  has  no  knowledge  and  understanding  of  EU  drugs 
policies. 

Most respondents (82% of responses) stated that they have good to very good knowledge and 
understanding  of  the  activities  of  the  Agency.  Only  18%  of  respondents  claimed  that  their 
knowledge  is  limited.  Additionally,  most  respondents  to  the  public  consultation  (78%  of 
respondents) declared having cooperated with the Agency or having used their services. 

Almost half of respondents (47% of respondents) had regular contact with the Agency, with 
most of them (37% of respondents) monthly, and 10% of respondents weekly. Only 14% of 
respondents did not have any contact with the Agency during the evaluation period. 

 (b)  Results of the Public Consultation 

The  analysis  of  the  results  of  the  public  consultation  as  well  as  the  individual  replies  are 
available online.132 The input  received through the  public consultation  was integrated in  the 
evidence provided in Annex 3 of the final report of the external evaluation and was taken into 
account in the case studies (Annex 4 of the final report of the external evaluation).  

The public consultation showed that the Agency is relevant in facilitating solutions for each of 
the challenges it was designed for, especially in providing reliable, objective, evidence-based 
and accessible information  on drugs and drug policies. This  has helped policy-makers at  all 
levels, as well as practitioners and individuals working in the field of drugs to have a better 
understanding of the drug situation in Europe, which they found has been beneficial in their 
professional lives. 

The  public  consultation  further  indicated  that  the  Agency  has  been  producing  important 
outputs in the form of a range of publications, which respondents regularly consult and find 
useful and which respond to their needs. Respondents to the public consultation stated that the 
Agency was a leading source of information on new drugs, EU drug policy and developments 
on the EU drug market. The most consulted publications were the flagship publications, such 
as the European Drug Report, although respondents stated that they would consult regularly 
also publications on specific topics relevant for their work.  

132  

https://ec.europa.eu/home-affairs/what-is-new/work-in-progress/initiatives/evaluation-emcdda.  

54 

 
 
                                                           
According to most of the respondents, the Agency has been contributing in a positive way to 
the  EU  Drugs  Strategy  and  its  Action  Plans,  in  particular  regarding  its  role  in  providing  a 
better  understanding  of  all  aspects  of  the  drugs  phenomenon  and  of  the  impact  of 
interventions  to  provide  a  sound  and  comprehensive  evidence-base  for  policies  and  actions. 
The Agency was considered useful in strengthening further the cooperation between the EU 
and third countries and international organisations on drug issues, as well as in contributing to 
the  reduction  in  the  demand  for  drugs,  drug  dependence  and  drug-related  health  and  social 
risks  and  harm.  Over  half  of  the  respondents  found  that  the  Agency  could  have  an  even 
greater  impact  or  improve  its  effectiveness  if  it  broadened  the  scope  of  its  monitoring,  by 
including  more  criminal  indicators  or  by  focusing  more  on  the  link  between  licit  and  illicit 
substances. 

Concerning coherence, most of the respondents found that Agency’s activities were coherent 
with other EU policies. They further stated that its activities are in synergy with the work of 
other agencies, especially with Europol or the United Nations Office on Drugs and Crime, as 
they focus on similar topics, but through  different angles, which provides a broader view of 
the  drug  situation.  Most  of  the  respondents  did  not  find  that  there  were  overlaps.  For  those 
that did,  the stakeholders stated that the Agency’s work  was  complementary  to  the work of 
other agencies, even if it did sometimes lead to ‘double-work’ as in the case of data reporting 
to the Agency and the United Nations Office on Drugs and Crime.  

The  EU  added  value  of  the  Agency  was  established  by  the  respondents  to  the  public 
consultation  when  most  of  them  considered  that  the  Agency’s  activities  had  improved 
Member States’ ability to monitor the drug situation in their country and to respond to drug 
problems,  as  well  as  to  inform  the  policy  debate  on  drug  issues.  Indeed,  the  public 
consultation found that the Agency had contributed to the understanding of the drug situation 
in  Europe  in  general  by  providing  credible  and  reliable  data  or  that  the  Agency  helped 
practitioners  to  do  their  work  more  effectively.  Should  the  Agency  cease  its  activities,  the 
public  consultation  stated  that  this  would  result  in  the  loss  of  high  quality,  objective  and 
reliable  knowledge.  For  future  development  considerations,  18%  of  the  total  respondents 
provided  a  reply.  These  replies  showed  that  the  Agency  could  go  into  different  directions, 
such  as  covering  a  wider  range  of  addictive  substances  or  behaviours.  Finally,  the  public 
consultation  established  that  the  future  evolution  of  the  Agency  would  most  likely  be  to 
continue providing quality information that is essential for policy-makers and practitioners in 
the field of drugs. 

55 

 
 
 
ANNEX IV – INTERVENTION LOGIC  

56 

 
 
ANNEX V – SUMMARY OF THE MAIN AREAS AND GOALS IN THE 2013-2015 
AND 2016-2018 STRATEGIES AND WORK PROGRAMMES133 

2013-2015 Strategy  

Main area 

Goal 

2016-2018 Strategy 

Main area 

Goal 

Data collection, 
analysis and 
quality 
assurance  

A coherent, reliable and valid 
data collection system, 
underpinned by a quality 
assurance framework  

Monitoring and 
understanding 
drug use and 
problems: key 
indicators and 
epidemiology  

Provide an integrated and 
insightful overview of the 
European drug situation by 
enhancing analysis of the 
epidemiological key indicators, 
including cross-indicator 
analysis and combined analysis 
with other sources of 
information, while ensuring the 
quality of the information 
collected by Member States and 
the Agency  

Cross-cutting 
strategic action 
area A: 
information 
collection and 
management  

Key strategic 
action area 2: 
early warning and 
threat assessment  

Ensure the validity, 
consistency and 
reliability of the Agency’s 
reporting system  

Support rapid EU response 
to new threats by providing 
EU institutions and 
Member States with 
prompt and scientifically 
sound information for 
action on NPS and 
emerging drug trends  

Monitoring 
demand 
reduction 
responses to 
drug-related 
problems  

Support high-quality service 
development by producing 
information and analysis on 
demand reduction interventions 
and best practices  

Key strategic 
action area 3: 
situation, 
responses and 
trend analysis  

Provide a holistic picture 
of the drug phenomenon, 
through an integrated and 
coherent core monitoring 
system  

Monitoring 
drug supply and 
supply 
reduction 
interventions  

Provide the EC and the Member 
States with a comprehensive 
overview of the supply of illicit 
drugs into Europe and of the 
responses developed to respond 
to it  

Cross-cutting 
strategic action 
area B: quality 
assurance  

Monitoring new 
trends and 
developments 
and assessing 
the risks of new 
substances  

Provide a timely and sound 
information and analysis 
platform for identifying 
emerging trends and threats 
related to NPS and their risks, 
new patterns of drug use, and 
new developments in drug 
availability  

Cross-cutting 
strategic action 
area C: 
cooperation with 
partners  

Ensure that the Agency’s 
tools, processes and 
outputs remain of high 
quality and fit for purpose 
through a process of 
continuous improvement 
and evaluation of efforts  

Enhance and further 
increase the quality of the 
services provided to EU 
and Member State 
stakeholders through a 
better strategic 
understanding of the drug 
phenomenon, characterised 
by strong partnership with 
key players at European 
and global level, and by 

133  

The areas in bold type are common to both 3-year strategies and work programmes, i.e. data collection, 
communicating results, governance, administration and IT systems. Source: Final report of the external 
evaluation, p. 10. 

57 

 
 
                                                           
Improving 
Europe’s 
capacity to 
monitor and 
evaluate 
policies  

Scientific 
coordination, 
research and 
content support  

Cooperation 
and 
collaboration 
with key 
partners  

Communica-
ting the 
Agency’s 
findings to 
external 
audiences  

Governance, 
management 
and networks  

Improve the understanding of 
European and global policy 
developments by providing 
relevant and timely drug policy 
data, analysis and expertise  

Produce high-quality scientific 
work through efficient working 
practices  

Support EU drug policy debate 
and effective action and 
increased capacity for reporting 
on drug use in non-EU 
countries with an emphasis on 
countries that represent a 
priority for EU action in the 
drugs area  

Information and analyses of 
high quality reach their 
intended audience in a timely 
and cost-efficient manner  

Attain good performance in 
carrying out the tasks set out 
in the Regulation and 
achievement of objectives. 
This will be accomplished 
through good governance and 
efficient management and 
leadership, supported by 
enhanced planning, 
monitoring and reporting and 
an effective internal control 
and risk management system  

Key strategic 
action area 1: 
communicating 
evidence and 
knowledge 
exchange  

Corporate area: 
governance 

Administration 
— supporting 
core business  

Ensure effective and efficient 
allocation and management of 
financial and human 
resources and assets, through 
further rationalising internal 
processes, while developing 
the quality of services and 

Corporate area: 
administration 
and ICT 

58 

knowledge transfer to EU 
priority third countries and 
regional programmes  

Provide policy and 
practice with better 
evidence for decision-
making and action, and 
serve as the European 
central reference point 
for drug-related 
information and analysis  

Function as a modern, 
efficient and forward-
looking EU 
administration, 
committed to providing 
high-quality services to 
its stakeholders and to 
EU citizens in general; in 
achieving this, the 
Agency will be guided by 
good governance, steered 
by sound management 
and leadership and 
operated by a motivated 
and high-performing 
workforce 

Ensure sound allocation 
and management of 
financial and human 
resources and assets, and 
management of the ICT 
infrastructure and 
services, through further 

 
 
 
 
 
 
 
 
support provided  

Information and 
communications 
technology  

Support the Agency in 
achieving its objectives by 
providing high-quality and 
efficient ICT services  

rationalising and 
automatising relevant 
processes and tools, 
enhancing efficiency and 
synergies, and developing 
the quality of services 
and support  

59