COMMISSION STAFF WORKING DOCUMENT on the ex post evaluation of the "Prevention and fight against crime" 2007-2013 programme (ISEC) Accompanying the document Report from the European Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions ex post evaluation for the period 2007 to 2013 of the "Prevention and fight against crime" programme (ISEC) and the "Prevention, preparadness and consequence management of terrorism and other security related risks" programme (CIPS)
Date: 2018-06-12
Institutions: Directorate-General for Migration and Home Affairs, European Commission
Subjects: EU programme, area of freedom, security and justice, arms trade, child protection, computer crime, fight against crime, offence, organised crime, protection of freedoms

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 12.6.2018  SWD(2018) 332 final  COMMISSION STAFF WORKING DOCUMENT on the ex post evaluation of the "Prevention and fight against crime" 2007-2013 programme (ISEC)  Accompanying the document Report from the European Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions  ex post evaluation for the period 2007 to 2013 of the "Prevention and fight against crime" programme (ISEC) and the "Prevention, preparadness and consequence management of terrorism and other security related risks" programme (CIPS) {COM(2018) 455 final} - {SWD(2018) 331 final}  
 
 
Contents 

1 

Introduction .................................................................................................................... 2 

2  Background to the initiative ........................................................................................... 3 

2.1 

Policy context ............................................................................................................. 3 

2.2  Baseline ...................................................................................................................... 4 

2.3 

The ISEC Programme as funding instrument ........................................................... 11 

3  Evaluation questions .................................................................................................... 14 

4  Method ......................................................................................................................... 14 

5  Limitations ................................................................................................................... 15 

6 Status of Implementation .................................................................................................. 17 

6. 1 Absorption Rates of the ISEC Programme ............................................................... 19 

7. Answers to the evaluation questions ............................................................................... 23 

7.1 Relevance ................................................................................................................... 23 

7.2 Effectiveness .............................................................................................................. 26 

7.3 Efficiency ................................................................................................................... 35 

7.4 Coherence .................................................................................................................. 42 

7.5 EU added value .......................................................................................................... 46 

8. Conclusions ..................................................................................................................... 50 

ANNEX 1 - Procedural information .................................................................................... 55 

ANNEX 2 - Stakeholder consultation ................................................................................. 56 

ANNEX 3 – Methodology ................................................................................................... 63 

ANNEX 4 - List of evaluation questions ............................................................................ 67 

ANNEX 5 - List of abbreviations and country codes .......................................................... 69 

ANNEX 6- Intervention logic of the ISEC Programme ...................................................... 71 

1 

 
 
 
 
 
 
 
 
 
 
 
1 

INTRODUCTION 

The "Prevention and Fight against Crime" 2007-2013 programme (ISEC) was established 
for  the  period  2007  to  2013  by  Council  Decision  No  2007/125/JHA1  as  part  of  the 
Framework Programme on Security and Safeguarding Liberties 2007-2013 (SSL). Its total 
allocated budget amounted to EUR 522 million for the whole period. The other component 
of  this  Framework  Programme  is  the  "Prevention,  Preparedness  and  Consequence 
Management of Terrorism and other Security-Related Risks (CIPS)", which is covered in a 
separate report. 

In  the  context  of  the  growing  professionalisation  and  internationalisation  of  criminal 
networks and activity and the evolution of existing security threats, the ISEC programme 
aimed  to  prevent  and  combat  all  forms  of  crime  at  EU  level,  including  in  particular 
terrorism, trafficking in human beings and offences against children, illicit drug and arms 
trafficking, corruption and fraud and to contribute to the development of the policies of the 
Union. 

The  legal  basis  of  the  ISEC  programme  required  the  Commission  to  submit  to  the 
European Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions the following evaluations reports: 

  An annual presentation on the implementation of the Programme.2 

  An  interim  evaluation  report  on  the  results  obtained  and  the  qualitative  and 
quantitative aspects of the implementation of the Programme by 31 March 2010.3 

  A communication on the continuation of the Programme by 31 December 2010.4 

  An ex post evaluation for the period 2007 to 2013 on the implementation of actions 
co-financed by the Fund5 by 31 December 2015. Due to the internal re-organisation 
of DG HOME, the reporting on this ex post evaluation was delayed to 2018. The 
present  staff  working  document  therefore  reports  on  this  ex  post  evaluation 
covering  the  evaluation  of  the  2007  to  2013  Annual  Work  Programmes  (AWPs) 
implemented  in  the  participating  Member  States,  based  on  the  findings  of  an  ex 

1   OJ L 58, 24.2.2007, p. 7–12. 
2   The State of play of the AWPs was presented at ISEC Committee meetings twice a year. 
3   See European Commission (2010) Evaluation of 'Prevention and Fight against Crime'  and 'Prevention, 
Preparedness and Consequence Management of Terrorism and other Security Related Risks' Programs - 
JLS/2010/ISEC-CIPS/001-F4. Available at:  
http://ec.europa.eu/smart-regulation/evaluation/search/results.do?fileId=136471. 

4   See  European  Commission  (2011)  Communication  from  the  Commission  to  the  European  Parliament 
and the Council on the Mid Term Evaluation of the Framework Programme "Security and Safeguarding 
Liberties" (2007-2013). Available at: 
 https://ec.europa.eu/home-affairs/sites/homeaffairs/files/financing/fundings/pdf/com2011-
318_final_16062011_en.pdf . 

5   Article  14(4)  of  Regulation  (EU)  No  513/2014  of  the  European  Parliament  and  of  the  Council  of  16 
April  2014  establishing,  as  part  of  the  Internal  Security  Fund,  the  instrument  for  financial  support  for 
police cooperation, preventing and combating crime, and crisis management and repealing the Council 
Decision No. 2007/125/JHA. 

2 

 
 
 
                                                            
 
 
post evaluation study carried out by an external consulting firm and building up on 
the previous evaluations mentioned above.6  

The ex post evaluation assessed the ISEC programme in light of its relevance (whether its 
objectives matched with societal needs), effectiveness (to what extent objectives have been 
achieved),  efficiency  (to  what  extent  costs  were  proportionate  to  the  achieved  benefits), 
coherence  between  the  actions  financed  by  the  instrument  as  well  as  the  added  value  of 
intervening at EU level. 

The interim evaluation of the Internal Security Fund (ISF)7, its Police component being the 
successor of the ISEC and CIPS programmes for the period 2014-2020, takes into account 
the  findings  and  conclusions  formulated  by  this  evaluation,  assessing  what  has  been 
implemented already under ISF-Police and what needs to be further addressed in the next 
Multiannual Financial Framework (MFF) post 2020. The interim evaluation of ISF is due 
in 20188. The Commission will submit an interim evaluation report of ISF by 30 June 2018 
to  the  European  Parliament,  the  Council,  the  European  Economic  and  Social  Committee 
and the Committee of the Regions. The results of the interim evaluation together with the 
results of the present ex-post evaluation will contribute to the shaping of the future policies 
in migration and security area, especially to the preparation of the new funding instruments 
in the MFF post 2020. 

2  BACKGROUND TO THE INITIATIVE 

2.1  POLICY CONTEXT 

The rationale behind setting up the ISEC programme was to develop necessary measures to 
“prevent,  detect,  investigate,  and  prosecute  all  forms  of  crime  efficiently  and  effectively, 
most  particularly  in  cases  with  a  trans-border  element”.9  The  foundation  for  Council 
Decision 2007/125/JHA establishing ISEC was provided within the three pillar structure of 
the Treaty of Maastricht10, which includes competences around preventing and combating 
crime, organised or otherwise. 

6   Austria,  Belgium,  Bulgaria,  Cyprus,  Croatia,  Czech  Republic,  Denmark,  Estonia,  Finland,  France, 
Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, 
Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, United Kingdom, and Croatia as of 2013. 

7   Regulation (EU) No 514/2014 of the European Parliament and of the Council of 16 April 2014 laying 
down the general provisions on the Asylum, Migration and Integration Fund and on the instrument for 
financial support for police cooperation, preventing and combatting crime, and crisis management, OJ L 
150, 20.5.2014, p. 112–142. 

8   Regulation (EU) No 514/2014, Article 57. Part of the interim evaluation is the mid-term review of the 
ISF national programmes of the Member States (Regulation (EU) No 515/2014, Article 8) which takes 
place  in  2017  and  2018.  The  purpose  of  the  mid-term  review  exercise  is  for  the  Commission  and  the 
Member  States  to  review  their  national  programmes  and  assess  the need  for  a  possible  revision of  the 
programme, in light of developments in Union and national policies through a questionnaire and bilateral 
dialogues with the Member States. In addition, if the need arises, the results of the mid-term review of 
the  national  programmes  may support  requests  for  additional  funding made  by  the  Commission  to  the 
budgetary authorities for the remaining implementation period. 

9   Council Decision 2007/125/JHA. 
10   http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=URISERV:xy0026  

3 

 
 
 
 
 
 
                                                            
Several legal, practical and policy developments introduced before 2007 had an influence 
on  the  adoption  of  ISEC.  A  predecessor  of  ISEC  was  the  Judicial  Cooperation  in 
Criminal  Matters  (AGIS)  framework  programme  established  by  Council  Decision 
2002/630/JHA.11  Its  main  aim  was  to  strengthen  Member  State  cooperation  between  the 
police, other law enforcement agencies and the judiciary. AGIS covered the period 2002-
2006 and supported practitioners in contributing to the development of EU policy in this 
area. 

In  November  2004,  the  European  Council,  building  on  the  conclusions  of  the  Tampere 
European Council 1999, reaffirmed its promotion of freedom, security and justice through 
the  prevention  of  and  fight  against  crime.  This  led  to  the  development  of  the  Hague 
Programme  (2004-2009),  focusing  on  strengthening  security  in  the  EU.  Of  particular 
relevance to the ISEC programme are two of the specific aims of the Hague Programme:  

  defining a balanced approach to migration and fighting trafficking in human beings 

and; 

  developing  a  strategic  concept  on  tackling  organised  crime  and  guaranteeing  the 

right balance between privacy and security while sharing information.  

The  subsequent  Stockholm  Programme  and  Action  Plan  (2010-2015),  called  for  the 
development of an EU Internal Security Strategy, underlying the need to fight cross-border 
crimes  such as  trafficking in  human beings, sexual abuse, sexual exploitation  of children 
and child pornography, cybercrime, corruption, counterfeiting and piracy and illicit drugs. 
The  financial  support  provided  under  ISEC  covered  indeed  these  areas  of  intervention. 
This  support,  along  with  the  abovementioned  EU  policy  developments,  underlines  the 
increasing importance of the Union's policies in the area of Freedom, Security and Justice, 
with the Lisbon Treaty and with the guidance provided by the Stockholm Programme and 
its Action Plan. 

On 22 November 2010, the Commission issued the Communication titled The EU Internal 
Security  Strategy  in  Action:  Five  steps  towards  a  more  secure  Europe,  which  proposed 
over the next four years, five strategic objectives for internal security and concrete actions. 
This  Communication  oriented  the  priorities  for  financial  support  provided  by  the  Union 
within the framework of ISEC and CIPS programmes. 

2.2  BASELINE 

At the beginning of the ISEC period, there was no common EU policy on the prevention 
and fight against crime and existing cooperation in this area took place in the form of inter-
governmental  cooperation  between  Member  States.  ISEC  thus  covered  a  wide  scope  of 
policy areas, leaving cooperation possibilities open for Member States as a way to foster 
closer cooperation the areas of trafficking in human beings, drug trafficking, cybercrime, 
economic  and  organised  crime,  corruption,  law  enforcement  cooperation,  child  sexual 
abuse,  radicalisation,  arms  trade  and  trafficking.  The  following  sub-sections  provide  an 
outline  of  key  policy  developments  in  the  different  thematic  areas  covered  by  the  ISEC 
Programme. 

11   2002/630/JHA:  Council  Decision  of  22  July  2002  establishing  a  framework  programme  on  police  and 

judicial cooperation in criminal matters (AGIS),   
http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32002D0630  

4 

 
 
 
 
                                                            
Trafficking in human beings 

According to Europol, human beings are trafficked by organised criminal groups (OCGs) 
mainly to serve illegal labour and sex markets12.  

There  was  an  overall  upward  trend  in  the  number  of  victims  of  human  trafficking 
registered  with  police,  NGOs  and  other  agencies  over  the  ISEC  period  in  the  EU. 
According  to  Eurostat  data,  the  number  of  identified  and  presumed  victims  of  human 
trafficking increased by 18 % between 2008 and 2010, with a total of 6 309 reported by EU 
Member  States  in  2008,  6  955  in  2009  and  7  410  in  201013.  However,  trends  across 
Member States have varied. 

At European  level, focus on human trafficking was high.  Indeed,  during the years of the 
implementation of ISEC, there were two main policy developments:  

  Directive  2011/36/EU  of  the  European  Parliament  and  of  the  Council  of  5  April 
2011  on  preventing  and  combating  trafficking  in  human  beings  and  protecting  its 
victims, and replacing Council Framework Decision 2002/629/JHA.14 

  The  EU  Strategy  towards  the  Eradication  of  Trafficking  in  Human  Beings  2012–

2016.15 

The Directive and the Strategy fit in the wider global action against trafficking in human 
beings to which the EU is committed. In 2009, the Council approved the Action-Oriented 
Paper on strengthening the EU external dimension on action against trafficking in human 
beings16.  The  EU  is  committed  to  strengthening  coordinated  action  amongst  Member 
States.  The  aim  is  also  to  act  in  countries  of  origin  and  transit  of  victims  and  to  raise 
awareness,  reduce  vulnerability,  support  and  assist  victims,  fight  the  root  causes  of 
trafficking  and  support  those  third  countries  in  developing  appropriate  anti-trafficking 
legislation (Directive 2011/36/EU (2)). 

Drug trafficking 

Drug trafficking is considered by Europol as the most dynamic crime area, with trafficking 
routes  continuously  diversifying.  Heroin,  cannabis,  cocaine  and  synthetic  drugs  are  the 
types that are most commonly illegally trafficked and consumed in the European Union17. 
Cannabis  is  the  most  frequently  consumed  drug  in  the  EU,  with  Cocaine  being  the 
second18.  Due  to  unlikely  decrease  in  consumption,  high  profits  and  low  risks,  many 

12   Europol,  SOCTA  2013.  Available  at  https://www.europol.europa.eu/content/eu-serious-and-organised-

crime-threat-assessment-socta 

13   Based  on  the  available  data  from  24  EU  Member  States,  excluding  Spain,  Ireland  and  the  United 

Kingdom for which data was missing. Source: 
https://emnbelgium.be/sites/default/files/publications/20130415_thb_stats_report_en.pdf. 

14   https://ec.europa.eu/anti-trafficking/sites/antitrafficking/files/directive_thb_l_101_15_april_2011_1.pdf 
15   http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52012DC0286:EN:NOT 
16   https://ec.europa.eu/anti-trafficking/eu-policy/action-oriented-paper-strengthening-eu-external-

dimension-against-trafficking-human-beings_en 

17   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 
18   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 

5 

 
 
 
 
 
                                                            
  
Organised  Criminal  Groups  (OCGs)  are  shifting  from  heroin  and  cocaine  trafficking  to 
cannabis19.  

Some Member States registered a slight decrease in consumption between 2007 and 2013, 
but the market appears to have stabilised20.  

Synthetic drugs and new psychoactive substances (NPS) have also increasingly become a 
challenge for European law enforcement authorities. In 2012, 70 psychoactive substances 
emerged  with  NPS  being  increasingly  sold  on  the  Internet21.  The  high  profitability  and 
continuous expansion of this market is likely to turn competition between OCGs more and 
more violent22. 

In terms of policy developments during or right after the period under evaluation, the EU 
published the EU Drugs Strategy 2013-2020 and the Action Plan 2013-201623, of which a 
progress report was published in  201524. The aim of both  the Strategy and of the Action 
Plan is to offer a platform for coordination among Member States and to promote an EU 
approach. The Strategy focuses around three areas: drug markets, health and social issues 
and evidence-based decision making. 

Cybercrime 

Cybercrime  can  include  several  crimes  conducted  through  the  Internet.  According  to 
Europol’s definition, cybercrime comprises of both ‘specific’ cybercrime offences such as 
hacking,  phishing  and  malware,  as  well  as  internet-enabled  fraud25.  Phishing  is  a 
constantly  growing  threat  in  the  EU,  used  to  gather  information  directly  from  unaware 
victims  as  well  as  to  install  malwares  in  people’s  devices  (e.g.  asking  them  to  follow  a 
link).  There  has  also  been  a  growing  trend  towards  the  hacking  of  service  providers  as 
opposed  to  single  users  as  a  more  efficient  way  to  gather  quickly  large  amounts  of 
sensitive data used consequently to carry out frauds. Finally, payment card fraud generates 
an  annual  income  of  around  EUR  1.5  billion  for  OCGs  in  the  EU,  having  caused 
substantial losses to the EU economy26. 

The  beginning  of  the  ISEC  programme  coincided  with  the  outburst  of  social  media  at 
global  level  (Facebook,  YouTube  and  later  Twitter  and  LinkedIn).  As  a  consequence  of 
this  boom,  new  opportunities  were  created  for  organised  crime  causing  additional 
challenges  for  law  enforcement  agencies.  Indeed,  nowadays  a  significant  share  of  the 
funding  invested  in  security  goes  to  the  ICT  (Information  and  Communications 
Technology) field27.  

19   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 
20   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 
21   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 
22   Op.cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta 
23

http://www.emcdda.europa.eu/attachements.cfm/att_212356_EN_EMCDDA_POD_2013_New%20

EU%20drugs%20strategy.pdf 

24   https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/policies/organized-crime-and-

human-trafficking/drug-control/docs/drugs_strategy_report_en.pdf 

25   Op. cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta  
26   Ibidem. 
27   Second expert workshop organised for this evaluation, ICF S.A. offices, Brussels, 6th of March 2017. 

6 

 
 
 
 
                                                            
 
One third of European citizens do not trust the internet for banking or payment services28. 
In  this  context  and  given  the  increasing  rate  of  cybercrime  against  both  individuals  and 
private sectors, the EU adopted a Cybersecurity Strategy in 201329. In the same year, the 
EU adopted a Directive on attacks against information systems30 which required Member 
States  to  strengthen  national  cyber-crime  laws  and  introduce  tougher  criminal  sanctions. 
The  2013  Cybersecurity  Strategy  consists  of  five  strategic  priorities:  to  achieve  cyber 
resilience,  reduce  cybercrime,  develop  cyber  defence  policy  and  capabilities,  develop 
industrial  and  technological  resources  for  cybersecurity  and  establish  a  coherent 
international cyberspace policy. 

Economic and financial crime  

reckless 

Economic  and  financial  crimes  include  fraud  and  money  laundering.  Fraud  is  a  broad 
category  covering  also  poor  business  practices  such  as 
investment, 
misrepresentation of financial statements and conspiring to manipulate inter-bank interest 
rates31. The most common economic fraud is value-added tax (VAT) fraud which causes a 
loss of VAT revenue. At EU level, the VAT fraud which takes advantage of movement of 
goods  between  different  jurisdictions  is  known  as  the  missing  trader  intra  community 
(MTIC) fraud. MTIC is a widespread criminal offence affecting many EU Member States 
and  perpetrated  from  both  inside  and  outside  the  EU.  It  deprives  Member  States  of  tax 
revenue  required  to  make  investments,  maintain  public  sector  services  and  service 
sovereign debt. The loss of VAT revenue due to MTIC fraudulent transactions is estimated 
at  between  EUR  45  and  53  billion32.  Advance  Fee  Fraud  and  Money  laundering  are  the 
other  most  common  economic  crimes.  It  is  estimated  that  the  sum  of  money  laundered 
globally amounts to between 2% and 5% of global GDP each year33. The difficulties linked 
to  international  cooperation  in  investigations,  differences  in  legislation  and  lack  of 
awareness  against  economic  and  financial  frauds  make  detection  of  Advance  Fee  Fraud 
generally low34. 

At  the  EU  level,  there  was  no  evident  policy  development  in  this  area  during  of  the 
implementation  period  of  the  ISEC  Programme  and  thus  the  main  legislative  instrument 
remains Directive 2005/60/EC (October 2005) on the prevention of the use of the financial 
system  for  the  purpose  of  money  laundering  and  terrorist  financing35.  The  Directive  was 
adopted  in  the  context  of  the  need  to  adjust  previous  legislation  (Council  Directive 

28   2012 Special Eurobarometer 390 on Cybersecurity. 
29   JOINT  COMMUNICATION  TO  THE  EUROPEAN  PARLIAMENT,  THE  COUNCIL,  THE 
EUROPEAN  ECONOMIC  AND  SOCIAL  COMMITTEE  AND  THE  COMMITTEE  OF  THE 
REGIONS,  Cybersecurity  Strategy  of  the  European  Union:  An  Open,  Safe  and  Secure  Cyberspace. 
2013. Available at http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52013JC0001. 

30   DIRECTIVE  2013/40/EU  OF  THE  EUROPEAN  PARLIAMENT  AND  OF  THE  COUNCIL  of  12 
August  2013  on  attacks  against  information  systems  and  replacing  Council  Framework  Decision 
2005/222/JHA. Available at 
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:218:0008:0014:EN:PDF 

31   Op. cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta  
32   Final Report of 30 June 2015, TAXUD/2013/DE/319, page 17. Available at 

https://ec.europa.eu/taxation_customs/sites/taxation/files/docs/body/ey_study_destination_principle.pdf 
33   Op. cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta. 
34   Ibidem. 
35   http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2005:309:0015:0036:EN:PDF  

7 

 
 
 
 
                                                            
  
91/308/EEC  (June  1991))  to  the  newly  created  international  scenarios  such  as  the  use  of 
the financial system and money laundering for terrorist financing. 

Corruption 

Although  Member  States  have  developed  over  time  the  necessary  legal  instruments  in 
place  to  prevent  and  fight  corruption,  the  EU  anti-corruption  Report  of  2014  found  that 
results  delivered  by  these  instruments  are  not  satisfactory  across  the  EU36.  Some  of  the 
issues  mentioned  include  anti-corruption  rules  not  always  being  vigorously  enforced,  the 
persistence  of  systemic  problems  and  a  lack  of  sufficient  capacity  for  the  relevant 
institutions to enforce the rules in this area. 

An “anti-corruption package” is in place since 2011 which aims to shape EU policies in the 
fight against corruption. It includes a Communication in Fighting Corruption in the EU37 
and  a  Commission  Decision  establishing  an  EU-anti  corruption  reporting  mechanism38. 
The  legislation  now  in  place  at  EU  level  is  the  Council  Framework  Decision 
2003/568/JHA of 22 July 2003 on combating corruption in the private sector39, which aims 
to establish passive and active corruption as criminal offences in all Member States and to 
ensure that penalties are effective, proportionate and dissuasive. 

Law enforcement cooperation 

With the entry into force of the Treaty of Lisbon and the abolition of the ‘pillar structure’, 
the  EU  gained  more  resources  to  promote  police  cooperation40.  Cooperation  and  policies 
are still developing, with attention focused on countering pan-EU threats and crime more 
effectively and doing so in compliance with fundamental rights and data protection rules.  

Two  important  legislative  instruments  in  place  to  facilitate  law  enforcement  cooperation 
are: the Council Decision 2008/615/JHA on the stepping  up of cross-border cooperation, 
particularly in combating terrorism and cross-border crime (the so called Prüm Decision) 
and  the  Council  Framework  Decision  2006/960/JHA  on  simplifying  the  exchange  of 
information and intelligence between law enforcement authorities of the Member States of 
the EU. While the latter sets up the rules for Member States’ law enforcement authorities 
to  exchange  existing  information  for  the  purposes  of  criminal  investigations,  Council 
Decision  2008/615/JHA  aims  to  improve  the  exchanges  of  information  between  the 
authorities  responsible  for  the  prevention  and  investigation  of  criminal  offences.  The 
decision sets out provisions with regard to: 

 

the automated access to DNA profiles, dactyloscopic data and certain national 
vehicle registration data; 

36   European Commission, EU anti-corruption Report, 2014, http://ec.europa.eu/dgs/home-affairs/e-

library/documents/policies/organized-crime-and-human-trafficking/corruption/docs/acr_2014_en.pdf . 

37   https://ec.europa.eu/home-

affairs/sites/homeaffairs/files/news/intro/docs/110606/308/1_en_act_part1_v121.pdf 

38   https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-is-
new/news/pdf/com_decision_c%282011%29_3673_final_en.pdf  
39 
 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32003F0568&from=EN. 
40   European Parliament, Fact sheets on the European Union, Police Cooperation, September 2016, 

available at http://www.europarl.europa.eu/atyourservice/en/displayFtu.html?ftuId=FTU_5.12.7.html. 

8 

 
 
 
 
 
                                                            
  supply of data in relation to major events; 
  supply of information in order to prevent terrorist offences; 
  other measures for stepping up cross-border police cooperation. 

At operational level, the main instrument for cooperation is Europol, which is central to the 
broader European internal security architecture. Since 1 January 2010, Europol has been an 
EU agency financed from the EU budget. Europol has been at the forefront of the EU law 
enforcement  response  to  emerging  threats:  in  January  2013  the  European  Cybercrime 
Centre (EC3)  was established,  which is  responsible, inter  alia, for the Internet Organised 
Crime  Threat  Assessment  (iOCTA).  The  Council  expanded  Europol’s  counterterrorism 
mandate  further  with  the  launch,  on  1  January  2016,  of  the  new  European  Counter-
Terrorism Centre, to which Member State counterterrorism experts are seconded to boost 
cross-border investigation capacity. Lastly, founded in 2000, the European Union Agency 
for  Law  Enforcement  Training  (CEPOL)  is  responsible  for  training  law  enforcement 
officials. 

Child sexual abuse 

According to Europol, the amount of video material depicting child sexual abuse available 
online  increases  steadily,  also,  due  to  innovations  in  video  sharing  technology41,  making 
new material most likely to be exchanged in non-commercial environments. The threat of 
online child sexual exploitation is expected to increase in the coming years corresponding 
with higher levels of demand for new child abusive material and continued technological 
developments42.  This  crime  has  serious  consequences  on  the  young  victims,  leading  to 
lasting mental issues and even suicide.  

There is no available data on this topic covering the implementation period of ISEC (2007-
2013), given that reporting of this issue through the ‘Threat Assessment from the Global 
Alliance Against Child Sexual Abuse Online’ only began in 2015. The Alliance is a joint 
EU-US initiative launched in December 2012. The 54 participating countries committed to 
four  targets:  better  identification  of  and  assistance  to  victims;  more  efforts  to  investigate 
cases of child sexual abuse and prosecute offenders; increased awareness among families 
and communities and reduction of availability of child pornography materials.  

During the ISEC implementation period, one legislative instrument was adopted in 2011 at 
EU  level  covering  this  crime  area,  Directive  2011/92/EU  on  combating  the  sexual  abuse 
and sexual exploitation of children and child pornography (replacing Council Framework 
Decision  2004/68/JHA)43.  The  aims  of  the  Directive  are  to  establish  minimum  rules 
concerning the definition of criminal offences and sanctions in the area of sexual abuse and 
sexual  exploitation  of  children,  child  pornography  and  solicitation  of  children  for  sexual 
purposes; to introduce provisions to strengthen the prevention of those crimes and protect 
victims. 

Radicalisation 

 Op. cit., https://www.europol.europa.eu/content/eu-serious-and-organised-crime-threat-assessment-socta  
Ibidem. 

41 
42 
43   http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32011L0093 

9 

 
 
 
 
 
                                                            
Terrorists and violent extremists have improved their use of the Internet and social media 
for  communication,  planning 
training  and 
propaganda44.  Many  experts  have  also  described the  process  of  self-radicalisation  among 
EU nationals45. Countering radicalisation is high on the Member State and EU agendas, in 
particular given the most recent terrorist attacks that have taken place on European soil. 

targeting,  recruiting, 

terrorist  attacks, 

At EU level, there were policy developments, during and after the evaluation period, which 
targeted  radicalisation.  The  first  pillar  of  the  EU  Counter-Terrorism  Strategy46  (2005) 
explicitly  focused  on  the  prevention  of  radicalisation  by  tackling  the  factors  that  lead  to 
radicalisation  and  violent  extremism.  Followed  by  the  Internal  Security  Strategy,  the 
Radicalisation  Awareness  Network  was  created  in  2011  on  that  basis47.  In  2014,  the  EU 
revised  its  Strategy  for  Combating  Radicalisation  and  Recruitment  to  Terrorism.48  The 
revised Strategy was adopted in recognition of the fact that radicalisation and terrorism are 
constantly evolving phenomena and, as a consequence, the priorities and approach of the 
EU in this area need to be constantly updated.  

Arms trade and trafficking 

It is difficult to discern a pattern in relation to the trafficking of firearms due to insufficient 
availability  of  data.  Available  data  for  a  handful  a  Member  States  suggest  that  firearms 
trafficking figures in recent years have remained stable.  

It is  also  difficult to  detect  a  trend in  the  number  of  deaths  caused annually  by  firearms. 
Data is only available for all 28 Member States for the years 2007-2010, and a clear pattern 
cannot be discerned for this data. The total number of deaths recorded in the EU seems to 
decrease over time but this may likely be a reflection of the absence of data than a trend.  

The main legislative instruments dealing with arms trade and trafficking at the EU level are 
Directive  91/477  (amended  by  Directive  2008/51)  and  Regulation  258/2012.  Directive 
91/477  introduced  a  number  of  provisions  regarding  both  the  possession  and  transfer  of 
arms and their transportation. The objective was to facilitate the free movement of goods 
while  ensuring  a  certain  level  of  protection  against  cross-border  crime  and  terrorism. 
Directive  2008/51  amending  Directive  91/47749  was  adopted  on  21  of  May  2008  and 
introduced a number of changes in the EU main legislative instrument dealing with arms 
possession and transfer.  

The main amendments introduced related to the innovative elements that the UN Protocol 
on  the  illicit  manufacturing  of  and  trafficking  in  firearms,  their  parts,  components  and 
ammunition,  signed  on  behalf  of  the  EU  by  the  Commission  in  2002,  added  to  the  EU 
regime.  These  innovative  elements  included  the  introduction  of  a  marking  system,  an 
explicit  penalisation  of  offences  related  to  illicit  arms  trafficking,  and  the  definition  of 
reactivation of deactivated arms. Together with an implementation report, the Commission 

44   Europol, EU Terrorism Situation & Trend Report (Te-Sat), 2014, 

https://www.europol.europa.eu/activities-services/main-reports/te-sat-2014-eu-terrorism-situation-and-
trend-report. 

45   Ibidem. 
46   http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%2014469%202005%20REV%204 
47   The first year of the RAN was funded under ISEC procurement after which it was funded through a 

separate DG HOME funds outside the ISEC Programme. 

48   http://data.consilium.europa.eu/doc/document/ST-9956-2014-INIT/en/pdf 
49   OJ L 179, 8.7.2008, p. 5–11. 

10 

 
 
 
 
                                                            
also  presented  a  proposal  for  a  Directive  of  the  European  Parliament  and  of  the  Council 
amending  Council  Directive  91/477/EEC  on  control  of  the  acquisition  and  possession  of 
weapons in 2015.50  

2.3  THE ISEC PROGRAMME AS FUNDING INSTRUMENT  

The  ISEC  programme  aimed  to  prevent  and  combat  all  forms  of  crime  at  EU  level, 
including in particular terrorism, trafficking in human beings and offences against children, 
illicit drug and arms trafficking, corruption and fraud and to contribute to the development 
of  the  policies  of  the  Union.  To  achieve  this,  it  sought  to  contribute  to  the  following 
general objectives51: 

1. The Programme shall contribute to a high level of security for citizens by preventing and 
combating crime, organised or otherwise, in particular terrorism, trafficking in persons and 
offences against children, illicit drug trafficking and illicit arms trafficking, corruption and 
fraud. 
2. Without prejudice to the objectives and powers of the European Community, the general 
objectives  of  the  Programme  contribute  to  the  development  of  the  policies  of  the  Union 
and of the Community. 

Further  to  these  general  objectives  were  the  following  specific  objectives52,  all  four  of 
which were linked to each of the general objectives and provided operational guidance to 
the implementation of the ISEC programme: 

(a)  To  stimulate,  promote  and  develop  horizontal  methods  and  tools  necessary  for 
strategically preventing and fighting crime and guaranteeing security and public order such 
as the work carried out in the European Union Crime Prevention Network, public-private 
partnerships, best practices in crime prevention, comparable statistics, applied criminology 
and an enhanced approach towards young offenders; 

(b)  To  promote  and  develop  coordination,  cooperation  and  mutual  understanding  among 
law enforcement agencies, other national authorities and related Union bodies in respect of 
the priorities identified by the Council in particular as set out by the Europol's Organised 
Crime Threat Assessment; 

(c) To promote and develop best practices for the protection and support witnesses; and 

(d) To promote and develop best practices for the protection of crime victims. 

The  programme  aimed  to  contribute  to  the  abovementioned  objectives  within  four 
themes53: 

50  https://ec.europa.eu/transparency/regdoc/rep/1/2015/EN/1-2015-750-EN-F1-1.PDF  
51   These are defined under Article 2 of Decision No 2007/125/JHA. 
52   These are defined under Article 3(2) of Decision No 2007/125/JHA. 
53   These are defined under Article 3(1) of Decision No 2007/125/JHA. 

11 

 
 
 
 
 
 
 
 
 
 
 
 
                                                            
(a) Crime prevention and criminology; 
(b) Law enforcement; 
(c) Protection and support to witnesses; 
(d) Protection of victims. 

These  themes  focus  the  intervention  by  grouping  the  policy  areas  covered  by  the 
programme into four main strands, each of which were in turn linked to one or several of 
the abovementioned specific objectives. In addition,  several thematic areas covered by the 
projects  implemented  under  the  ISEC  programme  were  identified;  these  are  discussed  in 
section 6. 

Implementation through Direct Management 

The  ISEC  programme  has  been  implemented  via  actions  under  the  2007-2013  AWPs, 
under direct management mode via projects supported by grants (representing 85% of total 
funding committed), or via contracts for services concluded following calls for tenders or 
administrative  arrangements  with  the  Joint  Research  Centre  (JRC)  (representing  15%  of 
total funding committed). 

Under  direct  management,  the  European  Commission  retained  full  responsibility  for 
implementation and carried out all programming and operational work. Political priorities 
and  objectives  for  projects  were  defined  in  the  AWPs  adopted  each  year,  and  calls  for 
proposals launched following the approval of each AWP. Member States were involved in 
the approval of AWPs, but they were not involved in the management of the Programme. 
Whilst the ISEC’s objectives were ‘fixed’ in the founding legislation54, the AWPs set the 
Programme  priorities  and  sub-priorities  and  could  be  revised  on  a  yearly  basis.  This 
allowed  the  Programme  a  degree  of  flexibility  when  adapting  to  changing  needs.  The 
process  of  setting  annual  priorities  in  the  AWPs  was  carried  out  in  regular  consultation 
with the Member States and through an inter-service Committee with the involvement of 
relevant policy units from the Directorate-General Migration and Home Affairs. Through a 
comitology  procedure,  Member  States  received  a  draft  of  the  AWP  and  were  invited  to 
provide their comments. Following incorporation of these comments, the final version of 
the AWPs was adopted via a Decision.  

Funding  was  provided  largely  through  action  grants  awarded  on  the  basis  of  calls  for 
proposals. Three types of grants were used, namely: 

(i)  action  grants  to  co-finance  specific  initiatives  awarded  through  general  and  targeted 
open calls for proposals (CFP), representing 73% of funding provided through grants; 

(ii)  action  grants  awarded  through  restricted  CFP  reserved  to  public  sector  entities  with 
whom ‘framework partnership agreements’ (FPA) have been established, representing 23% 
of funding provided through grants; and 

(iii)  operating  grants  aimed  at  supporting  the  activities  of  NGOs  with  a  European 
dimension, representing 4% of funding provided through grants. 

54   Council Decision 2007/125/JHA, 

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2007:058:0007:0012:EN:PDF 

12 

 
 
 
 
 
 
 
                                                            
 
Other  funding  channels  included  public  procurement  contracts,  involving  open  calls  for 
tenders  and  administrative  arrangements  with  the  JRC.  EU  Agencies,  funded  by  an  EU 
subsidy, were not entitled to apply under ISEC for the activities they are responsible for, as 
that  would constitute  double  funding.  However,  EUROPOL  became  an  EU  Agency  only 
on 1 January 2010. Before that, being another type of legal entity (inter-governmental), it 
was  indeed  funded  to  carry  out  a  number  of  projects.  However,  EU  Agencies  could 
participate in projects, provided that they paid for their own costs.  

According  to  Article  5  of  Decision  No  2007/125/JHA,  the  types  of  actors  who  could 
apply for ISEC funding included law enforcement agencies, other public and/or private 
bodies,  actors  and  institutions,  including  local,  regional  and  national  authorities,  social 
partners,  universities,  statistical  offices,  NGOs,  public-private  partnerships  and  relevant 
international bodies. 

According to Article 4(2) of Decision No 2007/125/JHA, the types of actions eligible for 
financial support from ISEC were: 

a) Actions improving operational cooperation and coordination (strengthening networking, 
mutual  confidence  and  understanding,  exchange  and  dissemination  of  information, 
experience and best practices); 

b) Analytical, monitoring and evaluation activities; 

c) Development and transfer of technology and methodology; 

d) Training, exchange of staff and experts; 

e) Awareness and dissemination activities. 

Overview of Main Characteristics of the Grants  

From  a  sample  of  151  projects  selected  for  in-depth  analysis,  the  vast  majority  (108, 
representing  72%)  were  transnational,  if  they  included  at  least  one  partner  organisation 
based  in  a  different  Member  State  to  the  coordinating  organisation55  while  only  a  small 
share were national (43, representing 28%). Overall, the average number of organisations 
involved in a project was five56. 

A more in-depth analysis of the geographical dimension of transnational partnerships in the 
sample (for 108 projects, as mentioned above) highlighted that coordinating organisations 
located  in  one  Member  State  which  established  multiple  partnerships  tended  to  do  so  in 
different  countries.57  This  indicates  that  when  there  were  opportunities  to  initiate  a  large 
number of partnerships, coordinating organisations did tend to make these partnerships as 
diverse as possible. Such opportunities are likely linked to a number of other factors such 

55   Additionally, one project – coordinated by Europol – was classed as transnational as it was coordinated 

by a trans-European organisation. 

56   It  should  be  highlighted  that  the  majority  of  national  projects  in  the  sample  concerned  single 

organisations whereas transnational projects included an average of seven organisations. 

57   For  example,  the  Netherlands  established  96  partnerships  with  organisations  located  in  22  different 

Member States for only 14 transnational projects. 

13 

 
 
 
 
 
                                                            
as  funding,  or  the  number  of  projects  coordinated  altogether  by  organisations  from  a 
specific country. In this case, for organisations in countries such as Italy, where the highest 
number  of  coordinating  organisations  were  located,  more  partnerships  were  established 
with  a  number  of  coordinating  organisations  in  different  Member  States,  compared  to 
organisations located in countries which coordinated fewer projects. 

A  salient  trend  was  that  coordinating  organisations  appear  to  partner  up  with  a 
coordinating  organisation  in  a  neighbouring  or  a  Member  State  belonging  to  the  same 
geographical category58. As a result, coordinating organisations located in newer Member 
States  (i.e.  EU-13)  were  more  likely  to  partner  up  with  a  coordinating  organisation  in 
another  EU-13  country  than  a  Western  European,  Southern European  or  Nordic  Member 
State.  Similarly,  the  number  of  partner  organisations  located  in  Nordic  Member  States 
taking  part  in  projects  was  higher  when  the  project  was  coordinated  by  another  Nordic 
country, compared to when coordinated by a Southern European or EU-13 Member State.  

Regarding  the  geographical  distribution  of  national  projects,  Italy  coordinated  the  largest 
number  of  national  projects  (9)59  followed  by  the  UK  (4).  Overall,  20  Member  States 
coordinated at least one national project. 

No clear association could be established between the type of call for proposals or thematic 
focus and partnership size. However, it is interesting to note that for the following thematic 
areas, no single size partnerships occurred (for those thematic areas which had at least five 
projects in the sample): trafficking of human beings, child sexual abuse and forensics. 

3  EVALUATION QUESTIONS 

The  evaluation  was  planned  and  tendered  before  the  Better  Regulation  Guidelines  were 
adopted on 19 May 2015, but still covered all the five key evaluation criteria required by 
the Guidelines i.e. relevance, effectiveness, efficiency, coherence and EU added value. To 
assess these criteria, the evaluation reviewed the 10 questions that are included in Annex 4. 

4  METHOD 

The  evaluation  relies  on  an  external  study  carried  out  between  August  2016  and  August 
2017  by  a  consulting  firm  specialised  in  evaluation.  The  study's  methodology  combined 
desk  research,  and  both  qualitative  and  quantitative  analysis.  It  required  a  systemic 

58   For  the  purpose  of  this  analysis,  Member  States  were  categorised  into:  the  EU-13  (Bulgaria,  Croatia, 
Cyprus,  Czech  Republic,  Estonia,  Hungary,  Latvia,  Lithuania,  Malta,  Poland,  Romania,  Slovakia,  and 
Slovenia);  Western  Europe  (Austria,  Belgium,  France,  Germany,  Ireland,  Luxembourg,  Netherlands, 
United  Kingdom);  Southern  Europe  (Greece,  Italy,  Portugal,  Spain);  and,  Nordic  Europe  (Denmark, 
Finland and Sweden). 

59   This  is  consistent  with  an  overrepresentation  identified  in  the  evaluation  of  coordinating  organisations 

located in Italy within the whole Programme. 

14 

 
 
 
 
 
                                                            
synthesis  of  the  evidence  regarding  the  implementation  of  the  ISEC  Programme. 
Information was drawn mainly from various primary and secondary sources, such as: 

  Primary  sources:  three  online  surveys  (coordinating  organisations,  partner 
organisations and project participants); stakeholder targeted consultations; thematic 
case studies (8); public consultation; expert workshops (2) and expert input. 

  Secondary  sources:  quantitative  data  on  all  ISEC  projects  provided  by  the 
Commission; in-depth analysis of a sample of projects (151 projects, representing 
27%  of  all  568  finalised  projects);  a  literature  review  of  EU  policy  baseline  and 
context and a literature review of national baseline and context. 

A more detailed overview of the methodology used for the evaluation study may be found 
in Annex 3. 

Information  was  triangulated  when  possible  to  ensure  validity  and  robustness.  Findings 
from the study are presented in this report together with the financial data extracted from 
ABAC60. 

5  LIMITATIONS 

The evaluation process encountered a number of difficulties that induced methodological 
limitations (for more details on the methodology see Annex 3). 

The lack of a baseline 

The  first  key  obstacle  encountered  in  the  evaluation  was  the  fact  that  establishing  a 
baseline for measuring either organised crime or the volume of crime levels across the EU 
is  difficult.  In  terms  of  organised  crime,  there  are  no  established  methodologies  for 
measuring in a comprehensive manner most illegal markets or activities covered by actions 
funded via ISEC. Similarly, the statistical information gathered on the volume of crime at 
EU level covers solely a few types of crimes (e.g. homicides, car-theft and robberies), and 
does  not  cover  unreported  crime.  It  has  also  proven  very  difficult  to  make  EU-wide 
comparisons due to differences in criminal justice systems, definitions and practices. These 
difficulties  made  it  problematic  and  indeed  not  possible  to  establish  a  common  EU 
baseline.  As  a  mitigation  measure,  during  the  inception  phase  of  the  evaluation,  external 
evaluators  were  asked  to  develop  a  set  of  indicators  with  a  view  to  mapping  as  far  as 
possible the national and EU context with regard to the different thematic areas covered by 
ISEC and to identify trends in these data before, during and after the programming period. 
However,  the  evaluators  encountered  significant  data  issues  when  trying  to  populate  the 
indicators and it was agreed that due to the lack of comparable data, this information would 
only be used for illustrative purposes. For this reason, a detailed overview of the EU policy 
developments in the thematic areas covered by the ISEC Programme over the period 2007-
2013 was developed instead, as presented in Section 2.1. 

Furthermore,  an  inherent  difficulty  when  measuring  the  extent  to  which  ISEC  helped  to 
improve  the  management  of  crime  –  and  thus  to  establishing  causal  relations  between 

60   ABAC is the Commission’s accrual-based online accounting system. 

15 

 
 
 
 
 
 
 
                                                            
ISEC projects and reported impacts – was the fact that due to the wide range of factors that 
influence crime and their dynamic interconnection61, it has not been possible to establish 
direct links between the funding provided to specific projects and the perceived outcomes 
or  impacts  in  the  relevant  areas  of  crime.  Therefore,  measuring  the  overall  impact  of 
specific funding instruments, including ISEC, on general or organised crime levels remains 
methodologically  challenging.  It  has  not  been  possible  to  establish  direct  causal  links 
between  the  ISEC  Programme's  intervention  or  identified  needs  on  the  one  hand  and 
observed  impacts  or  results  on  the  other  hand;  therefore,  effectiveness  has  been  mostly 
evaluated  in  terms  of  outputs  or  perceived  impacts  or  results–  which  were  more  easily 
measured and compared across MSs – rather than actual impacts or results. 

Time lag between Programme implementation and the Ex Post Evaluation 

There was a time lag between the implementation of the Programme (2007-2013) and the 
ex-post evaluation (August 2016-August 2017). This meant that the relevant beneficiaries 
and  national  and  EU  authorities  and  experts  were  often  no  longer  working  in  the  same 
position and were thus difficult to reach, which is reflected in a low response rate to some 
of  the  consultations  (see  Annex  3).  The  external  evaluator  took  some  initial  measures  to 
mitigate  these  difficulties,  such  as  sending  several  email  reminders  to  the  persons 
surveyed,  searching  for  alternative  contacts  when  possible,  raising  the  individual  partner 
organisation  interviews to  two  rather  than  one, and  extending  the  deadline  for  the  online 
survey carried out as part of the case studied. Overall, all evaluation questions have been 
answered to some extent, thanks to the use of triangulation of data where possible. Where 
the evidence base has been deemed to be too low to be reliable, this has been indicated.  

Monitoring, accessibility of data and consistency of project data 

Issues have been noted with regard to the monitoring of project progress and final results 
which hindered the assessment of relevance, effectiveness, efficiency and EU added value. 

Firstly,  the  outputs/results  of  the  projects  were  not  centrally  collected  and  the  final 
assessments of projects could have been carried out more comprehensively, elaborating on 
particular elements or activities of the project and their potential future use. The effect of 
these  limitations  on  effectiveness,  efficiency  and  EU  added  value  is  further  explained  in 
sections 9.2 and 9.5. 

While relevant information could be collected for all evaluation questions, a full overview 
of final  financial data and project data, especially  with regard  to a final absorption  rate62 
was not consistently available across all projects, given that data on final payments was not 

61   The range of factors known to influence crime levels is wide and includes: social and economic factors 
(such  as  (youth)  unemployment  rates),  law-enforcement  policies  (e.g.  number  of  police  officers  or 
CCTV  coverage),  and  criminal  justice  policies  (including  changes  in  imprisonment  rates),  taxation 
policies (e.g. when related to smuggling of excisable goods), etc. 

62   The absorption rate is defined as a relation of funding spent to funding committed and it is expressed as a 

percentage. 

16 

 
 
 
 
 
                                                            
available  for  projects  which  were  not  yet  complete63.  The  data  drawn  from  different 
sources was not always consistent. This has limited the overall findings of the evaluation to 
some  extent.  Where  possible,  estimates  have  been  used  to  calculate  absorption  rates  and 
implementation trends. However, the figures relating in particular to the last programming 
year should be used with caution. Given that no estimates were available for the JRC data, 
no absorption rate has been calculated for the JRC element of the procurement sector. 

Another limitation which affects the assessment of relevance and  effectiveness is that no 
peer review of the products/outputs was undertaken and results could not be measured due 
to the lack of ex ante targets. It was not feasible to compare the "cost of projects'" under 
ISEC to other similar projects due to the absence of comparable projects. This hindered the 
assessment of efficiency and effectiveness to a large extent, as it was difficult to compare 
project  costs  and  to  measure  the  extent  to  which  the  same  objectives  were  met  by  the 
different types of projects in the various areas of organised crime addressed by ISEC. 

To  mitigate  these  limitations,  the  assessment  of  effectiveness  and  efficiency  had  to  rely 
largely  on  the  qualitative  data  collected  through  the  stakeholder  consultations.  This, 
combined  with  the  absence  of  a  baseline,  proved  to  be  a  limitation  in  terms  of 
methodology. 

6 STATUS OF IMPLEMENTATION  

Initially, ISEC had an allocated budget of EUR 532 million as specified in the AWPs for 
the  period  2007  to  2013.  However,  the  allocated  budget  decreased  for  the  programming 
year 2011 due to financial resources needed to cope with the migration crisis, bringing the 
total allocated budget down to EUR 522 million.  

The budget allocated per programming year varied, with EUR 45 million for 2007 being 
the lowest allocated in a given AWP and EUR 117.57 million for 2012 the highest. There 
was  another  peak  in  the  budget  in  2010  (i.e.  EUR  99  million).  Across  the  programming 
period, grants constituted 70% to 90% of an AWP’s total budget available. 

The minimum EU co-financing increased from 50 000 euro (period 2007 to 2009) to 80 
000 euro in 2010 and remained at 100 000 euro from 2011 onwards. The maximum rate of 
the EU co-financing increased consistently with the increase in minimum EU grant, from 
70%  (period  2007-2009)  to  80%  in  2010  and  stayed  at  90%  from  2011  onwards  for  the 
remainder of the Programme implementation period. 

In total, 782 ISEC grants were implemented across the programming period. The number 
of  grants  implemented  per  year  remained  largely  the  same  over  the  period  2007-2010 
(between 95 and 101), before rising to 144 in 2011 and peaking to 153 in 2012, consistent 
with an increased allocated budget for that financial year. 

More  action  grants  were  funded  via  open  calls  (569  grants,  representing  73%  of 
implemented  grants)  than  via  restricted  calls  (213  grants,  representing  27%).  There  were 
32 monopoly grants (4%) and 180 framework partnership grants (23%) overall. Only one 
operating grant was implemented in the funding period. 

63   The final calculations on absorption rates include all projects up to 30/03/2017. 

17 

 
 
 
 
                                                            
Given that targeted calls were only introduced from 2010 onwards, a much higher number 
of grants were funded under the general open calls (216 action grants under targeted calls 
and 353 action grants under general calls). Half of the grants funded under targeted calls 
were  in  response  to  calls  relating  to  the  areas  of  financial  and  economic  crime  and 
cybercrime. 

The  Commission’s  monitoring  table  provided  an  overview  of  the  specific  thematic  areas 
covered  by  ISEC  projects  (17  in  total),  which  in  turn  fall  under  the  four  more  general 
themes of the programme (see section 2.3). An analysis of the projects implemented shows 
that each of four thematic areas (police cooperation64, economic and financial crime, law 
enforcement,  and  human  trafficking)  accounts  for  over  100  grants  and  that  together  they 
represent  58%  of  the  total  number  of  grants.  The  lowest  number  of  grants  was 
implemented in the areas of victims and policing mass events. 

Figure 1: Number of grants implemented by thematic area 

64   ISEC  projects  categorised  as  police  cooperation  include  training,  exchange  of  information  and  data 

among police forces in different Member States. 

18 

 
 
 
 
 
 
                                                            
The demand-driven design of the AWP (see section 7) resulted in a significant geographical 
imbalance  in  terms  of  the  number  of  projects  funded  per  coordinating  organisation  in  each 
Member State and thematic areas covered per Member State. While organisations from Italy 
and  Germany  coordinated  115  projects  and  82  projects  respectively,  organisations  from  15 
Member States (AT, CY, CZ, DK, EE, EL, FI, IE, LU, LV, MT, PT, SE, SI, SK) coordinated 
20 projects or fewer, with Malta and Luxembourg at the bottom of the scale (these countries 
only coordinated two projects each).  

When  taking  into  account  also  the  Member  State  location  of  partner  organisations  which 
participated in the projects, the outlook is similar with organisations from Italy (303), United 
Kingdom  (244),  Romania  (172),  Germany  (152)  and  Spain  (144)  being  particularly  active, 
while organisations from Luxembourg and Malta were relatively inactive in the Programme 
(with participation in 20 or less projects).  

Low levels of participation were  found to  be linked to  unfamiliarity with  the Programme, a 
lack  of  institutional  capacity  to  develop  project  proposals  and  to  understand  administrative 
requirements  (including  requirements  for  co-financing)  as  well  as  a  lack  of  contacts  and 
partners  in  other  EU  Member  States.  One  issue  highlighted  by  Member  States  was  the 
difficulty encountered in finding partners due to the different organisational structures across 
Member  States.  Another  issue  experienced  by  an  Eastern  European  Member  State,  which 
prevented them from applying, was the requirement for an application to respect a minimum 
value threshold65 which was considered by the Member State to be very high. 

In  order  to  achieve  a  better  geographical  spread  by  raising  awareness  of  the  Programme, 
regional  information  days  were  introduced;  these  took  place  in  seven  Member  States  (AT, 
HU, IT, RO, SE, SI, and the UK) in 2010, 2011 and 2012, encouraging participation in the 
Programme.  Member  State  representatives  were  also  asked  to  promote  the  Programme 
nationally. 
From  a  sample  of  151  projects  selected  for  in-depth  analysis66,  51%  of  all  projects  were 
coordinated  by  either  a  law  enforcement  agency  or  a  national  ministry.  Non-governmental 
organisations/  civil  society  organisations  and  university  or  research  organisations  were  also 
common coordinating organisations, representing 17% and 13% respectively. Other types of 
coordinating organisations were local authorities (6%), judicial authorities (5%), commercial 
entities from the private sector (3%), regional authorities (2%), European Networks, platforms 
or forums (1%) and other types (3%). 

6. 1 Absorption Rates of the ISEC Programme 

A total budget of EUR 522 million was allocated67 to the ISEC Programme of which just over 
EUR 413 million was committed and EUR 304 million spent (including funding provided via 

65  EUR 100,000 in 2011, EUR 50,000 in 2007-2008, and EUR 80,000 in 2010. 
66   This sample represents 27% of the 568 projects which were finalised by 30 June 2016. 
67   While the total amount for the ISEC Programme according to the planned funding for 2007-2013 was just 
over  EUR  532  million,  this  figure  takes  into  account  the  final  allocation  to  all  funding  mechanisms  and 
includes the changes brought to the initially allocated budget through the modifying decisions mentioned at 
the beginning of the section. 

19 

 
 
 
                                                            
action  grants  and  funding  provided  via  public procurement).  Not  all  available  funding  from 
the various calls was awarded, mostly due to project applications not meeting  the eligibility 
criteria  or  the  quality  criterion  for  being  awarded  funding.  Over  the  programming  period, 
ISEC had an overall absorption rate of 74%. 

Absorption rates of action grants (under open and restricted calls) 

Under  the  782  action  grants  implemented  in  the  ISEC  Programme,  a  total  of  EUR  352.9 
million  was  committed  and  EUR  261.2  million  was  spent,  with  a  total  absorption  rate  of 
74%68.  This  percentage  is  slightly  lower  than  commonly  observed  in  other  EU  funding 
programmes69. 

Grants under open calls: The majority of funds were committed and spent (73%) on action 
grants  under  open  calls  (general  and  thematic  calls)  which  generated  the  highest  number  of 
projects (569). 180 of the 782 grants selected for in-depth analysis were implemented under 
framework  partnership  calls,  representing  21%  of  budget  expenditure.  Overall,  over  the 
period  2007-2013  absorption  increased  from  68%  in  2007  to  79%  in  2013.  The  2011  ISEC 
interim  evaluation  report  had  found  a  poor  absorption  of  the  allocated  budget  and  it  was 
considered  likely  that  this  would  continue  in  the  following  years,  however,  a  clear  upward 
trend in terms of absorption across the period can be discerned (see Figure 2), following the 
use  of  mitigating  measures  such  as  the  modification  of  funding  mechanisms  following  the 
mid-term evaluation of the programme. 

Initially,  as  part  of  the  2007-2009  AWPs,  only  general  calls  were  included  under  the  open 
calls  for  proposals.  In  the  2010  AWP,  targeted  calls  were  added,  to  improve  focus  on  the 
priorities  outlined  in  the  AWPs.  The  targeted  calls  were  introduced  to  ensure  a  better 
distribution  across  key  themes  and  ensure  a  critical  mass  of  projects  within  each  thematic 
area,  in  agreement  with  the  Member  States  in  a  comitology  procedure.  Furthermore,  the 
targeted  calls  were  expected  to  speed  up  the  selection  procedure.  Before  this  change, 
difficulties were encountered in evaluating and comparing a very diverse pool of applications, 
given the broad thematic scope of ISEC, whereas with the adoption of the targeted calls it was 
possible  to  compare  between  and  within  themes.  The  targeted  calls  were  introduced  first  in 
the beginning of each year, with the general calls following in the spring/summer of the same 
year. This also made it possible to transfer any underspend on targeted calls into the general 
calls,  thus  ensuring  that  funds  could  be  allocated  to  more  projects  which  allowed  for  better 
absorption of the available funds. 

68   The absorption rates are calculated using the financial data included in the AWPs, the modifying decisions 
applied on these amounts and the DG Budget ABAC system. It should be noted that a total of 195 projects 
(1 in 2009, 6 in 2010, 35 in 2011, 68 in 2012 and 85 in 2013) were not yet closed. For these projects, the 
final invoice amount, as recorded in ABAC, has been used for the calculation of the absorption rate). The 
figures are still subject to potential change but it is not expected to be very significant as most projects used 
in the calculation of the absorption rate, i.e. 75%, are finalised. 

69   Several programme evaluations, including Youth in Action, the EGF, the ESF and the EU Structural Funds 
suggest that an absorption rate >80% is acceptable especially when the programme is introducing innovation 
and/or requiring new stakeholders to work together. A ‘typical’ absorption evolution starts between 60-70% 
to then, towards the end of the programme period, arrive at 80-95% (and in some cases even 100%). 

20 

 
 
 
                                                            
The increased effective uptake of resources over time could also be explained by the changes 
made to the Programme’s application procedures after the budgetary revision in 2010, which 
meant that project budgets were scrutinised before they were selected and awarded rather than 
at the stage of award and grant agreement. Between 2010 and 2011 the largest year-on-year 
improvement can be observed in the absorption rate (an increase of 7 percentage points). 

Figure 2: Committed and spent budget and absorption rate by AWP year  

Source: Data extracted from ABAC as of 31/03/2017. 

Grants  under  targeted  calls:  The  average  value  of  funds  committed  to  projects  under  the 
targeted  calls  (EUR  550  000)  is  almost  twice  as  high  as  the  average  value  of  projects 
committed under the general calls (EUR 350 000), although this is due to the very high value 
of one specific project implemented under the PNR targeted call, representing over EUR 17 
million70.  When  excluding  this  project,  the  average  value  of  funds  committed  to  projects  in 
the other ISEC targeted areas (EUR 378 000) is similar to an average value of projects under 
the general calls scheme. 

Grants  under  restricted  calls:  There  was  an  overall  lower  absorption  rate  over  the  2007-
2013 period of action grants within framework partnerships (70%) compared to other types of 
projects (other types of action grants – 77%, Grants to bodies in monopoly situations – 76%). 
This might be explained by the high number of projects that were not fully implemented or 
poorly  budgeted  in  the early  years  of  the  Programme  implementation  as  the  absorption  rate 
increased over time to 86% in 2013. 

70   Project implemented by French General Directorate of Customs and Excise in 2012. 

21 

 
 
 
 
 
                                                            
Framework  partnerships  were  established  following  a  call  for  applications  and  aimed  to 
provide  for  cooperation  with  ‘trusted  partners’,  such  as  public  bodies  or  organisations  with 
country-wide  responsibility  with  legal  personality  established  in  the  Member  States  (other 
types  of  bodies  were  not  eligible  to  apply).  Framework  partnerships  were  concluded  for  a 
period of four years and framework partners could undertake individual projects that may last 
up to  three years. The AWPs outlined the priorities  for framework partnerships  which  were 
very similar to those in the open calls. However, the use of framework partnerships did not 
have  the  intended  effect:  consulted  stakeholders  found  that  framework  partnerships  as  a 
funding  mechanism  did  not  provide  the  expected  level  of  flexibility  due  to  a  lengthy  and 
reportedly  bureaucratically  heavy  application  process.  In  addition,  both  open  and  restricted 
calls  had  very  similar  priorities71.  Under  the  successor  fund,  ISF-Police,  framework 
partnerships are no longer used. 

The procurement sector 

A total of EUR 60.6 million was committed during the Programme through procurement, of 
which  EUR  51.6  million  was  committed  to  various  procurement  items  and  nearly  EUR  9 
million  to  the  JRC.  Of  the  51.6  million  committed  during  the  Programme  to  procurement 
items, EUR 42.9 million was spent by 31 March 2017, representing an absorption rate of 83% 
(in total 349 public procurement projects were implemented). 

In addition, 14 administrative arrangements or actions with JRC were carried out. Originally, 
a budget of EUR 8.87 million was committed to actions and administrative arrangements with 
the JRC. The largest project had a commitment of EUR 2.3 million, and the smallest project 
was EUR 70 000.72 Final data on project expenditure was not available at the time of writing 
the  evaluation  report,  so  it  has  not  been  possible  to  provide  an  estimate  of  the  final  spent 
amount under the JRC actions. 

71   For example, although projects to support joint investigation teams (JITs) were specifically included in the 
calls  for  proposals  of  the  framework  partnerships,  they  were  also  funded  as  part  of  action  grants  through 
open calls for proposals. The long time needed to finalise the grant agreement (as well as amendments) is 
intrinsically incompatible with the nature of investigative work. In ISF-Police, the joint investigation teams 
have  been  allocated  under  the  eligible  actions  of  the  national  programmes  (under  the  shared  management 
modality)  and  the  ‘framework  partnership’  does  not  exist  in  the  new  Programming  period  as  a  type  of 
action. 

72   These figures are based on data extracted from ABAC. Final data on project expenditure was not available 

at the time of writing the report. 

22 

 
 
 
 
                                                            
7. ANSWERS TO THE EVALUATION QUESTIONS 

7.1 Relevance 

EQ1.  To  what  extent  did  the  objectives  and  the  actual  results  of  the  ISEC  programme 
correspond to the needs related to the prevention of and fight against crime? 

The assessment of the relevance of ISEC focused on the extent to which the objectives and 
the results of the programme corresponded to the needs related to the prevention of and fight 
against crime. 

With  regard  to  the  objectives,  the  ISEC  Programme  was  created  to  support  the 
implementation  of  the  EU  acquis  and  policy  initiatives  in  the  areas  of  prevention  and  fight 
against  crime.  The  nature  of  the  policy  areas  covered  by  ISEC  requires  a  transnational 
response. For example, in the area of trafficking in human beings, Eurostat statistics showed 
that the majority of registered victims (65%) come from other EU  Member States.73 Policy-
wise, in the backdrop of a single market and a Schengen area, facilitating ‘mutual trust’ and 
cooperation  among  law  enforcement  and  other  practitioners  in  the  Member  States  has  been 
generally seen as vital. 

Furthermore,  based  on  the  subsidiarity  principle,  ISEC  funding  was  not  meant  to  substitute 
national funding in the internal security area but rather to complement it, providing increased 
support  to  cross-border  cooperation.  In  this  sense,  specific  objective  (a)  relating  to 
“coordination, cooperation and mutual understanding among law enforcement agencies, other 
national authorities and related EU bodies”74 is highly relevant to the needs in the sector, as 
confirmed in the replies to the stakeholder consultation. Different categories  of stakeholders 
saw  such  cross-border  cooperation  as  highly  necessary,  especially  in  the  context  of 
internationalisation  of  crime  and  professionalization  of  criminal  groups.  Consulted 
stakeholders  agreed  that  given  the  financial  crisis  which  took  place  during  the  Programme 
period, transnational cooperation would not have been financed through alternative sources of 
funding, such as national budgets. 

Similarly,  specific  objective  (b)  relating  to  the  “promotion  and  development  of  horizontal 
methods and tools”75 was also seen by consulted stakeholders as relevant to the needs. ISEC 
has supported the development of tools, including large IT systems for the development of EU 
flagship  initiatives  in  crime  prevention  –  i.e.  Passenger  Name  Record  (PNR),  Prüm  and 
FIU.net76.  In  addition  to  IT  systems,  cross-border  responses  to  crime  require  common  tools 
and methodologies and the facilitation of their practical operationalisation. In this sense, this 
specific  objective  is  also  relevant  to  the  needs  in  the  area  of  prevention  and  fight  against 
crime.  

Overall, the relevance of these objectives has been further confirmed by the fact that 88% of 
participants  in  ISEC  projects  surveyed  responded  that  the  objectives  of  the  projects 
corresponded to their needs and/or the needs of their organisations. 

73   Eurostat (2015), Statistical Working Papers, Special edition on trafficking in human beings. 
74   Defined under Article 3(2) of Decision No 2007/125/JHA. 
75   Defined under Article 3(2) of Decision No 2007/125/JHA. 
76   FIU.net  is a  decentralised  and  sophisticated  computer  network  supporting  the  Financial  Intelligence  Units 

(FIUs) in the European Union in their fight against money laundering and the financing of terrorism. 

23 

 
 
                                                            
Specific objectives (c) and (d) concerned the development of best practices for protection and 
support  to  witnesses  and  victims.  Whereas  the  2007  and  2008  AWPs  included  these 
objectives as a separate priority area under the open calls for proposals, in the 2009 to 2013 
AWPs, the protection of victims was included as a transversal element within specific crime 
areas  included  under  the  different  priority  areas,  general  and  targeted  calls  focusing  on 
victims  of  human  trafficking,  victims  of  terrorism,  victims  of  radicalisation  and  organised 
crime and child victims of sexual exploitation. However, the ‘support to witnesses’ was not 
explicitly mentioned after the 2008 AWP, except as part of the objectives, while the 'support 
to  victims'  was  mentioned  in  2009-2013  as  priority  areas  for  the  open  and  targeted  calls 
relating  to  various  types  of  victims.  This  is  because  the  issues  relating  to  witnesses  and 
victims would be mainly covered under the Daphne III programme while under ISEC support, 
to victims was kept as a priority when it related to the prevention and fight against crime. This 
also meant that overlaps with the Daphne III were avoided (see section 9.4). 

These  last  two  specific  objectives  attracted  slightly  less  attention  than  the  two  first  ones  in 
terms  of  the  number  of  projects  implemented,  especially  with  regard  to  ‘support  for 
witnesses’ (please see section 6, figure 1 for an overview of number of projects implemented 
in these thematic areas). 

Demand-driven design of the Programme 
Whilst the ISEC’s objectives were ‘fixed’ in the founding legislation77 i.e. Council Decision 
2007/125/JHA,  the  continuous  relevance  of  the  Programme  was  ensured  by  the  process  of 
setting  the  ISEC  priorities  and  sub-priorities  which  were  identified  by  Member  States  and 
competent  Directorate-Generals  of  the  Commission78  on  a  yearly  basis  and  included  in  the 
AWPs. This allowed the Programme a degree of flexibility when adapting to changing needs. 
Member  State  representatives  as  well  as  Commission  representatives  considered  that  the 
process had run smoothly and their comments had been taken on board in the final approved 
versions.  

The  process  of  priority  formulation  and  AWP  adoption  lasted  about  six  months,  and  was 
followed by the publication of the calls for proposals. The AWPs reflected the priorities in the 
year before – following the ‘n+1 year’ model, ‘n’ being the year of the adoption of the AWP 
and  the  implementation  taking  place  one  year  later.  This  resulted  in  a time  lag  between  the 
formulation of priorities and their actual implementation. 

Some stakeholders commented that given the broad scope of the Programme, there were no 
major issues with the ‘n+1 year’ model, but a few suggested that the Programme could have 
better reflected the changing nature of law enforcement priorities. In their view, the design of 
ISEC made it difficult to be highly responsive or flexible to emerging needs, due to the time 
lapse of a full year between the drafting of the AWPs and the launch of the calls for proposals. 
This meant that specific areas might no longer be as relevant or as high on the political agenda 
as they were one year earlier. This was addressed in the successor funding instrument, ISF-

77   Council Decision 2007/125/JHA,  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2007:058:0007:0012:EN:PDF  

78   The  process  mentioned  for  the  identification  of  priorities  and  sub-priorities  was  carried  out  in  regular 
consultation  with  the  Member  States,  through  a  comitology  procedure  and  an  inter  service  group  with 
relevant DGs competent in the area of protection of critical infrastructure in the Commission, including DG 
CNECT, DG ECHO, DG SANCO, DG MOVE, DG ENER and the JRC. 

24 

 
 
 
                                                            
Police,  through  the  inclusion  of  emergency  assistance  “to  address  urgent  and  specific 
needs”.79 

Regarding the priorities set in the calls for proposals, most ISEC coordinating organisations 
and  partner  organisations  considered  these  to  be  relevant.  In  their  response  to  a  survey,  the 
coordinating  organisations  considered 
to  specific 
strategic/operational needs at national (62 responses or 66%) and EU level (72 responses or 
77%), and to the specific needs of target groups/ related stakeholders in the different thematic 
areas  (55  responses  or  60%).  Similar  percentages  of  co-beneficiaries  reported  the  same, 
although their response rate to the survey is much lower (35 responses in total).  

ISEC  priorities 

relevant 

to  be 

Project  participants  responded  very  similarly 
to  project  coordinating  and  partner 
organisations.  An  overwhelming  majority  (97%  or  91  responses)  of  participants  considered 
that  the  priorities  in  the  calls  for  proposals  corresponded  to  the  needs  at  EU  level,  while 
slightly less so to priorities at national level (94% or 88 responses) and the needs of the target 
group and related stakeholders in the respective thematic area (94% or 88 responses).  

Although a very limited number of responses (3) were received for the public consultation on 
ISEC conducted, two of the respondents stated that the projects and activities funded in their 
respective Member States (DE and IT) addressed needs related to the prevention of and fight 
against crime. 

Overall, the ISEC programme was considered relevant in terms of results, as these addressed 
the  needs  identified  through  the  initial  formulation  of  the  ISEC  programme  specific 
objectives.  However,  as  mentioned  in  section  5,  the  assessment  of  relevance  in  terms  of 
results  remains  limited  by  the  lack  of  a  central  repository  of  ISEC  project  deliverables  and 
results. 

The main results identified from ISEC projects were: enhanced networking, forms of sharing 
information  and  best  practices,  increased  levels  of  knowledge  and  skills  of  practitioners, 
improved  understanding  of  methods,  tools  and  practices  and  increased  take  up  of 
new/transferred methods, tools and practices.  
In-depth analysis of 151 projects80 showed that the most significant results of ISEC projects 
were enhanced networking and forms of sharing information and best practices (reported for 
75%  of  projects)  and  increased  levels  of  knowledge  and  skills  of  practitioners  (in  74%  of 
projects).  This  is  followed  by  improved  understanding  of  methods,  tools  and  practices  (in 
50% of projects) and increase take up of new/transferred methods, tools and practices (in 23% 
of  the  projects).  Results  relating  to  behavioural  changes  amongst  target  groups  were  less 
salient (reported only for 13% of projects). Please note that each project could have had more 
than one result. 

These  findings  are  supported  by  information  gathered  through  stakeholder  interviews,  in 
particular  with  authorities  in  the  Member  States,  whereby  the  programme  was  seen  as 
contributing  positively  to  the  results  obtained  in  the  different  EU  policy  areas.  Enhanced 
networking  and  cooperation  between  authorities  across  Member  States  were  identified  by 
several  stakeholders  as  one  of  the  main  results  of  the  ISEC  Programme.  Building  trust, 

79   Article 10, Regulation No 513/2014. 
80   Please note that each project could have had more than one result. 

25 

 
 
 
                                                            
increasing Member States’ motivation to work together and “improved Community feeling” 
were described as important outcomes of the Programme’s actions. 

Bearing  in  mind  that  law  enforcement  authorities  have  been  the  main  target  group  of  ISEC 
projects, stakeholders highlighted that ISEC activities “gave real and concrete added value to 
law enforcement authorities”.  

7.2 Effectiveness 

The assessment of effectiveness sought to measure the extent to which the ISEC programme 
contributed to its general and specific objectives81. Therefore, each of the evaluation questions 
answered in this section focuses on one of the programme’s general or specific objectives (see 
section 3): 

EQ2.  Did  the  ISEC  programme  contribute  to  better  security  for  citizens  and  if  so,  to  what 
extent?  To  what  extent  did  the  ISEC  programme  contribute  in  the  areas  of  preventing  and 
combating crime, organised or otherwise, in particular terrorism, trafficking in persons and 
offences  against  children,  illicit  drug  trafficking  and  illicit  arms  trafficking,  corruption  and 
fraud? 

As  mentioned  in  section  5,  the  exact  extent  to  which  ISEC  contributed  to  this  first  general 
objective  is  not  possible  to  quantify  and  it  is  difficult  to  establish  a  causal  relationship 
between ISEC actions and the impact of better security for EU citizens.  The evaluation also 
highlighted issues with regard to the monitoring of project progress and final results, which 
hindered the assessment of effectiveness  

Bearing these limitations in mind, overall, the evaluation findings suggest that ISEC achieved 
all of its objectives to some extent and achieved the expected outputs, which, it can be argued, 
contributed to crime prevention and, ultimately, to better security for EU citizens. The main 
results  identified  from  ISEC  projects  were  enhanced  networking,  forms  of  sharing 
information and best practices and increased levels of knowledge and skills of practitioners. 
This is followed by improved understanding of methods, tools and practices and increase take 
up  of  new/transferred  methods,  tools  and  practices.  The  majority  of  projects  accomplished 
their planned activities and achieved their objectives. In addition, ISEC played a key role in 
the  development  of  stable  transnational  cooperation  between  Member  States  and  their 
agencies  in  the  areas  of  crime  prevention  and  combatting,  as  well  as  better  exchange  of 
information, particularly in the field of forensics, drugs, PNR, cybercrime and trafficking  in 
human  beings.  National  authorities  interviewed  also  reported  that  ISEC  projects  fostered 
cooperation  between  agencies  in  the  same  Member  State,  increasing  synergies  between 
national Law Enforcement Agencies. 

Supporting these findings, most project coordinating organisations, partner organisations and 
participants surveyed perceived ISEC projects to have contributed to a great extent to better 

81   In order to do so, each of the evaluation questions answered in this section focus on one of the programme’s 
general or specific objectives: EQ2 deals with general objective 1; EQ3 deals with general objective 2; EQ4 
deals with specific objective a); EQ5 deals with specific objective b); EQ6 deals with specific objective c); 
and EQ7 deals with specific objective d). 

26 

 
 
 
 
                                                            
security for EU citizens. Only 13% of participants, 4% of coordinating organisations and none 
of  the  partner  organisations  respectively  thought  that  ISEC  project(s)  had  made  no 
contribution at all. This is further supported by the in-depth analysis of a sample of projects 
(151 in total), which shows that the majority of projects analysed achieved all their objectives 
(84%), while 15 % of projects partially achieved their objectives and only 1% (2 projects) did 
not. Similarly, the final assessments made of the project sample found 75% of projects to be 
excellent  or  very  good  with  beneficial  results  and  impacts  achieved;  20%  were  assessed  as 
acceptable/average, while 5% were marked as poor or very poor. 

In terms of achieving the project objectives, project coordinating organisations (91) responded 
that  the  objectives  were  fully  achieved  (82  responses),  while  only  nine  responses  found  the 
objectives  to  be  only  partially  achieved.  With  regard  to  the  quality  of  the  outputs,  the  final 
evaluations on projects completed at the end of the projects show that, in certain cases, some 
outputs  could have  been  more comprehensive, better  elaborated, and/or designed  in a better 
way to have a greater impact. Comments of this nature are made in about 15% of the sample 
of  projects  (5%  of  the  projects  were  rated  as  ‘poor’  or  ‘very  poor/not  acceptable’  and  10% 
were rated as ‘acceptable /average’). In a few instances, the transnational projects in question 
had a strong national focus and did not seemed to benefit the partner organisations from other 
Member States (whose involvement in the project was judged to have been rather limited) to a 
limited extent. 

EQ3. To what extent did the ISEC programme contribute to the development of the policies of 
the Union? 

Regarding  the  extent  to  which  the  Programme  contributed  to  the  second  general  objective, 
ISEC  projects  contributed  to  a  wide range  of  EU  policies  in the  field  of prevention  and  the 
combatting  of  crime,  such  as  the  Prüm  Decision,  Passenger  Name  Record  (PNR),  the  EU 
Action  Plan  on  the  security  of  Chemical,  Biological,  Radiological  and  Nuclear  (CBRN) 
materials, the EU Policy Cycle to tackle organised and serious crime (EMPACT), as well as 
contributing to the implementation of several EU Directives82. 

The PNR is one of the key flagship EU initiatives in the area of crime prevention supported 
by  ISEC  funding.  Directive  2016/681  aims  to  regulate  the  transfer  from  the  airlines  to  the 
Member States of PNR data of passengers of international flights, as well as the processing of 
this  data  by  the  competent  authorities.  ISEC-funded  projects  amounting  to  over  EUR  50 
million have been implemented in the area of PNR, with 15 Member States benefitting from 
these  projects.83  These  projects  supported  the  development  of  Passenger  Information  Units 
(PIUs) in the Member States for the collection and processing of PNRs and included a range 
of related activities, including the purchase of hardware and software; training and capacity-

82   ISEC  contributed  to  EU  Directives  such  as:  Directive  2011/93/EU  on  combating  sexual  abuse  and  sexual 
exploitation  of  children,  and  child  pornography;  Directive  2003/98/EC  on  the  re-use  of  public  sector 
information;  Directive  2011/62/EU  amending  Directive  2001/83/EC  on  the  Community  code  relating  to 
medicinal  products  for  human  use,  as  regards  the  prevention  of  the  entry  into  the  legal  supply  chain  of 
falsified  medicinal  products;  Directive  2011/36/EU  on  preventing  and  combating  trafficking  in  human 
beings and protecting its victims. 

83   AT, BG, EE, FR, FI, HU, IT, ES, LT, LV, NL, PT, RO, SI, SE. 

27 

 
 
 
 
                                                            
building for data transfers and data analysis as well as profiling and matching of PNR data. As 
the PNR systems became operational in some Member States only at the beginning of 2017, 
and  in  some  cases  later  due  to  technical  difficulties,  it  is  not  possible  to  comprehensively 
examine  the  effectiveness  of  projects  in  this  area.  However,  several  interviewed  Member 
State authorities84 saw the PNR projects as examples of successful ISEC contributions which 
are  expected  to  have  a  tangible  result.  Indeed,  considerable  financial  support  for  the 
implementation  of  PNR  continues  to  be  provided  to  Member  States  through  the  successor 
fund, ISF-Police.85 
The  Prüm  Decisions86  are  a  key  EU  policy  initiative  and  aim  at  closer  police  and  judicial 
cross-border cooperation in combating terrorism and cross-border crime, in particular in terms 
the 
of  exchange  of 
implementation of the Prüm Decisions amounting to over EUR 17 million and covering most 
EU  Member  States.  Projects  focused,  among  other  things,  on  building  operational  and 
technical capacity; the provision of equipment and automated solutions and training to users. 
The Projects aimed at the faster processing of crime-scene evidence, leading to more accurate 
DNA profile and fingerprint identification.  

information.  Over  30  projects  were 

implemented 

to  support 

Evidence  of  the  impact  of  these  projects  on  the  implementation  of  the  Prüm  Decisions  was 
drawn  from  the  analysis  of  project  final  reports.  In  Latvia,  a  2009  ISEC-funded  project87 
supported  the  understanding  of  technical  capacities  of  the  Prüm  DNA  data  exchange  and 
developed  software  for  the  Prüm  interface,  contributing  to  the  implementation  of  the 
Decision.  Similarly,  a  2009  ISEC-funded  project  implemented  in  Estonia88  established 
international  online  functionalities  of  the  Automated  Fingerprint  Identification  System 
(AFIS),  making  it  compliant  with  the  requirements  of  the  Prüm  Decisions.  A  project  in 
Germany established Mobile Competence Teams (MCT) which contributed to an exchange of 
methods and tools between Member States and supported ten Member States successfully in 
implementing the Decisions.  

Interviewed  Member  State  authorities  saw  such  projects  as  key  examples  of  ISEC’s 
contribution  to  the  exchange  and  management  of  information  and  intelligence  for  law 
enforcement.  According  to  interviewees,  projects  contributed  to  develop  functionalities  for 
exchanging data with other Member States which will be used on a daily basis for the Prüm 
data exchange. 

EQ4. To what  extent  did  the  ISEC programme contribute to promotion and development of 
horizontal  methods  and  tools  necessary  for  strategically  preventing  and  fighting  crime  and 
guaranteeing security and public order, such as the work carried out in the European Union 
Crime  Prevention  Network,  public-private  partnerships,  best  practices  in  crime  prevention, 

84   Authorities from EE, LT, LV, RO and SI. 
85   The recent "top-up" in financial support provided to Member States for PNR through ISF-Police highlights 

the importance given to developments in this area. 
86   Council Decisions 2008/615/JHA and 2008/616/JHA. 
87   Project “Building up the technical capacity of the State Police of the Republic of Latvia to implement the 
DNA data sharing according the Council decisions 2008/615/JHA and 2008/616/JHA. (Prüm Decisions)”. 
88   2009 AG project “Establishing the international online functionality of the existing Estonian AFIS according 

to Council Decision 2008/615/JHA”. 

28 

 
 
 
                                                            
comparable  statistics,  applied  criminology  and  an  enhanced  approach  towards  young 
offenders? 

From  a  sample  of  151  projects  analysed,  60%  included  as  an  activity  relating  to  specific 
objective  a),  the  development  of  horizontal  methods  and  tools;  these  were  spread  evenly 
across  the  different  thematic  areas  of  the  ISEC  Programme.  The  vast  majority  of  surveyed 
project  participants  (87%),  co-beneficiaries  (80%)  and  coordinators  (86%)89  considered  that 
ISEC projects had made a contribution to the achievement of the promotion and development 
of horizontal methods and tools in this area. Horizontal methods and tools for preventing and 
fighting crime include: 

 

investigation tools and techniques; 

  software tools, databases and technical solutions;  

  development of methodological frameworks, methodologies and the sharing of best 

practices. 

Investigation tools and techniques 

Among  the  sample  of  151  ISEC  projects  analysed  as  part  of  this  evaluation,  a  total  of  24 
projects  reported  to  have  had  outcomes  related  to  investigations.  Such  outcomes  were 
achieved  either  through  carrying  out  field  investigations  involving  transnational  law 
enforcement agencies (and the creation of Joint Investigation Teams  – JITs), or through the 
development  of  tools,  trainings,  guidelines  and/  or  best  practices  with  the  aim  to  increase 
investigators’  efficiency  and  effectiveness.  Projects  final  reports  show  several  examples  of 
positive results achieved by ISEC projects, some of which brought to the arrest and detention 
of  criminals  as  well  as  the  seizure  of  large  quantities  of  drugs.  For  example,  the  project 
“European Joint Investigation Teams in cross-border cold case review situations” supported a 
joint investigation between the UK, the Netherlands and Eurojust on several known criminal 
cases and involving the prosecution of a suspect in the UK (the suspect was convicted to a life 
sentence).90 Furthermore, the project created a JIT manual which will support knowledge and 
experience sharing with other Member States.  

Similarly,  the  project  “Pre-Trial  Investigation  M3”  developed  best  practices  in  fighting 
international  corruption  and  investigated  several  serious  crimes  with  the  aim  to  deter 
international  corruption  on  a  general  level  and  confiscate  criminal  profits.  It  also  fostered 
coordination and operational cooperation among law enforcement agencies (and other actors) 
of  Finland,  Croatia  and  Austria  as  well  as  Eurojust,  and  established  an  international  JIT, 
which was reported to be an effective method for fighting international corruption. Member 
State  authorities’  representatives  interviewed  also  confirmed  the  positive  results  of  ISEC 
projects supporting investigation activities. One Member State mentioned that ISEC funding 
made a significant contribution to investigations, as it gave national law enforcement agencies 

89   In  total  95  project  coordinators,  102  participants  and  32  co-beneficiaries  were  provided  responses  to  the 

survey. 

90   Final Report of project “European Joint Investigation Teams in cross-border cold case review situations”. 

29 

 
 
 
                                                            
the financial and technical ability to cooperate with and include third countries (e.g. non-EU 
countries) in investigation activities. 

Overall, the development of horizontal methods and tools has been found to have a potentially 
strong  multiplier  effect.  Innovative  investigative  methodologies  and  tools  on  specific  crime 
areas (e.g. counterfeit medicines, VAT fraud, methods in forensic science, etc.) can be tested 
in  a  few  Member  States  and  then  replicated  across  others.  Nevertheless,  the  evaluation 
showed  that  collecting  all  different  results  of  ISEC  with  a  view  to  ensuring  their  potential 
future  use  (instead  of  only  relying  on  dissemination  efforts  of  project  promoters  whose 
abilities may be limited), should have been taken up centrally at Commission level (or by one 
of its agencies). 

Software tools, databases and technical solutions 

The adoption of the PNR system has been mentioned by representatives from several Member 
States as an important tool for data exchange. Also in one Member State, ISEC contributed to 
the  implementation  and  update  of  three  software  programmes  which  helped  police  officers 
with their daily work. According to the interviewee, this was a large-scale and large-impact 
software, which also facilitated the exchange of information with other Member States. 

Data collected through the analysis of projects’ final reports show that only a limited number 
of  ISEC  funded  projects  covered  the  collection  and/or  sharing  of  comparable  statistics 
throughout the years of implementation of the Programme. A total of 15 projects (out of the 
151 projects) mentioned the collection/sharing of statistics as a project activity or objective, 
thus  showing  a  low  level  of  implementation  of  such  activities.  Furthermore,  none  of  the 
Member  State  representatives  interviewed  as  part  of  the  evaluation  reported  evidence  of 
results  in  terms  of  development  of  statistical  data.  However,  as  reported  in  final  reports 
analysed  as  part  of  the  sample  database,  some  projects  contributed  to  the  development  of 
comparable  statistics  or  tools  for  the  sharing  of  statistical  data  on  crime.  For  example,  the 
2007 project “European Statistical Database of Lethal Violence (ESDoLV)” aimed to lay the 
foundations  for  a  joint  European  statistical  database  on  lethal  violence.  Member  States 
involved in the project provided national statistics which were successfully compiled into one 
dataset  named  the  European  Homicide  Monitor  (EHM),  which  can  be  used  for  detailed 
analyses of trends, factors that foster lethal violence, preventive measures, sentencing policy 
and treatment of perpetrators. Another similar project produced a tool for the validation and 
utilisation  of  EU  statistics  on  human  trafficking  (TrafStat),  contributing  to  an  increased 
knowledge at EU level of THB data and statistics. 

Indeed, databases, IT systems and e-Platforms also have a potentially high multiplier effect as 
multiple  users  can  be  connected  electronically.  However,  projects  in  this  area  encountered 
several obstacles which hindered their effectiveness.  

For  example,  interviews  with  the  Europol  Financial  Intelligence  Unit  (FIU)  on  its 
decentralised  computer  network  supporting  the FIUs  in  the EU  in  their fight  against  money 
laundering and the financing of terrorism - the FIU.net91 - revealed a number of issues. The 
set-up  of  FIU.net  has  been  financed  through  ISEC  two-year  grants  amounting  to 

91   Council Decision 2000/642/JHA. 

30 

 
 
 
                                                            
approximately  EUR10  million  in  the  period  2007-2015.  FIU.net  was  led  by  the  Dutch 
Ministry of Security and Justice until 2016 and since 2016, the management was assumed by 
Europol. Europol is now responsible for operational developments, training initiatives and the 
operational running of the system. Before the beginning of the process of embedding it into 
the  Europol  system,  Europol  carried  out  a  number  of  assessments  at  infrastructure  level, 
programme  level,  DPO  assessment,  and  cryptographic  assessment  of  the  FIU.net.  Based  on 
these assessments, weaknesses were revealed in the security of the system. 

Another example was provided by a French authority, who mentioned issues encountered in 
the  implementation  of  PNR.  These  revolved  around  the  complexity  of  data  collection  when 
working with the airlines. The administration did not envisage at the outset of the project the 
degree  of  complexity  and  disparities  in  the  norms  and  technical  applications.  Due  to  the 
complexity  of  projects  such  as  those  funded  for  the  implementation  of  the  PNR,  more 
emphasis could have been placed on the monitoring of project implementation and results by 
technical experts and possibly, through the Programme, on creating opportunities for relevant 
organisations in the Member States which received such funding to exchange experiences and 
share  lessons  learnt  on  similar  activities.  This  is  partly  addressed  under  the  successor  fund 
ISF-Police,  as  regular  meetings  take  place  between  the  participating  Member  States  in  the 
form of the "AMIF-ISF Committee meetings", where Member States exchange best practices, 
share experiences and can provide feedback or direct questions about such issues. 

Development  of  methodological  frameworks,  methodologies  and  sharing  of  best 
practices 

One of the most tangible contributions in this area is the European Crime Prevention Network 
(EUCPN), funded through ISEC.92 EUCPN aims to facilitate the exchange of knowledge and 
accumulate  best  practices  and  information  in  the  area  of  crime  prevention.  In  2012,  an 
external evaluation was commissioned93 which found that EUCPN performed relatively well 
and has made good progress in relation to the objectives. However, a number of shortcomings 
were  identified  in  relation  to  the  added  value  of  some  of  the  contact  points  in  relation  to 
EUCPN’s functioning and visibility; its action at local level as well as tailoring its products to 
the target groups. One of the key findings of this evaluation was that a substantial amount of 
ISEC  crime  prevention  projects  supported  by  the  ISEC  Programme  were  implemented 
without  associating  or  even  informing  EUCPN.  Indeed,  when  reviewing  the  project 
documentation  of  the  sample  of  projects,  only  a  limited  number  of  ISEC  projects  mention 
dissemination  activities  in  relation  to  the  EUCPN.  This  has  been  confirmed  in  an  interview 
with  the  EUCPN  whereby  EUCPN  reported  not  having  received  results  of  ISEC-funded 
projects.  As  already  presented  above,  it  is  recommended  that  good  practices  and  results  of 
ISEC  projects  are  shared  with  EUCPN  which  can  potentially  act  as  a  ‘repository’  of  good 
practices provided there are workable dissemination channels in place. 

92   Set up by the Council Decision 2001/427/JHA, repealed by the Council Decision 2009/902/JHA. See 

http://eucpn.org/  

93   http://ec.europa.eu/dgs/home-affairs/e-library/documents/policies/organized-crime-and-human-

trafficking/crime-prevention/docs/20121130_eucpn_report_en.pdf . 

31 

 
 
 
                                                            
21 projects analysed as part of the evaluation described the development or sharing of crime 
prevention best practices, of which more than half tackled the areas of police cooperation, law 
enforcement  and  economic/  financial  crime.  Both 
the  projects’  final  reports  and 
Commission’s  final  evaluation  reports  showed  evidence  of  positive  outcomes  related  to  the 
sharing  of  best  practices  in  crime  prevention  at  EU  level,  particularly  between  law 
enforcement  agencies  and  other  practitioners.  For  example,  one  project  implemented  in 
Finland in  2010 contributed to  developing best practices in  fighting international  corruption 
and fostered cooperation among participating countries (AT and HR) and Agencies (Eurojust) 
as  well  as  between  law  enforcement  and  judicial  authorities.  The  project  developed  best 
practices through an investigation of several serious economic and financial crimes. Similarly, 
the 2011 project “Enhancing the administrative capacity of police  officers for prevention  of 
sexual crimes against children” contributed to the transfer of best practices to Bulgarian police 
officers, providing them with relevant tools for the prevention and detection of sexual crimes 
against children. 

In addition, a total of 6 ISEC projects out of the sample of 151 projects analysed referred to 
the enhancement of forensic capabilities at national level. Three Member State representatives 
commented  positively  on  the  results  of  ISEC  projects  in  the  field  of  forensics,  generally 
obtained  through  the  development  of  new  tools  or  methodologies  and  sharing  of  best 
practices.  For  example,  Greece  reported  that  the  Forensic  Science  Division  of  the  Hellenic 
Police (which was directly funded by ISEC) was able to obtain equipment that significantly 
upgraded  its  technical  capacities  both  in  scope  and  in  accuracy,  and  that  most  of  the 
equipment  of  the  DNA  Database  Section  was  funded  with  the  support  of  the  ISEC 
Programme. 

Furthermore,  the  European  Network  of  Forensic  Science  Institutes  (ENFSI)  network 
(monopoly  body  funded  by  ISEC)  has  contributed  through  the  sharing  of  data  and  good 
practices in this field. The network brought together more than 40 organisations competent in 
forensics  across  the  EU.  Best  practices  manuals  across  many  forensic  disciplines  were 
prepared (e.g. fingerprinting, DNA, handwriting, etc.). According to the ENFSI management 
interviewed,  the  ISEC  funding  has  been  instrumental  in  achieving  ENFSI’s  strategic 
objectives. 

Interviewed Member State representatives also commented positively on the development of 
best  practices  in  their  respective  countries  as  part  of  cooperation  initiatives  between 
transnational law enforcement agencies. 

However,  a  significant  area  for  improvement  identified  by  the  evaluation  and  by  most 
stakeholders  is  that  the  results  and  deliverables  of  ISEC  individual  projects  have  not  been 
compiled in a central repository (e.g. by the Commission or a relevant EU agency/network) to 
enhance  knowledge  sharing  and  transfers  of  practices  (as  mentioned  under  section  5).  For 
example,  methodologies  developed  to  investigate  certain  types  of  crimes,  reports,  analysis, 
best practices, training materials, etc. could have been compiled centrally and, consequently, 
the effectiveness of the programme could have been boosted if their potential multiplier effect 
and continuous usage for interested stakeholders in other Member States had been maximised. 

EQ5. To what  extent  did  the ISEC programme contribute  to promoting and development of 
coordination, cooperation and mutual understanding among law enforcement agencies, other 

32 

 
 
 
national  authorities  and  related  Union  bodies  in  respect  of  the  priorities  identified  by  the 
Council, in particular as set out by the Europol's Organised Crime Threat Assessment? 

Concerning  the  extent  to  which  the  ISEC  Programme  contributed  to  the  achievement  of 
specific objective b), the evidence collected during the evaluation shows that activities such as 
transnational  cooperation,  coordination  and  networking  (including  mutual  learning  and 
exchanges  of  good  practices)  between  law  enforcement  agencies,  national  authorities  and 
other  relevant  bodies  (e.g.  civil  society  organisations,  NGOs,  universities  and  research 
institutes)  were  implemented  by  a  large  number  of  ISEC  projects.  Stakeholders  reported 
strong evidence of outcomes and results related to transnational cooperation, coordination and 
mutual understanding.  

More than 50% (93 in total) of the 151 ISEC funded projects included in the sample analysed 
had the "promotion and development of coordination, cooperation and mutual understanding" 
as a project objective94, with 16 focussing solely on this objective (the large majority was also 
focusing  on  stimulating,  promoting  and  developing  horizontal  methods  for  preventing  and 
fighting crime). These 93 projects were implemented in 25 Member States, with the majority 
of the coordinating organisations located in Germany, Italy, the Netherlands and the UK and 
the coordinators consisting mostly of law enforcement authorities, national Ministries as well 
as universities and research institutes. Projects’ final reports show that more than half of the 
sampled  projects  (69  in  total)  of  projects  targeting  the  development  of  coordination, 
cooperation and mutual understanding were finalised according to plan. 

Overall,  the  analysis  of  final  reports  of  the  projects,  the  online  survey  as  well  as  feedback 
gathered  through  interviews  with  relevant  national  authorities  suggest  that  projects  made  a 
significant  contribution  to  the  promotion  and  development  of  coordination,  cooperation  and 
mutual understanding among law enforcement agencies, other national authorities and related 
Union  bodies.  The  most  commonly  reported  outcomes  from  the  sample  of  ISEC  projects, 
were:  improved  understanding  of  methods,  tools  and  practices  (reported  by  50%  of  the 
projects);  increased  levels  of  knowledge  and  skills  of  practitioners  (reported  by  74%  of  the 
projects); enhanced networking and forms of sharing information and best practices (reported 
by  75%  of  the  projects).  Furthermore,  stakeholders  also  reported  on  the  positive  outcomes 
related  to  the  operational  cooperation  between  transnational  law  enforcement  agencies  (e.g. 
police,  border  guards),  providing  examples  of  successful  joint  operations  leading,  for 
example, to the seizure of large quantities of drugs and to the arrest of wanted persons. 

Activities implemented through ISEC funding contributed to the development of more stable 
transnational  cooperation  between  Member  States  and  their  agencies,  as  well  as  better 
exchange  of  information,  particularly  in  the  field  of  forensics,  drugs,  PNR,  cybercrime  and 
trafficking in human beings. National authorities interviewed also reported that ISEC projects 
also  fostered  cooperation  between  agencies  in  the  same  Member  State, increasing  synergies 
between the Law Enforcement Agencies.95  

94   59%  of  them  funded  through  action  grants,  32%  through  Framework  Partnerships  and  9%  through 

monopoly organisations. 

95   For example Greece reported that ISEC supported greater cooperation between national actors such as the 
Criminal Investigation Agencies, Forensic Labs, Judicial Services, Coroners and Health Sector Agencies; in 
Romania  there  was  increased  cooperation  between  the  Department  for  Intelligence  and  Internal  Affairs 
within the Ministry of Internal Affairs and the Prosecutor’s Office with the High Court for Cassation and 
Justice within the Public Ministry. 

33 

 
 
                                                            
These outcomes are in line with the priorities identified in the 2013 Europol Organised Crime 
Threat  Assessment  (OCTA).96  However,  other  ISEC  projects  focussing  on  cooperation  and 
coordination  activities  covered  some  of  the  OCTA’s  priorities  to  a  lesser  extent  (e.g.  fraud, 
money laundering and counterfeit goods). 

The  2011  Europol  OCTA  identifies  “strong  and  effective  partnerships  with  the  private 
sector”97  as  a  priority  area,  particularly  for  the  purpose  of  crime  prevention  and  awareness 
raising. Evidence collected through the evaluation shows that cooperation between the public 
and private sector in Member States was supported by ISEC projects to a limited extent. 

However, in several instances, stakeholders interviewed did report evidence of public-private 
partnership or cooperation facilitated by ISEC activities:  

 

 

 

in Romania, air carriers established cooperation protocols for the exchange of 
information with law enforcement agencies in the context of a project on PNR in order 
to prevent possible threats related to terrorism and organised crime; 
in Greece, the procurement process for the equipment of forensic material provided an 
opportunity for members of the Forensic Science Division to meet with suppliers, who 
supported them with expertise and information on the operational needs of the 
Division and explored the opportunity for the potential development of tailor-made 
products which will support the Police’s efforts in the forensic field and; 
the Italian Medicines Agency and carried out by an international consortium of 
partners including both private (e.g. Alliance for Safe Online Pharmacies – ASOP) and 
public partners (e.g. Italian police forces)98. 

EQ6. To what  extent  did  the ISEC programme contribute  to promoting and development of 
best practices for the protection of and support to witnesses?; and 

EQ7. To what  extent  did  the ISEC programme contribute  to promoting and development of 
best practices for the protection of crime victims? 

From the sample of 151 projects analysed, a relatively low number of ISEC projects focused 
on  providing  protection  and  support  to  witnesses  or  victims  of  crime.  Seven  of  the  projects 
analysed  reported  impacts  associated  with  increased  support  and  protection  to  victims  and 
witnesses  of  crime.  All  seven  projects  entailed  awareness-raising  activities  as  well  as 
analytical research activities to a lesser extent. Among these projects, two were carried out in 
the  field  of  police  cooperation.  The  other  areas  covered  include:  law  enforcement,  child 
sexual abuse, human trafficking, economic and financial crime, and radicalisation. 

Reduction in the number of crime victims was only cited as an impact under three projects. It 
should  be  noted  that  this  is  influenced  by  many  exogenous  factors  which  are  outside  the 
control of the projects funded. The areas covered by the projects include child sexual abuse, 
cybercrime  and  radicalisation.  A  caveat  should  be  highlighted  regarding  the  2007  de-

96   The priority areas identified in the report were: facilitation of illegal immigration, trafficking in human 
beings, counterfeit goods with an impact on public health and safety, Missing Trader Intra Community 
(MTIC) fraud, synthetic drugs production and polydrug trafficking in the EU, cybercrime and money 
laundering. 

97   Europol - EU Organised Crime Threat Assessment 2011 
98   This project was called FAKESHARE II and is included in the Italian case study on financial crime.  

34 

 
 
 
                                                            
radicalisation project, given the reported shortcomings in dissemination and awareness-raising 
and the strong focus of the project on crime prevention at the local level. 

The low number of projects dedicated to these areas is further confirmed by a perceived low 
impact.  When  asked  about  the  perceived  contribution  of  ISEC  projects  to  best  practices  for 
the  protection  of  witnesses  and  crime  witnesses,  less  than  40%  across  the  three  stakeholder 
groups  of  respondents  (project  participants,  co-beneficiaries  and  coordinators)  replied  that 
their project made some contribution to these areas, while almost 50% responded that this was 
“not applicable”. 

7.3 Efficiency 

EQ8a. To what extent were the results of the ISEC programme achieved at a reasonable cost 
in terms of financial and human resources deployed? 

As  mentioned  in  section  6,  due  to  the  lack  of  an  initial  baseline  and  to  the  difficulties 
encountered  in  comparing  project  costs,  the  assessment  made  of  efficiency  below  relies 
largely on the qualitative data collected through the stakeholder consultations. 

Concerning  the  cost-efficiency  of  the  financial  resources  deployed,  as  mentioned 
previously,  a  budget  of  EUR  522  million  was  allocated  to  the  ISEC  Programme  of  which 
EUR 413 million was committed and EUR 304 million spent (including funding provided via 
action grants and funding provided via public procurement). There was an upward trend in the 
absorption of funds spent on action grants with an overall absorption rate of 74% across the 
period  2007-2013,  and  a  peak  of  79%  in  the  last  year.  The  EU  funding  provided  was 
perceived  by  the  vast  majority  of  consulted  stakeholders  as  having  been  sufficient  and  as 
having been highly cost-efficient for all activities implemented under ISEC. Particularly, the 
creation of networks was viewed as having a high value for money.  

At project level, a more in-depth analysis was carried out for a total of 146 projects using data 
taken  from  financial  cost  statements99.  Overall,  92%  of  the  estimated  budget  of  all sampled 
grant agreements together appears to have been spent. The underspend was primarily a result 
of:  

  Costs  for  travel  being  far  lower  than  estimated,  which  was  partially  due  to  standard 

day rates for subsistence and travel being higher than actual costs incurred; 

  Actual output (the difference between planned and actual events and participants) was 
lower than expected, although this could not be ascertained due to limited details on 
the exact targets set at the beginning and those achieved; 

  There  were  variations  between  the  estimated  and  spent  budget  due  to  difficulties  in 
forecasting  such  costs.  Some  organisations  underestimated  these  costs  and  needed 
more complex IT tools than initially envisaged or such costs were simply higher than 

99   The financial cost statement provides budgetary information about two stages of the project cycle, one at the 
start at the stage of the grant agreement on the basis of forecasted expenses, and one at the end consisting of 
the final budget of actual expenses. 

35 

 
 
 
 
                                                            
foreseen.  On  the  other  hand,  some  organisations  were  able  to  make  savings  on  IT 
costs, for example, by opting for available open source software. 

The  data  suggests  that  the  absorption  of  funding  improved  towards  the  end  of  the 
Programming period, with total final expenditures slightly closer to 100% in 2010-2013 than 
for 2007-2009 (although 2008 is an exception). 

Reported  issues  that  may  have  hampered  cost-efficiency  of  these  projects  were:  "Drop-outs 
and change in staff of the project" (mentioned for 29 out of 146 projects); "Problems related 
to  Commission  management"  (mentioned  for  22  out  of  146  projects);  "low  level  of 
participation  from  participants"  (mentioned  for  16  out  of  146  projects);  and  "lack  of 
cooperation  from  partners"  (mentioned  for  8  out  of  146  projects).  For  ten  projects,  issues 
highlighted  include  the  budget  reallocation,  lack  of  financial  resources  to  carry  out  some 
elements  of  the  project,  and  difficulties  in  having  to  pre-finance  the  project,  only  claiming 
reimbursement later. 

These  findings  are  supported  by  the  responses  of  coordinating  organisations  to  a  survey, 
where  the  majority  (58  respondents  representing  64.4%)  indicated  that  project  budget  was 
spent more or less as planned, while a third (32 respondents, representing 35.6%) highlighted 
that less was spent than planned. Reasons provided for this were the lower participation than 
expected in certain projects (9 respondents); differences between planned and actual spending 
including  lower  than  expected  travel  costs  (14  respondents);  having  carried  out  a  technical 
solution in-house rather than through an external contractor and thus making cost savings (2 
respondents);  shortening  of  the  length  of  training  events  as  their  foreseen  length  was 
considered  impractical;  fluctuations  in  prices  leading  to  costs  differing  from  the  forecast  (3 
respondents) but also lacking technical capacity (in IT staff) to contribute to a project activity; 
purchase orders exceeding those in planned budget; inability to carry out a part of the project 
due  to  legal  constraints  and  internal  procedures  deemed  complex;  choices  made  during  the 
project to carry out different activities from those planned; staff leaving before the end of the 
project; and internal difficulties of the organisation leading to lower absorption. 

Overall,  the  majority  of  coordinating  organisations  surveyed  found  EU  funding  to  be 
sufficient  for  the  activities  undertaken  (87.6%).  Among  the  coordinating  organisations  for 
whom  funding  was  not  sufficient,  issues  highlighted  were  the  difficulty  in  securing  co-
financing from other sources and in ensuring the continuity of certain outputs (i.e. websites, 
IT  tools);  higher  costs  for  specific  activities  than  expected;  difficulties  in  keeping  overhead 
costs  under the  ceiling;  and  having  some  of  the  funding  reclaimed  after  an  audit  due  to  the 
implementation  of  activities  not  foreseen  or  declared  costs  found  to  be  ineligible  for  EU 
funding. 
The external evaluation of the networks Railpol100, Tispol101 and Aquapol102 developed under 
projects for which grants were awarded to bodies in a monopoly situation and which consisted 

100   Railpol is an international network of the organisations responsible for policing the railways in EU Member 
States.  The  aim  is to  enhance  and  intensify  international  railway  police cooperation  in Europe,  to prevent 
threats and guarantee the effectiveness of measures against cross-border crime. 

101   Tispol  aims  at  cooperation  of  traffic  police  forces  of  Europe  in  order  to  improve  road  safety  and  law 
enforcement  on  the  roads  of  Europe.  It  acts  as  a  platform  for  learning  and  exchange  of  good  practices 
between the traffic police forces of Europe. 

102   Aquapol acts as a platform for learning and exchange of good practice for law enforcement in the domain of 
waterborne  transport  in  Europe.  Its  main  activities  are  exchange  of  intelligence,  exchange  of  operational 
information and experience and cross border cooperation in day-to-day law-enforcement work. 

36 

 
 
                                                            
of  rather  large  projects  (about  EUR  500  000  per  year),  found  no  indications  of  significant 
inefficiencies. Although the evaluation found that these projects/networks might benefit from 
a  better  demonstration  of  efficiency  and  value  for  money,  it  ultimately  concluded  that  “the 
networks are providing sufficient outputs to warrant their ongoing investment”.103 

Overall  resources  have  generally  been  considered  by  respondents  to  have  been  used  in  an 
efficient  way,  and  although  discrepancies  in  outputs  of  projects  were  found,  there  are  no 
indications  of  significant  differences  in  outcomes  pointing  directly  to  inefficient  use  of 
funding.  As  mentioned  in  section  6,  the  vastly  different  nature  of  each  project  makes  their 
direct  comparison  challenging  and  potentially  misleading.  Another  factor  is  that  a  lack  of 
detailed and harmonised reporting on outputs and results in final reports does not allow for a 
fully-fledged comparison. 

Concerning  the  cost-efficiency  of  the  human  resources  deployed,  one  of  the  key  issues 
highlighted by various stakeholders was the high staff turnover both in the Commission and in 
the other organisations coordinating or participating in or projects. In the Commission, across 
the period 2007-2013, there were around 24 different desk officers in total (OIA) responsible 
for following-up on ISEC projects and public procurement items at various times, as well as 
four programme  managers. Some  OIAs were  responsible for dozens of projects at the same 
time, which would make it difficult to have a detailed understanding of the situation in each 
project. This was further exacerbated by the fact that until 2011, projects were not required to 
submit an interim report. Some stakeholders alluded to this issue in interviews and the survey, 
pointing  out  that  frequent  staff  changes  made  communications  more  time-consuming,  with 
valuable knowledge getting lost in the process. 

Furthermore,  given  the  relative  newness  of  the  Programme  and  the  lack  of  resources  of  the 
Commission at the time to closely follow all ISEC projects from start to end, the evaluation 
showed  that  a  dedicated  technical  assistance  budget  could  have  been  useful.  Such  a  budget 
could have better supported the development of a state of the art monitoring system to collect 
and  analyse  data  on  financial  progress,  outputs  and  results  of  projects,  as  well  as  provided 
technical  expertise  throughout  the  lifecycle  of  projects  and  increased  the  dissemination  of 
results. 

Another issue raised in scoping interviews was the fact that the introduction of targeted calls 
in 2010 reportedly increased the workload for preparation, but also helped maximise the use 
of available budget and enhanced the quality of proposals.  

In  addition,  a  mixed  picture  emerges  regarding  perceptions  of  the  administrative  burden 
accompanying project implementation. Based on the evidence gathered, it can be concluded 
that  the  ISEC  Programme  had  a  similar  level  of  administrative  burden  to  other  EU 
programmes. The majority of coordinating organisations reported a positive experience with 
regard  to  the  application  and  award  procedure  and  20%  of  respondents  reported  that  no 
administrative  burden  was  perceived.  However,  the  vast  majority  did  perceive  some 
administrative  burden104.  The  survey  with  partner  organisations  confirms  similar  perceived 

103   Evaluation of Aquapol, Railpol and Tispol Final Report, 15 January, 2013. 
104   33% reported experiencing some administrative burden and 45% reported experiencing a significant or very 

significant level of administrative burden. 

37 

 
 
                                                            
levels  of  administrative  burden  in  the  implementation  of  ISEC  projects  compared  to  other 
non-EU funded projects.105 

Several  key  performance  indicators  related  to  the  Commission’s  management  of  the 
implementation  of  the  programme  were  stipulated  in  the  Financial  Regulation  and  are 
reported below106: 

1. Concerning the time to award, (the time taken from the call for proposals until the award 
decision –), although the vast majority of project coordinators surveyed and interviewed did 
not  perceive  major  problems  in  the  application  procedure,  a  number  of  coordinating 
organisations considered the time taken to announce the award decisions a challenge, with 14 
respondents  (representing  17%)  as  well  as  a  few  interviewees  finding  the  time  taken  too 
lengthy. In 2009, 2012 and 2013 the average time taken remained under the maximum limit of 
six months, however in 2007, 2008, 2010 and 2011 there were a few cases in which the time 
taken  went  over  this  limit.  Despite  these  exceptions,  the  average  time  taken  for  an  award 
decision decreased over the period with an average of 260 days in 2007 down to 162 in 2012 
and 2013 (see figure 3 below). 

Figure 3: Time taken from the call for proposals until the award decision 

2. A similar and more marked trend can be observed regarding the time to contract (the time 
taken  from  the  award  decision  until  the  grant  agreement  ),  with  the  average  time  per  year 
going down over the period, from an average of 203 days in 2007 to 73 in 2013 (see figure 4 
below).  The  average  time  across  the  whole  programming  period  was  119  days.  In  addition, 

105   37%  of  partner  organisations  reported  experiencing  significant  administrative  burden  and  33%  reported 
experiencing  some  administrative  burden, whilst 3%  reported  experiencing  very  significant  administrative 
burden.  Only  one  respondent  reported  experiencing  no  administrative  burden,  whilst  five  respondents  did 
not know. 

106   The  deadlines  are  provided  as  stipulated  in  the  Financial  Regulation  (Regulation  (EU,  EURATOM)  No 
966/2012  of  the  European  Parliament  and  of  the  Council  of  25  October  2012  on  the  financial  rules 
applicable  to  the  general  budget  of  the  Union  and  repealing  Council  Regulation  (EC,  Euratom)  No 
1605/2002). 

38 

 
 
 
 
 
                                                            
from  89  surveyed  coordinating  organisations  only  9  reported  issues  in  signing  the  grant 
agreement, while the remaining 80 viewed the grant agreement process as running smoothly. 

Figure 4: Time taken for the signature of the grant agreement 

3. Lastly, the evaluation found diverging trends for the time to make payments (the time taken 
to make the pre-financing payments) and the time taken to make the final payment. Most pre-
financing  payments  were  paid  on  time  throughout  the  programming  period,  with  95%  of 
payments on time in 2007, and 100% on time for the remaining years except 2012, when 78% 
of payments were made on time. In contrast to this highly positive trend, although on average 
the time taken to make final payments also improved to some extent across the programming 
period,  during  the  final  payment  stage  the  sample  of  projects  analysed  shows  an  on-time 
payment  ratio  between  45%  and  60%  for  all  years  except  2007  (23%)  and  2011  (65%).  A 
slower performance for the final  payment is  probably due  to  the emphasis on verifying  and 
checking all project deliverables and budget headings prior to authorising payment.  

EQ8b.  What  kinds  of  initiatives  or  approaches  were  adopted  to  simplify  access  to  and 
implementation of the actions funded by the programmes? 

Several  changes  were  introduced  during  the  ISEC  programming  period  as  a  response  to 
weaknesses  identified  in  the  first  years  of  implementation.  The  most  salient  changes  were 
those made to the application award process and the introduction of targeted calls from 2010 
onwards.  Another  significant  change  is  a  shift  in  favour  of  shared  management  under  the 
successor  fund,  ISF-Police.  These  changes  reflect  the  ability  and  flexibility  of  the 
Commission  to  respond  to  weaknesses  with  adjustments  to  the  design,  access  to  and/or 
implementation of the Programme.  

Concerning  the  changes  introduced  in  the  application  award  process,  throughout  the 
programming period, in addition to the required completion of budget estimation forms, staff-
cost  analysis  and  a  partnership  declaration,  applicants  were  asked  to  provide  the  following 
information in the application form for project grants: 

39 

 
 
 
 
 
 
  general  project  information  (including  types  of  activities,  partners  involved  and 

deliverables); 

  content  of 

the  project  (including  problem  description,  objectives,  relevance, 

innovation and EU added value); 

 

 

implementation  of  the  project  (approach  and  methodology,  description  of  activities, 
risks and opportunities); 

financial  management; 
dissemination of information. 

results 

(including  sustainability)  and  evaluation  and 

However, from 2010, applicants were also requested to provide the information according to 
work streams, indicators to measure results, evidence of previous programme experience and 
to  provide  more  details  on  project  partners.  While  adding  a  level  of  complexity,  these 
requirements increased the potential for higher quality projects, motivating the applicants to 
develop  a  more  rigorous  plan  and  cost  estimation  of  each  of  their  activities  as  well  as  to 
introduce  a  progress  monitoring  dimension.  However,  a  few  grant  beneficiaries  found  it 
demanding  to  report  project  expenses  in  detail,  using  templates  that  sometimes  changed 
during the project cycle. Some monopoly organisations found the budget rules to be very rigid 
and would have welcomed a greater degree of flexibility to better manage changes between 
budget lines. 

A  small  number  of  interviews  with  coordinating  organisations  highlighted  some  problems 
with  the  management  of  the  application  process.  Some  of  the  issues  encountered  included 
finding the application process time consuming and the online application website not being 
user-friendly. Suggestions for improvement made for the Commission were to put in place a 
dedicated  application  support  team  able  to  respond  quickly  and  effectively  to  queries. 
Introducing a section to deal with the general queries on the application procedure would have 
advantages and would make it possible to cover queries not covered by the existing guidance. 
Other suggestions made by coordinating and partner organisations to improve the flexibility 
of and access to the application procedure included: allowing the application to be submitted 
in  a  different  language  other  than  English;  allowing  projects  to  make  changes  to  the 
application/information  supplied  during  application  after  the  point  of  submission;  and 
reducing the amount of information required as part of the application process. 

With  a  view  to  increasing  access  to  the  Programme,  information  days  were  organised  in 
Brussels in 2010 and 2011 to explain the application procedures to potential applicants, with 
the aim of attracting a higher number of applications. Regional information days were held in 
seven Member States107 in 2010, 2011 and 2012. It is not possible to establish the impact of 
these information days based on the data available as the trends found are inconclusive. The 
trend in number of applications shows a decrease in the total number of applications per AWP 
year  from  2010  to  2011  (respectively  247  and  147),  however,  a  marked  increase  can  be 
observed from 2011 to 2012 in the total number of applications (respectively 147 and 315)108. 

The Commission also sought to improve the guidance documents for the calls for proposals. 
Although  the application forms became  more extensive during the programming  period, the 
guides also provided more detailed support. The increased length partly reflected the increase 

107   AT, IT, RO, HU, SE, SI, and the UK. 
108   It should be noted that in 2011 there was no call for framework partnership proposals. 

40 

 
 
                                                            
in  the  level  of  detail  required  in  the  application  form  and  the  introduction  (and  thus 
explanation) on the PRIAMOS system109. The majority of coordinating organisations reported 
a  positive  experience  with  regard  to  the  application  and  selection  procedure:  74%  of  online 
survey  respondents  indicated  that  they  had  encountered  no  specific  problems  with  the 
application  procedure  and  only  6%  of  respondents  reported  issues  with  understanding  the 
application documents. 

The  evaluation  found  that  the  share  of  ineligible  project  applications  amongst  all  rejected 
applications  went  down  over  time,  as  a  possible  consequence  of  the  changes  introduced. 
However,  other  exogenous  factors  including  improved  know-how  of  the  application  system 
and  rules  as  time  progressed,  and  the  application  experience  of  repeat  applicants,  may  also 
have been a reason for the decrease in ineligible projects. 
Between 2007 and 2013 there were 54 call for proposals110, with 4 in 2007, 6 in 2008, 5 in 
2009,  12  in  2010,  10  in  2011,  9  in  2012  and  8  in  2013.  The  possible  outcomes  of  project 
applications  were:  successful  applications;  unsuccessful  applications;  ineligible  applications 
and  withdrawn  projects.  The  overall  success  rate  over  2007-2013  was  42%  (from  1  700 
applications  made  over  2007-2013,  818  were  awarded,  including  via  reserve  lists)111.  The 
share of ineligible projects was much lower over the period 2011-2013 (ranging from 7% in 
2013  to  15%  in  2012)  than  2007-2010  (ranging  from  24%  in  2007  to  31%  in  2009).  This 
might  have  been  due  to  two  reasons:  1)  an  indication  that  eligibility  criteria  were  better 
followed  and/or  understood  in  later  years  of  ISEC  than  in  early  years  and  2)  thanks  to  the 
revision  of  the  Financial  Regulation,  applicants  could  be  contacted  to  request  further 
information and avoid ineligibility on purely formal grounds. 

Given the ‘newness’ of the Programme in the early years and the low capability or know-how 
of relevant stakeholders to write high-quality applications (in response to the Commission’s 
focus on quality rather than volume), a lower level of funding could have been considered at 
the start of the Programme, or the use of a technical assistance budget as recommended above 
(see section 9.2). To compensate for the lower fund absorption of some calls, ISEC provided 
for  the  possibility  of  transferring  remaining  funding  between  calls  (see  below).  It  is 
recommended  that,  for  future  programmes  in  the  field  of  crime  prevention  implemented 
through direct management, this flexibility is maintained.  

A  difficulty  encountered  with  general  calls  was  that  project  applications  covering  a  very 
diverse range of thematic policy areas needed to be compared in terms of quality as part of the 
selection  process,  a  significant  challenge  considering  the  very  different  target  groups  and 
aims.  Another  issue  was  the  concern  that  the  original  design  of  general  calls  did  not 
sufficiently meet the aim of supporting EU policy developments in specific areas. To address 
these  issues,  from  2010  onwards  targeted  calls  were  introduced  which  reflected  the  EU 
policy  priorities  more  closely  and  which  drove  the  budget  allocations  available  in  each 
thematic  area.  During  one  of  the  scoping  interviews  carried  out  for  the  evaluation,  it  was 
highlighted  that  several  positive  effects  of  this  introduction  could  be  observed,  such  as  an 

109   Which was introduced for applications for restricted calls in 2009 and for open calls in 2011. 
110   This figure includes all types except monopolies. 
111   Between 2007 and 2013 there were at least 520 non-successful projects, while another 339 applications were 
ineligible. At least 43 applications were rejected because the applicant did not have sufficient financial or 
operational resources for carrying out the project. 

41 

 
 
                                                            
increase in the level of efficiency and transparency112, an increase in the use of the available 
budget thanks to the fact that the non-awarded budget from the targeted calls was now used 
for  the  general  call,  maximising  available  budget;  and  a  diversification  of  the  types  of 
organisations receiving grants, with an analysis of the sample of 151 projects showing that the 
proportion  of  NGOs  and  civil  society  organisations  amongst  coordinating  organisations 
increased with the introduction of targeted calls.113 

In addition, a shift towards shared management as management mode was introduced for 
the  successor  fund,  ISF-Police.  Unlike  ISEC,  which  was  implemented  100%  under  direct 
management,  ISF-Police  is  being  implemented  on  the  basis  of  a  combination,  with  around 
60%  of  the  available  funding  being  allocated  to  national  programmes  under  shared 
management  and  40%  allocated  to  Union  actions  under  direct  management.  This  change  is 
expected to have an impact on the efficiency and effectiveness of the management of the ISF 
Programme. However, it has not been possible, within the scope of this ex-post evaluation, to 
identify and  determine whether the shift  to  shared management has  provided efficiency and 
effectiveness gains in the current ISF Programme. 

7.4 Coherence 

EQ9a.  To  what  extent  was  the  ISEC  programme  coherent  with  actions  related  to  the 
prevention of and fight against crime supported by other EU financial instruments? 

In  order  to  ensure  coherence  and  avoid  duplication  between  ISEC  and  other  EU 
programmes/interventions,  there  was  an  ongoing  consultation  process  from  the  planning 
phase  to  implementation  phase  of  projects,  which  included  the  following  actions:  an  inter-
service  consultation  (ISC)  carried  out  during  the  drafting  of  the  AWPs  where  the  different 
DGs had the possibility to provide comments, in the case of ISEC it included DGs such as DG 
TAXUD,  DG  JUST  and  DG  ECHO;  several  working  groups  at  Commission  level  covering 
ISEC, such as the Asset Recovery Office Platform (AROPLATFORM), where the needs and 
priorities of ISEC were discussed; the assessment of projects proposals on an ad-hoc basis to 
avoid  double-financing  of  a  project;114  the  involvement  of  other  DGs  in  DG  HOME's 
evaluation committee, to ensure synergies between policy fields. 

In  order  to  analyse  the  coherence  of  ISEC  with  other  EU  programmes,  the  following 
programmes  were  selected  for  examination:  Criminal  Justice  Support  Programme  (JPEN), 
Daphne  III  (Fight  against  violence),  7th  Framework  Programme  for  Research  and 
Technological  Development  (FP7)  and  Anti-fraud  programme  (Hercule  II).  An  analysis  of 
their coherence with ISEC is provided below. 

Coherence with JPEN 

112   In spite of the human resources needed to prepare targeted calls efficiency gains included better handling of 

financial verifications and the perceived level of quality of the content of the proposals selected. 

113   Under  the  general  calls  in  the  period  before  2010,  smaller  organisations  represented  only  18%  of  all 
coordinating organisations implementing grants, while in the period 2010-2013 they represented 45% of all 
coordinating organisations implementing grants under targeted calls. 
114   The project proposals had to list other EU funding in the application. 

42 

 
 
 
 
                                                            
JPEN  is  designed  to  contribute  to  mutual  recognition  in  criminal  matters  and  supports 
initiatives  for  the  exchange  of  best  practices  between  legal,  judicial  and  administrative 
authorities  and  the  legal  professions.  It  is  coherent  in  scope  to  ISEC  in  so  far  as  JPEN 
provides  exchange  of  information  and  best  practices  in  the  area  of  criminal  justice  and 
protection  of  victims.  There  is  thus  ample  scope  for  JPEN  and  ISEC  to  mutually  reinforce 
respective actions and results. A large number of actions selected for grants or tenders from 
2007  to  2013  under  JPEN  fell  into  training  activities  mainly  of  the  judiciary  or  of  legal 
practitioners, and mutual learning and exchanges between Member State authorities through 
for example seminars/symposia, showing scope for coherence with ISEC activities relating to 
‘training,  exchange  of  staff  and  experts’  and  ‘actions  on  operational  cooperation  and 
coordination’.  Other  JPEN  activities  included  development  of  e-Justice  at  EU/national  level 
and  support  and  advice  services,  through  for  example  websites,  which  had  scope  for 
coherence  with  ISEC  activities  regarding  the  ‘development  and  transfer  of  technology  and 
methodology’.  In  addition,  JPEN  activities  concerning  good  practice  dissemination  with  the 
purpose  of  supporting  mutual  learning  had  scope  for  coherence  or  overlaps  with  ISEC’s 
‘awareness  and  dissemination  activities’.  Regarding  potential  overlaps  at  project-level,  no 
particular  overlap  was  identified  as  JPEN  projects  focused  predominantly  on  judicial 
cooperation and tools to further such cooperation, which is outside the scope of ISEC. 

Coherence with Daphne III 

Another  programme  identified  with  potential  for  synergies  and/or  overlaps  with  ISEC  was 
Daphne III. Created as part of the European Framework Programme "Fundamental Rights and 
Justice",  Daphne  III  was  designed  to  provide  preventive  measures  against  all  forms  of 
violence  and  support  to  victims  and  groups  at  risk  (children,  young  people  and  women). 
Daphne III-funded actions were  mainly  led by national NGOs (including national platforms 
and  networks),  universities  and  European  networks,  platforms  and  forums.  In  line  with  the 
focus of the Programme on prevention, the projects mostly implemented were those related to 
‘awareness-raising,  information  and  dissemination’  activities,  ‘analytical  activities’  and 
‘mutual 
learning,  exchanges  of  good  practices  and  cooperation’,  with  scope  for 
coherence/overlap with similar ISEC activities. 

Analysis of ISEC’s last two specific objectives on ‘exchange of best practices for protection 
of victims and witnesses’ shows potential for complementarity with Daphne III’s objectives 
on ‘contribution to the protection of children, young people and women against all forms of 
violence’ and ‘providing support measures and protection of victims and groups at risk’. The 
2007  and  2008  ISEC  AWPs  included  the  protection  of  witnesses  and  victims  as  a  separate 
priority  area  under  the  open  calls  for  proposals  for  grants.  As  commented  by  Commission 
officials,  the  focus  on  victims  and  witnesses  was  later  reduced  in  ISEC’s  AWP  and 
subsequent calls for proposals encouraged the increase in synergies with Daphne III managed 
by the Directorate-General for Justice and Consumers (DG JUSTICE).115. In the 2009 to 2013 
Annual  Work  Programmes  the  protection  of  victims  in  specific  crime  areas  was  included 

115   Although the focus on protection of victims was narrowed to specific conditions (to terrorism and crimes), 
ISEC’s AWPs maintained the possibility for supporting projects on the protection of victims, as the basic act 
was not amended. 

43 

 
 
 
                                                            
under  the  priority  areas  of  the  open  targeted  and  general  calls  for  proposal  that  dealt  with 
victims  of  human  trafficking,  victims  of  terrorism,  victims  of  radicalisation  and  organised 
crime  and  child  victims  of  sexual  exploitation.  Complementarity  between  the  two 
programmes was achieved by the two Directorate-Generals (DG HOME and DG JUSTICE) 
working together on stimulating synergies. These were achieved between ISEC and Daphne 
III in particular in the area of trafficking in human beings, which showed the largest potential 
for complementarity/overlaps. 

Coherence with FP7 

In addition, potential for coherence was identified between the 7th Framework Programme for 
Research and Technological Development (FP7), which aimed to strengthen the scientific and 
technological  base  of  European  industry  and  to  encourage  its  international  competitiveness, 
while  promoting  research  that  supports  EU  policies.  One  of  the  themes  of  FP7  relates  to 
security and includes activities such as ensuring security of citizens, infrastructure and utilities 
and borders through technological solutions for civil protection, systems and services. These 
activities  are  complementary  to  the  ISEC  activities  on  ‘development  and  transfer  of 
technology  and  methodology’  and  ‘analytical,  monitoring,  evaluation  and  audit  activities’. 
Furthermore,  cross-cutting 
integration, 
interconnectivity and interoperability’, focus on technologies seeking to enhance cooperation 
among law enforcement and thus have similarities with the law enforcement theme of ISEC. 

including  ‘security  systems 

themes  of  FP7, 

Coherence with Hercule II 

Another EU programme with possible overlaps or coherence with ISEC is Hercule II, which 
focuses  on  the  objective  of  countering  fraud  and  other  illegal  activities  affecting  the  EU’s 
financial  interests.  Both  Programmes  aim  to  enhance  cooperation  between  law  enforcement 
authorities across the EU. However, Hercule II addresses the national authorities dealing with 
the EU's financial interests, which do not necessarily need to be law enforcement authorities. 
Furthermore, the scope of crime prevention in ISEC is much broader. 

Actions  funded  under  Hercule  II  included  technical  assistance  for  national  authorities,  in 
particular  providing  specific  knowledge,  equipment  and  IT  tools  facilitating  transnational 
cooperation,  and  training  activities  for  law  enforcement  agencies  which  contribute  in  a 
preventive and/or operational way to combating fraud detrimental to the financial interests of 
the  European  Union.  A  third  ‘legal  part’  included  co-financing  of  trainings,  seminars, 
comparative  studies,  and  conferences  seeking  to  address  the  legal  and judicial  protection  of 
the  EU  financial  interests  against  fraud,  corruption  and  any  other  illegal  activities.  Results 
from  the  stakeholder  interviews  suggest  that  the  scope  for  coherence  with  Hercule  II  is  the 
largest in activities regarding the development of technology and training of law enforcement 
staff.  Although  in  practice  Hercule  II  activities  are  focusing  on  protecting  EU’s  financial 
interests,  the  nature  of  these  activities  had  a  potential  for  overlaps  with  topics  which  were 
covered by ISEC. 

44 

 
 
 
 
EQ9b.  To  what  extent  was  the  ISEC  programme  coherent  with  actions  related  to  the 
prevention of and fights against crime supported by national resources of the Member States? 

The assessment of this evaluation question was limited by a lack of available data due to the 
fact that, according to  a number of  Commission  officials  interviewed, national actions were 
not  systematically  reviewed  on  their  potential  coherence/similarities  during  the  design  and 
implementation of ISEC. However, Member States received a draft of the Work Programme 
and were invited to provide their comments. Following incorporation of their comments, the 
final  version  of  the  AWP  was  then  adopted  via  a  Decision.  This  allowed  Member  States  to 
highlight  any  potential  overlaps  or  duplication  between  ISEC  activities  and  their  national 
actions. It can be argued that the Commission and Member States could have worked together 
further to ensure a higher degree of coherence, but this would have been challenging given the 
number of Member States involved and the high number of thematic areas covered by ISEC. 
Overall, the majority of the Member State representatives stated that ISEC was an excellent 
tool  that  enabled  the  Member  States  to  better  implement  actions  to  prevent  crime  in  the 
different areas and that the ISEC priorities responded to the practical cooperation needs of the 
Member States. 

EQ9c. To what extent is it possible to identify synergies between the results of actions funded 
under  the  ISEC  Programme  and  the  results  of  similar  actions  supported  under  other  EU 
Programmes  (such  as  the  Hercule  Programmes),  that  were  carried  out  by  different 
organisational entities in the Member States and/or third countries? 

Regarding  the  possibility  to  identify  synergies  between  results  of  ISEC  actions  and  similar 
actions supported under other EU Programmes, evidence collected from project coordinators 
via  an  online  survey  shows  that  33%  of  respondents  were  aware  of  other  activities 
implemented outside the project and covering similar objectives.116 Similarly, 31% of partner 
organisations  responded  that  they  were  aware  of  other  activities  implemented  at  national, 
international  and  EU  levels.  These  activities  included,  for  example,  a  Fundamental  Rights 
Agency  (FRA)  study  on  extreme  form  of  labour  exploitation,  and  analysis  conducted  by 
Transparency  International  and  ALTER-EU  regarding  lobbying  regulation  issues.  However, 
these results are not representative of partner organisations as this survey question received a 
low number of responses from them.117 

Examples of synergies provided by coordinating organisation survey respondents were: 

  Project  reference  JLS/2009/ISEC/CFP/AG:  the  project  coordinators  scheduled 
training  activities  with  CEPOL,  to  develop  a  common  platform  with  an  existing 
platform  created  by  another  project. According  to  the  survey  respondent,  the  project 
avoided overlapping and unnecessary expenditure of money on redundant activities; 

116   The  online  survey  to  coordinating  organisations  had  a  total  of  102  responses.  The  question  ‘While 
implementing the project(s) your organisation coordinated, were you aware of any other activities outside 
your project being implemented that covered similar objectives’ received 90 responses. 

117   The online survey to co-beneficiaries had a total of 110 responses. The question ‘While implementing the 
project(s) your organisation contributed to, were you aware of any other activities outside your project being 
implemented that covered similar objectives’ received 31 responses. 

45 

 
 
 
                                                            
  CRYME JLS/2007/ISEC/55: the project CRYME created a strong relationship with 
the Italian Ministry of Justice, helping to foster cooperation between several mosques 
and  institutions  and  to  raise  the  awareness  in  European  organisations  on  the 
importance of working in prison; 

  HOME/2012/ISEC/AG/4000004378:  The  survey  results  indicate  that  the  project 
revealed  emerging  risks  in  the  interconnection  between  radicalization,  counter-
radicalization  and  social  movements  of  different  kinds  (left,  right,  pro  /  anti-
immigrant)  by raising awareness  within intelligence services of various countries, as 
well as the police; 

  HOME/2011/ISEC/AG/400002575:  the  EURAD  project  created  strong  synergies 
between  the  partners  and  many  former  foreign  fighters,  as  well  as  organisations 
representing  ethnic  and  religious  minorities  in  many  countries,  including  Syria,  Iraq 
and  Libya  and  contributed  to  national  analysis  on  the  possibility  got  terrorists  to 
become  informers  as  an  alternative  to  criminalization.  The  project  contributed  to 
reporting  to  various  authorities  on  the  presence  of  non-Muslims  foreign  fighters  in 
several international and European countries. 

Concerning  co-beneficiaries,  only  ten  respondents  answered  the  question  on  the  creation  of 
synergies  and  types  of  synergies.  The  answers  included  similar  examples,  relating  to  the 
exchange of experiences or enhanced cooperation between project partners. 

7.5 EU added value 

EQ10.  To  what  extent  would  the  beneficiaries under  the ISEC  programme  (Member  States, 
Universities, Institutes, Associations, etc.) be able to carry out the activities necessary for the 
implementation  of  the  EU  policies  in  the  field  of  the  prevention  of  and  fight  against  crime 
without the support of the ISEC programme? 

The added value of the Programme is closely linked to the ‘importance’ of EU funding for the 
organisations involved and to its ability to foster transnational cooperation which would often 
not  have  taken  place  if  projects  were  supported  through  national  funding  alone.  Findings 
suggest  that  the  Programme  financing  was,  overall,  proportionate  to  the  funding  needs  of 
coordinating organisations in order for them to implement their project, with 88% of surveyed 
coordinating  organisations  finding  that  the  level  of  funding  was  sufficient  to  implement  the 
activities for the projects they coordinated. Results from the same survey further suggest that 
a  significant  proportion  of  the  activities  developed  under  ISEC  would  have  not  been 
developed  in  the  absence  of  ISEC  funding.  This  was  particularly  noted  for  activities  in 
specific  ISEC  thematic areas.  For  example,  an  interviewee reported that  ISEC  was the  only 
source of funding for CBRN-related activities in Spain and that without EU funding none of 
these activities would have been carried out, notably given the start of the economic crisis in 
2008. Another example, bearing in mind that over 30 projects were implemented to support 
the  implementation  of  the  Prüm  Decisions  (over  EUR  17  million)  and  covering  most  EU 
Member  States,  is  the  fact  that  several  interviewees  suggested  that,  without  ISEC  funding, 
Prüm data exchange would not have been developed due to a lack of national funding. 

46 

 
 
 
Furthermore, 87% of coordinating organisations118 reported that they would have struggled to 
find  national  or  other  funding  opportunities  to  fully  fund  and  implement  their  envisaged 
activities.  However,  13%  of  respondents  stated  that  they  would  have  been  fully  financed 
through  other  funding  sources  to  deliver  projects.  For  those  coordinating  organisations  for 
which funding was not available, the overall majority indicated that this was due to the fact 
that alternative sources of funding did not exist, while more than one fifth explained that their 
envisaged activities were not eligible for funding from other sources at several levels. In the 
difficult  financial  context  of  the  time,  many  organisations  reported  seeing  their  national 
funding  cut  and  some  being  forced  to  make  organisational  changes  (i.e.  cancelling  their 
membership to European networks or reducing the staff involved in these networks) following 
financial constraints. These findings confirm that ISEC filled a gap in this regard. 

Those  organisations  which  indicated  that  they  would  have  been  able  to  find  funding  from 
another source were also asked about their reasons for applying to ISEC funding. The table 
below presents the responses provided per reason. The fact that the vast majority of reasons 
provided for choosing ISEC over other funding sources (30 out of 36 responses) relates to an 
aspect  of  ISEC's  EU  added  value  suggests  that  the  programme’s  EU  added  value  was  the 
main reason for applying for funding. 

Table 1: Reasons for choosing ISEC over other funding sources 

Interviews  with  coordinating  organisations  further  suggest  that  ISEC  funding  made  a 
substantial  difference  to  the  scale  of  results  achieved.  Although  some  activities  could  have 
been  implemented  without  ISEC  funding,  most  projects  funded  through  monopoly  bodies 
would not have been able to operate at the same level and in all Member States. For instance, 
RAILPOL  started  as  a  regional  cooperation  initiative  between  Belgium,  Germany  and  the 
Netherlands, but thanks to ISEC funding, it grew to cover twelve additional Member States. 
AQUAPOL continued its activities after the ISEC funding ended but would not have started 
without this initial financial boost. 

The importance of ISEC funding is also demonstrated by the withdrawal of national members 
from ISEC-funded projects once the EU funding ended. For example, an interviewee reported 

118   90 organisations provided responses to this question. 

47 

 
 
 
                                                            
that some countries had to stop being members of RAILPOL as they were not able to finance 
their participation through national resources (e.g. to cover travel costs of staff to participate 
in  exchanges  or  training).  ISEC  funding  provided  financial  support  to  those  countries  (e.g. 
BG,  RO,  SI,  LT)  with  limited  national  resources  allocated  in  their  national  programmes  to 
specific  areas,  such  as  cross-border  cooperation,  exchanges  of  experience  and  technologies. 
On the other hand, some themes covered by ISEC were cross-cutting and also interesting for 
other  DGs  and  EU  Agencies,  which  meant  that  they  could  also  be  funded  through  other 
sources (notably in the fields of drugs trafficking, financial or economic crime). For example, 
cooperation amongst almost all partner organisations of the European Competition Network, a 
network  of  national  competition  authorities  of  EEA  countries  initially  called  the  European 
Antitrust  Training  in  Forensic  IT  (EAT_FIT)  and  funded  under  ISEC,  was  subsequently 
funded and coordinated by DG Competition. 

As  mentioned  above,  a  key  aspect  of  ISEC's  EU  added  value  lies  in  its  ability  to  foster 
transnational  cooperation.  Although  concerns  were  raised  during  the  first  two  programming 
years with regard to the actual contribution of partners to ISEC projects and the difficulties for 
some coordinating organisations to find the ‘right’ partner, overall and especially later on in 
the  Programme,  partnership  cooperation  was  considered  very  positive  by  the  stakeholders 
consulted. Changes introduced to the eligibility criteria of projects largely contributed to this. 

The important contribution of ISEC funding to transnational cooperation and implementation 
of  EU  law  in  the  various  areas  covered  by  the  prevention  and  fight  against  crime  was 
highlighted  during  a  high  number  of  interviews  with  all  types  of  stakeholders.  More 
specifically, some stakeholders commented positively on the advantages brought by the ISEC 
Programme,  which  allowed  for  the  funding  and  development  of  research  projects.  Other 
security related programmes (e.g. Horizon 2020) were considered by some stakeholders to be 
too technical and restrictive and they concluded therefore that research projects, such as for 
example the Organised Crime Portfolio, may have not been granted the necessary funding in 
the  absence  of  ISEC.  Even  though  the  amounts  involved  can  be  considered  to  be  relatively 
low,  or  sometimes  even  insignificant,  when  compared  to  the  national  funding  dedicated  to 
some  of  thematic  areas  covered,  stakeholders  overall  agreed  that  ISEC  made  a  significant 
contribution  to  national  law  enforcement  agencies  cooperating  closer  transnationally,  to 
building  mutual  trust,  and  to  the  formation  of  new  relationships  between  organisations 
working with counterparts in other Member States. Furthermore, based on the online survey 
to  project  participants,  when  asked  whether  their  participation  in  the  ISEC  project  made  a 
positive impact on their daily work, more than 81% (out of 107 respondents) confirmed that it 
did. Many mentioned that the Programme allowed them to cover those aspects which would 
usually  be  the  first  to  be  reduced  if  resources  were  scarce;  this  helped  to  increase  the 
possibility  of  action  in  all  relevant  areas  (e.g.  training  and  dissemination  activities)  and 
especially international cooperation. 

Transnational  partnerships  boosted  cooperation  and  mutual  trust  amongst  the  ISEC 
beneficiaries  also  after  the  end  of  the  projects  with  35%  of  the  respondents  to  the  online 
survey to coordinating organisations noting that cooperation amongst partners fully continued 
after  the  project  funding  ended.  However,  in  many  cases  the  actual  impact  at  EU  level  and 
thus  the  EU  added  value  of  national  projects  depends  to  a  certain  extent  on  the  actual 
promotion and dissemination of the transferable outcomes at EU level. The evaluation found 
room for improvement in this area (see section 8). 

48 

 
 
49 

 
 
 
 
8. CONCLUSIONS 

The  ISEC  programme  played  a  key  role  in  fostering  action  at  an  EU  level  in  the  areas  of 
preventing and tackling crime at a time where EU policies in these areas were non-existent or 
at early stages of development. ISEC contributed to a large extent to the achievement of its 
first two specific objectives, the promotion and development of horizontal methods and tools 
necessary for strategically preventing and fighting crime and guaranteeing security and public 
order,  as  well  as  the  promotion  and  development  of  cooperation  among  law  enforcement 
agencies and other national authorities and Union bodies. The programme contributed to the 
promotion and development of best practices for the protection and support of witnesses and 
crime victims to a lesser extent. 

Relevance 

In terms of relevance, the evaluation found that the ISEC objectives and results were overall 
highly relevant to the needs of Member States in the areas covered by the programme. This 
was largely thanks to the fact that the Programme covered a wide range of thematic areas and 
allowed for the provision of funding to a wide range of stakeholders. However, the specific 
objectives  concerning  the  development  of  best  practices  for  protection  and  support  to 
witnesses and victims attracted fewer projects. Furthermore, the demand-driven design of the 
programme sought to allow ISEC to target funding to Member State and EU needs. However, 
this  was  only  partly  achieved  given  that  this  demand-driven  design  resulted  in  a  significant 
geographical imbalance, with Member States like Italy and Germany coordinating very large 
numbers of projects, while the other Member States coordinated very few projects. In order to 
allow Member States more equal access to funding and to improve their participation, a shift 
towards shared management mode was introduced for the successor fund, ISF-Police. Unlike 
ISEC,  which  was  implemented  only  under  direct  management,  ISF-Police  is  being 
implemented on the basis of a combination, with around 60% of the available funding being 
allocated  to  national  programmes  under  shared  management  and  40%  allocated  to  Union 
actions under direct management. It is expected that this shift could increase the ISF-Police’s 
effectiveness and efficiency.  

The  evaluation  also  identified a  time  lag  between  the  formulation  of  priorities  under  AWPs 
and their actual implementation; this was due to the fact that the AWPs reflected the priorities 
set in the previous year, which meant that there was a one year difference between the setting 
of  priorities  and  their  implementation,  which  at  times  hindered  the  programme’s  relevance. 
This  was  partly  addressed  in  the  successor  fund,  ISF-Police,  through  the  inclusion  of 
emergency assistance “to address urgent and specific needs”.119 

119   Article 10, Regulation No 513/2014. 

50 

 
 
 
 
                                                            
Effectiveness 

The main results identified from ISEC projects were enhanced networking, forms of sharing 
information and best practices and increased levels of knowledge and skills of practitioners. 
This is followed by improved understanding of methods, tools and practices and the increased 
take  up  of  new  or  transferred  methods,  tools  and  practices.  The  programme’s  effectiveness 
was evidenced by the fact that the majority of projects accomplished their planned activities 
and achieved their objectives. ISEC projects were particularly effective in contributing to the 
development of stable transnational cooperation between Member States and their agencies, 
as well as better exchange of information, particularly in the field of forensics, drugs, PNR, 
cybercrime  and  trafficking  in  human  beings.  National  authorities  interviewed  also  reported 
that  ISEC  projects  fostered  cooperation  between  agencies  in  the  same  Member  State, 
increasing  synergies  between  the  law  enforcement  Agencies.  The  cooperation  between  the 
public and private sector in Member States was limited. 

Room  for  improvement  was  identified  the  monitoring  of  projects  and  the  compilation  of 
project results and deliverables. The evaluation found that the creation of a central repository 
containing  detailed  data  on  individual  ISEC  project  results  and  outputs  would  have  greatly 
enhanced  the  effectiveness  of  the  programme.  For  example,  methodologies  developed  to 
investigate  certain  types  of  crimes,  reports,  analysis,  best  practices,  training  materials,  etc. 
could have been compiled centrally and thus, their potential multiplier effect and continuous 
usage for other interested stakeholders in other Member States could have been maximized. 
On the basis of the above finding, for projects conducted under Direct Management under the 
successor  funding  instrument,  ISF,  a  repository  of  the  results  and  outputs  of  such  projects 
could be created and shared via, for example, the European Crime Prevention  Network,  the 
Critical Infrastructure Warning  Information  Network  (CIWIN),  the website  of  the  EU  Anti-
Trafficking Coordinator120 or via other Commission services. This has not yet been put into 
practice under ISF but could be taken into consideration for the next MFF. 

Furthermore,  given  the  relative  newness  of  the  Programme  and  the  lack  of  resources  of  the 
Commission at the time to closely follow all ISEC projects from start to end, the evaluation 
showed  that  a  dedicated  technical  assistance  budget  could  have  been  useful.  Such  a  budget 
could have better supported the development of a state of the art monitoring system to collect 
and  analyse  data  on  financial  progress,  outputs  and  results  of  projects,  as  well  as  provided 
technical  expertise  throughout  the  lifecycle  of  projects  and  increased  the  dissemination  of 
results. 

This has been addressed to a limited extent under the successor fund, ISF-Police, through the 
possibility for the financial instrument to contribute annually to technical assistance activities, 
which  could  be  used  for  the  development  of  a  monitoring  system.121  However,  under  ISF-

120   See http://ec.europa.eu/anti-trafficking/. 
121   Article 9, Regulation (EU) No 513/2014. 

51 

 
 
                                                            
Police  the  technical  assistance  available  cannot  be  used  for  staff  contracting  and  the 
development  and  implementation  of  a  monitoring  system  requires  also  additional  human 
resources on both the Member State and the Commission side. This could be addressed in the 
future MFF. 

Efficiency 

ISEC  was  found  to  be  efficient  to  a  large  extent.  In  general,  the  vast  majority  of  consulted 
stakeholders perceived EU funding to have been sufficient and to have had an overall high or 
very high value for money across all thematic areas. Flexibility of the Fund was appreciated 
by the stakeholders. No significant differences in the cost-effectiveness of individual projects 
were found following a comparison of similar financial inputs to their outputs and results.  

An upward trend in the absorption of funds spent on action grants with an overall absorption 
rate  of  74%  across  the  period  2007-2013,  and  a  peak  of  79%  in  the  last  year,  suggests 
improved  implementation  of  the  programme.  This  is  likely  to  be  a  result  of  changes 
introduced  during  the  ISEC  programming  period  and  of  increased  Member  State  and 
Commission  experience  with  the  programme.  Furthermore,  the  funding  provided  through 
ISEC was perceived by the vast majority of consulted stakeholders as having been sufficient 
and  as  having  been  highly  cost-efficient  for  all  implemented  projects.  In  particular,  the 
creation of networks was found to be highly cost-efficient. 

Although  the  evaluation  found  mixed  responses  regarding  the  perception  of  administrative 
burden related to the implementation of the programme when stakeholders where consulted, 
overall  it  was  concluded  that  the  ISEC  Programme  had  a  similar  amount  of  administrative 
burden to other EU programmes. 

However,  the  following  areas  for  improvement  in  terms  of  efficiency  were  identified: 
additional support could be provided to potential applicants during the application phase, the 
monitoring of project outputs and results could be enhanced by requesting more detailed final 
reporting  from  project  beneficiaries.  With  regard  to  improving  the  monitoring  of  project 
results  on  the  basis  of  the  required  reporting,  it  was  concluded  also  that  a  qualitative 
assessment of the results of the Programme was missing due to the absence of a peer review 
on  project  results  conducted  by  external  experts.  Although  under  ISF-P  this  has  slightly 
improved as Member States have to submit an Annual Implementation Report under Shared 
Management and Interim and Final Reports under Direct Management, the absence of a peer 
review  of  results  still  represents  a  shortcoming  that  could  be  addressed  under  the  next 
generation of Funds.  

As  already  stated  under  relevance,  with  regards  to  dissemination  of  project  outputs  and 
results,  consulted  experts  and  coordinating  organisations  recognised  that  the  absence  of  a 
central repository bringing together the findings of ISEC projects informing stakeholders on 

52 

 
 
 
available results has decreased the effectiveness and efficiency of the Programme to a certain 
extent. 

Coherence 

ISEC  supported  the  implementation  of  EU  obligations  and  cross-border  cooperation  among 
Member  States  in  a  large  number  of  crime  areas.  In  the  context  of  financial  crisis,  national 
funding was very scarce or not available for such cooperation and thus, no significant risk of 
duplication with national funding streams was found. ISEC was therefore deemed as highly 
coherent with initiatives at national level. 

A  substantial  scope  for  coherence  and  complementarity  was  identified  between  ISEC  and 
other  EU  Programmes,  namely  JPEN,  DAPHNE  III,  FP7  and  Hercule  II.  Thanks  to 
cooperation between DG Home and DG Justice, synergies were achieved between DAPHNE 
III  and  the  two  ISEC  specific  objectives  relating  to  the  promotion  and  development  of  best 
practices for protection and support of witnesses and crime victims, especially in the area of 
trafficking in human beings. Although potentially a theoretical overlap between ISEC and the 
remaining EU programmes is possible, no evidence of overlaps was found. 

Overall,  an  effective  coordination  at  both  design,  implementation  and  delivery  stage  was 
crucial to maximise the potential for coherence and complementarity and to avoid the risk of 
duplication. It can be concluded, on the basis of the evidence collected, that this was achieved 
to a large extent. 

EU Added Value 

In  line  with  the  strong  transnational  dimension  reflected  in  its  objectives,  the  ISEC 
programme was found to have had a high EU added value. Due to a lack of national funding 
for  transnational  cooperation  in  many  Member  States,  a  significant  part  of  activities 
developed  under  ISEC  would  not  have  been  developed  in  the  absence  of  funding  from  this 
programme. In other cases, ISEC funding made a substantial difference to the scale of results 
achieved. Even though other funding was available in similar policy areas, ISEC was unique 
in  the  high  level  of  funding  provided,  the  number  of  different  crime  areas  covered,  and  the 
solely transnational focus of the programme. 

In  the  vast  majority  of  cases,  coordinating  and  partner  organisations  considered  that 
cooperation and exchange of information with organisations from other countries helped them 
to  gain  more  knowledge  and  expertise  on  the  subjects  they  were  working  on.  However,  the 
EU  added  value  of  projects  depended  to  a  certain  extent  on  the  actual  promotion  and 
dissemination of the transferable outcomes at EU level, which was limited.  

53 

 
 
 
 
As mentioned above, the geographical distribution of coordinating and partner organisations 
that  received  funding  was  highly  uneven  and  concentrated  in  a  few  Members  States.  EU 
added value could have been higher if organisations had been better spread over the Member 
States. Various reasons seemed to have caused this uneven spread. Despite the Commission’s 
attempt  to  stimulate  applications  through  regional  information  days,  applications  were  still 
very  dependent  on  factors  such  as  the  Member  States’  Committee  representative’s  pro-
activeness,  knowledge  and  capacity  of  organisations  to  submit  high  quality  proposals  and 
their  compliance  with  the  eligibility  criteria.  As  mentioned  above,  this  geographical 
imbalance has been addressed through the shift towards shared management. Although under 
the current fund MS are entitled to implement transnational projects and/ or national projects 
with potential for transferability, the shift towards shared management could, however, mean 
that in practice many projects implemented by Member States under ISF-P tend to have less 
of a transnational dimension than projects previously conducted under CIPS, which due to the 
Direct management mode naturally had a higher transnational dimension. In consequence, it 
can be stated that the EU added value of the ISF-P fund has somehow decreased compared to 
ISEC. 

54 

 
 
 
 
 
ANNEX 1 - PROCEDURAL INFORMATION 

Leading Directorate-General 
Participating Units of DG HOME 

Participating DGs  

DG HOME 
A2 – Legal Affairs 
B2 – Visa Policy and document security 
B3 – Information Systems for Borders and Security 
C2 – Border Management and Schengen 
E1 – Union Actions 
E2 – National programmes for South and East 
Europe, evaluation, AMF/ISF Committee 
E3 - National programmes for North and West 
Europe, budget, MFF, agencies 
Secretariat-General 
DG BUDG 
DG ENV 
DG ENERGY 
DG GROW 
DG JUST 
November 2015 
2016/HOME/073  
Contract signed in August 2016 with ICF 
Consulting Services Limited 

Roadmap approval 
Agenda Planning  
External consulting firm 
specialised in evaluation 
Number of steering group meetings  5 
Last deliverable handed in 
Approval of the final report by 
Steering Group 

January 2018 
January 2018 

55 

 
 
 
 
 
 
ANNEX 2 - STAKEHOLDER CONSULTATION 

1. Overview 

A  large  part  of  the  ex  post  evaluation  of  the  Prevention  and  fight  against  crime  2007-2013 
(ISEC) Programme is the stakeholder consultations. The consultations were conducted by the 
European Commission and by an external consulting company specialised in evaluation ('the 
external  evaluators')  in  the  framework  of  their  contract  with  the  Commission  to  provide  a 
study  on  the  ex  post  evaluation  of  the  implementation  of  ISEC.  An  overview  of  the 
consultation  processes,  the  type  of  stakeholders  consulted  as  well  as  the  results  of  these 
consultations is provided below. 

1.1 The public consultation conducted by the European Commission  

From 14 July 2016 to 15 November 2016, the European Commission held an Internet-based 
public  consultation  on  the  ISEC  Programme  2007-2013  in  the  form  of  an  online 
questionnaire.  Contributions  were  sought  from  individuals  (experts,  beneficiaries),  public 
authorities,  intergovernmental  and  non-governmental  organisations,  social  partners  and  civil 
society,  academic  institutions,  international  organisations,  EU  Institutions  and  Agencies, 
based  in  EU  Member  States  or  third  countries.  The  aim  of  the  consultation  was  to  collect 
views and opinions on the results and impacts of projects and actions co-financed by the ISEC 
programme  during  2007-2013  and  to  assess  their  relevance,  effectiveness,  efficiency, 
coherence  and  EU  added  value.  The  final  number  of  respondents  was  very  low:  three 
participants from three Member States (Germany, Italy and Austria), all of them EU citizens. 
Given this low response rate, contributions cannot  be considered to  be representative of the 
targeted  stakeholders.  However,  the  results  may  provide  additional  insights  and  will  be 
presented below together with the findings of the other consultations. In addition, the fact that 
the information available on the identities of the participants is based on self-reported values 
which cannot be verified should be considered as a further limitation. 

1.2 The consultations conducted by the external evaluators 

In  addition  to  the  consultation  conducted  by  the  European  Commission,  the  external 
evaluators  conducted  targeted  consultations  with  a  range  of  stakeholders  through  online 
surveys, stakeholder consultations, thematic case studies, expert workshops and expert input. 
This section provides an overview of the type of consultation process that took place in each 
case. The unit responsible for the evaluation (DG  HOME, Unit  E2) supported the access to 
relevant  EU-level  stakeholders.  Stakeholders  were  approached  via  e-mail  introducing  the 
study  and  the  importance  of  their  contribution,  including  a  letter  of  introduction  from  DG 
HOME. 

1.2.1 Online surveys 

56 

 
 
 
The external evaluators launched three online surveys at the end of December 2016 addressed 
at coordinating organisations, partner organisations and project participants. 

Online survey for coordinating organisations 

This  online  survey  was  available  in  five  languages:  English,  French,  German,  Italian  and 
Spanish. The survey ran for four weeks – from 4 November 2016 to 5 December 2016. The 
goal of this survey was to invite all coordinating organisations of ISEC projects to participate 
by  answering  questions  on  the  evaluation  criteria  (relevance,  effectiveness,  efficiency, 
coherence, and EU added value). From a total of 658 contacts invited to complete the survey, 
156 responses were received (97 complete responses and 59 partial responses). 

Online survey for partner organisations 

The  coordinating  organisations  were  asked  to  send  this  survey  to  the  relevant  partner 
organisations of ISEC projects. Similarly to the online survey for coordinating organisations, 
this survey was open from 4 November 2016 until 5 December 2016 and it was available in 
the  same  five  languages.  A  total  of  110  responses  to  the  survey  were  received  of  which  37 
were complete and 73 partial responses. 

Online survey for project participants 

This online survey ran for 12 weeks – from 1 December 2016 until 28 February 2017. It was 
also  available  in  English,  French,  German,  Italian,  and  Spanish.  The  participants  were 
identified with the assistance of the coordinating organisations which forwarded the survey to 
the participants of their ISEC projects. The total number of replies of this online survey was 
285 from which 118 are complete responses and 167 are partial.  

1.2.2 Stakeholder consultation 

The  external  evaluators  carried  out  41  interviews  with  six  types  of  stakeholders.  The 
interviews  were  either  held  face-to-face  or  via  telephone  conversation.  The  six  types  of 
interview  conducted  for  the  purpose  of  the  evaluation  were:  scoping  interviews  with 
Commission  officials,  EU  agencies  and  offices,  Commission  officials  responsible  for  other 
EU  programmes,  European  and  international  organisations,  representatives  of  monopoly 
bodies,  and  Member  State  representatives.  The  external  evaluators  carried  out  these 
interviews in a semi-structured way based on questionnaires developed for each stakeholder 
type.  

1.2.3 Case studies 

Following the request of DG HOME, the external evaluators carried out thematic case studies. 
The  aim  of  these  case  studies  was  to  examine  more  closely  the  results  and  impacts  in  one 
specific thematic area, and where possible to link them to the wider context at EU level. In 
total 8 thematic case studies were carried with the participation of six Member States– Police 

57 

 
 
 
cooperation  (Germany),  Economic  and  financial  crime  (Italy),  Cybercrime  (Italy,  Ireland, 
UK),  Chemical,  biological,  radiological  or  nuclear  materials  and  explosives  (CBRNE) 
(Spain),  Drugs  and  firearms  trafficking  (UK),  Radicalisation  (The  Netherlands),  Forensics 
(Germany), and Organised crime (Italy). 

1.2.4 Expert workshop and expert input 

The external evaluators organised two expert workshops which were held in Brussels – one 
on  23  September  2016  and  one  on  6  March  2017.  Both  workshops  were  carried  out  with 
experts involved in the evaluation. The second workshop was followed by written inputs from 
the experts on the findings presented in the draft final evaluation report. 

2. Results 

The most significant results of the consultations carried out by the European Commission and 
the  external  evaluators are  grouped  in  this  section  and  presented according  to  the following 
evaluation criteria: relevance, effectiveness, efficiency, coherence, and EU added value. 

2.1 Relevance 

The assessment of the relevance of ISEC was focused on the extent to which the objectives 
and the design of the Programme corresponded to the needs related to the prevention of and 
fight against crime. Regarding  the relevance of  the objectives,  the stakeholder consultations 
confirmed  that  the  first  two  specific  objectives  "coordination,  cooperation  and  mutual 
understanding of law enforcement agencies, other national authorities and related EU bodies" 
and  "promotion  and  development  of  horizontal  method  and  tools"  were  highly  relevant  and 
necessary.  An  overview  of  the  opinions  of  the  respondents  to  the  surveys  on  the  ISEC 
programme  shows  that  69%  strongly  agreed  with  the  statement  that  the  objectives  and  the 
expected  results  of  the  ISEC  project  corresponded  to  their  needs/the  needs  of  their 
organisations. Concerning the relevance of the design of the programme, the results from the 
consultations,  the  interviews  and  the  expert  workshops,  show  that  in  broad  terms  the 
programme  allowed  for  flexibility  in  the  funding  of  different  types  of  actions  that  could 
evolve and change over time. 

In addition, the consulted stakeholders agreed on the importance of ISEC during the financial 
crisis, as no alternative for financing transnational cooperation in the area of prevention and 
fight against crime existed. Furthermore, the extensive scope of the programme was, overall, 
appreciated  by  the  applicants  and  covered  their  needs  on  the  ground.  The  European 
Commission  public  consultation  supports  these  positive  findings.  For  instance,  Question  7 
asked the participants to what extent, in their opinion, the projects and the activities funded in 
their country of residence addressed the needs related to prevention of and fight against crime. 
The  majority  (2/3)  replied  that  they  related  to  the  needs  to  a  significant  extent  and  the  rest 
(1/3) only "to a limited extent". 

2.2 Effectiveness 

58 

 
 
ISEC-funded projects contributed to achieving the general objectives of the Programme and 
its  specific  objectives  related  to  developing  horizontal  tools  and  methods,  and  cooperation, 
coordination and mutual understanding in the area of prevention and fight against crime. The 
evaluation of the effectiveness  of ISEC aimed to  assess the extent to  which the  Programme 
was  effective  in  achieving  those  general  and  specific  objectives.  Question  4  of  the 
Commission's public consultation asked participants whether, based on their experience, the 
actions  financed  by  ISEC  in  their  country  were  consistent  with  the  objectives  of  the  Fund. 
Two of the respondents replied that they "do not know" (DE and IT) and the third one replied 
that they were consistent "to a limited extent" (AT). 

Based on the evidence collected, the external evaluation findings suggest that ISEC broadly 
achieved  most  of  its  specific  objectives  which,  it  can  be  argued,  contributed  to  crime 
prevention  and,  ultimately,  better  security  for  EU  citizens.  Furthermore,  the  project 
coordinating organisations, partner organisations and participants surveyed perceived ISEC to 
have contributed to a great extent to better security of EU citizens. Only 13% of participants 
and 4% of coordinating organisations respectively responded that the ISEC projects made no 
contribution at all; none of the partner organisations reported this. Surveyed stakeholders also 
believed  that  ISEC  made  a  contribution  to  the  promotion  and  development  of  horizontal 
methods and tools, with 37% of coordinating organisations, 25% of partner organisations, and 
33%  of  participants  respectively  perceiving  ISEC  to  have  made  a  very  significant 
contribution. Only 1% of coordinating organisations believed that ISEC made no difference at 
all.  

Regarding  the  extent  to  which  projects  under  ISEC  achieved  their  objectives,  implemented 
activities  and  delivered  expected  outputs,  the  overall  evidence  collected  suggests  that  ISEC 
projects  have  largely  achieved  their  objectives  and  delivered  the  expected  outputs.  The 
external  evaluators'  in-depth  analysis  of  a  sample  of  projects  showed  that  the  majority  of 
projects  achieved  all  their  objectives  (84%),  while  15  %  of  projects  partially  achieved  their 
objectives and only 2 projects did not achieve their objectives. Similarly, the final assessments 
by the Commission of the sample of analysed projects shows that 75% of projects have been 
assessed  as  excellent  or  very  good  with  beneficial  results  and  impacts  achieved;  20%  of 
projects have been assessed as acceptable/average, while 5% of the projects were marked as 
poor or very poor. The vast majority of coordinating organisations surveyed (89) also reported 
that project activities were delivered fully (85%) or that most activities were delivered (14%). 
In  addition,  project  coordinating  organisations (91)  responded  that  the objectives  were  fully 
achieved  (82  responses),  while  according  to  nine  responses  objectives  were  only  partially 
achieved. 

2.3 Efficiency 

The efficiency of ISEC aimed to measure the extent to which the costs of the actions funded 
under  the  programme  were  justified  compared  to  the  outputs.  The  evaluation  focused  on 
examining  the  financial  efficiency  of  the  Prorgamme  across  the  funding  years  and  funding 

59 

 
 
mechanisms, the extent to which the resources were used efficiently, and the efficiency of the 
Commission's management of human resources. 

As  confirmed  by  the  stakeholders  consulted  and  desk  research  performed,  it  was  concluded 
that  the  ISEC  Programme  had  a  similar  amount  of  administrative  burden  as  other  EU 
programmes.  Question  8  of  the  Commission's  public  consultation  asked  whether  the 
participants considered that the effect of the actions carried out under the Fund were achieved 
to  a  reasonable  extent  in  terms  of  the  financial  and  human  resources  deployed.  Responses 
show  a  mixed  picture,  with  the  respondent  from  DE  finding  that  benefits  were  greater  than 
costs, the respondent from IT replying with "I don’t know" while the third respondent (AT) 
replied that the costs are greater than benefits and added that the costs in administration are 
high and with little use in return. Findings from the survey of coordinating organisations are 
also  mixed  regarding  the  overall  perception  of  administrative  burden  related  to  the 
implementation of ISEC. One in five respondents reported that no administrative burden was 
perceived  whereas  just  over  one  third  of  respondents  reported  experiencing  some 
administrative burden. In contrast, a quarter of respondents reported experiencing a significant 
level of administrative burden and just one in  five a very significant  level of administrative 
burden.  The  survey  with  partner  organisations  confirms  similar  levels  of  administrative 
burden  in  implementing  their  projects  compared  to  other  non-Commission  funded  projects: 
the  large  majority  of  partner  organisations  reported  experiencing  administrative  burden.  In 
this  regard,  37%  reported  experiencing  significant  administrative  burden  and  33%  reported 
experiencing  some  administrative  burden,  whilst  3%  reported  experiencing  very  significant 
administrative  burden.  Only  one  respondent  reported  experiencing  no  administrative  burden 
and five respondents reported that they did not know. As regards to the financial efficiency of 
the  Programme,  EU  funding  was  overall  perceived  to  have  been  sufficient  for  the  vast 
majority  of  consulted  stakeholders  and  an  overall  high  or  very  high  value  for  money  was 
pointed  out  across  all  activities.  The  majority  of  coordinating  organisations  (87,6%) 
considered the EU funding provided to be sufficient. However, 11 coordinating organisations 
did not find the funding available sufficient and they highlighted issues such as difficulty in 
securing co-financing from other sources and ensuring continuity of certain outputs.  

Concerning the efficiency of human resources employed, the management by the Commission 
was overall considered positively, although the results of the consultations noted the existence 
of some issues in the technical follow-up of projects, the guidance provided to stakeholders 
and the interpretation of rules.  

2.4 Coherence 

The  evaluation  criterion  of  coherence  refers  to  the  extent  to  which  the  intervention  did  not 
contradict  or  create  duplications  with  other  interventions  with  similar  objectives.  The 
assessment  targeted  coherence  with  other  EU  programmes  in  the  same  field  and  coherence 
with national programme and initiatives. 

60 

 
 
 
The  external  evaluators'  assessment  identified  substantial  scope  for  coherence  and 
complementarity  between  ISEC  and  other  selected  EU  Programmes  such  as  JPEN, 
DAPHNE III, FP7 and Hercule II. 
For  instance,  based  on  information  received  from  stakeholder  interviews,  the  scope  for 
coherence with Hercule II is the largest in activities regarding development of technology and 
training of law enforcement staff. For example, training financed under Hercule II addressed 
similar topics as ISEC. 

The  results  from  the  European  Commission's  public  consultation  were  not  sufficiently 
conclusive  to  substantiate  such  findings.  Question  10  of  the  consultation  asked  participants 
about  the  extent  to  which  they  consider  that  the  EU  funding  for  the  ISEC  programme  was 
coherent with other actions in the same field funded by other EU financial instruments. Only 
one respondent replied that the EU funding for ISEC is coherent with other actions. The other 
two did not know. 

The  examination  of  the  external  evaluators  concluded  that  ISEC  was  largely  coherent  with 
initiatives at national level. ISEC supported the implementation of EU obligations and cross-
border  cooperation  among  Member  States  in  the  different  crime  areas.  In  the  context  of 
financial  crisis,  national  funding  was  not  available  for  such  cooperation  and  thus,  no 
significant  risk  of  duplication  with  national  funding  streams  existed.  Question  11  of  the 
Commission's public consultation asked participants whether they considered the EU funding 
provided for the ISEC Programme coherent with the national actions of the Member States. 
The  replies  varied  from  coherent  (DE),  to  coherent  to  a  limited  extent  (AT),  while  one 
respondent  did  not  know  (IT).  Despite  the  mixed  results  from  the  public  consultation,  the 
assessment  of  the  external  evaluators  found  that  overall,  the  majority  of  Member  State 
representatives  stated  that  ISEC  was  an  excellent  tool  that  enabled  Member  States  to  better 
implement  actions  to  prevent  crime  in  the  different  areas,  and  that  the  ISEC  priorities 
responded to the practical cooperation needs of the Member States. 

2.5 EU added value  

The  external  evaluators  assessed  the  EU  added  value  of  ISEC  in  three  directions  –  the 
transnational dimension, partnership aspects, and the importance of the EU funding provided. 

While examining the transnational dimension of the EU added value of ISEC, the external 
evaluator  concluded  that  due  to  the  demand-driven  design  of  the  Programme  (based  on 
applications),  the  geographical  distribution  of  coordinating  and  partner  organisations  that 
received  funding  was  uneven  and  concentrated in  few  Member  States.  Italy,  Germany,  UK, 
the  Netherlands  led  around  50  percent  of  the  projects  during  the  evaluation  period  (2007-
2013). During the same evaluation period the involvement of organisations as coordinators of 
funded projects located in Cyprus, Denmark, Luxembourg and Malta was very limited – five 
or less projects were coordinated by organisations located in these Member States. Thus, the 
evaluator  concluded  that  the  EU  added  value  could  have  been  higher  if  organisations  in  all 
Member States had participated to a higher degree.  

61 

 
 
The  added  value  of  partnerships  was  considered  to  be  high.  The  interviews  with 
coordinating  organisations  suggested  that,  overall,  the  partnerships  worked  well  in  the  vast 
majority  of  cases,  and  that  they  were  viewed  as  a  positive  experience  leading  to  valuable 
outcomes. According to these interviewees, a critical factor for a successful partnership was a 
pre-existing  good  relationship  with  organisations  on  which  the  partnership  under  ISEC  was 
built. Following the date collected from the coordinating organisations, it was concluded that 
they were overall satisfied with the partnerships. They indicated that ISEC projects increased 
the  knowledge  and  the  expertise  of  the  organisations  involved  in  these  transnational 
exchanges  -  67%  of  the  coordinating  organisations  (87)  indicated  that  generally  the 
partnership brought as added value more knowledge and expertise on the topic of the project 
whereas 49% noted that the partnership led to increased knowledge of policies and practices 
in  other  countries.  An  additional  33%  indicated  that  ISEC  supported  them  in  establishing  a 
network that helped them in raising awareness of the activities implemented by the national 
organisations or in disseminating the results achieved. 

Concerning  the  importance  of  EU  funding  to  carry  out  the  activities,  the  online  survey 
results showed that 88% of the coordinating organisations reported that the level of funding 
from  the  Commission  was  sufficient  to  implement  the  activities  for  the  projects  they 
coordinated. Thus, a significant part of the activities developed under ISEC would not have 
been  developed  in  the  absence  of  the  Programme's  funding.  The  public  consultation  asked 
participants in Question 1 whether they considered that in their county the implementation of 
the  ISEC  programme  during  2007-2013  affected  positively  the  work  of  the  public 
administration in  the field  of prevention of and fight against crime. Two  of the respondents 
replied with "significantly" (DE and IT) and one replied with "partially" (AT). Organisations 
indicating that they would have been able to find funding from another source (13% or 12 out 
of  90)  were  also  asked  why  they  chose  to  apply  for  ISEC  funding.  Multiple  answers  were 
possible but it is clear from the answers that the EU added value of the Programme constitutes 
one  of  the  main  reasons  for  applying  for  ISEC  funding  instead  of  funding  from  another 
source, as this is the third most common reason for choosing ISEC. 

62 

 
 
 
 
 
ANNEX 3 – METHODOLOGY 

Study conducted by external evaluators  

The preparatory ex post evaluation study conducted by the external evaluators was regarded 
as  providing  the  most  robust  and  impartial  overview  possible  of  the  Fund  because  it  was 
highly structured: 

•  A regular and transparent dialogue took place between the Commission services and the 

contractors; 

•  The parameters of the contract were clearly set out and respected; 
•  All data sources were assessed and presented data are clearly labelled. 

Communication between the European Commission and the external evaluator 

The  study's  progress  was  followed  by  an  Inter-service  Steering  Group  (ISG)  comprised  of 
officials  from  the  following  European  Commission  services:  DG  for  Migration  and  Home 
Affairs,  the  DG  for  Environment,  the  DG  for  Development  and  Cooperation,  the  DG  for 
Humanitarian  Aid  and  Civil  Protection,  the  DG  for  Energy,  the  DG  for  Justice  and 
Consumers,  the  DG  for  Internal Market,  Industry,  Entrepreneurship  and Small  and Medium 
Enterprises,  the  DG  for  Budget,  the  Secretariat-General,  the  Joint  Research  Centre,  the 
European Anti-Fraud Office, as well as the external evaluators with their partners and experts. 
Regular meetings took place between the contractors and the ISG and structured feedback (in 
both  directions) was provided  throughout  the contract. This  two-way dialogue was enriched 
by  the  active  participation  in  the  ISG  of  policy  and  implementation  units  and  shadowed  by 
horizontal units and the Secretariat-General. 

Evaluation stages 

A structuring feature of the external evaluation was the segmentation of the tasks into clearly 
defined stages which were closely observed by all parties. These stages had been determined 
in the Terms of Reference. Adherence to the Terms of Reference made the study itself more 
efficient and transparent. 

1.  Inception and refinement of the methodological approach: The overall purpose of the 
Inception Phase was to lay the groundwork and further elaborate and validate the method 
of approach to this evaluation. 

2.  Data collection and analysis 

3.  Formulation of conclusions and recommendations: This phase aimed to bring together 

the data collected and to formulate findings, conclusions and recommendations. 

63 

 
 
 
 
 
 
 
 
4.  Revision  and  finalisation:  The  aim  of  the  final  phase  was  to  assure  the  quality  of  the 
deliverables  through  revision  and  finalisation  based  on  the  comments  of  the  Steering 
Group.  

Data sources 

1.  Primary data 

  Three online surveys: 

-  Coordinating organisations (95 respondents) 
-  Partner organisations (110 respondents) and 
-  Project participants (285 respondents). 

 

 Stakeholder consultations: 

-  Scoping interviews with Commission officials (10); 
-  EU agencies and offices (8); 
-  Commission officials responsible for other EU programmes (3); 
-  European and international organisations (1); 
-  Representatives of monopoly bodies (3), and; 
-  Member State representatives (16). 

  Thematic case studies (8): 

-  Police cooperation – Germany; 
-  Economic and financial crime – Italy; 
-  Cybercrime - Italy, Ireland and the UK; 
-  Chemical, biological, radiological or nuclear materials and explosives 

(CBRNE) – Spain; 

-  Drugs and firearms trafficking – UK; 
-  Radicalisation – the Netherlands; 
-  Forensics – Germany, and; 
-  Organised Crime - Italy 

  Analysis of public consultation responses (3 responses) and 
  Expert workshops (2) and expert input. 

2.  Secondary Data 

  An extensive quantitative analysis of the characteristics of all ISEC projects; 
 

In-depth analysis of a sample of ISEC projects (151 projects representing 27% 
of all 568 finalised projects); 

  Literature review of EU policy baseline and context, 
  Literature review of national baseline and context. 

Methodological limitations 

A  number  of  methodological  obstacles  were  encountered  throughout  the  evaluation  which 
affected, for a number of evaluation questions, the extent to which findings and conclusions 
could be formulated. 

The lack of a baseline 

64 

 
 
 
 
 
 
 
The first key obstacle encountered in the evaluation was the fact that establishing a baseline 
for measuring either organised crime or the volume of crime levels across the EU is difficult. 
In  terms  of  organised  crime,  there  are  no  established  methodologies  for  measuring  in  a 
comprehensive manner most illegal markets or activities covered by actions funded via ISEC. 
Similarly,  the  statistical  information  gathered  on  the  volume  of  crime  at  EU  level  covers 
solely  a  few  types  of  crimes  (e.g.  homicides,  car-theft  and  robberies),  and  does  not  cover 
unreported  crime.  It  has  also  proven  very  difficult  to  make  EU-wide  comparisons  due  to 
differences in criminal justice systems. These difficulties made it problematic and indeed not 
possible to establish a common EU baseline. 

Furthermore,  an  inherent  difficulty  when  measuring  the  extent  to  which  ISEC  helped  to 
improve the management of crime  – and thus to establishing causal relations between ISEC 
projects and reported impacts – was the lack of scientific consensus as to the range of factors 
that  influence  crime.  The  range  of  factors  known  to  influence  crime  levels  is  wide  and 
includes: social and economic factors (such as (youth) unemployment rates), law-enforcement 
policies  (e.g.  number  of  police  officers  or  CCTV  coverage),  and  criminal  justice  policies 
(including changes in imprisonment rates), taxation policies (e.g. when related to smuggling 
of excisable goods), etc. Attributing specific level of impact even of these established factors 
to changes in crime levels has not been done so far by academics. Therefore, measuring the 
overall impact of specific funding instruments, including ISEC, on general or organised crime 
levels  remains  methodologically  challenging.  It  has  not  been  possible  to  establish  direct 
causal links between the ISEC Programme's intervention or identified needs on the one hand 
and  observed  impacts  or  results  on  the  other  hand;  therefore,  effectiveness  has  been  mostly 
evaluated  in  terms  of  outputs  or  perceived  impacts  or  results–  which  were  more  easily 
measured and compared across MSs – rather than actual impacts or results. 

Time lag between Programme implementation and the Ex Post Evaluation 

There was a time lag between the implementation of the Programme (2007-2013) and the ex 
post  evaluation  (August  2016-August  2017).  This  meant  that  the  relevant  beneficiaries  and 
national  and  EU  authorities  and  experts  were  often  no  longer  working  in  the  same  position 
and  were  thus  difficult  to  reach,  which  is  reflected  in  a  low  response  rate  to  some  of  the 
consultations  (see  Annex  3).  The  external  evaluator  took  some  initial  measures  to  mitigate 
these difficulties, such as sending several email reminders to the persons surveyed, searching 
for alternative contacts when possible, raising the individual  partner organisation interviews 
to two rather than one, and extending the deadline for the online survey carried out as part of 
the case studied. Overall, all evaluation questions have been answered to some extent, thanks 
to the use of triangulation of data where possible. Where the evidence base has been deemed 
to be too low to be reliable, this has been indicated.  

Monitoring, accessibility of data and consistency of project data 

Issues  have  been  noted  with  regard  to  the  monitoring  of  project  progress  and  final  results 
which  hindered  the  assessment  of  effectiveness,  efficiency  and  EU  added  value.  Especially 
given the relative newness of the Programme and the lack of capacity at the time within the 

65 

 
 
 
 
Commission  to  closely  follow  projects  from  start  to  end,  a  dedicated  technical  assistance 
budget  could  have  supported  a  monitoring  system  to  collect  and  analyse  data  on  (financial) 
progress, outputs and results. The effect of these limitations on effectiveness, efficiency and 
EU added value is further explained in sections 9.2 and 9.5. 

While relevant information could be collected for all evaluation questions, a full overview of 
final  financial  data  and  project  data,  especially  with  regard  to  a  final  absorption  rate122  was 
not  consistently  available  across  all  projects,  given  that  data  on  final  payments  was  not 
available  for  projects  which  were  not  yet  complete123.  In  addition,  data  provided  by  the 
Commission  and  the  data  drawn  from  other  sources  were  not  always  consistent.  This  has 
limited the overall findings of the evaluation to some extent. Where possible, estimates have 
been  used  to  calculate  absorption  rates  and  implementation  trends.  However,  the  figures 
relating in particular to the last programming year should be used with caution. Given that no 
estimates were available for the JRC data, no absorption rate has been calculated for the JRC 
element of the procurement sector. 

Another limitation which affects the assessment of effectiveness is that the outputs/results of 
the  projects  were  not  centrally  collected  and  the  final  assessments  of  projects  by  the 
Commission were not carried out comprehensively, often consisting only of a brief paragraph 
on the overall project,  without elaborating on particular elements or  activities of the  project 
and  their  potential  future  use.  In  addition,  no  peer  review  of  the  products/outputs  was 
undertaken.  In  addition,  results  could  not  be  measured  due  to  the  lack  of  ex  ante  targets.  It 
was not feasible to compare the "cost of projects'" under ISEC to other similar projects due to 
the  absence  of  comparable  projects.  This  hindered  the  assessment  of  efficiency  and 
effectiveness to a large extent, as it was difficult to compare project costs and to measure the 
extent to which the same objectives were met by the different types of projects in the various 
areas of organised crime addressed by ISEC. 

To  mitigate  these  limitations,  the  assessment  of  effectiveness  and  efficiency  had  to  rely 
largely on the qualitative data collected through the stakeholder consultations. This, combined 
with the absence of a baseline, proved to be a limitation in terms of methodology. 

122  The absorption rate is defined as a relation of funding spent to funding committed and it is expressed as a 

percentage. 

123  The final calculations on absorption rates include all projects up to 30/03/2017. 

66 

 
 
 
 
 
 
                                                            
ANNEX 4 - LIST OF EVALUATION QUESTIONS 

Relevance 

EQ1.  To  what  extent  did  the  objectives  and  the  actual  results  of  the  ISEC  programme 
correspond to the needs related to the prevention of and fight against crime? 

Effectiveness  

EQ2.  Did  the  ISEC  programme  contribute  to  better  security  for  citizens  and  if  so,  to  what 
extent?  To  what  extent  did  the  ISEC  programme  contribute  in  the  areas  of  preventing  and 
combating  crime,  organised  or  otherwise,  in  particular  terrorism,  trafficking  in  persons  and 
offences  against  children,  illicit  drug  trafficking  and  illicit  arms  trafficking,  corruption  and 
fraud? 

EQ3. To what extent did the ISEC programme contribute to the development of the policies 
of the Union and of the Community? 

EQ4. To what extent did the ISEC programme contribute to promotion and development of 
horizontal  methods  and  tools  necessary  for  strategically  preventing  and  fighting  crime  and 
guaranteeing security and public order, such as the work carried out in the European Union 
Crime  Prevention  Network,  public-private  partnerships,  best  practices  in  crime  prevention, 
comparable  statistics,  applied  criminology  and  an  enhanced  approach  towards  young 
offenders? 

EQ5. To what extent did the ISEC programme contribute to promoting and development of 
coordination, cooperation and mutual understanding among law enforcement agencies, other 
national  authorities  and  related  Union  bodies  in  respect  of  the  priorities  identified  by  the 
Council, in particular as set out by the Europol's Organised Crime Threat Assessment? 

EQ6. To what extent did the ISEC programme contribute to promoting and development of 
best practices for the protection of and support to witnesses? 

EQ7. To what extent did the ISEC programme contribute to promoting and development of 
best practices for the protection of crime victims? 

Efficiency  

EQ8. To what extent were the results of the ISEC programme achieved at a reasonable cost in 
terms  of  financial  and  human  resources  deployed?  What  kinds  of  initiatives  or  approaches 
were  adopted  to  simplify  access  to  and  implementation  of  the  actions  funded  by  the 
programmes? 

Coherence 

EQ9. To what extent was the ISEC programme coherent with actions related to the prevention 
of  and  fights  against  crime  supported  by  other  EU  financial  instruments  and/or  by  national 
resources of the Member States, and to what extent is it possible to identify synergies between 
the  results  of  actions  funded  under  the  ISEC  Programme  and  the  results  of  similar  actions 

67 

 
 
supported under other EU Programmes (such as the Hercule Programmes), that were carried 
out by different organisational entities in the Member States and/or third countries? 

EU Added Value 

EQ10.  To  what  extent  would  the  beneficiaries under  the ISEC  programme  (Member  States, 
Universities, Institutes, Associations, etc.) be able to carry out the activities necessary for the 
implementation  of  the  EU  policies  in  the  field  of  the  prevention  of  and  fight  against  crime 
without the support of the ISEC programme? 

68 

 
 
 
 
 
ANNEX 5 - LIST OF ABBREVIATIONS AND COUNTRY CODES 

AFIS 
AGIS 
AOP 
AP 
AWP 
CBRN 
CIPS 

CEPOL 
Daphne II 
DG HOME 
DG TAXUD 
DG JUSTICE 
EMCDDA 
EUCPN 
EC3 
EUROSUR 
FPA 

Automated Fingerprint Identification System 
Judicial Cooperation in Criminal Matters  
Action-Oriented Paper 
Annual Programme 
Annual Work Programme 
Chemical, Biological, Radiological and Nuclear materials 
"Prevention,  Preparedness  and  Consequence  Management  of  Terrorism  and 
other Security-Related Risks 
European Union Agency for Law Enforcement Training 
Fight Against Violence 
Directorate-General Migration and Home Affairs 
Directorate-General Taxation and Customs Union 
Directorate-General for Justice and Consumers 
European Monitoring Centre for Drugs and Drug Addiction 
European Crime Prevention Network 
European Cybercrime Centre 
European Border Surveillance System 
Framework Partnership Agreements 

FRA 

Frontex 

Hercule II 
ICT 
ISEC 
ISF 
ITech 
JPEN 
JRC 
MAP 
MCT 
MFF 
MTIC 
NCC 
NER 
NPS 
OCG 
PNR 
SSL 
VAT 

Fundamental Rights Agency 

European  Border  and  Coast  Guard  Agency  as  of  16/10/2016,  previously 
European  Agency  for  the  Management  of  Operational  Cooperation  at  the 
External Borders of the Member States of the European Union 
Anti-fraud programme 
Information and Communications Technology 
"Prevention and Fight against Crime" 2007-2013 programme 
Internal Security Fund  
Information Technology 
Criminal Justice Support Programme 
Joint Research Council 
Multiannual Programme 
Mobile Competence Teams 
Multiannual Financial Framework 
Missing Trader Intra Community 
National Coordination Centre 
National Evaluation Report  
New Psychoactive Substances 
Organised Criminal Groups 
Passenger Name Record 
Framework Programme on Security and Safeguarding Liberties 2007-2013 
Value Added Tax 

69 

 
 
 
 
 
 
 
 
 
List of country codes 

AT 
BE 
BG 
CH 
CY 
CZ 
DE 
DK 
EE 
EL 
ES 
FI 
FR 
HR 
HU 
IE 
IS 
IT 
LT 
LU 
LV 
MT 
NL 
NO 
PL 
PT 
RO 
SE 
SI 
SK 
UK 

Austria  
Belgium 
Bulgaria 
Switzerland 
Cyprus 
Czech Republic 
Germany 
Denmark 
Estonia 
Greece 
Spain 
Finland 
France 
Croatia 
Hungary 
Ireland 
Iceland 
Italy 
Lithuania 
Luxembourg 
Latvia 
Malta 
Netherlands 
Norway  
Poland 
Portugal 
Romania 
Sweden 
Slovenia 
Slovakia 
United Kingdom  

70 

 
 
 
 
 
ANNEX 6- INTERVENTION LOGIC OF THE ISEC PROGRAMME 

71