Evaluation of the Hercule II Programme : final report.
Date: 2015-06-02
Institutions: European Anti-Fraud Office
Subjects: EU financing, EU programme, financial control, fraud, fraud against the EU, industrial counterfeiting

Evaluation of the  
Hercule II Programme 

Final report 

mmmll 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Europe Direct is a service to help you find answers  
to your questions about the European Union. 
Freephone number (*): 
00 800 6 7 8 9 10 11 

(*)  

  The information given is free, as are most calls  
  (though some operators, phone boxes or hotels may charge you). 

PREPARED BY  

Ramboll Management Consulting A/S  
Hanna-Maija Kuhn,  
Mathilde Heegaard,  
Anders Kragh Bingen,  
Ida Maegaard Nielsen 

FOR THE EUROPEAN COMMISSION  

European Anti-Fraud Office (OLAF)  
Directorate D - Policy 
Unit D1 - Policy Development & Hercule 
Contact:  
E-mail: OLAF-FMB-HERCULE@EC.europa.eu 
European Commission  
B-1049 Brussels  

LEGAL NOTICE  
This document has been prepared for the European Commission however it reflects the views only of the 
authors, and the Commission cannot be held responsible for any use which may be made of the information 
contained therein.  

More information on the European Union is available on the Internet (http://www.europa.eu).  
Luxembourg: Publications Office of the European Union, 2015  
ISBN 978-92-79-48241-0  
doi: 10.2784/287250  

© European Union, 2015  

December 2014    ii 

  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Table of Contents 

3. 

2. 

1.1 
1.2 

2.1 
2.2 
2.3 

Table of Contents .............................................................................................. iii 
Executive summary ........................................................................................... vi 
Introduction ....................................................................................... 1 
1. 
Methodology used ............................................................................... 2 
Contents of this report ........................................................................ 3 
Context analysis ................................................................................. 4 
The policy context .............................................................................. 4 
Objectives ......................................................................................... 4 
Activities carried out in the context of Hercule II .................................... 5 
Evaluation findings ............................................................................12 
Effectiveness .....................................................................................12 
Efficiency ..........................................................................................26 
Implementation .................................................................................31 
Utility ...............................................................................................36 
Acceptability .....................................................................................38 
EU added value .................................................................................39 
Complementarity ...............................................................................42 
Sustainability ....................................................................................44 
Conclusions ......................................................................................46 

3.1. 
3.2. 
3.3. 
3.4. 
3.5. 
3.6. 
3.7. 
3.8. 

4. 

Table of figures 

Figure 1: Overview of the sectors and activities of Hercule II ................................................ 5 
Figure 2: Budget per sector in 2007-2013 as specified in the annual work programmes ........... 6 
Figure 3: Total no. of grants and contracts funded under Hercule II, per sector ...................... 7 
Figure 4: Budget allocated to Technical Assistance and IT Support (EUR) and committed ......... 8 
Figure 5: Budget allocated to training, seminars and conferences (EUR) and committed ......... 10 
Figure 6: Budget allocated to IT Support (EUR) and committed ........................................... 11 
Figure 7: Volume of funding (grant or contract) in each sector per Member State. ................. 24 
Figure 8: To what extent  do you agree that  the  desired project  results have been achieved at 
reasonable costs? (N= 59) .............................................................................................. 26 
Figure 9: Did the project(s) start on time (N= 59) and did you receive the payment according to 
the payment schedule set in the grant agreement? (N= 57) ................................................ 29 
Figure  10:  To  what  extent  do  you  agree  that  the  grant  received  from  OLAF  was  sufficient  in 
order for the project to reach its objectives? (N= 59) ......................................................... 30 
Figure 11: Budgeted vs. total committed amount (EUR)...................................................... 32 
Figure  12:  To  what  extent  do  you  agree  with  the  following  statement?  –  The  Hercule  II 
Programme’s application procedure has been straightforward and easy to complete (N= 59) .. 33 
Figure  13:  To  what  extent  do  you  agree  with  the  following  statement?  –  The  reporting 
requirements during the project’s implementation have been easy to meet (N= 59) ............... 34 
Figure 14: To what  extent do you agree with  the  following statement?  –  The  support  received 
by OLAF during the project implementation has contributed to the project reaching the desired 
results (N= 59) .............................................................................................................. 35 
Figure  15:  To  what  extent  do  you  agree  with  the  following  statement?  -  The  results  of  the 
implemented project match the needs and problems experienced in our country (N= 68) ....... 37 
Figure  16:  Taking  into  account  the  costs  of  participation  and  the  benefits  of  the  project,  how 
useful did you find the participation? (N= 68) .................................................................... 38 
Figure  17:  Percentage  of  beneficiaries  who  agree  that  the  same  objectives  could  have  been 
achieved by means of national initiatives (N=68) ............................................................... 39 

December 2014    iii 

  
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

List of boxes 

Box 1: Conclusion on enhanced transnational and multidisciplinary cooperation ..................... 13 
Box 2: Conclusion on building networks throughout the Member States, acceding countries and 
candidate countries ........................................................................................................ 15 
Box  3:  Conclusion  on  the  facilitation  of  the  exchange  of  information,  experience  and  best 
practices ....................................................................................................................... 16 
Box 4: Conclusion on the provision of technical and operational support for the law enforcement 
authorities .................................................................................................................... 17 
Box  5:  Conclusion  on  facilitated  data  access  and  analysis  through  specific  databases  and  IT 
tools ............................................................................................................................. 18 
Box  6:  Conclusion  on  improved  support  for  investigations,  monitoring  and  intelligence  work 
through IT tools ............................................................................................................. 19 
Box  7:  Conclusions  on  intensified  measures  against  smuggling  and  counterfeiting  of  cigarettes
 ................................................................................................................................... 21 
Box 8: Conclusions on the differences between the effectiveness of the three sectors ............. 22 
Box 9: Conclusions on geographical balance ...................................................................... 25 
Box 10: Conclusions on efficiency .................................................................................... 30 
Box 11: Conclusions on implementation ............................................................................ 36 
Box 12: Conclusions on utility .......................................................................................... 37 
Box 13: Conclusions on acceptability ................................................................................ 38 
Box 14 Conclusions on EU added value ............................................................................. 41 
Box 15 Conclusions on complementarity ........................................................................... 43 
Box 16: Conclusions on sustainability ............................................................................... 45 

List of tables 
Table 1: Sectorial effectiveness in reaching Programme objectives ....................................... 21 

Annexes 

Annex A:  
Annex B:  
Annex C:  
Annex D:  

Annex E:  

Methodology (see separate document) 
Contribution stories (see separate document) 
Examples of concrete quantitative results (see separate document) 
Beneficiary  and  participation  survey,  observations,  all  (see  separate 
document) 
List of references 

December 2014    iv 

  
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

List of abbreviations 

AFCOS 

AGIS 

AMT 

AOSD 

BCP 
COCOLAF 

CTI 

DG 
DG HOME 

DG TAXUD 
EEA 

EFTA 

ELA 

Eucrim 

EUR 
IA 
OLAF 

EU 

IT 

Anti-Fraud Coordination Service 

Framework  programme  on  police  and  judicial  co-
operation in criminal matters 
Automated Monitoring Tool 

Authorising Officer by Sub-Delegation 

Border-Crossing Point 
Advisory  Committee  for  the  Coordination  of  Fraud 
Prevention 
Chinese Trade Information 

Directorate-General 
Directorate-General Home Affairs 

Directorate-General Taxation and Customs Union 
European Economic Area 

The European Free Trade Association 

Associations  for  European  Criminal  Law  and  for  the 
Protection of the EU Financial Interests 
The European Criminal Law Associations’ Forum 

Euro 
Impact Assessment 
European Anti-Fraud Office 

European Union 

Information Technologies 

IMSI 
ISEC Programme  

International Mobile Subscriber Identity 
Programme Prevention of and Fight against Crime  

JRC 

MS 
NGO 

N 
TFEU 
TOC 
VAT 

Joint Research Centre 

Member State 
Non-governmental organisation 

Number 
The Treaty on the Functioning of the European Union 
Theory of Change 
Value Added tax 

December 2014    v 

  
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Executive summary 

The present report contains the results of the independent evaluation of the Hercule II 
Programme (2007 to 2013) undertaken by Ramboll Management Consulting on behalf 
of OLAF between March and December 2014.  

The overall objectives of the evaluation were three-fold:  

1.  to  assess  the  extent  to  which  the  objectives  of  the  Programme  have  been 

achieved (the effectiveness);  

2.  to assess the efficiency of the Programme’s implementation to provide basis for 

the Commission’s improvement of the Programme’s overall performance; and  

3.  to  assess  the  EU  added  value  of  the  Programme’s  activities  and  the 

sustainability of its results.  

The Hercule II  Programme promoted activities designed to strengthen  EU action with 
regard to the prevention and combating of fraud affecting the financial interests of the 
Union,  including  the  fight  against  cigarette  smuggling  and  counterfeiting.  The 
relevance of the programme was not covered, as this has already been addressed and 
assessed  in  the  impact  assessment  accompanying  the  proposal  for  the  Hercule  II 
Programme  and  reiterated  in  the  impact  assessment  accompanying  the  proposal  for 
the Hercule III Programme. 

The  Programme  was  the  successor  of  the  first  Hercule  Programme,  which  was 
extended  by  Decision  878/2007/EC  for  the  period  2007-13  as  Hercule  II1.  The 
Programme was implemented through three sectors of activities: Technical Assistance; 
IT  Support;  and  Training,  Seminars  and  Conferences  (which  again  was  divided 
between anti-fraud training and a legal part). The total budget for the Programme was 
EUR  98.525.000  million.  The  Programme  has  been  further  continued  as  Hercule  III 
from  2014,  and  is  implemented  by  the  Policy  Directorate  of  the  European  Anti-Fraud 
Office (OLAF).                                                                                                                                                                                                                                                      

On 26 February 2014, the European Parliament and the Council of the European Union 
adopted Regulation (EU) No 250/2014 establishing a programme4 to promote activities 
in the field of the protection of the financial interests of the European Union (Hercule 
III  Programme)  and  repealing  Decision  No  804/2004/EC5.  This  programme  entered 
into force on 21 March 2014. 

The  Programme  ran  for  a  period  of  seven  years  to  align its  duration  with  that  of  the 
multiannual  financial  framework  laid  down  in  Council  Regulation  (EU,  Euratom)  No 
1311/2013. 
Overall,  the  evaluation  confirms  that  the  Hercule  II  Programme  has  reached  the 
objectives which it set out to achieve, and thus the Programme has delivered 
its intended impact, namely protecting the Union’s financial interests.  

The  Hercule  II  Programme  delivers  complementary  results  within  the  three  different 
sectors. Although it was not possible to judge which sector was more effective, it was 
clear that each sector has been very effective in reaching the expected outcomes, and 
the evaluation therefore recommends maintaining the structure of the Programme and 
continuing the funding of all three sectors.  

1 Decision 878/2007/EC of the European Parliament and of the Council of 23 July 2007 (OJ L193 of 25 July 
2007 (Hercule II programme)). 

December 2014    vi 

  
 
 
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

focus  on  cigarette  smuggling  and 
The  Hercule  II  Programme´s 
counterfeiting (compared to Hercule I) has led to an increased effort articulated by the 
multiple  activities  providing  funding  for  purchase  of  technical  equipment,  but  also 
through the views of the beneficiaries, as 75% of the survey respondents agreed that 
the Programme had been successful in this regard. 

increased 

The  evidence  collected  shows  that  Technical  Assistance,  through  the  provision  of 
equipment,  has  been  highly  effective  in  strengthening  technical  and  operational 
support  for  the  law  enforcement  and  customs  authorities.  It  was  generally  unlikely 
that  equipment  would  have  been  purchased  without  the  Programme’s  co-financing, 
and overall this increased the flexibility and capacity of the authorities.   

With  respect  to  Trainings,  Seminars  and  Conferences,  the  evaluation  concludes 
that  the  Hercule  II  Programme  enhanced  transnational  and  multidisciplinary 
cooperation, however primarily between Member States, and that the Programme has 
effectively  contributed  to  strengthening  networks  and  exchange  of  information.  The 
evaluation  shows  that  trainings,  seminars  and  conferences  with  a  legal  focus  where 
the most effective in this respect, hence Training, Seminars and Conferences were still 
effective,  but  to  a  lesser  extent,  when  it  came  to  developing  a  unified  level  of  skills 
and improving the readiness in the Member States to detect smuggled and counterfeit 
products.  In  regards  to  forming  networks,  both  formal  and  informal  networks  were 
established,  supporting  faster  information  exchange,  increased  access  to  information 
and  improved  response  to  fraud.  The  evaluation  showed  that  the  exchange  of 
information and best practices which took place during the events where assessed to 
be the most successful. The evaluation also shows that the  Programme has improved 
cooperation between practitioners and academics at the level of Member States. Based 
on  the  findings  concerning  Training,  Seminars  and  Conferences,  the  evaluation 
recommends  that  the  Commission  ensures  through  its  application  and  selection 
procedures that the Training, Seminars and Conferences provides a playing level field 
for networking through more targeted themes and participants.       

Finally, the IT Support facilitated access to data and analysis which was indicated to 
have  improved  investigations,  monitoring  and  intelligence  work  and  in  particular  the 
development  of  the  Automatic  Monitoring  Tool  (AMT)  had  significantly  improved  the 
monitoring and intelligence work of the national law enforcement authorities to detect 
undervaluation in imported goods. Specifically concerning facilitation of access to data 
bases in the Member States the  evaluation shows that the  IT Support has effectively 
contributed to this in some Member States, whilst it has only contributed to a limited 
extent in others. Hence, the evaluation also shows that the potential of the IT Support 
may not be fully exploited yet.  

including 

looked  at 

the  efficiency, 

The  evaluation  also 
the  Programme’s 
implementation,  and  concludes  that  there  are  indications  that  the  Hercule  II 
Programme  has  reached  its  desired  effects  at  reasonable  costs.  Although  it  is  not 
possible  to  conclude  whether  the  Programme  has  led  to  overall  efficiency  gains,  the 
overall  management  of  the  Programme  by  OLAF  is  deemed  efficient  i.e.  in  terms  of 
making the resources available to the beneficiaries in due time, quantity and quality. 
Nevertheless,  the  evaluation  showed  that  efficiency  gains  may  be  possible if  Member 
States  shared  best  practices  with  respect  to  purchasing  and  procurement  of 
equipment. 

The  evaluation  shows  the  importance  of  additional  monitoring  requirements  in  line 
with  the  current  detailed  targeted  reporting  on  results  achieved  with  the  technical 
equipment, and thereby improves  OLAF´s ability to monitor the results  achieved and 
the sustainability of the activities funded.  

December 2014    vii 

  
 
 
 
Evaluation of the Hercule II Programme 

The EU added value of the Hercule II Programme´s activities has been evaluated and 
the  activities  are  found  to  be  successful  in  providing  EU  added  value.  Generally  and 
across the different types  of actions, stakeholders agree that the objectives achieved 
by means of Hercule funding could not, or to a lesser extent, have been achieved with 
national  or  regional  means.  This  applies  to  all  three  sectors  of  the  Programme, 
although  Training,  Seminars  and  Conferences  are  assessed  as  particularly 
instrumental  in  facilitating  international  cooperation  and  establishing  networks  which 
would not otherwise have been formed. Meanwhile, the tools developed and provided 
under the IT Support sector and the Technical Assistance actions provide more direct 
value  to  the  investigations  of  the  national  customs  and  law  enforcement  authorities 
and,  as  such,  should  contribute  to  the  protection  of  the  financial  interests  of  the 
Union. 

The evaluation showed  that the Hercule II  Programme indeed was complementary to 
other programmes in the field. According to the stakeholders the  Programme is seen 
to  have  a  unique  profile  in  addressing  the  protection  of  the  financial  interests  of  the 
Union, which sets it apart from other programmes offered by the Commission i.e. the 
Customs,  Fiscalis  or  DG  HOME  programmes.  Technical  Assistance  was  particularly 
highlighted  as  a  feature  that  is  not  offered  by  other  EU  initiatives.  The  evaluation 
assesses  that  more  could  be  done  to  enhance  cooperation  and  communication 
between the Commission services.  

Finally,  the  Hercule  II  Programme  showed  that  sustainability  has  been  achieved. 
Overall,  the  beneficiaries  assess  that  the  purchased  equipment  and  its  use,  are 
sustainable.  In  relation  to  Training,  Seminars  and  Conferences,  the  participants  are 
generally confident that the lessons learned can be put into practice.  

December 2014    viii 

  
 
 
 
 
 
Evaluation of the Hercule II Programme 

1. Introduction 

The evaluation is undertaken in compliance with the provisions set out in Article 7 of 
the 2007 Decision establishing the Hercule II Programme2, according to which “by 31 
December 2014 the Commission (OLAF) shall present to the European Parliament and 
to the Council a report on the achievement of the objectives of the programme”. 

Ramboll  Management  Consulting  (Ramboll)  was  contracted  by  OLAF  to  carry  out  the 
evaluation for the Commission in the period from March until December 2014.  

As  specified  in  the  terms  of  reference,  the  objectives  of  the  evaluation  were  three-
fold:  

1.  to  assess  the  extent  to  which  the  objectives  of  the  programme  have  been 

achieved (the effectiveness);  

2.  to assess the efficiency of the programme’s implementation to provide a basis 
for  the  Commission’s  improvement  of  the  programme’s  overall  performance; 
and  

3.  to  assess  the  EU  added  value  of  the  programme’s  activities  and  the 

sustainability of its results.  

The relevance of the programme was not covered, as this has already been addressed 
and assessed in the impact assessment accompanying the proposal for the Hercule III 
Programme.3  During  meetings  with  the  Steering  Group,  it  became  clear  that  aspects 
related  to  implementation  and  complementarity  with  other  EU  programmes  were 
important for the Commission, which is why evaluation questions were added to  also 
cover these criteria.  

The evaluation covers the activities of the Hercule II Programme during the totality of 
its implementation, i.e. 2007-2013. 

On 26 February 2014, the European Parliament and the Council of the European Union 
adopted  Regulation  (EU)  No  250/20143  establishing  a  programme4  to  promote 
activities in the field of the protection of the financial interests of the European Union 
(Hercule  III  Programme)  and  repealing  Decision  No  804/2004/EC5.  This  programme 
entered  into  force  on  21  March  2014.  The  programme's  proposal  has  been 
accompanied  by  an  IA  that  also  addresses  some  of  the  issues  discussed  in  this 
evaluation. 

The Programme runs for a period of seven years to align its duration with that of the 
multiannual  financial  framework  laid  down  in  Council  Regulation  (EU,  Euratom)  No 
1311/2013.4 

2 Decision 878/2007/EC of the European Parliament and of the Council of 23 July 2007, OJ L193 of 25 July 
2007 (Hercule II Programme). 
3 Commission staff working paper: Impact assessment accompanying document to the 
Proposal for  a  Regulation  of  the  European  Parliament  and of  the  Council  on  the  Hercule  III  Programme  to 
promote activities in the field of the protection of the European Union’s financial interests. SEC (2011) 1610 
final, 19.12.2011. 
4  Council  Regulation  (EU,  Euratom)  No  1311/2013  of  2  December  2013  laying  down  the  multiannual 
financial framework for the years 2014-2020 (OJ L 347, 20.12.2013, p. 884). 

December 2014    1 

  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

1.1  Methodology used 

The  approach  to  the  evaluation  is  based  on  the  method  known  as  the  ‘contribution 
analysis’, which creates a framework for the evaluation, where information is gathered 
to  generate  hypotheses  about  links  between  for  example  activities,  outputs  and 
outcomes  (results),  and  then  testing  these  hypotheses  against  new  data.  This 
information is gathered based on the programme’s theory of change which is used to 
illustrate  how  the  Programme  activities  are  ultimately  intended  to  contribute  to  the 
long-term  impact  via  the  generation  of  outputs  and  outcomes.  The  theory  of  change 
for  the  programme  can  be  found  in  Annex  A.  In  practice,  the  contribution  analysis 
means that the more  ‘traditional’ data collection methods  of  surveys and stakeholder 
interviews  were  supplemented  by  20  contribution  case  studies5.  Based  on  these  case 
studies, contribution stories were developed, describing in detail the linkages between 
the  different  activities,  outputs,  outcomes  and  results.  A  summary  of  these 
contribution stories can be found in Annex B. 

All  in  all,  four  online-based  surveys  and  38  stakeholder  interviews  were  carried  out 
within  the  framework  of  the  evaluation,  in  addition  to  20  case  studies.  These  are 
described in more detail in Annex A. 

The  evaluation  is  based  on  an  evaluation  matrix,  where  the  evaluation  questions  are 
split into more specific sub-questions. An evaluation matrix is a table showing how the 
evaluation  questions  will  be  answered  based  on  data  sources.  The  evaluation  matrix 
can be found in Annex A. In this main body of the report, we provide answers to the 
evaluation  questions  based  on  carefully  selected  indicators,  evidence  and  judgement 
criteria,  which  are  analysed  through  a  set  of  analytical  strategies:  Descriptive, 
qualitative  and  contribution  analyses.  By  triangulating  the  findings  from  the  three 
strategies,  the  evaluation  combines  the  overview  and  broad  applicability  of  the 
descriptive  analysis  with  the  qualitative  analysis’  focus  on  the  programme  (as  a 
whole), and, finally, benefits from the detail-rich, project-specific evidence generated 
by the contribution analysis which takes into account the context of delivery. Not only 
does  this  ensure  a  clear  link  between  evidence  and  conclusions,  but  it  allows  us  to 
cross-check  the  findings  from  each  subset  of  analysis,  thereby  increasing  the 
credibility and validity of the evaluation findings. 

While the evaluation methodology is explained in more detail in Annex A, a number of 
considerations  are  relevant  to  keep  in  mind  while  examining  the  results  of  the 
evaluation. 

-  The  survey  of  participants  in  Training,  Seminars  and  Conferences  (hereinafter 
participant survey) had a response rate of 39% (574 out of 1464 responded to 
the  survey).  The  high  number  of  responses  should  be  assessed  carefully  as 
almost one in four respondents had participated in the digital forensics training, 
making this type of training, seminars and conferences overrepresented among 
the respondents. 

-  Whereas  the  survey  of  the  beneficiaries  of  the  Hercule  II  Programme 
(hereinafter  beneficiary  survey)  has  a  high  response  rate  (65%),  the  sample 
size is limited, as the survey was distributed to the most recent beneficiaries as 
this  was  initially  thought  to  be  sufficient.  It  became,  however,  apparent  that 
some grants and contracts from the early years of Hercule II (2007-2010) were 
not  well  covered  by  the  survey.  Moreover,  it  was  difficult  to  interview 
beneficiaries of the programme and their participants. 

-  The  share  of  respondents  per  Member  State  in  the  beneficiary  survey  is  not 
representative  of  the  division  of  funding  between  the  Member  States.  This 

5 Case study No. 2 on X-ray equipment for the Finnish customs was not carried out due to lack of data. 

December 2014    2 

  
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

means  that  the  survey  responses  are  not  necessarily  representative  of  the 
views of the Member States to the same degree as the Member States receive 
co-funding. 

This being said, as the survey is only one source  of data used in the evaluation, and 
the  survey  results  are  always  backed  by  secondary  data,  stakeholder  interviews 
and/or  case  studies,  these  limitations  are  not  likely  to  bias  the  outcome  of  the 
analysis. 

1.2  Contents of this report 

This report is structured as follows: 

Chapter  2  provides  an  analysis  of  the  context  in  which  the  Hercule  II  Programme  is 
carried out and implemented. 
Chapter  3  describes  the  evaluation  findings,  divided  into  the  different  evaluation 
criteria:  effectiveness,  efficiency,  implementation,  utility,  acceptability,  EU  added 
value, sustainability and complementarity. 
Chapter  4  consists  of  the  conclusions,  drawing  upon  the  main  findings  from  each 
evaluation criterion, and areas for improvement identified by the evaluation. 

December 2014    3 

  
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

2. Context analysis 

The  Hercule  II  Programme  promotes  activities  designed  to  strengthen  EU  action  with 
regard to the prevention and combating of fraud affecting the financial interests of the 
Union,  including  the  fight  against  cigarette  smuggling  and  counterfeiting.  The 
Programme  covers  the  period  from  1  January  2007  to  31  December  2013.  It  is  the 
successor  of  the  first  Hercule  Programme6,  which  was  extended  by  Decision 
878/2007/EC  for  the  period  2007-13  as  Hercule  II.  This  decision  introduced  an 
emphasis on the fight against cigarette smuggling and counterfeiting7. The Programme 
has been further  continued as  Hercule  III  from 2014, which entered into force  on 21 
March 2014.  

The  Hercule  Programme  is  implemented  by  the  Policy  Directorate  of  the  European 
Anti-Fraud Office (OLAF)8. 

2.1  The policy context 

The Treaty on the Functioning of the European Union (TFEU) determines that Member 
States  shall  coordinate  their  action  aimed  at  protecting  the  financial  interests  of  the 
Union  against  fraud  and  take  the  same  measures  to  counter  fraud  affecting  the 
financial  interests  of  the  Union  as  they  take  to  counter  fraud  affecting  their  own 
financial  interests.9  The  Convention  on  the  protection  of  the  European  Communities’ 
financial interests, which entered into force in 2002, requires that fraud affecting both 
expenditure  and  revenue  must  be  punishable  by  effective,  proportionate  and 
dissuasive  criminal  penalties  in  every  Member  State.10  A  Commission  Anti-Fraud 
Strategy11  is  in  place  to  improve  prevention,  detection  and  the  conditions  for 
investigations  of  fraud  and  to  achieve  adequate  reparation  and  deterrence,  with 
proportionate and dissuasive sanctions, and respecting the due process, especially by 
introducing anti-fraud strategies at Commission service level respecting and clarifying 
the different responsibilities of the various stakeholders.12  

2.2  Objectives 

In  the  context  of  the  overall  objective  of  countering  fraud  and  any  other  illegal 
activities  affecting  the  Union’s  financial  interests,  the  Hercule  II  was  specifically 
designed to: 

  enhance transnational and multidisciplinary cooperation between the  actors in the 
fight  against  fraud  damaging  the  Union's  financial  interests,  i.e.  the  competent 
authorities  in  the  Member  States,  the  Commission  and  the  European  Anti-Fraud 
Office (OLAF); 
reinforce  the  networks  for  exchange  of  information,  experience  and  best  practice 
between Member States, the accession countries and the candidate countries; 

 

6  Established  by  Decision  No  804/2004/EC  of  the  European  Parliament  and  of  the  Council  to  promote 
activities in the field of the protection of the Community’s financial interests (OJ L 143, 30.4.2004, p. 9–14) 
7  This  is  due  to  the  legal  obligations  of  the  Commission  stemming  from  the  Anti-Contraband  and  Anti-
Counterfeiting Agreement with Philip Morris International signed in 2004.  
8  Commission Decision of 28 April 1999  establishing  the  European  Anti-fraud  Office  (OLAF), 1999/352/EC, 
ECSC, Euratom (OJ L 136, 31.5.1999, p. 20) as amended by Commission Decision 2013/478/EU (OJ L 257, 
28.9.2013, p. 19). 
9 Article 325 (TFEU), C 115/47 
10 Council Act of 26 July 1995 drawing up the Convention on the protection of the European Communities’ 
financial interests [Official Journal C 316 of 27.11.1995]. 
11  It  should  be  noted  that  the  Anti-Fraud  Strategy  has  been  mandatory  in  each  Commission  service  since 
end 2013.  
12  Communication  from  the  Commission  to  the  European  Parliament,  the  Council,  the  European  Economic 
and Social Committee, and the Committee of the Regions and the Court of Auditors on the Commission Anti-
fraud Strategy COM/2011/376. 

December 2014    4 

  
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

  provide  operational  and  technical  support  to  law  enforcement  agencies  in  the 

Member States, particularly the customs authorities; 

  strike a geographical balance by including, if possible, all Member States, acceding 

countries and candidate countries; 

  multiply  and  intensify  the  measures  in  the  areas  identified  as  the  most  sensitive, 

particularly in the field of cigarette smuggling and counterfeiting. 

2.3  Activities carried out in the context of Hercule II 

According to Council Decision No 804/2004/EC, the activities supported by Hercule II 
pertain to three categories referred to as ‘sectors’, namely (i) Technical Assistance for 
national  and  regional  authorities,  (ii)  Training,  Seminars  and  Conferences  and  (iii)  IT 
Support13. Each sector encompasses a number of activities as summarised in  Figure 1 
pursuing  a  specific  set  of  operational  sub-objectives14  consistent  with  the  overarching 
strategy of the Programme. 

Figure 1: Overview of the sectors and activities of Hercule II15 

The allocation of funds across the different sectors is made on an annual basis through 
the annual work programmes which constitute a financing decision16.  

Hercule II (2007-2013) has a total budget of EUR 98.5 million.17 The annual reports of 
the Hercule II Programme18 as well as the mid-term evaluation of the Programme show 
the commitments per sector of activity:  

13 A detailed description is provided below. 
14 The wording is taken from SEC(2011) 1610 final. 
15  Source:  Elaboration  by  Ramboll  Management  Consulting  based  on  the  Commission  Decisions  on  the 
adoption of a financing decision for 2007-2013 within the framework of the Community action programme 
to  promote  activities  in  the  field  of  the  protection  of  the  Community's  financial  interests  (Hercule  II 
Programme). 
16 Within the meaning of Article 75(2) of Regulation (EC, Euratom) No 1605/2002. 
17 Since the start of the Hercule programme in 2004 and its continuation into the Hercule II between 2007 
and 2013, the contribution of the Union for the implementation of the programme has been financed from 
budget  line  24.0201  of  the  General  Budget  of  the  European  Union  (General  measures  to  combat  fraud  - 
Hercule II). Until 2003, support for bodies and activities in the field of fight against fraud was paid out of the 
general  budget  of  the  European  Union  under  the  lines  A03600  and  A03010  (Conferences,  congresses  and 
meetings in connection with the activities of the associations of European lawyers for the protection of the 
financial interests of the Community) and under line B5-910 (General measures to combat fraud). 
18 The term annual report is used with reference to the Annual Overview with Information on the Results of 
the  Hercule  II  Programme  (as  per  Article  7  of  Decision  878/2007  of  23  July  2007),  accompanying  the 
Annual  Report  from  the  Commission  to  the  European  Parliament  and  the  Council  on  the  Protection  of  the 
European Union's financial interests - Fight against fraud. 

December 2014    5 

  
 
 
 
 
 
 
 
 
                                           
 
Evaluation of the Hercule II Programme 

Figure  2:  Budget  per  sector  in  2007-2013  as  specified  in  the  annual  work 
programmes19 

It can be seen that the division of the budget between sectors has not been subject to 
great  deviations  over  the  years,  with  the  exception  of  anti-fraud  training  receiving 
considerably  more  funding  in  2010-2011,  with  the  funding  for  Technical  Assistance 
diminishing  equally.  The  total  annual  budget  has  also  remained  on  a  stable  level, 
ranging between EUR 13.7 and 15 million.  

In accordance with the provisions laid down in the Financial Regulation, funding under 
the Hercule II Programme can take two legal forms:  
  grants, following calls for proposals; and  
  public procurement contracts, following calls for tender. 

In the case of grants, the bodies eligible for receiving funding are listed in Article 3 of 
the  Hercule  II  Decision.  As  for  procurement  contracts,  this  legal  form  was  included 
into  the  basic  act  establishing  Hercule  II  with  the  aim  of  streamlining  all  operational 

19 Source: Elaboration by Ramboll Management Consulting based on the data provided by the Annual Work 
Programmes  2007-2013  and  amending  decisions  (annual  budget  commitments  made  by  the  Commission 
per  sector  and  including  both  grants  and  procurement)  and  cross-referenced  with  Hercule  II  mid-term 
review and Annual overviews. It is to be noted that for year 2007 a breakdown of the data on the budget 
reserved  to  the  Training  sector  was  not  available  and,  thus,  the  value  reported  includes  both  operational 
and  legal  training.  The  initial  allocation  provided  for  in  the  Annual  Work  Programme  2007  (before  being 
modified) reserved EUR 3,63 mil to the Training sector (EUR 3,25 mil to the operational part and 0,38 mil 
for the legal part). 

December 2014    6 

  
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

expenditure  related  to  general  anti-fraud  activities.20  The  awarding  of  public 
procurement  contracts  allows  for  acquiring  goods  and  services  and  for  supporting 
coordination activities with the Member States. 

The  annual  division  of  the  budget  as  well  as  the  overall  number  of  grants  and 
contracts  signed  by  OLAF  within  the  context  of  the  Hercule  II  Programme,  where  in 
total  52.8%  of  funds  were  allocated  to  grants  and  47.1%  were  allocated  to 
procurement21.  Table  1  below  shows  the  annual  number  of  grants  and  contracts 
allocated under the Hercule II Programme from 2007-2013.   

Figure 3: Total no. of grants and contracts funded under Hercule II, per sector22 

Training  etc.  - 
Anti-fraud part 
39 

Training etc. –  
legal part 
5 

IT 
Support 
9 

Total  no.  of  grants 
and contracts 
83 

Year 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

Sector 

Technical 
Assistance 
30 

31 

29 

29 

21 

52 

33 

51 

42 

35 

49 

33 

29 

TOTAL 

225 

278 

0 

0 

3 

3 

15 

11 

37 

14 

14 

11 

12 

17 

7 

84 

96 

85 

78 

85 

117 

80 

624 

Each of the sectors is described in more detail below. 

Sector A: Technical Assistance 
Technical Assistance represents the largest sector of activities in terms of budget. The 
projects  implemented  under  this  heading  are  aimed  at  providing  specific  knowledge, 
equipment  and  information  technology  tools  to  national  authorities,  at  improving  the 
quality  of  the  operational  technical  support,  at  enhancing  staff  exchange  and  at 
supporting joint operations.  

The  projects  falling  within  the  scope  of  this  sector  can  be  broadly  grouped  into  two 
strands:  the  ‘investigation  support’  strand  generally  concerned  with  the  fight  against 
fraud  and  corruption;  and  the  ‘cigarettes’  strand  aimed  at  strengthening  the 
applicants’ capacity to combat cigarette smuggling and counterfeiting. Funds are used 
for  supporting  the  purchase  of  new  technical  equipment  (or  to  pay  for  the 
maintenance  of  existing  equipment),  such  as  electronic  and  mobile  surveillance 
devices  (e.g.  IMSI23  catchers)  and  inspection  systems  (e.g.  Automated  Number  Plate 
Recognition  System,  ANPRS),  for  analysis  of  digital  evidence,  for  encrypted 
communications and for conducting inspections of vehicles and containers (e.g. X-ray 
scanners,  animals  used  in  the  detection  of  illicit  goods).  In  parallel,  appropriate 
supporting activities in the form of training are provided by the Programme to customs 
staff to make the best use of the equipment. 

20 Communication from the Commission on the implementation of the Hercule Programme and its extension 
during the period 2007-2013. COM (2006) 339 final. 
21 0.1% was spent directly by OLAF on supporting own initiative joint actions related to tobacco trafficking in 
accordance  with  the  Commission  Decision  of  7.2.2013  concerning  the  adoption  of  a  financing  decision  for 
2013 in the framework of the Hercule II Programme. C(2013) 612 final. 
23  International  Mobile  Subscriber  Identity  can  be  used  to  identify  the  user  of  a  mobile  network  and  is  a 
unique identification associated with all networks. It can also be used to acquire details of the mobile in the 
home location register (HLR) or as locally copied in the visitor location register. 
23  International  Mobile  Subscriber  Identity  can  be  used  to  identify  the  user  of  a  mobile  network  and  is  a 
unique identification associated with all networks. It can also be used to acquire details of the mobile in the 
home location register (HLR) or as locally copied in the visitor location register. 

December 2014    7 

  
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Funds  take  the  form  of  grants  or  procurement  contracts.  Grants  are  awarded 
according  to  the  co-financing  principle  whereby  the  total  eligible  expenditure  cannot 
exceed 50%. Procurement, e.g. framework contracts with laboratories for the analysis 
of seized cigarettes, represents an additional tool for providing funding, although this 
form only concerns a minor share of the budget.24  

Figure  4  below  shows  the  breakdown  of  budgets  and  amounts  committed  by  year 
based  on  the  annual  reports.  In  the  years  2010  and  2011,  the  below-average 
allocation  of  funds  was  accompanied  by  an  increase  in  the  resources  for  (Anti-fraud) 
training and, to a lesser extent, to IT Support.  

Figure  4:  Budget  allocated  to  Technical  Assistance  and  IT  Support  (EUR)  and 
committed25  

Total budget TA 

Total committed TA 

% committed26 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

9,455,208 

7,089,000 

7,770,000 

6,250,000 

6,150,000 

7,350,000 

7,000,000 

9,414,489 

7,155,941 

7,785,636 

6,757,726 

4,429,588 

6,581,729 

6,110,923 

100% 

101% 

100% 

108% 

72% 

90% 

87% 

Sector B: Anti-fraud and legal training 
The  projects  falling  within  this  sector  are  aimed  at  enhancing  and  developing  the 
operational, legal and judicial protection of the Union’s financial interests, at spreading 
best practices and at promoting networking. 

While  not  formally  separated  in  the  Decision  878/2007/EC,  activities  in  the  training 
sector are divided into two groups pursuing different operational objectives27.  

The first group is commonly referred to as  ‘anti-fraud training’ and possesses a more 
operational orientation. Anti-fraud training covers the following actions: 

1.  Training in the form of conferences, seminars, e-learning etc. in the area of the 
fight  against  fraud,  including  the  fight  against  cigarette  smuggling  and 
counterfeiting (grants); 

2.  Organisation  by  OLAF,  using  a 

in  organising  events 
(procurement),  of  high-level  conferences  and  ad-hoc  training  actions  focused 
on the protection of the EU financial interests;  

firm  specialised 

3.  Organisation  of  training  sessions  in  digital  forensics  training  by  a  business 
company  (procurement).  The  training  consists  of  courses  given  to  both 
beginners and advanced students for improving their skills and competences in 
the use of tools and software used during forensics examinations of computers, 
tablets,  mobile  phones  or  any  other  digital  device.  The  course  includes  tools 

24 In 2013 the amount committed was EUR 250,000 out of a total of EUR 6,110,923. 
25 Source: Annual reports of the Hercule II Programme for years 2011-2013 and Hercule II mid-term review 
for years 2007-2010. Commitment amounts correspond to the amounts stated in the grant agreements as a 
part  to  be  financed  by  OLAF.  Final  payment  can  be  less  than  the  committed  amount,  as  actual  costs  are 
sometimes less than initially envisaged. 
26  The  programme  can  commit  more  than  100%  to  a  certain  sector,  as  funds  can  be  transferred  across 
sectors  in  accordance  with  the  Financing  Decision  indicating  that  ‘Cumulated  changes  in  allocations  to 
specific actions are not considered to be substantial, provided that they do not significantly affect the nature 
and objective of the work programme’.  
27 Article 1(2) of the Decision 878/2007/EC. 

December 2014    8 

  
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

used  to  collect  evidence  from  devices  running  under  different  operating 
systems.28 

The  second  type  of  training  is  the  ‘legal  training’  which  is  aimed  at  improving  the 
cooperation  between  competent  authorities,  practitioners  and  academics,  at  raising 
awareness among judicial and other professions about the interaction between Union 
law  and  national  criminal  law,  at  promoting  and  disseminating  research  and 
information in this field, and at developing the most appropriate legal instruments for 
protecting  the  EU  financial  interests.  OLAF  provides  support  through  grants  to  the 
Associations  for  European  Criminal  Law  and  for  the  Protection  of  the  EU  Financial 
Interests  (hereinafter  ELA)  as  well  as  to  universities,  research  institutes,  Member 
State  administrations  and  non-profit  actors.  Concrete  actions  consist  of  conferences 
and  seminars  for  public  prosecutors,  research  projects  and  studies  on  topical  issues 
concerning  the  protection  of  the  EU  financial  interests,  and  the  production  and 
dissemination of a quarterly review (Eucrim).  

In  the  case  of  projects  falling  within  the  scope  of  the  anti-fraud  training,  funds  are 
allocated  through  grants  (following  calls  for  proposals)  and  public  procurement 
contracts (on the basis of calls for tenders). Procurement contracts are signed by OLAF 
with external service providers for the organisation of events such as conferences and 
ad-hoc  training,  e.g.  digital  forensics  training.  In  the  case  of  grant  agreements,  the 
co-financing  principle  applies  with  the  maximum  co-financing  rate  set  at  80%  of  the 
eligible expenditure. 

As  far  as  the  legal  part  is  concerned,  all  of  the  funds  are  allocated  through  calls  for 
proposals,  i.e.  through  grant  agreements.  On  average  EUR  620,000  have  been 
allocated  each  year  for  this  part,  amounting  to  4%  of  the  total  funds  for  Hercule  II. 
This  allowed  for  financing  between  5-7  projects  compared  to  the  15-25  applications 
submitted  on  average  on  a  yearly  basis  following  one  (2013)  or  two  calls  for 
proposals.  As  in  the  other  cases,  the  principle  of  co-financing  applies  and  the  rate  is 
set at 90%.  

Figure 5 below illustrates the budgets and commitments based on the annual reports.  

28 Source: OLAF. 

December 2014    9 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure  5:  Budget  allocated  to  training,  seminars  and  conferences  (EUR)  and 
committed29 

Total budget 
training, 
seminars and 
conferences 
anti-fraud 

Total 
committed 
training, 
seminars and 
conferences 
anti-fraud 

% 
committed30 

Total budget 
training, 
seminars 
and 
conferences 
- legal part 

Total 
committed 
training, 
seminars 
and 
conferences 
- legal part 

2,698,908 

2,746,950 

102% 

% 
committed 

4,067,000 

3,968,139 

4,010,000 

3,313,254 

5,100,000 

4,476,897 

5,200,000 

4,226,795 

3,250,000 

2,507,494 

98% 

83% 

88% 

81% 

77% 

700,000 

700,000 

3,000,000 

3,505,811 

117% 

700,000 

652,327 

551,154 

524,867 

93% 

79% 

75% 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

Sector C: IT Support for national authorities 
The  funding  provided  under  the  heading  of  Sector  C  is  aimed  at  providing  specific 
access to databases and IT tools facilitating the access to data and analyses.   

Since 2010, three measures have been implemented in the field of IT, namely: 

  The  renewal  of  subscriptions  to  external  databases  providing  detailed 
information on companies, ship movements, cargo manifests and trade figures: 
Seasearcher  (replaced  Seadata  in  2011);  NTLEX  (information  on  cargo 
manifest  data);  GRS  (general  company  data);  DBAI  (specific  and  financial 
information  on  companies);  GTA  (trade  data)  and  CTI  (Chinese  Trade 
Information)31 

  The  extension  of  the  administrative  arrangements  with  the  European 
Commission’s  Joint  Research  Centre  (JRC)  for  the  development  of  specific  IT 
tools  for  intelligence  purposes  such  as  Contraffic,  an  advanced  risk-analysis 
system  in  the  maritime  transportation  domain,  and  the  Automated  Monitoring 
Tool (AMT), an ad-hoc software that provides analysis of trade flow data at EU 
level and detects suspicious movements; 

  The  purchase  of  technical  and  scientific  services  for  determining  the  origin  of 

seized cigarettes with the aim of building a centralised database. 

The  budget  allocated  to  this  sector  is  illustrated  in  Figure  6  below.  The  funds  are 
distributed through public procurement. The contract is bilateral: it imposes reciprocal 
obligations  on  the  contracting  authority  (OLAF)  and  on  the  economic  operator,  with 
the  operator  providing  the  contracting  authority  with  the  product  or  service  it  has 
ordered. The contracting authority monitors  the provision of the product or service it 
has ordered. 

29 Source: Annual reports of the Hercule II Programme for 2011-2013 data and Hercule II mid-term review 
for 2007-2010 data. The source used for 2007-2010 data (mid-term review of the Hercule II Programme) 
does  not  specify  the  amounts  for  anti-fraud  training  and  training,  seminars  and  conferences  –  legal  part. 
These are included jointly. 
30  The  programme  can  commit  more  than  100%  to  a  certain  sector,  as  funds  can  be  transferred  across 
sectors  in  accordance  with  the  Financing  Decision  indicating  that  ‘Cumulated  changes  in  allocations  to 
specific actions are not considered to be substantial, provided that they do not significantly affect the nature 
and objective of the work programme’. 
31 C-Hawk (cargo manifest data) was also available until November 2011. 

December 2014    10 

  
 
 
 
 
  
  
 
  
  
 
  
  
 
  
  
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure 6: Budget allocated to IT Support (EUR) and committed32 

Total Budget IT 

Total Committed IT 

% committed33 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

1,570,883 

2,644,000 

2,220,000 

2,750,000 

2,950,000 

2,950,000 

3,300,000 

1,563,561 

2,675,920 

2,203,633 

2,544,904 

2,886,734 

3,574,427 

2,581,895 

100% 

101% 

99% 

93% 

98% 

121% 

78% 

2.3.1 

Beneficiaries of Hercule II 

According  to  Council  Decision  804/2004/EC,  the  bodies  eligible  for  funding  under 
Hercule II include: 
  All  national  and  regional  authorities  of  a  Member  State  and  non  EU-countries34, 
which  promote  the  protection  of  the  EU  financial  interests.  Examples  of 
beneficiaries  include  Customs,  Ministries  of  Finance  and  Justice,  law  enforcement 
bodies, audit bodies and the judiciary. 

  Research and educational institutes that have had legal personality for at least one 
year, are established and operating in a Member State and non EU-countries34 and 
promote the strengthening of EU action to protect the Union’s financial interests; 
  Non-profit making bodies that have had legal personality for at least one year, are 
established  and  operating in  a  Member  State  and  non-EU  countries34  and  promote 
the strengthening of Union action to protect the Union’s financial interests.  

The participation of a wider set of representatives from non-EU countries35 is facilitated 
because  the  related  expenditures  are  considered  eligible  for  funding.  Following  the 
application  of  the  conditions  under  the  Hercule  II  basic  act,  no  third  countries  were 
eligible  to  apply  for  funding.  Thanks  to  the  transnational  character  of  the  funded 
activities, participants from Member States or non-EU countries, that do not carry out 
actions, can still benefit from the Programme, particularly through access to databases 
and  participation  in  conferences,  seminars  and  training  organised  by  other  Member 
States. 

32 Source: Annual reports of the Hercule II Programme for 2011-2013 data and Hercule II mid-term review 
for 2007-2010 data. 
33  The  Programme  can  commit  more  than  100%  to  a  certain  sector,  as  funds  can  be  transferred  across 
sectors  in  accordance  with  the  Financing  Decision  indicating  that  “Cumulated  changes  in  allocations  to 
specific actions are not considered to be substantial provided that they do not significantly affect the nature 
and objective of the work programme’”. 
34 According to Article 3 of Council Decision 804/2004/EC these include: acceding countries; the EFTA/EEA 
countries in accordance with the conditions laid down in the EEA agreement; candidate countries associated 
with  the  European  Union  on  the  basis  of  the  conditions  stipulated  in  the  association  agreements  or  their 
additional  protocols  on  participation  in  Community  programmes  concluded  or  to  be  concluded  with  those 
countries. 
35  Including  Balkan  countries  (forming  part  of  the  stabilisation  process  for  the  countries  of  South-Eastern 
Europe), the Russian Federation, countries covered by the Neighbourhood Policy and certain countries with 
which the Union has concluded agreements for mutual assistance in customs matters. 

December 2014    11 

  
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

3. Evaluation findings 

In  this  section  we  provide  an  overview  of  the  main  evaluation  findings  under  each 
evaluation criterion: Effectiveness, efficiency, implementation, utility, acceptability, EU 
added value, complementarity and sustainability.  

The  evaluation  findings  are  grounded  on  a  combination  of  descriptive  analysis, 
qualitative  analysis  and  contribution  analysis,  which  are  based  on  different  data 
sources, including surveys, stakeholder interviews and case studies.36  

For  each  evaluation  criterion  (effectiveness,  efficiency  etc.),  a  number  of  subsections 
are identified based on the specific evaluation questions and structured accordingly.  

3.1.  Effectiveness 

The main evaluation question covering the effectiveness of the Programme concerned 
the  extent  to  which  it  reached  its  objective  of  promoting  activities  in  the  field  of  the 
protection  of  the  financial interests  of  the  Union,  Article  1  of  the  Decision.  As  follows 
from  the  evaluation  methodology  chosen,  the  contribution  of  Hercule  II  to  the 
protection of the financial interests of the EU is examined through a number of themes 
which  combined  provide  insight  into  how  the  Programme  reached  its  objectives.  The 
themes that are examined are based on the activities and objectives of the Hercule II 
Programme, as specified in the Decision establishing the Hercule II Programme.  

Enhanced  transnational  and  multidisciplinary  cooperation  between  Member 
States’ authorities, the Commission and OLAF (Evaluation Question 1.1) 
One  of  the  aspects  in  relation  to  which  the  effectiveness  of  the  Programme  is 
measured  is  transnational  and  multidisciplinary  cooperation,  transnational  networks 
and technical and operational support. The Programme had the ambition of enhancing 
transnational  and  multidisciplinary  cooperation  between  Member  States’  authorities 
and  the  Commission.  This  assessment  is  based  on  the  evidence  collected  through 
surveys  (to  beneficiaries  and  participants),  interviews  with  key  stakeholders  from 
Member  States  and  institutions  and  a  contribution  analysis  based  on  eight  case 
studies. 

Descriptive analysis 
The survey  results show that with regard to  transnational cooperation,  only 62%37 
of the respondents to the beneficiary survey assessed that the project to some or to a 
high degree has enhanced transnational cooperation. In relation to multidisciplinary 
cooperation  between  Member  States’  authorities  and  the  Commission,  63%  of  the 
respondents believe that their latest project has to some or to a high degree enhanced 
multidisciplinary cooperation between Member States’ authorities and the Commission. 
In particular, among beneficiaries  who conducted anti-fraud training, more than 80% 
found  that  the  activity  had  improved  cooperation.  This  can  be  explained  by  the  fact 
that  anti-fraud  training  is  often  intended  to  improve  cooperation,  whereas  this  was 
rarely an objective of Technical Assistance projects.  

Another  indicator  of  the  Programme’s  effectiveness  in  terms  of  transnational 
cooperation is the extent to which it has increased the participants’ understanding of 
both  Union  and  national  mechanisms  for  protection  of  the  financial  interests  of  the 

36  Contribution  analysis  is  only  used  to  assess  the  effectiveness  of  the  Hercule  II  Programme,  as  it  is,  by 
definition,  a  method  to  assess  “the  contribution  of  the  programme’s  activities  to  its  objectives”,  which  is 
covered by the evaluation question effectiveness. 
37 The evaluation norm expected that 70% of the respondents would agree to some or a high extent.  

December 2014    12 

  
 
 
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Union.  According  to  the  results  from  the  participant  survey,  more  than  80%  of  the 
respondents  considered  that  the  most  recent  event  funded  by  Hercule  II  they  had 
attended enhanced their understanding of national mechanisms and EU mechanisms. 

Qualitative analysis 
Stakeholders from Member States assessed both cooperation between Member States 
and cooperation between the Commission and Member States’ authorities. Firstly, 10 
out  of  16  of  the  interviewed  stakeholders  assessed  that  the  Programme  greatly 
enhanced  transnational  cooperation  between  Member  States.  Hence,  the  qualitative 
analysis  of  the  stakeholder  interviews  helps  nuance  the  findings  of  the  beneficiary 
survey  by  suggesting  that  transnational  cooperation  has  been  greatly  enhanced, 
especially through Training, Seminars and Conferences. The same stakeholders judged 
that  the  Hercule  II  Programme  had  only  enhanced  the  multidisciplinary  cooperation 
between  Member  States’  authorities  and  the  Commission  to  some  extent,  thereby 
confirming  the  survey  findings  with  regard  to  multidisciplinary  cooperation.  This 
suggests that, indeed, the Programme may not have made as notable contributions to 
this as was the case with transnational cooperation. However, it should be underlined 
that Training, Seminars and Conferences as well as anti-fraud training were generally 
assessed  to  enhance  the  multidisciplinary  cooperation  between  Member  States’ 
authorities and the Commission. 

Contribution analysis 
The contribution analysis confirmed that anti-fraud training and training, seminars and 
conferences  with  a  legal  focus  were  the  main  drivers  in  enhancing  transnational 
cooperation.  The  findings  from  case  studies  demonstrate  that  the  increased 
coordination between Member States was generally achieved when the events helped 
to  put  in  place  common  terminology,  definitions  of  fraud  and  irregularities  as  well  as 
when the events highlighted areas where Member States could benefit from increased 
cooperation.  Based  on  this,  three  Member  States  established  networks  which 
increased  awareness  of  the  benefits  of  cooperation.  The  case  studies  demonstrated 
that  there  were  even  examples  of  enhanced  transnational  and  multidisciplinary 
cooperation  reached  through  the  purchase  of  the  technical  equipment,  although  this 
has been to a limited extent. With regard to enhancing the understanding of relevant 
mechanisms,  a  majority  of  case  studies  supported  this,  and  highlighted  that  the 
events  made  a  contribution  to  it.  This  contribution  happened  because  Programme 
activities increased coordination and enhanced exchanges of experiences.  

Box 1: Conclusion on enhanced transnational and multidisciplinary cooperation 

The Hercule II Programme enhanced transnational and multidisciplinary cooperation, 
however,  primarily  among  Member  States.  Several  sources  confirm  that  anti-fraud 
training  and  training,  seminars  and  conferences  with  a  legal  focus  were  the  main 
drivers  in  achieving  this.  In  some  instances  Technical  Assistance  also  contributed  to 
enhanced  transnational  and  multidisciplinary  cooperation.  The  Programme  also 
effectively increased the beneficiaries’ and participants’ understanding of both EU and 
national  mechanisms  for  protection  of  the  financial  interests  of  the  Union.  The 
evaluation findings suggest that this increased understanding of the mechanisms also 
contributed positively to Member States becoming increasingly aware of the benefits 
derived from enhanced cooperation. 

December 2014    13 

  
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Building  networks  throughout  the  Member  States,  acceding  countries  and 
candidate countries (Evaluation Question 1.2) 
The previous section showed that the majority of survey respondents and interviewed 
stakeholders considered that the Programme increased transnational cooperation, and 
indicated  that  networks  were  a  factor  in  this.  This  section  further  investigates  the 
extent  to  which  the  Programme  has  built  networks  throughout  the  Member  States, 
acceding  countries  and  candidate  countries  as  well  as  between  practitioners  and 
academics. 

Descriptive analysis 
In relation to the extent to which the Programme has built networks between Member 
States, only 47% of all respondents to the beneficiary survey agree or strongly agree 
that  the  latest  project  they  were  involved  in  has  built  strong  networks  between 
Member States. However, although the beneficiary survey shows that the Programme 
may have succeeded to a low extent, it also showed that this was not the case for all 
sectors, for example with regard to anti-fraud training where 82% respondents agree 
or strongly agree that the training built strong networks.  

Qualitative analysis 
There were affirmative assessments from 7 out of 16 Member State stakeholders, who 
assessed  that  the  Programme  helped  build  networks  throughout  the  Member  States, 
acceding  countries  and  candidate  countries.  According  to  the  stakeholders,  the 
networks  had  three  effects,  namely  faster  information  exchange  between  Member 
States,  increased  access  to  information  from  other  Member  States  and  improved 
response to fraud due to this information.  

Contribution analysis 
The contribution analysis shows that the Programme led to strengthened networks and 
information  exchange.  The  case  study  on  Automated  Number  Plate  Recognition 
System  showed  that  meetings  between  officials  specifically  concerning  equipment 
have  strengthened  the  relations  between  the  participating  Member  States  and  their 
customs  authorities.  Also  staff  exchanges  have  improved  networking,  although  the 
number  of  participants  in  these  exchanges  remains  limited.  The  analysis  also  shows 
that exchange with colleagues both  interpersonally and with regard to data exchange 
remains limited, as information exchange is automated through the equipment feeding 
information into the database. At the level of information networks, there has been a 
significant effect, as the Member States involved can access information via a common 
database. The analysis shows that there is room for improvement in terms of building 
stronger networks across Member States, for example by ensuring staff exchanges. In 
this  way  networks  could  primarily  be  built  between  Member  State  authorities,  the 
Commission  and  OLAF.  Without  building  networks  where  a  group  of  Member  States 
proactively  shares  information  with  each  other,  the  support  seems  to  have 
strengthened  the  internal  national  intelligence  network  more  than  cross-border 
intelligence networks. 

Another  example  of  a  network  is  the  digital  forensics  network,  in  which  500  to  600 
officials take part. It is an informal network that was a result of the training sessions 
organised by an external provider and OLAF. This network was built and continuously 
strengthened through Hercule II activities.  

December 2014    14 

  
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Box  2:  Conclusion  on  building  networks  throughout  the  Member  States,  acceding 
countries and candidate countries 

The  Hercule  II  Programme  has  been  effective  in  establishing  and  strengthening 
networks.  Both  informal  and  formal  networks  between  Member  State  authorities,  the 
Commission and OLAF have been established, with the majority of the networks being 
informal  in  nature.  These  networks  support,  among  other  things,  faster  information 
exchange between national authorities38, increased access to information and improved 
response  to  fraud.  In  addition,  the  Programme  has  contributed  to  improved 
cooperation  between  practitioners  and  academics  in  the  Member  States.  Of  the 
different  sectors  supported  within  the  Hercule  II  Programme,  anti-fraud  training  has 
been the most effective in relation to building networks. The effects of the Programme 
could  be  further  strengthened  by  ensuring  more  personal  contact  between  different 
administrations, for example through staff exchanges. 

Facilitation  of  the  exchange  of  information,  experience  and  best  practices 
(Evaluation Question 1.3) 
Hercule II was intended to facilitate the exchange of information, experience and best 
practices  between  Member  States  in  order  to  enhance  cooperation  across  the  EU. 
Seminars  and  conferences,  anti-fraud  training and  training  with  a legal  focus  are  the 
three key Programme activities intended to contribute to doing so.  

Descriptive analysis 
Based  on  the  results  from  the  beneficiary  survey,  69%  of  beneficiaries  agreed  or 
strongly  agreed  that  the  latest  project  funded  by  Hercule  II  they  had  implemented 
facilitated  the  exchange  of 
information,  experience  and  best  practices.  For 
beneficiaries who conducted anti-fraud training and training, seminars or conferences 
with  a  legal  focus,  100%  either  agreed  or  strongly  agreed  that  the  project  has 
facilitated  exchange  of  information,  experience  and  best  practices.  In  comparison, 
44% of the Technical Assistance beneficiaries agreed or strongly agreed. 

Qualitative analysis 
The  qualitative  analysis  shows  that  11  out  of  16  stakeholders  from  Member  States 
assessed  that  the  Programme  had  made  an  important  contribution to  the  capacity  of 
national  administrations  by  facilitating  the  exchange  of  information,  experience  and 
best  practices  to  a  high  extent.  This  is  in  line  with  the  findings  of  the  beneficiary 
survey. There were only few accounts from institutional stakeholders, suggesting that 
such  exchange  may  be  less  in  focus  or  perceived  as  evident.  Stakeholders  confirm 
that anti-fraud and legal training facilitated the exchange to a higher extent, which is 
aligned  with  the  findings  from  the  beneficiary  survey.  There  was  generally  more 
emphasis on exchange of information and experience, rather than best practices. One 
stakeholder  pointed  out  that  sharing  best  practices  has  helped  Member  States  build 
capacity  to  counter  fraud,  for  example  because  smaller  Member  States  learnt  about 
best practice procedures from larger Member States, which have more expertise. 

Contribution analysis 
The  contribution  analysis  indicated  that  the  Programme’s  main  contribution  to 
increasing  the  exchange  of  experience  between  Member  States  was  made  through 
Training, Seminars and Conferences, where participants discussed during the informal 
sessions of the conference. Some evidence suggested that these discussions led to the 
exchange of experience related to instruments and possibilities which could potentially 
contribute positively to the fight against crimes which threaten the financial interests 

38  This  information  exchange  does  not  refer  to  the  use  of  IT  tools,  but  to  the  fact  that  staff  in  national 
authorities  is  using  the  networks  to  contact  their  counterparts  in  other  Member  States  in  order  to 
spontaneously exchange information. 

December 2014    15 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

of  the  Union.  The  contribution  analysis  indicated  that  an  underpinning  condition  of 
success  was  that  the  events  generally  manage  to  attract  participants  who  were 
experts  in  the  same  area,  thus  facilitating  meaningful  exchanges.  The  contribution 
story presented under Evaluation Question 1.1 presented strong evidence, supporting 
that  the  Programme  made  a  significant  contribution  to  increase  the  exchange  of 
experience between Member States.  

Box  3:  Conclusion  on  the  facilitation  of  the  exchange  of  information,  experience  and 
best practices 

The  Hercule  II  Programme  facilitated  exchange  of  information,  experience  and  best 
practice. Several sources confirmed that the beneficiaries who were conducting anti-
fraud  training  and  training,  seminars  or  conferences  were  more  successful  in 
facilitating the exchange of information, experience and best practices. The exchange 
of  information,  experience  and  best  practice  which  took  place  during  events  funded 
by the Programme was the most successful. 

Provision  of  technical  and  operational  support  for  the  law  enforcement 
authorities of the Member States (Evaluation Question 1.4) 
In  total,  the  Hercule  II  Programme  provided  EUR  51  million39  in  funding  to  Technical 
Assistance.  Due  to  the  lack  of  information  on  tangible  results  of  increases  in  the 
number  of  arrests,  smuggled  goods  and  convictions  as  a  result  of  the  technical 
equipment  purchased,  an  assessment  of  quantitative  improvements  that  the 
Programme  may  have  contributed  to  cannot  be  provided.  However,  the  qualitative 
evidence gives a detailed account of how the Programme helped provide technical and 
operational support for law enforcement authorities.  

Qualitative analysis  
8 out of 16 Member State stakeholders and one NGO stakeholder were able to provide 
clear confirmation that the Programme had provided technical and operational support 
for the law enforcement authorities of the Member States in their fight against illegal 
cross-border  activities,  emphasising  support  for  customs  authorities.  An  example  of 
this  was  that  the  new  equipment  gave  authorities  more  flexibility,  i.e.  previously 
scanners  were  huge  and  processing  slow,  while  the  new  scanners  are  faster  and 
smaller.  The  installation  is  cheaper,  meaning  that  authorities  can  move  the  scanners 
around (beyond BCPs). 

Contribution analysis 
The contribution analysis provides very strong evidence showing that  the Programme 
provided technical and operational support for the law enforcement authorities of the 
Member  States  in  their  fight  against  losses  to  the  Union  budget.  There  were  several 
examples of how Technical Assistance provided law enforcement authorities with high-
quality  equipment  that  contributed  towards  ensuring  better  collection  of  evidence  of 
corruption,  fraud  and  smuggling  in  Member  States.  By  upgrading  the  technical 
equipment  available  to  investigative  units,  the  operational  work  of  law  enforcement 
authorities  was  supported.  This  was  achieved  because  the  availability  of  specific 
devices (e.g. false document detectors, tracking devices) allowed authorities to gather 
intelligence, which they could then use in investigations. In the absence of these tools, 
it  is  questionable  whether  the  same  quality  and  quantity  of  intelligence  could  be 
gathered.  Case  studies  showed  that  this  equipment  increased  the  capacity  of  law 
enforcement,  which  meant  that  more  operations  could  take  place  simultaneously, 
because  more  equipment  was  available  (such  as  specially  equipped  vehicles,  IT 
hardware  and  software).  Importantly,  the  case  studies  demonstrated  that it is  highly 

39  Budgets  allocated  to  sector  A,  Technical  Assistance  according  to  the  Hercule  II  Intermediate  review 
(2010) and the annual reports in 2011-2013. 

December 2014    16 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

unlikely that the equipment would have been purchased without support from Hercule 
II. 

Box  4:  Conclusion  on  the  provision  of  technical  and  operational  support  for  the  law 
enforcement authorities 

The  Hercule  II  Programme  delivered  the  intended  results,  namely  strengthening  law 
enforcement  and  customs  authorities  in  the  intended  way,  that  is,  by  providing 
Technical Assistance. The combined evidence indicates that it was achieved to a high 
extent  and  evidence  shows  that  it  was  generally  unlikely  that  the  equipment  would 
have  been  purchased  without  the  Programme’s  co-financing.  There  were  many 
examples  of  how  Technical  Assistance  provided  law  enforcement  with  high-quality 
equipment  that  contributed  towards  ensuring  better  collection  of  evidence  of 
corruption,  fraud  and  smuggling in  Member  States.  In  addition,  findings  showed  that 
on  a  more  overall  level  Technical  Assistance  increased  the  flexibility  of  the  law 
enforcement and capacity of law enforcement authorities in Member States. 

Facilitating access to data and analysis through specific databases and IT 
tools (Evaluation Question 1.5) 
The  IT  Support  activities  are  completely  implemented  by  procurement.  OLAF 
purchases  licences  to  external  databases  and  contracts  the  Joint  Research  Centre  to 
develop  IT  tools.  In  addition,  OLAF  has  a  framework  contract  with  a  company  that 
performs analysis of seized counterfeit cigarettes. These resources are made available 
to  the  Member  States  through  OLAF  and  are  intended  to  facilitate  data  access  and 
analysis. While this section primarily analyses  the effects  of the access to databases, 
the following section will focus on the IT tools.  

Descriptive analysis 
In  2013,  OLAF  conducted  a  small  survey  of  the  use  of  OLAF’s  external  databases 
(apart  from  the  CTI),  which  collected  the  views  of  22  officials  from  Member  States. 
Although the findings of the survey should be interpreted with care since 31% of the 
respondents were from Germany, the survey results do provide some indication of the 
utility  of  the  databases.  Across  databases  the  survey  shows  that  the  access  to  the 
databases was assessed as related to actions for the protection of the Union’s financial 
interests  by  all  respondents.  Moreover,  77%  of  the  respondents  considered  that  the 
access to databases was very relevant when assessing whether they had added value 
for  the  fight  against  fraud.  The  remaining  respondents  assessed  it  as  fairly  relevant, 
suggesting that overall, the access to databases made a positive contribution to data 
access  and  analysis.  These  findings  are  supported  by  the  survey  of  database  users 
(apart from the CTI) made in connection with a case study for this evaluation. In this 
survey  the  respondents  also  agreed  that  without  access  to  this  information,  the 
authorities would be less successful in preventing illegal imports into the Union. 

There was no  relevant  descriptive data available in relation to the  effectiveness  of  IT 
tools. 

Qualitative analysis 
The qualitative analysis provided further insights into how the IT tools facilitated data 
access  and  analysis.  However,  the  information  comes  primarily  from  institutional 
stakeholders,  as  stakeholders  from  Member  States  were  not  able  to  assess  it.  This 
suggests that there may be low  awareness  of  the databases  among the target group 
or  that  Member  States  are  not  aware  that  the  Programme  provides  access  to  the 
databases.  Among  institutional  stakeholders,  an  interviewee  explained  that  the 
Programme provides  access to the databases because this EU procurement of access 
increases  the  availability  of  information  to  a  broader  EU  audience  at  a  lower  cost.  If 

December 2014    17 

  
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

the  Programme  had  not  done  so,  Member  States  would  need  to  purchase  access 
themselves,  which  would  possibly  exclude  some  Member  States,  in  particular  newly 
acceded countries. Therefore, the EU procurement of access ensures that information 
can  be  accessed  across  Member  States.  The  institutional  stakeholders  assessed  that 
databases are a fundamental tool, which is used during both the pre-operational and 
operational  phases.  The  access  to  databases  increased  the  information  available  to 
authorities, which is essential given the cross-border nature of the majority of customs 
fraud.  

Contribution analysis 
In general, there were two strands in the case study findings on the access to external 
databases.  One  showed  that  database  users  find  the  information  provided  by  the 
databases  useful  and  indicates  that,  while  Member  States  could  potentially  acquire 
some  data  through  channels  not  supported  by  the  Programme,  it  would  be  more 
costly.  The  other  strand  suggests  that  the  distribution  of  access  through  national 
contact  points  has  not  ensured  that    all  authorities  within  Member  states  may  be 
benefitting  from  the  access  to  data.  One  possible  explanation  for  this  may  be  that 
there is no strategy for disseminating information concerning access to the databases. 
Findings  indicate  that  the  access  also  differs  between  Member  States,  where  some 
may be benefitting more from it than others.  

Box 5: Conclusion on facilitated data access and analysis through specific databases 
and IT tools 
The  Hercule  II  Programme  has  been  effective  in  facilitating  access  to  data  in  some 
Member States, while it has only contributed to a limited extent in others. The findings 
suggest that by distributing access to data only through national contact points, there 
is  an  increased  risk  that  not  all  relevant  authorities  within  the  Member  States  are 
benefitting  from  the  access  to  data.  Findings  indicate  that  the  same  may  be  said  for 
the  access  to  data  across  all  Member  States,  where  some  may  be  benefitting  more 
from the data than others. 

Improved support for investigations, monitoring and intelligence work 
through IT tools (Evaluation Question 1.6) 
This  aspect  is  closely  linked  to  the  analysis  presented  above,  which  established  that 
databases have facilitated data access. The present section’s analysis therefore builds 
on this in order to assess the extent to which the data access and analysis contributed 
to  improving  support  for  investigations,  monitoring  and  intelligence  work  through  IT 
tools.  

Contribution analysis  
The case studies showed that the Automatic Monitoring Tool developed by the JRC and 
funded  through  the  Hercule  II  Programme  produces  (almost)  real-time  data.  This 
allows authorities to react much faster and at an earlier stage, possibly at the time of 
customs  clearance,  meaning  that  suspected  fraudsters  could  potentially  be  caught  in 
the  act,  increasing  the  number  of  detections.  This  has  helped  some  Member  States 
initiate  investigations  of  suspected  fraudsters,  and  has  in  a  few  instances  led  to  the 
recovery  of  duties  and  VAT  due,  although  only  small  amounts.  Meanwhile,  it  is 
expected  that  further  planned  developments  will  lead  to  improvements  in  customs 
authorities’  monitoring  work,  which  will  increasingly  discourage  companies  from 
attempting fraud and thus prevent losses to the Union budget.  

Investigations  were  also  improved  through  the  information  provided  by  the  external 
databases, and the survey  respondents (to the mini-survey carried out during a case 

December 2014    18 

  
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

study)  agreed  that  without  access  to  this  information,  the  authorities  would  be  less 
effective  in  preventing  illegal  imports  into  the  Union.  For  example,  the  CTI  has 
contributed  to  preventing  losses  to  the  EU  budget,  because  the  information  drawn 
from  the  database  is  crucial  to  the  investigations  of  national  customs  authorities  and 
helps  recover  large  sums  annually.  While  the  survey  respondents  found  it  difficult  to 
assess to what extent the information provided by the other databases contributed to 
the  prevention  of  smuggling  of  cigarettes  and  crime  more  generally,  they  strongly 
agreed that they would be less successful in preventing such crimes without access to 
this information.  

Box 6: Conclusion on improved support for investigations, monitoring and intelligence 
work through IT tools 

The Hercule II Programme developed features of the Automatic Monitoring Tool (AMT) 
which  improved  the  monitoring  and  intelligence  work  of  national  law  enforcement 
authorities.  The  tool  has  significantly  improved  the  authorities’  ability  to  detect 
undervaluation  of  imported  goods,  and  it  has  increased  efficiency,  freeing  up  human 
resources  and  time  for  other  investigation  work.  The  AMT’s  potential  can,  however, 
not  be  considered  fully  reached  until  the  planned  improvements  have  been  finalised, 
thereby rendering the tool more useful to national authorities in their work to prevent 
losses  to  the  EU  budget.  In  addition,  the  Hercule  II  Programme  provided  access  to 
information  through  external  databases,  i.e.  CTI,  and  thereby  helped  national 
authorities  to  improve  their  investigations,  which  would  otherwise  have  been  less 
successful. 

Cigarette smuggling and counterfeiting (Evaluation Question 1.8) 
Activities  to  support  the  fight  against  smuggling  and  counterfeiting  of  cigarettes  are 
funded  under  Technical  Assistance  ‘The  purchase  of  special  equipment  and  tools’,  as 
well  as  under  the  Training  Sector  (Conferences  and  seminars  for  customs)  and  IT 
Support  (maintenance  and  update  of  a  database  with  information  on  the  chemical 
composition  and  origin  of  seized  smuggled  and  counterfeited  cigarettes) 40.  The 
effectiveness of these activities will be assessed by examining the extent to which the 
Programme has multiplied and intensified the measures in the areas identified as the 
most sensitive, in particular cigarette smuggling and counterfeiting.  

Separate  calls  for  proposals  are  organised  by  OLAF  to  ensure  that  actions  are 
concretely  directed  at  the  fight  against  cigarette  smuggling  and  counterfeiting. 
However,  here  the  evaluator  had  limited  data available  on  the  numbers  of  arrests  or 
numbers  of  seizures  of  smuggled  cigarettes  and  counterfeit  products,  which  made  it 
difficult  to  assess  the  effectiveness  of  these  actions  in  purely  quantitative  terms.  For 
the same reason it was not possible to determine whether the Programme contributed 
to  increased  numbers  of  arrests  or  increased  numbers  of  seizures  of  smuggled 
cigarettes  and  counterfeit  products.  However,  the  qualitative  analysis  provides  an 
overall  assessment  of  whether  the  fight  against  smuggling  and  counterfeiting  of 
cigarettes was intensified and the contribution analysis provides concrete quantitative 
evidence from case studies.  

Descriptive analysis 
Overall,  the  beneficiary  survey  showed  that  75%  of  beneficiaries  agreed  or  strongly 
agreed  that  the  Programme  multiplied  and  intensified  the  measures  in  the  areas 
identified  as  the  most  sensitive,  in  particular  cigarette  smuggling  and  counterfeiting. 

40 Annual Report 2013. 

December 2014    19 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

90% and 88% respectively of the beneficiaries who carried out actions in the Technical 
Assistance  sector  and  in  the  area  of  anti-fraud  training  agree  or  strongly  agree  that 
the Programme has intensified the available measures in targeting cigarette smuggling 
and counterfeiting. 

Qualitative analysis 
Three  Member  State  stakeholders  were  able  to  provide  assessments  of  the  extent  to 
which  the  Programme  multiplied  and  intensified  the  measures  in  the  areas  identified 
as  the  most  sensitive,  in  particular  cigarette  smuggling  and  counterfeiting.  It  was 
highlighted that Technical Assistance has supported the law enforcement authorities to 
a  high  extent.  For  instance,  baggage  scanners  were  installed  at  airports  and 
international  post  offices,  which  has  enabled  scanning  travellers’  luggage  and 
packages  arriving  into  a  Member  State.  As  effective  as  these  scanners  are,  the 
stakeholder  pointed  out  that  the  scanners  cannot  determine  whether  a  product  is 
counterfeit, and therefore the use of them has to be combined with trained personnel. 
Similar observations were made by another stakeholder, who assessed that  Technical 
Assistance  enabled  customs  authorities  to  detect  smuggled  cigarettes  entering  the 
country.  

One  stakeholder  assessed  that  the  Programme  had  indeed  multiplied  and  intensified 
measures,  thereby  helping  to  counter  cigarette  smuggling  and  fraud.  At  the  same 
time, the stakeholder noted that it had been difficult to benefit fully from this due to 
the  scarcity  of  national  co-financing.  In  this  regard,  it  was  emphasised  that  if  the 
results of Hercule II projects were better documented, it would facilitate the Member 
State in obtaining national co-financing.  

Contribution analysis 
A  case  study  confirmed  that  the  equipment  would  not  have  been  purchased  in  four 
Member States without the co-financing provided by Hercule II. 

Due to the lack of quantitative data on seizures of smuggled or counterfeit cigarettes, 
the evidence is limited. This makes it difficult to estimate precisely the extent to which 
the actions have multiplied and intensified existing measures. However, the qualitative 
data  provided  strong  evidence  showing  that  the  Programme  intensified  measures  in 
the  area  of  cigarette  smuggling,  because  it  contributed  to  projects  which  improved 
staff’s  analytical  capacity  to  counter  cigarette  smuggling.  The  increased  analytical 
capacity was for example achieved though working visits with peers in the UK, France 
and  Romania.  After  the  project,  personnel  put  this  knowledge  to  use  in  their  daily 
work and were better at detecting smuggled cigarettes.  

Another  example  was  that  the  Programme  contributed  to  sharing  knowledge  on 
effective measures across Member States. For example, the dissemination and update 
of  written  guidelines  on  how  to  manage  risks  related  to  cigarette  smuggling,  or  by 
providing training that helped increase the knowledge of customs personnel in relation 
to how cigarettes may be concealed in vehicles and how to dismantle vehicles (without 
causing unnecessary damage). 

The evidence collected indicated that four drivers made the projects successful. Firstly, 
the  participants  were  experts,  which  ensured  detailed  discussions.  Secondly,  in  the 
case  of  working  visits,  the  host  administrations  were  welcoming  and  ready  to  share 
their  experiences,  which  ensured  that  the  training  took  place  in  a  supportive 
environment and facilitated a meaningful exchange of experience across participating 
states.  Thirdly,  the  organising  authorities  had  previous  experience  with  planning  and 
carrying  out  similar  projects,  which  supported  a  smooth  implementation.  Finally,  the 
projects  received  the  support  they  required  from  OLAF,  which  also  supported  project 
implementation.  

December 2014    20 

  
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

Box  7:  Conclusions  on  intensified  measures  against  smuggling  and  counterfeiting  of 
cigarettes 

The Hercule II Programme’s increased focus on cigarette smuggling and counterfeiting 
(compared to Hercule I) has led to an increased effort in this respect. This is not only 
shown  by  the  multiple  activities  providing  funding  for  the  purchase  of  technical 
equipment,  but  also  through  the  views  of  the  beneficiaries  as  75%  of  the  survey 
respondents  agreed  that  the  Programme  had  been  successful  in  this  regard.  The 
contribution  analysis  supports  the  findings in  terms  of  the  equipment  purchased,  but 
also  in  terms  of  the  training  activities,  which  have  been  aimed  at  increasing  the 
national  authorities’  capacity  to  fight  cigarette  smuggling  and  counterfeiting.  Only 
limited  quantitative  evidence  exists  concerning  the  effects  of  these  intensified 
measures, which is assessed to be mainly due to the lack  of dedicated monitoring at 
the Member State level. 

Differences between the effectiveness of the three sectors (Evaluation 
Question 1.9) 
The  three  sectors  of  the  Hercule  II  Programme  are  assessed  in  relation  to  the 
objectives  which  each  of  them  are  intended  to  achieve.  In  other  words,  we  do  not 
assess  whether  IT  Support  led  to  increased  cooperation  between  practitioners  and 
academics, since IT Support was never intended to lead to this objective.  

The  table  below  shows  which  of  the  intended  objectives  the  sectors  were  most 
effective at achieving, and which they were least effective at. It should be noted that 
these  have  been  selected  based  on  how  strong  the  evidence  was  in  support  of  the 
objective having been reached.  

Table 1: Sectorial effectiveness in reaching Programme objectives 

Sector 

Most effective  

Least effective  

Technical Assistance  Strengthened technical and 

Enhanced transnational and 

Objective 

operational support for law 

multidisciplinary cooperation between 

enforcement and customs authorities 

Member State authorities and the 

& 

Commission, striking a geographical 

Improved investigations, monitoring 

balance 

and intelligence work 

Training, Seminars 

Enhanced transnational cooperation 

Unified level of skills, equipment and 

and Conferences 

between Member State authorities  

readiness in Member States to detect 

& 

smuggled and counterfeit products 

Strengthened networks and 

exchange of information 

IT Support 

Improved investigations, monitoring 

Enhanced transnational and 

and intelligence work 

multidisciplinary cooperation between 

Member State authorities and the 

Commission 

The  evidence  shows  that  Technical  Assistance  strengthened  technical  and 
operational  support  for  law  enforcement  and  customs  effectively  because  it  helped 
Member States acquire  new  equipment. The same was the case  when assessing how 
Technical  Assistance  improved  investigations,  monitoring  and  intelligence  work.  In 
comparison,  there  was  little  evidence  suggesting  that  the  objectives  were  reached 
through enabling joint operations and staff exchanges or promoting common technical 
standards (as set out in the theory change).  

December 2014    21 

  
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

The  evaluation  shows  that  Technical  Assistance  made  only  a  limited  contribution  to 
enhanced  transnational  and  multidisciplinary  cooperation  between  Member  State 
authorities  and  the  Commission.  The  key  explanation  identified  was  that  Technical 
Assistance  projects  tended  to  focus  on  one  Member  State  and  only  occasionally 
involved  multiple  Member  States  (although  there  were  two  examples  of  this).  In 
addition, Technical Assistance projects only infrequently intend to reach this objective. 

With regard to Training, Seminars and Conferences, these were assessed to have 
effectively  enhanced  transnational  cooperation  between  Member  State  authorities. 
Evidence  showed  that  this  objective  was  reached  through  two  mechanisms,  namely 
the  increased  coordination  between  Member  States,  which  followed  after  events 
organised  under  the  Programme,  as  well  as  through  the  increased  exchange  of 
experience  which  often  occurred  during  events.  Common  for  both  mechanisms  was 
that  these  activities  made  Member  States  aware  of  areas  where  there  were  mutual 
benefits  to  be  reaped  from  increased  transnational  cooperation.  With  respect  to 
multidisciplinary  cooperation  or  cooperation  with  the  Commission,  there  was  limited 
evidence showing that this part of the objective had been achieved.  

Training,  Seminars  and  Conferences  was  judged  the  least  effective  at  leading  to  a 
unified level of skills, equipment and readiness in Member States to detect smuggled 
and counterfeit products. However, there was some evidence indicating that this may 
have  been  achieved  through  conferences  by  facilitating  discussions  on  the  definitions 
of fraud, terminology and disseminations of control methods.  

Finally,  IT  Support  was  shown  to  have  been  most  effective  at  improving 
investigations,  monitoring  and  intelligence  work,  primarily  through  the  provision  of 
access  to  data.  The  reason  is  that  this  data  feeds  into  anti-fraud  work  being  done  in 
Member  States.  There  were  indications  that  the  access  to  databases  carried  even 
more  potential,  if  further  awareness  of  them  was  raised  and  their  features  better 
explained. At the same time, the case studies suggested that limited national capacity 
or interest may be hindering some from fully benefitting from the access to databases 
provided through the Programme.  

Based  on  the  above,  it  is  not  possible  to  judge  whether  a  specific  sector  has  been 
more  successful in contributing to the  outcomes and impacts  of  the Programme than 
others, since the three sectors deliver complementary results.  

The  evaluation  found  limited  evidence  of  how  interaction  between  the  sectors  had 
worked  during  the  Programme,  suggesting  that  different  authorities  within  Member 
States  were  more  likely  to  implement  projects  in  a  certain  sector.  Moreover,  few 
examples  of  coordination  of  projects  across  sectors  were  found,  while  projects  were 
coordinated  within  one  sector  (often  Technical  Assistance)  between  national 
authorities.  

Box 8: Conclusions on the differences between the effectiveness of the three sectors 

The  Hercule  II  Programme  delivers  complementary  results  within  the  three  different 
sectors supported, hence it is not possible to judge whether a specific sector has been 
more  effective  in  contributing  to  the  outcomes  and  impacts  of  the  Programme  than 
others.  For  example,  Technical  Assistance  has  been  highly  effective  in  providing 
strengthened  technical  and  operational  support  for  law  enforcement  and  customs 
authorities,  as  well  as  in  improving  investigations,  monitoring  and  intelligence  work. 
The  sectors’  contribution  to  enhanced  transnational  and  multidisciplinary  cooperation 
between Member State authorities and the Commission was very limited and also not 
the  main  objective  of  providing  Technical  Assistance.  Training,  Seminars  and 
Conferences  has,  for  its  part,  enhanced  transnational  cooperation  between  Member 

December 2014    22 

  
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

State authorities and strengthened networks and exchange of information, while it has 
been  less  effective  in  developing  unified  levels  of  skills,  equipment  and  readiness  in 
the  Member  States  to  detect  smuggled  and  counterfeit  products.  IT  Support  has,  in 
support  of  the  Technical  Assistance  sector,  improved  investigations,  monitoring  and 
intelligence work. 

Geographical balance (Evaluation Question 1.7) 
In  the  following  section  we  analyse  the  extent  to  which  the  Programme  has  struck  a 
geographical  balance  by  involving  all  Member  States,  acceding  countries  and 
candidate countries in the activities financed under the Programme.  

Descriptive analysis  
The secondary data provided by OLAF shows that the Programme has to some extent 
struck  a  geographical  balance.  All  the  Member  States  have  received  funding  through 
the Hercule II Programme. However, Croatia was the only acceding country41 that has 
been  involved  as  a  beneficiary  of  the  Hercule  II  Programme.  No  candidate  countries 
have been direct beneficiaries of the Programme, since none were eligible for funding 
in line with the provisions set out in Article 3 of the Hercule II Programme.  

There  is  a  significant  difference  between  the  Member  States  that  have  been 
beneficiaries of the Hercule II grant. When considering both the volume of funding and 
the  number  of  actions,  it  is  clear  that  Belgium,  France  and  Germany  are  the  three 
Member  States  which  have  both  received  most  funding  and  carried  through  most 
projects.  This  is  mainly  due  to  the  fact  that  service  providers  under  procurement 
contracts have been located in these three countries, increasing the amount of funding 
for,  in  particular,  Training,  Seminars  and  Conferences  for  these  Member  States. 
Poland, Hungary, Romania and Spain constitute a cluster of Member States that have 
had  the  second  highest  volume  of  funding  and  number  of  actions.  Czech  Republic, 
Denmark,  Sweden  and  Bulgaria  have  received  a  limited  amount  of  funding  and  were 
involved  in  few  projects.  However,  it  is  important  to  mention  that  a  high  volume  of 
funding does not necessarily indicate a high number of projects and vice versa.  

The figure below gives a complete overview of the volume of funding by country and 
by  sector,  however,  excluding  IT  Support,  which  does  not  consist  of  grants  or 
contracts awarded to national authorities or organisations.  

41 Croatia became an EU Member State on 1 July 2013. 

December 2014    23 

  
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure 7: Volume of funding (grant or contract) in each sector per Member State.42  

42 Source: Elaboration of Ramboll Management Consulting based on data provided by OLAF (List of contracts 
or grant agreements – at Level 2 commitment – funded under Hercule II budget broken down by sector of 
activity and country). 

December 2014    24 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

All the Member States have received funding through the Technical Assistance sector, 
with  Spain,  Poland  and  Hungary  being  the  biggest  beneficiaries  in  monetary  terms, 
followed  by  Belgium,  Germany,  Lithuania,  the  Netherlands,  Portugal,  Romania  and 
Finland. These Member States are all characterised by either ports of entry into the EU 
or  external  land  borders  of  the  EU  towards  the  East,  thus  confirming  that  Technical 
Assistance is often awarded to Member States with external borders.  

With  regard  to  the  number  of  participants  in  projects  per  Member  State,  acceding 
country  and  candidate  country,  there  is  only  limited  information  on  where  the 
participants  over  the  period  were  from.  One  example  can  be  found  in  the  digital 
forensics  training,  where  approximately  20%  of  the  respondents  to  the  Hercule  II 
participant  survey  who  had  attended  digital  forensics  training  (n  =  145)  were  from 
countries other than Member States, including candidate countries. Although this does 
not provide a complete overview of the number of participants per  candidate country 
during  the  Programme,  it  does  suggest  that  participants  from  candidate  countries  do 
have  access  to  Hercule  II  activities  such  as  digital  forensics  training  and  the  AFCOS 
conference.  In  the  case  of  the  latter,  one  case  study  examined  two  AFCOS 
conferences,  which  took  place  in  the  then-candidate  country  Croatia  and  candidate 
country Turkey, and thereby demonstrated how countries besides Member States have 
been involved in projects funded under the Programme. Another example is that in the 
beneficiary  survey  11%  of  the  respondents  worked  in  countries  outside  the  EU,  with 
Serbia,  Kosovo*43,  Switzerland  and  the  former  Yugoslav  Republic  of  Macedonia 
accounting for half of those respondents. This gives some indication of  how countries 
other than Member States participated in events under the Programme.  

Box 9: Conclusions on geographical balance 

While  all  the  Member  States  have  benefitted  from  the  Hercule  II  Programme,  there 
are significant differences in terms of both amounts of funding and numbers of actions 
carried  out  by  the  Member  States.  All  the  Member  States  have  received  Technical 
Assistance,  with  Spain,  Poland  and  Hungary  being  the  biggest  beneficiaries  in 
monetary terms, followed by Belgium, Germany, Lithuania, the Netherlands, Portugal, 
Romania and Finland. These Member States are all characterised by either busy ports 
of  entry  into  the  European  Union  or  external  land  borders  of  the  European  Union 
towards the East. As acceding country, Croatia, which is now a member of the EU, was 
a  direct  beneficiary  of  the  Programme.  In  terms  of  Training,  Seminars  and 
Conferences,  funding  has  generally  been  allocated  to  newer  Member  States  (Poland, 
Hungary  and  Romania),  with  exception  of  Belgium,  France  and  Germany,  where 
numerous training providers with framework contracts directly with OLAF are located. 
The  objective  for  geographical  balance  has  been  abandoned  for  the  Hercule  III 
Programme. 

43 * This designation is without prejudice to positions on status, and is in line with UNSCR 1244/99 and the 
ICJ Opinion on the Kosovo declaration of independence. 

December 2014    25 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

3.2.  Efficiency 

The  main  evaluation  question  covering  the  efficiency  of  the  Programme  was  whether 
the  same  effects  could  have  been  achieved  with  lower  costs,  if  different  instruments 
currently  used  for  the  implementation  of  the  Programme  had  been  applied.  This  is 
examined through different areas of efficiency, as presented below. 

Ability  to  achieve  the  desired  effects  at  reasonable  costs  (Evaluation 
Question 2.1) 

In order to examine Hercule II’s efficiency, it is relevant to assess whether the desired 
effects have been achieved at a reasonable cost. The annual budget of the Hercule II 
Programme  has  varied  between  EUR  13.7-15  million  per  year,  with  78-117  actions 
receiving  funding  per  year.  When  looking  at  the  efficiency  of  the  Hercule  II 
Programme,  a  challenge  exists  in  the  assessment  of  the  cost-effectiveness  ratio,  as 
the different sectors of activities render very different types of effects.  

According  to  the  beneficiary  survey’s  respondents,  the  desired  project  results  were 
achieved  at  reasonable  costs.  Here,  90%  of  all  respondents  agree  or  strongly  agree 
with the relevant statement.  

Figure  8:  To  what  extent  do  you  agree  that  the  desired  project  results  have  been 
achieved at reasonable costs? (N= 59)44  

Our  case  studies  show  that  while  some  beneficiaries  of  Technical  Assistance  are  able 
to  report  on  the  number  of  x-ray  scans  and  of  the  number  and/or  value  of  detected 
smuggled  goods,  such  as  clothes  or  cigarettes,  carried  out  with  the  new  equipment, 
the evaluators have not found any cases where it has been possible to compare these 
numbers  to  a  baseline,  i.e.  the  situation  before  the  technical  equipment  was 
purchased and put into use. This means that it is not possible to indicate the efficiency 
of  the  Programme  in  monetary  terms,  i.e.  its  estimated  financial  impact  (prevented 
losses) compared to its total cost. 

The main reason for this is that the monitoring systems are at this stage not yet fully 
up and running, as for example the final technical report and the final implementation 
report have only been used in their current format since 2013 (see section 3.3 below).  

44 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    26 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

By  collecting  data  from  numerous  secondary  sources,  including  the  three  annual 
reports  on  the  implementation  of  the  Hercule  II  Programme  and  the  final  technical 
reports of the actions that were covered in our case studies, the estimates provided by 
the  beneficiaries  on  the  financial  impact  of  the  detection  carried  out  through  the 
Hercule II financed technical equipment amount to more than EUR 270 million, which 
is three-fold the total cost of the Programme (EUR 98 million)45.  

Efficiency  gains  delivered  by  the  Programme  (Evaluation  Questions  2.2,  2.3 
and 2.446) 

During  the  evaluation  a  number  of  efficiency  gains  were  identified  within  the  two 
sectors Technical Assistance and Training, Seminars and Conferences.  

Technical Assistance 

A notable finding was identified during a case study on the purchase of x-ray scanners 
for  customs  as  a  technical  support  tool  for  detecting  the  presence  of  cigarettes  and 
tobacco.  When  comparing  the  different  experiences  in  the  different  Member  States  it 
can be seen that while one customs authority had problems finding a suitable supplier 
and  as  a  result  experienced  high  prices  on  the  product,  another  experienced  lower 
prices  than  expected  on  the  equipment  procured.  This  was  assessed  to  be  due  to 
increased  competition  on  the  European  market  from  suppliers  in  third  countries.  The 
equipment sought and procured in the two Member States was not identical; however, 
the different experiences still suggest that there may be advantages to be gained from 
sharing  experiences  between  the  Member  States  on  the  type  of  equipment  used  and 
how it is procured.  

Training, Seminars and Conferences 

Data is not available on the costs of training per participant for the different types of 
training,  seminars  and  conferences  funded  by  Hercule  II.  This  means  that  it  is  not 
possible  to  assess  to  what  extent  savings  have  been  derived  from  the  specialised 
training  organised  jointly,  i.e.  the  training  being  organised  in  the  way  that 
representatives of several Member States can join it, as opposed to training organised 
in one Member State, in the national language, and only being open for participation 
for persons from that Member State. 

The  figures  show  that  in  total  EUR  26  million  were  budgeted  (based  on  the  annual 
work  programmes)  for  anti-fraud  training  throughout  the  course  of  the  Hercule  II 
Programme. A rough estimate shows that more than 19,000 persons have participated 
in training, seminars or conferences funded in 2007-2013. 

It  is  challenging  for  the  participants  in  training,  seminars  and  conferences  to  assess 
whether their organisation has been able to save money as a result of the specialised 
training  activities  –  27%  of  the  respondents  do  not  know  whether  this  has  been  the 
case.  However,  64%  of  those  who  have  attended  anti-fraud  training  are  aware  and 
agree  that  their  organisation  has  been  able  to  save  money,  compared  to  organising 
similar training nationally. What the case studies and contribution analysis indicate is 
that the Hercule II Programme has strongly contributed to the organisation of training, 
seminars  and  conferences,  which  would  often  not  have  been  carried  out  without  the 
support from the Hercule II Programme.  

For  example,  when  looking  at  the  digital  forensics  training,  which  is  organised  bi-
annually by an external provider who has signed a framework contract with OLAF for 
the  provision  of  training,  the  case  study  indicates  that  by  organising  the  training 

45 This number should be looked at with care, as it is by no means complete and only sums the cases where 
the beneficiaries have been able to assess the financial impact and have reported on it in writing. The case 
studies do not cover all the Hercule II actions, which could suggest that in reality the number is likely to be 
much higher. A full list of sources used to reach this amount can be found in Annex C. 
46  Based  on  discussions  with  the  Steering  Group,  evaluation  questions  2.2.,  2.3  and  2.4  (as  stated  in  the 
revised interim report) were modified, and this has been section revised. 

December 2014    27 

  
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

collectively, it is possible for the organisers to ensure the best possible trainers for the 
courses.  On  average  one  trainer  per  five  participants  is  present  at  the  training 
sessions.  With  the  EUR  600-800,000  per  year  funded  through  the  Hercule  II 
Programme,  approximately  12047  persons  have  received  highly  specialised  training 
during  each  session  and  650  different  persons  altogether  from  the  EU  as  well  as 
candidate  and  potential  candidate  countries.  These  included,  among  others,  Serbia, 
Albania,  Kosovo*48,  the  former  Yugoslav  Republic  of  Macedonia  and  Bosnia  and 
Herzegovina which have participated in the training, many of  which reached a higher 
level of expertise by participating several times (on different subjects). 

However, one aspect that emerges from the case studies that in particular in relation 
to training, seminars and conferences, the beneficiaries tend to overestimate the costs 
in  the  budgeting  phase  in  order  to  ensure  that  they  can  get  all  the  eligible  costs 
covered  in  the  end.  The  beneficiaries  overestimate  costs  to  avoid  having  to  shoulder 
any  exceeded  costs  on  a  national  level.  While  this  is  understandable  from  the 
beneficiaries’  point  of  view,  it  can  be  counterproductive  for  the  Programme,  as  the 
eligible costs to be funded will be lower than those budgeted for. 

Overall,  it  seems  that  the  organisation  of  training,  seminars  and  conferences  under 
Hercule II serve other purposes of more qualitative nature, as already described under 
the evaluation criterion effectiveness. 

Ability  to  make  resources  available  to  the  beneficiaries  in  due  time,  in  the 
appropriate quantity and in the appropriate quality (Evaluation Question 2.5) 

An important aspect of efficiency is to assess whether the resources have been made 
available  to  the  beneficiaries  in  due  time,  in  the  appropriate  quantity  and  in  the 
appropriate quality. 

A  clear  majority  of  the  projects  (83%  of  those  surveyed)  started  on  time.  Based  on 
the  answers  of  the  survey  respondents  as  well  as  the  case  studies  it  seems  that  a 
reason  for  delays  is  usually  found  at  the  beneficiary’s  end,  rather  than  with  OLAF.  It 
was mentioned that securing the national co-financing can take time, which can delay 
projects,  although  this  is  an  initial  condition  for  obtaining  co-financing  from  the 
Programme. A limited number of contracts had been signed late, delaying the project 
start. 

47 With an average cost of EUR 350,000 per two-week training session and an average of 120 participants, 
the  cost  per  participant  for  a  training  session,  including  flights  and  accommodation,  amounts  to 
approximately EUR 2,900. 
48 * This designation is without prejudice to positions on status, and is in line with UNSCR 1244/99 and the 
ICJ Opinion on the Kosovo declaration of independence. 

December 2014    28 

  
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure  9:  Did  the  project(s)  start  on  time  (N=  59)  and  did  you  receive  the  payment 
according to the payment schedule set in the grant agreement? (N= 57)49 

OLAF has also been efficient in providing the final payment according to the payment 
schedule set in the grant agreement. This has been the case according to 91% of the 
survey  respondents.  Both  of  the  above  are  positive  signs  of  efficient  project 
management on the part of OLAF50. 

An  interesting  aspect  of  the  Programme’s  efficiency  is  also  the  beneficiaries’ 
perception  of  the  sufficiency  of  the  grant.  Across  the  59  different  projects  that 
responded  to  the  survey,  there  is  an  overall  perception  that  the  grant  received  from 
OLAF was sufficient in order for the project to reach its objectives, with 71% agreeing 
to a high degree with the relevant statement.  

49 Source: Ramboll Management Consulting, Survey to beneficiaries. 
50 They can also be partly related to the entry into force of the new Financial Regulation on 1 January 2013; 
the Hercule II budget was implemented for the first time based on the new Regulation in 2013. The main 
changes have been related to the requirement to inform applicants within six months of the outcome of the 
evaluation  procedures  as  well  as  to  shortened  payment  delays  and  more  detailed  requirements  on  the 
publication  of  awarded  grants.  Commission  staff  working  document,  Annual  Overview with  Information  on 
the Results of the Hercule II Programme in 2012. SWD (2014) 247 final, 17.7.2014. Despite the document’s 
name  referring  to  the  Hercule  II  Programme  in  2012,  the  document  covers  the  implementation  of  the 
Programme in 2013. 

December 2014    29 

  
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure  10:  To  what  extent  do  you  agree  that  the  grant  received  from  OLAF  was 
sufficient in order for the project to reach its objectives? (N= 59) 51 

This  aspect  is  also  interesting  in  light  of  the  new  Hercule  III  Programme,  where  the 
maximum EU funding for the proposed actions amounts to 80%  of the  eligible costs. 
For  exceptional  and  duly  justified  cases,  this  percentage  may  even  be  increased  to  a 
maximum of 90% of the eligible costs.52  

Despite  these  positive  statements,  the  beneficiaries  mentioned  that  securing  the 
national  co-financing  to  cover  the  rest  of  the  budget  has  not  always  been  easy. 
Furthermore,  in  some  countries  with  high  VAT,  additional  costs  were  incurred  due  to 
the  addition  of  VAT  on  top  of  the  national  co-financing.  Similar  statements  were 
provided in some stakeholder interviews, where the necessity for a higher co-financing 
rate  was  mentioned  by  some  stakeholders.  However,  with  respect  to  the  technical 
equipment,  the  beneficiaries  often  mention  that  the  co-funding  has  helped  them 
ensure  replacement  of  old  equipment  earlier  than  would  have  been  possible  with 
national funding only.  

Box 10: Conclusions on efficiency 

The  Hercule  II  Programme  shows  indications  of  having  reached  its  desired  effects  at 
reasonable costs. However, as is visible from the evaluations’ assessment above, it is 
not possible to conclude whether the Programme has led to significant efficiency gains. 
Therefore,  additional  monitoring  data  to  specify  the  concrete  cost-benefit  ratio  of  in 
particular  the  Technical  Assistance  sector  is  needed  (please  see  section  3.3  for  more 
information  on  ex-post  monitoring).  The  evidence  points  to  efficient  project 
management  in  terms  of  making  the  resources  available  to  the  beneficiaries  in  due 
time, and in the appropriate quantity and quality. However, the different experiences 
still  suggest  that  there  may  be  advantages  to  be  gained  from  sharing  experiences 
between the Member States on the type of equipment used and how it is procured.  

51 Source: Ramboll Management Consulting, Survey to beneficiaries. 
52 Hercule III Programme 2014-2020. Call for Proposals: Technical Assistance   for  the  fight  against  EU 
fraud. Deadline 1 September 2014. 

December 2014    30 

  
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

3.3.  Implementation 

When  discussing  the  implementation  of  the  Hercule  II  Programme,  it  is  relevant  to 
analyse the implementation in three  different layers. Firstly, it is relevant to examine 
how  the  Programme  management  is  organised  internally  at  OLAF.  Secondly,  it  is 
relevant  to  look  at  the  application  and  selection  phase,  and  thirdly  at  the 
communication with ongoing projects as well as their reporting and monitoring. 

Internal organisation at OLAF (Evaluation Question 3.1) 
While until 2012 the responsibility for the overall management of the Programme was 
decentralised to a number  of units and departments (each responsible for  one  sector 
or  sub-sector  of  activity),  the  reorganisation  of  the  Office  in  2012  led  to  the 
centralisation of the coordination and management functions into a single unit, namely 
Unit  D.5,  a  unit  within  OLAF’s  Policy  Directorate,  whose  Head  of  Unit  acts  as  the 
authorising  officer  by  sub-delegation  (AOSD),  and  which  manages  the  Hercule  II 
Programme.  

In  line  with  the  Financial  Regulation,  there  is  a  separation  of  competencies  for  the 
implementation  of  the  Hercule  II  Programme  between  unit  D.5  and  unit  O.2  (Human 
Resources & Budget). 

Unit D.5 drafts the Financing Decisions, establishes strategy documents for the annual 
implementation of the Hercule Programme, monitors and reports on the implemented 
activities  and  coordinates  with  stakeholders.  Unit  0.2  deals  with  the  financial  and 
budgetary management of the Hercule II Programme.  

The financial circuit of the management of the Hercule II Programme is decentralised 
with  a  hierarchical  independence  from  the  AOSD  of  the  persons  performing  financial 
(initiation  and)  verification.  In  respect  of  the  ‘four-eyes’  principle  (Financial 
Regulation,  art.  66.5),  the  aspects  of  an  operation  (operational  and  financial)  are 
verified  by  at  least  one  member  of  staff  other  than  the  one  who  initiated  the 
operation. 

Coordination  between  the  activities  carried  out  under  Hercule  and  other  activities 
carried  out  by  OLAF,  ad  hoc  inter-service  meetings  with  other  Commission  DGs  and 
Member States is ensured through regular meetings of the Advisory Committee for the 
Coordination  of  Fraud  Prevention  (COCOLAF)  and  through  inter-service  consultation 
among  Commission  services  on  Annual  Work  Programmes.  COCOLAF  is  composed  of 
representatives of the Member States’ authorities.  

Application and selection phase (Evaluation Question 3.1) 
Grants  are  allocated  on  the  basis  of  a  procedure  following  a  call  for  proposals 
published at OLAF’s website and in the Official Journal. All proposals are collected and 
checked against the formal as well as the technical requirements as set out in the calls 
by  an  ad-hoc  established  Evaluation  Committee  made  up  of  at  least  three  persons 
representing  at  least  two  organisational  entities  of  the  institutions  or  other  bodies. 53 
Based  on  the  eligibility,  exclusion,  selection  and  award  criteria54,  the  Evaluation 
Committee  makes  a  recommendation  to  the  Authorising  Officer  containing  the  list  of 
immediately  approved  projects,  the  immediately  rejected  ones  and  those  requiring 
additional information. 

The selection and award criteria for projects to be funded are specified in a financing 
decision,  which  is  adopted  on  an  annual  basis.  An  examination  of  the  financing 

53 As defined in Article 116(1) of the previous Financial Regulation and Article 133(1) of the current Financial 
Regulation whose members are to be appointed by the authorising officer, i.e. the Head of Unit D.5. This is 
in line with the provisions set out in Article 178 of the previous Implementing Rules and 204.1 of the current 
Rules of Application (Article 133 of the Financial Regulation). 
54 From Article 2c Commission Decision No 878/2007/EC establishing the Hercule II Programme. 

December 2014    31 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

decisions  shows  that  the  selection  procedures  do  take  into  account  the  objectives  of 
the  Programme,  as  laid  down  in  the  further  elaboration  and  specification  of  the 
Hercule  II  Decision.  The  examination  also  showed  that  the  selection  procedure  takes 
into account that the funding should lead to an “improved knowledge of OLAF and its 
working  methods,  identification  of  shortcomings  in  the  systems  of  preventing  and 
combating  fraud,  as  well  as  optimal  dissemination  of  the  results  among  a  broader 
public”.  The  figure  below  shows  the  amounts  budgeted  and  finally  committed  each 
year  of  the  Programme’s  implementation.  As  can  be  seen  during  most  years,  the 
amount  budgeted  exceeds  the  amount  committed.  This  may  be  related  to,  for 
example,  the  quality  of  the  applications.  However,  the  share  of  commitment  is 
relatively high, indicating an effective implementation of the budget. 

Figure 11: Budgeted vs. total committed amount (EUR)55 

The beneficiaries consider the application procedures to be straightforward and easy to 
complete.  This  is  illustrated  by  the  survey  to  the  beneficiaries,  where  81%  of  the 
respondents agreed or strongly agreed with the relevant statement. 

55 Source: Elaboration by Ramboll Management Consulting based on the data provided by the Annual Work 
Programmes  2007-2013  (budget  commitments  made  by  the  Commission  per  sector  and  including  both 
grants and procurement) and cross-referenced with Hercule II mid-term review and Annual overviews. 

December 2014    32 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

Figure 12: To what extent do you agree with the following statement? – The Hercule II 
Programme’s  application  procedure  has  been  straightforward  and  easy  to  complete 
(N= 59)56 

One  of  the  beneficiaries  specified  that  while  application  procedures  are  never  ‘easy’, 
he  considered  that  the  application  procedure  of  Hercule  II  guarantees  transparency, 
concurrence and adequate reporting  based on  a reasonable effort, and that it is  thus 
appropriate. 

In general, the beneficiaries who had received grants through Hercule II had not noted 
any changes in the Programme’s procedures over the years, but it was mentioned that 
there  is  a  learning  effect  which  makes  it  easier  to  apply  for  funding  a  second  time 
around,  having  filled  in  the  application  forms  once  previously.  There  were  also 
beneficiaries who mentioned that the learning effect is undermined by the fact that the 
templates have been changed several times over the years due to implementation of 
audit recommendations and the organisational change in 2012, which led to an update 
of the reporting templates related to Technical Assistance (see below). 

Communication, monitoring and reporting (Evaluation Question 3.1) 
During  and  after  the  project,  communication,  monitoring  and  follow-up  activities  are 
performed.57  Beneficiaries  under  Hercule  II  are  required  to  submit  two  documents, 
namely a final financial report and a final technical report following the finalisation of 
the project and the expiry of the grant agreement. The former includes information on 
the  financial  implementation  of  the  action,  such  as  invoices  and  incurred  costs.  The 
latter  comprises  a  detailed  description  of  the  action  and  of  its  impacts  on  the 
investigative  and  operational  activities  of  national  and  regional  administrations  (e.g. 
the  results  enabled  or  facilitated  through  the  equipment  purchased  with  the  EU’s 
financial support).  

56 Source: Ramboll Management Consulting, Survey to beneficiaries. 
57  Commission  Decision  No  878/2007/EC  establishing  the  Hercule  II  Programme  provides  that  the 
Commission (or a qualified outside body) may carry out an audit throughout the lifetime of the contract or 
the agreement and up to five years following the last payment. This set of monitoring tools and indicators 
was  identified  as  a  priority  by  the  Commission  in  its  communication  for  the  continuation  of  Hercule  after 
2006. COM(2006) 339 final. 

December 2014    33 

  
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

The  submission  of  these  documents  is  compulsory  as  they  enable  Unit  D.5  to  make 
recommendations  to  Unit  O.2  concerning  the  final  payment.  The  intelligence  gained 
through these reports represents the basis for the compilation of the annual reports on 
the  implementation  of  the  Programme  that  the  Commission  submits  to  the  European 
Parliament and the Council. 

In  the  case  of  the  Technical  Assistance  sector,  the  template  for  the  final  technical 
report  is  recent  and  has  only  been  implemented  since  2013.  With  regard  to  Training 
(including  the  Legal  Part),  the  beneficiaries  must  submit  a  summary  event  feedback 
form, in addition to the above mentioned documents. This feedback form is based on a 
standardised  questionnaire  to  guarantee  comparability  and  consistency  across 
projects.  

The  beneficiaries  considered  that  the  reporting  requirements  during  the  project’s 
implementation  are  easy  to  meet:  78%  of  the  respondents  either  strongly  agreed  or 
agreed with the relevant statements in the survey. No striking differences between the 
views of the beneficiaries and the different sectors or types of activities were detected. 

Figure 13: To what extent do you agree with the following statement? – The reporting 
requirements during the project’s implementation have been easy to meet (N= 59) 58 

OLAF  has  introduced  a  new  type  of  reporting,  e.g.  in  the  Technical  Assistance  sector 
which required more detailed targeted reporting on results achieved with the technical 
equipment. This type of reporting should be continued. 

Besides  these  formalised  reporting  requirements,  additional  quality  checks  of  the 
funded activities exists, including the attendance to events by OLAF staff with financial 
management  responsibilities  and/or  with  technical  expertise  as  well  as  systematic 
dialogues  between  the  OLAF  staff  and  the  national  services  concerned  with  the 
purchase of technical equipment. 

The beneficiaries are highly satisfied with the support provided by Unit D.5 during the 
implementation of their projects. The information collected through case studies shows 
that the communication between OLAF and the beneficiaries works well and that OLAF 
is  able  to  answer  to  a  majority  of  the  questions  asked  by  beneficiaries.  This  is 
supported both by the stakeholder interviews and by the survey results, where 44% of 
the respondents  strongly agree and another 36% agree that the support  received by 

58 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    34 

  
 
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

OLAF  during  the  project  implementation  has  contributed  to  the  project  reaching  the 
desired results. 

Figure 14: To what extent do you agree with the following statement?  – The support 
received  by  OLAF  during  the  project  implementation  has  contributed  to  the  project 
reaching the desired results (N= 59)59 

Project monitoring is primarily based on the information provided by the beneficiary in 
their  final  technical  and  financial  reports.  Incurred  costs  reported  in  the  financial 
reports  are  verified  through  a  thorough  examination  of  receipts  (and  other 
documentation  provided).  In  addition,  the  results  of  the  projects  are  reported  in  the 
final technical reports.  

The  case  studies  showed  that  although  the  Commission  has  the  right  to  carry  out  a 
verification  of  the  results  reported  in  the  technical  reports,  only  a  small  number  of 
verifications  took  place.  The  evaluation  identified  three  reasons  for  why  verification 
rarely takes place. 

Firstly,  as  mentioned  by  an  institutional  stakeholder,  the  beneficiaries  are  primarily 
enforcement  authorities  of  the  Member  States,  which  means  that  a  certain  level  of 
peer-to-peer  cooperation  exists  between  the  beneficiaries  and  OLAF.  Risks  are 
therefore  considered  to  be  low  and  an  ex-post  verification  of  the  results  reported  in 
the final technical reports is rarely regarded as necessary.  

Secondly,  the  final  technical  reports  are  required  to  be  submitted  a  short  time  after 
the  grant  agreement  has  expired,  which  makes  it  unlikely  that  concrete  results  have 
materialised,  in  particular  in  the  case  of  Technical  Assistance.  Verification  would  not 
change this, as it takes time for the projects to deliver effects.  

Thirdly,  if  a  verification  mission  takes  place  (with  Commission  staff  visiting  project 
beneficiaries  and  monitoring  the  use  of  the  technical  equipment  purchased)  this  may 
not necessarily lead to a better understanding and an improved overview of the use of 
the  equipment,  because  such  a  verification  only  is  a  snapshot  of  the  use  of  the 

59 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    35 

  
 
 
 
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

equipment. A permanent monitoring of use of the equipment is not cost effective and 
would even be counter-productive in some operations. 
Based  on  the  evaluator’s  examination  of  the  final  technical  reports  of  numerous 
grants, it can be seen that the quality and content of these reports vary greatly. One 
notable  observation  in  this  regard  is  that  the  information  on  results  included  in  the 
reports  is  often  scattered  and  of  poor  quality.  While  there  are  positive  examples  of 
final technical reports,  where  the beneficiaries could clearly describe  the added value 
of  the  support  provided  by  Hercule  II,  the  reports  often  lack  the  specificity  that  is 
necessary for OLAF to report on the effectiveness of the Programme.  

OLAF follows the Financial Regulation and its requirements, and it is expected that the 
introduction  of  final  implementation  reports  (submitted  one  year  after  the  final 
payment) will improve  the monitoring situation. However, it seems that more can be 
done to improve the reporting requirements in order to obtain more information about 
the results of the projects. A more targeted monitoring of the project results and their 
contribution to the objectives of the Hercule Decision seems to be necessary.  

Box 11: Conclusions on implementation 

Evidence  points  to  a  positive  assessment  of  the  organisational  change  carried  out  in 
2012  on  the  implementation  of  the  Hercule  II  Programme.  Furthermore,  the 
beneficiaries are overall positive about the application procedures to be followed within 
the  Programme.  While  the  beneficiaries  consider  the  reporting  requirements  to  be 
relatively easy to meet, the evidence showed that more should be done to improve the 
reporting  requirements.  By  ensuring  the  right  types  of  data  are  requested  from  the 
beneficiaries  in  the  final  technical  report,  OLAF’s  ability  to  monitor  the  results  and 
impacts of the activities funded can be improved. 

3.4.  Utility 

The  utility  of  the  Programme  concerns  the  extent  to  which  the  results  of  the 
interventions match the needs in the Member States and at the Union level when the 
Programme was adopted.  

Needs, problems and issues in the Member States (Evaluation Question 4.1) 

The  beneficiary  survey  shows  that  the  vast  majority  (86%)  of  the  68  respondents 
state that they either strongly agree or agree that the results of the project they have 
implemented  match  the  needs  and  problems  in  their  country.  In  particular,  the  anti-
fraud  training  projects  have  been  assessed  to  match  the  country’s  needs  to  a  very 
high  extent,  as  94%  answer  that  they  either  strongly  agree  or  agree  that  there  is  a 
match between the project and their needs and problems.  

December 2014    36 

  
 
 
 
 
 
Evaluation of the Hercule II Programme 

Figure 15: To what extent do you agree with the following statement? - The results of 
the  implemented  project  match  the  needs  and  problems  experienced  in  our  country 
(N= 68)60 

Looking  at  the  specific  activities,  the  participant  survey  indicates  that  the  ‘digital 
forensics  training’  is  the  activity  considered  to  match  the  organisations’  needs  best. 
75% of the respondents strongly agree with this, while only 30.8% of the respondents 
who participated in ‘legal conference and seminars’ strongly believe that these events 
have matched their organisations’ needs.  

Needs, problems and issues at Union level (Evaluation Question 4.2) 

The  overall  objective  of  the  Programme  is  to  protect  the  Union’s  financial  interests. 
Hercule  II  is  the  Commission’s  (OLAF)  instrument    of  offering  the  organisations 
(national  and  regional  authorities,  research  and  educational  institutes  and  non-profit 
making  bodies)  practical  and  effective  support  to  promote  the  protection  of  the  EU 
financial interests. 

Furthermore,  the  Hercule  II  Programme  introduced  a  distinct  focus  on  cigarette 
smuggling and counterfeiting as these account for  a substantial threat to the Union’s 
financial interests.  

The  Programme  also  helps  raise  awareness  of  activities  that  contribute  to  protecting 
the  Union’s  financial  interests.  One  example  of  this  can  be  seen  in  the  AFCOS 
conferences. However, the visibility and awareness of the Hercule II Programme could 
be further developed.  

Box 12: Conclusions on utility 

The beneficiaries and participants are satisfied with the Hercule II Programme’s ability 
to match their needs at the Member State level. The funded activities have a broader 
focus  than  the  Member  States’  needs,  as  they  contribute  to  the  protection  of  the 
financial  interests  of  the  Union.  The  Hercule  II  Programme  has  reached  its  different 
objectives,  and  thus  it  is  assessed  that  the  Programme  has  contributed  to  the 
prevention of losses to the Union’s budget. 

60 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    37 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

3.5.  Acceptability 

The  acceptability  should  be  seen  as  an  extension  of  the  utility  as  it  assesses  the 
usefulness  of  the  Programme,  in  particular  when  considering  the  costs  and  the 
benefits from the perspective of the beneficiaries. 

Usefulness  and  satisfaction  with  the  Programme  (Evaluation  Questions  5.1 
and 5.2) 

The  projects  supported  by  the  Hercule  II  Programme  are  overall  assessed  by  the 
respondents to the beneficiary survey to be  very useful (75%) or useful (22%) when 
considering the benefits in relation to the cost of participation. The anti-fraud training 
is considered to be especially relevant as 88% of the respondents state that this area 
has been very useful, and the remaining 12% state that the training has been useful. 

Figure  16:  Taking  into  account  the  costs  of  participation  and  the  benefits  of  the 
project, how useful did you find the participation? (N= 68) 61 

Overall,  most  of  the  participants  in  the  supported  training,  seminars  and/or 
conferences find the activities useful and are satisfied with their participation. 94% of 
the  participants  find  their  participation  either  very  useful  or  useful  when  considering 
the  time  and  money  spent  on  the  activity.  As  described  in  the  section  on  utility 
(above),  the  digital  forensics  training  is  the  event  assessed  to  best  match  the 
participants’ needs, followed by round-table activities. 

The  stakeholder  interviews  confirm  the  findings  from  the  two  surveys  concerning  the 
usefulness  of  the  Programme.  The  stakeholders  highlight  that  the  Programme  has 
been particularly useful in the area of customs. In this regard, stakeholders mentioned 
that the purchases of devices and equipment are expensive in the area of customs and 
that funds (and co-financing) can be difficult to obtain from the national budgets. This 
makes the Programme a valuable source of support for such investments.   

Box 13: Conclusions on acceptability 

The Hercule II Programme is accepted by beneficiaries and stakeholders. The benefits 
of  the  Programme  clearly  overshadow  the  costs  of  participation.  The  Programme  is 
particularly useful for customs authorities. 

61 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    38 

  
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

3.6.  EU added value 

The  main  evaluation  question  covering  EU  added  value  of  the  Hercule  II  Programme 
concerns  whether  the  interventions  have  allowed  for  achieving  objectives  that  could 
not, or to a lesser extent, have been achieved by interventions undertaken at national 
or  regional  level  (Evaluation  Question  6).  In  other  words,  the  evaluation  examined 
whether  it  would  have  been  possible  to  carry  out  the  same  activities  and  achieve 
similar  results  through  national  or  regional  funding  if  the  Hercule  II  Programme  had 
not existed.  

The  answer  to  this  question  is  based  on  the  opinions  of  beneficiaries,  training  and 
conference  participants  as  well  as  other  stakeholders,  collected  through  surveys, 
interviews  and  case  studies.  To  the  extent  that  this  information  is  available,  the 
answer also draws on evidence provided by organisers of training and conferences on 
the added value of these events.  

Overall,  the  majority  of  the  surveyed  and  interviewed  stakeholders  agree  that  the 
Hercule II Programme provides an added value that could not have been achieved (to 
the same extent) by national or regional initiatives and funds. 

Figure 17: Percentage of beneficiaries who agree that the same objectives could have 
been achieved by means of national initiatives (N=68)62 

As  Figure  17  shows,  65%  of  the  surveyed  beneficiaries  either  disagreed  (46%)  or 
strongly disagreed (19%) that the objectives achieved by means of Hercule II funding 
could  also  have  been  achieved  through  national  initiatives.  Similarly,  51%  of  the 
surveyed  participants  in  Hercule-funded  training,  conferences  and  seminars  agreed 
that they could not have achieved the same benefits from a similar event organised at 
national  or  regional  level.  This  finding  is  lower  than  expected.63  Meanwhile,  this  is  in 
part due to the fact that a large share of both groups of respondents found it difficult 
to assess this question. Only 6% of beneficiaries and 12% of participants in Hercule-
funded  events  considered  that  the  Programme  did  not  provide  a  particular  EU  added 
value. 

62 Source: Ramboll Management Consulting, Survey to beneficiaries. 
63  The  norms  set  for  this  evaluation  question  was  that  70%  of  beneficiaries  and  70%  of  participants  in 
training agreed or strongly agreed. 

December 2014    39 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

From the perspective of participants, the participant survey showed that in total, 54% 
of the respondents in the participant survey stated that the latest training, seminar or 
conference  they  had  participated  in  took  place  in  a  country  other  than  the  one  they 
stated  as  the  country  they  work  in.  Overall,  this  suggests  that  another  aspect  of  EU 
added  value  is  providing  trans-European  training,  which  otherwise  could  not  have 
been funded.  

The  mainly  positive  assessment  of  the  added  value  of  having  an  EU  programme  for 
fighting  fraud  and  protecting  the  financial  interests  of  the  Union  is  further  supported 
by  the  statements  given  by  stakeholders  through  interviews  and  through  the 
contribution analysis. 

With  regard  to  Technical  Assistance,  several  stakeholders  interviewed  considered 
that  having  EU  financial  support  committed  through  Hercule  helps  bring  attention  to 
the importance of this type of equipment to the work of customs and law enforcement 
authorities, and essentially raises awareness of the protection of the financial interests 
of  the  country  and  the  Union.  The  contribution  analysis  showed  that  for  the  vast 
majority of projects, the technical equipment could not have been funded exclusively 
through  national  means.  Several  project  managers  underlined  that  without  the 
Programme,  they  could  not  have  purchased  new  equipment,  and  in  some  instances 
restricted  national  budgets  meant  that  national  co-financing  was  difficult  to  obtain. 
There were very few exceptions to this tendency, but some case studies indicated that 
had  Hercule  II  not  existed  it  would  have  taken  longer  to  purchase  equipment,  and 
thus not been impossible.  

In relation to Training, Conferences and Seminars, an important EU added value of 
the Hercule II Programme pertains to the facilitation of international cooperation. This 
appears to be especially true for the digital forensics training, which has contributed to 
exchange of knowledge and the establishment of networks across Member States. The 
participants  agreed  that  these  results  could  not  have  been  achieved  without  the 
existence of Hercule II as national budgets rarely cover this type of training, because 
of its international angle.  

In interviews, participants in the digital forensics training stated that the training was 
invaluable to help develop the basic competencies of their national cyber-crime units, 
and  that  it  was  necessary  for  the  investigation  of  cyber-crime  and  economic  fraud 
cases,  many  of  which  had  a  cross-border  element.  Moreover,  the  varied  mix  of 
nationalities and job roles ensured that there were possibilities for participants to meet 
law  enforcement  colleagues  with  a  wide  range  of  experience  and  backgrounds.  Many 
participants attended multiple events and over the period began to form closer contact 
with  colleagues  from  cyber-crime  departments  in  other  countries.  The  establishment 
of  such  practitioners’  networks  has  improved  access  to  necessary  information  in 
connection  with  international  investigations.  Consequently,  the  networks  led  to  an 
increase in formal requests for information, which contributed to a faster collection of 
information  on  international  cases.  The  interviewed  participants  assessed  that  this 
could  not  have  been  achieved  without  the  existence  Hercule  II  as  it  could  not  have 
been covered by national budgets.  

Another example was identified in the case study on the Hercule II-funded conference 
on  Rural  Development  Fraud.  This  conference  provides  another  good  example  of  the 
added  value  of  the  Programme,  in  terms  of  facilitating  international  cooperation  and 
sharing  knowledge  and  practices.  The  conference  was  judged  to  have  allowed  all 
Member States to come together to discuss, and this would not have been possible if 
the costs were not covered by the Programme. Similarly, the case study on the AFCOS 
conferences  showed  that  participants  in  these  conferences  shared  experiences, 
learned  from  each  other’s  best  practices  and  coordinated  or  adopted  relevant 
activities.  Cooperation  has  clearly  increased  as  a  result  of  the  conference  and  it  has 
been a key forum for the new Member States to discuss anti-fraud coordination. 

December 2014    40 

  
 
 
 
 
Evaluation of the Hercule II Programme 

For IT Support, the AMT project in particular is said to have resulted in concrete and 
valuable  tools  that  the  Member  States’  authorities  assess  could  not  have  been 
developed  at  national  level.  In  relation  to  the  access  to  databases,  the  evidence 
collected  through  a  case  study  is  slightly  ambiguous  in  terms  of  establishing  the 
extent to which EU added value is realised. It is clear that the database users find the 
information  provided  by  the  databases  useful  and  there  are  indications  that,  while  it 
might  be  possible  to  acquire  this  information  elsewhere,  it  would  be  more  costly  for 
the users’ organisations. The fact that the database users generally agree that without 
access  to  this  information  their  organisations  would  be  less  successful  in  preventing 
illegal imports into the Union suggests that there is an EU added value of having such 
access to information. 

Looking  closer  at  the  different  sectors  of  the  Hercule  II  Programme,  there  are  no 
significant  differences  in  the  beneficiaries’,  stakeholders’  or  case  studies’  assessment 
of the EU added value of the Programme, regardless of whether they were in charge of 
implementing  a  project  related  to  Technical  Assistance,  Training/Conference  or  IT 
Support.  Only  very  few  of  the  surveyed  beneficiaries  agree  that  the  same  objectives 
could have been achieved at local or national level.  

Box 14 Conclusions on EU added value 

The  Hercule  II  Programme  has  been  successful  in  providing  EU  added  value.  All 
sources  confirm  this  assessment.  Generally  and  across  the  different  types  of  actions, 
stakeholders  agree  that  the  objectives  achieved  by  means  of  Hercule  funding  could 
not,  or  to  a  lesser  extent,  have  been  achieved  with  national  or  regional  means.  This 
applies to all three sectors of the  Programme. However, there are variations between 
the  sectors  in  terms  of  the  types  of  added  value  that  the  different  actions  provide. 
Training  and  conferences  are  particularly  instrumental  in  facilitating  international 
cooperation  and  establishing  networks.  Meanwhile,  the  tools  developed  and  provided 
under the IT Support sector and the Technical Assistance actions provide more direct 
value  to  the  investigations  of  the  national  customs  and  law  enforcement  authorities. 
As  such,  the  Hercule  II  actions  clearly  contribute  to  the  protection  of  the  financial 
interests of the Union. 

December 2014    41 

  
 
 
 
 
Evaluation of the Hercule II Programme 

3.7.  Complementarity 

The  main  evaluation  question  to  be  answered  for  the  complementarity  evaluation 
criteria  is:  To  what  extent  are  complementarity  and  lack  of  overlaps  ensured 
between  Hercule  II  and  other  related  EU-funded  initiatives?  (Evaluation 
Question 8). This question is mainly concerned with the policy areas of the European 
Commission  that  are  the  most  closely  linked  to  the  objectives  of  the  Hercule 
Programme  and  covered  by  DG  HOME  and  DG  TAXUD,  and  the  extent  to  which 
complementarity is ensured between programmes in these fields and Hercule II. 

The  answer  to  this  question  is  based  on  desk  research,  interviews  with  stakeholders 
and contribution analysis.  

Complementarity with other EU-funded initiatives 

Overall,  the  interviewed  stakeholders  from  Member  States  did  not  find  it  difficult  to 
distinguish the Hercule II Programme and its objectives from related EU programmes, 
such  as  DG  TAXUD’s  Customs  2020  or  DG  HOME’s  Programme  ISEC  (Prevention  of 
and Fight against Crime). In general, the stakeholders considered that there were no 
real risks of overlaps.  

The  impact  assessment  drafted  for  the  preparation  of  the  decision  on  the  Hercule  II 
Programme clearly considered the complementarity and risk of overlaps with other EU 
programmes. Within the impact assessment, due attention is given to examining how 
the actions of Hercule II may be complementary to actions funded under the FISCALIS 
(DG TAXUD) programmes.  

In  the  view  of  stakeholders  from  Member  States  it  is  clear  that  there  are  common 
objectives between DG HOME initiatives and funds such as the European Border Fund; 
but they also considered it to be quite clear that while DG HOME initiatives are focused 
on  such  issues  as  human  rights,  migration,  anti-terrorism  and  crime  prevention,  the 
Hercule II Programme had a clear financial focus, also in its activities related to crime 
prevention,  which  the  interviewees  found  distinguishes  the  Hercule  activities  from 
those of DG HOME. 

Among  interviewed  stakeholders  from  Member  States,  Customs  2020  (DG  TAXUD) 
was regarded as more  of a  complementary programme to  Hercule II  due to  its focus 
on  exchange  of  personnel,  for  instance,  and  the  exchange  of  knowledge.  In  this 
regard, Technical Assistance was highlighted as a distinctive feature of the Hercule II 
Programme. One example of this is the emphasis that Hercule II placed on anti-fraud 
activities.  This  is  also  supported  by  the  answers  from  the  survey  to  beneficiaries, 
which shows that a clear majority of project managers find that Hercule II is unique in 
what it does. 

Ensuring coordination in order to avoid overlaps 

In  the  impact  assessment  for  the  launch  of  the  new  Hercule  III  Programme  to  be 
running  between  2014  and  2020,  it  is  stated  that  during  the  Hercule  II  Programme 
period, steps have been taken to guard against the risk of overlapping with DG HOME 
activities.  For  instance,  the  purchase  of  scanners  for  Member  State  border  services 
was preceded by contacts with DG HOME to make sure there was no overlap with their 
envisaged  actions64.  So,  while  no  specific  DG  HOME  programmes  or  activities  are 
mentioned  as  having  been  under  consideration  during  the  preparation  of  the 
programme  under  evaluation,  steps  have  been  taken  under  the  programming  period 
to  ensure  this65.  Furthermore,  as  mentioned  above,  complementarity  and  potential 

64  Commission  staff  working  paper:  impact  assessment,  Accompanying  document  to  the  Proposal  for  a  regulation  of  the  European 
Parliament  and  of  the  Council  on  the  Hercule  III  programme  to  promote  activities  in  the  field  of  the  protection  of  the  European 
Union’s financial interests; p. 39. 
65 Communication from the Commission on the implementation of the Hercule programme and its extension during the period 2007-
13; p. 16. 

December 2014    42 

  
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

overlaps  with  the  TAXUD’s  Fiscalis  programme  were,  according  to  the  Impact 
Assessment, also thoroughly assessed prior to the programme implementation. 

Interviewed  representatives  of  DG  HOME  found  that  under  the  ISEC  Programme 
(Programme  Prevention  of  and  Fight  against  Crime),  there  could  be  some  overlaps 
between the call for projects within cybercrime and financial and economic crime and 
the  activities  of  Hercule  II.  Other  DG  HOME  representatives,  however,  disagreed  and 
said that there are differences between the particular focus of Hercule  II and the  DG 
HOME-funded activities.  In general, the representatives judged that  efforts  are made 
to ensure complementarity and avoid overlaps. However,  they were not sure  of what 
and  how  much  –  other  than  the  inter-service  consultations  on  the  annual  activity 
reports – and they assessed that more could perhaps be done to accommodate this. 

Similarly, the representative of DG TAXUD pointed to the inter-service consultation as 
a  means  of  avoiding  risk  of  overlaps  and  ensuring  complementarity,  but  also 
mentioned that apart from this there was no specific communication with the Hercule 
II  Programme  management  to  prevent  overlaps.  That  said,  the  representative  found 
that,  while  there  is  a  common  objective  between  Hercule  II  and  the  TAXUD 
programmes,  Fiscalis  and  Customs  2020,  namely  fighting  fraud,  they  address  this 
from different, complementary perspectives. 

Box 15 Conclusions on complementarity 

Complementarity in the preparation and implementation of the Hercule II Programme 
has  been  sufficiently  ensured.  Stakeholders  generally  agree  that  Hercule  II  has  a 
distinctive  profile  in  addressing  the  protection  of  the  financial  interests  of  the  Union, 
which  sets  it  apart  from  other  programmes  of  DG  HOME  and  TAXUD.  Technical 
Assistance  was  particularly  highlighted  as  a  feature  that  is  not  offered  by  other  EU 
initiatives. While efforts have been made to coordinate the planning of the different EU 
programmes and some mechanisms are in place to ensure that this happens (e.g. the 
inter-service  consultation),  more  could  be  done  to  enhance  cooperation  and 
communication between DGs. 

December 2014    43 

  
 
 
 
Evaluation of the Hercule II Programme 

3.8.  Sustainability 

Sustainability concerns the Programme’s ability to maintain the positive effects of the 
various  interventions  after  the  end  of  support  (Evaluation  Question  7).  The 
responsibility  of  the  projects’  sustainability  lies  with  beneficiaries  as  they  are 
responsible for the project results after the grant period ends.  

Technical Assistance (Evaluation Question 7.1) 

Generally, the surveys show a strong consensus concerning the equipment being used 
after  a  project  has  ended.  71%  of  all  respondents  strongly  agree  that  the  technical 
equipment  is  being  used.  The  statements  are  vaguer  for  beneficiaries  from  customs 
authorities  where  only  60%  strongly  agree  compared  to  for  example  77%  in  law 
enforcement. 

Figure 20: To what extent do you agree with the following statement? - The technical 
equipment is used after the project has ended66 

When it comes to the skills of the people operating the equipment there is no doubt as 
to  whether  they  possess  the  necessary  qualifications,  since  90%  of  the  respondents 
either strongly agree or agree that the personnel have the necessary qualifications. 

It  could  further  be  examined  whether  there  are  any  barriers  for  Member  States  in 
obtaining  funding  for  training  in  how  to  operate  equipment  purchased  with  Technical 
Assistance.  However,  numerous  case  studies  suggest  that  providers  of  specialised 
technical equipment often provide training to staff on how to operate the equipment.  

Training, Seminars and Conferences (Evaluation Question 7.1) 

The participant survey concludes that 89% of the respondents either strongly agree or 
agree  that  they  have  been  or  expect  to  be  able  to  use  the  lessons  learned  from  the 
supported  event.  The  law  enforcement  authorities  stand  out  as  being  the  most 
positive.  However,  the  degree  of  respondents  strongly  agreeing  to  the  statement  is 
lower compared to the beneficiary survey.  

66 Source: Ramboll Management Consulting, Survey to beneficiaries. 

December 2014    44 

  
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

In  relation  to  networks,  88%  of  the  respondents  either  strongly  agree  or  agree  that 
networks have been maintained as a result of the training. 82% declare that they still 
make use of the network.  

As  can  be  seen  from  the  effectiveness  analysis  (section  3.1),  the  anti-fraud  training 
has  been  considered  as  the  most  effective  in  terms  of  sustaining  networks.  This  was 
also confirmed by the AFCOS case study where the ability to share the lessons learned 
across  borders  and  establish  networks  were  highlighted  as  key  outcomes  of  the 
conferences.   

Box 16: Conclusions on sustainability 

The  Hercule  II  Programme  is  overall  seen  as  being  sustainable.  The  beneficiaries 
assess that the purchased equipment and the operation of it is indeed sustainable. In 
relation  to  Training,  Seminars  and  Conferences,  the  participants  are  generally 
confident  that  the  lessons  learned  can  be  put  into  practice,  hence  sustainability  has 
been achieved. 

December 2014    45 

  
 
 
 
Evaluation of the Hercule II Programme 

4. Conclusions 

The  table  below  presents  the  evaluation´s  16  conclusion  boxes,  which  were  included 
under each section above. In cases were the evaluation identified recommendations in 
relation to the conclusions, these have highlighted in bold in the boxes below. 

Conclusions 

1 

2 

3 

4 

The  Hercule  II  Programme  enhanced  transnational  and  multidisciplinary 
cooperation,  however  primarily  between  Member  States.  Several  sources 
confirm  that  anti-fraud  training  and  training,  seminars  and  conferences 
with  a  legal  focus  were  the  main  drivers  in  achieving  this.  In  some 
instances  Technical  Assistance  also  contributed  to  enhanced  transnational 
and  multidisciplinary  cooperation.  The  Programme  also  effectively 
increased the beneficiaries’ and participants’ understanding of both EU and 
national mechanisms for protection of financial interests of the Union. The 
evaluation  findings  suggest  that  this  increased  understanding  of  the 
mechanisms  also  contributed  positively  to  Member  States  becoming 
increasingly aware of the benefits derived from enhanced cooperation. 

The  Hercule  II  Programme  has  been  effective  in  establishing  and 
strengthening  networks.  Both  informal  and  formal  networks  between 
Member  State  authorities,  the  Commission  and  OLAF  have  been 
established,  with  the  majority  of  the  networks  being  informal  in  nature. 
These  networks  support,  among  others,  faster  information  exchange 
between  national  authorities67,  increased  access  to  information  and 
improved  response  to  fraud.  In  addition,  the  Programme  has  contributed 
to  improved  cooperation  between  practitioners  and  academics  in  the 
Member  States.  Of  the  different  sectors  supported  within  the  Hercule  II 
Programme,  anti-fraud  training  has  been  the  most  effective  in  relation  to 
building  networks.  The  effects  of  the  Programme  could  be  further 
strengthened by ensuring more personal contact between different 
administrations, for example through staff exchanges. 

The Hercule II  Programme facilitated exchange of information, experience 
and  best  practice.  Several  sources  confirmed  that  the  beneficiaries  who 
were  conducting  anti-fraud  training  and  training,  seminars  or  conferences 
were  more  successful  in  facilitating  the  exchange  of  information, 
experience  and  best  practices.  The  exchange  of  information,  experience 
and  best  practice  which  took  place  during  events  funded  by  the 
Programme was the most successful. 
The  Hercule  II  Programme  delivered  the  intended  results,  namely 
strengthening  law  enforcement  and  customs  authorities  in  the  intended 
way,  that  is,  by  providing  Technical  Assistance.  The  combined  evidence 
indicates that it was achieved to a high extent and evidence shows that it 
was  generally  unlikely  that  the  equipment  would  have  been  purchased 
without the Programme’s co-financing. There were many examples of how 
the  Technical  Assistance  provided  law  enforcement  with  high  quality 
equipment that contributed towards  ensuring better collection of evidence 

67  This  information  exchange  does  not  refer  to  the  use  of  IT  tools,  but  to  the  fact  that  staff  in  national 
authorities  is  using  the  networks  to  contact  their  counterparts  in  other  Member  States  in  order  to 
spontaneously exchange information. 

December 2014    46 

  
 
 
 
 
 
 
 
                                           
Evaluation of the Hercule II Programme 

5 

6 

7 

8 

of corruption, fraud, and smuggling in Member States. In addition, findings 
showed  that  on  a  more  overall  level  Technical  Assistance  increased  the 
flexibility  of  the  law  enforcement  and  capacity  of  law  enforcement 
authorities in Member States. 
The Hercule II Programme has been effective in facilitating access to data 
in some Member States, whilst is has only contributed to a limited extent 
in  others.  The  findings  suggest  that  the  distributing  the  access  to 
data  through  national  contact  points  has  not  ensured  that  all 
relevant authorities within the Member States are benefitting from 
the access to data. Findings indicate that the same may be said for 
the  access  to  data  across  all  Member  States,  where  some  may  be 
benefitting more from the data than others. 

The Hercule II Programme developed features of the Automatic Monitoring 
Tool  (AMT)  which  improved  the  monitoring  and  intelligence  work  of 
national  law  enforcement  authorities.  The  tool  has  significantly  improved 
the  authorities’  ability  to  detect  undervaluation  of  imported  goods,  and  it 
has  increased  efficiency,  freeing  up  human  resources  and  time  for  other 
investigation  work.  The  AMT’s  potential  can  however  not  be  considered 
fully reached until the planned improvements have been finalised, thereby 
rendering  the  tool  more  useful  to  national  authorities  in  their  work  to 
prevent  losses  to  the  EU  budget.  In  addition,  the  Hercule  II  Programme 
provided  access  to  information  through  external  databases,  i.e.  CTI,  and 
thereby  helped  national  authorities  to  improve  their  investigations,  which 
would otherwise have been less successful. 
The  Hercule  II  Programme’s  increased  focus  on  cigarette  smuggling  and 
counterfeiting (compared to Hercule I) has led to an increased effort in this 
respect. This is not only shown by the multiple activities providing funding 
for  purchase  of  technical  equipment,  but  also  through  the  views  of  the 
beneficiaries  as  75%  of  the  survey  respondents  agreed  that  the 
Programme  had  been  successful  in  this  regard.  The  contribution  analysis 
supports  the  findings  in  terms  of  the  equipment  purchased,  but  also  in 
terms  of  the  training  activities,  which  have  been  aimed  at  increasing  the 
national  authorities’  capacity 
fight  cigarette  smuggling  and 
counterfeiting.  Only  limited  quantitative  evidence  exists  concerning 
the  effects  of  these  intensified  measures,  which  is  assessed  to  be 
mainly due to the lack of dedicated monitoring at the Member State 
level. 
The Hercule II Programme delivers complementary results within the three 
different  sectors  supported,  hence  it  is  not  possible  to  judge  whether  a 
specific sector has been more effective in contributing to the outcomes and 
impacts of the Programme than others. For example, Technical Assistance 
has  been  highly  effective  in  providing  strengthened  technical  and 
operational  support  for  law  enforcement  and  customs  authorities,  as  well 
as  in  improving  investigations,  monitoring  and  intelligence  work.  The 
sectors’  contribution  to  enhanced  transnational  and  multidisciplinary 
cooperation  between  Member  State  authorities  and  the  Commission  was 
very  limited  and  also  not  the  main  objective  of  providing  Technical 
Assistance.  Trainings,  seminars  and  conferences  have,  for  their  part, 
enhanced transnational cooperation between Member State authorities and 
strengthened networks and exchange of information, while they have been 
less effective in developing unified levels of skills, equipment and readiness 
in the Member States to detect smuggled and counterfeit products. The IT 

to 

December 2014    47 

  
 
 
 
 
Evaluation of the Hercule II Programme 

9 

10 

11 

12 

13 

Support  has,  in  support  of  the  Technical  Assistance  sector,  improved 
investigations, monitoring and intelligence work. 

While  all  the  Member  States  have  benefitted  from  the  Hercule  II 
Programme,  there  are  significant  differences  in  terms  of  both  amounts  of 
funding and numbers of actions carried out by the Member States. All the 
Member States have received Technical Assistance, with Spain, Poland and 
Hungary  being  the  biggest  beneficiaries  in  monetary  terms,  followed  by 
Belgium,  Germany,  Lithuania,  the  Netherlands,  Portugal,  Romania  and 
Finland. These Member States are all characterised by either busy ports of 
entry  into  the  European  Union  or  external  land  borders  of  the  European 
Union towards the East. Croatia, which is now a member of the EU, was a 
direct  beneficiary  of  the  Programme.  In  terms  of  trainings,  seminars  and 
conferences, funding has generally been allocated to newer Member States 
(Poland,  Hungary  and  Romania),  with  exception  of  Belgium,  France  and 
Germany,  where  numerous  training  providers  with  framework  contracts 
directly with OLAF are located. The objective for geographical balance has 
been abandoned for the Hercule III Programme. 

The Hercule II Programme shows indications of having reached its desired 
effects  at  reasonable  costs.  However,  as  is  visible  from  the  evaluations´ 
assessment  above,  it  is  not  possible  to  conclude  whether  the  Programme 
has led to significant efficiency gains. Therefore, additional monitoring data 
to specify the concrete cost-benefit ratio of, in particular for the Technical 
Assistance  sector,  is  needed  (please  see  section  3.3  for  more  information 
on  ex-post  monitoring).  The  evidence  points  to  an  efficient  project 
management 
in  terms  of  making  the  resources  available  to  the 
beneficiaries in due time, quantity and quality.  

However, the different experiences still suggest that there may be 
advantages  to  be  gained  from  sharing  experiences  between  the 
Member  States  on  the  type  of  equipment  used  and  how  it  is 
procured.  

Evidence  points  to  a  positive  assessment  of  the  organisational  change 
carried  out  in  2012  on  the  implementation  of  the  Hercule  II  Programme. 
Furthermore,  the  beneficiaries  are  overall  positive  about  the  application 
procedures  to  be  followed  within  the  Programme.  While  the  beneficiaries 
consider  the  reporting  requirements  to  be  relatively  easy  to  meet,  the 
evidence  showed  that  more  should  be  done  to  improve  the  reporting 
requirements.  By ensuring the right types of data is requested from 
the  beneficiaries  in  the  final  technical  report,  OLAF’s  ability  to 
monitor  the  results  and  impacts  of  the  activities  funded,  can  be 
improved. 
The  beneficiaries  and  participants  are  satisfied  with  the  Hercule  II 
Programme’s  ability  to  match  their  needs  at  the  Member  State  level.  The 
funded activities have a broader focus than the Member States’ needs, as 
they contribute to the protection of the financial interests of the Union. The 
Hercule  II  Programme  has  reached  its  different  objectives,  and  thus  it  is 
assessed  that  the  Programme  has  contributed  to  the  prevention  of  losses 
to the Unions budget. 
The  Hercule  II  Programme  is  accepted  by  beneficiaries  and  stakeholders. 
The  benefits  of  the  Programme  clearly  overshadow  the  costs  of 
participation. The Programme is particularly useful for customs authorities. 

December 2014    48 

  
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

14 

15 

16 

The  Hercule  II  Programme  has  been  successful  in  providing  EU  added 
value.  All  sources  confirm  this  assessment.  Generally  and  across  the 
different  types  of  actions  stakeholders  agree  that  the  objectives  achieved 
by  means  of  Hercule  funding  could  not,  or  to  a  lesser  extent,  have  been 
achieved with national or regional means. This applies to all three sectors 
of  the  Programme.  There  are  in  the  meantime  variations  between  the 
sectors  in  terms  of  the  types  of  added  value  that  the  different  actions 
in 
provide.  Training  and  conferences  are  particularly 
facilitating international cooperation and establishing networks. Meanwhile, 
the  tools  developed  and  provided  under  the  IT  Support  sector  and  the 
Technical Assistance actions provide more direct value to the investigations 
of  the  national  customs  and  law  enforcement  authorities.  As  such,  the 
Hercule  II  actions  clearly  contribute  to  the  protection  of  the  financial 
interests of the Union. 

instrumental 

Complementarity  in  the  preparation  and  implementation  of  the  Hercule  II 
Programme  has  been  sufficiently  ensured.  Stakeholders  generally  agree 
that Hercule II has a distinctive profile in addressing the protection of the 
financial interests of the Union, which sets it apart from other programmes 
of DGs HOME and TAXUD. Technical Assistance was particularly highlighted 
as a feature that is not offered by other EU initiatives. While efforts have 
been  made  to  coordinate  in  the  planning  of  the  different  EU 
programmes and that some mechanisms are in place to ensure that 
this  happens  (e.g.  the  inter-service  consultation)  more  could  be 
done to enhance cooperation and communication between DGs. 
The  Hercule  II  Programme  is  overall  seen  as  being  sustainable.  The 
beneficiaries assess that the purchased equipment and the operation of it, 
is  indeed  sustainable.  In  relation  to  Training,  Seminars  and  Conferences, 
the participants are generally confident that the lessons learned can be put 
into practice, hence sustainability has been achieved. 

December 2014    49 

  
 
 
 
 
 
Evaluation of the Hercule II Programme 

Annex E: List of references 

Anti-Contraband and Anti-Counterfeit Agreement and General Release dated as of July 9, 2004 
Among Philip Morris International Inc., Philip Morris Products Inc., Philip Morris Duty Free Inc., 
and  Philip  Morris  World  Trade  Sarl.,  The  European  Community  Represented  by  the  European 
Commission and Each Member State 

Commission of the European Communities (2006): Communication from the Commission on the 
implementation of the Hercule Programme and its extension during the period 2007-2013. COM 
(2006) 339 final, Brussels, 28.6.2006 

Commission  of  the  European  Communities  (2009):  Commission  Decision  of  28  April  1999 
establishing the European Anti-fraud  Office  (OLAF),  1999/352/EC, ECSC, Euratom  as  amended 
by Commission Decision 2013/478/EU 

Consolidated Version of the Treaty on the Functioning of the European Union (TFEU), Art. 325, 
2012, C 115/47. 

Council of the European Union (1995): Council Act of 26 July 1995 drawing up the Convention 
on  the  protection  of  the  European  Communities’  financial  interests  [Official  Journal  C  316  of 
27.11.1995] 

Council  of  the  European  Union  (2002):  Council  Regulation  (EC,  Euratom)  No  1605/2002 of  25 
June  2002  on  the  Financial  Regulation  applicable  to  the  general  budget  of  the  European 
Communities  (OJ  J  248  of  16/09/2002),  as  last  amended  by  Regulation  (EU,  Euratom)  n° 
1081/2010 of the European Parliament and of the Council of 24 November 2010 

Regulation  (EU,  Euratom)  no  966/2012  of  the  European  Parliament  and  of  the  Council  of 
25/10/2012  on  the  financial  rules  applicable  to  the  general  budget of  the  Union  and  repealing 
of  26/10/2012) 
Council  Regulation 

(EC,  Euratom)  No  1605/2002 

L298 

(OJ 

Commission  Delegated  Regulation  (EU)  No  1268/2012  of  29  October  2012  on  the  rules  of 
application of Regulation (EU, Euratom) No 966/2012 of the European Parliament  
and of the Council on the financial rules applicable to the general budget of the Union 
(OJ L 362, 31.12.2012, p. 1) 

Council  of  the  European  Union  (2011):  Commission  staff  working  paper.  Impact  assessment 
accompanying document to the Proposal for a Regulation of the European Parliament and of the 
Council on the Hercule III programme to promote activities in the field of the protection of the 
European Union’s financial interests. SEC (2011) 1610 final, 19.12.2011 

Decision No 804/2004/EC establishing a Community action programme to promote activities in 
the  field  of  the  protection  of  the  Community's  financial  interests  (Hercule  II  Programme),  as 
amended by Decision No 878/2007/EC of the European Parliament and of the Council of 23 July 
2007 [OJ L 143, 30.4.2004, p. 9–14] 

European  Commission  (2011):  Communication  from  the  Commission  to  the  European 
Parliament,  the  Council,  the  European  Economic  and  Social  Committee,  and  the  Committee  of 
the  Regions  and  the  Court  of  Auditors  on  the  Commission  Anti-fraud  Strategy  COM/2011/376 
final, Brussels, Brussels, 24.6.2011 

European  Commission  (2013):  Commission  Decision  of  7.2.2013  concerning  the  adoption  of  a 
financing decision  for 2013 in the framework of the Hercule II  Programme. C(2013) 612 final, 
Brussels, 7.2.2013 

European Commission (2014): Report from the Commission to the European Parliament and the 
Council.  Protection  of  the  European  Union’s  Financial  interests  —  Fight  against  fraud  2013. 
Annual Report. Brussels, 17.7.2014, COM(2014) 474 final 

December 2014    50 

  
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation of the Hercule II Programme 

European Commission (2014): Annual Overview with Information on the Results of the Hercule 
II Programme in 2012. SWD(2014) 247 final, Brussels, 17.7.2014  

Commission  Decisions  on  the  adoption  of  a  financing  decision  for  2007-2013  within  the 
framework  of  the  Community  action  programme  to  promote  activities  in  the  field  of  the 
protection of the Community's financial interests (Hercule II Programme) 

European  Commission  (2012):  Commission  Decision  of  21  February  2012  on  the  annual 
financing of the  expenditure of the "Community  action programme to promote activities in the 
field of the protection of the Community's financial interests (Hercule II  Programme)" for 2012 
(C(2012)/1057), 8.3 

December 2014    51 

  
 
 
 
 
 
Evaluation Methodology 

Annex A 

mmmll 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Table of Contents 

Overall framework: contribution analysis .............................................................. 3 
Definitions and terminology.............................................................................. 3 
Embedded theory of change ............................................................................. 5 
Evaluation questions ....................................................................................... 8 
Evaluation matrix............................................................................................ 9 
Surveys ...........................................................................................................24 
Survey to the beneficiaries ..............................................................................24 
Survey to the participants ...............................................................................29 
Stakeholder interviews ......................................................................................34 
Contribution analysis and case studies ................................................................35 
Steps  3  and  4:  develop  the  embedded  theory  of  change  and  assess  and  prioritise 
alternative explanations ..................................................................................35 
Step 5: Collect new data identified by prior steps...............................................39 
Step 6: Analyse and report on the contribution of Hercule II ...............................60 

December 2014 2 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Overall framework: contribution analysis 

The approach to the evaluation of the Hercule II Programme was based on the method 
known as “contribution analysis”. This method was selected because the main challenge 
in  the  evaluation  of  Hercule  II  concerns  the  establishment  of  causal  linkages,  i.e.  the 
contribution  of  the  actions  funded  by  Hercule  II  (known  as  “interventions”)  to  the 
results of the work to protect the financial interest of the Union. 

In  the  case  of  Hercule  II,  there  is  a  need  for  analysis  which  can  establish  causal 
contribution. In other words, this evaluation must be founded on an analytical approach 
which  can  identify  to  what  extent  Hercule  II  contributed  to  realising  its  objectives,  as 
specified in the Hercule II Decision (outcomes). In contribution analysis, this is done by 
using iterative theory building and testing of hypotheses and alternative explanations to 
indicate  attribution  of  outcomes  and  impacts.  Contribution  analysis  therefore  offers  a 
strong  alternative  to  classic  impact  evaluations  which  rely  on  quasi-experimental 
designs  that  are  not  feasible  for  this  evaluation.  Alternative  explanations  are 
mechanisms  or  factors  that  can  explain  the  emergence  (or  non-emergence)  of  an 
outcome or impact, unrelated to the programme, and should be examined.  

Contribution analysis structures the analysis around gathering information which helps 
generate  hypotheses  about  the  links  between  e.g.  activities  and  outcomes  and  then 
tests these against the  new data which is collected. Furthermore, potential alternative 
explanations  are  taken  into  consideration,  which  will  strengthen  the  validity  of  the 
evaluation  of  the  Hercule  II  Programme  because  it  provides  a  sound  analysis  of  how 
and to what extent these alternative explanations have helped or inhibited the ability of 
the  programme  to  reach  its  outcomes.  These  alternative  explanations  have  been, 
together  with  inhibitors  and  drivers,  been  included  in  an  embedded  theory  of  change, 
which shows how the activities of the Hercule II Programme in reality contribute to the 
objectives  of  the  programme.  The  contribution  analysis  is  described  in  more  detail  in 
the section concerning case studies. 

Definitions and terminology 

In the table below, key evaluation terms are described and explained, as employed in 
the final report and its annexes. 

Table 2: Definitions of key concepts and terms 

Term 
Intervention Logic  The  chain  of  assumptions  which  illustrates  how  change  is  to  be  achieved  in  an 

Definition 

Contribution 
analysis 

Inputs 

Activities 

Outputs 

Outcomes 

Impact 

Indicator 

intervention, and what the objectives are. Sometimes also called theory of change. 
Analysis  of  an  intervention’s  likely  contribution  to  overall  objectives,  taking  into 
account  the  strength  of  causal  mechanisms  and  assumptions  underpinning  the 
intervention. 
The  resources  used  to  carry  out  activities,  includes  financial,  human,  technical 
resources. 
The  activities  of  the  Hercule  II  programme  as  defined  by  Decision  No  878/2007/EC 
are the instruments, or the activities of the programme that are employed to bring 
about the desired outputs and outcomes: technical assistance; training, seminars and 
conferences;  and  IT  support  The  activities  are  under  direct  control  of  the 
programme/intervention. 
Outputs  are  the  direct  products  of  the  activities,  i.e.  a  scanner  is  purchased,  a 
conference is organised, access to IT products is purchased. The outputs are also part 
of what the programme controls directly. 
The  intermediate  effects  of  the  programme  which  enable  the  achievement  of  the 
impacts  of  the  programme.  The  outcomes  of  the  programme  are  the  desired  short 
and  medium  term  effects  –  resulting  from  the  activities  and  outputs.  The  results 
can be influenced directly by the intervention, but are not under direct control. 
The  impact  of  the  Programme  corresponds  to  what  in  Decision  No  878/2007/EC  is 
referred  to  as  the  “overall  objective”  of  the  programme.  It  is  the  wider  effect  or 
impact  that  the  intervention  aspires  to  ultimately  bring  about,  by  means  of  the 
programme  inputs,  activities,  outputs  and  outcomes.  The  impact  can  only  be 
influenced indirectly, and is to a large extent subject to external factors. 
A quantitative or qualitative measure, used as a tool to measure performance and 

December 2014 3 

 
 
 
 
 
 
 
Judgement 
criteria 
Baseline 

Effectiveness 
Efficiency 

Implementation 

Utility 
Acceptability 

EU added value 

Sustainability 
Complementarity 
Contextual driver 

Contextual 
inhibitor 

Internal driver 

Internal inhibitor 

Data collection 

Survey 

Population 

Semi-structured 
interview 

Explorative 
interview 
Structured 
interview 
Interview guide 

Case study 

Intervention 

 Evaluation of Hercule II Programme 

progress of outputs, outcomes and impacts produced by the intervention. 
Quantitative  values  and/or  normative  descriptors  of  how  judgement  will  be 
made on the success of an intervention. 
The situation before an intervention started, measured by collecting information 
on indicators prior to start of intervention 
The extent to which the objectives set for the programme are achieved. 
The  extent  to  which  the  desired  programme  effects  are  achieved  at  a  reasonable 
cost. 
Description and analysis of how the programme has been organised and managed by 
OLAF and the beneficiaries 
The usefulness of the achieved impacts of an intervention or programme. 
The  extent  to  which  we  can  observe  changes  in  the  perception  of  the  intervention 
(positive or negative) by the targeted stakeholders and/or by the general public 
EU added value is defined as the “value” resulting from an EU intervention that 
is additional to the “value” that would have resulted from intervention at national or 
regional levels. 
The likelihood of the effects of Hercule II to last after the programme has ended 
The extent to which Hercule II supports and usefully supplements other policies? 
Mechanisms or factors in the context of Hercule II, or the specific intervention (funded 
action),  external  to  the  programme,  that  can  have  affected  the  results  positively 
(instead of the programme/project itself) 
Mechanisms or factors in the context of Hercule II, or the specific intervention (funded 
action),  external  to  the  programme,  that  can  have  affected  the  results  negatively 
(instead of the programme/project itself) 
Internal  mechanisms  or  factors  within  the  beneficiary’s  or  OLAF’s  organisation, 
unrelated  to  Hercule  II,  that  can  have  affected  the  results  positively  (instead  of  the 
programme/project itself) 
Internal  mechanisms  or  factors  within  the  beneficiary’s  or  OLAF’s  organisation, 
unrelated to Hercule II, that can have affected the results negatively (instead of the 
programme/project itself) 
All activities undertaken to gather information for the evaluation, e.g. interviews, case 
studies, surveys. 
A  set  of  structured  questions  targeted  to  a  defined  population  or  sample  of 
population  in  order  to  gather  information,  opinions  and  perceptions  on  a  particular 
subject,  area  or  alike,  from  a  group  of  individuals.  Can  be  done  via  the  web, 
electronically, in writing, phone etc. 
The individuals to be targeted in the data collection. Either the whole population 
is targeted, or more often, a sample of the population is selected to represent the 
views of the whole population. 
An  interview  centred  on  themes,  ensuring  coverage  of  important  topics  with 
open-ended  questions,  leaving  it  open  for  the  respondent  to  add,  amend  and 
comment freely. 
An interview intended to generate understanding and knowledge of an area, with 
descriptive questions and, when relevant, perceptions of the respondent. 
An  interview  with  closed/highly  structured  questions,  on  very  specific  topics  or 
issues.  
A guide for the interviewer with instructions, themes and/or questions to follow 
during an interview. 
A targeted hypothesis, set of questions and data collection activities, serving 
to  examine  and  illustrate  a  specific  issue,  activity  or  theme.  Provides  evaluative 
knowledge and conclusions on the case in question, and is also used in overall 
evaluation  findings  to  analyse  the  contribution  of  Hercule  II  to  the  overall 
objectives of the programme. 
An intervention is a time-limited, targeted activity undertaken to bring about change 
and to reach a certain objective. 

December 2014 4 

 
 
 
 Evaluation of Hercule II Programme 

Embedded theory of change 

An intervention logic, or a theory of change, is intended to provide an overview of the 
overall  Programme  logic  –  how  the  Programme  activities  are  ultimately  intended  to 
contribute  to  the  long-term  impact  via  the  generation  of  outputs  and  outcomes.  The 
embedded  theory  of  change  builds  on  this,  but  shows  how  the  evidence  collected 
confirmed the Programme activities, their outputs and outcomes. 

During  the  inception  phase  a  preliminary,  embedded  theory  of  change  (TOC)  was  set 
out,  and  based  on  these  18  hypotheses  about  the  causal linkage  between  Programme 
activities, output and outcomes were set out in the interim report. 

Due  to  the  complexity  of  the  three  different  sectors  of  Hercule  II,  three  different  TOC 
have been developed, each providing an overview of which activities are documented to 
lead  to  certain  outputs  and  outcomes  in  the  sector  they  describe.  In  each  TOC  the 
linkages  for  which  there  was  the  strongest  evidence  are  illustrated  by  a  weighted 
arrow, whilst the linkages for which there was limited evidence are illustrated in a thin 
arrow.  

Contribution  stories  are  a  narrative  of  how  the  Programme  activities  contributed  to 
outputs, outcomes and ultimately the long term impact of the Programme. In total, six 
contribution  stories  are  presented  in  annexes  to  the  evaluation  and  these  provide  an 
important source of evidence in the new, embedded TOC, because apart from analysing 
the linkages, the contribution analysis also uncovered two types of influencing factors: 
One  type  is  drivers,  which  contribute  positively  to  the  Programme  achieving  its 
intended  results,  and  another  type  is  inhibitors,  which  hinder  the  Programme  in 
achieving  its  intended  results.  Whilst  the  contribution  stories  give  a  more  complete 
overview  of  the  Programme  activities’  effects  ‘on-the-ground’  in  Member  States,  as 
they are based on concrete case studies of Hercule II actions, these case studies  have 
also  been  supported  by  evidence  from  surveys,  secondary  data  and  interviews  with 
stakeholders (see Annex A). 

Technical Assistance  
The  evidence  confirmed  all  of  the  linkages.    Particularly  strong  evidence  showed  that 
support  for  Technical  Assistance  lead  to  technical  equipment  being  acquired,  and 
further  to  improved  investigations,  monitoring  and  intelligence  work.  Evidence  also 
showed  that  the  Technical  Assistance  lead  to  strengthened  technical  and  operational 
support for law enforcement and customs authorities. 

December 2014 5 

 
 
 
 
 
Figure 18: Embedded theory of change – Technical Assistance 

 Evaluation of Hercule II Programme 

A  number  of  influencing  factors  were  identified.  Two  external  inhibitors  were  the 
economic  crisis,  which  led  to  limited  national  budget  for  co-financing,  and  changing 
modus  operandi  of  criminals,  which  sometimes  meant  that  the  equipment  purchased 
was  less  effective  in  detecting  fraud  than  expected.  A  third  external  factor  was 
changing  trade  flows,  which  can  both  contribute  to  and  hinder  the  effectiveness  of 
equipment (depending on where it was placed). Internal to the Member States, national 
budgetary restrictions both contributed to and hindered participation in the Programme. 
The  reason  being,  that  limited  budgets  made  Hercule  II  a  more  important  source  of 
funding,  but  also  reduced  the  availability  of  national  co-financing.  No  plausible 
alternative explanations were identified. 

Training, Seminars and Conferences 
All of the intended linkages were confirmed, but to very different extents. Amongst the 
strongest links from activity to output was the link showing that trainings, seminars and 
conferences  helped  Member  States  coordinate  activities 
leading  to  enhanced 
cooperation,  as  well  as  enhanced  understanding  of  EU  and  national  mechanisms.  In 
addition, the evaluation demonstrates how an increase in the exchange of experiences 
could  be  shown  to  enhance  cooperation.  Despite  strong  evidence  that  Member  State 
officials  and  other  actors  received  training  to  fight  against  fraud,  there  is  less  strong 
evidence showing that it in fact led to the intended outcomes.  

December 2014 6 

 
 
 
 
Figure 19: Embedded theory of change – trainings, seminars and conferences 

 Evaluation of Hercule II Programme 

The  most  notable  external  influencing  factor  was  the  participation  of  experts,  which 
showed  to  significantly  increase  the  extent  to  which  experiences  were  exchanged. 
Although  an  internal  influencing  factor,  namely  the  organisers’  experience  with 
implementing trainings, seminars or conferences, could increase the participation of the 
right  experts  (i.e.  those  with  substantial  knowledge  of  the  subject  matter  at  hand),  it 
remains outside the control of  organisers and  the Programme who, in the  end, attend 
the  events.  Another  internal  influencing  factor  was  the  limited  national  funds  for 
organising international events, which increased Member States interest in applying for 
funding under Hercule II. No plausible alternative explanations were identified. 

IT Support 
Although  all  intended  linkages  were  confirmed,  the  Programme’s  most  significant 
achievement  through  the  IT  Support  was  to  facilitate  access  to  data  (and  to  a  lesser 
extent exchange of data). The data derived from the databases, which were funded by 
Hercule  II,  could  be  shown  to  have  been  particularly  successful  at  improving 
investigations,  monitoring  and  intelligence.  Another  strong  linkage  was  that  the 
facilitated  data  access  and  exchange  led  to  multiplied  and  intensified  measures,  in 
particular in the field of cigarette smuggling and counterfeiting.  

December 2014 7 

 
 
 
 
 
Figure 20: Embedded theory of change – IT Support 

 Evaluation of Hercule II Programme 

The  Programme’s  achievements  were  influenced  by  the  quality  of  the  data  available 
(from  providers);  the  more  accurate  and  timely  this  was,  the  more  useful  it  was  for 
authorities,  for  example,  monitoring  trade  flows.  Two  factors  internal  to  the  national 
administrations  were  also  noted,  namely  that  a  lack  of  awareness  of  the  IT  Support 
may  be  hindering  all  Member  States  (and  different  authorities  within  them)  from  fully 
benefitting  from  the  Programme’s  IT  Support.  Next,  the  national  capacity  to  use  data 
from those databases could either strengthen or weaken the achievement of intensified 
measures  against  cigarette  smuggling,  or  intelligence  work.  This  was  illustrated  by 
differences in usage across Member States. 

Evaluation questions 

the  evaluation  criteria  of  effectiveness,  efficiency, 
The  evaluation  covers 
implementation,  utility,  acceptability,  EU  added  value, 
sustainability  and 
complementarity.  The  main  evaluation  questions  were  operationalised  in  the  inception 
phase  and  more  specific  sub-questions  were  developed.  These  can  be  found,  together 
with  the  relevant  indicators,  judgement  criteria  and  data  collection  tools  in  the 
below68.
evaluation 

presented 

matrix 

68 Please note that in the main body of the report the order of evaluation questions has 
been  modified  in  such  a  way  that  the  chapters  on  sustainability  and  complementarity 
have been reversed. 

December 2014 8 

 
 
 
 
 
 
 
                                           
Evaluation matrix 

Table 3: Evaluation matrix 

Criteria 

Effectivenes
s 

Evaluation 
question 
Q.1 
To  what  extent 
has  the  Hercule 
Programme 
II 
its 
reached 
objective 
of 
promoting 
activities  in  the 
the 
field 
of 
protection 
of 
the 
financial 
interests  of  the 
Union? 

Operationalis
ation 
Q.1.1 
To  what  extent 
has 
the 
Programme 
enhanced 
transnational 
and 
multidisciplinar
y  cooperation 
between 
Member 
States’ 
authorities,  the 
Commission 
and OLAF? 

Q.1.2 
To  what  extent 
has 
the 
Programme 
built  networks 
throughout  the 
MS,  acceding 
countries  and 

Indicator/descriptor 

Share  of  beneficiaries  considering 
that  the  Programme  has  increased 
cooperation 

of 

beneficiaries 

Share 
AND 
participants  considering  that  the 
Programme 
increased 
understanding  of  EU  and  national 
mechanisms/by  type  of  instrument 
and type of participant 

has 

Number  of  trainings,  seminars  and 
conferences 

Number of joint operations 

Number of staff exchanges 

Share  of  beneficiaries  considering 
that  the  Programme  has  increased 
cooperation 

Share  of  practitioners  AND  of 
academics 
in 
Programme  activities  who  consider 
cooperation  between 
that 

participating 

the 

 Evaluation of Hercule II Programme 

judgment 

Norm/ 
criteria 
70% 
the 
of 
beneficiaries  consider 
that  the  Programme 
increased 
has 
cooperation 

70% 
the 
of 
beneficiaries  and  70% 
participants 
of 
the 
the 
consider 
that 
Programme 
has 
increased 
understanding  of  EU 
and 
national 
mechanisms 

Data 
collection 
Monitoring  data 
from 
final 
reports 

Analytical 
strategy 
Descriptive 
data 

Qualitative 
Data Analysis 

Contribution 
Analysis 

Survey 
beneficiaries 

of 

of 
Survey 
training/confere
nce participants 

Stakeholder 
interviews 

Case studies 

70% 
the 
of 
beneficiaries  consider 
that  the  Programme 
has 
increased 
cooperation 

Survey 
beneficiaries 

of 

Descriptive 
data 

Monitoring  data 
final 
from 
reports 

Qualitative 
Data Analysis 

70%  consider  that  the 
cooperation  between 

Survey 

of 

Contribution 
Analysis 

December 2014 9 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Criteria 

Evaluation 
question 

Operationalis
ation 
candidate 
countries? 

Q.1.3 
To  what  extent 
the 
has 
programme 
facilitated 
exchange 
information, 
experience  and 
best practices? 

the 
of 

Q.1.4 
To  what  extent 
has 
the 
Programme 
provided 
technical  and 
operational 
support  for  the 
law 
enforcement 
authorities  of 
the  MS  in  their 

Indicator/descriptor 

practitioners  and  academics  has 
the 
improved  as  a 
Programme 

result  of 

Number  of  networking  activities 
during the course of the Programme 

Number of networks created 

Number  of  networks  existing  at  the 
end of the Programme 

Number of legal studies published 

Number of research projects carried 
out 

Number of comparative law reviews 
published 

of 

the 
views 
Qualitative 
beneficiaries  of  the  contribution  of 
the  technical  equipment  to  the 
numbers 
seizures, 
convictions  and/or  financial  impact 
of seizures. 

arrests, 

of 

Share  of  beneficiaries  who  consider 
that the training of personnel in the 
use  of  the  technical  equipment  is  a 
contributing  factor  to  the  increases 
(if any) in the numbers. 

 Evaluation of Hercule II Programme 

Analytical 
strategy 

judgment 

Norm/ 
criteria 
practitioners 
and 
has 
academics 
improved as a result of 
the Programme 

50%  of  the  networks 
the 
created  during 
course 
the 
Programme  exist  also 
at 
the 
the  end  of 
Programme 

of 

Data 
collection 
training/confere
nce participants 

Stakeholder 
(in 
interviews 
particular  MS, 
acceding 
countries 
candidate 
countries) 

and 

Case studies 

Increased  numbers  of 
arrests* 

Monitoring  data 
from 
final 
reports 

Descriptive 
data 

Increased  numbers  of 
seizures  of  illicit  and 
smuggled goods* 

Increased  numbers  of 
convictions* 

Increased 
financial 
impact  of  seizures  of 

Survey 
beneficiaries 

of 

Interviews  with 
selected 
beneficiaries, 
including  both 
customs 

Qualitative 
Data Analysis 

Contribution 
Analysis 

December 2014 10 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Criteria 

Evaluation 
question 

against 
cross 

Operationalis
ation 
fight 
illegal 
border 
activities, 
emphasizing 
support 
customs 
authorities? 

for 

Indicator/descriptor 

Number of devices purchased 

Numbers of arrests 

Numbers  of  seizures  of  illicit  and 
smuggled goods 

Estimated 
seizures  of 
goods 

financial 

of 
illicit  and  smuggled 

impact 

Numbers  of  convictions  that  were 
made  possible  with  equipment 
funded under the programme 

 Evaluation of Hercule II Programme 

Analytical 
strategy 

Data 
collection 
authorities  and 
other 
beneficiaries 

Case studies 

judgment 

Norm/ 
criteria 
illicit  and  smuggled 
goods*69 

70% 
the 
of 
beneficiaries  consider 
that  the  increases  in 
the  numbers  are  the 
result  of  the  technical 
equipment purchased 

the 

70% 
the 
of 
beneficiaries  consider 
arrests, 
that 
seizures, 
convictions 
and/or financial impact 
of  seizures  could  not 
reached 
have  been 
without  the  technical 
equipment purchased 

the 
of 
70% 
beneficiaries  consider 
that  the  training  of 
personnel in the use of 
technical 
the 
equipment 
a 
contributing  factor  to 
the  increases  (if  any) 
in the numbers. 

is 

69  The  judgement  criteria  marked  with  *  will  be  supplemented  with  baseline  data  in  the  data  collection  phase  to  ensure  that  an  increase  can  be 
documented. 

December 2014 11 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Evaluation 
question 

Operationalis
ation 

Indicator/descriptor 

Norm/ 
criteria 

judgment 

Data 
collection 

Analytical 
strategy 

 Evaluation of Hercule II Programme 

Share  of  beneficiaries  who  consider 
that  the  IT  tools  developed  and 
provided  through  the  Programme 
have  facilitated  data  access  and 
analysis 

Share  of  beneficiaries  who  consider 
increased 
that  data  exchange 
through the Programme 

for 

Share  of  beneficiaries  who  consider 
that  the  IT  tools  developed  and 
provided 
investigations, 
monitoring  and  intelligence  work 
law 
provided  support 
their 
enforcement  authorities 
fight  against  illegal  cross  border 
activities 

the 
in 

for 

Hit  rate  (statistics  on  usage)  of  the 
databases 

Q.1.5 
To  what  extent 
have 
the 
specific 
databases  and 
IT 
tools 
provided 
through 
Programme 
facilitated  data 
access 
and 
analysis? 

the 

the 

Q.1.6 
To  what  extent 
IT 
have 
tools  provided 
for 
investigations, 
monitoring  and 
intelligence 
work  provided 
support  for  the 
law 
enforcement 
authorities 
their 
against 
cross 
activities? 

in 
fight 
illegal 
border 

Monitoring  data 
from 
final 
reports 

Descriptive 
data 

Qualitative 
Data Analysis 

Contribution 
Analysis 

Survey 
beneficiaries 

of 

Interviews  with 
selected 
beneficiaries, 
including  both 
customs 
authorities  and 
other 
beneficiaries 

Case studies 

IT 

the 
of 
70% 
beneficiaries  consider 
tools 
that 
the 
and 
developed 
provided  through  the 
Programme 
have 
facilitated  data  access 
and analysis 

70% 
the 
of 
beneficiaries  consider 
that  data  exchange 
increased  through  the 
Programme 

IT 

the 
of 
70% 
beneficiaries  consider 
tools 
that 
the 
and 
developed 
for 
provided 
investigations, 
monitoring 
and 
intelligence 
work 
for 
provided  support 
the  law  enforcement 
authorities 
their 
in 
fight  against 
illegal 
cross border activities 

Evidence  of  active  use 
of  the  databases  by 

December 2014 12 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Criteria 

Evaluation 
question 

Operationalis
ation 

Indicator/descriptor 

a 

by 
all 

Q.1.7 
To  what  extent 
has 
the 
Programme 
stricken 
geographical 
balance 
including 
Member 
States, 
Acceding 
States 
candidate 
in 
countries 
the 
activities 
financed  under 
the 
Programme? 
Q.1.8 
To  what  extent 
has 
the 
Programme 
multiplied  and 

and 

Number  of  projects  in  each  sector 
per  Member  State,  Acceding  State 
and candidate country 

Number  of  participants  in  projects 
(f.ex.  participants  in  trainings)  per 
Member  State,  Acceding  State  and 
candidate country 

(grant  or 
funding 
Volume  of 
contract) in each sector per Member 
State, Acceding State and candidate 
country 

Share  of  beneficiaries  who  consider 
that  the  technical  equipment  has 
increased the detection of smuggled 
and/or 
cigarettes 
counterfeit products 

tobacco, 

or 

 Evaluation of Hercule II Programme 

judgment 

Norm/ 
criteria 
the  Member  State 
authorities 

broad 
Evidence 
of 
of 
representation 
Member 
States, 
Acceding  States  and 
candidate  countries  as 
participants 
project 
in 
(for 
training 
and 
conferences) 

example 

Data 
collection 

Analytical 
strategy 

Monitoring  data 
final 
from 
reports 

Descriptive 
data 

Qualitative 
Data Analysis 

Interviews  with 
stakeholders70 

Case studies 

Increased  numbers  of 
arrests* 

Monitoring  data 
from 
final 
reports 

Descriptive 
data 

Increased  numbers  of 
seizures  of  smuggled 

Survey 

of 

Qualitative 
Data Analysis 

70 In particular to uncover the reasons for participation and/or non-participation of Member States in the activities funded by the different sectors of 
Hercule II. 

December 2014 13 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
 Evaluation of Hercule II Programme 

judgment 

Norm/ 
criteria 
cigarettes 
and 
counterfeit products*71 

Data 
collection 
beneficiaries 

Analytical 
strategy 

Contribution 
Analysis 

Criteria 

Evaluation 
question 

Operationalis
ation 
intensified  the 
in 
measures 
areas 
the 
as 
identified 
most 
the 
sensitive, 
in 
particular 
cigarette 
smuggling  and 
counterfeiting? 

Indicator/descriptor 

Share  of  beneficiaries  who  consider 
that  the  training  has  increased  the 
skills  of  customs  staff  in  detecting 
smuggled  cigarettes  or  tobacco, 
and/or counterfeit products 

Number of devices purchased 

Numbers of arrests 

Numbers  of  seizures  of  smuggled 
cigarettes and counterfeit products 

Contribution  of  activities  within  the 
different  sectors  to  the  outcomes 
and impacts of the Programme 

Q1.9 
some 
Have 
and 
sectors 
types 
of 
activities  been 
more  effective 
in  reaching  the 
objectives 
of 
Hercule  II than 
others? 

that 
in 

70  %  of  beneficiaries 
the 
consider 
increases 
the 
numbers are the result 
of 
technical 
equipment purchased 

the 

70%  of  beneficiaries 
consider  that  training 
has increased the skills 
of  customs  staff 
in 
detecting 
smuggled 
cigarettes  or  tobacco 
counterfeit 
and/or 
products 
The  evaluation  results 
indicate  that  a  specific 
sector  has  been  more 
in 
successful 
contributing 
the 
outcomes  and  impacts 
Programme 
of 
than others 

the 

to 

Interviews  with 
selected 
beneficiaries 

Interviews  with 
OLAF’s 
Task 
Group 
Cigarette’s 
partners 

Case studies 

All 
relevant 
data  collection 
activities 

Contribution 
Analysis 

71  The  judgement  criteria  marked  with  *  will  be  supplemented  with  baseline  data  in  the  data  collection  phase  to  ensure  that  an  increase  can  be 
documented. 

December 2014 14 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Efficiency 

Operationalis
ation 
Q.2.1 
To  what  extent 
have 
the 
desired  effects 
been  achieved 
at 
reasonable 
costs? 

Evaluation 
question 
Q.2 
Could  the  same 
effects 
have 
been  achieved 
with lower  costs 
if 
different 
instruments 
currently  used 
the 
for 
implementation 
of 
the 
Programme 
were applied? 

Q.2.2 
To  what  extent 
savings 
have 
derived 
been 
from 
the 
collective 
procurement of 
specialised 
equipment? 

Indicator/descriptor 

Average cost of training per person 

Estimated 
seizures  of 
goods 

financial 

of 
illicit  and  smuggled 

impact 

Estimated 
of 
financial 
detected  fraud  related  to  funding 
and corruption 

impact 

Ratio 
impact/cost  

estimated 

financial 

Share  of  beneficiaries  who  consider 
that  the  input/output  ratio  was 
reasonable 

Cost  per  beneficiary 
collective  procurement 
specialised equipment 

for 
of 

the 
the 

Cost  for  the  procurement  of  the 
specialised  equipment  if  not  carried 
out collectively 

Cost  per  registered  user  of  a 
database in relation to the total cost 
of the procurement 

Norm/ 
criteria 
Estimated 
impact > cost 

judgment 

financial 

70% 
the 
of 
beneficiaries  consider 
the  input/output  level 
was reasonable 

 Evaluation of Hercule II Programme 

Data 
collection 
Monitoring  data 
final 
from 
reports 

Survey 
beneficiaries 

of 

Case studies 

Analytical 
strategy 
Quantitative 
Data Analysis 

Descriptive 
data 

Qualitative 
Data Analysis 

the 

Cost  per  beneficiary 
for 
collective 
procurement  <  cost  if 
the  procurement  had 
not  been  carried  out 
collectively72 

Monitoring  data 
from 
final 
reports 

Quantitative 
Data Analysis 

Descriptive 
data 

Survey 
beneficiaries 

of 

Case studies 

72  Information  on  the  cost  if  the  procurement  had  not  been  carried  out  collectively  will  be  gathered  during  the  case  studies  concerning  training 
activities. Interviews with training organisers and training participants are foreseen to provide estimates, which will be verified by contacts to training 
organisers non-affiliated with the Hercule II Programme. 

December 2014 15 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Evaluation 
question 

Operationalis
ation 
Q.2.3 
To  what  extent 
savings 
have 
derived 
been 
the 
from 
collective 
procurement of 
databases 
to 
be  used  by 
stakeholders? 
Q.2.4 
To  what  extent 
savings 
have 
derived 
been 
from 
the 
specialised 
training 
organised 
jointly73? 
Q.2.5 
Have 
resources 
achieve 
objectives 
made 
been 
available to the 
beneficiaries 
-  In due time 
-  In 

the 
to 
the 

Indicator/descriptor 

Cost  per  beneficiary 
the 
collective procurement of databases 

for 

Cost  for  the  procurement  of  the 
carried  out 
databases 
collectively 

if  not 

the 

judgment 

Norm/ 
criteria 
Cost  per  beneficiary 
collective 
for 
procurement  <  cost  if 
the  procurement  had 
not  been  carried  out 
collectively 

Cost  per  participant  for  specialised 
training  organised  jointly  (digital 
forensics  training  used  as  proxy 
indicator) 

Cost  per  participant  in 
digital 
forensics 
training  <  cost  if  the 
training  had  not  been 
organised jointly 

Cost  for  the  specialised  training  if 
not organised jointly 

Share  of  beneficiaries  who  state 
that  they  received  the  payments 
according  to  the  payment  schedule 
set in the grant agreement 

Share  of  beneficiaries  who  consider 
that  the  grant  received  from  OLAF 
was  sufficient 
for  the 
project to reach its objectives 

in  order 

90% 
the 
of 
beneficiaries state that 
they 
the 
received 
payments  according  to 
the  payment  schedule 
grant 
set 
agreement  

the 

in 

of 
70% 
beneficiaries 

the 
agree 

 Evaluation of Hercule II Programme 

Analytical 
strategy 
Quantitative 
Data Analysis 

Descriptive 
data 

Data 
collection 
Monitoring  data 
final 
from 
reports 

Survey 
beneficiaries 

of 

Case studies 

Monitoring  data 
from 
final 
reports 

Quantitative 
Data Analysis 

Descriptive 
data 

Survey 
beneficiaries 

of 

Case studies 

Survey 
beneficiaries 

of 

Quantitative 
Data Analysis 

Interviews  with 
OLAF 

Descriptive 
data 

Case studies 

Qualitative 
Data Analysis 

73 With the word “jointly” we refer to the training being organised in the way that representatives of several Member States can join it, as opposed to 
training organised in one Member State, in the national language, and only being open for participation for persons from that Member State. 

December 2014 16 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Evaluation 
question 

Operationalis
ation 

appropriate 
quantity 

-  In 

appropriate 
quality 

Indicator/descriptor 

Opinion  of  Olaf  on  whether  the 
resources were made available  
-  In due time 
-  In appropriate quantity  
-  In appropriate quality 

judgment 

Norm/ 
criteria 
that the grant received 
was 
OLAF 
from 
sufficient  in  order  for 
the project to reach its 
objectives 

Representatives 
of 
Olaf  consider  that  the 
resources  were  made 
available  
-  In due time 
-  In appropriate 

quantity 

-  In appropriate 

quality 

Implementat
ion 

Q.3 
How  is  Hercule 
II implemented? 

Q.3.1 
How  is  Hercule 
II implemented 
at  the  level  of 
OLAF? 

Share  of  beneficiaries  who  consider 
that  the  selection  procedures  are  a 
contributing  factor  to  the  projects 
reaching their objectives 

The 
selection 
procedures  take  into 
account  the  objectives 
of the Programme 

consider 

Qualitative  views  of  stakeholders 
who 
the 
administrative  arrangements  are  a 
contributing 
the 
Programme reaching its objectives 

factor 

that 

to 

the 

Share  of  beneficiaries  who  consider 
that 
administrative 
arrangements  are  a  contributing 
factor to the projects  reaching their 
objectives 

the 
of 
70% 
beneficiaries  consider 
selection 
that 
the 
procedures 
a 
are 
contributing  factor  to 
the  projects  reaching 
their objectives 

Stakeholders  consider 
that the administrative 
arrangements  are  a 
contributing  factor  to 

 Evaluation of Hercule II Programme 

Data 
collection 

Analytical 
strategy 

Descriptive 
data 

Qualitative 
Data Analysis 

Contribution 
Analysis 

research 
selection 

Desk 
of 
procedures 
(calls 
for 
proposal)  and 
administrative 
arrangements 

Survey 
beneficiaries 

with 

Interviews  with 
stakeholders, 
including OLAF 

Case studies 

December 2014 17 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Criteria 

Evaluation 
question 

Operationalis
ation 

Indicator/descriptor 

Time  spent  by  administrators  on 
vs. 
programme  management 
surveillance 
for 
investigative functions 

support 

and 

Financial resources available for the 
management of the Programme 

Human  resources  available  for  the 
management of the Programme 

Utility 

Q.4 
To  what  extent 
did  the  results 
of 
the 
interventions 
correspond  with 
needs, 
the 
and 
problems 

Q.4.1 
To  what  extent 
did  the  results 
of  the  projects 
implemented 
to 
correspond 
the 
needs, 
problems  and 

Share  of  beneficiaries  who  consider 
that  the  project  results  correspond 
to  the  needs,  problems  and  issues 
in 
the 
their  Member  State  at 
moment of Programme adoption 

 Evaluation of Hercule II Programme 

Data 
collection 

Analytical 
strategy 

judgment 

Norm/ 
criteria 
the 
Programme 
reaching its objectives 

70% 
the 
of 
beneficiaries  consider 
that the administrative 
arrangements  are  a 
contributing  factor  to 
the  projects  reaching 
their objectives 

current 

The 
implementation 
system 
for 
results  monitoring  and 
follow up of projects 

allows 

current 

gather 
to 

The 
implementation 
system  has  the  ability 
lessons 
to 
learned 
further 
improve 
implementation 
70%  of  beneficiaries 
consider 
the 
results 
project 
correspond 
the 
needs,  problems  and 
issues in their Member 
State  at  the  moment 
Programme 
of 

that 

to 

Survey 
beneficiaries 

of 

Descriptive 
data 

Survey 
of 
training/confere
nce participants 

Qualitative 
Data Analysis 

Stakeholder 

Contribution 
Analysis 

December 2014 18 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Criteria 

Evaluation 
question 
issues 
to  be 
at 
addressed 
the  moment  the 
Programme  was 
adopted? 

Operationalis
ation 
issues 
experienced  by 
the 
Member 
States  at  the 
of 
moment 
Programme 
adoption? 

 Evaluation of Hercule II Programme 

Indicator/descriptor 

Norm/ 
criteria 
adoption 

judgment 

Data 
collection 
interviews 

Analytical 
strategy 

Monitoring  data 
from 
final 
reports 

Case studies 

Q.4.2 
To  what  extent 
did  the  results 
of  the  projects 
and/or 
Programme 
to 
correspond 
the 
needs, 
problems  and 
issues 
experienced  at 
the  Union  level 
at  the  moment 
of  Programme 
adoption? 
Q.5.1 
How  do 
beneficiaries 
assess 
cost-benefit 
ratio 

the 

the 

of 

Share  of  stakeholders  who  consider 
that  the  project  and/or  Programme 
results  correspond  to  the  needs, 
problems  and  issues  at  the  Union 
level at the moment the Programme 
was adopted 

to 

Stakeholders  consider 
that the project and/or 
results 
Programme 
correspond 
the 
needs,  problems  and 
issues  at  the  Union 
level  at  the  moment 
the  Programme  was 
adopted74 

Share  of  beneficiaries  who  consider 
the  Programme  to  be  useful  and/or 
are satisfied with the Programme 

Share  of  stakeholders  who  consider 
the Programme to be useful 

70% 
the 
of 
beneficiaries  consider 
the  Programme  to  be 
useful, 
into 
account  the  costs  and 
of 
benefits 

taking 

Survey 
beneficiaries 

of 

Descriptive 
data 

Survey 
of 
training/confere
nce participants  

Qualitative 
Data Analysis 

Acceptability  Q.5 

To  what  extent 
do 
the 
stakeholders 
accept 
the 
Programme  and 

74 No judgement criteria in % has been indicated, as it is not foreseen that the number of stakeholders interviewed will allow for a judgement of their 
responses based on percentages. A more qualitative assessment will be used instead. 

December 2014 19 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Indicator/descriptor 

Average 
person75 

cost  of 

training  per 

Criteria 

Evaluation 
question 
consider it to be 
in 
useful, 
particular  when 
an  analysis  of 
the  costs  and 
benefits is made 
the 
from 
perspective 
of 
the 
beneficiaries? 

Operationalis
ation 
participating  in 
the 
Programme? 

Q.5.2 
How  do 
the 
participants  in 
training, 
seminars 
and/or 
conferences 
assess 
cost-benefit 
ratio 
of 
participating  in 
the 
Programme? 

the 

 Evaluation of Hercule II Programme 

Analytical 
strategy 
Quantitative 
Data Analysis 

Data 
collection 

Stakeholder 
interviews 

Monitoring  data 
from 
final 
reports 

Case studies 

Survey 
beneficiaries 

of 

Qualitative 
Data Analysis 

Stakeholder 
interviews 

judgment 

Norm/ 
criteria 
participation 

the 
70% 
of 
are 
beneficiaries 
the 
satisfied  with 
programme, 
taking 
into  account  the  costs 
and 
of 
benefits 
participation 

in 

70% 
the 
of 
participants  find  their 
the 
participation 
programme  useful  or 
very useful, taking into 
account  the  costs  and 
of 
benefits 
participation 

Stakeholders  consider 
the  Programme  to  be 
useful 
70% 
the 
of 
beneficiaries  consider 
the  objectives 
that 
could best be achieved 
at the EU level  

EU 
value 

added 

the 

Q.6 
Have 
interventions 
allowed 
achieving 
objectives 
that 
could  not,  or  to 

for 

N/A  analytical 
question 

Opinions  of  the  beneficiaries  and 
stakeholders 
the 
objectives  could  have  been  better 
achieved  at  national  or  regional 
level 

of  whether 

Share  of  participants  in  training, 

Stakeholders  consider 

Case studies 

75 The average cost of digital forensics training per person will be used in the case studies to enquire into the cost and benefit ratio of the training from 
the point of view of the beneficiaries and participants. 

December 2014 20 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Evaluation 
question 
a  lesser  extent, 
be  achieved  by 
interventions 
undertaken 
national 
regional level? 

at 
or 

Operationalis
ation 

Indicator/descriptor 

seminars  and/or  conferences  who 
consider  that  they  could  not  have 
received  the  same  benefits  through 
activities  organised  and  funded  at 
national or regional level. 

Evidence  of  the  projects  having 
provided EU added value 

 Evaluation of Hercule II Programme 

judgment 

Norm/ 
criteria 
that 
the  objectives 
could best be achieved 
at the EU level76 

Data 
collection 

Analytical 
strategy 

the 
of 
70% 
in 
participants, 
seminars 
training, 
conferences 
and/or 
they 
that 
consider 
have 
could 
same 
received 
through 
benefits 
organised 
activities 
and  funded  at national 
or regional level 

not 

the 

Sustainabilit
y 

Q.7 
To  what  extent 
are  the  positive 
effects  of 
the 
intervention 
likely 
after 

last 
the 

to 

Q.7.1 
To  what  extent 
is  the  technical 
equipment 
used  after  the 
intervention 
has ended? 

financial 

of 
Estimated 
seizures  of 
illicit  and  smuggled 
goods  during  the  intervention  and 
after the intervention has ended*77 

impact 

Estimated 
of 
financial 
detected  fraud  related  to  funding 

impact 

results 
The  project 
correspond  with 
the 
EU  added  value  plans 
in the project proposal 
Case studies show that 
the 
technical 
equipment  purchased 
is  in  active  use  and 
has  been  maintained 
after  the  intervention 
has ended 

Monitoring  data 
from 
final 
reports 

Descriptive 
data 

Survey 
beneficiaries 

of 

Qualitative 
Data Analysis 

Contribution 

76 No judgement criteria in % has been indicated, as it is not foreseen that the number of stakeholders interviewed will allow for a judgement of their 
responses based on percentages. A more qualitative assessment will be used instead. 
77  The  judgement  criteria  marked  with  *  will  be  supplemented  with  baseline  data  in  the  data  collection  phase  to  ensure  that  an  increase  can  be 
documented. 

December 2014 21 

 
 
 
 
 
 
 
 
 
 
 
                                           
Criteria 

Evaluation 
question 
intervention  has 
ended? 

Operationalis
ation 

Q.7.2 
To  what  extent 
are 
the 
beneficiaries 
using 
the 
lessons learned 
from 
participation  in 
Hercule 
activities  

Complement
arity 

Q.8 
To  what  extent 
are 
complementarit
y  and  lack  of 
overlaps 
ensured 
between 

are 

Q.8.1 
How 
complementari
ty  and  lack  of 
overlaps 
ensured 
between 
Hercule  II  and 

Indicator/descriptor 

and corruption* 

Evidence  of  active  use  of  the 
technical equipment  

the 

lessons 

Share  of  beneficiaries  who  state 
that 
from 
participation  in  joint  operations  or 
staff  exchanges  are  being  used 
after the intervention has ended 

learned 

Share  of  training,  seminar  and/or 
who 
conference 
consider  that  they  have  been  able 
to  use  the  lessons  learned  in  their 
work after the training/conference 

participants 

Opinions  of  stakeholders  and  OLAF 
the 
on 
the  way 
complementarity 
of 
overlaps have been ensured 

in  which 
and 

lack 

 Evaluation of Hercule II Programme 

Analytical 
strategy 
Analysis 

Data 
collection 
Survey 
of 
training/confere
nce participants 

Case studies 

Norm/ 
criteria 

judgment 

Case studies show that 
the  beneficiaries  are 
able  to  operate  the 
equipment 
technical 
purchased 
70% 
the 
of 
beneficiaries state that 
learned 
lessons 
the 
from  participation 
in 
joint  operations  or 
staff  exchanges  are 
being  used  after  the 
intervention has ended 

of 

the 
70% 
in 
participants 
seminars 
training, 
and/or 
conferences 
state  that  they  have 
been  able  to  use  the 
lessons 
in 
their  work  after  the 
training/conference 
Impact  Assessments 
take into consideration 
complementarity  and 
lack of overlaps 

learned 

Stakeholders  consider 
that  full  attention  has 
been given to ensuring 

Desk research 

Descriptive 
data 

Stakeholder 
interviews, 
including  with 
other DGs 

Interviews  with 

Qualitative  data 
analysis 

December 2014 22 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Indicator/descriptor 

judgment 

Norm/ 
criteria 
complementarity  and 
avoiding overlaps  

Data 
collection 
OLAF 

Analytical 
strategy 

Criteria 

Evaluation 
question 
Hercule  II  and 
related 
other 
EU-funded 
initiatives?  

Operationalis
ation 
programmes 
funded  by  DG 
HOME? 

are 

Q.8.2. 
How 
complementari
ty  and  lack  of 
overlaps 
ensured 
between 
Hercule  II  and 
programmes 
funded  by  DG 
TAXUD? 

December 2014 23 

 
 
 
 
 Evaluation of Hercule II Programme 

Surveys 

All in all four online surveys were carried out during the evaluation, each of them with 
the help of Ramboll’s online survey software, SurveyXact. The surveys were directed at 

1.  Beneficiaries of the Hercule II Programme 
2.  Participants  in  trainings,  seminars  and  conferences  co-financed  by  the  Hercule  II 

Programme 

3.  Users  of  the  IT  databases  for  which  access  has  been  purchased  through  the  Hercule  II 

Programme 

4.  Participants in the seminar on "How to Prevent and fight Rural Development Fraud" held 

in Zagreb on 14-15 October 2013 

While  the  information  on  the  last  two  surveys  has  fed  directly  into  the  relevant  case 
studies,  the  surveys  to  beneficiaries  and  participants  are  used  as  direct  source  in  the 
evaluation.  Below,  some  background  information  is  provided  on  each  of  these  two 
surveys, for  example in terms of  response  rates, countries where the  respondents are 
based and types of actions they represent. 

The  beneficiary  and  participant  surveys  are  used  as  one  of  the  three  main  sources  of 
primary data in the main evaluation report and they represent the descriptive analysis, 
as  specified  under  the  evaluation  criterion  of  effectiveness  in  section  3.1.  of  the  main 
body of the report. 

Survey to the beneficiaries 

The survey was open 19 May-30 June 2014. All in all 104 respondents were contacted, 
out  of  whom  68  completed  the  survey.  This  means  that  the  response  rate  was  65%. 
While  the  survey  was  open,  three  reminders  were  sent  out  to  those  persons  who  had 
not yet responded to the survey – on 3, 16 & 25 June 2014. 

The  respondents  contacted  were  beneficiaries  of  the  Hercule  II  Programme,  who  had 
received  signed  grant  agreements  as  a  result  of  calls  for  proposals  organised  in  2012 
and 2013. These persons received an individual link to a survey directly to their e-mail 
address. 

The 68 respondents represented mainly law enforcement and customs authorities, but 
also representative of the judiciary, educational institutions and NGOs were represented 
among the respondents. 

Figure 21: What type of an organisation do you represent? (N= 68) 

The majority of the EU Member States were represented among the respondents, but in 
particular Italian, Polish and Romanian respondents were numerous. 

December 2014 24 

 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 22: Which country do you work in? (N= 68) and In which Member State was 
your latest project conducted? (N=59) 

 Evaluation of Hercule II Programme 

The  survey  was  submitted  in  English  only,  which  led  to  some  concerns  about  the  bias 
this may have caused in terms of respondents without sufficient language skills leaving 
the survey unanswered. However, this does not appear to have had  an impact on the 
responses,  as  the  response  rate  is  relatively  high  and  the  respondents  represent  a 
majority  of  the  Member  States,  indicating  that  it  has  been  possible  for  those 
beneficiaries who wished to respond to the survey, to submit their response. 

In  order  to  assess  whether  those  Member  States  that  are  the  main  receivers  of  co-
funding  through  the  Hercule  II  Programme  were  also  the  ones  contributing  to  a  high 
extent  to  the  survey,  the  evaluators  have  compared,  on  the  one  hand,  the  division  of 
funding  accepted  by  the  Commission  for  each  Member  State78  (excluding  the  IT 
database  projects),  and  the  share  of  respondents  from  each  Member  State.  This 
comparison can be seen in the figure below. 

As can be seen, the shares of respondents and funding differ from each other clearly in 
particular in case of the following Member States:  

  Belgium (high share of funding – small share of respondents) 
  Bulgaria (high share of respondents - small share of funding) 
  Czech Republic (high share of funding – small share of respondents) 
  Greece (high share of funding – no respondents) 
Italy (small share of funding – high share of respondents) 
 
Poland (small share of funding – high share of respondents) 
 
  Romania (small share of funding – high share of respondents) 

There  can  be  different  explanatory  factors  to  this.  Firstly,  it  is  possible  that  in  the 
countries, where there is a high share of respondents but a small share of funding, the 
grants  have  been  relatively  small  and  there  have  been  several  of  them,  making  the 
potential  list  of  respondents  to  the  survey  longer.  Secondly,  it  is  possible  that  the 
countries  with  a  high  share  of  funding  but  small  share  of  respondents  received  the 
funding during the first years of the implementation of the Hercule II Programme, and 
this  would  indicate  that  grants  and  contracts  from  those  years  are  not  sufficiently 

78 The data available to the evaluator provides an overview of ”funding accepted” by the 
Commission for each grant and procurement contract. This information can be analysed 
per country where the main beneficiary is located.  

December 2014 25 

 
 
 
 
 
 
 
 
 
                                           
covered  in  the  beneficiary  survey.  In  any  case  this  analysis  shows  that  the  survey 
results should be interpreted with some care in terms of the geographical distribution of 
the respondents. 

Figure 23: Funding accepted vs. share of respondents in the survey 

 Evaluation of Hercule II Programme 

A  bit  more  than  a  half  of  the  respondents  were  beneficiaries  of  grants  for  technical 
assistance for the fight against fraud, 25% beneficiaries of anti-fraud training and 19% 
beneficiaries  of  trainings,  seminars  or  conferences  with  a  legal  focus.  This  shows  that 
the  beneficiaries  of  training,  seminars  or  conferences  with  a  legal  focus  are 
overrepresented  among  the  survey  respondents,  as  these  types  of  activities  only 
receive approximately 5% of the funding under Hercule II. 

December 2014 26 

 
 
 
 
 
Figure 24: The latest project you have been involved in, which has been (co-)financed 
by OLAF under the Hercule II Programme? (N= 68) 

 Evaluation of Hercule II Programme 

It  can  be  seen  from  the  figure  below  that  59%  (or  40  persons)  of  the  respondents 
represented  organisations  that  had  received  funding  for  more  than  one  project  under 
the Hercule II Programme.  

Figure 25: Has your organisation received funding for more than one project under the 
Hercule II programme? (N= 68) 

This  was  in  particular  the  case  for  the  organisations  carrying  out  technical  assistance 
projects,  where  25  stated  having  received  funding  through  the  Hercule  II  Programme 
previously. 

Figure 26: If yes, what other programme categories have you been involved in (please 
tick all the relevant categories) (N=40) 

A  challenge  in  terms  of  the  evaluation  was  to  reach  to  the  organisations  that  have 
received  funding  through  the  Hercule  II  Programme  during  the  first  years  of  its 
implementation,  i.e.  2007-2010.  A  (non-exhaustive)  cross-check  of  organisations  that 
received  the  survey  (104  recipients)  and  a  list  of  organisations  having  received  co-
funding through the Hercule II Programme in 2007-2011, based on different secondary 
data, shows that approximately one fourth of  the recipients of the survey represented 
organisations that had also received funding prior to 2011.  

This  indicates  that  the  survey  does  not  thoroughly  take  into  account  the  views  of 
organisations  that  have  received  funding  through  the  Hercule  II  Programme  in  2007-
2010. 

The  respondents  to  the  beneficiary  survey  were  mainly  project  managers,  but  there 
were also project owners or project users among the respondents.  

December 2014 27 

 
 
 
 
 
 
 
 
 
 
 
Figure 27: Please consider the latest project you were involved in (under the Hercule II 
programme) and select from the options below the one which best describes your role 
in the project (N=68) 

 Evaluation of Hercule II Programme 

All anti-fraud trainings and 58% of the trainings, seminars or conferences with a legal 
focus  had  encompassed  more  than  one  Member  State  or  third  country.  In  the  case  of 
technical  assistance,  the  share  was  only  12%,  but  this  is  an  interesting  finding  as  it 
shows  that  also  technical  assistance  projects  can  encompass  more  than  one  Member 
State and truly provide transnational value in this sense. 

Figure 28: Did the latest project that you were involved in encompass more than one 
Member State or Third Country? (N= 59) 

Apart  from  two  respondents,  all  the  beneficiaries  surveyed  were  recipients  of  grants 
rather than signatories of procurement contacts. 

Figure 29: Was the support for your latest project awarded as a grant or through a 
procurement contract? (N=59) 

Their  projects  were  to  a  high  extent  still  on-going  (54%)  and  only  42%  of  the 
respondents’  latest  projects  had  been  completed.  This  should  be  kept  in  mind  while 
analysing  the  survey  results,  as  concluding  on  the  results  and  impacts  of  on-going 
projects is only possible to a limited extent. 

December 2014 28 

 
 
 
 
 
 
 
Figure 30: What is the status of your latest project? (N=59) 

 Evaluation of Hercule II Programme 

The  beneficiary  survey  was  examined  to  identify  any  outliers,  i.e.  respondents  who 
have provided throughout highly positive or negative answers, drawing the averages of 
the  survey  to  one  direction  or  another.  It  appears  that  this  is  not  the  case:  the 
organisations  that  have  provided  negative  answers  on  some  questions  are  positive  on 
other  regards,  whereas  the  organisations  that  have  provided  highly  positive  answers 
throughout the questionnaire are also critical on other aspects of the programme. This 
leads  the  evaluator  to  conclude  that  apart  from  the  challenge  of  a  relatively  small 
number  of  respondents  and  the  share  of  all  Hercule  II  co-funded  projects,  the  survey 
results can be considered valid. 

Survey to the participants 

The  survey  was  launched  on  19  May  2014  and  sent  to  a  total  of  1464  participants  in 
trainings, seminars and conferences co-funded by the Hercule II Programme. Following 
three  reminders,  which  were  sent  to  the  recipients  who  had  not  yet  responded  to  the 
survey on 2, 16 and 25 June 2014, the survey was closed on 30 June. 

The final response rate is 39%, or 574 completed surveys. 

As  can  be  seen  from  the  figure  below,  the  respondents  represent  all  the  EU  Member 
States  as  well  as  a  number  of  non-EU  countries.  Romanian,  German,  Bulgarian  and 
Italian respondents were represented to a high extent among the survey respondents.  

Figure 31: Which country do you work in? (N= 574) 

The  non-EU  countries  that  were  represented  include  the  following  (where  the  person 
specified the country that they work in): 

December 2014 29 

151039244285155145311231139221637729761583171263010203040506070 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Kosovo*79: 8 
Montenegro: 3 
Switzerland: 8 
The former Yugoslav Republic of Macedonia: 5 
Iceland: 4 
Albania: 3 
Norway: 2 
Serbia: 12 
Bosnia and Herzegovina: 3 

More than half of the respondents represent law enforcement authorities, followed by customs 
authorities and judicial authorities.  

Figure 32: What type of an organisation do you represent? (N= 573) 

*note: including media  (3), law firms  and legal professionals (4),tobacco industry (1), 
consulting company (1), bank (1), private company of other kind (2). 

The  majority  of  the  respondents  had  participated  in  a  training,  seminar  or  conference 
organised  by  OLAF  (these  could  be,  for  example,  the  AFCOS  conferences).  131 
respondents had participated in a digital forensics training.80 

79 This designation is without prejudice to positions on status, and is in line with UNSCR 1244/99 and the ICJ 
Opinion on the Kosovo declaration of independence 
80 The respondents were asked to specify the latest training, seminar or conference they 
have participated in. 

December 2014 30 

 
 
 
 
 
 
 
 
 
                                           
Figure 33: Please categorise the latest training, seminar or conference you have 
participated in? (N= 574) 

 Evaluation of Hercule II Programme 

The  trainings,  seminars  or  conferences  that  the  respondents  had  participated  in,  had 
taken place, mainly, in Croatia, Germany and Romania.  

Figure 34: In which Member State was your latest training, seminar or conference held? 
(N= 574) 

The  cross-national  aspect  of  the  trainings,  seminars  and  conferences  becomes  clear 
when crossing the countries the persons work in with the countries in which their latest 
training, seminar or conference was held.  

In  total,  54%  of  the  respondents  in  the  participant  survey  stated  that  the  latest 
training,  seminar  or  conference  they  had  participated  in  took  place  in  a  country  other 
than the one they  stated as the country they  work in. As can be seen from the figure 
below,  in  concrete  numbers,  in particular  the Croatian  training,  seminar  or  conference 
participants had travelled to another EU Member State for their latest training, seminar 
or  conference.  The  shares  of  persons  travelling  were  also  high  in  Belgium,  Germany, 
Greece, Hungary, the Netherlands and the UK. Instead, only 5% of the Spanish, 7% of 

December 2014 31 

1471265141429233211131390320160,0%10,0%20,0%30,0%4927134034071149023475346800627070102205020406080100120140160 
 
 
 
 
 
 
 
 
the  Polish  and  15%  of  the  Bulgarian  respondents  had  travelled  to  another  EU  MS  for 
their latest training, seminar or conference. 

Figure  35:  Percentage  of  survey  respondents  working  in  country  X,  participating  in  a 
training, seminar or conference in another country (N= 511) 

 Evaluation of Hercule II Programme 

What is striking in the responses of the training, seminar and conference participants, is 
that in 90% of the cases the latest training, seminar or conference had taken place in 
either  2013  or  2014.  This  means  that  the  respondents  have  the  experiences  from  the 
events  still  fresh  in  their  minds,  but  it  can  also  mean  that  it  is  difficult  for  the 
respondents, in particular for those who had participated in an event in 2014, to assess 
the sustainability of the lessons learned.  

December 2014 32 

 
 
 
 
 
 
Figure 36: When did the latest training, seminar or conference that you participated in 
take place? (N= 574) 

 Evaluation of Hercule II Programme 

Furthermore, only 39% of the respondents had participated in more than one training, 
seminar or  conference  under the Hercule  II Programme. This means that the majority 
would  not  be  able  to  assess  the  longer  term  effects  and  sustainability  of  their 
experiences either based on older events. 

Figure  37:  Have  you  participated  in  more  than  one  training,  seminar  or  conference 
under the Hercule II Programme? (N= 574) 

These limitations should be kept in mind when assessing the results of the participant 
survey. 

December 2014 33 

 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Stakeholder interviews 

All in all 38 stakeholder interviews were carried out during the course of the evaluation 
with the following categories of stakeholders: 

-  OLAF personnel 
-  Member State authorities 
-  Representatives of different DGs of the European Commission 
- 

International organisations 

The  interviews  were  mainly  carried  out  by  phone,  some  face-to-face  and  a  limited 
number  of  responses  were  provided  in  writing.  This  was  usually  the  case  due  to  the 
need to verify the interview responses by the hierarchy. 

Concerning  the  views  of  the  stakeholders,  it  should  be  mentioned  that  the  level  of 
knowledge  that  the  stakeholders  had  of  the  Hercule  II  Programme  differed  greatly. 
While  some  were  well  aware  of  the  Programme,  its  set-up  and  the  projects  funded, 
others  were  only  able  to  express  their  views  on  the  basis  of  a  project  they  had 
participated  in,  and  others  were  answering  from  the  point  of  view  of  larger  political 
priorities in the field of protection of financial interests of the EU. 

This means that the stakeholders were not always able to make qualitative statements 
and  the  evidence  base  to  some  of  the  questions  lies  on  a  limited  number  of 
stakeholders. Only the views of the persons who were able to comments on a topic are 
taken into account for each evaluation question. 

The stakeholder interviews are used as one of the three main sources of primary data 
in  the  main  evaluation  report  and  they  represent  the  qualitative  analysis,  as  specified 
under  the  evaluation  criterion  of  effectiveness  in  section  3.1.  of  the  main  body  of  the 
report. 

December 2014 34 

 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Contribution analysis and case studies 

As  mentioned  previously,  the  evaluation  was  structured  around  the  overall 
methodology  of  contribution  analysis,  which  helped  the  evaluator  frame  both  the 
evaluation  questions  as  well  as  the  data  collection  tools.  In  the  figure  below  the 
different  evaluation  steps  are  described,  each  representing  different  activities  during 
the evaluation. 

Whereas  steps  1  and  2  have  been  described  previously  (evaluation  matrix  with  the 
evaluation  questions;  beneficiary  and  participant  surveys),  we  will  in  this  section 
provide an overview of how the remaining steps have been carried out. 

Figure 38: Steps involved in contribution analysis of Hercule II 

Steps  3  and  4:  develop  the  embedded  theory  of  change  and 
assess and prioritise alternative explanations 

Based  on  preliminary  research  carried  out  during  the  inception  phase  and  interim 
phases,  the  evaluators  developed  18  hypotheses  describing  assumptions  that  link  the 
programme  activities,  outputs,  outcomes  and 
impacts  together.  These  were 
supplemented  by  preliminary  influencing  factors,  which  were  used  as  the  basis  for 
assessing  both  influencing  factors  and  alternative  explanations  to  the  evaluation 
findings. The hypotheses and preliminary influencing factors can be  found in the table 
below. 

Table 4: Hypotheses to be tested in the case studies 
Hypot
hesis 
No. 

Linkage to the original 
intervention logic and 
hypotheses 

Influencing factors (preliminary 
and fictive) 

H1 

By providing technical assistance for 
national authorities, MS authorities 
can acquire technical equipment, 
which strengthens networks and 
exchange of information, leading to 

By providing technical assistance for 
national authorities, MS authorities can 
acquire technical equipment, which 
strengthens networks and exchange of 
information. The equipment is however 

December 2014 35 

Step 1•Identify the contribution question(s)•Refining the evaluation questions and the evaluation matrix with the help of context analysis and explorative interviews with OLAFStep 2•Gather and review evidence on impacts•Beneficiary and participant surveys•Interviews with stakeholdersStep 3•Develop the embedded theory of change•Define the assumptions linking the Programme activities, outputs, outcomes and impacts in concise but detailed hypothesesStep 4•Assess and prioritise focus on alternative explanation•Strengthen the analysis by critically assessing the alternative explanations.Step 5•Collect new data identified by prior steps•Conduct case studies to collect systematic knowledge of the mechanisms at play in realising the expected outcomes, examining the validity of the hypothesesStep 6 •Analyse and report on the contribution of HerculeII•Using the data to careful address the strength of the hypotheses and revise the embedded theory of change to accurately reflect the evidence reviewed and gathered. Phase 1:StructuringPhase 2:Data collectionPhase 3:Synthesis 
 
 
 
 
 
 
 
 
 
H2 

H3 

H4 

H5 

prevention of losses to the EU 
budget. 

By providing technical assistance for 
national authorities, MS authorities 
can acquire technical equipment, 
which enhances transnational and 
multidisciplinary cooperation 
between the MS and Commission, 
leading to prevention of losses to 
the EU budget. 

By providing technical assistance for 
national authorities, MS authorities 
can acquire technical equipment, 
which strengthens the technical and 
operational support for law 
enforcement and customs 
authorities, leading to prevention of 
losses to the EU budget. 

By providing technical assistance for 
national authorities, MS authorities 
can acquire technical equipment, 
which improves investigations, 
monitoring and intelligence work, 
leading to prevention of losses to 
the EU budget. 

By providing technical assistance for 
national authorities, MS authorities 
can acquire technical equipment, 
which multiplies and intensifies 
measures in the areas identified as 
the most sensitive, particularly in 
the field of cigarette smuggling and 
counterfeiting, leading to prevention 
of losses to the EU budget. 

H6 

By providing technical assistance for 
national authorities, joint 
operations and staff exchange 
can be enabled, which strengthens 
networks and exchange of 

 Evaluation of Hercule II Programme 

not being used for information 
exchange between MS, as the national 
authorities focus on domestic issues. 
The activity does thus not lead to a 
more unified prevention of losses to 
the EU budget. 
By providing technical assistance for 
national authorities, MS authorities can 
acquire technical equipment, which 
enhances transnational and 
multidisciplinary cooperation between 
the MS and Commission will be 
achieved. However, the technical 
equipment is not relevant to all MS 
and not suited for transnational 
cooperation. The activity does thus not 
lead to a more unified prevention of 
losses to the EU budget. 
By providing technical assistance for 
national authorities, MS authorities can 
acquire technical equipment. This 
equipment cannot however be properly 
used due to lack of human resources in 
the MS. The activity does thus not lead 
to prevention of losses to the EU 
budget. 

By providing technical assistance for 
national authorities, MS authorities can 
acquire technical equipment, which 
improves investigations, monitoring 
and intelligence work. This equipment 
cannot however be fully utilized by the 
staff as they lack the capacity and 
necessary training to operate the 
equipment. The activity does thus not 
lead to a more unified prevention of 
losses to the EU budget. 
By providing technical assistance for 
national authorities, MS authorities can 
acquire technical equipment, which 
multiplies and intensifies measures in 
the areas identified as the most 
sensitive, particularly in the field of 
cigarette smuggling and counterfeiting. 
Though the equipment is relevant it 
does not add to the existing measures 
in the areas of cigarette smuggling and 
counterfeiting. The activity does thus 
not lead to an increased prevention of 
losses to the EU budget. 
By providing technical assistance for 
national authorities, joint operations 
and staff exchange can be enabled. 
The MS authorities however do not 
have the resources to prioritise and 

December 2014 36 

 
 
 
 
H7 

H8 

H9 

H10 

H11 

information, leading to prevention of 
losses to the EU budget. 

By providing technical assistance for 
national authorities, joint 
operations and staff exchange 
can be enabled, which enhances 
transnational and multidisciplinary 
cooperation between the MS and 
Commission, leading to prevention 
of losses to the EU budget. 

By providing technical assistance for 
national authorities, joint 
operations and staff exchange 
can be enabled, which strengthens 
the technical and operational 
support for law enforcement and 
customs authorities, leading to 
prevention of losses to the EU 
budget. 

By organising training, seminars and 
conferences, MS coordinate 
activities, which enhance the 
understanding of Union and national 
mechanisms, leading to prevention 
of losses to the EU budget. 

By organising trainings, seminars 
and conferences, exchanging 
experiences in the Member 
States is increased, which 
enhances understanding of Union 
and national mechanisms, leading to 
prevention of losses to the EU 
budget. 

By organising trainings, seminars 
and conferences, exchanging 
experiences in the Member 
States is increased, which raises 
the awareness of the judiciary and 
other branches of the legal 

 Evaluation of Hercule II Programme 

train the exchanged staff. 
Consequently, the joint operation and 
staff exchange have a limited effect 
and does not lead to prevention of 
losses to the EU budget.   
By providing technical assistance for 
national authorities, joint operations 
and staff exchange can be enabled, 
which enhances transnational and 
multidisciplinary cooperation between 
the MS and the Commission. The joint 
operations and staff exchanges prove 
to be limited and do not increase the 
cooperation between the MS and the 
Commission significantly. The activity 
does thus not lead to prevention of 
losses to the EU budget. 
By providing technical assistance for 
national authorities, joint operations 
and staff exchange can be enabled, 
which strengthens the technical and 
operational support for law 
enforcement and customs authorities. 
The law enforcement and customs 
authorities do not spend the required 
resources on involving the staff being 
on exchange in their daily work, which 
makes the value of technical and 
operational support limited. The 
activity does thus not lead to 
prevention of losses to the EU budget. 
By organising training, seminars and 
conferences, MS coordinate activities. 
However, the activities are considered 
more relevant by some MS than others 
and there is a limited coordination 
between them. The activity does thus 
not lead to prevention of losses to the 
EU budget. 
By organising trainings, seminars and 
conferences, the exchanging of 
experiences in the Member States is 
meant to increase. However, the types 
of experiences exchanged are not 
considered important for some MS, 
which means that there will not be 
enhanced understanding of Union and 
national mechanisms. The activity does 
thus not lead to prevention of losses to 
the EU budget. 
By organising trainings, seminars and 
conferences, the exchanging of 
experiences in the Member States is 
meant to increase, raising the 
awareness of the judiciary and other 
branches of the legal profession for the 

December 2014 37 

 
 
 
H12 

H13 

H14 

H15 

H16 

profession for the protection of the 
financial interests of the Union, 
leading to prevention of losses to 
the EU budget. 

By organising trainings, seminars 
and conferences, research and 
dissemination of knowledge is 
increased, which improves 
cooperation between practitioners 
and academics, leading to 
prevention of losses to the EU 
budget. 

By organising trainings, seminars 
and conferences, research and 
dissemination of knowledge is 
increased, which raises the 
awareness of the judiciary and other 
branches of the legal profession for 
the protection of the financial 
interests of the Union, leading to 
prevention of losses to the EU 
budget. 

By providing IT support, specific 
databases and IT tools are 
developed, which improves 
investigations, monitoring and 
intelligence work, leading to 
prevention of losses to the EU 
budget. 

By providing IT support, data access 
and exchange are facilitated, which 
improves investigations, 
monitoring and intelligence 
work, leading to prevention of 
losses to the EU budget. 

By providing technical assistance for 
national authorities, common 
technical standards can be 
promoted, which leads to unified 
level of skills, equipment and 
readiness in the MS to detect 
smuggled and counterfeit 

 Evaluation of Hercule II Programme 

protection of the financial interests of 
the Union. The event has however not 
been targeted specifically enough to 
the judiciary and other branches of the 
legal procession, which means that 
awareness is not raised. The activity 
does thus not lead to prevention of 
losses to the EU budget. 
By organising trainings, seminars and 
conferences, research and 
dissemination of knowledge take place. 
However, the participants do not 
sufficiently represent practitioners and 
academia, which means that research 
and dissemination of knowledge are 
not increased and cooperation is not 
improved. The activity does thus not 
lead to prevention of losses to the EU 
budget. 
By organising trainings, seminars and 
conferences, research and 
dissemination of knowledge take place. 
The representatives of the judiciary 
and other branches of the legal 
profession do however not see the 
benefit of working transnationally, 
which means that their awareness for 
the protection of the financial interests 
of the Union is not increased. The 
activity does thus not lead to 
prevention of losses to the EU budget. 
By providing IT support, specific 
databases and IT tools are developed. 
The developed databases and IT tools 
are however not relevant for the 
investigations, monitoring and 
intelligence work of the Member 
States. The activity does thus not lead 
to prevention of losses to the EU 
budget. 
By providing IT support, data access 
and exchange are facilitated. However, 
the MS consider the use of the 
databases too complicated and 
bureaucratic, meaning that 
investigations, monitoring and 
intelligence work are not improved. 
The activity does thus not lead to 
prevention of losses to the EU budget. 
By providing technical assistance for 
national authorities, common technical 
standards can be promoted. Some MS 
do however not have the financial 
means to participate in financing of 
technical equipment, which means that 
a unified level of equipment is not in 

December 2014 38 

 
 
 
H17 

H18 

products, leading to prevention of 
losses to the EU budget. 

By organising anti-fraud training to 
MS officials and other actors, MS 
officials are trained to fight against 
fraud, which leads to unified 
level of skills, equipment and 
readiness in the MS to detect 
smuggled and counterfeit 
products, leading to prevention of 
losses to the EU budget. 

By organising anti-fraud training to 
MS officials and other actors, MS 
officials are trained to fight against 
fraud, which multiplies and 
intensifies measures in the 
areas identified as the most 
sensitive, particularly in the 
field of cigarette smuggling and 
counterfeiting, leading to 
prevention of losses to the EU 
budget. 

 Evaluation of Hercule II Programme 

place. The activity does thus not lead 
to prevention of losses to the EU 
budget. 
By organising anti-fraud training to MS 
officials and other actors, MS officials 
are trained to fight against fraud. The 
participation in the trainings is 
however geographically uneven 
between the MS, meaning that the 
training does not lead to unified level 
of skills in the MS to detect smuggled 
and counterfeit products. The activity 
does thus not lead to prevention of 
losses to the EU budget. 
By organising anti-fraud training to MS 
officials and other actors, MS officials 
are trained to fight against fraud. 
Participation in EU financed projects is 
however not prioritised by the 
hierarchy in the relevant MS, which 
means that the activity does not lead 
to prevention of losses to the EU 
budget. 

Step 5: Collect new data identified by prior steps 

In  order  to  examine  the  18  hypotheses,  21  thematic  and  specific  case  studies  were 
selected to be carried out, divided as follows: 

  11 case studies on Technical Assistance 
  8 case studies on training, seminars and conferences  
  2 case studies on IT support 

Each case study examined at least two (2) hypotheses and relevant influencing factors 
or  alternative  explanations  for  these  hypotheses.  The  data  used  for  the  case  studies 
consisted  of  desk  research  of  grant  applications,  final  technical  and  financial  report, 
participant  evaluation  forms  and  other  relevant  project  documents;  interviews  with 
project  managers,  project  users  or  other  relevant  persons  related  to  the  project(s)  in 
question;  and  drawing  on  primary  data  collection  activities  carried  out  within  the 
framework  of  the  evaluation,  including  surveys.  For  two  of  the  case  studies  an  own 
online survey was developed and distributed. 

20  of  the  case  studies  were  finalised  during  the  time  of  draft  final  report,  due  to 
challenges  in  reaching  the  relevant  interviewees  in  case  study  2  (Procurement  of 
technical equipment for electronic and mobile surveillance in the fight against cigarettes 
smuggling in Finland). More than 29 grant agreements and procurement contracts were 
covered by the case studies. 

December 2014 39 

 
 
 
 
 
 
 
 
 
Table 5: List of case studies and hypotheses 

Sector  of 
activity 

Case 
study 
No. 

Type  of 
Case 
Study 

Title  of 
Study 
description 

the  Case 
short 

and 

Country  Year 

Technical 
Assistance 

1 

Thematic  The  purchase  of  x-ray 
scanners  for  customs 
as 
technical  support 
tool  for  detecting  the 
presence  of  cigarettes 
and tobacco. 

IE, 

BE, 
ES, PL 

2011/ 
2012 

This  case  study  will 
address  a  number  of 
actions  aimed  at  the 
purchase 
x-ray 
scanners. 

of 

Grant nr: 

TA IE 2012 D5 050 

TA BE 2012 D5 011 

TA ES 2012 D5 088 

TA IE 2012 D5 005 

TA PL 2012 D5 089 

Size 
of 
commitment 
(and 
co-
financing 
rate) 

of 

for 

(BE) 
Purchase 
X-Ray 
scanner 
customs: 
€850,000 
(50%) 

(ES) 
Purchase 
of 
back  mobile 
scanners: 
€586,061 
(48.4%) 

(IE)  Purchase 
mobile 
of 
XRay  scanner 
for  customs: 
€95,000 
(48.7%); 

(IE) 
Procurement 
of 
scanners: 
€45,000 
(50%) 

X-Ray 

 Evaluation of Hercule II Programme 

Type 
Action 

of 

Hypothesis 1 

Hypothesis 2 

Grants 

H5: 

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which  multiplies 
and 
intensifies 
measures  in  the 
identified 
areas 
as 
the  most 
sensitive, 
particularly 
the 
field 
cigarette 
smuggling 
counterfeiting, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

in 
of 

and 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

December 2014 40 

 
 
 
 
 
(PL)  Purchase 
of 
X-Ray 
scanners  and 
video 
cameras: 
€81,466 
(50%) 

Technical 
equipment 
mobile 
for 
and electronic 
surveillance: 
€1,200,000 
(50%) 

2 

Specific 

FI  

2012 

Procurement 
of 
technical equipment for 
electronic  and  mobile 
surveillance in the fight 
against 
cigarettes 
smuggling. 

The  case  study  will 
address  one  action  for 
the 
of 
purchase 
technical  equipment  in 
FI. 

Grant nr:  

TA FI 2013 D5 013 

 Evaluation of Hercule II Programme 

Grant 

H5: 

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
can 
authorities 
acquire  technical 
equipment, 
which  multiplies 
intensifies 
and 
measures  in  the 
identified 
areas 
the  most 
as 
sensitive, 
particularly 
the 
field 
cigarette 
smuggling 
counterfeiting, 
to 
leading 
of 
prevention 
losses  to  the  EU 
budget. 

in 
of 

and 

3 

Specific 

Procurement 

of  PL 

2012 

Technical 

Grant 

H3:  

H4:  

December 2014 41 

 
 
 
technical equipment for 
the  acquisition  and 
analysis 
digital 
of 
evidence and analysis. 

This  case  study  will 
action 
an 
address 
the 
at 
aimed 
procurement 
of 
computer 
forensic 
laboratory equipment. 

Grant nr: 

TA PL 2013 D5 001 

4 

Thematic  The 

of 
provision 
technical support in the 
fight against corruption 
– The Romanian case. 

RO 

2011/ 
2012 

directed 

This  case  study  will 
include  a  number  of 
actions 
to 
technical 
provide 
the 
support 
Romanian 
Anticorruption  General 
Directorate  in  the  fight 

to 

the 

equipment 
for 
acquisition 
and  analysis 
of 
digital 
analysis: 
€312,390 
(47.5%) 

(2011) 
Technical 
equipment 
for  electronic 
surveillance: 
€74,900 
(49.9%) 

(2012) 
Technical 
endowment: 
€115,500 
(49.8%); 

 Evaluation of Hercule II Programme 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
of 
prevention 
losses  to  the  EU 
budget. 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which 
strengthens 
technical 
operational 
support  for  law 
enforcement  and 
customs 
authorities, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

the 
and 

Grants 

H3:  

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which 
strengthens 
technical 

the 
and 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 

December 2014 42 

 
 
 
against corruption. 

Grant nr: 

TA RO 2011 C5 019 

TA RO 2012 D5 039 

TA RO 2012 D5 065 

TA RO 2012 D5 071 

5 

Thematic  Procurement 

the 

of 
surveillance  equipment 
in 
fight  against 
fraud to  obtain refunds 
in  VAT,  the  Spanish 
case. 

This  case  will  focus  on 
two  actions  directed  at 
the  procurement  of 
surveillance  equipment 
to 
Spanish 
authorities, which were 
successfully  used  in  a 
case  of  fraud  to  obtain 
refunds of VAT through 

the 

the 

(2012) 
Technical 
endowment 
of 
national 
anticorruption 
directorate: 
€45,320 
(50%); 

(2012) 
Technical 
support 
in 
fighting  fraud 
and 
corruption: 
€85,928 
(50%) 

special 

Procurement 
of 
investigation 
equipment: 
€211,548 
(42.3%); 

Purchase 
of 
technical  and 
electronic 
devices: 
€164,908 
(50%) 

ES 

2012 

 Evaluation of Hercule II Programme 

monitoring  and 
intelligence 
work,  leading  to 
of 
prevention 
losses  to  the  EU 
budget. 

operational 
support  for  law 
enforcement  and 
customs 
authorities, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

Grant 

H3:  

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which 
strengthens 
technical 
operational 
support  for  law 
enforcement  and 

the 
and 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 

December 2014 43 

 
 
 
a  virtual  trade  of  olive 
oil. 

Grant nr:  

TA ES 2012 D5 054 

TA ES 2013 D5 003 

6 

Thematic  Development  of  an 
Number 
Automated 
Recognition 
Plate 
in 
System 
Estonia,  Lithuania  and 
Latvia 
their 
and 
interfacing network. 

(ANPRS) 

EE,  LT, 
LV 

2012 

This  case  will  focus  on 
a  series  of  actions 
directed  to  the  three 
countries  in  order  to 
develop and implement 
their ANPRS and to the 
an 
completion 
interfacing 
project 
which  linked  the  three 
national 
systems, 
enhancing  cooperation 
against illicit  trafficking 
of goods. 

of 

Grant nr: 

TA LV 2012 D5 070 

(EE) 
Procurement 
of 
surveillance 
and 
equipment: 
€87,636 
(49.9%) 

tracking 

(LV) Purchase 
and 
installation  of 
vehicle 
tracking 
system: 
€98,778 
(48%) 

the 

(LT)  To  be 
specified 
once 
information 
about 
the 
Programme’s 
are 
actions 
provided. 

 Evaluation of Hercule II Programme 

prevention 
of 
losses  to  the  EU 
budget. 

customs 
authorities, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

Grants 

H1:  

H2:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which 
strengthens 
networks 
exchange 
information, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

and 
of 

H6: 

providing 

By 
technical 
assistance 
national 
authorities,  joint 

for 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  enhances 
transnational 
and 
multidisciplinary 
cooperation 
between  the  MS 
and 
Commission, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

H7: 

providing 

By 
technical 

December 2014 44 

 
 
 
 
 
 Evaluation of Hercule II Programme 

and 
operations 
staff 
exchange 
can  be  enabled, 
which 
strengthens 
networks 
exchange 
information, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

and 
of 

H8: 

for 

providing 

By 
technical 
assistance 
national 
authorities,  joint 
and 
operations 
staff 
exchange 
can  be  enabled, 
which 
strengthens 
technical 
operational 
support  for  law 
enforcement  and 
customs 
authorities, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

the 
and 

for 

assistance 
national 
authorities,  joint 
operations  and 
staff  exchanges 
can  be  enabled, 
which  enhances 
transnational 
and 
multidisciplinary 
cooperation 
between  the  NS 
and 
Commission, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

December 2014 45 

 
 
 
7 

Specific 

Technical  support  for 
the  purchase  of  fuel 
unloading systems 

Grant nr: 

TA LV 2012 D5 051 

LV 

2012 

€58,800 
(50%) 

 Evaluation of Hercule II Programme 

Grant 

H3:  

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
can 
authorities 
acquire  technical 
equipment, 
which 
strengthens 
technical 
operational 
support  for  law 
enforcement  and 
customs 
authorities, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

the 
and 

8 

Specific 

Technical  and  video 
surveillance capacity 

PT 

Grant Nr: 

TA PT 2010 C5 018 

2010 

€56,100 

Grant 

H4: 

H2: 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities 
can 
acquire  technical 
equipment, 
which 
investigations, 

improves 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  enhances 

December 2014 46 

 
 
 
9 

Specific 

Procurement  of  audio 
tools  for  investigation 
by  the  Czech  General 
for 
Directorate 
Customs 

Grant Nr: 
TA CZ 2013 D5 004 

 Evaluation of Hercule II Programme 

monitoring  and 
intelligence 
work,  leading  to 
of 
prevention 
losses  to  the  EU 
budget. 

transnational 
and 
multidisciplinary 
cooperation 
between  the  MS 
and 
Commission, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

CZ 

2012 

€26,250 

Grant 

H16: 

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

for 

providing 

By 
technical 
assistance 
national 
authorities, 
common 
technical 
standards can be 
promoted,  which 
leads  to  unified 
level  of  skills, 
equipment 
and 
readiness  in  the 
to  detect 
MS 
smuggled 
and 
counterfeit 
products, leading 
to  prevention  of 
losses  to  the  EU 
budget. 

December 2014 47 

 
 
 
 
 Evaluation of Hercule II Programme 

H3:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
can 
authorities 
acquire  technical 
equipment, 
which 
strengthens 
technical 
operational 
support  for  law 
enforcement  and 
customs 
authorities, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

the 
and 

Grant 

H16: 

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities, 
common 
technical 
standards can be 
promoted,  which 
leads  to  unified 

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 

December 2014 48 

10 

Specific 

equipment 
Technical 
for 
and 
mobile 
electronic  surveillance 
for Helsinki Police. 

FI 

2010 

2011 

€117,263.93 
(48.13%) 

€125,000 
(49.3%) 

Grant nr:  

TA FI 2010 C5 015 

TA FI 2011 C5 053 

 
 
 
11 

Specific 

Implementation 
mobile forensic tools 

Grant nr: 

TA IT 2012 D5 062 

of 

IT 

2012 

€228,048 
(50%) 

 Evaluation of Hercule II Programme 

investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

level  of  skills, 
equipment 
and 
readiness  in  the 
to  detect 
MS 
smuggled 
and 
counterfeit 
products, leading 
to  prevention  of 
losses  to  the  EU 
budget. 

Grant 

H16: 

H4:  

for 

providing 

By 
technical 
assistance 
national 
authorities,  MS 
authorities  can 
acquire 
technical 
equipment, 
which  improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
of 
prevention 
losses  to  the  EU 
budget. 

for 

providing 

By 
technical 
assistance 
national 
authorities, 
common 
technical 
standards can be 
promoted,  which 
leads  to  unified 
level  of  skills, 
equipment 
and 
readiness  in  the 
to  detect 
MS 
and 
smuggled 
counterfeit 
products, leading 
to  prevention  of 
losses  to  the  EU 
budget. 

Training, 

12 

Specific 

Training  of  customs  SK 

2012 

€39,350 

Grant 

H17:  

H18:  

December 2014 49 

 
 
 
 
(79.9%) 

Seminars 
and 
Conference 

officers  in  the  area  of 
fight against corruption 
and  fraud,  particularly 
tobacco 
against 
products 
smuggling 
and counterfeiting  

The  case  will  address 
an  action  aiming  at 
providing 
financial 
support  for  training  of 
custom officers. 

Grant Nr: 

09 

TRAIN 
OLAF/2011/D7/035 
D7/AH/KH DED 4168 

SK 

 Evaluation of Hercule II Programme 

leads 

organising 

By 
anti-fraud 
to  MS 
training 
officials 
and 
other  actors,  MS 
officials 
are 
trained  to  fight 
fraud, 
against 
which 
to 
unified  level  of 
skills,  equipment 
and  readiness  in 
the  MS  to  detect 
smuggled 
and 
counterfeit 
products, leading 
to  prevention  of 
losses  to  the  EU 
budget. 

organising 

By 
anti-fraud 
training  to  MS 
officials 
and 
other actors, MS 
officials 
are 
trained  to  fight 
against 
fraud, 
which  multiplies 
and 
intensifies 
measures  in  the 
identified 
areas 
as 
the  most 
sensitive, 
particularly 
the 
field 
cigarette 
smuggling  and 
counterfeiting, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

in 
of 

13 

Specific 

Increasing the capacity 
of  Bulgarian  customs 
to combat trafficking of 
illicit tobacco products. 

BG 

2012 

aimed 

The case focuses on an 
at 
action 
analytical 
increasing 
capacity 
threat 
analysis 
assessment 

in 

Increasing 
the  capacity 
BG 
of 
customs 
to 
combat 
trafficking  of 
illicit  tobacco 
products: 
€36,590 
(80%) 

Grant 

H17: 

H18:  

organising 

By 
anti-fraud 
to  MS 
training 
officials 
and 
other  actors,  MS 
officials 
are 
trained  to  fight 
fraud, 
against 

organising 

By 
anti-fraud 
training  to  MS 
officials 
and 
other actors, MS 
officials 
are 
trained  to  fight 
fraud, 
against 

December 2014 50 

 
 
 
 
 
and  operational 
analysis. 

risk 

Grant nr: 

27 

TRAIN 
OLAF/2012/D5/083 
D5/JK/KH 2012-4906 

BG 

IT 

2011 

Counteraction 
to 
the  EU 
financial 
fraud, 
strategies 
and 
tools: 
89,331.46 
(80%) 

audit 
€ 

14 

Specific 

financial 

Counteraction  to  the 
EU 
fraud. 
Strategies  and  audit 
tools 

organised 

seminar 
A  half-day 
in 
were 
Italian 
nineteen 
regional 
offices 
(managing  authorities, 
certification  and  audit 
offices/administrations) 
for  eighty  participants 
per seminar.  

A two-day seminar was 
organised 
in  Rome 
which  was  attended  by 
from 
representatives 

 Evaluation of Hercule II Programme 

leads 

which 
to 
unified  level  of 
skills,  equipment 
and  readiness  in 
the  MS  to  detect 
smuggled 
and 
counterfeit 
products, leading 
to  prevention  of 
losses  to  the  EU 
budget. 

which  multiplies 
and 
intensifies 
measures  in  the 
identified 
areas 
as 
the  most 
sensitive, 
particularly 
field 
the 
cigarette 
smuggling  and 
counterfeiting, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

in 
of 

Grant 

H10:  

H11: 

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
States 
is 
increased,  which 
enhances 
understanding  of 
Union 
and 
national 
mechanisms, 
to 
leading 
prevention 
of 
losses  to  the  EU 

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
States 
is 
increased, which 
the 
raises 
of 
awareness 
judiciary 
the 
and 
other 
branches  of  the 
legal  profession 
for 
the 
protection of the 

December 2014 51 

 
 
 
 
the  relevant  national 
institutions  from  three 
Member 
States 
Hungary, 
(Greece, 
Romania), 
three 
countries 
candidate 
(Croatia, 
Former 
Yugoslav  Republic  of 
Iceland, 
Macedonia, 
Montenegro, 
Turkey) 
and Albania. 

Grant nr: 

18 

TRAIN 
OLAF/2011/D7/054 
D7/AH/KH DED-4319 

IT 

15 

Specific 

DE 

2013 

European 
Public 
Prosecutor’s 
Office 
– 
Protection  of 
EU’s 
the 
financial 
interests: 
€117,580.16 
(80%) 

European 
Public 
Prosecutor’s  Office  – 
Protection  of  the  EU’s 
financial interests 

A three-day conference 
participants 
with 
from  public 
mostly 
prosecution 
services 
dealing  with  questions 
international 
of 
cooperation  and 
the 
fight against fraud. The 
comprises 
event 
and 
presentations 
the 
discussion 
a 
setting-up 
Public 
European 

on 
of 

 Evaluation of Hercule II Programme 

budget. 

financial 
interests  of  the 
Union, 
leading 
to  prevention  of 
losses  to  the  EU 
budget. 

Grant 

H11: 

H13: 

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
States 
is 
increased,  which 
raises 
the 
awareness of the 
judiciary 
and 
other  branches 
legal 
of 
profession 
for 
the  protection  of 

the 

and 

organising 

By 
trainings, 
seminars 
conferences, 
and 
research 
dissemination  of 
knowledge 
is 
increased, which 
the 
raises 
of 
awareness 
judiciary 
the 
and 
other 
branches  of  the 
legal  profession 
for 
the 
protection of the 

December 2014 52 

 
 
 
 
 
Office 

Prosecutor’s 
(EPPO). 

Grant nr: 

02 

TRAIN 
OLAF/2013/D5/043 
D5/JK/KH 2013-5328 

DE 

16 

Specific 

How  to  Prevent  and 
Fight 
Rural 
Development  Fraud  – 
Zagreb, Croatia 

OLAF 

2013  
(14-
15/10/13) 

Committed:  € 
143,951.05  
Paid: 
€62,437.25   

Procurement 
Contract  – 
MCI Benelux 
S.A. 

Contract nr: 

Organised 

under 

 Evaluation of Hercule II Programme 

the 
financial 
interests  of  the 
Union, leading to 
of 
prevention 
losses  to  the  EU 
budget. 

financial 
interests  of  the 
Union, 
leading 
to  prevention  of 
losses  to  the  EU 
budget. 

H12: 

and 

organising 

By 
trainings, 
seminars 
conferences, 
research 
and 
dissemination  of 
is 
knowledge 
increased,  which 
improves 
cooperation 
between 
practitioners  and 
academics, 
to 
leading 
of 
prevention 
losses  to  the  EU 
budget. 

H10:  

H9:  

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 

and 

organising 

By 
trainings, 
seminars 
and 
conferences,  MS 
coordinate 

December 2014 53 

 
 
 
 
Framework 
Service 
Contract  OLAF/D6/JPL 
D(2010/798076) 

17 

Thematic  Computer 

Forensics 

OLAF 

Training 

Twice 
year 

a 

€  890  259 
(committed) 

(2013) 
Computer 
Forensic 
Training 
Winter 
€449,600 

: 

Will  address 
all  contracts 
from  2007  to 
2013 

aimed 

employed 

This  case  study  will 
address  a  number  of 
at 
actions 
funding 
the 
organisation of training 
sessions  in  computer 
targeting 
forensics, 
staff 
by 
national  or 
regional 
administrations  of  the 
EFTA/EEA 
MS, 
countries 
and 
countries 
candidate 
which  promote 
the 
strengthening  of  EU 
action  to  protect  the 
EU's financial interests, 
of 
with 

aim 

the 

Procurement 
(framework 
and  service 
contract) 
Insig2   

 Evaluation of Hercule II Programme 

activities,  which 
enhances 
understanding 
of  Union  and 
national 
mechanisms, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

in 
experiences 
Member 
the 
States 
is 
increased,  which 
enhances 
understanding  of 
Union 
and 
national 
mechanisms, 
to 
leading 
of 
prevention 
losses  to  the  EU 
budget. 

H10:  

H9:  

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
is 
States 
increased,  which 
enhances 
understanding  of 
and 
Union 
national 
mechanisms, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

organising 

By 
trainings, 
seminars 
and 
conferences,  MS 
coordinate 
activities,  which 
enhance 
understanding 
of  Union  and 
national 
mechanisms, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

December 2014 54 

 
 
 
 
improving their skills in 
the  use  of  tools  and 
software  for  forensics 
while 
examination, 
exchanging 
experiences 
information. 

and 

Contract nr: 

Framework 
Contract 
OLAF/D5/D9/2013 

Service 
number 

18 

Specific 

"Fight  against 
affecting 

frauds 

RO  and 
HR 

2012 

€50,000 
(79.9%) 

EU's  budget  -  Sharing 
Romanian 
the 
experience  with 
the 
Croatian authorities." 

A  series  of 
training 
activities  organised  by 
DLAF  –  Fight  Against 
Department, 
Fraud 
Romania.  The  aim  of 
the  project  was 
to 
Croatian 
inform 
the 
officials 
the 
experience 
Romanian 
AFCOS 
(Anti-Fraud 
Coordination 
Service 
and  national  contact 
OLAF) 
with 
point 

about 
of 

 Evaluation of Hercule II Programme 

Grant 

H10:  

H9:  

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
States 
is 
increased,  which 
enhances 
understanding  of 
Union 
and 
national 
mechanisms, 
to 
leading 
prevention 
of 
losses  to  the  EU 
budget. 

organising 

By 
trainings, 
seminars 
and 
conferences,  MS 
coordinate 
activities,  which 
enhances 
understanding 
of  Union  and 
national 
mechanisms, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

December 2014 55 

 
 
 
following  accession  to 
the  EU.  The  action 
the 
with 
dealt 
EU's 
of 
protection 
interests 
financial 
the  European 
under 
enlargement 
policy, 
of 
reporting 
irregularities  to  OLAF 
and 
and 
recovery 
fraud 
follow-up 
related cases. 

on 

Grant nr: 

TRAIN  2011  RO  D7 
056 

19 

Thematic  Anti-Fraud 

Coordination  Services 
(AFCOS) conferences.  

1)  To  bring  together 
AFCOS  representatives 
the  Member 
from 
States, Acceding 

States, 
Candidate 
Countries and potential 
Candidate Countries; 

(2) to present the work 
of  OLAF  and  AFCOS 
and their cooperation; 

(3) 
discuss 
challenges  relevant  to 

to 

Multiple  2007-2013 
(the  case 
study  will 
look  closer 
at 
2012 
and 2013) 

(2012) 
Dubrovnik: 
€98,209.17 

(2013) 
Ankara: 
€108,437.17 

 Evaluation of Hercule II Programme 

Procurement  H10:  

H9:  

and 

organising 

By 
trainings, 
seminars 
conferences, 
exchanging 
in 
experiences 
Member 
the 
States 
is 
increased,  which 
enhances 
understanding  of 
Union 
and 
national 
mechanisms, 
leading 
prevention 

to 
of 

organising 

By 
trainings, 
seminars 
and 
conferences,  MS 
coordinate 
activities,  which 
enhances 
understanding 
of  Union  and 
national 
mechanisms, 
leading 
to 
of 
prevention 
losses  to  the  EU 
budget. 

December 2014 56 

 
 
 
IT Support 

20 

Specific 

mutual cooperation; 

(4) to share experience 
and best practices. 

Grant nr: 

TRAIN  AFCOS  CONF 
2012 2013 

Access 
external 
to 
database  China  Trade 
Information  Database 
(CTI) 

This  case  study  will 
the  actions 
address 
the  creation 
funding 
and 
of 
renewal 
subscription to external 
database CTI. 

OLAF 

Yearly 

(2010) 
Goodwill 
China 
Business 
2010: 
€30,036 

(2010) 
Goodwill 
China 
Business 
2011: 
€31,536 

 Evaluation of Hercule II Programme 

losses  to  the  EU 
budget. 

IT 

Procurement  H15: 

n/a 

By  providing  IT 
data 
support, 
and 
access 
exchange 
are 
facilitated,  which 
improves 
investigations, 
monitoring  and 
intelligence 
work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

December 2014 57 

IT 

(2011) 
External 
Database 
China  Trade 
2012: 
€43,310 

(2011) 
Goodwill 
China 
Business 
2010-2011 
Chapter 

IT 

9: 

 
 
 
 
 
 Evaluation of Hercule II Programme 

€1,636 

(2012) 
External 
Database  CTI 
(1/1 
– 
31/3/2013): 
€11,215 

(2013) 
External 
Database  CTI 
(1/4 
– 
31/12/2013): 
€34,184.5 

(2013) 
External 
Database  CTI 
(1/1 
- 
31/12/2014): 
€59,99681 

21 

Thematic  Automated  Monitoring 

OLAF 

Tool 

This  case  study  will 
the  actions 
address 
the 
funding 
development  by 
the 
JRC of the AMT, aiming 
at 
providing 
intelligence  to  national 
authorities. 

2011/2013  (2008)82  AMT 
3: 

Step 
€270,299 

(2008)  AMT 
3: 
Step 
€247,189 

(2011/2012) 
AMT  Step  4 
(24  months): 
€400,000 

Procurement 

H14: 

n/a 

(JRC) 

By  providing  IT 
support,  specific 
databases and IT 
are 
tools 
developed, which 
improves 
investigations, 
monitoring  and 
intelligence 

81 Committed in 2012. 
82 To be verified. 

December 2014 58 

 
 
 
                                           
 Evaluation of Hercule II Programme 

(2013)  AMT 
Step 
4: 
€25,300 

work,  leading  to 
prevention 
of 
losses  to  the  EU 
budget. 

December 2014 59 

 
 
 
 
 
 
During the case studies 49 different persons were interviewed.  

 Evaluation of Hercule II Programme 

Step 6: Analyse and report on the contribution of Hercule II 

Once  the  case  studies  had  been  finalised,  the  hypotheses  were  divided  between  the 
different  evaluation  questions  they  cover,  and  the  relevant  findings  from  the  case 
studies on the strength of each hypothesis were drawn together.  

In the first step, the embedded theory of change, a draft of which was developed in the 
inception  phase,  was  updated  to  show  how  the  different  activities  of  the  Hercule  II 
Programme  have  led  to  the  outputs,  outcomes  and  finally,  impacts.  The  updated 
embedded theories of change are included in the main body of the report, section 2.4.  

For each of the themes covered under the overall evaluation question on effectiveness 83, 
a contribution story was developed. A contribution story is a narrative, which focusses 
on the theme at hand  and provides a concrete description of each hypothesis covered 
and  each  of  the  linkages  in  these  hypotheses,  based  on  the  case  studies  carried  out. 
The  narratives  provide  an  assessment  of  the  extent  to  which  the  programme  reached 
its specific objectives. The contribution stories are all collected together and presented 
in  Annex  B.  In  these  contribution  stories  the  strength  of  linkages  is  specified  for  each 
hypothesis and each linkage. The linkages are assessed as either being strong, medium 
or weak. These are based on the findings in the case studies and are used as follows: 

Strong  evidence:  all  or  the  majority  of  the  evidence  in  the  case  study  points  to  the 
existence  of  the  linkage.  Examples  can  be  provided  and  there  is  both  qualitative  and 
(where possible) quantitative evidence to support the existence of the linkage. 

Medium  evidence:  some  evidence  in  the  case  study  points  to  the  existence  of  the 
linkage. However, this evidence cannot always be triangulated due to the lack of data, 
but the relevant stakeholders all agree to the existence of the linkage. 

Weak  evidence:  there  is  none  or  limited  (i.e.  statement  by  one  person)  evidence  of 
the existence of a linkage. No quantitative evidence could be provided in support of the 
linkage. 

83 As this evaluation criterion is the only one where the contribution of the programme 
to its objectives was examined, it is also the only one where contribution analysis has 
been employed directly. 

December 2014 60 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                           
CONTRIBUTION STORIES 
Annex B 

mmmll 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Table of Contents 

Introduction ...................................................................................................... 3 
Reading annex B ............................................................................................. 3 
Contribution  story  1:  Enhanced  transnational  and  multidisciplinary  cooperation 
between Member States’ authorities and the Commission ....................................... 4 
Contribution  Story  2:  Building  networks  throughout  the  Member  states,  acceding 
countries and candidate countries ........................................................................ 8 
Contribution  story  3:  facilitating  the  exchange  of  information,  experience  and  best 
practices .........................................................................................................10 
Contribution  story  4:  provision  of  technical  and  operational  support  for  the  law 
enforcement authorities of the Member States in their fight against illegal cross border 
activities, emphasizing support for customs authorities .........................................12 
Contribution  story  5:  Facilitating  data  access  and  analysis  through  specific  databases 
and IT tools provided through the programme .....................................................16 
Contribution story 6: support to the law enforcement authorities in their fight against 
illegal  cross  border  activities  through  IT  tools  provided  for  investigations,  monitoring 
and intelligence work ........................................................................................19 
Contribution story 7: Multiplying and intensifying the measures in the areas identified 
as the most sensitive, in particular cigarette smuggling and counterfeiting ..............21 

December 2014 2 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Introduction 
Annex  B  contains  7  contribution  stories  which  relate  to  the  core  evaluation  questions 
(all  evaluation  questions  except  for  “Geographical  balance”).  The  contribution  stories 
provide  the  detailed  analysis  behind  the  contribution  analysis  presented  in  the  final 
report.  Similar  to  annex  D,  the  present  annex  does  not  contain  any  evaluative 
conclusions,  since  it  only  provides  background  information  on  data  collection.  Aside 
from  the  contribution  stories,  the  evaluation´s  conclusions  are  based  on  several 
additional  data  sources,  including  stakeholder  interviews  and  surveys,  and  therefore 
the conclusions can be found in the final report.  

Reading annex B 
Annex  B  is  closely  related  to  annex  A,  in  which  the  intervention  logic  is  included.  The 
hypotheses which are presented in this annex were derived from the intervention logic, 
which  was  developed  as  part  of  this  evaluation.  It  is  therefore  recommended  that 
annex A is consulted prior to reading annex B.  

The  primary  evidence  in  annex  B  is  included  in  a  table  under  each  contribution  story. 
Each  table  contains  the  most  important  information  in  relation  to  the  contribution 
analysis.  Column  1  of  the  table  refers  to  the  hypothesis  which  is  being  tested,  and 
column 2 shows which specific link in the hypothesis was examined.  Column 3 lists the 
case studies which were used to test the hypotheses, and column 4 shows the overall 
assessment of the evidence´s strength in support of the specific link. The final column 
5 provides a summary of the findings on each link of the hypotheses. It is important to 
note  that  these  summaries  are  not  the  evaluations  conclusions  (those  are  in  the  final 
report).  These  summaries  are  partial  assessments  of  the  evidence  in  support  of  each 
hypothesis´ specific link. For ease of reference, the row which  is the most informative 
in relation to the overall evaluation question is highlighted in turquoise in all tables. 

In  some  cases,  the  contribution  stories  include  assessments  of  “influencing  factors” 
which  are  believed  to  affect  the  contribution  that  Hercule  II  makes  towards  achieving 
its  objectives.  Importantly,  these  are  contextual  factors  which  the  programme  cannot 
influence  directly.  Influencing  factors  can  roughly  be  divided  into  “inhibitors”  and 
“drivers”.  Inhibitors  are  factors  outside  the  programme´s  control,  which  can  make  it 
more difficult for the programme to contribute to its objectives. Conversely, divers are 
factors  which  either  make  it  easier  for  the  programme  to  contribute  towards  its 
objectives, or which strengthen the programme´s contribution to those objectives.  

December 2014 3 

 
 
 
 
 
 
 
 
 
Contribution  story  1:  Enhanced  transnational  and 
multidisciplinary  cooperation  between  Member  States’ 
authorities and the Commission  
The contribution analysis will first provide a brief overview of the overall findings on the 
two  hypotheses  which  belong  under  evaluation  question  1.2.  This  information  is 
summarised in the table below. 

December 2014 4 

 
 
 
 
 
Table 6 Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

H2 and H7 

H2  

1.  By providing 
technical 
assistance for 
national 
authorities…(lea
ding to link no.1 
or 3) 

2.  …MS authorities 
can acquire 
technical 
equipment 
which (leading 
to link no.4) 

Overall 
strength  of 
evidence  in 
support  of 
the link 
Strong 

Summary of findings 

Two  case  studies  show  that  the  programme’s  support  has  clearly  strengthened  the 
authorities’ technical capacity and helped them to purchase the equipment. 

6 and 8 

6 and 8 

Strong 

The  case  studies  find  that  purchasing  such  equipment  cannot  be  done  through  the 
national budget as it may not be high on the priority list in Member States.  

H7 

3.  …joint 

6 

Medium 

operations and 
staff exchange 
can be enabled 
(leading to link 
no.4) 

H2 and H7 

4.  …enhances 

6 and 8 

transnational 
and 
multidisciplinary 
cooperation 
between the MS 
and Commission 

There was limited evidence showing that joint operations and staff exchanges were 
increased,  although  some  staff  exchange  followed  the  operationalisation  of  the 
equipment  (EE,  LV,  LT,  Case  study  no.  6).  At  the  same  time,  the  communication 
between the Baltic States was increased after the implementation of the equipment. 

Strong: 0 
Medium: 1 
Weak:1 

The  case  studies  provided  demonstrated  that  the  purchase  of  the  technical 
equipment  has  had  limited  effect  on  enhancing  transnational  and  multidisciplinary 
cooperation  between  the  Member  States  and  Commission.  Transnational  and 
multidisciplinary  cooperation  has  only  been  enhanced  as  a  result  of  the  project  to 
some extent - in one case between the Baltic States (EE, LV, LT, Case study no. 6), 
and  in  the  other  between  Spain,  Germany  and  the  UK  (ES,  Case  study  no.  8). 
However, the evidence does not support enhanced cooperation between the Member 
States  and  the  Commission.  In  short  the  case  studies  suggest  that  technical 
assistance projects had limited impact when it came to  enhancing  cooperation, but 
that some projects did achieve it, although only to some extent.  

December 2014 5 

 
 
 
 
 
 
 
 
 
 
 
 
Hypothesis 
No. 

Analysed link 

Case 
studies  

Overall 
strength  of 
evidence  in 
support  of 
the link 

H9, 
and 12 

H10 

5.  By organising 
training, 
seminars and 
conferences… 
(leading to link 
no.6, 7 or 9) 

14, 
15, 
16, 17, 18 
and 19 

Strong:5 
Medium: 1 
Weak: 0 

H9 

6.  …MS coordinate 

activities which 
… (leading to 
link 8) 

16, 17, 18 
and 19 

Strong:3 
Medium:1 
Weak:0 

Summary of findings 

The case studies showed overall strong evidence in support of it, demonstrating that 
Hercule  II  did  enable  Member  States  in  organising  training,  seminars  and 
conferences. This was  primarily due to the fact that the costs of such events could 
not be shouldered by national administrations alone. In one case study the evidence 
in  support  of  the  link  was  assessed  as  ‘medium’,  because  findings  showed  that 
without  the  programme’s  support  it  might  have  taken  longer  to  ensure  funding  for 
the  training,  but  the  events  would  still  have  taken  place,  although  at  a  later  date 
(RO, Case study no. 18).  

The findings from case studies demonstrate that the increased coordination between 
Member  States  was  generally  achieved  when  the  events  helped  put  in  place 
common  terminology,  definitions  of  fraud  and  irregularities  as  well  as  when  the 
events  highlighted  areas  where  Member  States  could  benefit  from  increased 
cooperation.  In  one  case  study,  it  was  noted  that  focus  on  coordination  may  have 
been somewhat removed due to interest in networking and exchanging experiences 
(Procurement, Case study no.16).  

H10 

7.  …exchanging 

experiences in 
the Member 
States is 
increased 
(leading to link 
no. 8) 

8.  …enhance the 
understanding 
of Union and 
national 
mechanisms  

H9 
H10 

and 

14, 
17, 
and 19 

16, 
18 

Strong:4 
Medium:1  
Weak: 0 

The  case  studies  provided  strong  support,  and  notable  examples  of  exchanges 
include Member States sharing fraud detection techniques and risk analysis activities 
with each other (Procurement, Case study 16). Other examples were the exchange 
of methods and knowledge on detection and analysis techniques (Procurement, case 
study no. 17). 

14, 
17, 
and 19 

16, 
18 

Strong:3 
Medium:1  
Weak:1 

A majority of the case studies provided strong evidence, highlighting that the events 
contributed  to  enhancing  the  understanding  of  relevant  mechanisms,  in  particular 
within the participating newly acceded Member States and candidate countries. The 
case study indicated that the conferences have focused on both Union mechanisms 
as well as relevant national examples from selected Member State. As shown in the 
analysis of the previous links, this happened both due to increased coordination and 

December 2014 6 

 
 
 
 
 
 
 
 
 
 
 
 
Hypothesis 
No. 

Analysed link 

Case 
studies  

Overall 
strength  of 
evidence  in 
support  of 
the link 

H12  

9.  …research and 

15 

Strong 

H12 

dissemination of 
knowledge is 
increased, 
which… (leading 
to link no. 10) 

10. …improves 
cooperation 
between 
practitioners 
and academics 

Summary of findings 

enhanced exchanges of experiences. In this case, the preliminary influencing factor 
(set  out  in  the  interim  report  for  H10)  is  rejected  and  thus  shows  that  Member 
States did find exchanging experiences relevant.  

The case study found strong evidence in support of Hercule II having contributed to 
an  increase  in  research  and  dissemination  of  knowledge.  It  is  worth  noting  that 
knowledge  was  both  disseminated  from  OLAF  to  Member  States  and  in  between 
Member States and mainly through formal presentations.  

15 

Strong 

This sharing of research and knowledge was also shown to have happened between 
practitioners and academics, although limited evidence makes it difficult to provide 
further examples. 

December 2014 7 

 
 
 
 
 
 
 
 
 
 
 
 
Contribution Story 2: Building networks throughout the 
Member  states,  acceding  countries  and  candidate 
countries 
The  contribution  analysis  will  provide  an  overview  of  the  overall  findings  on  the  two 
hypotheses  which  belong  under  evaluation  question  1.2.  This  information 
is 
summarised in the table below. 

December 2014 8 

 
 
 
 
 
 
Table 7 Contribution story findings 

Hypot
hesis 
No. 

H1  and 
H6 

H1 

Analysed link 

Case 
studies  

6 

1.  By providing 
technical 
assistance for 
national 
authorities…  

Overall 
strength  of 
evidence  in 
support  of 
the link 
Strong 

6 

Strong 

2.  …MS authorities 
can acquire 
technical 
equipment, which… 
(leads to link 4) 

H6 

3.  …joint operations 

6 

Medium 

and staff exchange 
can be enabled, 
which… 

H1  and 
H6 

4.  …strengthens 
networks and 
exchange of 
information 

6 

Summary of findings 

The  case  study  showed  that  the  programme  technical  assistance  to  national 
authorities,  which  relies  on  targeted  authorities  actually  applying  for  funding  under 
the  Hercule  II.  In  this  case,  OLAF  provided  50  %  of  the  project  budget,  which 
indicates that the programme did indeed provide significant technical assistance.  

Evidence from the case study indicates that Member States were enabled to acquire 
technical  equipment.  The  case  study  found  that  the  co-financing  from  the 
programme  enabled  the  Members  States  to  purchase  equipment  that  otherwise 
would have been unaffordable. 

The  case  study  found  that  joint  operations  and  staff  exchange  take  place  only  to  a 
limited degree.  However, there have been exchange of  staff and meetings between 
officials  among  the  Member  States  and  with  the  network  in  place,  future  joint 
operations may be possible. 

(for 
Strong 
exchange  of 
information) 
Medium  (for 
strengthenin
g networks) 

The  case  study  found  that  the  networks  and  information  exchange  were 
strengthened, but with room for improvement. Meetings between official specifically 
on  the  equipment  have  strengthened  the  relations  and  network  between  the 
participating  states  and  their  customs  forces.  Staff  exchanges  have  improved 
networking, although the number of participants in these exchanges remains limited. 
The case study also shows that the information exchange is automated through the 
equipment feeding information into the database and  does not rely on interpersonal 
contact. On the level of information networks, however, there has been a significant 
effect, as the Baltic States exchange information via the common database and have 
agreements on information exchange.  

December 2014 9 

 
 
 
 
 
 
 
 
 
 
 
The  interim  report  set  out  two  preliminary  influencing  factors  which  could  potentially 
influence the programme’s ability to strengthen networks and exchange of information, 
placing  emphasis  on  building  networks.  The  first  preliminary  inhibitor  was  that  if 
national authorities are using technical equipment only for domestic issues and not for 
information  exchange  between  Member  States,  then  the  contribution  to  building  and 
strengthening  networks  will  be  weak.  The  second  inhibitor  was  that  if  Member  State 
authorities  did  not  have  the  resources  to  prioritise  and  train  the  exchanged  staff. 
Consequently,  the  joint  operation  and  staff  exchange  have  a  limited  effect  on 
strengthening  networks.  The  case  study  shows  that  the  former  factor  was  confirmed, 
because  the  case  study  points  out,  “the  support  seems  to  have  strengthened  the 
internal  national  intelligence  more  than  cross  border  intelligence”.  The  reluctance  to 
communicate  across  borders  to  a  higher  degree  could  also  be  rooted  in  the  lack  of 
exchange offers, training and motivation to use networks for daily tasks, for instance by 
using  channels  that  would  encourage  direct  communication.  The  second  preliminary 
influencing factor was neither rejected nor confirmed.  

Contribution story 3: facilitating the exchange of 
information, experience and best practices  

The  contribution  analysis  will  provide  an  overview  of  the  overall  findings  on  the  two 
hypotheses  which  belong  under  evaluation  question  1.3.  This  information 
is 
summarised in the table below. It should be noted that the exchange of experiences is 
a cross-cutting output in the sense that it feeds into several outcomes (as shown in the 
embedded theory of change, see section 2.4 in the main body of the report). Therefore, 
findings  from  the  contribution  analysis  under  evaluation  question  1.1  should  also  be 
taken  into  account  when  assessing  the  programme’s  overall  contribution  to  facilitating 
the exchange of information, experience and best practices. 

December 2014 10 

 
 
 
 
 
 
 
 
 
Table 8 Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

H11 
H13 

and 

1.  By organising 

15 

trainings, seminars 
and conferences… 
(leading to link no. 
2 or 3) 

Overall 
strength  of 
evidence  in 
support  of 
the link 
Strong 

H11 

2.  …exchanging 

15 

Medium 

experiences in the 
Member States is 
increased, which… 
(leading to link no. 
4) 

H13 

3.  ..research and 

15 

Medium  

dissemination of 
knowledge is 
increased (leading 
to link no. 4) 

H11 
H13 

and 

4.  raises the 

15 

Medium  

awareness of the 
judiciary and other 
branches of the 
legal profession for 
the protection of 
the financial 
interests of the 
Union 

Summary of findings 

The  case  study  shows  that  the  support  from  Hercule  II  enabled  the  German 
authorities to organise an international conference. The evidence suggested that 
the organisation of such conferences is expensive, and based on this, it is unlikely 
that  the  conference  could  have  been  organised  without  co-financing  from  the 
Hercule II Programme. 

The  evidence  gives  a  mixed  image  of  the  conference’s  effects  on  increasing  the 
exchange  of  experiences  between  Member  States.  The  conference’s  main 
contribution  to  increasing  the  exchange  of  experiences  between  Member  States 
happened  during  the  conference’s  informal  sessions  (the  social  programme), 
where  participants  could  discuss  freely,  for  example  experiences  related  to 
instruments  and  possibilities  which  could  potentially  contribute  positively  to  the 
fight against crimes which threaten the financial interests of the EU. 

The  case  studies  indicate  that  the  conference  made  a  notable  contribution  in 
disseminating  knowledge  from  OLAF  to  Member  States  and  documented  that 
several presentations during the conference explored different EU legal systems. 
Moreover,  the  case  study  suggested  that  the  conference  contributed  to  an 
exchange  of  information  between  speakers  and  participants,  and,  there  were 
indications  that  the  conference  helped  foster  networks  between  officials  from 
different  Member  States,  but  it  went  unconfirmed  whether  these  were  used  to 
increase research and dissemination of knowledge after the conference ended.  

The presentations informed participants of what the EPPO could be  used for and 
helped  emphasize  the  importance  of  the  EPPO.  This  was  assessed  to  have 
increased  the  participants’  awareness  of  the  fact  that  both  Member  States  and 
the EU have financial interests. In this regard, it was judged that the conference 
led  to  an  increased  acceptance  of  the  idea  of  an  EPPO,  and  as  a  result  also 
increased  willingness  to  cooperate  with  the  European  Public  Prosecutor's  Office 
(when  it  has  been  set  up).  In  addition,  the  conference  helped  collecting 
suggestions on how to further develop the draft regulation on the EPPO.  

December 2014 11 

 
 
 
 
 
 
 
 
 
 
 
 
When assessing the strength of evidence supporting link two and three, it is possible to 
compare  the  findings  according  to  the  evaluation  question’s  distinction  between 
information,  experiences  and  best  practices.  The  case  study  indicated  that  the 
conference was particularly successful at contributing to exchanging information on the 
establishment  of  the  EPPO  through  the  formal  session  of  the  conference,  whilst  the 
informal  moments  provided  opportunities  to  exchange  experiences.  There  was  limited 
evidence  suggesting  that  best  practices  were  in  focus  at  the  conference.  This  may  be 
due to the often very practical nature of best practices, which are more likely to relate 
directly  to  anti-fraud  measures,  rather  than  organisational  aspects  or  actions  which 
have a more legal focus.   

The  interim  report  set  out  two  potential  inhibitors,  which  could  have  hindered  the 
project  from  contributing  to  the  exchange  of  information,  experiences  and  best 
practices, as  well as stifled the conference  from raising the awareness of the judiciary 
and other branches of the legal profession for the protection of the financial interests of 
the Union. These were (1) if the conference did not targeted specifically enough to the 
judiciary  and  other  branches  of  the  legal  profession;  and  (2)  if  the  representatives  of 
the  judiciary  and  other  branches  of  the  legal  profession  did  not  see  the  benefit  of 
working transnationally. Both of these influencing factors were rejected, as participation 
in the conference had indeed been targeted at the right participants, who also attended 
the  event.  The  conference  organisers  had  made  a  conscious  effort  in  this  regard  by 
extending 60% of all invitations to specific individuals personally. Although the number 
of  participants  was  slightly  lower  than  expected,  this  could  not  be  shown  to  have  had 
negative  effects  on  the  achievements  of  the  conference.  In  part,  the  rejection  of  the 
two  influencing  factors  may  be  explained  by  an  identified  driver,  namely  that  the 
organising authority had previous experience with organising international conferences 
which  were  co-financed  by  the  programme.  This  is  assessed  ensured  that  the 
conference was a match with the objectives of Hercule II, and that the skills required to 
ensure participation were in place.  

Contribution story 4: provision of technical and 
operational support for the law enforcement authorities 
of the Member States in their fight against illegal cross 
border activities, emphasizing support for customs 
authorities 

The  contribution  analysis  will  provide  an  overview  of  the  overall  findings  on  the  two 
hypotheses  which  belong  under  evaluation  question  1.4.  This  information 
is 
summarised in the table below.  

December 2014 12 

 
 
 
 
 
 
 
 
 
 
 
 
Table 9 Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

H3,  H4  and 
H8 

1.  By 

providing 

for 

technical 
assistance 
national 
authorities… 
(leading  to  link 
no. 2 or 3) 

H3, H4 

2.  …MS 

authorities  can 
acquire 
technical 
equipment… 
(leading  to  link 
no. 4 or 5) 

1,  4,  5, 
6,  7,  8, 
9, 
10 
and 11. 

1,  4,  5, 
7,  8,  9, 
10  and 
11 

Overall 
strength 
of 
evidence 
in  support 
of the link 
Strong: 11 

Medium:2 

Summary of findings 

The  case  studies  showed  that  there  were  no  major  obstacles,  such  as  lack  of  human 
resources  or  a  discouraging  application  procedure,  which  hindered  national  authorities 
applying  for  co-financing.  Yet,  project  managers  did  welcome  changes  which  could 
further simplify the application procedure.  

Strong: 12 

The case studies showed that authorities’ were able to acquire technical equipment with 
support  from  the  programme.  This  relied  on  two  important  elements,  namely,  the 
authorities’  ability  to  carry  out  the  project  (e.g.  procurement  procedures)  and  the 
availability  of  national  co-financing.  All  case  studies  showed  that  the  authorities  were 
indeed able to acquire technical equipment under the programme.  

H8 

3.  …joint 

6 

Medium 

operations  and 
staff  exchange 
can 
be 
enabled, 
which…(leadin
g to link no.4) 

The case study showed that staff exchanges were introduced following the installation of 
the  technical  equipment,  but  the  information  indicates  that  exchanges  and  joint 
operations are still limited. The communication between the Baltic States has increased 
but  mostly  through  the  system  and  not  explicitly  through  meetings  or  any  joint 
activities.  Case  study  no.  6  was  intended  to  be  the  only  case  study  testing  this. 
However,  case  study  5  also  reported  examples  of  cooperation  with  other  law 
enforcement authorities both within and outside Spain as being a direct consequence of 
the  co-financed  equipment84.  Overall,  there  is  medium  evidence  in  support  of  this 
linkage. 

H3 and H8 

4.  …strengthens 

the 

technical 

4,  5,  6, 
7 and 9 

Strong: 3 

The fourth linkage examines whether the technical assistance contributed to providing 
technical  and  operational  support  for  the  law  enforcement  authorities  of  the  Member 

84 Since the case study did not systematically test the H8, it is included only as anecdotal evidence and does not figure in the overall assessment of evidence in support of the third link.  

December 2014 13 

 
 
 
 
 
 
 
 
 
                                           
H4 

and 
operational 
support for law 
enforcement 
and 
authorities. 
5.  …improves 

customs 

investigations, 
monitoring and 
intelligence 
work. 

Medium:1 

States in their fight against illegal cross border activities. 

Weak:1 

1,  4,  5, 
7,  8,  9, 
10  and 
11. 

Strong:3 

Medium:2 

Weak:3 

The  case  studies  showed  that  investigations,  monitoring  and  intelligence  work  was 
improved in four key ways: 
  Operational  support  through  gathering  intelligence:  By  upgrading  the  technical 
equipment  available  to  investigative  units  the  operational  work  of  law  enforcement 
authorities was supported. This was achieved because the very availability of specific 
devices  (e.g.  false  document  detectors,  tracking  devices)  allowed  for  generating  the 
intelligence  used  to  carry  on  the  investigations.  In  the  absence  of  these  tools  it  is 
questionable whether the same quality and quantity of intelligence could be gathered 
(ES, Case study no. 5).  

  Technical  support  through  improving  the  quality  of  evidence:  The  evidence 
indicates  that  the  equipment  strengthened  the  technical  and  operational  support  to 
law  enforcement  and  customs  authorities.  Specifically,  the  equipment  improved  the 
authority’s ability to ensure that recordings can be certified and high quality and used 
in court cases. Regrettably, there were few examples of when the system had actually 
been put to use and what results it has yielded so far (CZ, Case study no. 9). 

  Technical  and  operational  support  through  increased  capacity:  The  equipment 
increased  the  capacity  of  law  enforcement,  which  meant  that  more  operations  could 
take  place  simultaneously,  because  more  equipment  was  available  (such  as  specially 
equipped  vehicles,  IT  hardware  and  software).    Additionally,  the  equipment  has 
provided  new  technical  and  operational  support,  which  has  enabled  law  enforcement 
to more effectively monitor suspects (RO, case study no.4). 

  Technical  and  operational  support  through  increased  cooperation:  Although 
there  is  limited  evidence  showing  that  technical  and  operational  support  was 
strengthened  through  joint  operations  and  staff  exchanges,  the  information indicates 
that  the  programme  has  provided  relevant  technical  support  for  the  customs 
authorities  in  the  Baltic  States.  The  authorities  in  the  three  Member  States  became 
able to access relevant intelligence on potential smugglers and the number of seizures 
appears  to  have  increased  after  the  implementation  of  the  system  (EE,  LT,  LV,  Case 
study no.6). 

December 2014 14 

 
 
 
 
 
In the interim report an influencing factor was attached to both H3 and H8. For H3 the 
influencing factor stated that if there was a lack of human resources in the Member the 
contribution of the equipment towards the protection of the EU budget, may be limited. 
The  assumption  was  that  if  there  were  insufficient  human  resources,  the  equipment 
might not be fully exploited. The evidence collected through the case studies reject this 
influencing  factor,  and  in  several  cases  shows  that  new  equipment  had  increased  the 
efficiency of staff. One tangible example of this was a project implemented in Romania, 
where  IT  experts  received  a  combination  of  hardware,  software  and  training,  which 
provided  them  with  increased  capacity  and  efficiency  in  scanning  hard  drives.  The 
project  manager  said  that  this  has  significantly  reduced  the  resources  needed  for 
certain standard exercises, for example, because they can now scan 60 hard drives in 5 
days instead of two weeks.  

Table 10: New drivers identified 

New  influencing  factors  were  confirmed  through  the  case  studies.  One  notable 
influencing factor which may have contributed positively to the equipment’s usefulness 
for law enforcement authorities was that the equipment suppliers provided training for 
staff  intended  to  use  it  in  four  case  studies  (CZ,  ES,  LV,  RO).  Another  factor  was 
previous  technical  assistance,  in  the  sense  that  the  projects’  achievements  are 
assessed  to  have  been  positively  influenced  by  the  complementarity  of  technical 
equipment  purchased  under  different  projects  increased  the  overall  usefulness  of  the 
equipment  purchased  under  specific  projects.  This  is  especially  clear  in  the  case  of 
OLAF/2011/C5/019,  were  the  equipment  was  used  in  combination  with  equipment 
purchased under OLAF/2009/C4/014 (not reviewed in the case study).  

Table 11: New inhibitors identified 

One  inhibitor  was  that  due  to  the  rapid  development  of  technology,  the  constant 
refinement  of  the  modus  operandi  of  the  criminal  organisations  and  the  changing 
needs  of  the  authorities,  the  equipment  is  only  temporarily  at  the  forefront  of  the 
technological  barrier  (ES).  This  means  that  the  quicker  equipment  is  made 
operational,  the  more  useful  it  will  be.  However,  the  case  studies  uncovered  several 
examples delays in putting the equipment to use.  

One  case  study  showed  that  the  delay  was  caused  by  a  change  in  national  legal 
requirements  that  prolonged  implementation  (LV,  no.7),  whilst  another  case  study 
showed  that  issues  with  getting  permission  to  install  equipment  caused  delays  (LV, 
no.6).  A  third  case  study  demonstrated  that  an  unexpected  delay  in  getting  national 
co-financing affected the start of the procurement process negatively and thus delayed 
the project as a whole (RO). These delays in putting the equipment in use may have 
prevented the programme for reaching its full potential in strengthening technical and 
operational support for law enforcement and customs authorities. 

The  interim  report  also  set  out  an  influencing  factor  specifically  for  H8,  namely  that 
joint  operations  and  staff  exchange  will  not  strengthen  law  enforcement  and  customs 
authorities, if there are not sufficient national resources in the host authority to actively 
involve  the  visiting  staff.  This  influencing  factor  could  neither  be  confirmed  nor 
rejected. No new influencing factors were identified in relation to H8.  

The narrative analysis did not identify any alternative explanations, and it can therefore 
be  concluded  that  the  contributions  listed  above  can  be  accredited  to  the  combined 
impact of the programme.  

December 2014 15 

 
 
 
 
 
 
 
Contribution story 5: Facilitating data access and 
analysis through specific databases and IT tools 
provided through the programme  

The contribution analysis will first provide a brief overview of the overall findings on the 
hypotheses  which  belong  under  evaluation  question  1.5.  As  explained  above,  this 
section  will  only  address  the  two  first  links  of  each  hypothesis.  The  information  is 
summarised in the table below. 

December 2014 16 

 
 
 
 
 
Table 12 Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

H14 
H15 

and  

20 and 21 

1.  By providing 
IT support… 
(leading to 
link 2) 

Overall 
strength  of 
evidence  in 
support  of 
the link 
Strong 

H14  

2.  …specific 

21 

Strong 

databases and 
IT tools are 
developed, 
which… 
(leading to link 
3 and 4) 

Summary of findings 

The  case  studies  showed  that  for  the  duration  of  the  programme  the  IT  sector 
accounted for between 15% and 23% of the annual budgets. A substantial share 
of  this  funding  was  allocated  to  renewing  subscriptions  to  databases.  In  total, 
between EUR 2.2 and EUR 2.4 million were allocated to external databases each 
year, out of a budget for all IT tools ranging between EUR2.2 million (2009) and 
EUR3.3  million  (2013).  The  user  statistics  and  the  surveys  conducted  by  OLAF 
and  by  the  evaluator  indicated  that  there  was  overall  satisfaction  with  this 
support. 

In comparison, EUR 400,000 were allocated to the AMT project from the Hercule 
II  budget  in  2011,  followed  by  an  additional  EUR25,300  in  2013  through  a 
contract  extension  to  finance  a  workshop  with  participants  from  JRC,  OLAF  and 
the Member States. The case studies provide evidence of overall satisfaction with 
this support. 

The evidence provides a more complex picture of how the IT tools developed by 
JRC  and  the  access  to  external  databases  have  facilitated  access  to  and 
exchange  of  data.  IT  systems  (Contraffic  and  AMT)  have  been  developed.  This 
was  done  in  the  framework  of  multiple  consecutive  administrative  agreements 
concluded  between  OLAF  and  JRC  for  the  development  of  the  AMT.  The  case 
study  suggested  that  it  would  not  have  been  possible  to  develop  such  a  tool  at 
national  level.  This  suggested  that  it  makes  good  sense  to  have  such  a  tool 
developed  at  EU  level,  by  the  JRC,  for  the  benefit  of  all  Member  States. 
Moreover, case study indicated that the responsible people at OLAF and JRC are 
considered  highly  competent.  Meanwhile,  another  important  factor  could  be 
shown  to  have  contributed  positively  to  the  development  of  the  tool  has  also 
been  the  close  collaboration  between  the  JRC  and  Member  States  and  the 
information and feedback provided by some Member States. 

December 2014 17 

 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
Hypothesis 
No. 

Analysed link 

Case 
studies  

H15 

20 

3.  …data access 
and exchange 
are facilitated, 
which… 
(leading to 3 
and 4) 

Overall 
strength  of 
evidence  in 
support  of 
the link 
Medium 

Summary of findings 

There  were  two  strands  in  the  case  study  findings.  One  strand  showed  that 
database  users  find  the  information  provided  by  the  databases  useful  for  their 
work.  Here,  it  was  also  indicated  that  Member  States  could  potentially  acquire 
some data through channels not supported by the programme, it would be more 
costly.  The  other  strand  suggests  that  by  distributing  access  through  national 
contact points, not all authorities within Member states may be benefitting from 
the access and exchange of data. Findings indicate that the same can be said for 
the access across Member States, where some may be benefitting more from the 
access  to  data  than  others.  One  observation  which  provides  some  insight  into 
this is the low number of respondents to the  mini-survey (distributed as part of 
the  case  study)  which  seems  to  be  a  result  of  national  contact  points  being 
reluctant  to  distribute  it.  This  points  us  towards  an  important  inhibitor,  namely 
that some national administrations may not have the ability to make  use of the 
access  to  information  provided.  This  could  be  happened  due  to  a  lack  of 
awareness, which means that information about the access to databases may not 
reach  all  persons  who  would  benefit  from  it.  This  may  be  hindering  the 
programme from reaching its full potential in facilitating access to and exchange 
of data. 

December 2014 18 

 
 
 
 
 
 
 
 
 
 
Contribution story 6: support to the law enforcement 
authorities in their fight against illegal cross border 
activities through IT tools provided for investigations, 
monitoring and intelligence work 

The  contribution  analysis  will  first  provide  a  brief  overview  of  the  overall  findings  on 
linkage  3  of  the  hypotheses  which  belong  under  evaluation  question  1.6.  This 
information is summarised in the table below. 

December 2014 19 

 
 
 
 
 
 
 
 
 
 
Table 13 Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

Overall 
strength  of 
in 
evidence 
support 
of 
the link 

H14 
H15 

and  

2.  …improves 

20 and 21  Strong 

investigations, 
monitoring and 
intelligence 
work… 

Summary of findings 

The  case  study  found  that  the  tested  features  of  the  AMT  tool,  while  not 
completely perfected yet, already help increase the effectiveness and efficiency of 
national customs authorities’ investigations and monitoring work. One of the main 
benefits of the AMT is that it will (with time) produce signals of trade abnormalities 
at a much earlier point in time. The data available for auditors at national level is 
at  least  one  year  old,  whereas  the  AMT  produces  (almost)  real-time  data.  This 
gives the authorities the possibility to react much faster and earlier, possibly even 
at  the  time  of  customs  clearance,  meaning  that  fraudsters  could  potentially  be 
caught  in  action,  increasing  the  number  of  detections.  The  trade  information 
produced  by  the  AMT  has  helped  some  Member  States  initiate  further 
investigations  of  potential  fraudsters,  and  this  has  in  a  few  instances  led  to  the 
recovery of duties and VAT due. However, thus far only small amounts.  

Meanwhile,  there  is  still  room  for  improvement  in  the  timely  availability  of  data 
and  the  signalling  of  trade  abnormalities.  When  these  improvements  have  been 
made it will lead to improvements in customs authorities’ monitoring work that it 
may  discourage  companies  from  attempting  fraud  and  thus  prevent  losses  to  the 
EU budget. 

The  case  study  found  that  the  information  provided  by  the  external  data  bases 
help  national  authorities’  investigations.  Without  access  to  this  information,  the 
authorities would be less successful in preventing illegal imports into the Union. In 
what regards the  CTI database  specifically, there is a clear indication that access 
to  information  from  this  database  helps  prevent  losses  to  the  EU  budget,  as  the 
information  drawn  from  the  database  is  crucial  to  the  investigations  of  national 
customs authorities and helps recover large sums every year.  

December 2014 20 

 
 
 
 
 
 
 
 
 
 
 
Contribution story 7: Multiplying and intensifying the 
measures in the areas identified as the most sensitive, 
in particular cigarette smuggling and counterfeiting 

The  contribution  analysis  will  provide  an  overview  of  the  overall  findings  on  the 
hypothesis which belong under evaluation question 1.8. This information is summarised 
in the table below.  

December 2014 21 

 
 
 
 
 
 
 
Table 14: Contribution story findings 

Hypothesis 
No. 

Analysed link 

Case 
studies  

H5 
H16 

and 

H5 

1.  By providing technical 
for 
assistance 
national  authorities… 
(leads to link 2 and 3) 

2.  MS authorities can 
acquire technical 
equipment, which… 
(leads to link 6) 

Overall  strength 
in 
of  evidence 
support  of  the 
link 
Strong:4 
Medium:  
Weak:  

1,  9,  10 
and 11 

1 

Strong: 

H16 

10 

9, 
and 11 

Strong:1 
Medium: 1 
Weak:1 

Summary of findings 

In general, the case studies clearly demonstrated that the programme had 
co-financed technical equipment with national authorities. 

The  case  study  showed  that  the  equipment  would  not  have  been  bought 
without co-financing (BE, ES, IE, PL, Case study no.1), thus confirming that 
the  programme  enabled  the  authorities  to  acquire  equipment.  The 
preliminary influencing factor (set out in the interim report) suggested that 
if Member States could not contribute with the necessary co-financing, then 
they would not be able to purchase technical equipment. Whilst this factor 
was rejected by the case studies under review here, it cannot be ruled out 
the  budget  restrictions  in  some  Member  States  may  have  prevented 
national authorities from applying for more technical assistance.  

Evidence  was  mixed.  One  case  study  confirmed  that  the  purchased 
equipment promoted common technical standards, specifically by providing 
different  units  within  the  same  authority  with  the  same  equipment. 
Additionally,  this  authority  also  provided  staff  with  training  on  how  to  use 
the  new  equipment  (IT,  Case  study  no.  11).  In  another  case  study,  the 
evidence  in  support  of  the  link  was  less  strong,  but  suggesting  that  there 
had been an increase in the technical standards for the national department 
and  that  it  had  improved  cooperation  between  Police  and  Customs  in  the 
Member States (FI, Case study no.10). Finally, the last case study provided 
evidence that the technical equipment increased the technical standards of 
the  national  authority,  but  no  findings  suggested  that  this  lead  to  the 
promotion of common technical standards (CZ, Case study 9).  

3.  common 

technical 
be 
can 
which 

standards 
promoted, 
…(leads to link 7) 
4.  By  organising  anti-
fraud  training  to  MS 

H17 
H18 

and 

12 
13  

and 

Strong:2 
Medium:  

The  case  studies  demonstrated  that  the  programme  helped  finance 
training,  seminars  and  conference,  which  the  Member  States  would 

December 2014 22 

 
 
 
 
 
 
 
 
 
 
 
Hypothesis 
No. 

Analysed link 

Case 
studies  

Overall  strength 
of  evidence 
in 
support  of  the 
link 
Weak: 

Summary of findings 

otherwise not have organised. 

H17 
H18 

and 

officials  and  other 
actors... (leads to link 
5) 
5.  MS 

are 
trained 
fight 
against fraud, which… 
(leads to link 6 or 7) 

officials 

to 

H5, 
H18 

and 

6.  …multiplies and 

intensifies measures 
in the areas identified 
as the most sensitive, 
particularly in the 
field of cigarette 
smuggling and 
counterfeiting 

and 

12 
13 

Strong:2 
Medium:  
Weak: 

12 

1, 
and 13 

Strong:1 
Medium: 0 
Weak:285 

There was strong evidence showing that these events had in fact provided 
Member State officials with anti-fraud training. 

The case study findings were unable to provide concrete examples of types 
of  intensified  measures.  The  strongest  evidence  in  support  of  the  linkage 
showed  that  the  project  has  intensified  measures  in  the  area  of  cigarette 
smuggling,  by  conducting  working  visits  with  peers  in  the  UK,  France  and 
Romania.  Thereby,  they  improved  their  analytical  capacity  to  counter 
cigarette smuggling and evidence showed that the participants had put this 
knowledge into use in their daily work. The same project also contributed to 
the dissemination and update of the Guidance on Risk Management, which 
was  disseminated  and  enhanced  officials  understanding  about  how  to 
intensify  measures  against  fraud  (BG,  Case  study  no.  13).  Another  case 
study  showed  how  the  project  by  providing  training  helped  increase  the 
knowledge  of  customs  personnel  of  how  cigarettes  may  be  concealed  in 
vehicles  and  how  to  dismantle  vehicles  without  causing  unnecessary 
damage (SK, Case study no.12).  

H16 
H17 

and 

7.  …leads to unified level 
of skills, equipment 
and readiness in the 
MS to detect 
smuggled and 
counterfeit products 

10, 
12 

9, 
11, 
and 13 

Strong:1 
Medium: 3 
Weak:1 

A  case  study  showed  that  thanks  to  the  purchase  of  new  equipment,  and 
the  subsequent  promotion  of  common  technical  standards,  contributed 
strongly to a unified level of skills, equipment and readiness in the Member 
States. One case study found that because the same equipment is used by 
a  vast  number  of  law  enforcement  authorities  in  the  EU,  a  higher  level  of 
uniformity of skills and readiness across Member States (IT). 

85 Case study 1: The strength of evidence in support of link 6 was originally assessed to be ’medium-weak’ after a closer review, it is assessed as ’weak’.  

December 2014 23 

 
 
 
 
 
 
 
 
 
 
                                           
The expected preliminary influencing factor, namely that the equipment did not add to 
the  existing  measures  in  the  areas  of  cigarette  smuggling  and  counterfeiting,  could 
neither  be  confirmed  nor  rejected.  Also,  there  was  no  evidence  suggesting  that  anti-
fraud  training  was  not  being  supported  by  hierarchy,  rejecting  another  preliminary 
influencing factor.   

The evidence collected indicated that there were three drivers in achieving succeeded 
at  intensifying  said  measures  through  training.  Firstly,  the  participants  were  experts 
and supported the detailed discussions and, secondly, welcoming and participating host 
administrations  were  (in  the  case  of  working  visits),  which  ensured  that  the  training 
took  place  in  a  supportive  environment  and  facilitated  a  meaningful  exchange  of 
experiences  across  participating  states.  Thirdly,  training  events  were  more  likely  to 
successfully  equip  officials  to  counter  fraud  when  the  organising  authorities  had 
previous  experience  with  planning  and  carrying  out  similar  projects,  and  received  the 
support they required from OLAF. 

December 2014 24 

 
 
 
 
 
 
mmmll 

 
 
 
Examples of concrete 
quantitative results 
Annex C 

mmmll 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Examples of quantitative results across data sources  

Annex C shows only compiled figures from each of the sources listed in the grey rows. 
The  figures  have  not  been  compiled  across  sources  or  treated.  The  Annual  Reports 
from 2012 and 2013, the Impact Assessment from 2011 and the Intermediate Review 
2007, may be using figures from previous reports and project reports. Therefore, the 
quantitative  results  cannot  be  compared  or  compiled  across  the  sources  or  with  the 
individual case study tables. 

Given  that  there  was  no  clear  indication  on  the  quantitative  data  collection  and  data 
management  process  in  the  original  documents,  Ramboll  Management  Consulting 
cannot  make  assumptions  on  how  the  quantitative  data  was  collected  or  treated 
before being compiled in the present table. 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Germany  German 
Ministry 
Finance 
Customs 

of 
– 

Annual Report 2013 

1 October 
2012- 

Mobile 
scanner 

x-ray 

 31 
December 
2013 

scans 

17,318 
performed 
Seized items: 
20 kg raw opium 
15,840 
polo shirts 
22 kg hashish 
150,000 cigarettes 

counterfeit 

Spain 

n/a 

Guardia  Civil 
together  with 
Spanish 
customs 

Equipment 
targeting 
cigarette 
smuggling 
counterfeiting 

and 

Spain 

Guardia  Civil 
in cooperation 
with 
Portuguese 
Guardia 

n/a 

False  document 
detectors 
and 
tracking devices 

10 persons arrested 
Seized items: 
465,440  packets  of 
cigarettes  
Financial 
impact 
estimated  at  EUR  2.2 
million 

7 persons arrested 
Seized items: 
25,000 
cigarettes  
1,000,000 

packets 

of 

86  A  systematic  quantitative  data  collection  was  conducted  from  the  Annual  Reports  (2013,  2012,  2011),  the 
Impact  Assessment Report  (2011),  the  Intermediate  Review Report  2007 and from the case studies  reports.  The 
quantitative data was centralised in a database. For the data collected for the countries recurring in the database, 
an  in-depth  data  analysis  was  realised  and  data  validation  measures  were  taken  in  order  to  address  the  risk  of 
overlapping  data  as  well  as  ensure  data  accuracy.  The  methods  used  for  data  validation  included  data  screening 
and  data  verification with the  performance  of  relational tests  between  different  variables  that fall under  potential 
overlapping categories and, thereafter, the manual reconciling of invalid/overlapping values. However, the methods 
applied  in  order  to  increase  data  validity  do  not  fully  preclude/mitigate  the  risk  of  overlapping  data,  which  may 
occur  due  to  insufficient  details  regarding  the  quantitative  data  in  the  original  documents  from  which  the 
quantitative data was extracted. 

December 2014 2 

 
 
 
 
 
 
 
 
                                           
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Nacional 
Republicana 

counterfeited 
garments 
Cash 

Spain 

Guardia Civil  

n/a 

Tracking devices 

packets 

24 persons arrested 
Seized items: 
15,470 
cigarettes 
Financial 
estimated 
62,500 

at 

impact 
EUR 

of 

Spain 

Guardia Civil 

n/a 

Undercover  video 
surveillance 
camera 

72 persons arrested 
Financial 
impact 
estimated  at  EUR  8.9 
million 

Romania  Romanian 

2013 

National 
Anticorruption 
Directorate 

Technical  and  IT 
forensic 
equipment 

Greece 

Hellenic 
Financial 
Police  

n/a 

Forensic  Software 
and Devices 

17  investigations  (14 
in 2013) 
25 persons arrested 
Seized items: 
(among 
51,000 
cigarettes 
Financial 
impact 
estimated  at  EUR  75 
million 

others) 
of 

packets 

lubricants 
(impact 
at  more 

8  persons  arrested 
(fuel smuggling) 
6  persons  arrested 
(cigarette smuggling) 
Seized items: 
55 tons of oil 
1,158 LPG cylinders 
1,102 
barrels 
estimated 
than EUR 1,000,000); 
Numerous 
tanks  
16 tank trucks  
Approximately 
1,620,630 
packets  
540  tobacco  packs  (25 
gram each) 
Weapons 

illegal  oil 

cigarette 

December 2014 3 

 
 
 
 
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Latvia 

Latvian  State 
Revenue 
Service 

1 November 
2013 - 28 
February 
2014 

Unloading 

Fuel 
System 

Annual Report 2012 

Lithuania  Lithuanian 

2011 

Law 
Enforcement 
Agency 

Spain 

Spanish  Law 
Enforcement 
Agency 

2011 

Upgraded 
and 
software 
hardware 
for 
computer forensic 
investigation 
and 
duplicator 

forensic 

and 

Cameras 
transmission 
devices 

Finland 

Law 

Finnish 
Enforcement 
Agency 

2011 

Technical 
equipment 

Other vehicles  

individual 

26 infringements 
Seized items: 
6,700 
cigarettes  
9,079  litres  of  illegal 
fuel 
Detection  of  fuel  for 
which excises were not 
paid  (based  on  the  old 
equipment) 

38 suspects identified 
Financial 
impact 
estimated  at  EUR  19.6 
million 

imports 

31  persons  arrested 
(illegal 
of 
hydrocarbons) 
14  persons  arrested 
(fraud concerning VAT) 
9  persons  arrested 
(counterfeiting  of  Euro 
notes and coins) 
Seized items: 
Financial 
impact 
estimated  at  EUR  60 
million  (illegal  imports 
of hydrocarbons) 
Financial 
impact 
estimated  at  EUR  5 
million 
(fraud 
concerning VAT) 
Financial 
impact 
estimated  at  EUR  240 
050  (counterfeiting  of 
Euro notes and coins) 

10 suspects brought to 
court 
Seized items: 

December 2014 4 

 
 
 
 
 
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Romania   Romanian 

2011 

Law 
Enforcement 
Agency 

Technical 
equipment 

Denmark  Danish 

Law 

Enforcement 
Agency 

December 
2011- 
June 2012 

Scanners 
items) 

(6 

Poland 

Law 

Polish 
Enforcement 
Agency 

Annual Report 2011 

2011 

Technical 
equipment 

Slovakia  Slovak 

Law 

2012 

Enforcement 
Agency 

Romania  Romanian 

n/a 

Law 
Enforcement 
Agency 

Technical 
equipment 

Technical 
equipment 

EUR 200,000 
4 million cigarettes 
impact 
Financial 
estimated  at  EUR  900 
000 

Estimated 
financial 
impact of EUR 300,000 
EUR 3 million (fraud) 
Discovered  the  abuse 
of  EUR  5  million  EU 
funds 

Approximately  30,000 
scans and controls 
370 infringements 
Seized items: 
87,220 cigarettes 
kg 
Almost  24 
smoking tobacco 
Drugs 
(30,000  XTC 
tabs,  8  kg  marihuana 
and 91 kg qat)  
Medicines 
10,000 pieces)  

(almost 

of 

112  persons  arrested 
(12  months  following 
acquisition) 
Seized items: 
32  consignments  of 
illicit goods 
Estimated 
impact 
million 

financial 
of  EUR  2 

large  number  of 
A 
criminals  and  corrupt 
law  enforcement  staff 
arrested 
Seized items: 
Substantial amounts of 
smuggled 
tobacco, 
cigarettes 
Liquors 
commodities 

other 

and 

December 2014 5 

 
 
 
 
 
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Impact Assessment Report 2011 

Ireland  

Port of Dublin 

December 
2009 

Mobile scanner 

Malta 

n/a 

May 2009 

Intermediate Review 2007 

n/a 

2007 

n/a 

All 
actions 
funded 
under 
Hercule 
II 

6 

million 
(January 

Almost  4,000  scans 
(January 2010 and 
June 2011) 
Seized items: 
Over 
cigarettes 
2010-June 2011) 
2,111 litres of spirits 
5,877 litres of wine  
48 kgs of cannabis 
Counterfeit items 
8  million 
(January 2010) 
6.5  million  cigarettes 
(April 2010) 

cigarettes 

capability: 
Increased 
capable  of  scanning  a 
40ft shipping container 
in just 1.5 minutes 
Seized items: 
50 million contraband 
cigarettes  
over  16,000  kilos  of 
tobacco  
over 
million 
contraband  cigarettes 
(2010) 
nearly 
million 
5 
contraband  cigarettes 
(first half of 2011) 

20 

to 

discovered 
EUR 

Frauds 
amounting 
3,462,821 
Seized items: 
21,491,440 cigarettes 
5,500 litres of alcohol 
1,262,900 
fuel/gasoil 

litres  of 

All 

n/a 

2008 

n/a 

Frauds 

discovered 

December 2014 6 

 
 
 
 
 
 
 
 
 
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

actions 
funded 
under 
Hercule 
II 

All 
actions 
funded 
under 
Hercule 
II 

n/a 

2009 

n/a 

amounting  to 
81,175,024 
Seized items: 
266,766,634 
cigarettes 
500,000 
alcohol 
10,030 
fuel/gasoil 

litres 

litres 

  EUR 

of 

of 

discovered 
  EUR 

Frauds 
amounting  to 
Seized items: 
4,056,886  
49,578,069 cigarettes 
8,575,885 
alcohol 

litres  of 

Case study Report 1 

Ireland 2  Dublin Airport  September 

2011-
December 
2013 

Mobile 
x-ray 
inspection  van  to 
scan/monitor l for 
cigarettes 
and 
drugs 

Seized items: 
13  seizures  of  tobacco 
(largest  detection  of 
5,000 cigarettes in one 
bulk) 

January 
2013- 
March 2014 

X-ray scanners 
Video  endoscopes 
(2 items) 

660,800 cigarettes 

Poland 

of 
and 

Customs 
offices 
Gołdap 
Gronowo;  
Border 
Customs 
Enforcement 
in 
Units 
Bezledy  and 
Grzechotki 

Ireland 1  Dublin 

Airport, 
Shannon 
Airport 
Rosslare Port 

and 

October 
2012- 
March 2014 

Cabinet 
scanners 
items) 

X-ray 
(2 

of 

kilograms 

Seized items: 
1,873,000 cigarettes  
202 
tobacco  
Drugs, liquor, weapons  
Estimated 
financial 
value  of  EUR  520,000 
recovered  excise 
in 
duties  and  15,000  in 
VAT 

December 2014 7 

 
 
 
 
 
 
Country  Organisation  Time 

Spain 

n/a 

period 

January 
2013 
December 
2013 

–

 Evaluation of Hercule II Programme 

Type 
equipment 

of 

Quantifiable 
results86 

Backscatter 
scanning  mobile 
(and  3 
system 
years 
maintenance)  
CS 2 

2,857  scans  carried 
out,  of  which  211 
allowed  the  detection 
of illicit goods 
Seized items: 
36,000 
cigarettes 

packs 

of 

Case Study Report 4 

Romania  National 

2011-2012  Software 

Anticorruption 
General 
Directorate 

upgrades87,  

Romania  National 

2011-2012  Surveillance  and 

Anticorruption 
General 
Directorate 

communication 
equipment 

four 

pursuit 
Criminal 
against 
civil 
servants  indicted  and 
sent to court  
Seized items: 
EUR 345,000  

n/a;  only  qualitative 
data available 

Romania  National 

2011-2012  Vehicles (2 items)  n/a;  only  qualitative 

Anticorruption 
General 
Directorate 

Romania  National 

Anticorruption 
General 
Directorate 

data available 

2011-2012  Software, training 
for 
software, 
surveillance  and 
recording 
equipment 

Case Study Report 5 

Enhanced  capabilities: 
scan  60  hard  drives  in 
5  days  instead  of  2 
weeks 

Spain 

Spanish  Law 
Enforcement 
Agency 

October 
2012 

Spain 

Spanish  Law 
Enforcement 

December 
2013 

catcher 

Monitoring 
devices  for  GSM 
communications 
IMSI 
system 
Video  and  audio 
tools 
recording 
and 
tracking 
devices 
Monitoring 
devices  for  GSM 

87 persons arrested 

Seized items: 
57,000 

cigarettes 

87 Digital wireless surveillance system (including repeater kit), digital compact system and modules for audio-video 
recording 

December 2014 8 

 
 
 
 
 
 
 
                                           
 Evaluation of Hercule II Programme 

Country  Organisation  Time 

period 

Type 
equipment 

of 

Quantifiable 
results86 

Agency 

false 

communications 
Portable 
document 
detectors 
than 100 items) 

(more 

garments 

boxes 
100,000 
worth EUR 2 million 
25,000 
smuggled 
worth 
115,000 

of 
cigarettes 
EUR 

boxes 

over 

Case Study Report 6 

Estonia 

Estonian  Tax 
and  Customs 
Board, 
Estonia 

2013 

Automated 
Vehicle  Number 
Plate  Recognition 
System 

Latvia 

National 
Customs 
Board, Latvia 

2013 

Automated 
Vehicle  Number 
Plate  Recognition 
System 

Seized items: 
1,180,000 
smuggled 
cigarettes  (in  2012, 
120,000 
smuggled 
cigarettes) 
202 
kilograms 
narcotics  (in  2012,  43 
kilograms narcotics) 

Seized items: 
1,927,800 
smuggled 
cigarettes  (in  2012, 
smuggled 
224,580 
cigarettes) 

Lithuania  Customs 

2013 

Department, 
Lithuania 

Automated 
Vehicle  Number 
Plate  Recognition 
System 

Seized items: 
88,000,000  smuggled 
cigarettes  (in  2012, 
73,000,000  smuggled 
cigarettes) 

Case Study Report 8 

End of 
2011 - 
present 

Tracking  devices, 
night  vision  and 
video  surveillance 
equipment 

Portugal  Technical 

Support  Unit 
of 
the 
Portuguese 
criminal police 
- 
Polícia 
Judiciária 

10  missions  (NB.  No 
information  on 
clear 
which 
precisely 
missions 
the 
equipment  had  been 
deployed) 
300  missions  per  year 
the 
all 
(with 
equipment, 
including 
the  one  purchased 
through the grant) 

Case Study Report 9 

Czech 

General 

November  Specialised  audio  6 persons suspected 

December 2014 9 

 
 
 
 
 
 
 
 
 
 
 
 
Country  Organisation  Time 

Republic  Directorate  of 

Customs 

period 

2013-
March 
2014 

 Evaluation of Hercule II Programme 

Type 
equipment 

of 

Quantifiable 
results86 

for 

system 
surveillance  
‘CEDAR 
Cambridge  Series 
Forensic 
III 
System: 
Host 
System’ 

kg 

at: 

tobacco 

impact 
EUR 

Seized items: 
680 kg tobacco  
2041 
leaves 
Financial 
estimated 
243,000 
(NB.  Loose  correlation 
between 
the 
equipment  purchased 
under  the  grant  and 
the 
the 
results  of 
operation, 
the 
aforementioned 
example  provided 
is 
inconsistent  with  the 
documentation  stating 
that 
the  equipment 
from  the  grant  only 
became  operational  in 
this 
April,  whereas 
operation  ended 
in 
March) 

Case Study Report 10 

Finland  

Helsinki Police 
Department 
Finnish 
customs 

2010 

and 

Mobile 
electronic 
surveillance 
equipment, 
tracking devices 

Evidence  in  corruption 
and fraud cases 
Seized items: 
Smuggled cigarettes 
Financial 
impact 
estimated  at  EUR  1 
million 

Case Study Report 11 

Italy 

Guardia 
Finanza 

di 

- 
January 
March  2014 
(delayed) 

for 

Kits 
forensic 
extraction 

the 

(NB. 
n/a 
Implementation  report 
finalised 
in  March 
2015) 

December 2014 10 

 
 
 
 
 
 
 
 
 
mmmll 

 
 
 
Beneficiary and participant 
surveys: all observations 

Annex D 

mmmll 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Evaluation of the Hercule II Programme 

Table of Contents 

Beneficiary survey ............................................................................................. 3 
Participant survey .............................................................................................12 

December 2014 2 

 
 
 
 
 
 
 
 Evaluation of the Hercule II Programme 

Beneficiary survey 

What type of an organisation do you represent? 

Customs 
Judiciary 
Law enforcement 
Education 
NGO 
other 
Total 

Which country do you work in? 

Austria 
Belgium 
Bulgaria 
Croatia 
Cyprus 
Czech Republic 
Denmark 
Estonia 
Finland 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Latvia 
Lithuania 
Luxembourg 
Malta 
Netherlands 
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 
United Kingdom 
None of the above, please specify 
Total 

Respondents 
20 
2 
30 
5 
5 
6 
68 

Respondents 
1 
2 
5 
0 
1 
2 
0 
4 
1 
2 
2 
0 
3 
2 
12 
3 
3 
0 
0 
0 
10 
0 
9 
1 
0 
5 
0 
0 
0 
68 

Percent 
29,4% 
2,9% 
44,1% 
7,4% 
7,4% 
8,8% 
100,0% 

Percent 
1,5% 
2,9% 
7,4% 
0,0% 
1,5% 
2,9% 
0,0% 
5,9% 
1,5% 
2,9% 
2,9% 
0,0% 
4,4% 
2,9% 
17,6% 
4,4% 
4,4% 
0,0% 
0,0% 
0,0% 
14,7% 
0,0% 
13,2% 
1,5% 
0,0% 
7,4% 
0,0% 
0,0% 
0,0% 
100,0% 

December 2014 3 

 
 
 
 
  
  
 Evaluation of the Hercule II Programme 

The  latest  project  you  have  been  involved  in,  which  has  been  (co-)financed  by  OLAF 
under the Hercule II programme? 

Respondents 

Percent 

Technical  assistance  for  the  fight  against 
fraud 
Anti-fraud training 
Training,  seminar  or  conference  with  a  legal 
focus 
None of the categories above 
Do not know 
Total 

38 
17 

13 
0 
0 
68 

55,9% 
25,0% 

19,1% 
0,0% 
0,0% 
100,0% 

Has your organisation received funding for more than one project under the Hercule II 
programme? 

Yes 
No 
Total 

Respondents 
40 
28 
68 

Percent 
58,8% 
41,2% 
100,0% 

If yes, what other programme categories have you been involved in (please tick all the 
relevant categories) 

Respondents 

Percent 

Technical  assistance  for  the  fight  against 
fraud 
Anti-fraud training 
Training,  seminar  or  conference  with  a  legal 
focus 
None the categories above, please elaborate 
Do not know 
Total 

25 
14 

8 
0 
3 
40 

62,5% 
35,0% 

20,0% 
0,0% 
7,5% 
100,0% 

Please  consider  the  latest  project  you  were  involved  in  (under  the  Hercule  II 
programme) and select from the options below the one which best describes your role 
in the project 

Project manager 
Project owner 
Project user 
Other 
Total 

Respondents 
48 
3 
9 
8 
68 

In which Member State was your latest project conducted? 

Austria 
Belgium 
Bulgaria 
Croatia 
Cyprus 
Czech Republic 

Respondents 
1 
1 
4 
0 
1 
1 

Percent 
70,6% 
4,4% 
13,2% 
11,8% 
100,0% 

Percent 
1,7% 
1,7% 
6,8% 
0,0% 
1,7% 
1,7% 

December 2014 4 

 
 
 
  
  
 
  
  
  
Denmark 
Estonia 
Finland 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Latvia 
Lithuania 
Luxembourg 
Malta 
Netherlands 
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 
United Kingdom 
Total 

 Evaluation of the Hercule II Programme 

0 
4 
1 
3 
1 
0 
3 
2 
10 
2 
3 
0 
0 
0 
9 
0 
7 
1 
0 
5 
0 
0 
59 

0,0% 
6,8% 
1,7% 
5,1% 
1,7% 
0,0% 
5,1% 
3,4% 
16,9% 
3,4% 
5,1% 
0,0% 
0,0% 
0,0% 
15,3% 
0,0% 
11,9% 
1,7% 
0,0% 
8,5% 
0,0% 
0,0% 
100,0% 

Did  the  latest  project  that  you  were  involved  in  encompass  more  than  one  Member 
State or Third Country? 

Yes 
No 
Total 

Respondents 
25 
34 
59 

Percent 
42,4% 
57,6% 
100,0% 

Was the support for your latest project awarded as a grant or through a procurement 
contract? 

Grant 
Procurement contract 
Do not know 
Total 

What is the status of your latest project? 

Start-up phase 
Ongoing 
Completed 
Dropped during its implementation 
Other 
Total 

Respondents 
57 
2 
0 
59 

Respondents 
1 
32 
25 
0 
1 
59 

Percent 
96,6% 
3,4% 
0,0% 
100,0% 

Percent 
1,7% 
54,2% 
42,4% 
0,0% 
1,7% 
100,0% 

December 2014 5 

 
 
 
 
  
  
  
 Evaluation of the Hercule II Programme 

Has the project enhanced transnational cooperation between the Member States? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know 
Total 

Respondents 
9 
6 
13 
29 
11 
68 

Percent 
13,2% 
8,8% 
19,1% 
42,6% 
16,2% 
100,0% 

Has  the  project  enhanced  transnational  cooperation  between  the  Member  States  and 
the Commission? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know 
Total 

Respondents 
9 
9 
20 
19 
11 
68 

Percent 
13,2% 
13,2% 
29,4% 
27,9% 
16,2% 
100,0% 

In  your  opinion,  has  the  project  enhanced  multidisciplinary  cooperation  between 
Member States’ authorities, the Commission and OLAF? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know 
Total 

Respondents 
8 
14 
22 
21 
3 
68 

Percent 
11,8% 
20,6% 
32,4% 
30,9% 
4,4% 
100,0% 

To  what  extent  do  you  agree  with  the  following  statement?  The  project(s)  has  built 
strong networks between Member States 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know 
Total 

Respondents 
3 
10 
10 
14 
18 
13 
68 

Percent 
4,4% 
14,7% 
14,7% 
20,6% 
26,5% 
19,1% 
100,0% 

To  what  extent  do  you  agree  with  the  following  statement?  The  project(s)  has 
facilitated exchange of information, experience and best practices 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know 

Respondents 
2 
6 
7 
15 
32 
6 

Percent 
2,9% 
8,8% 
10,3% 
22,1% 
47,1% 
8,8% 

December 2014 6 

 
 
 
  
  
  
  
  
 Evaluation of the Hercule II Programme 

Total 

68 

100,0% 

To  what  extent  do  you  agree  with  the  following  statement?  The  technical  and 
operational  support  provided  by  the  project  has  improved  the  effectiveness  of  my 
organisation in fighting illegal cross border activities 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know 
Total 

Respondents 
1 
1 
4 
20 
36 
6 
68 

Percent 
1,5% 
1,5% 
5,9% 
29,4% 
52,9% 
8,8% 
100,0% 

To  what  extent  do  you  agree  that  the  Hercule  II  programme  has  intensified  the 
available measures in targeting cigarette smuggling and counterfeiting? 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
1 
2 
3 
21 
30 
11 
68 

Percent 
1,5% 
2,9% 
4,4% 
30,9% 
44,1% 
16,2% 
100,0% 

Please assess the following statement: The desired project results have been achieved 
at reasonable costs 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
1 
1 
20 
33 
4 
59 

Percent 
0,0% 
1,7% 
1,7% 
33,9% 
55,9% 
6,8% 
100,0% 

Please  assess  the  following  statement:  The  procurement  of  specialised  equipment 
through the project(s) has enabled our organisation to save money 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
0 
1 
12 
15 
5 
33 

Percent 
0,0% 
0,0% 
3,0% 
36,4% 
45,5% 
15,2% 
100,0% 

December 2014 7 

 
 
 
  
  
  
 
  
 Evaluation of the Hercule II Programme 

Please assess the following statements: Our organisation has been able to save money 
as a result of the specialised training activities we have participated in 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Did the project(s) start on time? 

Yes 
No 
Do not know 
Total 

Respondents 
1 
0 
5 
5 
8 
7 
26 

Respondents 
49 
9 
1 
59 

What was the reason that the project did not start on time? 

The contract was signed late 
The financing arrived late 
Lack of adequate personnel at project start 
External contextual issues delayed the start 
The project needed to beredefined 
New  partners  needed  to  be  included  in  the 
project 
Other, please specify 
Total 

Respondents 
5 
0 
0 
3 
1 

0 
2 
9 

Percent 
3,8% 
0,0% 
19,2% 
19,2% 
30,8% 
26,9% 
100,0% 

Percent 
83,1% 
15,3% 
1,7% 
100,0% 

Percent 
55,6% 
0,0% 
0,0% 
33,3% 
11,1% 

0,0% 
22,2% 
100,0% 

Did  you  receive  the  payments  according  to  the  payment  schedule  set  in  the  grant 
agreement? 

Yes 
No 
Do not know 
Total 

Respondents 
52 
2 
3 
57 

Percent 
91,2% 
3,5% 
5,3% 
100,0% 

To what extent do you agree that the grant received from OLAF was sufficient in order 
for the project to reach its objectives? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know 
Total 

Respondents 
0 
2 
13 
42 
2 
59 

Percent 
0,0% 
3,4% 
22,0% 
71,2% 
3,4% 
100,0% 

December 2014 8 

 
 
 
  
  
  
  
  
 Evaluation of the Hercule II Programme 

To  what  extent  do  you  agree  with  the  following  statements,  considering  the  latest 
project you have been involved in? The Hercule II programme's application procedure 
has been straight forward and easy to complete 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
3 
6 
28 
20 
2 
59 

Percent 
0,0% 
5,1% 
10,2% 
47,5% 
33,9% 
3,4% 
100,0% 

To  what  extent  do  you  agree  with  the  following  statements,  considering  the  latest 
project  you  have  been  involved  in?  The  reporting  requirements  during  the  project’s 
implementation have been easy to meet 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
5 
4 
33 
13 
4 
59 

Percent 
0,0% 
8,5% 
6,8% 
55,9% 
22,0% 
6,8% 
100,0% 

To  what  extent  do  you  agree  with  the  following  statements,  considering  the  latest 
project  you  have  been  involved  in?  The  support  received  by  OLAF  during  the  project 
implementation has contributed to the project reaching the desired results 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
1 
0 
5 
21 
26 
6 
59 

Percent 
1,7% 
0,0% 
8,5% 
35,6% 
44,1% 
10,2% 
100,0% 

To  what  extent  do  you  agree  with  the  following  statement?  The  results  of  the 
implemented project match the needs and problems experienced in our country 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
3 
0 
3 
24 
35 
3 
68 

Percent 
4,4% 
0,0% 
4,4% 
35,3% 
51,5% 
4,4% 
100,0% 

December 2014 9 

 
 
 
  
  
  
  
 Evaluation of the Hercule II Programme 

Taking  into  account  the  costs  of  participation  and  the  benefits  of  the  project,  how 
useful did you find the participation? 

Not useful 
Somewhat useful 
Useful 
Very useful 
Do not know/cannot assess 
Total 

Respondents 
0 
2 
15 
51 
0 
68 

Percent 
0,0% 
2,9% 
22,1% 
75,0% 
0,0% 
100,0% 

Taking  into  account  the  costs  of  participation  and  the  benefits  of  the  project,  how 
satisfied have you been with your participation? 

Very dissatisfied 
Dissatisfied 
Satisfied 
Very satisfied 
Do not know/cannot assess 
Total 

Respondents 
0 
1 
17 
50 
0 
68 

Percent 
0,0% 
1,5% 
25,0% 
73,5% 
0,0% 
100,0% 

Please  state  to  what  extent  you  agree  with  the  following  statements?    Our 
organisation’s  objectives  are  better  achieved  today  as  a  result  of  participating  in  the 
Hercule II programme 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
1 
1 
35 
29 
2 
68 

Percent 
0,0% 
1,5% 
1,5% 
51,5% 
42,6% 
2,9% 
100,0% 

Please state to what extent you agree with the following statements? If the Hercule II 
programme  had  not  existed,  other  EU  level  programmes  or  initiatives  could  have 
achieved the same objectives 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
6 
25 
13 
3 
2 
19 
68 

Percent 
8,8% 
36,8% 
19,1% 
4,4% 
2,9% 
27,9% 
100,0% 

Please state to what extent you agree with the following statements? If the Hercule II 
programme  had  not  existed,  other  national  programmes  or  initiatives  could  have 
achieved the same objectives 

Strongly disagree 
Disagree 

Respondents 
13 
31 

Percent 
19,1% 
45,6% 

December 2014 10 

 
 
 
  
  
  
  
  
 Evaluation of the Hercule II Programme 

Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

6 
2 
2 
14 
68 

8,8% 
2,9% 
2,9% 
20,6% 
100,0% 

Please state to what extent you agree with the following statement(s)? - The technical 
equipment is used after the project has ended 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
1 
0 
1 
6 
27 
3 
38 

Percent 
2,6% 
0,0% 
2,6% 
15,8% 
71,1% 
7,9% 
100,0% 

Please state to what extent you agree with the following statement(s)?  - The persons 
operating the technical equipment have the necessary skills and qualifications 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
0 
1 
11 
23 
3 
38 

Percent 
0,0% 
0,0% 
2,6% 
28,9% 
60,5% 
7,9% 
100,0% 

Please state to what extent you agree with the following statement(s)?  - The lessons 
learned  from  participating  in  the  programme  are  being  used  after  the  end  of  the 
project 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
0 
0 
0 
24 
39 
5 
68 

Percent 
0,0% 
0,0% 
0,0% 
35,3% 
57,4% 
7,4% 
100,0% 

December 2014 11 

 
 
 
 
 
  
  
  
 Evaluation of the Hercule II Programme 

Participant survey 

Which country do you work in? 

Austria 
Belgium 
Bulgaria 
Croatia 
Cyprus 
Czech Republic 
Denmark 
Estonia 
Finland 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Latvia 
Lithuania 
Luxembourg 
Malta 
Netherlands 
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 
United Kingdom 
None of the above 
Total 

Respondents 
15 
10 
39 
24 
4 
28 
5 
15 
5 
14 
53 
11 
23 
11 
39 
22 
16 
3 
7 
7 
29 
7 
61 
5 
8 
31 
7 
12 
63 
574 

What type of an organisation do you represent? 

Customs authorities 
Judicial authorities 
Law enforcement authorities 
Research and educational institutes 
Non profit organisation 
Other 
Total 

Respondents 
104 
56 
284 
31 
35 
63 
573 

Percent 
2,6% 
1,7% 
6,8% 
4,2% 
0,7% 
4,9% 
0,9% 
2,6% 
0,9% 
2,4% 
9,2% 
1,9% 
4,0% 
1,9% 
6,8% 
3,8% 
2,8% 
0,5% 
1,2% 
1,2% 
5,1% 
1,2% 
10,6% 
0,9% 
1,4% 
5,4% 
1,2% 
2,1% 
11,0% 
100,0% 

Percent 
18,2% 
9,8% 
49,6% 
5,4% 
6,1% 
11,0% 
100,0% 

December 2014 12 

 
 
 
  
  
 Evaluation of the Hercule II Programme 

Please categorise the latest training, seminar or conference you have participated in? 

Respondents 

Percent 

Conferences/seminars  organised  by  OLAF 
(anti-fraud part) 
Conferences/seminars  co-funded  by  OLAF 
(grants - anti-fraud part) 
E-learning 
Meeting 
Round Table 
Workshop 
Symposium 
Exchange of experience and good practices 
Teamwork  visits/Exchange  of  specialised 
staff 
Technical Training 
Digital forensics training 
Conference and seminars (legal part) 
Publication  and  distribution  of  scientific 
periodical  on  the  protection  of  the  EU's 
financial interests 
Comparative  Law  study  on  legal  aspects  of 
protection of EU financial interests 
Dissemination activities 
Annual  meeting  of  the  Presidents  of  the 
Associations  for  European  Criminal  Law  and 
for  the  protection  of  the  EU  Financial 
Interests 
None of the above 
Total 

147 

126 
5 
14 
14 
29 
2 
33 

2 
11 
131 
39 

0 

3 
2 

0 
16 
574 

25,6% 

22,0% 
0,9% 
2,4% 
2,4% 
5,1% 
0,3% 
5,7% 

0,3% 
1,9% 
22,8% 
6,8% 

0,0% 

0,5% 
0,3% 

0,0% 
2,8% 
100,0% 

In which Member State was your latest training, seminar or conference held? 

Austria 
Belgium 
Bulgaria 
Croatia 
Cyprus 
Czech Republic 
Denmark 
Estonia 
Finland 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Latvia 
Lithuania 
Luxembourg 
Malta 
Netherlands 

Respondents 
4 
9 
27 
134 
0 
34 
0 
7 
1 
14 
90 
23 
47 
5 
34 
6 
8 
0 
0 
6 

Percent 
0,7% 
1,6% 
4,7% 
23,3% 
0,0% 
5,9% 
0,0% 
1,2% 
0,2% 
2,4% 
15,7% 
4,0% 
8,2% 
0,9% 
5,9% 
1,0% 
1,4% 
0,0% 
0,0% 
1,0% 

December 2014 13 

 
 
 
  
  
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 
United Kingdom 
Total 

 Evaluation of the Hercule II Programme 

27 
0 
70 
1 
0 
22 
0 
5 
574 

4,7% 
0,0% 
12,2% 
0,2% 
0,0% 
3,8% 
0,0% 
0,9% 
100,0% 

When  did  the  latest  training,  seminar  or  conference  that  you  participated  in  take 
place? 

2008 
2009 
2010 
2011 
2012 
2013 
2014 
Total 

Respondents 
2 
1 
1 
12 
39 
259 
260 
574 

Percent 
0,3% 
0,2% 
0,2% 
2,1% 
6,8% 
45,1% 
45,3% 
100,0% 

Have  you  participated  in  more  than  one  training,  seminar  or  conference  under  the 
Hercule II programme? 

Yes 
No 
Total 

Respondents 
221 
353 
574 

Percent 
38,5% 
61,5% 
100,0% 

If  yes,  what  other  categories  of  training(s),  seminar(s)  or  conference(s)  have  you 
participated in? 

Respondents 

Percent 

Conferences/seminars  organised  by  OLAF 
(anti-fraud part) 
Conferences/seminars  co-funded  by  OLAF 
(grants - anti-fraud part) 
E-learning 
Meeting 
Round Table 
Workshop 
Symposium 
Exchange of experience and good practices 
Teamwork  visits/Exchange  of  specialised 
staff 
Technical Training 
Digital forensics training 
Conference and seminars (legal part) 
Publication  and  distribution  of  scientific 
periodical  on  the  protection  of  the  EU's 
financial interests 

73 

56 
2 
19 
14 
41 
5 
24 

1 
16 
83 
13 

3 

33,2% 

25,5% 
0,9% 
8,6% 
6,4% 
18,6% 
2,3% 
10,9% 

0,5% 
7,3% 
37,7% 
5,9% 

1,4% 

December 2014 14 

 
 
 
  
  
  
 Evaluation of the Hercule II Programme 

Comparative  Law  study  on  legal  aspects  of 
protection of EU financial interests 
Dissemination activities 
Annual  meeting  of  the  Presidents  of  the 
Associations  for  European  Criminal  Law  and 
for  the  protection  of  the  EU  Financial 
Interests 
None of the above, please specify 
Total 

4 
6 

6 
2 
220 

1,8% 
2,7% 

2,7% 
0,9% 
100,0% 

In  your  opinion  has  the  latest,  or  previous,  training(s),  seminar(s)  or  conference(s) 
you  have  attended  enhanced  your  understanding  of  EU  mechanisms  protecting  the 
financial interests of the Union? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know/cannot assess 
Total 

Respondents 
19 
39 
243 
258 
15 
574 

Percent 
3,3% 
6,8% 
42,3% 
44,9% 
2,6% 
100,0% 

In  your  opinion  has  the  latest,  or  previous,  training(s),  seminar(s)  or  conference(s) 
you  have  attended  enhanced  your  understanding  of  national  mechanisms  protecting 
the financial interests of the Union? 

Not at all 
To a limited degree 
To some degree 
To a high degree 
Do not know/cannot assess 
Total 

Respondents 
21 
52 
257 
228 
16 
574 

Percent 
3,7% 
9,1% 
44,8% 
39,7% 
2,8% 
100,0% 

Please state to what extent you agree with the following statements in relation to the 
training(s),  seminar(s)  or  conference(s)  you  have  attended?  -  a.  The  cooperation 
between  practitioners  and/or  academics  has  improved  as  a  result  of  my  participation 
in the training(s), seminar(s) or conference(s) 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
7 
15 
61 
286 
161 
44 
574 

Percent 
1,2% 
2,6% 
10,6% 
49,8% 
28,0% 
7,7% 
100,0% 

December 2014 15 

 
 
 
  
  
  
 Evaluation of the Hercule II Programme 

Please state to what extent you agree with the following statements in relation to the 
training(s),  seminar(s)  or  conference(s)  you  have  attended?  -  b.  The  exchange  of 
information,  experience  and  best  practices  between  practitioners  and/or  academics 
has  improved  as  a  result  of  my  participation  in  the  training(s),  seminar(s)  or 
conference(s) 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
4 
17 
44 
250 
228 
31 
574 

Percent 
0,7% 
3,0% 
7,7% 
43,6% 
39,7% 
5,4% 
100,0% 

To  what  extent  do  you  agree  that  the  training(s),  seminar(s)  or  conference(s)  you 
have  attended  have  contributed  to  establishing  and/or  strengthening  networks 
between practitioners and/or academics 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Would you categorise the(se) network(s) as… 

Formal 
Informal 
Do not know/Cannot assess 
Total 

Respondents 
6 
5 
54 
278 
210 
21 
574 

Respondents 
168 
261 
59 
488 

Percent 
1,0% 
0,9% 
9,4% 
48,4% 
36,6% 
3,7% 
100,0% 

Percent 
34,4% 
53,5% 
12,1% 
100,0% 

To  what  extent  do  you  agree  that  the  training(s),  seminar(s)  or  conference(s)  you 
have attended effectively matched the needs of your organisation? 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Respondents 
4 
7 
29 
230 
295 
9 
574 

Percent 
0,7% 
1,2% 
5,1% 
40,1% 
51,4% 
1,6% 
100,0% 

Taking into  account  the  costs  to  yourself  and  your  organisation  (time  and  money)  of 
participation  and  the  benefits  of  the  training(s),  seminar(s)  or  conference(s),  how 
useful did you find your participation? 

Not useful 
Somewhat useful 

Respondents 
3 
26 

Percent 
0,5% 
4,5% 

December 2014 16 

 
 
 
  
  
  
  
  
Useful 
Very useful 
Do not know 
Total 

 Evaluation of the Hercule II Programme 

160 
377 
8 
574 

27,9% 
65,7% 
1,4% 
100,0% 

Taking  into  account  the  costs  of  participation  and  the  benefits  of  the  training(s), 
seminar(s) or conference(s), how satisfied have you been with your participation? 

Very dissatisfied 
Dissatisfied 
Satisfied 
Very satisfied 
Do not know 
Total 

Respondents 
3 
9 
183 
374 
5 
574 

Percent 
0,5% 
1,6% 
31,9% 
65,2% 
0,9% 
100,0% 

Please state to what extent you agree with the following statement? I could not have 
achieved the same benefits by participating in training(s), seminar(s) or conference(s) 
organised at national or regional level 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

10 
55 
118 
199 
151 
41 
574 

1,7% 
9,6% 
20,6% 
34,7% 
26,3% 
7,1% 
100,0% 

Please state to what extent you agree with the following statement(s)? - I have been 
able to/I expect to be able to use the lessons learned from the training(s), seminar(s) 
or conference(s) I have attended, in my daily work. 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know 
Total 

Respondents 
1 
16 
39 
279 
230 
9 
574 

Percent 
0,2% 
2,8% 
6,8% 
48,6% 
40,1% 
1,6% 
100,0% 

Please  state  to  what  extent  you  agree  with  the  following  statement(s)?  -  The 
network(s)  established  and/or  strengthened  through  my  participation  in  training(s), 
seminar(s) or conference(s) still exist. 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know 
Total 

2 
7 
39 
285 
146 
9 
488 

0,4% 
1,4% 
8,0% 
58,4% 
29,9% 
1,8% 
100,0% 

December 2014 17 

 
 
 
  
  
 Evaluation of the Hercule II Programme 

Please  state to what  extent you agree  with the following statement(s)?  -  I  still make 
use  of  the  network(s)  established  and/or  strengthened  through  my  participation  in 
training(s), seminar(s) or conference(s). 

Strongly disagree 
Disagree 
Neither agree or disagree 
Agree 
Strongly agree 
Do not know/cannot assess 
Total 

Overall Status 

New 
Distributed 
Partially Complete 
Complete 
Rejected 
Total 

Respondents 
0 
15 
59 
263 
138 
13 
488 

Respondents 
0 
786 
0 
574 
2 
1362 

Percent 
0,0% 
3,1% 
12,1% 
53,9% 
28,3% 
2,7% 
100,0% 

Percent 
0,0% 
57,7% 
0,0% 
42,1% 
0,1% 
100,0% 

December 2014 18 

 
 
 
  
  
 
 
 
 
 
 
 
 
 
 
J 

OB-02-15-373-EN-N