COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the evaluation of the Union's finances based on the results achieved
Date: 2012-11-21
Institutions: European Commission
Subjects: EU financing, cost-effectiveness analysis, evaluation method

EUROPEAN 
COMMISSION 

Brussels, 21.11.2012  
SWD(2012) 383 final 

COMMISSION STAFF WORKING DOCUMENT 
Accompanying the document 

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND 
THE COUNCIL 

on the evaluation of the Union's finances based on the results achieved 

{COM(2012) 675 final} 

EN 

   EN 

 
 
 
 
  
 
 
SECTION 1: OVERVIEW OF EVALUATIONS COMPLETED IN 2011.......................................3 
SECTION 2: RETROSPECTIVE EVALUATIONS ..................................................................25 
01 – Economic and Financial Affairs ......................................................................26 
02 – Enterprise .........................................................................................................30 
04 – Employment and social affairs.........................................................................38 
05 – Agriculture and Rural Development ................................................................50 
06 – Mobility and Transport ....................................................................................61 
07 – Environment.....................................................................................................68 
08 – Research...........................................................................................................70 
09 – Information Society and Media .......................................................................87 
11 – Maritime Affairs and Fisheries ........................................................................90 
12 – Internal Market.................................................................................................90 
13 – Regional Policy................................................................................................90 
14 – Taxation and Customs Union...........................................................................90 
15 – Education and Culture......................................................................................90 
16 – Communication................................................................................................90 
17 – Health and Consumer Protection .....................................................................90 
18 – Home affairs.....................................................................................................90 
19 - External Relations.............................................................................................90 
20 – Trade ................................................................................................................90 
21 – Development and Relations with African, Caribbean and Pacific (ACP) States
..................................................................................................................................90 
22 – Enlargement .....................................................................................................90 
23 – Humanitarian Aid.............................................................................................90 
26 – Commission's administration ...........................................................................90 
29 – Statistics ...........................................................................................................90 
32 – Energy ..............................................................................................................90 
33 – Justice...............................................................................................................90 
SECTION 3: PROSPECTIVE EVALUATIONS ......................................................................90 
04 – Employment and social affairs.........................................................................90 
08 – Research...........................................................................................................90 
11 – Maritime Affairs and Fisheries ........................................................................90 
16 – Communication................................................................................................90 
20 – Trade ................................................................................................................90 
22 – Enlargement .....................................................................................................90 
32 – Energy ..............................................................................................................90 

SECTION 4: REFERENCES TO REPORTING ON PERFORMANCE IN THE ACTIVITY 
STATEMENTS (AS) TO THE DRAFT BUDGET 2013 .........................................................90 

2 

 
 
 
SECTION 1: OVERVIEW OF EVALUATIONS COMPLETED IN 2011 

This overview lists the titles of the available evaluations in order of Activity Based Budgeting 
Activity within each Policy Area.  

The different types of studies are indicated as follows: retrospective evaluations are in plain 
script, prospective evaluations are indicated by italic script and evaluation-related studies are 
flagged by an asterisk. 

Of  all  evaluations,  62%  concerned  expenditure  programmes;  14%  covered  regulatory 
activities,  linked  to  regulations,  directives,  regulatory  policy  communications  and  'soft  law'; 
13%  concerned  communication,  information  and  coordination  activities;  3%  internal 
administrative processes of the EU Institutions with the remaining 8% in other categories. 

3 

 
 
 
 
ABB-ACTIVITY  

N° 

HEADING 

01 02 

Economic and 
monetary union 

01 03 

International 
economic and 
financial affairs 

01 04 

Financial operations 
and instruments  

ABB-ACTIVITY  

N° 

HEADING 

02 02 

Competitiveness, 
industrial policy, 
innovation and 
entrepreneurship  

01 – Economic and Financial Affairs 

TITLE OF EVALUATION 

•  Evaluation of the Joint Harmonised EU Programme of Business and 

Consumer Surveys  

•  Ex-post evaluation of the Euratom Loan Facility 

02 – Enterprise  

TITLE OF EVALUATION 

•  Final Evaluation of the Competitiveness and Innovation Framework 

Programme 

•  Final evaluation of the Entrepreneurship and Innovation Programme 

•  Evaluation  of  the  Executive  Agency  for  Competitiveness  and 

Innovation (EACI) 

•  Final Evaluation of the Lead Market Initiative 

• 

Interim evaluation of Erasmus for Young Entrepreneurs 

02 03 

Internal market for 
goods and sectoral 
policies 

•  Evaluation  of  the  Directive  2009/142/EC  on  appliances  burning 

gaseous fuels 

•  Ex-Post  Evaluation  on  Enhancing  the  Implementation  of  the 

Internal Market Legislation Relating to Motor Vehicles 

02 04 

Cooperation — Space 
and security 

02 05 

European satellite 
navigation 
programmes (EGNOS 
and Galileo) 

4

 
 
 
 
 
 
 
 
 
03 – Competition 

TITLE OF EVALUATION 

ABB ACTIVITY  

N° 

HEADING 

03 03 

Cartels, anti-trust and 
liberalisation 

03 
AWBL-
03 

03 
AWBL-
04 

Control of State aid 

Merger control 

ABB ACTIVITY  

N° 

HEADING 

04 – Employment and social affairs 

TITLE OF EVALUATION 

•  Evaluation of ESF support to gender equality 

04 02 

European Social Fund 

•  Evaluation  of  ESF  Support  for  Enhancing  Access  to  the  Labour 
Market and the Social Inclusion of Migrants and Ethnic Minorities 

•  Evaluation of the capacity of the ESF delivery systems to attract and 

support OP target groups 

04 03 

Working in Europe - 
Social dialogue and 
mobility 

04 04 

Employment, social 
solidarity and gender 
equality 

•  Assessing  the  effectiveness  and  the  impact  of  the  Social  OMC  in 

preparation of the new cycle 

•  European  Year  2010  for  combating  poverty  and  social  exclusion- 

Ex-post evaluation 

•  Support for carrying out ex-ante evaluation for the new PROGRESS 

Programme post-2013* 

• 

Impact  of  enlargement  and 
arrangements on labour mobility within the EU* 

the 

functioning  of 

transitional 

5

 
 
 
 
 
 
 
 
04 05 

04 06 

European 
Globalisation 
Adjustment Fund 
(EGF) 

Instrument for 
Preaccession (IPA) - 
Human resources 
development 

ABB ACTIVITY  

N° 

HEADING 

05 – Agriculture and Rural Development  

TITLE OF EVALUATION 

05 02 

Interventions in 
agricultural markets 

05 03 

Direct aids 

05 04 

Rural development 

05 05 

05 06 

Pre-accession 
measures in the field 
of agriculture and 
rural development 

International aspects 
of "Agriculture and 
rural development" 
policy area 

05 07 

Audit of agricultural 
expenditure 

•  Evaluation of income effects of direct support (also covered by 05 03) 

•  Evaluation of CAP measures applied to the sugar sector (also covered by 05 

03) 

•  Evaluation of CAP measures applied to the dairy sector (also covered 

by 05 03) 

•  Evaluation of  CAP  measures applied to the sheep  and goat sector  (also 

covered by 05 03) 

•  Evaluation of promotion and information actions for products  

•  Evaluation of the exceptional  market support  measures in the poultry 

and egg sector  

6

 
 
 
 
 
 
 
 
 
 
05 08 

Policy strategy and 
coordination of 
"Agriculture and 
Rural Development" 
policy area 

ABB ACTIVITY  

N° 

HEADING 

06 – Mobility and Transport  

TITLE OF EVALUATION 

06 02 

Inland, air and 
maritime transport 

•  Evaluation of the Marco Polo Programme 

•  Evaluation  of  Regulation  881/2004  establishing  the  European 

Railway Agency 

•  Ex post evaluation on EU Passenger Ship Safety Legislation 

•  Mid-term evaluation of TEN-T programme 

07 – Environment  

EVALUATION PROJECTS 

•  Mid-term  Evaluation  of 

the  Sustainable  Consumption  and 

Production and Sustainable Industrial Policy Action Plan 

06 03 

06 06 

Trans-European 
networks 

Research related to 
transport 

ABB ACTIVITY  

N° 

HEADING 

07 02 

07 03 

07 12 

Global 
environmental and 
Climate Action 
affairs 

Development and 
implementation of 
Union environmental 
policy and legislation 

Implementation of 
Union policy and 
legislation on climate 
action 

7

 
 
 
 
 
 
 
 
 
 
07 13 

Climate 
mainstreaming and 
innovation 

ABB ACTIVITY 

N° 

HEADING 

08 – Research  

TITLE OF EVALUATION 

08 02 

08 03 

08 04 

Cooperation — 
Health 

• 

Interim  Evaluation  of  the  Innovative  Medicines  Initiative  Joint 
Undertaking 

Cooperation — 
Food, agriculture and 
fisheries, and 
biotechnology 

Cooperation — 
Nanosciences, 
nanotechnologies, 
materials and new 
production 
technologies 

•  Meta-analysis of "Bio-Technology", "Agriculture", "Food", "Marine 

and Maritime", and Horizontal themes 

• 

Interim  Assessment  of  the  Research  Public  Private  Partnerships  in 
the  European  Economic  Recovery  Plan:  Energy-efficient  Buildings, 
Factories of the Future, and European Green Cars Initiative  

08 05 

Cooperation — 
Energy 

•  First  Interim  Evaluation  of  the  Fuel  Cells  and  Hydrogen  Joint 

Undertaking (FCH JU) 

08 06 

08 07 

08 08 

08 09 

Cooperation — 
Environment 
(including climate 
change) 

•  Stock-Taking  of  Results  and  Impacts  of  EU-Funded  Environmental 

Research* 

•  State  of  the  Art  and  Forward-Looking  Analysis  of  Environmental 

Research and Innovation* 

Cooperation — 
Transport (including 
aeronautics) 

• 

Interim  Evaluation  of  EU  FP7  Transport  Research,  notably  within 
Theme  7  of  the  Cooperation  Programme  "Transport  (including 
Aeronautics)" 

Cooperation — 
Socioeconomic 
sciences and the 
humanities 

Cooperation — Risk-
sharing finance 
facility (RSFF) 

08 10 

Ideas 

08 12 

Capacities — 
Research 
infrastructures 

8

 
 
 
 
 
 
 
 
 
08 13 

08 14 

08 15 

08 16 

08 17 

08 18 

08 19 

08 20 

08 21 

08 22 

08 23 

Capacities — 
Research for the 
benefit of small and 
medium-sized 
enterprises (SMEs) 

Capacities — 
Regions of 
knowledge 

Capacities — 
Research potential 

Capacities — 
Science in society 

Capacities — 
International 
cooperation activities  

Capacities — Risk-
sharing finance 
facility (RSFF) 

Capacities — 
Support for coherent 
development of 
research policies 

Euratom — Fusion 
energy  

Euratom — Nuclear 
fission and radiation 
protection 

Completion of 
previous framework 
programmes and 
other activities 

Research programme 
of the research fund 
for coal and steel 

• 

Impact Assessment of the Regions of Knowledge Programme 

• 

Impact Assessment of the Research Potential Programme 

•  Fusion Energy – State of Development and Future Role* 

•  Ex Post Evaluation of NMP FP6 - Project Level 

08 
AWBL 
02  

Policy strategy and 
coordination 

•  Assessing the Effectiveness of Simplification Measures under FP7* 

•  Long-Term Impact of the Framework Programmes* 

•  Review  of  S&T  Cooperation  between  the  European  Union  and  the 

Republic of Argentina (2006-2010)* 

•  Review  of  S&T  Cooperation  between  the  European  Union  and  the 

Republic of Chile (2007-2011)* 

9

 
 
 
 
 
 
 
 
 
 
09 – Information Society and Media 

TITLE OF EVALUATION 

•  Safer Internet Evaluation 

•  CIP ICT PSP Final (Second Interim) Evaluation  

• 

eContent Plus Final Evaluation  

10 – Direct Research 

TITLE OF EVALUATION 

ABB ACTIVITY  

N° 

HEADING 

09 02 

09 03 

09 04 

09 05 

Regulatory 
framework for the 
Digital Agenda 

Information and 
communication 
technologies take-up 

Cooperation - 
Information and 
communication 
technologies (ICTs) 

Capacities — 
Research 
infrastructures 

ABB ACTIVITY  

N° 

HEADING 

10 02 

10 03 

10 04 

Directly financed 
research operational 
appropriations — 
Seventh framework 
programme (2007 to 
2013) — EU 

Directly financed 
research operational 
appropriations — 
Seventh framework 
programme (2007 to 
2011 and 2012 to 
2013)  — Euratom 

Completion of 
previous framework 
programmes and 
other activities 

10

 
 
 
 
 
 
 
 
10 05 

Historical liabilities 
resulting from 
nuclear activities 
carried out by the 
Joint Research 
Centre pursuant to 
the Euratom Treaty 

ABB ACTIVITY  

N° 

HEADING 

11 02 

Fisheries markets 

11 03 

International 
fisheries and law of 
the sea 

11 – Maritime Affairs and Fisheries  

TITLE OF EVALUATION 

•  Mission d’évaluation à mi-parcours du Plan Régional de Surveillance 

des Pêches dans le Sud-Ouest de l’Océan Indien 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Madagascar (2007-2012) 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Cape Verde (2007-2011) 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Guinea Bissau (2007-2011) 

•  An  ex-ante  evaluation  for  the  conclusion  of  a  new  Fisheries 
Partnership  Agreement  (FPA)  and  Protocol  between  the  European 
Union and Mauritius 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Mauritania,  and  an  ex-ante  evaluation  and  analysis  of  impacts  of  a 
potential future Protocol on sustainability 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Gabon,  and  an  ex-ante  evaluation  and  analysis  of  impacts  of  a 
potential future Protocol on sustainability 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Mozambique (2007-2011) 

•  An  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and 
Greenland 

11

 
 
 
 
 
11 04 

Governance of the 
Common Fisheries 
Policy 

11 06 

European Fisheries 
Fund (EFF) 

11 07 

11 08 

Conservation, 
management and 
exploitation of living 
aquatic resources 

Control and 
enforcement of the 
Common Fisheries 
Policy 

11 09 

Maritime policy 

ABB ACTIVITY  

N° 

HEADING 

12 02 

Single Market 
Policy 

12 04 

Free movement of 
capital, company 
law and corporate 
governance 

ABB ACTIVITY  

N° 

HEADING 

13 03 

European Regional 
Development Fund 
and other regional 
operations 

• 

Interim evaluation of DG MARE external communication activities 

• 

• 

Interim  evaluation  (study)  of  the  European  Fisheries  Fund  (2007-
2013) 

Interim  evaluation  of  the  European  Fisheries  Fund  (2007-2013)
(Synthesis of the 26 national evaluation reports) 

12 – Internal Market  

TITLE OF EVALUATION 

•  Evaluation of the Professional Qualifications Directive 

• 

Impact and Effectiveness of EU Public Procurement Legislation  

13 – Regional Policy  

TITLE OF EVALUATION 

•  Enterprise  support:  support  to  SMEs  and  large  enterprises  in  Italy, 
including a comparison of grants and other financial instruments* 

•  Data-Linking and Impact Evaluation in Northern Ireland* 

•  Study  on  the  contribution  of  local  development  in  delivering 
interventions  co-financed  by  the  European  Regional  Development 
Fund (ERDF) in the periods 2000-06 and 2007-13* 

12

 
 
 
 
 
 
 
 
•  Policy analysis on the performance of cohesion policy 2007 – 2013
Task 1: Policy Paper on Renewable Energy and Energy Efficiency of 
Residential 
Housing
Task 2: Country reports on achievements of cohesion policy 

•  Counterfactual 

policy 
evaluation 
Work package 2: Examples from Support to Innovation and Research 

cohesion 

impact 

of 

•  Ex  post  evaluation  of  cohesion  policy  interventions  2000-2006 
financed 
Fund 
the 
Work Package B: Cost-benefit analysis of selected transport projects 

Cohesion 

by 

•  Ex  post  evaluation  of  cohesion  policy  interventions  2000-2006 
ISPA) 

financed  by 
Work Package C – Cost benefit analysis of environment projects 

the  Cohesion  Fund 

(including 

former 

•  Ex  post  evaluation  of  cohesion  policy  interventions  2000-2006 
financed  by  the  Cohesion  Fund  (including  former  ISPA)  Work 
Package D – Management and Implementation  

•  Study  on  the  relevance  and  effectiveness  of  ERDF  and  Cohesion 
Fund support to regions with specific geographical features – islands, 
mountainous and sparsely populated areas* 

14 – Taxation and Customs Union 

TITLE OF EVALUATION 

13 04 

Cohesion Fund 

13 05 

Pre-accession 
operations related to 
the structural 
policies 

13 06 

Solidarity Fund 

ABB ACTIVITY  

N° 

HEADING 

14 02 

14 03 

Policy strategy and 
coordination for the 
Directorate-General 
for Taxation and 
Customs Union 

International aspects 
of taxation and 
customs 

14 04 

Customs policy 

•  Mid-term Evaluation of Customs 2013 

13

 
 
 
 
 
 
 
 
 
14 05 

Taxation policy 

•  Mid-term Evaluation of Fiscalis 2013 

•  A retrospective evaluation of the elements of the VAT system 

15 – Education and Culture   

TITLE OF EVALUATION 

•  External  Evaluation  of  the  European  Institute  of  Innovation  and 

Technology 

• 

• 

Interim  Evaluation  of  External  Cooperation  Agreements  in  Higher 
Education, Training and Youth with the US and Canada 

Interim Evaluation of the Lifelong Learning Programme (LLP) 2007-
2013 

•  Ex-post evaluation of 2010 European Capitals of Culture and Interim 
evaluation  of  selection  and  monitoring  procedures  of  ECOC  2010-
2016 

•  Evaluation of Preparatory Actions and special events in the field of 

sport 

•  Youth in Action – Interim evaluation 

ABB ACTIVITY  

N° 

HEADING 

15 02 

Lifelong learning, 
including 
multilingualism 

15 04 

15 05 

15 07 

Developing cultural 
and audiovisual 
cooperation in 
Europe 

Encouraging and 
promoting 
cooperation in the 
field of youth and 
sports 

People — 
Programme for the 
mobility of 
researchers 

14

 
 
 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

16 02 

Communication and 
the media 

16 03 

"Going Local" 
communication 

16 04 

16 05 

Analysis and 
communication tools 

Fostering European 
citizenship 

ABB ACTIVITY  

N° 

HEADING 

17 02 

Consumer policy 

16 – Communication 

TITLE OF EVALUATION 

• 

Interim evaluation of Euranet – The European Radio Network 

•  Evaluation  of  the  Management  Partnership’s  Actions  within  the 

scope of the 2008-2010 Communication Plans in Portugal 

•  Ex-post  Evaluation  of  the  Management  Partnership  concluded 
between  the  Republic  of  Austria  and  the  European  Commission  for 
the period of 2008-2010 

•  Ex-Ante Evaluation of the European Year of Citizens 2013 

17 – Health and Consumer Protection   

TITLE OF EVALUATION 

•  Evaluation of the European Consumer Centres' Network (ECC-Net) 

•  Evaluation  of  Consumer  Education,  Information  and  Capacity 

building actions 

•  1st  interim  evaluation  of  the  Public  Health  Executive  Agency 

(Executive Agency) (EAHC since July 2008) 

•  Ex-post  evaluation  of  the  Public  Health  Programme  (PHP)  2003-

17 03 

Public health 

2008 

•  Mid-Term Evaluation of the EU Health Strategy 2008-2013 

•  Mid-Term Evaluation of the Health Programme (2008-2013) 

15

 
 
 
 
 
17 04 

Food and feed 
safety, animal 
health, animal 
welfare and plant 
health 

•  Evaluation of the EU policy on animal welfare (EUPAW) 

•  Evaluation of the EU-Reference Laboratories in the field of food and 

feed safety and animal health and live animals 

•  Evaluation of the Community Plant Variety Right Acquis 

•  Evaluation of the EU legislative framework in the field of cultivation 
of  GMOs  under  Directive  2001/18/EC  and  Regulation  (EC)  No 
1829/2003, and the placing on the market of GMOs as or in products 
under Directive 2001/18/EC 

ABB ACTIVITY  

N° 

HEADING 

18 – Home affairs 

TITLE OF EVALUATION 

18 02 

Solidarity — 
External borders, 
return, visa policy 
and free movement 
of people 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council on the application of Directive 2003/109/EC concerning the 
status of third-country nationals who are long-term residents* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council  on  the  application  of  Directive  2005/71/EC  on  a  specific 
procedure  for  admitting  third  country  nationals  for  the  purposes  of 
scientific research* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council  on  the  Application  of  Directive  2004/114/EC  on  the 
conditions of admission of third country nationals for the purposes of 
studies,  pupil  exchange,  unremunerated 
training  or  voluntary 
service* 

•  Report  from  the  Commission  to  the  European  Parliament,  the 
Council,  The  European  Economic  and  Social  Committee  and  the 
Committee  of  the  Regions  on  the  results  achieved  and  on  the 
quantitative  and  qualitative  aspects  of  implementation  of  the 
European Return Fund for the period 2008-2009 (report submitted in 
accordance  with  Article  50(3)(b)  of  Council  Decision  575/2007/EV 
of 23 May 2007)* 

•  Report  from  the  Commission  to  the  European  Parliament,  the 
Council,  The  European  Economic  and  Social  Committee  and  the 
Committee  of  the  Regions  on  the  results  achieved  and  on  the 
quantitative and qualitative aspects of implementation of the External 
Borders  Fund  for  the  period  2007-2009  (report  submitted  in 
accordance  with  Article  52(3)(b)  of  Council  Decision  574/2007/EV 
of 23 May 2007)* 

16

 
 
 
 
18 03 

18 05 

18 08 

Migration flows - 
Common 
immigration and 
asylum policies 

Security and 
safeguarding 
liberties 

Policy strategy and 
coordination 

ABB ACTIVITY  

N° 

HEADING 

19 02 

19 03 

19 04 

19 05 

19 06 

19 08 

Cooperation with 
third countries in the 
area of migration 
and asylum 

Common foreign 
and security policy 
(CFSP) 

European Instrument 
for Democracy and 
Human Rights 
(EIDHR) 

Relations and 
cooperation with 
industrialised third 
countries 

Crisis response and 
global threats to 
security 

European 
Neighbourhood 
Policy and relations 
with Russia 

19 09 

Relations with Latin 
America 

•  Report  from  the  Commission  to  the  European  Parliament,  the 
Council,  the  European  Economic  and  Social  Committee  and  the 
Committee of the Regions on the results achieved and on qualitative 
and quantitative aspects of implementation of the European Refugee 
Fund for the period 2005-2007 (COM(2011)2)* 

•  External Evaluation of the European Migration Network 

•  Mid-term  evaluation  of  the  Framework  Programme  “Security  and 

Safeguarding Liberties (2007-2013) (SSL)”  

19 – External Relations 

TITLE OF EVALUATION 

•  Evaluation of the crisis response and preparedness components of the 

European Union’s Instrument for Stability (IfS) 

•  Evaluation  des  opérations  d'aide  budgétaire  de  la  Commission 

Européenne à la Tunisie entre 1996 et 2008 

•  Evaluation de la coopération de l'UE avec la Tunisie de 1995 à 2008 

17

 
 
 
 
 
 
 
 
 
 
19 10 

Relations with Asia, 
Central Asia and 
Middle East (Iraq, 
Iran, Yemen) 

•  Evaluation  of  the  European  Commission's  co-operation  with  the 

Philippines 

•  Evaluation of the European Commission's co-operation with overseas 

countries and territories 

•  Thematic  evaluation  of  European  Commission  support  to  conflict 

prevention and peace building (CPPB) 

19 11 

Policy strategy and 
coordination for 
‘External relations’ 
policy area 

•  Thematic  evaluation  of  the  EC  Support  in  Employment  and  Social 

Inclusion sectors in partner countries 1999-2009 

•  Thematic evaluation of the European Commission support to Justice 

and Security System Reform (JSSR) 

•  Study  on  legal  instrument  and  lessons  learned  from  the  evaluations 

managed by the joint evaluation unit 

ABB ACTIVITY  

N° 

HEADING 

20 02 

Trade policy 

20 – Trade  

TITLE OF EVALUATION 

•  Evaluation of the Commission's Market Access Database (MADB) 

•  Trade  SIA  relating  to  the  negotiation  of  a  comprehensive  economic 

and trade agreement between the EU and Canada 

21 – Development and Relations with African, Caribbean and Pacific (ACP) States  

ABB ACTIVITY  

N° 

HEADING 

21 02 

Food security 

21 03 

Non-State actors in 
development 

TITLE OF EVALUATION 

18

 
 
 
 
 
 
 
21 04 

Environment and 
sustainable 
management of 
natural resources, 
including energy 

21 05 

Human and social 
development 

21 06 

21 07 

21 08 

Geographical 
cooperation with 
African, Caribbean 
and Pacific (ACP) 
States 

Development 
cooperation actions 
and ad hoc 
programmes 

Policy strategy and 
coordination for 
"Development and 
relations with ACP 
States" policy area 

•  Evaluation  of  the  European  Commission's  co-operation  with  the 

Dominican Republic  

•  Evaluation  of  the  European  Commission's  co-operation  with  the 

Republic of Malawi  

•  Evaluation  conjointe  des  opérations  d'aide  budgétaire  au  Mali  entre 

2003 et 2009 

ABB ACTIVITY  

N° 

HEADING 

22 – Enlargement  

TITLE OF EVALUATION 

• 

Interim  Evaluation  of  Cross-Border  Programmes  between 
Candidate/Potential  Candidate  Countries  (Intra-Western  Balkan 
Borders)  under  the  Cross-Border  Cooperation  Component  of  IPA; 
Report II – Performance of assistance 

22 02 

Enlargement process 
and strategy 

•  Retrospective  evaluation  of  EU  pre-accession  assistance  to  CBC 

programmes in Bulgaria and Romania 

•  Thematic evaluation of EU's support to civil society in the Western 

Balkans and Turkey 

•  Thematic 

Evaluation 

of 

IPA-Funded 

Information 

and 

Communication Activities 

19

 
 
 
 
 
 
 
 
 
•  Thematic  evaluation  of  EU  pre-accession  multi-beneficiary 
assistance  to  Western  Balkans  and  Turkey  in  the  fields  of 
environment and disaster risk reduction 

•  Review of Twinning in Turkey* 

•  Evaluation of Twinning versus Technical Assistance 

•  Strategic/Interim Evaluation of EU IPA Pre-Accession Assistance to 

Albania 

• 

(Ex-ante)  Evaluation  to  support  the  preparation  of  pre-accession 
financial instruments beyond 2013  

•  Strategic/Interim Evaluation of EU IPA Pre-Accession Assistance to 

Kosovo* (under UNSCR 1244/1999) 

•  Strategic/Interim  evaluation  of  IPA  pre-accession  assistance  to 

Serbia 

•  Evaluation  on  stakeholder  participation 

in  programming  and 

implementation of pre-accession assistance to Turkey 

22 
AWBL-
03  

Enlargement pre-
accession negotiations 

ABB ACTIVITY  

N° 

HEADING 

23 02 

Humanitarian aid 
including aid to 
uprooted people, 
food aid and disaster 
preparedness 

23 – Humanitarian Aid 

TITLE OF EVALUATION 

• 

• 

Evaluation  and  Strategy  Orientation  of  DG  ECHO-Funded  Health 
Sector  Activities  in  Burmese  Refugee  Camps  in  Thailand  (2004-
2009) 

Evaluation  of  DG  ECHO's  Actions  in  the  Water  Sanitation/Public 
Health Sector in Zimbabwe 

• 

Evaluation of the DG ECHO action in Uganda 

•  Real-time  evaluation  of  humanitarian  action  supported  by  DG 

ECHO in Haiti 2009-2011 

23 03 

Civil Protection 
Financial Instrument 

•  Commission evaluation of the Civil Protection Mechanism and the 

CP Financial Instrument 2007-2009  

20

 
 
 
 
 
26 – Commission's administration 

TITLE OF EVALUATION 

• 

• 

• 

Evaluation  of  internal  communication  and  staff  engagement 
strategy of the European Commission  

Ex-ante  evaluation  of 
the  contract  "Processing  of  public 
procurement  notices  for  publication  in  the  Supplement  to  the 
Official Journal of the European Union, hereafter OJ S"* 

Ex-ante  evaluation  of  the  contract  "Performance  of  publication 
services"* 

ABB ACTIVITY  

N° 

HEADING 

26 01 

Administrative 
expenditure of 
‘Commission’s 
administration’ 
policy area 

26 02 

Multimedia 
production 

26 03 

26 
AWBL -
27 

Services to public 
administrations, 
businesses and 
citizens 

Policy strategy and 
coordination for 
'Personnel and 
Administration' 
Directorate-General 

ABB ACTIVITY  

N° 

HEADING 

29 – Statistics  

TITLE OF EVALUATION 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council on the quality of fiscal data reported by Member States in 
2010* 

29 02 

Production of 
statistical 
information  

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council on the implementation of Regulation (EC) No 450/2003 of 
the European Parliament and of the Council concerning the labour 
cost index (LCI)* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council on the implementation of Decision (EC) No 1297/2008/EC 
of  the  European  Parliament  and  of  the  Council  of  16  December 
2008 on a Programme for the Modernisation of European Enterprise 

21

 
 
 
 
 
 
 
and Trade Statistics (MEETS)* 

•  Rolling review of the Education Statistics* 

•  Rolling review of the Information Society (IS) Statistics* 

•  Rolling Review on Inland Waterways Transport Statistics* 

•  Rolling Review - Labour Force Survey Statistics* 

•  Rolling Review - Maritime Transport Statistics* 

•  Rolling Review - EU National Accounts* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council  on  the  implementation  of  Decision  No  1608/2003/EC  of 
the  European  Parliament  and  of  the  Council  on  science  and 
technology statistics* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council  concerning  short-term  statistics  as  required  by  Council 
Regulation (EC) No 1165/1998 of 19 May 1998* 

•  Report  from  the  Commission  to  the  European  Parliament  and  the 
Council  on  the  implementation  of  Council  Regulation  (EC, 
Euratom) No 58/97 and of the Regulation (EC) No 295/2008 of the 
European  Parliament  and  of  the  Council  concerning  structural 
business  statistics  (recast)  which  repealed  and  replaced  Regulation 
No 58/97* 

31 – Language Services 

TITLE OF EVALUATION 

22

ABB ACTIVITY  

N° 

HEADING 

31 
AWBL -
02 

Interpreting and 
linked activities 

31 
AWBL -
03 

Logistic assistance 
for Commission 
events (LACE) 

31 
AWBL -
04 

Conference 
organisation and 
advice 

 
 
 
 
 
 
 
31 

AW
BL- 
11 

Translations 

ABB ACTIVITY  

N° 

HEADING 

32 03 

Trans-European 
networks 

32 – Energy 

TITLE OF EVALUATION 

•  Mid-term  evaluation  on  the  European  Energy  Programme  for 

Recovery (EERP) 

•  Mid–term  evaluation  on  horizontal  issues  concerning  energy 

savings in the EU 

32 04 

Conventional and 
renewable energies 

•  Report  of  the  Mid-term  Evaluation  on  the  Background  study  for 

energy supply side efficiency framework 

•  Report on the final evaluation of the Intelligent Energy Europe II 

• 

• 

Report  on  Ex-ante  Evaluation/impact  assessment  of  a  possible 
successor to the intelligent Energy II Programme 

Ex-Ante  Evaluation  preparing  a  Euratom  Council  Regulation  on 
Union  support 
the  nuclear  decommissioning  assistance 
programmes in Bulgaria, Lithuania and Slovakia 

for 

32 05 

Nuclear energy 

32 06 

Research related to 
energy 

23

 
 
 
 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

33 02 

Fundamental rights 
and citizenship 

33 03 

Justice in criminal 
and civil matters 

33 – Justice 

TITLE OF EVALUATION 

•  Report on the interim evaluation of the "DAPHNE III Programme 

2007-2013"  

•  Report  on  the  interim  evaluation  of  the  "Fundamental  Rights  and 

Citizenship Programme 2007-2013"  

• 

Evaluation of the impact of the EU instruments affecting children’s 
rights  with  a  view  to  assessing  the  level  of  protection  and 
promotion of children’s rights in the EU 

•  Communication from the COM to the EP and the Council Consular 
Protection for EU citizens in third countries: state of play and way 
forward* 

•  Report  from  the  COM  to  the  EP  and  the  Council  on  the 
implementation  since  2007  of  the  Council  Framework  decision  of 
13  June  2002  on  the  European  arrest  warrant  and  the  surrender 
procedures between Member States (COM(2011) 175 final)* 

•  Report on the interim evaluation of the specific Programme "CIVIL 

Justice"; (JCIV) 2007-2013  

• 

• 

Interim evaluation report on the Criminal Justice Programme 

Study  for  an  external,  independent  assessment  of  the  European 
Action on Drugs* 

33 04 

Drugs prevention and 
information 

•  Report  from  the  COM  on  the  functioning  of  Council  Decision 
2005/387/JHA  on  the  information  exchange,  risk  assessment  and 
control of new psychoactive substances (COM(2011) 430 final)* 

•  Report on the interim evaluation of the specific Programme "Drug 

prevention and information"; (DPIP) 2007-2013  

33 05 

Policy strategy and 
coordination 

33 06 

Equality 

• 

The Mid-term Evaluation of PROGRESS - Final Report 

24

 
 
 
 
 
 
SECTION 2: RETROSPECTIVE EVALUATIONS 

This  section  provides  fact  sheets  for  118  retrospective  evaluations  (i.e.  interim  and  ex  post 
evaluations)  and  39  evaluation-related  studies 
focusing  on  monitoring  or 
implementation), with information on their findings and conclusions. 

(e.g. 

25

 
 
01 – Economic and Financial Affairs 

Evaluation of the Joint Harmonised EU Programme of Business and Consumer Surveys 

ABB activities: 

01 02 Economic and monetary union 

Timing:  

 27/02/2012  

Background, scope and focus  

ECFIN’s  business  and  consumer  surveys  summarise  the  attitudes  and  judgement  of  a  large 
number  of  business  managers  and  consumers,  with  harmonised  and  comparable  results  being 
published on a monthly basis 

The emphasis of this evaluation is on improving the functioning of the BCS with a view to better 
meeting  users'  needs  and  to  raising  the  efficiency  of  DG  ECFIN's  management  of  the 
Programme. 

The evaluation concerns the implementation and use of the BCS in the Member States. The main 
evaluation issues to be covered in the exercise are the effectiveness, efficiency and utility of the 
instrument.  In  particular,  the  exercise  should  include  an  analysis  of  user  needs  and  their 
appreciation of the surveys. 

Summary of findings and conclusions  

The main findings and conclusions of this evaluation are as follows: 

Meeting users’ needs and expectations 

Overall, the BCS Programme is well aligned with users’ needs and expectations. However, there 
are some issues requiring attention. These are as follows: 

Relevance: the programme is not considered relevant by users who are specifically interested in 
data  for  Ireland  (due  to  lack  of  data  for  the  period  2008  to  2011).  The  relevance  of  some 
components  of  the  programme  is  greatly  reduced  by  their  limited  sectoral  coverage  (Flash 
Indicator  is  only  available  for  the  consumer  sector)  or  non-availability  of  country  level  data 
(Flash Indicator and the Financial Services Survey). 

Timeliness: a number of users expressed frustration that the BCS data are released later than the 
Flash PMI and the national surveys (in some cases).  

Comparability:  the  two  main  concerns  expressed  by  the  users  are  (a)  a  relatively  short  time 
series for the Services and the Financial Services survey; and, (b) non-homogeneity of approach 
across  partner  institutes  e.g.  the  reference  period  for  the  questions  being  different  across 
countries (six months instead of three months). 

Methodology: some methodological approaches were criticised by the users notably, the use of 

26

 
 
 
 
 
 
 
 
 
DAINTES for seasonal adjustment; the practice of assigning equal weights to all questions in the 
construction of indices; and the phrasing of survey questions which in their present form, seek to 
elicit information regarding intended behaviour as opposed to the actual behaviour of economic 
agents.  

Interpretability:  users  have  expressed  demands  for  more  guidance  on  how  to  interpret  survey 
questions as well as survey answers.  

Transparency:  BCS  surveys  are  quite  transparent  in  terms  of  the  information  available  on  DG 
ECFIN’s website notably, comprehensive metadata by sector and country; a methodological user 
guide; press releases; list of partner institutes; and, contact details. However, users are often not 
aware  of  the  existence  of  this  information  and,  in  some  cases,  have  requested  for  further 
information.  

Accessibility: a number of external users and even some internal users consider that the visibility 
and  accessibility  of  the  survey  products  could  be  greatly  improved.  Some  users  expressed  a 
preference for data to be made available in alternative formats (.CSV or access database). 

Frequency:  some  users  would  like  the  European  Business  Cycle  Indicators  (EBCI)  and 
Investment Survey to be produced on a monthly basis, and the question on capacity utilisation to 
be asked on a monthly basis 

New needs arising from the financial and economic crisis 

The financial and economic crisis has reinforced the importance of early warning signals. Due to 
greater economic uncertainty and instability, there is a greater urgency to access data in a timely 
manner. For many users, the crisis has shifted emphasis from EU or Euro-area level to a country 
level analysis.  

Assessment of EU added value of the programme 

The  main  added  value  of  the  programme  is  that  it  generates  harmonized  data  which  enables 
cross-country  comparisons  and  the  construction  of  EU/  Euro-area  indicators.  Furthermore,  the 
programme promotes capacity building and knowledge sharing among partner institutes.  

Comparison with alternative survey products notably, the PMI 

The  relative  strengths  of  BCS  products  as  compared  to  the  PMI  are:  its  wider  sectoral  and 
geographical  coverage;  availability  of  data  for  sub-sectors;  longer  time  series;  larger  sample 
sizes; and, methodological transparency. 

The scope for cost savings  

The  evaluation  did not  find  any  compelling  reasons  for  discontinuing  survey products  that  are 
less used. On the other hand, there is a case to be made for improving the usability of the surveys 
that  are  relatively  less  used  by  improving  their  visibility,  expanding  their  coverage  and/or 
improving the survey questionnaires.  

Availability of the report on Europa 

http://ec.europa.eu/economy_finance/evaluation/completed/index_en.htm  

27

 
 
 
 
Ex-post evaluation of the Euratom Loan Facility 

ABB activities: 

01 04 Financial operations and instruments  

Timing:  

 13/06/2011  

Background, scope and focus  

The Facility was  created by Council Decision 77/270/Euratom of 29 March 1977 and Council 
Decision  94/179/Euratom  of  21  March  1994  extended  the  Facility  to  some  non-member 
countries. 

Considering the current policy context of the drive to develop low carbon technologies, the fight 
against climate change and the EU2020 targets, it is considered opportune to evaluate whether 
the scope, objectives  and the limits fixed by the current Council Decisions remain appropriate 
and effective. 

The scope of the evaluation concerns mainly: The relevance and EU added value of the Facility; 
The functioning of the Facility; The assessment of the parameters of the Facility; 

The  main  criteria  for  the  evaluation  of  the  Facility  are  the  relevance,  effectiveness,  efficiency 
and coherence of the instrument so as to fulfil its legal and institutional mandate. 

Summary of findings and conclusions  

The main findings and conclusions of the evaluation are as follows: 

–  The underlying intervention logic of the Euratom Loan Facility remains valid in the context 
of  the  EU’s  increasing  dependence  on  energy  imports;  high  and  volatile  oil  prices;  projected 
growth in electricity consumption within the EU; and, the need to cut greenhouse gas emissions 
to mitigate the impact of climate change. 

–  The  overall  objectives  of  the  Euratom  Loan  Facility  are  strongly  aligned  with  the  EU’s 
policy objectives relating to secure and  affordable energy supply; climate change; job creation 
and economic growth; and, promotion of nuclear safety and security in third countries. 

–  The  Euratom  Loan  Facility  has  promoted  and  accelerated  the  development  of  the  EU’s 
nuclear energy sector through direct financing of economically viable and environment friendly 
projects.  It  is  estimated  that  Euratom  loans  co-financed  21  per  cent  of  the  total  investment  in 
new builds in the EU over the period 1977 to 2003. 

–  By enabling investment in the nuclear sector, the Euratom Loan Facility has contributed to 
the decarbonisation and diversification of the EU’s sources of energy supply. 

–  A  majority  of  the  plants  co-financed  by  Euratom  loans  are  still  in  operation,  generating 
114,142 GWh of low carbon electricity annually (representing circa 6 per cent of the EU’s gross 

28

 
 
 
 
 
 
 
 
 
electricity generation and 12 per cent of nuclear electricity generation). 

In the absence of this indigenous production capacity, the EU would be importing an additional 
10Mtoe of energy on an annual basis. Secondary benefits of the Facility include the creation of 
6,000 highly skilled jobs at the plants under operation plus jobs and output creation in the wider 
economy through backward and forward linkages. 

–  The expansion of the geographic scope of the Euratom Facility in 1994 to CEEC and CIS 
was  relevant  and  appropriate.  Loans  approved  under  the  1994  Decision  directly  contributed  to 
safety  enhancements  and  promoted  greater  transparency  of  nuclear  operations  in  Bulgaria, 
Romania  and Ukraine. Safety improvements financed by Euratom loans have helped bring the 
nuclear  installations  in  these  countries  in  line  with  internationally  recognised  nuclear  safety 
principles  and  standards.  Euratom  lending  was  also  crucial  in  achieving  wider  reform  in  these 
countries  such  as  the  creation  and  funding  of  decommissioning  funds;  reform  of  electricity 
tariffs; and, increase in the scale of nuclear insurance.  

–  The Facility has an important ‘signalling’ effect i.e. an EU endorsement of the project which 
provides  a  positive  message  to  the  market,  Governments  and  the  public  about  the  project’s 
economic  and  technological  viability;  and,  a  ‘catalytic’  effect  i.e.  Euratom  lending  helps 
leverage financing from other sources. 

–  The Euratom Loan Facility provides loans on attractive terms to  borrowers. The European 
Commission operates on a non-profit basis and passes on the benefits of its AAA/Aaa rating to 
its borrowers. The difference between the cost of capital raised on the market and the cost of the 
Euratom loan represents the financial added value of the Facility in the case of each project. 

–  As regards the financing needs of the nuclear sector, the evaluation identifies a financing gap 
for new builds and large scale infrastructure for demonstration of next generation technologies. 
Additional,  exceptional  financing  needs  (as  yet  hard  to  quantify)  might  also  arise  from  safety 
improvements/ upgrades required as a result of the EU ‘stress tests’. 

–  The  amount  available  for  new  loans  within  the  current  ceiling  is  EUR  626  million.  The 
present  resources  and  borrowing  ceilings  for  the  Facility  would  not  be  adequate  to  meet  the 
expected future demand for loans. 

–  The financial management and implementation arrangements for the Euratom Loan Facility 
have worked extremely well and there is evidence of them being effective: all loans within the 
EU  have  been  fully  repaid;  there  has  been  no  recourse  to  the  EU  budget  guarantee  during  the 
lifetime of the Facility; and, the Euratom Loan Facility has delivered its stated objectives. 

–  However,  some  operational  aspects  of  the  Facility  could  be  improved.  There  is  scope  to 
enhance  the  visibility  of  the  Facility  and  the  processes  relating  to  the  procurement  of  external 
expertise could be streamlined. 

Availability of the report on Europa 

http://ec.europa.eu/economy_finance/evaluation/completed/index_en.htm#loan_facility  

29

 
 
 
 
02 – Enterprise  

Final Evaluation of the Competitiveness and Innovation Framework Programme 

ABB activities: 

02 02 Competitiveness, industrial policy, innovation and 
entrepreneurship  

Timing:  

 December 2011 

Background, scope and focus  

The  Competitiveness  and  Innovation  Framework  Programme  (CIP)  aims  to  foster  the 
competitiveness  of  enterprises,  in  particular  of  SMEs;  to  promote  innovation,  including  eco-
innovation; to develop a sustainable, competitive, innovative and inclusive information society 
and to promote energy efficiency and renewable energy. The CIP was established by Decision 
No  1639/2006/EC  which  included  specific  provisions  relating  to  the  evaluation  of  the 
Programme.  The  final  evaluation  of  the  CIP  provided  an  overall  assessment  of  the  CIP  by 
analysing  the  existing  evaluations  of  each  of  the  three  component  programmes  (EIP,  ICT-PSP 
and IEE), with a special emphasis on tracing cross-cutting issues and themes. 

Summary of findings and conclusions  

The  evaluation  concludes  that  the  framework  programme  as  a  whole  and  its  specific 
programmes individually are all performing well, in line with expectations at the launch of the 
programme. The CIP has become a major vehicle for promoting innovation. Innovation in this 
case  is  viewed  as  a  relatively  open  process  going  beyond  the  simple  focus  on  technological 
development.  It  is  a  broader  activity  that  encompasses  developments  in  the  service  sector  as 
much  as  in  manufacturing  and  relates  to  processes  and  business  models  as  much  as  products, 
which in turn makes a major contribution to the competitiveness of the European economy.  

The  evaluation  reports  that  major  measures  such  as  Financial  Instruments,  Enterprise  Europe 
Network, innovation and eco-innovation are on track to achieve the targets set. The evaluation 
also  draws  attention  to  small  budget  items  which  are  far  from  negligible  in  their  effects.  For 
example, the action programme for reducing administrative burdens and its follow-on measures 
are estimated to be delivering savings for enterprises valued at over €40 billion. 

The  evaluation  concludes  that  the  current  economic  and  financial  crisis  has  underlined  the 
significance  of  many of the issues that the CIP was designed to  address,  while the urgency of 
measures to meet the 2020 climate targets continues to increase. 

The  utility  of  the  CIP  is  thus  higher  than  ever,  especially  since,  notwithstanding  the  scope  for 

30

 
 
 
 
 
 
 
 
 
 
 
further  improvements,  the  effectiveness  of  the  instruments  used  has  increased  over  time.  The 
results  of  the  evaluation  were  presented  to  the  CIP  joint  committee  meeting  as  well  as  at  the 
expert workshop on the effectiveness of the CIP which was organised by the Policy Department 
of the European Parliament. 

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/cip_final_report_cses_300112_en.pdf 

Final evaluation of the Entrepreneurship and Innovation Programme 

ABB activities: 

02 02 Competitiveness, industrial policy, innovation and 
entrepreneurship  

Timing:  

 April 2011 

Background, scope and focus  

The  Entrepreneurship  and  Innovation  Programme  (EIP)  aims  to  support  entrepreneurship  and 
innovation and to promote the development and growth of SMEs across the EU. The programme 
(2007-2013)  was  established  as  one  of  the  three  ‘pillars’  of  the  CIP  by  Decision  No 
1639/2006/EC which included specific provisions relating to the evaluation of the Programme. 
The  final  evaluation  of  the  Programme  was  designed  to  address  the  relevance,  coherence  and 
synergies, effectiveness, efficiency, sustainability, and utility of the EIP, and where possible the 
distribution of funding with regard to sectors.  

Summary of findings and conclusions  

The  final  evaluation  of  the  EIP  concluded  that  the  programme  is  on  track  to  achieve  the 
anticipated  impacts.  It  is  effectively  benefitting  end-users,  in  particular  SMEs,  and  its  major 
strength is that it is able to concentrate on core issues for SMEs in a direct and practical way. 
The  measures  under  the  EIP,  notably  the  financial  instruments  and  eco-innovation,  have 
effectively  created  conditions  for  real  replication  in  the  market.  The  eco-innovation  first 
application  and  market  replication  funding  scheme  is  successfully  addressing  an  important 
market failure in the implementation of new ideas and approaches in products and services that 

31

 
 
 
 
 
 
 
 
 
 
 
 
 
benefit the environment. Innovation actions, with their focus on clusters, innovation in services 
and  eco-innovation,  are  regarded  as  successful  and  appropriate,  especially  by  those  who  have 
participated in them. The actions have been characterised by easier procedures and shorter time 
to contract than comparable award and management processes in the research and development 
area.  The  Enterprise  Europe  Network  is  a  major  policy  asset  for  the  EU’s  relationship  with 
enterprises.  It  has  considerable  further potential  in  terms  of  the  engagement  of  SMEs,  with its 
integration into the business support system across Europe, and the strength of the personal links 
between its members. Financial Instruments are achieving their objectives of facilitating access 
to  finance  for  the  start-up  and  growth  of  SMEs.  The  GIF  facility  and  SMEG  loan  and  micro 
credit  windows  are  relevant  to  the  needs  of  European  SMEs;  they  fulfil  a  demand  for  finance 
which otherwise would not have been met. The findings and recommendations of this evaluation 
have guided the preparation of the follow-up programmes, COSME and Horizon 2020.   

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/110426_final_report_en.pdf 

Evaluation of the Executive Agency for Competitiveness and Innovation (EACI) 

ABB activities: 

02 02 Competitiveness, industrial policy, innovation and 
entrepreneurship  

Timing:  

 May 2011 

Background, scope and focus  

Art. 25 under Council Regulation (EC) No 58/2003 requires an external evaluation of executive 
agencies  on  a  three  year  basis.  This  evaluation  covers  the  second  three  years  of  operation  of 
EACI,  between  1  July  2008  and  31  December  2010  focusing  on  the  implementation  of  the 
Intelligent Energy Europe Programmes (IEE I and IEE II) as well as part of the Entrepreneurship 
and  Innovation  Programme  (EIP)  in particular  the  Enterprise  Europe  Network,  Eco-innovation 
and  IPeuropeAware  initiative  and  Marco  Polo  I  and  II  programmes.  The  purpose  of  this 
evaluation  was  to  examine  how  the  EACI  has  handled  specific  project  management  tasks 
(project application and award procedures, monitoring, financial aspects, etc.). 

Summary of findings and conclusions  

32

 
 
 
 
 
 
 
 
 
 
 
 
The  evaluation  confirmed  that,  overall,  the  EACI  is  performing  well  and  is  an  efficient  and 
effective delivery mechanism for the initiatives for which it has operational responsibility. The 
main  recommendations  addressed  improvements  needed  to  the  EACI  staff  retention  rate;  the 
short average period of staff remaining at the Agency and the relatively high costs arising from 
the staff turnover rate, should be avoided in order to increase operational effectiveness. It may be 
necessary to tackle this issue jointly with other Executive Agencies as it could involve changes 
in  current  regulations.  The  evaluation  also  recommended  further  IT  improvements  throughout 
the Agency’s systems to enhance its performance. This was judged as particularly necessary in 
the case of the Enterprise Europe Network where an effective, reliable IT backbone for delivery 
of services and support for administrative functions is needed. Finally, the evaluation called for 
further  development  of  the  operational  indicators  used  by  the  EACI:  in  order  to  provide  on-
going updates and feedback on the effectiveness of programmes and initiatives, the EACI should 
develop operational indicators, in collaboration with parent DGs, rather than doing so at longer 
intervals of 2-3 years.  

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/110512__evaluation_eaci_en.pdf 

Final Evaluation of the Lead Market Initiative 

ABB activities: 

02 02 Competitiveness, industrial policy, innovation and 
entrepreneurship  

Timing:  

 July 2011 

Background, scope and focus  

The  Lead  Market  Initiative  (LMI)  aims  to  promote  greater  innovation  in  Europe  and  is  now  a 
central  feature  of  Europe  2020.  It  promotes  the  development  of  6  specific  lead  markets:  bio-
based  products,  eHealth,  protective  textiles,  sustainable  construction,  recycling  and  renewable 
for  3-5  years).  The  Commission 
energies 
Communication that launched LMI envisaged that a final evaluation report on the first cycle of 
the LMI would be presented in 2011. The purpose of this evaluation is to prepare the report. 

(corresponding  action  plans  operational 

33

 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The  evaluation  report  of  the  first  cycle  of  the  LMI  concludes  that  the  major  strength  of  the 
initiative was targeting interrelated policy areas for promising markets that would not be picked 
up  by  other  policy  frameworks.  LMI  has  promoted  the  use  of  demand-side  policy  tools  (e.g. 
public procurement, standardisation, regulation and complementary actions) as an important new 
element  in  innovation  policy.  The  six  markets  initially  chosen  as  targets  continue  to  show  a 
marked potential for further growth. In four sectors (i.e. bio-based products, e-Health, protective 
textiles,  and  sustainable  construction)  the  LMI  has  showed  clearly  that  co-ordinated  demand–
side  initiatives  at  European  level  are  valuable  instruments  for  promoting  market  uptake  of 
innovative products. It is important that there should be follow-up to the results achieved under 
the Action Plans. In particular the full impact of the work of the procurement networks will only 
be felt if there is effective dissemination and take-up of the initial results.  

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/final-eval-lmi_en.pdf 

Interim evaluation of Erasmus for Young Entrepreneurs 

ABB activities: 

02 02 Competitiveness, industrial policy, innovation and 
entrepreneurship  

Timing:  

 May 2011 

Background, scope and focus  

Under Erasmus for Young Entrepreneurs, new entrepreneurs can acquire the skills necessary to 
manage an enterprise by working with an experienced entrepreneur in another EU member State 
for between 1 to 6 months. The rational for the Erasmus for Young Entrepreneurs is provided by 
the  Small  Business  Act  for  Europe.  The  objective  of  the  interim  evaluation  of  Erasmus  for 
Young Entrepreneurs was to evaluate the relevance, effectiveness and efficiency of the actions 
implemented in the framework of the Pilot Project and the Preparatory Action. 

34

 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The evaluation showed that there is a strong rationale for the Erasmus for Young Entrepreneurs 
scheme. There is positive feedback from new entrepreneurs on the contribution of stays to their 
business  ideas.  The  mobility  scheme  is  likely  to  increase  the  quality  of  start-ups,  SME 
innovation as well as the SMEs’ internationalisation. The research suggests it has the potential to 
help create a significant number of jobs if it is expanded. Some shortcomings (e.g. a backlog of 
unmatched new and host entrepreneurs) are due to teething problems rather than to fundamental 
design faults. The evaluation gave a number of recommendations how to tackle these issues and 
improve the scheme.  

The  participation  in  the  programme  has  been  intensifying  since  last  year.  Whereas  it  took  16 
months  to  organise  400  exchanges  in  2010,  the  last  400  exchanges  have  been  organised  in  11 
months.  This  partially  results  from  the  implementation  of  evaluation  recommendations  in 
relation  to  the  IT  matching  tool,  the  increased  management  of  the  “dormant”  entrepreneurs’ 
applications, the increased number of available host entrepreneurs in countries in high demand 
and the better EU coverage of the network of Erasmus for Entrepreneurs local contact points. 

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/eye_final_report_en.pdf 

Evaluation of the Directive 2009/142/EC on appliances burning gaseous fuels 

ABB activities: 

02 03 Internal market for goods and sectoral policies 

Timing:  

 March 2011 

Background, scope and focus  

Directive  2009/142/EC  on  Appliances  Burning  Gaseous  Fuels  (GAD)  came  into  force  on  1 
January 1996 and aims to promote free circulation of gas appliances within the internal market 
and achieve a high standard of consumer protection. The purpose of the evaluation is to assess 
its  relevance,  utility,  effectiveness  and  efficiency  in  order  to  identify  the  strengths  and 
weaknesses of the Directive and assess the potential for its future improvement. 

35

 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The evaluation concluded on the overall efficiency and effectiveness of the implementation and 
functioning  of  the  gas  appliance  directive  in  meeting  its  main  objectives  of  ensuring  free 
movement  of  products  and  protecting  against  risks  to  safety  and  health.  The  issues  identified 
relate to necessary clarifications and technical updates of its current provisions and to the scope 
of the directive. These issues are being further analysed in the context of the alignment of the 
directive  with  the  new  legislative  framework,  scheduled  for  2013:  a  public  consultation  took 
place  between  December  2011  and  March  2012,  and  an  impact  assessment  study  is  being 
carried-out (due to be completed in September 2012). 

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/03_2011_finalreport_gas_en.pdf 

Ex-Post Evaluation on Enhancing the Implementation 
of the Internal Market Legislation Relating to Motor Vehicles 

ABB activities: 

02 03 Internal market for goods and sectoral policies 

Timing:  

 June 2011 

Background, scope and focus  

Directive  2007/46/EC  establishes  a  framework  for  the  approval  of  motor  vehicles  and  their 
trailers, and of systems, components and separate technical units intended for such vehicles. The 
Directive was subject to a combined ex-post evaluation and impact assessment study aiming at 
supporting the preparation of a new legislative proposal for the directive. The process was in line 
with  the  new  requirement  in  the  Commission's  Working  Methods  (reiterated  in  the  Smart 
Regulation Communication of October 2010) to base all significant proposals for a revision or a 
new legislative measure on an evaluation of policy framework already in place. 

36

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The  evaluation  concluded  that  the  Directive  has  proven  its  relevance,  although  the  scope  may 
not  be  sufficient  to  address  future  market  changes.   Additionally,  there  are  still  problems  with 
unsafe and/or non-compliant automotive products  being placed on the EU  market. Finally, the 
effectiveness  of  Directive  2007/46/EC  relies  significantly  on  the  quality  and  performance  of 
technical  services  and  also  on  the  ability  to  ensure  conformity  of  production.  Actions  which 
undermine  the  effectiveness  of  technical  services  (such  as  type  approval  hopping)  also 
undermine the Directive's effectiveness. Therefore, the revision of the directive should consider 
proposing specific measures to address the stated problems. 

Availability of the report on Europa  

http://ec.europa.eu/enterprise/dg/files/evaluation/report_internal_market_legislation_en.pdf 

37

 
 
 
 
 
04 – Employment and social affairs 

Assessing the effectiveness and the impact of the Social OMC in preparation of the 
new cycle 

ABB activities: 

04 04 Employment, social solidarity and gender equality 

Timing:  

 May 2011 

Background, scope and focus  

This is the first comprehensive, external evaluation of the Social OMC, with a twofold purpose:  

 To develop a reference methodology for evaluation of the effectiveness and the impact of 

1. 
the Social OMC which can be used by the SPC and the European Commission and  

 To  apply  this  methodology  in  an  assessment  of  the  Social  OMC  2006-2010  which  can 

2. 
inform decisions about its role and shape after 2010.  

The  evaluation  was  carried  out  by  a  consortium  led  by  PPMI,  in  association  with  AEIDL,  the 
Austrian  Institute  for  SME  research,  CSES  and  Tarki  social  research  institute,  between 
November 2009 and May 2011.  

Summary of findings and conclusions  

The  first  part  of  the  evaluation  provided  an  overview  of  existing  evaluation  methodology  and 
results  since  the  start  of  the  OMC's  in  the  social  field  around  2000,  and  included  the 
development  of  a  draft  framework  of  analysis.  The  second  part  applied  this  framework  of 
analysis  to  the  evaluation  of  the  effectiveness  and  impact  of  the  social  OMC  since  2006  (till 
2009).  Effectiveness  is  defined  as  the  extent  to  which  the  complex  institutional  mechanism  of 
the  OMC  (agreeing  on  common  objectives,  regular  reporting  at  the  EU  and  national  level, 
evidence-based  mutual  learning,  monitoring  of  progress  based  on  a  set  of  commonly  agreed 
indicators) functioned successfully; whereas impact is the extent to which the Social OMC has 
actually influenced policies and policy-making processes at the domestic level. 

Main evaluation findings 

Effectiveness of the institutional framework  

–  Several  changes  in  the  political  environment  contributed  positively  to  the  effectiveness  of 
the  social  OMC:  the  continuous  improvement  of  the  set  of  common  indicators,  including  the 
extension into the fields of pensions and health/long-term care after the streamlining decision in 

38

 
 
 
 
 
 
 
 
 
 
 
2006;  the  extension  of  the  reporting  cycle  (decreasing  the  administrative  burden)and  the 
introduction of thematic focus years (deemed as a success by most stakeholders); improvements 
in  the  format  of  various  mutual  learning  processes  (notably  by  linking  them  better  to  the 
preparation of the Joint Report).  

–  Paradoxically,  the  social  OMC  is  still  seen  as  a  process  by  and  for  "insiders",  with  low 
public visibility, and limited awareness among some of the key stakeholders.  

Likeliness of impacts on policy decisions  

–  The  social  OMC  lacks  binding  targets,  does  not  lead  to  compulsory  recommendations  and 
does not provide for legal sanctions in case of non-compliance.  

–  However,  it  is  and  important  and  innovative  component  of  EU  governance  which  accepts 
normative  and  institutional  differences  among  Member  States  and  provides  a  framework  for 
coordination  based  on  comparison  and  mutual  learning.  It  works  in  combination  with  other 
frameworks of cooperation (initiated by OECD, World Bank, ILO etc.) and influences polices by 
drawing attention to particular issues, at a particular moment in time (such as child poverty), by 
offering  formats  for  discussion,  procedural  templates  and  concepts  for  reflection.  Very  often, 
actors use concepts and themes with clear linkages to the social OMC (such as active ageing), 
but often without recognising such linkages.  

Substantive impacts of the social OMC  

–  The  cognitive  and  discursive  impacts  may  lead  to  impacts  on  decisions,  depending  on 
whether they already pre-existed, were made available for mutual learning and used in the way 
which was politically and economically feasible in their contexts. The eleven in-depth country 
studies provided a wide range examples where the  social OMC  has influenced policy thinking 
which led in turn to policy decisions (e.g. in Denmark, where it triggered the formulation of a 
national homelessness strategy for 2008-2011).  

–  Overall,  the  social  OMC  can  be  credited  with  developing  a  conceptual  and  procedural 
toolbox  that  helped  the  social  policy  actors  (especially  in  inclusion)  to  assert  a  more  balance 
position  with  regards  to  policies  which  have  been  traditionally  viewed  as  growth-oriented  and 
thus  more  important  (e.g.  linking  the  struggle  against  poverty  to  the  wider  debate  on  social 
protection,  and  providing  a  reference  point  for  the  discourse  on  adequacy  as  well  as  financial 
sustainability of pensions).  

Procedural impacts of the social OMC  

–  The social OMC clearly defines a concept of good governance which is a combination of a 
genuine  involvement  of  all  actors  at  all  levels  and  across  the  policy  spectrum  and  decision-
making based on data and evidence which is comparable across Member States.  

–  The social OMC has the clearest procedural impacts on the involvement of non-state actors 
(in particular at the EU level) during the National Strategy Report process and on strengthening 
statistical capacities. The actual evidence on the success of such interaction with non-state actors 
is mixed, even if it had a positive influence in building their capacity. The collection, availability 
and  usage  of  data  has  especially  improved  in  countries  with  less  developed  national  (or  sub-
national structures for data collection. 

–  The impact on vertical co-ordination across all levels of government has been limited, as in 
most countries the social OMC remains very much a central government process, even if there 
are some countervailing examples of good practices.  

–  The impact on horizontal coordination was probably greater than on vertical coordination, as 
it  was  normal  practice  to  ask  non-social  ministries  to  provide  their  inputs  into  the  National 
Strategy Report, mostly as a formality, but in some cases providing opportunities for interaction 
that would otherwise not have taken place.  

39

 
 
Streamlining between the three strands  

–  The social inclusion strand has been the most advanced in terms of stakeholder activism and 
development of common indicators. During 2006-2010, the two strands (pensions and health/LT 
care)  caught  up,  by  developing  common  indicators,  integrating  into  the  mutual  learning 
programme, creating the ASIP network and building a stronger stakeholder basis, but much still 
remains to be done in all these fields.  There have been important developments in the pension 
strand during this period, while the health/LTC strand remains the weakest, not least because the 
policy field is institutionally more fragmented and consequently more heterogeneous.  

–  The  policy  rationale  for  putting  together  the  three  strands  in  2006  has  been  contested  by 
many policy actors, and as a result there is little evidence that streamlining has led to joined-up 
approaches and synergies between the three strands at domestic level.  

–  However,  some  evidence  of  positive  effects  starts  to  materialise,  notably  through  the 
awareness of actors of the possible connections and aims of the other policies.  

Availability of the report on Europa  

http://ec.europa.eu/social/main.jsp?catId=701&langId=en  

Evaluation of ESF support to gender equality 

ABB activities: 

04 02 European Social Fund 

Timing:  

 January 2011 

Background, scope and focus  

The aim of the study was to provide an assessment of the ESF's (2007-2013) support to gender-
equality  policy  by  examining  the  gender  sensitivity  of  ESF  programmes,  on  the  basis  of 
available evidence on the programming process and initial implementation. Specific findings on 
each  Member  State  are  presented  in  27  country  reports  and  focused  thematic  analysis  is 
presented  in  6  thematic  reports  on  "Enhancing  women’s  access  to  employment",  "Vertical 
segregation", "Horizontal segregation", "Reconciliation", "Participation of women in enterprise 
creation and growth", "Education and training". The synthesis report highlights similarities and 
differences  in  Member  States'  approaches  and  concludes  on  the  main  ways  in  which  the  ESF 
intervention generates or is expected to generate European added value. The study was carried 

40

 
 
 
 
 
 
 
 
 
 
 
 
 
out by consortium GHK Consulting Ltd and Fondazione G. Brodolini. 

Summary of findings and conclusions  

There is undisputed European Added Value in the ESF's support to the gender equality policy, 
mostly acknowledged as support for the creation of a recognized and autonomous policy space 
for  gender  equality  (e.g.  the  ESF  has  'pushed'  gender  equality  on  the  policy  agenda  in  most 
Member States) and capacity building. For some Member States, the ESF contribution has been 
very important also from a budgetary point of view, as it has allowed implementing actions for 
which funds would otherwise not be available.  

The  gender  equality  theme  has  been  considered  by  all  Member  States  in  ESF  Operational 
Programmes  but  with  important  differences  in  the  level  of  consideration.  A  comprehensive 
policy effort analysis (using a composite index developed specifically for this evaluation) shows 
unsatisfactory  levels  of  consideration  of  gender  issues,  especially  in  the  Member  States  with 
more  severe  gender  gaps.  The  dual  approach  requirement  was  in  practice  translated  into  a 
significant  decrease  of  gender-specific  actions  (comparing  to  the  previous  period)  and  over-
dependence  on  gender  mainstreaming,  which  is  however  not  appropriately/effectively  applied; 
alarmingly, there is not yet a common clear understanding of its theoretical underpinnings and 
operational  implications.  Member  States  are  still  not  investing  sufficiently  to  increase  their 
internal capacity in this field. This is a serious matter of concern as the contribution of the ESF 
to  gender  equality  in  the  current  cycle  will  be  crucially  dependent  on  how  well  the  gender 
mainstreaming principle is applied. 

ESF  impact  in  this  field  will  also  be  determined  by  how  the  relatively  limited  resources  for 
gender equality specific actions are focussed on critical issues. Worryingly, the actions devised 
remain overly geared to the supply side (e.g. helping women to access jobs) while many crucial 
aspects  and  causes  of  gender  inequality  are  not  sufficiently  addressed  (e.g.  educational 
segregation,  cultural  and  social  stereotypes)  limiting  the  ESF's  potential  to  contribute  to  the 
objective of better employment for women by influencing critical issues such as the way jobs are 
created (which and where) and the way work is organised.  

Progress  of  implementation  has  been  slower  for  the  gender  equality  specific  actions  (than  for 
other types of actions) and in some cases resources have been diverted towards other objectives 
as  a  response  to  the  economic  crisis.  This  indicates  a  relatively  weak  status  of  the  gender 
equality objectives which, in the absence of a formally funded priority, are easily overcome by 
other policy objectives or weakened by administrative difficulties.  

Whilst the ESF is still able to produce added value in relation to this policy field, there are some 
signals of diminishing returns or steps backward. In addition to what has been mentioned above, 
in  most  Member  States  the  ESF  is  reinforcing  already  funded  strategies  instead  of 
complementing  and  innovating.  Innovative  and  effective  approaches  previously  encouraged 
through the ESF funded EQUAL, have been abandoned in this period (e.g. partnership among 
women’s organizations and institutions) and this is openly regretted by stakeholders.  

On  the  basis  of  these  findings  the  evaluation  addressed  a  number  of  recommendations  to 
Member States and to the European Commission. Most of the recommendations to the European 
Commission have already been adopted and have influenced the Commission's proposal for the 
new regulations on the Structural Funds and the ESF. 

41

 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/social/keyDocuments.jsp?type=0&policyArea=0&subCategory=0&country=0
&year=0&advSearchKey=evaluationesf&mode=advancedSubmit&langId=en  

Evaluation of ESF Support for Enhancing Access to the Labour Market and the 
Social Inclusion of Migrants and Ethnic Minorities 

ABB activities: 

04 02 European Social Fund 

Timing:  

 May 2011 

Background, scope and focus  

The Europe2020 Strategy gives priority to ‘empowering people in inclusive societies’ and sets a 
headline  target  of  increasing  overall  employment  rates  to  at  least  75%.  This  will  require  a 
substantial  increase  in  labour  participation  rates  and  the  better  integration  of  migrants  and 
minorities  in  the  workforce,  which  is  especially  crucial  in  the  context  of  demographic  ageing. 
The  Europe2020  target  of  reducing  poverty  and  social  exclusion  by  25%  is  also  pertinent  for 
migrants  and  minorities,  and  especially  the  Roma,  Europe’s  largest  minority  and  most 
disadvantaged social group.  

The evaluation aims were:  

1.  Assess the European Social Fund's contribution to enhancing access to the labour market and 
the social inclusion of migrants and ethnic minorities;  

2.  Identify  the  most  effective  means  of  tackling  barriers  to  the  social  and  labour  market 
integration of migrants and ethnic minorities;  

3.  Provide  guidelines  for  the  Member  States  on  the  implementation  of  ESF  Operational 
Programmes (OPs) in the 2007-2013 programming period in order to target beneficiaries from 
these target groups more effectively;  

4.  Make  recommendations  looking  ahead  to  the  role  of  the  ESF  in  promoting  access  to  the 
labour market for migrants and ethnic minorities post-2013.  

The evaluation scope included core ESF in the 2000-2006 and 2007-2013 programming periods 
respectively,  and  the  ESF-financed  EQUAL  Community  Initiative  2000-2006.  The  evaluation 
was carried out by the Centre for Strategy & Evaluation Services (CSES). 

42

 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

EU Added Value: The ESF has addressed an important funding gap by providing resources to 
promote access to the labour market and the social inclusion of migrants and minorities (‘volume 
effects’). The ESF has brought additional targeted funding support to these groups that would not 
otherwise  have  been  available  through  national  support.  The  ESF  in  general  (and  EQUAL  in 
particular) have had important ‘scope effects’ by extending coverage to wider target groups that 
had  not  previously  been  addressed  through national programmes,  or  had only received  limited 
attention.  

–  There  is  increased  attention  promoting  the  labour  market  and  social  inclusion  of  migrants 
and  minorities  in  ESF  in  the  2007-2013  period  compared  with  2000-2006.  This  is  reflected  in 
higher levels of expenditure on these groups and in further attention to these groups in ESF OP 
strategies: Participation by migrants and minorities in ESF appears to be significantly higher in 
the 2007-2013 period. In total, 1.19 m people with a migrant or minority background have taken 
part. This equates to approximately 8.7% of all ESF beneficiaries. 

–  Achievements regarding ESF support to Individuals: the ESF made a positive contribution in 
helping people with a migrant and minority background to overcome barriers to labour market 
access, to improving language skills and strengthening basic skills for employability such as ICT 
skills,  basic  literacy  and  communication  skills.  Moreover,  the  ESF  has  promoted  workforce 
adaptability for migrants through activities to strengthen their professional qualifications and in 
industrial areas undergoing restructuring. Migrants and minorities have been encouraged through 
the ESF to take the ‘first steps’ into further education and training through non-formal learning. 
ESF has also addressed an important gap in vocational training and apprenticeship provision in 
some  EU  countries  for  migrant  youth.  ESF  has  strengthened  the  recognition  of  qualifications 
from third countries. Entrepreneurship has been promoted among migrants and minorities. ESF 
also  promoted  the  social  inclusion  of  individuals  with  a  migrant  and  minority  background,  for 
example,  by  fostering  pathways  to  employment  and  promoting  a  reduction  in  the  numbers  of 
early school leavers among young people with a migrant and Roma background. 

–  Regarding  systems  and  structures,  the  capacity  of  public  institutions  to  promote  the 
integration of people with a minority background has been strengthened as a result of both core 
ESF  and  EQUAL.  Knowledge  has  been  gained  and  experiences  shared  between  public 
institutions  and  migrant  and  minority  NGOs.  The  ESF  has  helped  to  improve  the  quality  of 
initial  integration  services,  such  as  the  reception  of  migrants  from  third  countries  (e.g. 
development  of  online  information  portals  about  relevant  services  in  the  most  common 
immigrant languages). The ESF has promoted the modernisation of public employment services 
(PES),  developing  intercultural  competences  among  staff  working  there  and  increasing 
awareness about the specific needs of job-seekers with a migrant and minority background and 
how to tailor support services accordingly. Moreover, the ESF has promoted the creation of new 
networks and organisational structures to promote the integration of migrants and minorities, as 
well  as  the  establishment  of  vertical  coordination  mechanisms  to  strengthen  joint  working  on 
Roma inclusion between the national, regional and local levels. 

in 

–  Migrants  and  minorities 
lifecycle:  Through  multiannual 
the  ESF  Programme 
programming, the ESF has encouraged the development of a long-term approach to promoting 
the  integration  of  migrants  and  minorities,  and  in  tackling  the  multifaceted  barriers  to  their 
inclusion.  While  the  ESF  strengthened  the  capacity  of  migrant  and  minority  representative 
NGOs and self-organised associations to design and implement projects, their ability to take part 
in  national  programming  processes  was  often  limited.  More  positively,  in  project  delivery, 
migrant and minority NGOs often played a key role, sometimes working to implement packages 
of  support;  The  direct  participation  of  migrants  and  minorities  in  ESF  project  implementation 
was found to be effective in reaching groups at a distance from the labour market. 

43

 
 
The  ESF  has  also  strengthened  horizontal  coordination  mechanisms  to  promote  greater 
cooperation  in  Structural  Funds  management  and  implementation  structures.  The  ESF  has 
promoted an  ‘integrated approach’ to the social and labour market integration of  migrants  and 
minorities  that  links  employment  with  education,  and  housing  with  health.  This  was  more 
evident in 2007-2013 than 2000-2006, suggesting EQUAL played a positive role in this regard. 
With regard to financial implementation, there do not appear to have been absorption problems 
in  relation  to  ESF  support  to  migrants  and  minorities  in  2000-2006,  although  some  problems 
were identified in 2007-2013. 

Monitoring and evaluation: In 2000-2006, there was a lack of participation data on migrants and 
minorities.  In  2007-2013,  following  the  introduction  of  Annex  XXIII  of  the  Implementing 
Regulations, there have been improvements in data availability, however, the quality of data still 
varies for ethnic minorities, with evidence of under-estimates due to data protection issues and a 
reluctance  to  self-ascribe  ethnicity  (especially  among  the  Roma).  There  was  insufficient 
emphasis at Member State level on assessing the real achievements of ESF support to migrants 
and  minorities.  In  particular,  there  was  too  great  a  focus  on  outputs  and  a  lack  of  attention  to 
results indicators. 

Availability of the report on Europa  

http://ec.europa.eu/social/keyDocuments.jsp?type=0&policyArea=0&subCategory=0&country=0
&year=0&advSearchKey=evaluationsocialinclusion&mode=advancedSubmit&langId=en&order
By=docOrder 

European Year 2010 for combating poverty and social exclusion - Ex-post evaluation

ABB activities: 

04 04 Employment, social solidarity and gender equality 

Timing:  

 November 2011  

Background, scope and focus  

In recent years the European Union has organised a series of 'European Years', designed to draw 
attention  to  a  particular  theme  of  interest  to  people  and  organisations  at  large,  as  well  as  to 
policy makers and professionals. 2010 was designated the European Year Against Poverty and 
Social Exclusion. The goal was to raise awareness, build new partnerships, create a momentum 
and get a strong political commitment to fighting poverty in Europe.  

An evaluation was carried out by ECORYS and assessed how the EY2010 played out and what 

44

 
 
 
 
 
 
 
 
 
 
 
impacts  it  has  had  and  may  have  in  future.  The  research  included  review  of  background 
documentation and data; surveys and individual consultations as well as in-depth studies of the 
results of the Year in Bulgaria, Italy, Lithuania, Sweden and the UK; and attendance at events 
and several activities. 

Summary of findings and conclusions  

The  Year  proved  highly  relevant  and  topical  to  a  wide  audience,  notably  as  it  took  place 
against the background of persisting economic and financial crisis. This meant added impact but 
at the same time posed challenges in terms of communicating messages on what can sometimes 
be a sensitive topic. The Year complemented existing EU initiatives in the field and increased 
the  visibility  of  people  experiencing  poverty  and  social  inclusion.  By  involving  NGOs,  social 
partners  and  other  stakeholders,  the  Year  addressed  the  need  to  expand  participation  in  the 
design and implementation of social inclusion policies. In individual countries, EY2010 funding 
allowed  a  range  of  small-scale  activities  to  take  place,  which  could  not  otherwise  have  been 
implemented. In most countries activities to address the needs of all the main groups at risk were 
organised (children, youth, women, elderly, homeless people, migrants and ethnic minorities). 

In terms of the effectiveness and impact of the EY2010, progress was made on all four of the 
EY2010 objectives. At EU level in particular commitment and concrete actions featured high in 
the evaluator's assessment of achievements (also because of a final Declaration of the Council on 
the  Year).  Key  players  at  national  level  however  remain  cautious  about  the  extent  to  which 
similar commitments and concrete actions will take root, especially given current constraints on 
public spending. EY2010 activities in the majority of countries attracted strong participation by 
stakeholders, particularly NGOs and associations. At EU level too, the Year afforded valuable 
opportunities for stakeholders to come together, make connections and identify areas of common 
interest. In terms of the impact on the general public, evidence suggests that information on the 
Year may have reached about one tenth of Europeans through the broadcast media. One of the 
goals of the EY2010 was to involve actors that are not traditionally involved in social inclusion 
policies, and this was achieved to a certain extent at national level; perhaps less so at EU level. 
Notably, a number of innovative approaches were tested during the Year: using social media for 
communication, engaging with journalists and participatory approaches to events. Examples of 
policy  innovation  were  identified  in  a  few  countries:  advanced  participatory  approaches  to 
policy planning, evidence-based social policy, social entrepreneurship. Links with other policy 
themes were made by some initiatives, notably in relation to health, justice and finance. 

The Year should have lasting effects at national level in terms of visibility and capacity of the 
participating  stakeholders  and  the  development  of  fresh  approaches  to  communicating  and 
promoting  social  inclusion.  At  EU  level  there  is  cautious  optimism  about  high-level 
developments in EU policy (specifically the headline targets on reducing poverty), which were 
helped  by  the  momentum  and  interest  in  poverty  and  social  exclusion  created  by  the  Year.  In 
addition,  the  final  Declaration  of  the  Council  represented  a  firm  commitment  of  the  EU  and 
Member States to go beyond awareness-raising and give the Year ‘a strong political legacy that 
delivers concrete results.  

It took some time to set up the system at EU and national level, largely because of the decision 
to  use  administrative  structures  that  were  already  accredited  by  the  EC  for  managing  EU 
funding. However, on the whole, the management of the Year by the European Commission 
was  judged  satisfactory  by  National  Implementation  Bodies  (NIBs)  and  stakeholders.  The 
Consultative  Committee  supporting  the  NIBs  and  the  EU  expert  stakeholders  group  were 
considered  both  valuable  instruments.  Procedures  were  judged  clear,  although  there  were 

45

 
 
 
requests for more guidance at an earlier stage. Cooperation between NIBs and the EY2010 task 
force worked satisfactorily, although for communication activities the division of tasks between 
EU  and  national  levels  was  not  always  clear  and  smooth.  Cooperation  with  the  national 
correspondents  of  the  communication  contractor  was  satisfactory  in  most  countries,  despite 
some initial uncertainty on their roles. EC representations were involved and in certain countries 
also active in organising their own events. 

Although  the  EY2010  is  now  over,  it  succeeded  in  injecting  new  momentum  into  the  fight 
against poverty and social exclusion in Europe. As a result, those involved in the field have been 
energised and inspired to step up their efforts, collaborate across themes and look at new ways to 
tackle  the  challenges  ahead.  This  mobilisation  has  increased  the  capacity  of  a  range  of  key 
players to make a difference and a very strong policy lead from the EU has re-focused the future 
agenda.  At  an  especially  challenging  time  for  the  economy  and  society,  the  EY2010  opened 
many  people’s  eyes  to  the  reality  of  poverty  and  social  exclusion  in  Europe;  recognised  the 
current  and  potential  contributions  of  stakeholders  and  policy-makers,  and  reinforced  the 
importance  of  listening  to  the  voices  of  those  experiencing  poverty  and  social  exclusion 
themselves.  Maintaining  the  momentum  started  by  the  Year  must  now  become  the  focus  of 
attention. 

Availability of the report on Europa  

http://ec.europa.eu/social/main.jsp?langId=en&catId=22  

Evaluation of the capacity of the ESF delivery systems to attract and support OP 
target groups 

ABB activities: 

04 02 European Social Fund 

Timing:  

 March 2011 

Background, scope and focus  

The purpose of the evaluation was to analyse the capacity of the delivery systems put in place in 
the  Member  States  to  attract  and  support  OP  target  groups  as  foreseen  by  the  ESF  Regulation 
Articles 2, 3 and 10 (including the target groups that are difficult to reach). The attraction and 
support of target groups requires a diversity of implementing bodies (IBs) and/or beneficiaries 
with  sufficient  knowledge  of  the  needs  of  target  groups,  as  well  as  appropriate  powers  and 
capacity.  

The evaluation covered EU27 in the programming period 2007-13, elaborating 18 case studies 
from 14 MS. The following methods were used: desk research, interviews with Member States 

46

 
 
 
 
 
 
 
 
 
 
 
representatives  of  ESF  Managing  Authorities,  case  studies  and  group  discussions  with 
stakeholders and experts. The evaluation was carried out by Eureval in consortium with Rambøll 
and Ecorys between December 2009 and March 2011. 

Summary of findings and conclusions  

The main findings of the evaluation are: 

1.  The  definition  of  clearly-identified,  simply-defined  and  easily-monitored  target  groups 
within the OP, accompanied by appropriate indicators for those groups is a key factor of success. 

2.  Existing  national  or  local  policies  for  supporting  target  groups  significantly  reinforce  the 
capacity  of  ESF  delivery  systems  to  reach  those  groups.  Moreover,  attracting  and  supporting 
target groups requires interventions on different levels. 

3.  Genuine  representation  of  or  dialogue  with  the  target  groups  in  the  design  and 
implementation of the OP can be a key success factor in attracting and supporting such groups. 

4.  Flexibility in the formal management arrangements can assist the attraction and support of 
target groups. 

5.  The  attraction  and  support  of  target  groups  requires  a  diversity  of  IBs  and/or  beneficiaries 
with  sufficient  knowledge  of  the  needs  of  target  groups,  as  well  as  appropriate  powers  and 
capacity. 

6.  Some MAs have not taken sufficient steps to simplify or reduce the administrative burden, 
creating the risk that some small organisations or NGOs are unable to participate in an activity, 
but there exist ways to better adapt application and payment procedures to those organisations. 

The  report  contains  a  set  of  10  recommendations  and  a  policy  proposal  based  on  these 
recommendations and also taking into account the on-going policy discussion: 

1.  A clear statement is needed that a main objective of the ESF is to contribute to the European 
Employment  Strategy  (EES)  in  a  strategic  way.  For  the  EES  to  be  a  strategic  European 
document, guiding the spending of the European Social Fund will require some changes. Those 
changes need to keep in mind that national priorities and the link between the OP, the NSRF and 
the  NRP  are  taken  into  account.  The  required  changes  are  addressed  by  the  following 
recommendations. 

2.  The  Integrated  Guidelines  have  a  catch-all  nature  that  makes  them  less  appropriate  for 
ensuring strategic contribution. It is therefore recommended that the OPs should be based on the 
NRP.  Furthermore  the  ESF  should  contribute  to  specific  measures  of  the  NRP  (and  not  to  the 
overall priorities or objectives). 

3.  One of the main reasons for lack of strategic linkage was that processes around the NRP and 
the  OP  are  coordinated,  but  in  various  MS  only  to  a  small  extent  integrated.  It  is  therefore 
recommended  that  a  country  specific  procedure  or  structure  is  designed  for  coordination  and 
communication purposes, linking the NRP and the OP processes. 

4.  Member States should be encouraged to use the ESF to fund the implementation of country 
specific recommendations when relevant. 

5.  It should be assessed  whether the NSRF becomes unnecessary as a strategic document for 
the  OP  if  the  OP  is  to  contribute  strategically  to  the  NRP.  In  general  it  will  be  necessary  to 
choose whether the OP should strategically be linked with the NSRF or the NRP. 

6.  and 7. The different cycles of the EES and the ESF make a strategic linkage difficult. It is 

47

 
 
 
therefore recommended, that:  

–  The policy cycles of the NRP and the OP are synchronised;  

–  The OP and the NRP are more directly linked, to avoid unnecessary bureaucracy. 

8.  ESF should mainly contribute to the EES through the results of activities and less through 
the transfer of ideas. 

9.  The  transfer  of  innovative  ideas  from  EES  and  ESF  is  especially  weak  when  it  comes  to 
ensuring that new ideas from EES are integrated in and funded by the ESF. It can therefore be 
recommended  that  clear  strategies  for  and  systems  to  support  a  transfer  of  ideas  are  designed. 
Specific procedures can be designed at MS level, in order to identify ESF relevant EES ideas, 
transfer these to the ESF and ensure implementation with or integration into ESF activities. 

10.  It is quite unlikely that “bottom up” transfer of ideas and innovations from ESF to EES are 
easily organised and it is therefore recommended not to invest too much energy and effort into it. 

Availability of the report on Europa  

http://ec.europa.eu/social/keyDocuments.jsp?type=0&policyArea=0&subCategory=0&country=0
&year=0&advSearchKey=evaluationesf&mode=advancedSubmit&langId=en 

Impact of enlargement and the functioning of transitional arrangements on labour 
mobility within the EU 

Evaluation related study

ABB activities: 

04 04 Employment, social solidarity and gender equality 

Timing:  

 July 2011  

Background, scope and focus  

The  overall  objective  of  the  study  was  to  firstly,  assess  the  functioning  of  the  transitional 
arrangements,  the  impact  of  enlargement  and  effects  of  workers'  mobility  on  the  economy, 
labour markets and social situation in the EU; secondly, prepare country-specific assessments in 
order to provide in-depth information on the impact of enlargement and mobility from Bulgaria 
and Romania; thirdly, estimate the mobility potential from the candidate and potential candidate 
countries. 

The study was carried out by the National Institute of Economic and Social Research (NIESR). 

48

 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The main focus of this study is an assessment of the macro-economic impact on both host and 
home  countries  of  the  increased  labour  mobility  that  has  resulted  from  the  two  recent  EU 
enlargements. The estimates to end-2009 suggest that: 

–  Since the 2004 enlargement, about 1.8 per cent of the EU-8 population has moved to the 
EU-15, raising the host country population by 0.3 per cent. Of this, approximately 75 per cent 
can be attributed to the enlargement process itself.  

–  Since 2007, about 4.1 per cent of the EU-2 population has moved to the EU-15, raising the 
host country population by a further 0.3 per cent. Of this, just over 50 per cent can be attributed 
to the enlargement process itself.  

–  The  potential  level  of  output  in  Bulgaria,  Romania  and  Lithuania  may  be  permanently 
reduced  by  5-10  per  cent  as  a  result  of  the  population  shifts  towards  the  EU-15  since  2004. 
Latvia and Estonia can also expect a permanent scar of at least 3 per cent on the potential level 
of  output  in  their  economies.  While  remittances  can  partially  offset  the  negative  effects  on 
growth in sending countries in the short- to medium-term, they cannot fully address the loss of 
labour input on capacity output in the longer-term.  

–  The impact on GDP per capita is much smaller than the impact on total GDP, but also tends 
to be negative in the sending countries, especially given the age structure of migrants, who are 
predominantly of working age.  

–  As for the receiving countries, the macro-economic impact of the population shifts from the 
EU-8 and EU-2 to the EU-15 since 2004 is expected to be small, possibly raising the long-run 
level  of  potential  output  by  about  ½  per  cent.  The  impact  on  Ireland  is  expected  to  be  more 
significant,  perhaps  raising  the  potential  level  of  GDP  by  3  per  cent  in  the  long-run.  The  UK 
may also benefit from a rise in potential output of just over 1 per cent.  

–  The long-run impact on GDP per capita is expected to be negligible in the EU-15, but may 
be  slightly  positive,  depending  on  the  productive  capacity  of  inward  migrants.  Outflows  of 
remittances are expected to have only a marginal negative effect on host countries.  

–  There is clear evidence that the pattern of transitional restrictions in place at the beginning of 
the  2004  enlargement  diverted  mobile  workers  away  from  traditional  destinations  –  namely 
Germany  –  and  towards  the  more  easily  accessed  labour  markets  in  the  UK  and  Ireland. 
However,  we  should  not  over-emphasize  the  magnitude  of  this  impact,  as  macro-economic 
developments and demographics have also played a role in the location decision, and in many 
cases appear to have played the dominant role. Transitional restrictions may have also played a 
certain  role  for  the  EU-2  economies,  although  the  rise  in  the  unemployment  rate  in  Spain  can 
explain about half of the nearly 10 percentage point loss of EU-2 migrant stock share between 
2006 and 2009.  

–  As  of  June  2011,  workers  from  the  EU-2  still  faced  some  restrictions  on  access  to  labour 
markets in Belgium, Germany, Ireland, France, Italy, Luxembourg, the Netherlands, Austria, the 
UK  and  Malta.  The  second  phase  of  the  transitional  arrangements  for  the  2007  enlargement 
came to an end on 31 December 2011, at which point the governments in these countries decide 
on whether or not to extend the restrictions for a further two years.  

–  The  findings  suggest  that,  due  to  network  effects,  transitional  arrangements  can  have 
permanent  effects  on  the  pattern  of  migration.  This  may  be  of  particular  importance  in  host 
countries  where  the  working  age  share  of  the  population  is  in  decline.  Countries  that  retain 
restrictions  are  expected  to  have  a  lower  level  of  potential  output  in  the  long-run  as  a  result. 

49

 
 
 
Allowing full labour market access has not been found to have significant adverse effects on the 
host economy.  

Availability of the report on Europa  

http://ec.europa.eu/social/main.jsp?langId=en&catId=89&newsId=1108&furtherNews=yes  

Direct access to main report: http://ec.europa.eu/social/BlobServlet?docId=7120&langId=en  

Direct access to country case studies: 
http://ec.europa.eu/social/BlobServlet?docId=7191&langId=en  

Direct access to executive summary: 
http://ec.europa.eu/social/BlobServlet?docId=7193&langId=en  

50

 
 
 
 
 
05 – Agriculture and Rural Development 

Evaluation of income effects of direct support  

ABB activities: 

05 02 Interventions in agricultural markets 

05 03 Direct aids 

Timing:   May 2011 

Background, scope and focus  

The  evaluation  analysed  the  effects  of  the  direct  support  schemes  laid  down  in  Council 
Regulation (EC) 1782/2003 (later Council Regulation (EC) 73/2009) on the income of farmers. 
The  effectiveness  and  efficiency  of  the  implementation  of  direct  support  with  respect  to 
achieving the objective of ensuring a fair standard of living for the agricultural community was 
also analysed, as was the coherence of direct  payments with  measures under the Single Market 
Organisation and rural development measures. 

The  evaluation  covered  the  EU-27  over  the  period  2005-2008.  The  analysis  distinguished 
between seven agricultural sectors (field crops, dairy, grazing livestock, granivores, horticulture, 
permanent crops, mixed farms), the choices of implementation of the direct payment schemes in 
different Member States/regions, different farm size classes and the types of farm organisation, 
and the geographical location. 

Summary of findings and conclusions  

The  evaluation  concluded  that  direct  payments  have  contributed  to  enhancing  the  income  of 
farmers and have played an important role in generating farm income. This is particularly true for 
grazing livestock farms (beef, sheep and goats), field crops, mixed farms and dairy farms. 

The study underlined the role of direct payments in strengthening the cohesion between regions, 
in  particular  in  the  sectors  of  field  crops,  milk,  grazing  livestock  (other  than  dairy  farms)  and 
mixed farms. It also showed that direct payments have contributed to reducing the existing gap 
between the average farm income per labour unit of small and large farms. 

The  evaluation  indicates  that  direct  payments  have  made  a  positive  and  robust  contribution  to 
the  stability  of  the  income  of  farmers,  in  particular  on  field  crop,  grazing  livestock  and  mixed 
farms.  The  stability  of  farmers'  income  was  fostered  by  direct  payments  to  a  larger  extent  on 
smaller farms in comparison to medium and large farms. 

For the purpose of the evaluation, the regional GDP/employee was used as benchmark for a fair 

51

 
 
 
 
 
 
 
 
 
standard of living of the agricultural community. The analysis showed that direct payments have 
helped reduce the existing gap between farmers' income and the regional GDP/employee. 

The  conclusions  indicated  that  direct  payments  have  been  crucial  in  ensuring  the  economic 
viability of farms, in particular those specialised in field crops, grazing livestock, mixed farming 
and, partly, dairy farming. 

The evaluation showed that in the years analysed (2004, 2006 and 2007) the efficiency of direct 
payments  in  terms  of  providing  income  support  to  farmers  was  quite  high.  Yet,  notable 
differences across regions were also indicated. 

The results of the statistical analysis pointed out that direct payments have been coherent with the 
measures  under  the  single  Common  Market  Organisation  (CMO)  and  rural  development 
measures  in  relation  to  the  objective  of  enhancing  farmers'  income  and  also  in  terms  of 
stabilising farmers' income. 

Availability of the report on Europa  

http://ec.europa.eu/agriculture/eval/reports/income/index_en.htm 

Evaluation of CAP measures applied to the sugar sector  

ABB activities: 

05 02 Interventions in agricultural markets 

05 03 Direct aids 

Timing:   December 2011 

52

 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The  evaluation  examined  the  effects  of  the  2006  sugar  reform  (Council  Regulations 
318/20061,  320/2006  and  1782/20032)  on  the  sugar  supply  chain,  including  the farm  sector 
and sugar producers, refiners and isoglucose producers. 

The evaluation covered the period from 2006 until the present time. The period between 2001 and 
2005 served as a reference. In principle, all EU Member States were covered with a focus on six 
Member States that were  analysed in  more detail:  Finland, France, Germany,  Italy, Poland  and 
UK. 

Summary of findings and conclusions  

Impacts on production and geographical distribution of the sugar beet sector. As a direct 
impact of sugar quota renunciations, the average volume of beet produced in the EU-27 in 2008-
10 was 19% lower than in 2003-05. After the reform the trend of improvements in yields 
accelerated as did the concentration of production in larger farms. These changes were different 
among Member States and led to a further geographical concentration of production in the beet 
belt3. 

Impacts on market orientation and competitiveness of sugar beet producing farmers. Due 
to  the  reduction  in  the  beet  minimum  price,  the  distortive  effect  of  CAP  support  greatly 
decreased. The evaluation confirmed that farmers'  production choices were  more linked to the 
beet price level. The competitiveness of beet production improved, mainly because the reform 
led the least efficient growers to give up beet production. 

Impacts on sugar producers·  The sugar quota renunciations reached 5.8 million tonnes, which 
is  almost  the  initially  established  target  for  the  EU  quota  reduction.  The  quotas  were  largely 
reduced  in  Member  States  outside  the  beet  belt,  and  sugar  production  was  abandoned  in  five 
Member States. The reform directly contributed to a decrease of quota sugar prices as a result of 
the reduction of the sugar  reference price. However,  since  the  end  of  2009,  the  EU  price  level 
remained  well  above  the  reference  price  because  of  exceptionally  high  world  market  sugar 
prices.  The  reform  significantly  accelerated  the  restructuring  process  of  the  sector  with 
altogether 41% of the 179 factories operating in 2005/06 closing down, mostly those with low 
and  medium  capacity.  The  evaluation  concluded that  on  average,  the  competitiveness  of  sugar 
producers improved, although this improvement was not equal among Member States. 

Impacts  on  isoglucose  producers.  Since  isoglucose  is  made  from  cereals,  the  reform  had  an 
indirect  negative  effect  on  its  relative  profitability  compared  to  the  beet  sugar  (due  to  the 

1 Later integrated into Council Regulation 1234/2007 

2 Later replaced by Council Regulation 73/2009 

3 EU sugar production is concentrated in regions considered the most competitive for sugar production. These are 
located  in  a  zone  ranging  from  the  UK,  France,  Belgium,  the  Netherlands,  Germany  to  Poland,  generally 
referred to as the "beet belt". 

53

 
 
 
                                                 
reduction of sugar beet  minimum prices). The reform therefore increased isoglucose quotas  as 
compensation,  whilst  maintaining  the  possibility  for  isoglucose  producers  to  renounce  quotas1. 
As a result of the reform, the isoglucose quota increased from 0.5 to 0.6 million tones in the EU-
25.  Half  of  the  existing  isoglucose  production  units  were  dismantled  within  the  restructuring 
scheme.  Isoglucose  production  is  now  concentrated  in  five  Member  States  (instead  of  fifteen 
before the reform). 

Impacts  on  sugar  refiners.  The  removal  of  the  strong  protection,  which  used  to  benefit  the 
traditional full-time refiners, together with the decrease in EU sugar production and the expected 
increase  in  raw  sugar  imports,  led  to  an  increase  in  the  number  of  full-time  refiners,  with  new 
refineries  in  Member  States  that  had  no  such  activity  before.  However,  after  the  transitional 
period, supplies of raw cane sugar to the EU market decreased. Indeed, the attractiveness of the 
EU market for raw sugar exports depends on the price gap between the EU and other markets. 

Social aspects. The closure of 41% of the sugar factories inevitably led to job losses. However, 
the evaluation underlined that a significant proportion of these jobs would have been lost even 
without  the  reform,  as  the  restructuring  was  an  ongoing  process  in  the  sector.  The  evaluation 
concluded  that  although  the  sugar  restructuring  scheme  intended  to  limit  negative  social 
impacts, the requirements imposed by EU legislation did not go beyond national labour market 
legislation.  Yet,  the  existence  of  restructuring  aid  received  by  the  companies  weighed  in  the 
negotiations between the companies and employees, in favour of the employees. 

Availability of the report on Europa  

http://ec.europa.eu/agriculture/eval/reports/sugar-2011/index_en.htm  

Evaluation of CAP measures applied to the dairy sector  

ABB activities: 

05 02 Interventions in agricultural markets 

05 03 Direct aids 

Timing:   November 2011 

Background, scope and focus  

1 In the EU-25, 0.3 million tonnes of quotas were distributed for free while 0.2 million tones were renounced. 

54

 
 
 
 
 
 
 
 
 
 
 
                                                                                                                                            
The evaluation of CAP measures applied to the dairy sector covered the measures of the CMO for 
dairy (later integrated into the Single CMO) and direct payment schemes as far as these schemes 
are  used  by  beneficiaries  in  the  dairy  sector.  The  evaluation  addressed  the  effect  of  these 
measures on the prices and the production of milk and the impact on dairy farmers. It also looked 
further  downstream  on  the  consequences  for  the  price  of  dairy  products,  market  balance, 
competitiveness and industry structure. The evaluation covered the EU-27 over the period since 
1 July 2004, when first cuts in intervention prices were implemented. 

Summary of findings and conclusions  

1.  Raw milk 

In the period evaluated, the domestic supply became less determined by quota ceilings and more 
responsive to milk prices, with quotas no longer being always filled for most Member States. The 
dairy  policy  contributed  to  a  falling  structural  excess  supply  (from  2004  onwards),  to  a  better 
balance  of  demand  and  supply  and  to  a  gradual  convergence  of  the  EU  prices  towards  world 
market prices during 2004-2006. 

In  2007-2009,  the  effects  of  the  CAP  measures  on  internal  prices  were  masked  by  the  price 
turbulence originating from the world market commodity boom. During 2007/08, which was a 
period of abnormally high world market prices, export refunds and intervention buying were no 
longer operational in the months when the world prices exceeded the intervention price levels. 
In 2009, on the downside of the price spike, although intervention buying-in was activated, the 
(average) EU-27 raw milk price fell significantly 

The  evaluation  showed  that  during  the  time  covered  by  the  accounting  data  available  (up  till 
2007)  trends  in  dairy  farm  income  and  profitability  of  dairying  relative  to  other  commodity 
sectors, were maintained. Market orientation improved; cost-competitiveness, however, did not 
improve. 

2.  Milk products 

EU  dairy  product  prices  remained  stable  until  2006  due  to  export  refunds  and  public 
intervention. Volatility increased after 2007 due to the commodity boom on world markets. 

Generally,  the  EU  was  not  competitive  at  world  market  prices.  During  the  evaluation  period, 
however, the price gap relative to the world market declined due to lower intervention prices for 
milk products (butter, SMP), and increasing world market prices. 

The evaluation did not allow to draw strong conclusions with respect to the impacts of policies 
on  the  structure  of  the  processing  industry.  However,  it  was  noted  that  concentration  and 
consolidation of firms increased in most Member States. 

3.  Efficiency, coherence and relevance of the policy 

Policy efficiency generally increased. The total cost of the dairy support policy declined whilst 
the market balance improved and producer income levels were maintained. 

Overall,  the  evaluation  found  a  good  degree  of  coherence  between  the  CAP  dairy  measures, 
rural development measures and state aids. Pillar 1, rural development and national aid measures 
operate at different levels and scales, giving them a complementary character. 

Milk producers' concerns over income, production flexibility and expansion were met, but at the 

55

 
 
cost  of  a  heavier  administrative  burden  and  more  exposure  to  price  risk.  Processors  now  also 
face more price and market risk. 

Society benefits from increased efficiency and a greater focus on sustainability. Consumers are 
potential  beneficiaries  of  lower  product  prices,  but  these  benefits  are  conditional  on  the 
transmission of lower milk prices along the supply chain. 

Availability of the report on Europa  

http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/dairy-sector-
2011_en.htm  

Evaluation of CAP measures applied to the sheep and goat sector  

ABB activities: 

05 02 Interventions in agricultural markets 

05 03 Direct aids 

Timing:   November 2011 

Background, scope and focus  

The evaluation examined the impacts of the direct aids implemented in the framework of Council 
Regulation  (EC)  No  1782/2003  and  of  the  market  measures  laid  down  in  Council  Regulation 
(EC) No 1234/2007 (formerly laid down in Council Regulation (EC) No 2529/2001) in the sheep 
and goat sector. 

The impacts examined concerned sheep and goat production, the supply of sheep and goat meat to 
the  processing  industry,  profitability,  competitiveness  and  market  orientation,  farmers'  income, 
efficiency, relevance and coherence. The evaluation covered the period 2005-2009. 

The  geographical  scope  was  the  EU-27.  In-depth  analyses  covered  the  main  producing  Member 
States: Spain, Italy, Greece, France, the United Kingdom, Ireland and Romania for the sheep sector, 
and Spain, France and Greece for the goat sector. 

Summary of findings and conclusions  

56

 
 
 
 
 
 
 
 
 
 
 
Sheep  livestock  decreased  especially  in  the  four  Member  States  oriented  exclusively  or 
predominantly towards meat production (UK, Ireland, France and Spain). Decoupling accelerated a 
trend existing before the 2003 CAP reform and led to a greater reduction of livestock in economically 
vulnerable areas (e.g. Scottish Highlands and Islands, Epirus in Greece). The sector was negatively 
affected  by  external  factors  (rising  fodder  prices,  epizootic outbreaks). The payments granted in 
Italy and Greece in the framework of Article 69 of Council Regulation (EC) 1782/2003 were low and, 
therefore, had little impact. 

The milk sector was neither affected by the change in the CAP measures nor by the negative external 
factors  which  influenced  the  meat  sector.  The  number  of  milking  ewes  remained  stable  in  Italy, 
Greece, France, Spain, and grew in Romania. 

The number of goats tended to decrease in Greece and Spain and grew in France. The drivers 
were not policy-related; they were linked to the market. 

Meat production decreased in the evaluation period in the analysed Member States (with the exception 
of  Romania),  following  the  diminishing  number  of  sheep.  After  the  implementation  of  the  2003 
CAP reform, the decreasing trend accelerated. 

The  quantity  of  sheep  and  goat  milk  used  for  processing  increased  during  the  evaluation  period 
under the influence of market-related factors. 

In the sheep meat sector, the average profitability contracted or remained stable (Ireland) in the main 
meat producing Member States. The reform did not prevent the profitability of sheep meat farms 
from deteriorating. 

In the sheep milk sector, in all five Member States analysed, price competitiveness and profitability 
were negatively affected by increased production costs, especially for fodder. 

In the goat sector, profitability improved due to milk price increases, despite the significant increase 
in production costs. These trends have no link to the reform. 

In  all  Member  States  and  sectors  studied,  direct  aids  contributed  to  increasing,  maintaining  or 
limiting the decline in farmers' income. In the sheep meat sector, the support offered through the 
rural development programmes is considered critical to maintaining the sector. 

The  administrative  burden  decreased  with  decoupling;  however,  cross-compliance  increased  the 
control workload. 

Electronic identification was introduced in the sheep and goat sector in order to help prevent and/or 
quickly contain epizootics. However, its effects vary widely between Member States due to different 
conditions of implementation. 

Promotional  measures  contributed  to  the  improvement  of  the  competitiveness,  to  increasing 
consumption and to a better marketing of farmers' products. 

The  reform  did  not  contribute  to  improving  the  competitiveness  of  the  sheep  and  goat  sector.  The 
adaptations that were implemented by some farmers (depending on Member State and production 
type:  extensification,  concentration,  improved  feed  self-sufficiency)  were  facilitated  by 
decoupling,  but  did  not  offset  the  effects  of  external  factors  (epizootics  and  fodder  costs)  on 
production costs, which have increased in most cases. 

Availability of the report on Europa  

57

 
 
 
http://ec.europa.eu/agriculture/evaluation/index_en.htm  

58

 
 
 
 
Evaluation of promotion and information actions for agricultural products  

ABB activities: 

05 02 Interventions in agricultural markets 

Timing:   November 2011 

Background, scope and focus  

The evaluation examined the relevance and effectiveness of the EU information and promotion 
policy  for  agricultural  products1  with  respect  to  achieving  the  objectives  laid  down  in  Council 
Regulation  3/2008,  as  well  as  its  coherence  with  other  promotion  measures  applied  under  the 
CAP,  notably  in  the  fruit  and  vegetable  and  the  wine  sectors  and  under  the  rural  development 
policy. 

The examination covered the period between 2002 and 2010. In principle, all EU Member States 
benefitting from EU co-financed promotion programmes were covered. The Member States that 
were the main beneficiaries of promotion programmes and the Member States which took part in 
multi-country promotion programmes, were examined in more depth2. 

Summary of findings and conclusions  

The evaluation concluded that while the scope of the products and themes, and the list of third 
countries eligible for the scheme are adequate for achieving the objectives of the policy, there is 
a lack of prioritisation and strategy behind them. 

The  themes  of  the  Commission  initiatives3  organised  within  the  examination  period  (organic 
farming campaign and campaign to promote European quality schemes in third countries) were 
judged to be of Community interest and corresponded to the main priorities of the Regulation.  

1 The main objectives of EU information and promotion scheme is to improve the image and raise awareness of 
the  quality  and  the  specific  production  methods  of  EU  agricultural  products  in  order  to  reverse  their  declining 
consumption, expand the demand or open new markets, depending on the case. 
2 France, Germany, Greece, Italy, the Netherlands, Poland, Portugal, Spain, UK. 

3  The  Commission  may  launch  initiatives  on  themes  and  products  that  are  of  Community  interest  where 
appropriate proposal has been submitted by proposing organisations. 

59

 
 
 
 
 
 
 
 
 
 
                                                 
Although  the  EU  promotion  policy  framework  has  encouraged  multi-country  programmes  and 
multi-product programmes, these programmes have not been widespread so far. The number of 
multi-country and multi-product programme proposals was limited by the specific difficulties in 
their design and implementation. 

As regards the evolution of the effectiveness of promotion programmes, there was widespread 
use  of  good  practices  amongst  proposing  organisations.  Market  analysis,  SMART  objectives, 
clear  targets,  well  adapted  actions  and  good  monitoring  methods  were  widely  supported  and 
adopted  practices  among  the  proposing  organisations.  This  was  reflected  in  an  improving 
standard  of  the  selected  proposals  and  it  seems  that  the  performance  of  programmes  was 
improving, as a result of good practice, leading to better attainment of the objectives. 

Concerning  management  aspects,  the  evaluation  revealed  that  while  the  overall  framework  for 
managing  promotion  programmes  is  clearly  defined,  the  two-step  selection  process  and  the 
implementation  procedures  affected  the  management  of  the  policy,  in  particular  in  the  case  of 
multi-country programmes.  

The evaluators concluded that EU promotion programmes have an important leverage effect on 
generic,  multi-country  and  multi-product  promotion  programmes.  Without  EU  co-funding, 
promotion  programmes  would  have  a  much  smaller  scale  and  would  be  funded  mainly  by  the 
private sector, and would thus be brand-oriented and not generic.  

The  evaluation  examined  the  coherence  of  the  information  and  promotion  actions  with  other 
promotion  measures  applied  under  the  CAP,  in  particular  measures  applied  in  the  wine  sector 
and  in  the  fruit  and  vegetable  sector,  as  well  as  measures  under  the  rural  development  policy. 
The promotion and information scheme is coherent with these other CAP measures in terms of 
objectives, messages conveyed, groups targeted and channels used. However, it was noted that 
the  coexistence  of  an  increased  number  of  promotion  measures  resulting  from  recent  CAP 
reforms may limit the overall efficiency of the EU promotion policy. 

Availability of the report on Europa  

http://ec.europa.eu/agriculture/eval/reports/promotion/index_en.htm   

Evaluation of the exceptional market support measures in the poultry and egg sector  

ABB activities: 

05 02 Interventions in agricultural markets 

Timing:   November 2011 

60

 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The evaluation of the exceptional market support measures for poultry and eggs dealt with the 
seven measures taken under Commission Regulation (EC) No 1010/2006 in 2006 to tackle the 
strong market disturbances, such as plummeting demand and prices, caused by a lack of consumer 
confidence  due  to  Avian  Flu.  These  disturbances  occurred  despite  hardly  any  outbreaks  of  the 
disease among domestic birds. This was the first experience of use of exceptional measures in 
response to a perceived lack of EU consumer confidence. 

The  evaluation  covered  the  period  from  autumn  2005  to  spring  2007.  It  included  all  Member 
States  that  requested  payments  under  the  scheme.  Case  studies  were  carried  out  for  the  main 
recipients of the scheme: Greece, Spain, France, Italy, Hungary, the Netherlands and Poland. 

Summary of findings and conclusions  

The  evaluation  concluded  that  the  common  approach  at  the  EU  level  where  Member  States 
could choose from a menu of seven measures, created a European added value and proved to be 
adequate as a response. 

The  long  preparation  time  of  the  measures  -  a  legal  base  had  first  to  be  created  which  took 
several months - prevented that the measures had a direct influence on the short-run stabilization 
of  the  market.  However,  already  the  announcement  of  the  measures  may  have  had  an  indirect 
stabilizing impact via market expectations. 

The measures were certainly effective with respect to covering a substantial part of the severe 
income  losses  that  occurred  in  the  sector,  with  farmers  in  non-integrated  systems  considerably 
more  impacted  than  those  in  integrated  systems.  The  measures  were  coherent  with  the  actions 
taken by the industry in the Member States. 

According to the evaluation, the exceptional measures were coherent with the actions taken by 
the industry, and helped maintain the Internal Market by avoiding national interventions. 

The exceptional measures were also coherent with export refunds. While export refunds targeted 
existing  surplus  production,  the  exceptional  measures  were  used  to  prevent  future  production 
adding to this surplus. 

The  evaluation  provided  also  the  conclusion  that  in  case  of  consumer  confidence  collapse  the 
Commission should have in place instruments, for this and other agricultural sectors, that can be 
used swiftly without delay to increase the stabilizing effect on the market. 

Furthermore the evaluation concluded that the availability of data to assess market disturbances 
during a crisis and to adequately monitor and evaluate the measures, should be reviewed. 

Availability of the report on Europa  

http://ec.europa.eu/agriculture/eval/reports/poultry-eggs/index_en.htm  

61

 
 
 
 
06 – Mobility and Transport  

Evaluation of Regulation 881/2004 establishing the European Railway Agency 

ABB activities: 

06 02 Inland, air and maritime transport 

Timing:   April 2011 

Background, scope and focus  

Regulation (EC) No 881/2004 of 29 April 20041 as subsequently amended by Regulation (EC) 
No 1335/2008 of 16 December 20082 ("the Regulation") specified the creation of the European 
Railway  Agency  ("the  Agency",  "ERA").  Established  in  2005  in  Valenciennes  (France),  ERA 
helps  to  build  an  integrated  European  railway  area  by  reinforcing  rail  safety  and  promoting 
interoperability.  It  develops  common  technical  standards  and  common  approaches  to  safety, 
working  closely  with  stakeholders  from  the  rail sector  as  well  as  with  national  authorities,  the 
EU  institutions  and  other  interested  parties.  Featuring  a  dedicated  Safety  Unit,  ERA  also 
monitors and reports on rail safety in the EU. 

Article  43  of  the  Regulation  provides  for  an  evaluation  of  the  implementation  of  the 
Regulation five years after the Agency takes up its duties. 

The main objectives of the evaluation study were: 

to assess whether and how the main objectives of the Regulation were effectively 

– 
attained (implementation of the Regulation), 

to assess how the instruments and procedures provided by the Regulation have been 

– 
implemented (the functioning of the Agency), 

–  and to identify areas of improvement (including possible future roles of the Agency). 

Summary of findings and conclusions  

The main findings of the evaluation study are the following: 

1.  Positive  steps  towards  the  completion  of  the  initial  objectives  of  the  Regulation,  however 
many of them are likely to continue to remain valid for the longer term. 

2.  The  Agency  has  made  reasonable  progress  as  regards  its  contribution  to  the  fulfillment  of 

1 OJ L 164, 30.4.2004, p. 1–43 

2 OJ L 354, 31.12.2008, p. 51-59 

62

 
 
 
 
 
 
 
 
 
 
                                                 
policy  objectives  (they  relate  to  the  overall  effectiveness  of  the  railway  regime  for  which  the 
Agency has only a contributory role). 

3.  The  effectiveness  of  the  instruments  laid  down  by  the  Regulation  for  the  fulfillment  of  the 
objectives  (such  as  registers  and  databases)  was  assessed  as  useful  to  stakeholders,  but  use  or 
awareness of some of them seems limited. 

4.  The costs incurred for the fulfillment of the objectives are broadly in line with comparable 
EU agencies. 

5.  The  performance  of  the  Agency  is  overall  satisfactory;  however,  there  are  areas  of 
opportunity for improvement, like introduction of key performance indicators or increasing the 
effectiveness of the working parties. 

6.  The  effectiveness  and  functioning  of  the  Agency  and  its  working  practices  as  regards  the 
structures  created  by  the Regulation  is  overall  assessed  positively,  nevertheless  there  are  areas 
where the functioning of the Agency could be improved (such as quality control, more effective 
governance structure, greater flexibility regarding staff contracts). 

7.  As  regards  potential  future  roles  of  the  Agency,  provision  of  advice  and  support, 
dissemination  of  railway-related  information  and  training  and  involvement  in  international 
cooperation and promotion of EU standards were identified as the most supported. 

Availability of the report on Europa  

http://ec.europa.eu/transport/evaluations/doc/2011_era-evaluation-881-2004.pdf 

Evaluation of the Marco Polo Programme 

ABB activities: 

06 02 Inland, air and maritime transport 

Timing:  

 18/04/2011  

Background, scope and focus  

The evaluation of the programme was foreseen in the Regulation (EC) No 1692/2006 of the European 
Parliament  and  of  the  Council,  establishing  the  second  'Marco  Polo  programme  for  the  granting  of 
Community  financial  assistance  to  improve  the  environmental  performance  of  the  freight  transport 
system ('Marco Polo 11) (OJ L 328, 24.11.2006). As required by the abovementioned Regulation, the 
evaluation covers the period 2003 - 2010 and represents both the ex-post evaluation of the Marco Polo I 
programme  and  the  mid-term  evaluation  of  the  Marco  Polo  Π  programme.  The  evaluation  has  been 
commissioned  in  order  to  provide  information  on  the  results  delivered  by  the  programme  and  to 

63

 
 
 
 
 
 
 
 
 
 
 
contribute to the development of new funding and policy instruments supporting transport services. 

Summary of findings and conclusions  

1.  Marco Polo was conceived as a key programme for promoting sustainable transport. It 
supports the shift of freight traffic from the road to other modes of transport, such as sea, rail and 
inland waterways. There have been two Marco Polo programmes: Marco Polo I with the budget 
of €102 million which ran between 2003 and 2006 and the ongoing Marco Polo II, established 
for the period 2007 – 2013 with a substantially increased budget of €450 million and extended 
scope. 

2.  The most important features of the programme are its transparency and high accuracy with 
which the achieved results can be measured and quantified as well as the direct relationship 
between the EU funding and the results. 

3.  The outsourced management of the programme has strengthened its implementation and 
allowed the Commission to concentrate on the key policy issues. 

4.  While Marco Polo I has realised a significant modal shift (21.4 billion of tonnes-kilometre 
shifted off the road), which can be quantified into substantial environmental benefits expressed 
in hundreds of million euro, the ambitious objectives established for the programme before 
coming into being, have not been fully achieved (45% for Marco Polo I)1. 

5.  A significant modal shift is also forecasted for Marco Polo II, but as the programme is still 
on-going, the final results are not yet available. 

6.  The economic recession, which was reported over the period 2008 – 2009, may have had an 
adverse impact on the success rate of projects financed under Marco Polo I and Marco Polo II 
programmes. 

7.  There is a budgetary underperformance of the programme both in terms of commitments and 
payments. 

8.  Majority of projects continue after the funding period, which demonstrates that a short term 
funding may lead to a long term change. 

9.  There is likely some deadweight in the programme, in the meaning that a certain proportion 
of supported projects would have gone ahead in the absence of Marco Polo funds, whereby MP 
allowed either to accelerate the set-up or to increase the scale foreseen.  

10.  Some concerns have been expressed about the possible distortion of competition in the 
maritime sector as a result of Marco Polo funding; however there has been no evidence about 
any significant adverse effects on competition in the maritime transport.   

11.  Marco Polo programme is currently the only financial instrument at the EU level which aims 
at improving environmental efficiency of freight transport along with reducing greenhouse gas 
emissions from freight transport and reducing congestion on European road networks. Therefore, 
in the next financial framework funding should also be provided to support sustainable freight 
services while consideration need to be taken of whether modal shift shall remain the main criterion 
of support.  

1 Results achieved at the end of 2010 

64

 
 
 
 
 
                                                 
Availability of the report on Europa  

http://ec.europa.eu/transport/evaluations/doc/2011_marco-polo-programme-2003-2010.pdf 

Ex post evaluation on EU Passenger Ship Safety Legislation 

ABB activities: 

06 02 Inland, air and maritime transport 

Timing:  

 April 2011 

Background, scope and focus  

This  set  of  legislative  measures  was  formulated  essentially  in  response  to  the  Estonia  ferry 
disaster  in  1994  and  the  technical  basis  for  the  legislation  is  over  10  years  old.  The  review 
included identifying, in consultation with Member State experts, what areas should be addressed 
further. In parallel, it was considered prudent to undertake a separate external ex-post evaluation 
of the EU passenger ship safety legislative framework. The Directives concerned were: 

–  Directive  2009/45/EC  on  safety  rules  and  standards  for  passenger  ships  (a  codification  of 
Directive 98/18/EC and its amendments); 

–  Directive 98/41/EC on the registration of persons on board; 

–  Directive  99/3  5/EC  on  mandatory  surveys  for  ro-ro  ferry  and  high-speed  passenger  craft 
services; and 

–  Directive  2003/25/EC  on  specific  stability  rules  for  ro-ro  passenger  ships  (making  the 
intergovernmental 1996 Stockholm Agreement regulations applicable throughout the Union). 

The main objectives of the study were (1) to identify any aspects of the current regulations that 
were not efficient, effective, useful or relevant; and (2) to identify areas where action at EU level 
could address potential safety gaps and related obstacles to the internal market. 

Summary of findings and conclusions  

As they stand, the Directives are considered to be relevant, efficient, effective and have utility. 
The aim pursued by the Directives is achieved at a reasonable cost.  

There is a significant consensus amongst stakeholders that amendments to Directives 009/45/EC 
and Directive 99/35/EC are necessary.  

65

 
 
 
 
 
 
 
 
 
 
 
Directive 2009/45/EC: 

1.  The scope of the Directive should be expanded to cover ships made of material other than 
steel; historic/traditional ships; and sailing ships; 

2.  A more tailored approach should be taken to regulating safety standards for small ships; 

3.  Clarification  of  certain  definitions  (port  areas;  High  Speed  Passenger  Craft  (HSPC));  and 
guidelines for carrying out annual surveys were necessary; and 

4.  A Member State reporting obligation on the implementation of the Directive would ensure 
an on-going overview. 

Directive 99/35/EC: 

1.  A reduction in the number of standard surveys; and 

2.  Harmonisation of procedures with Directive 2009/16/EC on port state Control. 

The following constitute the main recommendations of the evaluation report. 

Directive 2009/45/EC 

1.  Extension  of  scope  to  include  ships  made  of  material  other  than  steel  –  adapting  the 
Directive to market trends and technological improvements; 

2.  Adoption of well-targeted standards for small ships of below 24 m, perhaps considering an 
application to below 12 passengers; 

3.  Elaboration  of  a  comprehensive  solution  to  address  the  needs  of  the  niche  market  for 
transport of passengers by historic/traditional and sailing ships; 

4.  Elaboration  of  new  standards  (adopting  and  adapting  those  to  be  made  at  IMO  level)  for 
particular passenger ships: cruise tenders; offshore workers (e.g., to service windmill farms); 

5.  Incorporation of the IMO Polar Code, once agreed, into EU acquis for domestic voyages; 

6.  Consideration of an extension of the current regulatory framework to intra-EU traffic; 

7.  Publication of guidelines on: the meaning of 'port area'; definition of HSPC; survey work, in 
particular timeframes and coordination amongst a Member State's competent authorities; and 

8.  Biennial implementation report requirement. 

Directive 99/35/EC 

1.  Inclusion of provisions to increase unscheduled 'surprise' inspections and to reduce standard 
inspections; 

2.  Harmonisation of inspections with those in Directive 2009/16/EC; and 

3.  Biennial implementation report requirement, along with better use of current data uploaded 
to EMSA's central database by all EU competent authorities. 

Availability of the report on Europa  

66

 
 
 
 
 
http://ec.europa.eu/transport/evaluations/doc/2011_passenger_ship_safety.pdf 

Mid-term evaluation of TEN-T programme 

ABB activities: 

06 03 Trans-European networks 

Timing:  

 March 2011 

Background, scope and focus  

In order to give financial support to the implementation of the TEN-T guidelines, the Parliament 
and  Council  adopted  Regulation  No  680/2007/EC  specifying  general  rules  for  the  granting  of 
Community  aid  in  the  field  of  trans-European  networks.1  The  Regulation  required  a  mid-term 
review of the TEN-Т Programme by end 2010. 

The  mid-term  evaluation  aimed  at  assessing  the  relevance  and  effectiveness  of  the  TEN-Т 
Programme (2007- 2013) in the context of the Trans-European Transport Network.  The period 
evaluated was 2007-2010. The main objective was to formulate overall conclusions and possible 
recommendations  on  the  implementation  of  the  TEN-Т  Programme  with  a  view  to  providing 
input to the revision of TEN-Т Programme and policy. 

Summary of findings and conclusions  

The main findings of the mid-term evaluation study are the followings: 

Since the start of the current financial perspective (2007-2013), the Programme governance had 
improved: the TEN-Т Executive Agency is providing more control over the public money that is 
spent, the selection of projects through proposal calls is more rigorous and leads to better project 
delivery.  More  than  90%  of  the  Programme  funds  have  been  allocated  and  where  the  earliest 
projects since 2007 did not perform as required the funds have already been reallocated. 

However the Programme is behind schedule on completion: a significant number of the largest 
projects  in  the  Multi-Annual  Programme  will  be  completed  after  2013,  by  2015.  The  projects 
that have been completed to date tend to be projects of common interest because they are shorter 
and because they are less complex than the Priority Projects. 

A number of the recent EERP (European Economic Recovery Programme) projects are already 

1 OJ L 162/1,  22 June 2007, pages 1-10 

67

 
 
 
 
 
 
 
 
 
 
                                                 
late whereas they had been specifically selected to be completed over a short period. This will 
mean that there is little chance that the TEN-Т network can be fully operational by 2020. 

The Priority Projects, the dorsal spine of the network, are not delivering the expected effects. A 
few  Priority  Projects  are  completed  and  numerous  sections  are  finalised  but  some  key  parts  -
such as cross-border sections - are missing and explain why the TEN-T network is an assembly 
of  largely  national  sections,  often  poorly  interlinked,  rather  than  a  proper  physical  and 
interoperable network. 

Most  Priority  Projects  focus  on  rail:  eighteen  address  rail  and  two  address  inland  waterways, 
without achieving a coherent network. In spite of the focus given to rail, these projects have not 
resulted  in  a  Single  European  Railway  Areal  and  are  still  experiencing  bottlenecks  and 
significant interoperable obstacles.  

The  ex-post  and  mid-term  review  reports  conclude  therefore  that  there  is  a  sub-optimal, 
economic,  social  and  territorial  cohesion,  sub-optimal  functioning  of  the  internal  market  and 
sub-optimal use of infrastructure and resources. 

Horizontal  Projects  contribute  to  the  removal  of  "soft"  but  nonetheless  real  barriers  and 
bottlenecks  and  therefore  contribute  to  the  achievement  of  the  objectives  of  the  Programme. 
Compared to "hard" infrastructure projects, the time requirement for interoperability effects can 
be shorter, however changes to behaviour and standardisation of tools or vehicles takes time and 
this needs to be recognised by the Programme. 

Although there are many examples of what bottlenecks are on the European transport network, 
there  is  still  no  clear  definition  of  what  they  are:  a  physical  lack  of  infrastructure,  a  lack  of 
common operating procedures, or a lack of smooth operations at the borders? This needs to be 
addressed in a revision of the Guidelines. 

The objectives of the Programme are so general that it makes any evaluation of the Programme 
successes difficult. The Programme has been the catalyst to a number of key pieces of transport 
infrastructure in Europe, and has been playing a part in the structuring of the transport network 
by allowing transport investments to be focussed. Its political leverage is high but its financial 
leverage is poor. The Programme has clearly made a positive contribution to the mobility needs 
of  the  European  citizens  and  goods.  However  some  aspects  of  the  Programme  need  to  be 
improved  which  requires  a  revision  of  the  Guidelines  and  Regulation  and  of  some  internal 
aspects of the Programme practices. 

Availability of the report on Europa  

http://ec.europa.eu/transport/evaluations/doc/2011_ten_t.pdf 

68

 
 
 
 
07 – Environment  

Mid-term Evaluation of the Sustainable Consumption and Production and 
Sustainable Industrial Policy Action Plan 

ABB activities: 

07 03 Development and implementation of EU 
environmental policy and legislation 

Timing:  

 September 2011 

Background, scope and focus  

The objective of this evaluation is to provide the Commission with key findings and lessons of 
experience  from  the  implementation  of  the  Commission’s  Sustainable  Consumption  and 
Production  and  Sustainable  Industrial  Policy  Action  Plan.  The  evaluation  will  present  those 
findings  and  lessons,  along  with  a  set  of  detailed  recommendations,  in  a  report  designed 
primarily to provide the Commission’s policymakers and managers with a valuable aid both for 
the  on-going  implementation  of  the  current  Action  Plan  and  its  review  due  in  2012.  The 
evaluation  will  also  assess  whether  the  evolution of  the  general  environmental,  economic,  and 
social context is being properly addressed, and if the element included in the Action Plan are still 
relevant.  

Summary of findings and conclusions  

Strength of rationale has grown and detailed prioritisation remains a subject of (helpful) debate  

There  is  a  clear  and  increasing  rationale  for  action  relating  to  sustainable  consumption  and 
production  and  the  instruments  grouped  under  the  AP  do  address  the  majority  of  the  issues 
involved. There is some inevitable (given the breadth of issues) discussion of the prioritisation 
between  the  objectives,  but  apart  from  an  under  representation  of  social  and  possibly 
competitiveness issues the balance appears satisfactory. 

The  AP  has  made  a  positive  start  on  improving  Commission  policy  making  but  there  remains 
more to be done 

The AP has improved coordination between the DGs most involved in SCP/SIP, this is a very 
useful and positive outcome but these efforts need to be maintained and potentially widened to 
include other DGs. There are some instruments (see recommendations for examples) outside the 
AP which could usefully be more formally aligned with it.  

Striving for consistency of approach between instruments is sound but achieving this in practice 

69

 
 
 
 
 
 
 
 
 
 
is proving difficult 

The objectives of the AP related to improving cooperation and alignment between EU and MS 
instruments,  e.g.  between  Ecolabel  and  MS  level  labelling  schemes  are  recognised  as  worthy, 
with  some,  albeit  limited,  progress  being  achieved.  This  limited  progress  relates  to  a 
combination of the complexity of the compromises involved and possibly a lack of resourcing. 

The AP has helped present a coherent policy picture but is somewhat stuck in Brussels  

Some  MSs  have  been  helped  by  both  the  high  level  statement  of  objectives  (and  collation  of 
instruments)  that  the  AP  offers  and  there  are  some  good  examples  of  instrument  specific 
progress in MSs. However the AP is  more influential on Commission policy making than MS 
policy making. 

The AP and its instruments lack targets and data, making evaluation difficult  

The lack of firm targets for the AP and for most of the instruments it covers is a weakness. This, 
combined  with  a  lack  of  data,  has  made  quantified  evaluation  difficult.  Monitoring  requires  a 
combination of high level indicators and instrument specific outputs. 

The breadth of issues involved in this area make it a true cross DG activity  

Overall  control  of  the  AP  and  its  instruments  under  one  DG  would  simplify  implementation, 
however the fundamental issues involved are genuinely cross DG so this approach (which would 
inevitably exclude the non lead DGs to a certain extent) would not be recommended. 

Satisfactory progress in adding value  

The AP has helped its constituent instruments become more than the sum of their parts, but there 
remains significant scope (and need) for more 

Faster progress is needed and there are opportunities to address this  

From a review of high level indicators, it appears that the rate of improvement in a selection of 
key issues is slow and mixed between MSs (though the most recent data only covers part of the 
AP's life). Therefore there is a case for more ambitious policy instruments to increase the rate of 
positive  change  in  this  area.  The  development  of  the  Resource  Efficiency  initiative  offers  the 
opportunity to address this. 

Availability of the report on Europa  

http://ec.europa.eu/environment/eussd/pdf/14.%20SCP-SIP%20AP%20Mid%20Term%20-
%20Final%20Report.pdf 

70

 
 
 
 
08 – Research  

Assessing the Effectiveness of Simplification Measures  
under FP7 

Evaluation related study

ABB activities: 

08 AWBL 02 Policy strategy and coordination 

Timing:  

 08/2011 

Background, scope and focus  

The overall objective of this study was to provide a well-founded and robust assessment of the 
effectiveness of the simplification measures and initiatives implemented under FP7. As such the 
study is part of the work contributing to FP7 evaluation exercises. The study was also part of the 
evidence base for the Ex-ante Impact Assessment of the Rules for Participation for the Horizon 
2020 Programme. 

Summary of findings and conclusions  

The  study  concludes  that stakeholders  -  while  understanding  that  a  programme  such  as  FP7  is 
complex  per  se  -  still  see  significant  room  for  improvement.  Simplification  remains  a  key 
challenge  in  their  minds.  According  to  stakeholders,  the  most  successful  FP7  simplification 
measures introduced under FP7 are: 

–  The introduction of the Unique Registration Facility (URF); 

–  A major reduction in the number of certificates related to financial statements that must be 
provided with periodic claims; 

–  A considerable reduction in ex-ante controls and revised protective measures for financially 
weaker participants; 

–  The extension of lump sum financing for subsistence and accommodation costs. 

Measures that were considered as potentially important by stakeholders, but are not perceived as 
having been successfully implemented include:  

–  The introduction of the possibility of ex-ante certification of the accounting methodology for 
recurring participants; 

–  A  clearer  definition  of  eligible  costs,  and  improvements  to  the  services  and  guidance 
documents for applicants; 

–  A simpler cost reporting system; 

71

 
 
 
 
 
 
 
 
 
–  A simplified support rate per type of activity. 

As regards the administrative burden (time spent on scientific tasks not included) and according 
to  the  study  results,  the  most  time-consuming  project  life  cycle  step  for  participants  is  project 
management, 
followed  by  application/selection,  negotiation  and  audit.  The  absolute 
administrative  effort  depends  on  a  number  of  factors  including  participants'  experience  with 
Framework  Programmes,  participants'  size  and  profile,  participants'  internal  organisation,  the 
degree of centralisation of tasks by project coordinators, and the project size. 

In  addition  to  these  findings,  the  study  identified  a  number  of  simplification  areas  meriting 
attention  by  the  Commission  and  provides  corresponding  recommendations,  for  instance  to 
make better use of communication as a powerful simplification tool. The study also recommends 
to monitor the simplification effects continuously and to measure simplification impacts. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/pdf/archive/fp7-evidence-
base/evaluation_studies_and_reports/simplification_study_report.pdf 

Long-Term Impact of the Framework Programmes 

Evaluation related study

ABB activities: 

08 AWBL 02 Policy strategy and coordination 

Timing:  

 09/2011 

Background, scope and focus  

This  study  was  intended  as  an  initial  exploration  of  the  long-term  impacts  of  the  Framework 
Programmes  through  a  number  of  six  selected  areas,  and  of  the  usefulness  of  coupling 
scientometric techniques with a case-based approach to impact. 

Summary of findings and conclusions  

1.  In Quantum Information Processing Computing, the Framework Programme picked up the 
emergence  of  a  new  field  of  science  and  technology,  helped  it  establish  scientific  and 

72

 
 
 
 
 
 
 
 
 
 
 
technological agendas, organise and grow in Europe to such an extent that the EU appears fully 
competitive  with  the  other  world  R&D  leaders.  Europe  has  the  technological  basis  and  has 
started to develop standards for doing so and for continuing to maintain strong positions in the 
global computing and communications industries. 

2.  In  brain  research,  the  FP  has  made  important  contributions  in  imaging  and  helped  support 
and integrate the European research community. The FP has been important in keeping Europe 
‘in the game’ in this field. 

3.  In Stratospheric Ozone research, the Framework Programme has made a major contribution 
by  growing  and  helping  coordinate  the  European  research  community,  not  least  through 
organizing multinational research campaigns to provide a better evidence base for policy. It has 
helped the European research community move from lagging far behind the USA to working at 
the  global  frontier.  Research  results  have  shaped  the  evolving  Montreal  Protocol  requirements 
and  have  been  so  influential  at  the  policy  level  that  Europe  has  achieved  the  Protocol’s  2020 
targets ten years ahead of schedule. 

4.  In  Solar  Photovoltaics  (PV),  the  Framework  Programme  has  expanded  the  European 
research community and enabled it to work at the technological frontier – not only in first- but 
also  in  second-  and  third  generation  Solar  PV.  Demonstration  projects  and  complementary 
renewable energy policies have helped develop markets for Solar PV and establish a significant 
European presence in the supply industry. 

5.  In  automotive,  the  Framework  Programme’s  role  has  been  to  sustain  longer-term  research 
and  research  in  areas  such  as  fuel  efficiency,  emissions  and  safety  that  create  not  only  private 
advantages for the industry but significant public goods. Exploiting the industry’s desire to self-
organise  to  define  R&D  directions  and  road  maps  has  been  a  powerful  way  to  coordinate  the 
longer-term R&D effort and has supported a long series of product and process innovations that 
help maintain Europe’s position among the global leaders in this industry. 

6.  The  Manufuture  Technology  Platform  is  of  interest  more  for  its  potential  than  for  any 
socioeconomic impacts achieved at this point. It underlines the importance of coordination and 
self-organisation as mechanisms to integrate research.  

7.  The most important commonality among these stories is to do with coordination by enabling 
self-organisation. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/long_t
erm_impact_of_the_fp.pdf#view=fit&pagemode=none  

73

 
 
 
 
 
 
Impact Assessment of the Regions of Knowledge Programme 

ABB activities: 

08 14 Capacities - Regions of Knowledge 

Timing:  

 04/2011 

Background, scope and focus  

The  study  aimed  to  evaluate  the  impact  of  the  "Regions  of  Knowledge"  (RoK)  programme 
regarding: 

-  Transnational cooperation of clusters 

-  Regional growth and competitiveness 

-  Regional  investment  in  research  and  innovation,  incl.  potential  synergies  with 
Structural  Funds,  Competitiveness  and  Innovation  Programme  and  any  other  source  of 
funding 

- 

Inclusion of more regions into the ERA. 

Summary of findings and conclusions  

1.  A targeted programme playing a non-negligible role in the smart specialisation of regions  

-  Most projects focus on areas of regional strategic importance 

-  Development of sectoral’s regional innovation strategies through the couple ‘state of the art 
analyses / Strategic Research Agendas + Joint Action Plans in regions which have often already 
carried out ‘generalist’ regional innovation strategies.  

2.  Outcomes are to a large extent ‘intangible’ including: 

 Improvement of clusters’ strategic management, enhancement of expertise and competence 

- 
in regional authorities, strengthening of collaboration within and between the clusters 

3.  The  five  most  potential  impacts  of  the  participation  to  a  "Regions  of  Knowledge"  project 
according to the survey are: 

-  An improved strategic vision in the cluster area; 

-  The formation of new, long-term relationships with clusters at the EU level; 

-  More knowledge transfer between research organisations and enterprises; 

-  Access to a pool of complementary competencies; 

74

 
 
 
 
 
 
 
 
 
 
- 

Improvement in the strategy making process.  

4.  "Regions  of  Knowledge"  sets  the  fundaments  for  future  impacts  to  occur  in  terms  of  an 
enhanced regional economic competitiveness through R&D activities. 

5.  Balanced partnerships in terms of ‘triple helix’ and newly established regional linkages 

-  Public  bodies  are  the  main  partners  followed  by  private  partners  and  universities  and 
research organisations.  

-  Long-lasting  effects  esp.  in  terms  of  inter-regional  collaboration  but  also  strengthening  of 
intra-regional links & communication channels between the triple helix  

-  Progressive up scaling of the programme over the years to increase the level of cooperation 
between regional research-driven clusters  

-  Fundaments for gaining a critical mass & the potential for international competitiveness 

-  A facilitator was the mentoring dimension of the RoK programme 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/assess
ment_of_the_impact_of_the_regions_of_knowledge_programme.pdf 

Review of S&T Cooperation between the European Union and the Republic of 
Argentina (2006-2010) 

Evaluation related study

ABB activities: 

08 AWBL-02 Policy strategy and coordination 

Timing:  

 05/2011 

Background, scope and focus  

1.  Analyse  the  S&T  cooperative  activities  2006-2010  under  the  EU  Research  Framework 
Programmes (FP) so as to draw up a pattern both in terms of areas/topics and types of research 
(science led, technology led, trade led, global issues led), if possible; 

2.  Identify success factors as well as bottlenecks and obstacles for on-going activities or their 
further development; 

75

 
 
 
 
 
 
 
 
 
 
 
3.  Broadly review EU Member States' bilateral activities with Argentina; 

4.  Assess complementarities/synergies and overlaps between different EU-Argentina activities; 

5.  Analyse the extent to which the EU-Argentina S&T cooperation is mutually beneficial; 

6.  Identify  S&T  areas/actors/instruments  for  which  there  are  promising  prospects  for 
developing the cooperation.  

Summary of findings and conclusions  

–  The review identified a number of useful opportunities for future cooperation, e.g. through: 

Increased  EU  focus  on  large  and  complex,  autonomously  managed  and  adapted 

- 
‘projects’ such as ETPs, JTIs, JPIs, and the European Innovation Partnerships. 

-  Converging of EU-RTDI- and EU-LAC-RTDI-cooperation policies in terms of priorities and 
approaches 

- 

Increased EU focus on leveraging national funds (of the Member States)  

- 
Increased EU efforts to better use resources through more coherence and synergies between 
different  programmes,  and  through  better  priority  setting,  helped  by  the  EU  Delegation  in 
Buenos Aires. 

-  ABEST, the EU-AR Liaison Office, is actively involved in political work and dissemination 
activities,  e.g.  via  10  National  Contact  Points  (NCP)  and  an  NCP  network  throughout  the 
country. 

-  A growing interest of Europe for Latin American countries in general as exemplified e.g. by 
an  increased  propensity  of  students  and  young  researchers  to  be  internationally  mobile,  and 
ability  to  host  young  European  researchers  having  difficulties  to  get  appropriate  positions  and 
conditions in their home countries. 

-  The  global  necessity  to  share  research  facilities  to  share  costs,  therefore  creation  and 
development of international platforms. 

-  Attraction of European centres of excellence to set up joint centres in Argentina 

-  Strengthened  connections  to  the  activities  of  the  Ministry  of  the  Economy  and  its 
programmes could enhance the uptake of research for innovation in SMEs and elsewhere 

-  Simplification for making cooperation more attractive.  

Availability of the report on Europa  

http://ec.europa.eu/research/iscp/pdf/review_eu-argentina_s&t_coopagree.pdf  

76

 
 
 
 
 
 
Interim Evaluation of the Innovative Medicines Initiative Joint Undertaking 

ABB activities: 

08 02 Cooperation - Health 

Timing:  

 07/2011 

Background, scope and focus  

To  assess  the  quality  and  efficiency  of  the  IMI  Joint  Undertaking  and  progress  towards  the 
objectives set (set in Council Regulation (EC) 73(2008), Article 11.2).  

Summary of findings and conclusions  

The  overall  result  of  the  evaluation  of  the  Innovative  Medicines  Initiative  Joint  Undertaking 
(IMI JU) is positive. 

–  Through the IMI JU, Europe has succeeded in establishing a new business model between 
public  and  private  sectors,  which  unites  research  strengths  across  European  pharmaceutical 
industry,  academia  and  small  to  medium  enterprises  (SMEs).  The  consortia  formed  carry  out 
focussed  research  addressing  problems  of  immediate  relevance  to  industry  and  future  public 
health. To have formed and embedded this new, applied, research environment is a significant 
achievement for Europe. 

–  By  facilitating  enhanced  cooperation  between  academic,  SMEs,  patient  organisations, 
regulatory authorities and the pharmaceutical industry, the IMI JU enables mutual learning and 
the opportunity to build understanding of respective rationales and approaches, with benefits to 
all parties. This is powerful.  

–  The scientific scope of the initiative is well targeted, embodied in the IMI Research Agenda, 
and the IMI JU has had the foresight to ensure that the Research Agenda is updated regularly.  

–  The financial resources available to the IMI JU, totalling € 2 bn, make this the largest public 
private partnership in health research in the world.  

– 
IMI constitutes a novel model for implementing the concept of “open innovation”. No other 
European programme has enabled cross-company collaboration within the pharmaceutical sector 
on the scale that has been achieved with IMI.  

Having identified many positive points, the Panel also identified certain weaknesses: 

- 

Internal governance structures are not yet working optimally; 

77

 
 
 
 
 
 
 
 
 
 
 
-  Proactive communication activities have been lacking; 

-  The  advisory  potential  of  several  stakeholders,  such  the  European  Medicines  Agency 
(EMA), is not exploited fully by the IMI JU; 

-  The lack of identified and used key performance indicators by the IMI JU risks making the 
output of the whole initiative diffuse. 

The Panel therefore came up with seven recommendations: 

1.  Continuously improve stakeholder involvement in IMI supported research projects. 

2.  Continuously  ensure  EFPIA  and  Commission  commitment 
sustainability. 

to  IMI’s  success  and 

3.  Ensure excellence and exploit new ways to support IMI scientific objectives. 

4.  Improve IMI communication. 

5.  Reinforce and streamline decision making and well-functioning processes. 

6.  Ensure best use of IMI results and IMI sustainability. 

7.  7. Develop monitoring and evaluation processes. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive  

Ex Post Evaluation of NMP FP6 - Project Level 

ABB activities: 

08 22 Completion of previous Framework Programmes and 
other activities 

Timing:  

 08/2011 

Background, scope and focus  

To get further in-depth information and insight into: 

- 
the  factors  that  affected  the  extent  and  range  of  impacts  of  NMP  projects  at  individual, 
organisational,  societal,  economic  and  environmental  levels  across  both  Europe  and  more 
globally; 

78

 
 
 
 
 
 
 
 
 
 
 
 
the  programme  aspects  and  policy  angles  that  should  be  retained  (or  altered)  in 

– 
future NMP programmes. 

Summary of findings and conclusions  

1.  The FP6 NMP programme attracted most of the top EU research institutions and companies. 

2.  The programme participants underlined that the FP6 NMP programme was a strategic tool to 
elaborate  novel  RTD  activities  that  they  would  not  have  been  able  to  perform  otherwise.  The 
main strength of the programme was the ability to bring together the best European, and in many 
cases,  international  research  groups,  raising  the  visibility  of  individual  participants  and 
improving trans-European networking. 

3.  The participants considered S&T goals as the most important reason for participating in the 
programme, followed by economic and health/environment goals. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive  

Meta-analysis of "Bio-Technology", "Agriculture", "Food", "Marine and Maritime", 
and Horizontal themes 

ABB activities: 

08 03 Cooperation - Food, Agriculture and Biotechnology" 

Timing:  

 05/2011 

Background, scope and focus  

The aim of the above evaluation study was to  assess the impact of all Framework Programme 
projects from 2000-2010 in the areas of:  

–  Sustainable production and management of biological resources from land, forest and 
aquatic environments;  

–  Food, health and well-being;  

79

 
 
 
 
 
 
 
 
 
 
 
 
–  Life sciences, biotechnology and biochemistry for sustainable non-food products and 
processes. 

Summary of findings and conclusions  

Results: The projects in this theme have increased the body of knowledge, delivered innovation 
and supported EU policies, thus helping Europe address global societal challenges. According 
to a survey, 64% of coordinators believe that their projects enhance the competitiveness of the 
participants, close to 5% of projects have directly led to the creation of new companies; 82% of 
projects created temporary jobs during the project's implementation and 35% created new posts 
after the end of the project. 

Impacts: Overall, FP projects have had a substantial impact on improving the knowledge base 
in  Food,  Agriculture,  Fisheries  and  Biotechnologies,  through  high  scientific  productivity 
combined with novel technological output. 340 firms in the manufacturing sector of food and 
beverages  that  have  introduced  a  new  product  or new  process  have  received  funds  from  FP5 
and FP6 programmes. Impacts on policies have also been high: according to the survey, more 
than 60% of FP projects have provided inputs to European policies, 56% to national policies, 
and 25% to international agreements. 49% of FP funded project coordinators stated that their 
project  had  positive  environmental  impacts.  The  impact  of  projects  on  the  development  and 
consolidation  of  the  ERA  is  also  found  to  be  very  high:  84%  of  coordinators  consider  that 
participation in FP projects has consolidated their permanent network of partners, and half of 
them stated that their participation has contributed to launching new European projects. 

EU2020  and  Innovation  Union:  The  bio-based  economy,  addressing  the  key  societal  global 
challenges of limited resources, food security, health and climate change contributes to Europe 
2020 on a broad front.  

Competitiveness  and  international  comparison:  Biotechnology  publications  in  2010  ‘Nature‘ 
journals cited US and Canadian authors in 486 cases and European authors in 641 cases, while 
Europe  accounted  for  35%  of  worldwide  biotechnological  patent  applications  in  2006, 
compared to 40% for the USA.  

European-added  value:  Research  and  innovation  in  Food,  Agriculture  and  Fisheries  and 
Biotechnologies has an intrinsic high European added value. The sectors address pan-European 
challenges  which  require  pan-European  research  and  innovation.  Moreover,  almost  70%  of 
coordinators  confirm  that  their  participation  in  an  FP-funded  project  has  delivered  leverage 
effects.  Co-ordination  and  synergy  with  national  R&I  policies  and  investments  is  another 
important aspect of European added value.  

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive  

80

 
 
 
 
 
 
First Interim Evaluation of the Fuel Cells and Hydrogen Joint Undertaking (FCH JU) 

ABB activities: 

08 05 Cooperation - Energy 

Timing:  

 06/2011 

Background, scope and focus  

The  objective  of  this  evaluation  was  to  assess  the  effectiveness,  efficiency  and  quality  of  the 
FCH JU operations, both with regard to the Joint Undertaking and its operating bodies and the 
technical activities carried out by its  members and project participants. In particular, emphasis 
was put on the progress of the FCH JU towards its objectives as set up in article 2 of the Council 
Regulation establishing the FCH JU.  

Summary of findings and conclusions  

The FCH JU is an achievement on its own and represents a valuable instrument for the European 
Union that should be maintained and supported to implement its work as originally envisaged.  

However, the experts have also identified a number of issues encountered by the FCH JU as well 
as some areas where its operation could be improved: 

the current legal framework as a “Community/Union body” is not best-suited to industry led 

- 
public-private partnerships like JTIs and should be streamlined; 

funding rates for FCH JU projects have proved variable from year to year and, in addition, 

- 
always considerably lower than those of FP7; 

inadequate  resources  of  the  Programme  Office  for  effective  project  monitoring  and 

- 
management; 

- 

- 

- 

 insufficient collaboration with Member States’ related programmes; 

lack of a robust project monitoring and assessment;  

lack of an international cooperation strategy.  

In order to address these issues the experts' panel made a series of recommendations, grouped in 
five broad categories: 

1.  Reinforce portfolio management 

2.  Ensure high agility of operations and adaptability to changing competitive forces 

3.  Improve visibility, communication and outreach 

81

 
 
 
 
 
 
 
 
 
 
 
4.  Improve collaboration and alignment with Member States 

5.  Ensure high efficiency of operations. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

Impact Assessment of the Research Potential Programme 

ABB activities: 

08 15 Capacities – Regions of Knowledge 

Timing:  

 09/2011 

Background, scope and focus  

The study aimed to address the following considerations regarding the Research Potential 
programme: 

– 

– 

Impact oriented analysis of projects 

Innovation and I.P. capacity building measures 

–  Analysis of the implementation per thematic domain and for the whole activity 

–  First thoughts for the next period (2014-2020) 

Summary of findings and conclusions  

1.  Programme considered to be simple and well understood by applicants:  

–  High numbers of proposals received (despite low success rate) 

–  Many  proposals scored 14 and above and were rejected 

2.  All FP7 thematic domains are well represented among  funded projects  

–  Excellent geographical distribution 

82

 
 
 
 
 
 
 
 
 
 
 
 
 
3.  Goods links with partnering organisations  

–  Average of 7 partnering organisations per project 

4.  Increased participation in the European Research Area 

– 

– 

– 

– 

– 

– 

Increased recognition of research quality 

Increased publications 

Increase attention to patenting 

Improved cooperation with end-users 

Increased involvement in regional social and economic development 

Improved relationship with S&T policy makers 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

Review of S&T Cooperation between the European Union and the Republic of Chile 
(2007-2011) 

Evaluation related study

ABB activities: 

08 AWBL-02 Policy strategy and coordination for Research 
Directorate-General 

Timing:  

 11/2011 

Background, scope and focus  

1.  Analyse the S&T cooperative activities 2006-2010 under the EU Research FP so as to draw 
up  a  pattern  both  in  terms  of  areas/topics  and  types  of  research  (science  led,  technology  led, 
trade led, global issues led), if possible; 

2.  Identify success factors as well as bottlenecks and obstacles for on-going activities or their 
further development; 

3.  Broadly review EU Member States' bilateral activities with Chile;  

4.  Assess complementarities/synergies and overlaps between different EU-Chile activities; 

5.  Analyse the extent to which the EU-CL S&T cooperation is mutually beneficial; 

83

 
 
 
 
 
 
 
 
 
 
6.  Identify  S&T  areas/actors/instruments  for  which  there  are  promising  prospects  for 
developing the cooperation.  

Summary of findings and conclusions  

1.  The  staff  of  CONICYT’s  International  Directorate  is  highly  qualified  and  motivated. 
However,  given  its  broad  spectrum  of  tasks  regarding  bilateral,  multilateral,  regional  and  EU-
S&T cooperation, it seems not well endowed in terms of personnel and of budget. It is difficult 
to harness fully the potential of international S&T cooperation. 

2.  The NCP network would benefit from more resources. 

3.  ‘Institutional liaison offices’ could be established, supported from Chile’s S&T budget and, 
if needed, twinned with successful offices in EU MS. 

4.  The  low  enterprise  participation  could  be  improved  through  a  broader  dissemination  and 
hands-on training, and systematic links of the SME NCP with national innovation actors. 

5.  The  increased  EU  focus  on  leveraging national  funds  of  the  MS  could  provide  indications 
and incentives to further develop instruments or approaches to open ERA to the world. 

6.  In addition to aiming at more programme synergies, the synergies inherent in well designed 
and  well  implemented  projects  could  be  increased  through  strengthening  national  and 
institutional contact points and support throughout the project cycle. 

7.  More training of administrators at universities and research institutions could be a good way 
to accompany researchers in consortia building.  

8.  Strengthening  connections  between  FP  projects  with  Chilean  participants  and  bilateral 
cooperation projects with EU MS and other countries could be driven by CONICYT and/or the 
EC delegation in Santiago. 

9.  A national programme to accompany the preparation of EU project proposals could help, as 
well as any activity of institutional actors to disseminate the information about opportunities for 
collaboration with the EU.  

10.  This summary couldn’t be complete without mentioning the big obstacle to opening ERA to 
the  world  –  procedures,  administration,  cost  eligibility  (e.g.  VAT)  and  late  payment  in  EU 
projects. Already difficult for European participants, this is even more difficult for researchers 
from ‘over the oceans’.  

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

84

 
 
 
 
 
 
 
 
 
Interim Assessment of the Research Public Private Partnerships in the European Economic 
Recovery Plan: Energy-efficient Buildings, Factories of the Future, and European Green Cars 
Initiative  

ABB activities: 

08 04 Cooperation — Nanosciences, nanotechnologies, 
materials and new production technologies 

Timing:  

 05/2011  

Background, scope and focus  

The overall objective is to assess the progress achieved in the first two years from the launch of 
the  public  private  partnerships  (PPPs),  including  the  effectiveness  and  efficiency  of  the  PPPs 
mechanisms  and  structures  and  how  the  PPP  implementation  has  contributed  to  the  objectives 
set  in  the  European  Economic  Recovery  Plan  and,  as  appropriate,  bring  forward  proposals  for 
how to further develop the PPPs. 

Summary of findings and conclusions  

1.  The  research  PPPs  are  seen  as  a  useful  scheme  for  organising  Research  and  Innovation 
topics with direct industrial utility and the model could be developed further.  

2.  The PPPs have all been successful in engaging top industrial companies, SMEs and research 
organisations within Europe, increasing significantly the large industry and SME participation. 

3.  The  three  research  PPPs  were  targeted  at  providing  research  and  innovation  support  to 
strategically important industries at a time when industrial funding of such actions was in danger 
of being cut back due to the economic crisis. However, while the review found that the research 
PPPs have been an effective response to the crisis, it is unlikely that they will achieve the aim of 
making a difference to the competiveness of European industry unless they are given longer term 
support.  One  criticism  from  industrial  stakeholders  is  that  the  current  level  of  funding  is  not 
sufficient  given  the  economic  importance  of  the  industries  they  are  targeting.  To  make  a 
significant impact on the world stage in terms of European competitiveness, the total combined 
public and private expenditure in this area would need to be considerably increased. 

4.  The  three  research  PPPs  have  facilitated  a  closer  working  relationship  between  the 
Commission and industry in the setting of goals and longer-term research programme objectives. 
This has allowed industry to commit to longer-term strategies for research investment. 

5.  Further  work  needs  to  be  undertaken  to  streamline  the  processes  associated  with  PPPs, 
maintaining  the  efficiency  of  the  calls  and  unifying  the  procedures  across  the  various 
participating themes. 

6.  To  properly  address  the  industrial  needs  relating  to  global  competitiveness,  the  research 

85

 
 
 
 
 
 
 
 
 
PPPs  need  to  support  activities  relating  to  the  “Valley  of  Death”,  in  particular  issues  where 
proven  concepts  and  prototypes  fail  to  progress  to  successful  wealth  creating  products  and 
services. While some of the Valley of Death issues require commercially sensitive, proprietary 
actions, others, such as proving of concepts for European added-value on an industrial scale and 
the  development  of  early  markets,  can  be  issues  that  can  be  tackled  precompetitively.  The 
research PPPs need to include innovation actions that address these near market issues. 

7.  Dissemination activities associated with research PPPs need to be strengthened.  

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

Interim Evaluation of EU FP7 Transport Research, notably within Theme 7 of the 
Cooperation Programme "Transport (including Aeronautics)" 

ABB activities: 

08 07 Cooperation – Transport (including Aeronautics) 

Timing:  

 02/2011 

Background, scope and focus  

The  interim  evaluation  of  the  Seventh  European  Framework  Programme  (FP7)  Transport 
research aims to contribute to the overall FP7 interim evaluation. 

Summary of findings and conclusions  

1.  The FP7 Transport programme exhibits a strong European added value 

2.  The approach adopted for the FP7 is an outstanding improvement compared to the previous 
FPs 

3.  The Transport programme demonstrates stronger ambitions with regards to cross-modal and 
cross-cutting activities, yet allows them to materialise more concretely 

86

 
 
 
 
 
 
 
 
 
 
 
 
 
 
4.  The procedures for priority-setting and Work programme design are well-defined 

5.  European Technology Platforms (ETPs) add value to the FP7 Transport programme 

6.  European  Technology  Platforms  contribute  to  gear  Member  States  public  and  private 
research towards common goals 

7.  The FP7 Transport programme strengthens and broadens the ‘ERA in Transport’ 

8.  The capacity of the FP7 to attract the most important players in research and innovation in 
transport is uneven among sectors 

9.  All necessary conditions are set for projects to deliver their expected results 

10.  FP7  funds  applied  mid-term  research  for  projects  that  will  need  follow-ups  to  lead  to 
innovation 

11.  FP7 funds research with high additionality 

12.  The role of the Small and Medium Enterprises (SMEs) in the projects is important 

13.  The cooperation with ICPC partners is still limited 

Recommendations:  

1.  The  EC  should  reaffirm  and  strengthen  its  integrated,  holistic  and  systemic  approach  in 
future FP7 work programmes and in FP8 

2.  The  EC  should  reinforce  and  modify  the  modalities  of  its  support  to  multi-modal  research 
activities 

3.  The EC should redefine its approach of cross-cutting issues 

4.  The EC should increase the development of new means of communication and diffusion of 
FP7 Transport programme results 

5.  The  EC  should  enhance  the  internal  human  resources  and  budget  assigned  to  the 
management  of  the  FP  Transport  programme  in  order  to  maintain  sufficient  project  and 
programme coordination 

6.  The EC should identify the lessons-learned from the Clean Sky Joint Technology Initiative 
and reflect upon possible replication in other transport modes 

7.  The EC should explore ways and instruments to reinforce European exploratory research 

8.  Increasing articulation between the Framework Programme and the national programmes of 
the Member States 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/pdf/archive/fp7-evidence-
base/evaluation_studies_and_reports/interim_evaluation_of_eu_fp7_transport_research.pdf#view
=fit&pagemode=none  

87

 
 
 
 
 
 
09 – Information Society and Media 

CIP ICT PSP Final (Second Interim) Evaluation  

ABB activities: 

09 03 ICT Take-up 

Timing:  

 20/07/2011 

Background, scope and focus  

The  final  (second  interim)  evaluation  of  the  Competitiveness  and  Innovation  Framework 
Programme ICT Policy Support Programme (CIP ICT PSP) focuses on the effectiveness of the 
new intervention instruments and on their relevance to the needs and problems in diffusing and 
deploying  Information  and  Communication  Technologies  (ICT)  particularly  in  public  sector-
related  activities.  This  includes  efficiency  of  resource  use,  effectiveness  in  engaging 
organisations that can meet the objectives of the programme, emphasis on tangible outcomes and 
sustainable  follow-up,  and  the  extent  to  which  the  programme  is  contributing  to  wider  EU 
strategic objectives. 

Summary of findings and conclusions  

Based on a review of available evidence and detailed project analysis the expert panel notes and 
recommends: 

1.  The positive, unique and innovative role of the ICT PSP. The ICT PSP has a range of 
positive  attributes  (a  unique  and  pioneering  innovation  instrument,  directly  policy-
related,  complementary  portfolio  of  instruments,  wide  stakeholder  participation)  that 
contributes to its strengths and differentiates it from research support programmes.  

2.  Continuing  the  ICT  PSP  as  an  innovation  deployment  policy  instrument.    The 
unique  features  of  the  ICT  PSP  are  its  focus  on  deployment  of  ICT  innovations  and 
addressing systemic and organisational risks, and these features should be maintained in 
any follow-up.. 

3.  Differentiating  the  ICT  PSP  from  the  Framework  Programme  in  any  post-2013 
follow-up.  There  are  major  differences  between  support  for  generation  of  new 
knowledge  and  support  for  diffusion  and  deployment  of  innovative  services.  These 
differences need to be reflected in policy design and delivery.  

Within the overall context of the expert panel’s positive appraisal of the role and importance of 
this  kind  of  programme,  its  innovative  nature  and  efficient  administration,  the  following 
Recommendations  are  made  to  improve  the  functioning  and  increase  the  impacts  of  the 
Programme in terms of developing longer-term sustainable cross-border public interest services: 

88

 
 
 
 
 
 
 
 
 
 
–  Tightening selection processes, strengthening project reviews and making them more 
systematic.  

–  Sustaining sustainability. 

–  Developing a strategic approach to critical mass.  

–  Expanding stakeholder involvement.  

–  Paying more attention to end users.  

–  Overcoming hurdles to SME participation. 

–  Rethinking Thematic Networks.  

–  Linking with other programmes.  

Availability of the report on Europa  

http://ec.europa.eu/cip/files/cip/docs/cip_ict_psp_interim_evaluation_report_2011_en.pdf 

Safer Internet Evaluation 

ABB activities: 

09 02 Regulatory framework for the Digital Agenda  

Timing:  

 01/11/2011 

Background, scope and focus  

The overall aim of the Safer Internet Programme is to promote safer use of the Internet and other 
communication technologies, to educate users — particularly children, parents and carers — in 
this regard, and to fight against illegal content and harmful conduct online. 

As  an  interim  evaluation,  the  exercise  gives  an  analysis  of  the  relevance  of  the  programme 
concerning  its  design,  efficiency  and  effectiveness.  In  addition,  the  evaluation  indicates  the 
sustainability  and  impact  the  programme  has  had  with  a  special  emphasis  on  the  coherence  of 
the programme. 

Summary of findings and conclusions  

89

 
 
 
 
 
 
 
 
 
 
 
 
 
The evaluation results are positive. They show that the programme has evolved to keep up with 
the  technological  and  sociological  considerations  surrounding  the  safer  use  of  the  Internet  by 
minors. It responds well to changing demands in the external environment as evidenced by the 
current emphasis on social media and cyber-bullying. However, to ensure the inclusion of new 
issues  the  programme  needs  to  continue  to  explore  new  issues  and  have  a  long-term  strategic 
vision. 

The evaluation results further show that the programme is influential in relation to other national 
and international activities; literature, research and results from the programme are widely used 
and  quoted.  In  addition,  recommendations  from  the  last  programme  evaluations  have  been 
broadly incorporated into the programme. 

The  programme  was  run  efficiently.  There  are  good  levels  of  communication  between  the 
related  EU  programmes  at  policy  level.  The  programme  is  also  effective  according  to  the 
evaluation. The achievements of the programme concerning the existence of hotlines, helplines 
and awareness centres in the majority of Member States are highlighted in the evaluation. The 
evaluation also concludes that the current focus and structure of  the programme is appropriate 
and  does  not  call  for  significant  change  in  the  next  period.  The  support  and  funding  from  the 
programme  is  crucial  for  the  projects  to  exist,  especially  considering  the  current  economic 
constraints. 

The overall picture of the programme is very positive; there is however room for improvements 
concerning some matters according to the evaluation findings. According to some stakeholders 
there still needs to be done further work to ensure that there is direct involvement of children. 
The projects call for longer funding periods to ensure better continuity and periods of reflection. 
There  is  a  need  for  the  programme  to  keep  up  to  date  with  technological  advances  –  through 
market  intelligence,  knowledge  enhancement  projects  and  general  knowledge  exchange.  The 
programme must also be able to follow the developments in national contexts concerning issues 
related  to  legislation.  The  risk  with  national  budget  cuts  and  subsequent  change  of  priorities 
implies  that  there  is  a  need  to  ensure  that  national  and  regional  priorities  are  coherent  with 
European priorities to maximise the effect in the time to come. 

RECOMMENDATIONS  

-  The  programme  should  not  broaden  its  objectives  but  maintain  its  current  ones.  It  is 
recommended to maintain a long-term strategic vision on the programme and its issues.  

-  The  efficiency  of  the  programme  could  be  improved  through  longer  funding  periods  for  the 
projects and fewer restrictions on the internal allocation of funds.  

-  The  networks  of  the  hotlines,  helplines  and  awareness  centers  should  be  further  coordinated 
through joint events and working groups. In addition it is recommended to look at measures for 
increasing  the  consistent  commitment  of  industry  as  well  as  further  increase  the  international 
networking opportunities of key stakeholders. 

- Concerning effectiveness it will be advisable to increase awareness of the work of the Safer 
Internet Programme to address a potential reduction in national/regional political support. 

Availability of the report on Europa  

http://ec.europa.eu/information_society/activities/sip/docs/prog_2009_2013/FINAL%20REPORT
%2020120124.pdf 

90

 
 
 
 
 
eContent Plus Final Evaluation  

ABB activities: 

09 03 ICT Take-up 

Timing:  

 9/9/2011 

Background, scope and focus  

The objective of the Programme was to make digital content in Europe more accessible, usable 
and exploitable, facilitating the creation and diffusion of information, in areas of public interest, 
at Community level. 
In order to achieve the overall aim of the Programme, the following lines of action were 
addressed: 
(a) facilitating at Community level access to digital content, its use and exploitation; 
(b) improving quality and enhancing best practice related to digital content between content 
providers and users, and across sectors; 
(c) reinforcing cooperation between digital content stakeholders and awareness. 

Via those lines of action, the Programme targeted areas which had a public interest and which 
would not have developed, or would have developed at a slower pace, if left to the market: 
geographic information, educational content and cultural and scientific/scholarly content digital 
libraries. 

Pursuant  to  Article  5(3)  of  the  Programme  Decision,  the  Commission  was  to  submit  a  final 
evaluation  report  at  the  end  of  the  Programme.  This  evaluation  has  been  carried  out  with  the 
assistance of a panel of external experts who evaluated the effectiveness of the Programme, its 
achievements,  its  impact,  its  sustainability  and  its  complementary  with  other  initiatives  within 
and  beyond  the  European  Union,  as  well  as  with  national  initiatives.The  findings  of  this  final 
evaluation are based on available documentation, Commission data, a survey questionnaire, and 
follow-up interviews with selected project participants. 

Summary of findings and conclusions  

The evaluators found that the Programme has been successful in achieving the objectives set out 
in the Programme Decision and in the annual work programmes adopted for its implementation. 
It has contributed to making digital content in Europe more accessible, usable and exploitable, 
facilitating  the  creation  and  diffusion  of  information,  in  areas  of  public  interest.  More 
specifically, the evaluation gave rise to the following conclusions: 

-  The  Programme  achievements  are  in  line  with  its  objectives:  The  Programme  succeeded  in 
creating  better  conditions  for  accessing,  using,  re-using  and  exploiting  digital  material,  thus 
making it possible to build added-value products and services across Europe in the target areas 
selected. It helped content stakeholders (providers and users) realise the full potential of digital 

91

 
 
 
 
 
 
 
 
 
 
content,  and  users  are  now  able  to  access  and  use  such  content,  irregardless  of  location  or 
language; 

- The Programme was managed efficiently;  

- The impact on organisations participating in the Programme can only be estimated, since many 
projects are still running or have just ended. Indicatively, most organisations already recognise 
that  participation  in  eContentplus  projects  has  increased  international  and  cross-border 
collaboration  and  made  them  more  aware  of  the  economic  and  cultural  value  of  the  digital 
content  they  own.  In  the  area  of  digital  libraries,  the  eContentplus  Programme  is  the  first 
programme  whose  main  impact  has  been  to  make  material  directly  available  to  individual 
citizens on a very large scale. In the area of educational content, a significant increase in the use 
of  the  underlying  content  isexpected  as  a  result  of  programme  support.  In  addition,  the 
involvement of national Ministries of Education in many of the projects is also likely to have an 
impact  on  the  inclusion  of  eLearning  in  curricula.With  regard  to  geographic  information, 
programme  support  is  expected  to  increaseinteroperability  within  a  number  of  thematic  areas 
under the INSPIRE Directive. 

Challenges that have to be addressed are the following:  Increased public funding is needed to 
finance  large-scale  digitisation,  alongside  initiatives  with  private  partners  provided  that  they 
allow  a  general  accessibility  of  Europe's  common  cultural  heritage  online.  Fragmentation  and 
complexity in the current licensing system also hinders the digitisation of a large part of Europe's 
recent cultural heritage. Rights clearance must be improved, and the role and responsibilities of 
private  and  public  organisations  for  digitising  orphan  works  as  well  as  material  that  is 
incopyright but no longer commercially available should be clarified. 

MAIN RECOMMENDATIONS 

There  is  an  increasing  demand  for  quality  digital  content  in  Europe  from  citizens,  students, 
researchers,  SMEs  and  other  business  users  or  re-users  wishing  to  exploit  digital  content 
resources  to  create  services.  This  calls  for  continued  support  at  European  level  to  increase  the 
availability of European digital content, in particular in areas of public interest where progress is 
likely  to  be  slow  and  where  access  to  content  is  limited  by  language  and  cultural  barriers  in 
particular. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0548:FIN:En:PDF  

92

 
 
 
 
 
 
11 – Maritime Affairs and Fisheries  

Interim evaluation (study) of the European Fisheries Fund (2007-2013) 

ABB activities: 

11 06 European Fisheries Fund (EFF)  

Timing:  

 February 2011 

Background, scope and focus  

The European Fisheries Fund (EFF) is the financial instrument of the Common Fisheries Policy 
(CFP) for 2007-2013. The EFF total budget amounts to € 4.3 billion and seeks to grant financial 
support  to  the  European  fisheries  sector  for  the  period  2007-2013  to  help  it  adapt  to  changes 
required within the sector.  

The objective of this interim evaluation is to perform an up-to-date assessment of the uptake and 
first  impacts  of  EFF  measures  implemented  in  the  Member  States  as  well  as  underpin  impact 
analysis of the future financial instrument. It is also worth to mention that the evaluation took 
place at an early stage in the implementation of the programme. 

Summary of findings and conclusions  

Partnership  principle  is  generally  well  applied,  despite  under-representation  of  women’s  and 
environmental organisations. Partnership between EC and MS is satisfactory, but coordination is 
slightly too formal.  

National  Strategic  Plan  is  a  broad  document  which  enables  a  better  strategic  view  on  the 
development of the sector in line with the CFP, but it entails a time-consuming process and is 
disregarded once the OP is in place.  

Although useful in enhancing the reliability of Operational Program (OP) management, the set 
up and assessment of the Management Control System (MCS) was a lengthy and costly process 
(for both EC and MS). In certain MS, human resources were focused more on the closure of the 
FIFG programme(s) rather than the launch of the EFF programme. The shift towards one single 
OP has mainly benefited the EC whilst, on the whole, it has increased the management costs for 
most MS. Only some of these costs 

Shift  to  one  single  OP  has  significantly  increased  the  programme  complexity  as  well  as 
administrative  burden  in  Federal  States  (and  more  generally  MS  having  a  decentralised 
management).The  workload  in  terms  of  the  audit  function  has  significantly  increased  since 
FIFG. Although enabling better control and lowering financial risks, the administrative burden is 
too  high  for  small  MS  that  have  to  establish  the  same  structures  and  systems  regardless  the 
programme size.  

93

 
 
 
 
 
 
 
 
 
 
Administrative burden for the final beneficiaries has generally been reduced. It remains higher in 
new  MS  where  the  required  paperwork  to  accompany  applications  may  prevent  potential 
beneficiaries from applying.  

Some  good  practices  are  observed  in  assisting  and  reaching  potential  beneficiaries  which  is  a 
key  factor  for  fostering  new  projects.  This  is  all  the  more  important  as  OP  are  increasingly 
focused on developing innovative projects, which require more technical assistance to potential 
beneficiaries.  

Regarding  monitoring  systems,  existing  indicators  required  by  the  EC  are  focused  on  output. 
They  are  numerous,  not  relevant  for  all  MS,  lack  a  common  definition  of  the  units  to  be 
measured,  and  are  therefore  not  completely  reliable.  Indicators  are  not  reported  upon 
systematically nor verified, so they may be incomplete and contain errors 

Finally, the program effectiveness has been undermined by low commitment rates under some 
measures  linked  with  the  issue  of  access  to  private  co-financing.  The  list  and  organisation  of 
current  EFF  measures  requires  review,  so  that  the  fund  can  be  tailored  in  the  future  to  avoid 
overlapping and respond to the most relevant needs. Whilst simple ‘replacement’ measures have 
been effective in terms of commitments, their focus is too individual and short term. This might 
limit the expected impacts in terms of environmental sustainability of the sector. 

Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/eff_interim_evaluation_en.pdf  

Interim evaluation of DG MARE external communication activities 

ABB activities: 

11 04 Governance of the common fisheries policy 

Timing:  

 February 2011 

Background, scope and focus  

In  view  of  an  increased  demand  at  the  political  level  from  both  EU  institutions  and  Member 
States to communicate more often and effectively, the European Commission views its mission 
of communicating about the EU and its policies as a high priority. DG MARE has launched an 
evaluation  that  covers  the  external  communication  activities  relating  to  both  the  Common 
Fisheries Policy (CFP) and the Integrated Maritime Policy (IMP) performed over the 2007-2010 
period. 

94

 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

DG MARE is currently facing important policy challenges, especially with regard to the reform 
of the CFP. They are also linked to communication challenges considering on the one hand the 
extension  in  the  scope  of  the  types  of  stakeholders  that  goes  together  with  the  Integrated 
Maritime Policy (IMP) development and on the other hand the “going local” trend of the CFP 
which takes more and more into account the regional concerns. Means are limited, but areas for 
concrete action may be easily identified.  

The following recommendations may be formulated. They are presented by priority: 

1.  Revise  the  strategy  framework  to  build  a  strategic  document  with  clear  objectives,  actions 
and target audiences declined into a real communication plan. 

The communication is composed of four axes: target audiences, objectives, messages and tools. 
DG MARE should define a communication plan with a matrix view that shows the articulation 
between target audiences, objectives, tools and messages. A better targeted strategy in terms of 
audiences may help better address all communication objectives. 

2.  Refine  the  target  audience  definition  and  classification  to  better  adapt  the  strategy  to  their 
needs 

This recommendation consists of two priority tasks: Refine the definition of target audiences and 
classify  them  in  a  more  detailed  manner  to  be  able  to  better  take  their  needs  into  account  and 
better  analyse  the  needs  of  each  target  audience  and  reflect  upon  a  process  or  tool  that  may 
enable to collect and update the needs analysis. 

3.  Adapt  some  of  the  key  messages  to  improve  CFP  image  and  develop  confidence  and 
acceptance among the priority target audiences, i.e. the fishermen 

4.  Better  manage  existing  communication  tools  to  develop  accessibility,  readability  and 
understandability  of  key  information  regarding  CFP  and  IMP.  Improving  knowledge  and 
understanding among the different target audiences  could be easily achieved by improving the 
existing communication tools. 

5.  Put in place new strategic tools to improve efficiency 

The  development  of  an  effective  performance  management  of  communication  tools  could  be 
achieved through the development of an appropriate monitoring system with SMART (Specific, 
Measurable, Achievable, Relevant and Time-bound) indicators. These indicators would be used 
to follow up the implementation of communication activities and to assess the effectiveness. A 
set of useful indicators could include both output and result indicators (no. of participants at an 
event per target audience, etc.) 

6.  Establish a newsletter tailored to the needs of each individual (good practice and expectation 
from most stakeholders) 

One  main  recommendation  from  the  stakeholders  met  during  the  evaluation  process  relates  to 
the  creation  of  a  newsletter  with  content  tailored  to  their  specific  needs.  Indeed,  they  often 
complain  about  the  fact  that  the  information  they  search  for  is  hard  to  find  on  the  one  hand, 
while they are overwhelmed with information (from DG MARE and other sources) that they do 
not need and do not have time to read on the other hand. 

95

 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/external_communication/index_en.htm  

Interim evaluation of the European Fisheries Fund (2007-2013) 
Synthesis of the 26 national evaluation reports 

ABB activities: 

 11 06 European Fisheries Fund (EFF) 

Timing:  

 December 2011  

Background, scope and focus  

The purpose of this report is to summarise the contents of the national interim evaluation reports 
on  the  European  Fisheries  Fund  (2007-2013)  and  to  highlight  the  main  points  and 
recommendations that emerge. It also aims to provide a basis for the organisation of a “strategic 
debate “with Member States to be undertaken by the end of 2011, which should also focus more 
widely  on  the  content  and  progress  of  the  implementation  of  national  strategic  plans,  and 
develop the exchange of best practice among Member States, as prescribed in Article 16 of the 
EFF regulation. 

Summary of findings and conclusions  

Since  2007,  the  sector  was  mainly  impacted  by  the  financial  /  economic  crisis  and  2007/2008 
fuel  crisis  which  resulted  in  reduced  access  to  private  finance,  stronger  public  expenditure 
control increased financial and economic pressure on the sector (rising fuel costs, fish feed costs 
increase, rising unemployment), a decline in demand for fisheries products, and prices that have 
stagnated/ fallen. A few MS however highlight a rather limited or very temporary impact of fuel 
crisis on the sector. The crisis was sometimes seen as an opportunity to promote innovation and 
higher  productivity  in  the  sector,  to  switch  to  more  economical  methods  of  fishing  and  to 
restructure the fleet. 

Most evaluation reports reach the conclusion that the management and control systems in place 
are  rather  satisfactory,  with  clear  and  efficient  coordination  between  the  different  structures 
involved in programme management, despite some different evolutions and characteristics. 

Monitoring Systems as a whole work well, but the quality and definitions of indicators should be 
improved. The usefulness of the indicators system in terms of evaluating the programmes results 
is not good. 

96

 
 
 
 
 
 
 
 
 
 
 
From a priority axis perspective, at 31 December 2010, the commitments by axes amounted to 
47%  of  total  EFF  budget  (M€  575)  for  Axis  1,  43%  (M€  518)  for  Axis  2,  40%  (M€  451)  for 
Axis 3, 6% (M€ 34) for Axis 4 and 21% (M€ 35) for Axis 5. Axis 1 projects thus account for the 
greatest proportion of commitments, due to their shorter term nature. Axis 4 achievement rate is 
the lowest, indicating that whilst groups have been formed, FLAG group projects are generally 
still in the very early stages of implementation. 

Generally  speaking,  sustainable  development  is  effectively  taken  into  account  by  EFF 
programmes,  whilst  the  consideration  given  to  gender  and  equal  opportunity  issues  is  more 
variable. The overall contribution of EFF programmes to the promotion of equal opportunity is 
neutral  or  marginally  positive.  The  overall  contribution  of  EFF  programmes  to  environmental 
sustainability is improving, although it remains unclear. 

Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/eff_evaluation/index_en.htm  

Mission d’évaluation à mi-parcours du Plan Régional de Surveillance des Pêches dans le Sud-
Ouest de l’Océan Indien 

ABB activities: 

 11 03 International fisheries and law of the sea 

Timing:  

 November 2011 

Background, scope and focus  

La Commission européenne a décidé conformément à ses engagements internationaux de lutter 
contre  la  pêche  INN,  notamment  au  travers  de  son  "Plan  d'action  communautaire  en  vue 
d’éradiquer la pêche INN"1. A ce titre, et suite aux engagements des Chefs d'Etat de la COI en 
2005, et à la déclaration de 2007 des Ministres de la COI en charge de la pêche, une convention-
cadre de partenariat intitulée "COI – Plan régional de surveillance des pêches dans le sud-ouest 
de  l’océan  Indien"  a  été  signée  le  24  janvier  2007  à  Mahé,  Seychelles,  entre  la  CE  et  le 
Secrétariat  général  de  la  COI.  Cette  convention  cadre  fixe  les  modalités  de  mise  en  œuvre  du 
projet intitulé "Projet régional de surveillance des pêches dans le sud-ouest de l'océan Indien." 
(PRSP).  Le  PRSP  est  structuré  autour  d’un  objectif  politique  et  d’un  environnement  législatif 

1 COM(2002)0180 final Communication from the Commission - Community action plan for the 
eradication of illegal, unreported and unregulated fishing 

97

 
 
 
 
 
 
 
 
 
                                                 
défini par la réunion ministérielle de Mahé, d’une méthode et d’une liste d’actions détaillée dans 
la convention cadre et les conventions spécifiques.  

Summary of findings and conclusions  

Pertinent,  le  projet  s’inscrit  pleinement  dans  les  orientations  générales  de  l’UE  en  matière  de 
pêche responsable et durable et de coopération régionale ainsi que dans la politique de la COI. 
Le projet dispose d’une forte cohérence interne. Le délai entre l’engagement des Etats membres 
de la COI à se doter d’une stratégie régionale de pêche et de lutter conjointement contre la pêche 
IUU et la mise en œuvre du projet aura été exceptionnellement court (24 mois) pour un projet de 
cette  envergure.  Le  dispositif  organisationnel  construit  et  le  fonctionnement  sur  un  mode 
"réseau"  a  particulièrement  bien  fonctionné.  Il  y  a  eu  adéquation  entre  la  nature  des  actions 
engagées par le projet et les procédures financières imposées. Les résultats obtenus par le projet 
sont conformes aux résultats attendus à l’origine du projet. En termes d'efficience des moyens 
engagés  par  la  CE  et  les  Etats  membres,  les  résultats  doivent  être  considérés  comme  très 
satisfaisants. Il y a une unanimité des administrations nationales et du Secrétariat Général de la 
COI  à  considérer  ce  projet  comme  un  succès  et  à  le  voir  se  pérenniser  sans  interruption,  y 
compris au prix d’une sollicitation financière plus grande des Etats membres. 

Availability of the report on Europa  

Not yet published. 

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement 
(FPA) between the European Union and Madagascar (2007-2012) 

ABB activities: 

11 03 International fisheries and law of the sea  

Timing:  

 November 2011 

Background, scope and focus  

98

 
 
 
 
 
 
 
 
 
This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  EU  and  Madagascar  (2007-2012).  The  evaluation 
considers  the  Protocol  in  terms  of  its  relevance,  coherence,  effectiveness,  efficiency  and 
sustainability 

Summary of findings and conclusions  

The  ex-post  evaluation  of  the  Fisheries  Partnership  Agreement  between  Madagascar  and  the 
European Community in terms of effectiveness indicates that the agreement reaches most of its 
goals. 

The reference tonnage has been set at a level close to the actual catch, which does not entail a 
disproportionate  cost  for  the  agreement.  For  the  European  fleets,  the  access  costs  are 
advantageous, especially for the surface longliners. The agreement has an important economic 
leverage. The agreement can therefore be qualified as efficient. 

The  FPA  relevance  has  been  verified.  The  European  tuna  seiners  fleets  fishing  in  the  Indian 
Ocean  need  access  to  fishing  grounds  that  represent  a  significant  share  of  their  total  catch, 
allowing them also to land their catch close to their fishing grounds. The surface longliner fleet 
targets a resource in the Malagasy waters it could barely find in any other area where it could 
operate.  The  agreement  allows  Madagascar  to  monetize  an  under-exploited  natural  resource 
which  is  not  currently  targeted  by  its  own  industry  and  which  feeds  into  an  important 
processing  industry  in  Madagascar.  On  this  particular  point,  the  surface  longliners  has  less 
impact than the tuna seiners. Support for SCS and sanitary control meets Madagascar’s needs 
to perform their duties at a satisfactory level. 

The  agreement  contributes  to  the  viability  of  the  European  tuna  supply  chain  by  granting 
access  to  the  Madagascar  fishing  zone  under  a  multiannual  stable  legal  framework.  It 
reinforces the implementation of management measures at the national and regional level and 
contributes to maintaining fishing activities in the region. The European fleets do not fish on 
the  continental  shelf  and  therefore  do  not  compete  with  local  traditional  fishermen.  FPA 
sectoral support is allocated to actions which contribute strongly to the Malagasy fishing sector 
viability 

Availability of the report on Europa  

Not yet published. 

99

 
 
 
 
 
An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Cape Verde (2007-2011) 

ABB activities:   

11 03 International fisheries and law of the sea 

Timing:   October 2010, published on 22/7/2011 

Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  Protocol  to  the  Fisheries 
Partnership Agreement (FPA) between the EU and Cape Verde (2007-2011). It covers the period 
of 1 September 2006 to 31 August 2011. The evaluation considers the Protocol in terms of its 
relevance, coherence, effectiveness, efficiency and sustainability 

Summary of findings and conclusions  

For the EU the Fisheries Partnership Agreement has provided access to fishing opportunities for 
EU  fleet  segments  from  fishery  dependent  areas,  created  employment,  and  provided  for 
additional supplies to the EU market. There is a realistic potential to link catches by EU vessels 
to Cape Verdean processing and export activities. Although it has had no impact on IUU fishing 
until  now,  the  Agreement  has  potential  to  achieve  this  in  future.  The  Agreement  provides  an 
additional  dimension  to  the  Special  Partnership  between  the  EU  and  an  important  Peripheral 
Region  Nation,  and  could  complement  a  future  Atlantic  dimension  to  the  EU’s  Integrated 
Maritime  Policy.  There  are  specific  synergies  with  a  number  of  EDF  regional  development 
programmes (ACP Fish II, SFP and the forthcoming regional MCS programme implemented by 
CSRP). The Agreement may therefore be considered coherent with the EU’s fisheries, maritime 
and development policies. 

For  Cape  Verde  the  Agreement  has  provided  about  24%  of  the  financial  means  for  public 
investment  in  fisheries,  thus  contributing  towards  economic  development  and  sustainability  of 
the  sector.  It  has  had  a  particular  impact  (along  with  donor  support)  on  building  institutional 
capacity,  providing  facilities  for  small  scale  fisheries  and  improving  compliance  with  EU 
sanitary  conditions  for  trade  in  fishery  products,  all  important  conditions  for  increasing  the 
economic contribution of the fishery sector. 

Overall the Agreement has also allowed the EU and the Cape Verde Authorities to maintain a 
policy  dialogue,  with  a  view  to  promoting  responsible  fishing,  although  the  success  of  the 
partnership  approach  has  so  far  been  limited  by  the  low  level  of  engagement,  a  sub-optimal 
intervention design and weak integration with linked donor programmes. Nevertheless, despite 
the under-performance of some aspects of the Agreement, it appears that it is strongly in the 
interest  of  both  parties  to  conclude  a  new  protocol  that  would  prolong  this  partnership 
between Cape Verde and the European Union. 

100

 
 
 
 
 
 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/cape_verde_2010/index_en.htm  

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Guinea Bissau (2007-2011) 

ABB activities:   

11 03 International fisheries and law of the sea 

Timing:  

 September 2010, publication on 22/7/2011  

Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  EU  and  Guinea  Bissau  (2007-2011).  It  covers  the 
period of 16 June 2007 to 15 June 2011. The evaluation considers the Protocol in terms of its 
relevance, coherence, effectiveness, efficiency and sustainability. 

Summary of findings and conclusions  

The Fisheries Partnership Agreement has provided access to fishing opportunities for EU fleet 
segments  from  fishery  dependent  areas,  created  employment,  and  provided  for  additional 
supplies to the EU market. The Agreement accounts for about one quarter of all transfers from 
the  EU  to  Guinea  Bissau,  and  the  Agreement  therefore  provides  an  important  supplementary 
pillar  of  support.  The  Agreement  has  started  to  deliver  important  developmental  outcomes  in 
terms of reduced IUU fishing, as well as the prospects of increased trade in fishery products and 
is  therefore  coherent  with the  EU’s  national  and  regional  development  approaches.  Its  support 
for  fisheries  MCS  means  that  the  Agreement  is  coherent  with  the  EU’s  approach  to  reducing 
IUU  fishing.  The  Agreement  has  also  allowed  the  EU  and  the  Guinea  Bissau  Authorities  to 
maintain  a  policy  dialogue,  with  a  view  to  promoting  responsible  fishing.  As  a  conclusion,  it 
appears that it is strongly in the interest of both parties to conclude a new protocol that would 
prolong this partnership between Guinea Bissau and the European Union. 

101

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/guinea_bissau_2010/index_en.htm  

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Mauritania, and an ex-ante evaluation and analysis of 
impacts of a potential future Protocol on sustainability 

ABB activities:   

11 03 International fisheries and law of the sea 

Timing:  

 March 2011  

Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  existing  Protocol  to  the 
Fisheries Partnership Agreement (FPA) between the EU and Mauritania. It covers the period of 
1 August 2008 to 31 July 2012. The evaluation considers the Protocol in terms of its relevance, 
coherence, effectiveness, efficiency and sustainability. It also provides a prospective analysis of 
impacts  and  ex-ante  evaluation  of  a  future  Protocol,  in  order  to  provide  sufficient  data  and 
information for the negotiation and implementation of a new Protocol. 

Summary of findings and conclusions  

The  scientific  advice  should  be  reinforced,  both  on  the  level  of  Mauritania  and  of 
CECAF/COPACE.  It  would  be  appropriate  that  COPACE  become  an  independent  body  being 
able to deliver high quality scientific advice. On the other hand, the Joint scientific Committee 
EU/Mauritania should be provided with a permanent secretariat.  

Related to the pelagic fishery, Member States concerned have to find a distribution key as the 
management  of  the  global  quota  appears  not  to  be  efficient.  Parallel  to  that,  it  would  be 
appropriate to suppress the mechanism of additional catches, paid exclusively by the EU budget. 

In  terms  of  sectoral  support,  the  monitoring  of  the  expenses  should  be  substituted  rather  by 
indicators  of  results.  Moreover,  a  conditionality  related  to  the  achieved  progress  has  to  be 
introduced (payment in tranches). 

There  is  a  need  for  a  better  synergy  between  development  policy  instruments  (EDF)  and  the 
FPA.  EDF  could  possibly  build  up  some  fisheries  actions,  supporting  more  infrastructure 

102

 
 
 
 
 
 
 
 
 
 
 
projects. 

It is desirable to maintain an annual rhythm to organise Joint Committee sessions which are an 
important tool to smoothly manage the Protocol. 

Presence  of  scientific  observers  on  board  of  EU  vessels  should  be  more  emphasized  and  EU 
ship-owners should get more involved.  

Concerning the cross-border management of the pelagic stocks, there is a need for coordination 
between the FPA with Morocco in order to introduce a regional approach.  

Availability of the report on Europa  

Not yet published. 

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Gabon, and an ex-ante evaluation and analysis of impacts 
of a potential future Protocol on sustainability 

ABB activities:   

11 03 International fisheries and law of the sea 

Timing:  

 June 2011 

Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  existing  Protocol  to  the 
Fisheries Partnership Agreement (FPA) between the EU and Gabon. It covers the period of 2005 
to  2011.  The  evaluation  considers  the  Protocol  in  terms  of  its  relevance,  coherence, 
effectiveness, efficiency and sustainability. It also provides a prospective analysis of impacts and 
ex-ante  evaluation  of  a  future  Protocol,  in  order  to provide  sufficient  data  and  information  for 
the negotiation and implementation of a new Protocol. 

Summary of findings and conclusions  

The relevance of the agreement is verified for both parties. EU distant-water tuna fishing vessels 
need an access to a fishing zone that represents about 10% of their catches in the Atlantic. The 

103

 
 
 
 
 
 
 
 
 
 
 
 
 
agreement gives to Gabon the opportunity to derive income from their fishery resources present 
in the remote parts of the  EEZ in the absence of national fishing vessels able to exploit them. 
The  agreement  does  not  have  any  particular  impact  on  the  sustainability  of  exploitation  of 
fishery  resources  in  the  EEZ  of  Gabon  as  the  European  vessels  exploit  fishing  possibilities 
within  the  rules  set  by  the  Regional  Fisheries  Management  Organisation  in  charge  of 
management and conservation of this shared tuna resource. The agreement is likely to contribute 
indirectly  to  the  biological  viability  of  exploitation  through  the  financial  support  allocated  to 
control  and  surveillance  and  research,  but  the  contribution  cannot  be  evaluated  in  more  detail. 
Several  recommendations  are  proposed  to  improve  the  performances  of  a  new  protocol  of 
agreement.  These  recommendations  concern  the  level  of  fishing  possibilities,  improving  the 
transparency of the agreement at national level, a revision of the implementing modalities of the 
sectoral partnership to take into consideration input indicators as opposed to output indicators, 
and a proactive role of the EU in the  monitoring and coordination of interventions of external 
donors in the fisheries sector. 

Availability of the report on Europa  

Not available until end of the negotiation. It is confidential at the moment. 

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Mozambique (2007-2011) 

ABB activities:  

11 03 International fisheries and law of the sea  

Timing:  

 June 2011 

Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  EU  and  Mozambique  (2007-2011).  The  evaluation 
considers  the  Protocol  in  terms  of  its  relevance,  coherence,  effectiveness,  efficiency  and 
sustainability 

104

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The first Protocol of the Fisheries Partnership Agreement between the EU and Mozambique has 
clearly  benefited  the  mutual  interests  of  both  parties.  For  the  EU,  it  has  proved  to  be  highly 
relevant  to  the  Common  Fisheries  Policy  (since  it  provides  access  to  fishing  opportunities  for 
EU vessels, supporting their regional presence in the SW Indian Ocean), with associated (albeit 
modest)  financial  and  economic  benefits  to  the  EU.  For  Mozambique  it  has  allowed  the 
generation  of  financial  income  from  national  fisheries  resources  which  at  present  the  country 
does not have the capacity to exploit.   

The issue of piracy has impacted severely on the utilisation of the Protocol, but it should also be 
noted  that  vessels  which  have  continued  to  use  the  opportunities  have  experienced  increasing 
catch rates in throughout the 4 years. The FPA provided about 60% of the income generated by 
Mozambique  from  this  fishery  and  has  therefore  clearly  been  in  Mozambique’s  interest. 
Mozambique has elected to apply all of this to the fishery sector budget, thus delivering general 
benefits to the sector. However it derives no other economic benefits from the presence of the 
EU vessels. Until now, the policy support measures adopted by the parties to the FPA have not 
been specifically linked to the Fisheries Master Plan. The Agreement has had less relevance to 
Mozambican  fisheries  policy  as  set  out  in  the  poverty  reduction  strategy  and  Fisheries  Master 
Plan  II,  although  it  has  contributed  indirectly  via  the  sectoral  support  measures  which  have 
helped  to  promote  sustainable  exploitation  of  resources,  to  strengthen  the  fight  against  IUU 
activities, and to strengthening institutional capacity of the sector. In conclusion, although there 
are  concerns  regarding  the  efficiency  of  the  Agreement  and  the  effectiveness  so  far  of  the 
partnership component, it has proved overall to be a useful tool for furthering the mutual policy 
objectives of the parties.  

Availability of the report on Europa  

Not available until end of the negotiation. It is confidential at the moment. 

An ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement (FPA) 
between the European Union and Mozambique (2007-2011) 

ABB activities:  

11 03 International fisheries and law of the sea  

Timing:  

 September 2011 

105

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

This  evaluation  provides  a  retrospective  ex-post  evaluation  for  the  existing  Protocol  to  the 
Fisheries  Partnership  Agreement  (FPA)  between  the  EU  and  Greenland  and  Denmark  (2007-
2012) The evaluation considers the Protocol in terms of its relevance, coherence, effectiveness, 
efficiency and sustainability.  

Summary of findings and conclusions  

The ex-post evaluation of the FPA between the EU and indicates that the agreement has clearly 
been of immense mutual benefit to the parties.  

It is highly relevant to the policy objectives of both. The FPA has been effective, with a good 
level of utilisation of the fishing opportunities provided with associated financial and economic 
benefits,  and  has  allowed  the  implementation  of  a  much  strengthened  policy  framework  for 
sustainable fisheries by Greenland (particularly in terms of research and fisheries control).  

The  FPA  has  not  been  as  efficient  as  it  could  have  been,  due  to  the  inclusion  of  some  quotas 
which  are  not  available,  or  not  commercially  viable,  all  of  which  have  contributed  to 
underutilisation of some fishing opportunities.  

Given  the  major  achievements  in  ensuring  the  functioning  of  the  Fisheries  Partnership 
Agreement, and the goodwill and mutual benefits this has generated, all efforts should be made 
to keep it in place by renewal of the Protocol, and to ensure a more sustainable implementation 
of the Fisheries Partnership Agreement in future. 

FPA sectoral support is allocated to actions which contribute strongly to the Greenlandic fishing 
sector viability. 

The parties are therefore recommended to consider establishing a renewed protocol for a period 
of three years from 2013. This will also allow for new measures within the reformed Common 
Fisheries  Policy  to  be  reflected  in  the  design  of  a  new  Protocol,  and  also  allow  adjustment  to 
achieve  coherence  with  other  emerging  EU  policy  areas,  including  the  EU’s  Arctic  Policy  in 
relation to maritime and environmental matters. 

Availability of the report on Europa  

In the process of preparation for publication. 

106

 
 
 
 
 
 
 
 
12 – Internal Market  

Evaluation of the Professional Qualifications Directive 

ABB activities concerned:  

12 02 Single Market Policy 

Timing:  

 5/07/2011  

Background, scope and focus  

The  Professional  Qualifications  Directive  (Directive  2005/36/EC)  is  aimed  at  facilitating 
mobility within the EU: it defines a set of rules allowing professionals qualified in one Member 
Sate to exercise their profession in another Member State. 

The  purpose  of  the  evaluation  was  to  assess  how  the  rules  for  the  recognition  of  professional 
qualifications  work  in  practice  and  whether  there  is  scope  for  improvement.  The  evaluation 
covered  the  various  situations  in  which  the  Directive  applies  (establishment  and  temporary 
mobility),  the  different  regimes  set  out  in  the  Directive  (general  system  and  automatic 
recognition)  and  the  most  relevant  horizontal  provisions  (assessment  of  language  skills, 
recognition of third country diplomas, administrative cooperation and assistance to citizens). 

The  evaluation  of  the  Directive  was  undertaken  with  a  view  to  modernising  the  rules  for  the 
recognition  of  qualifications  and  facilitating  the  mobility  of  workers,  as  announced  in  the 
Communication on the Single Market Act in October 2010. The findings of the evaluation were 
used to prepare the impact assessment for the legislative proposal modernising the Professional 
Qualifications Directive (proposal adopted on 19/12/2011).  

Summary of findings and conclusions  

The main findings of the evaluation are presented below: 

–  The  legislation  on  the  recognition  of  professional  qualifications  has  been  effective  in 
facilitating  labour  mobility  within  the  EU,  in  particular  for  the  professions  benefiting  from 
automatic  recognition.  The  mobility  of  professionals  is  determined  by  a  variety  of  factors 
(language,  family  situation,  education,  remuneration)  and  in  most  cases,  the  recognition  of 
professional  qualifications  does  not  constitute  an  obstacle  but  rather  supports  access  to  the 
profession  in  another  Member  State.  The  experience  gained  by  competent  authorities  in  the 
recognition  of  qualifications  will  be  particularly  helpful  in  tackling  the  decline  of  the  active 
working population in the EU and the increasing demand for highly-qualified jobs. 

–  The acquis on professional qualifications constitutes a pragmatic approach to the differences 

107

 
 
 
 
 
 
 
 
 
 
 
existing between Member States in the regulation of professions and in education systems. The 
procedures set out in the Directive offer guarantees for professionals seeking the recognition of 
their qualifications, including in cases where the profession is not regulated in the host Member 
State.  However,  the  objective  of  establishing  transparent,  uniform  and  quick  recognition 
procedures  has  not  been  achieved:  the  findings  of  the  evaluation  show  that  there  are  still 
concerns about access to information and the complexity and length of procedures, in particular 
under the general system. 

–  The new system introduced for the free provision of services constitutes a first simplification 
for professionals moving on a temporary basis. The outcome of the evaluation shows that there 
is still space for further simplification of the administrative requirements under this regime. 

–  The  legislation  on  the  recognition  of  professional  qualifications  has  not  undermined  the 
quality of professional services. Some provisions of the Directive need to be further examined in 
order  to  reach  the  right  balance  between  the  need  to  facilitate  mobility  and  the  public  interest 
(e.g.  provisions  on  language  skills  to  be  clarified  in  the  view  of  strengthening  patients'  safety; 
minimum  training  requirements  to  be  reviewed  in  order  to  adapt  to  the  evolution  of  the 
professions). 

Availability of the report on Europa :  

http://ec.europa.eu/internal_market/qualifications/docs/news/20110706-evaluation-directive-
200536ec_en.pdf 

Impact and Effectiveness of EU Public Procurement Legislation  

ABB activities concerned:   12 02 Single Market Policy 

Timing:  

 26/06/2011 

Background, scope and focus  

Under the Community rules public sector procurement must follow transparent open procedures 
ensuring  fair  conditions  of  competition  for  suppliers  (apart  from  exempted  sectors  and  for 
purchases below thresholds). 

The  current  legislative  package  of  public  procurement  Directives  approved  in  2004  (DIR 
2004/17/EC  and  DIR2004/18/EC),  aimed  at  simplifying  and  modernising  procurement 
procedures,  for  example  by  facilitating  electronic  procurement  in  the  public  sector  or  by 

108

 
 
 
 
 
 
 
 
 
 
 
 
 
 
introducing  more scope for dialogue between contracting authorities and tenderers.  

After  almost  four  years  from  the  deadline  for  transposition  of  the  above  Directives  into  the 
national legislation (31.1.2006), there is the need to get an interim evaluation of the effects of 
such Directives in fostering the internal market  

The  evaluation  provides  an  overview  of  the  current  functioning  of  EU  public  procurement 
legislation in MS in order to assess the progress that has been made in applying the EU rule and 
to provide indications as to whether the desired objectives have been achieved.  

 This will help the Commission respond to concerns about the relevance or cost-effectiveness of 
the current legislative rules, and identify areas where the Directives are subject to improvement. 
The  evaluation  could  also  serve  as  the  starting  point  for  preparing  the  modernisation  of  EU 
procurement legislation. 

Summary of findings and conclusions  

The  evaluation  draws  on  a  comprehensive  body  of  evidence  and  new  independent  research.  It 
finds that EU public procurement Directives have helped to establish a culture of transparency 
and outcome-driven procurement in the EU. This has triggered competition for public contracts, 
and  generated  savings  and  improvements  in  the  quality  of  procurement  outcomes.  Open  and 
competitive  public  procurement  has  driven  down  costs  by  around  4%,  generating  savings  of 
approximately  €20  billion.  This  far  exceeds  the  costs  generated  by  the  regulatory  framework, 
which are estimated to be €5 billion.  

While responses to the consultation reveal diverging views on the priority that should be given 
to  each  of  the  different  objectives  of  the  reform  there  is  strong  support  for  simplification, 
improving  market  access,  notably  for  SMEs  and  fostering  innovation.  There  is  also  a  certain 
consensus  that  streamlining  the  procedures  and  making  them  more  flexible  is  particularly 
important and that all actors of the procurement world could greatly benefit from it. 

The Commission will draw on the evidence collected in the evaluation, and the insights obtained 
from  stakeholder  consultation,  to  prepare  its  legislative  proposals  before  the  end  of  2011.  The 
priorities for legislative reform will be debated at a landmark public procurement conference in 
Brussels on June 30th. 

Additionally, the evaluation reveals scope for efforts to strike a better balance between the costs 
of the regulatory system and the resulting benefits – particularly for lower value purchases. The 
average  public  procurement  procedure  takes  108  days  and  costs  €28,000  -  3/4  of  which  is 
accounted for by the costs of preparing tenders (5.4 tenders received on average) and 1/4 for the 
contracting  public  authority.  The  worst  performing  Member  States  take  3  times  longer  to 
complete a purchase than the best-performing.  

The  feedback  statement  on  the  Green  Paper  consultation  reveals  that  all  different  stakeholder 
groups  (public  authorities,  business,  civil  society  organisations,  legal  experts  as  well  as 
individual citizens) have a strong interest in the upcoming reform.  

The  findings  from  the  evaluation  and  the  responses  to  the  Green  Paper  provide  a  substantial 
body of  evidence  and  opinion  on  the  functioning  of  existing  legislation,  as  well  as  identifying 
some  scope  for  improvement.  They  will  be  invaluable  in  the  preparation  of  the  legislative 
proposal, which will be published by the Commission before the end of 2011.  

The next step in this legislative reform process will be a Commission-hosted conference on June 
30th.  This  will  give  leading  policy-makers,  procurement  practitioners  and  civil  society  the 
opportunity to discuss the priorities for the legislative reform. The evaluation and responses to 

109

 
 
 
the Green Paper set the scene  

Availability of the report on Europa:  

http://ec.europa.eu/internal_market/publicprocurement/modernising_rules/evaluation/index_en.ht
m 

110

 
 
 
 
 
 
13 – Regional Policy  

Policy analysis on the performance of cohesion policy 2007 – 2013 
Task 1: Policy Paper on Renewable Energy and Energy Efficiency of Residential Housing 
Task 2: Country reports on achievements of cohesion policy 

ABB activities: 

13 04 Cohesion Fund 

Timing:  

 December 2011 

Background, scope and focus  

There are 27 papers delivered by the network of experts whose role is to synthesize evidence on 
the  performance  of  Cohesion  policy  in  the  2007-2013  period  by  examining  the  physical  and 
financial  performance  of  the  operational  programmes  and  evaluation  and  other  evidence 
available.  Every  year  the  expert  network  produces  country  reports  on  the  achievements  of 
cohesion  policy  and  on  a  selected  theme.  All  reports  look  at  the  physical  and  financial 
performance  of  the  operational  programmes  and  analyses  evidence  coming  from  evaluations 
undertaken in the Member States. 

For 2011 the network of experts produced for every Member State a report on the achievements 
of cohesion policy and a policy paper on renewable energy and energy efficiency in residential 
housing.  The  network  also  identified  examples  of  good  practice  in  evaluation  (interesting 
methods,  good  data,  etc.).  The  main  findings  from  all  country  reports  are  summarized  in  the 
synthesis  reports  which  contain  comparative  analysis  and  trends  across  the  EU  (the  synthesis 
report has been delivered in February 2012; therefore there is no specific reference to it in this 
document).  

Summary of findings and conclusions  

The findings emerged from the 27 analyses for Renewable Energy and Energy Efficiency of 
Residential Housing are:  

−  Substantial  amount  of  support  provided  by  national  governments  across  the  EU  for 
renewable  energy,  even  if  difficult  to  estimate  in  practice,  and  far  exceeds  the  resources 
allocated from the ERDF and Cohesion Fund to this; 

−  There is need to clarify the role of Cohesion policy, if it is used to supplement the support 
given by national measures or it has a relevant contribution in funding it; 

−  Scale  of  support  for  increasing  energy  efficiency  in  housing  is  also  difficult  to  estimate; 
however, it is small as compared with the support going to renewables; 

111

 
 
 
 
 
 
 
 
 
 
−  There  is  a  clear  role  for  Cohesion  policy  to  provide  support  to  investment  to  energy 
efficiency, so contributing to social cohesion and territorial balance through helping to even up 
housing  and,  therefore,  living  conditions  across  the  Union,  especially  in  the  EU12  countries 
where  the  need  for  improvements  in  the  energy  efficiency  of  housing  is  considerable,  though 
also in some of the more depressed parts of the EU15 

The findings emerged from the 27 analyses of the achievements of cohesion policy are:  

−  delays in implementing programmes have expanded the funding remaining to be spent over 
the remainder of the period and increased the risk that priority will be given to absorption instead 
of to the most effective way of pursuing development objectives; 

− 
the austerity measures implemented in many countries together with tight budget constraints 
mean that the EU funds are likely to be the major, and in some countries almost the only, source 
of finance for development expenditure over the next few years; 

− 
though  constraints  on  government  expenditure  give  an  incentive  for  the  authorities 
concerned to increase the  effectiveness  of the  spending which does take place, in a number of 
countries there is a question-mark over their capacity to do so in practice; 

− 
the information in the Annual Implementation Reports is , in general, in a form which makes 
it difficult to assess progress in implementing programmes in a meaningfully way and to identify 
what has tangibly been achieved through the expenditure so far carried out; 

− 
the  greater  freedom  given  to  Member  States  with  regard  to  evaluation  has  resulted  in  the 
number  and  scope  of  those  carried  out  varying  widely  across  the  EU,  with  some  countries 
making  little  effort  to  evaluate  policy  outcomes,  but  with  most  undertaking  more  focused 
evaluations than in the past. 

−  at  the  same  time  there  are  signs  of  progress  in  implementing  programmes,  much  more 
evidence  of  achievements  from  the  expenditure  carried  out  and  clearer  indications  that  the 
change  in  approach  to  evaluations  for  the  current  programming  period  is  having  the  intended 
effect.  In  many  Member  States,  therefore,  more  relevant  and  more  detailed  evaluations  of 
specific measures and issues are being carried out. 

These reports are being used to communicate the results of Cohesion Policy programmes. DG 
Regio is using them in its dialogue with the Member States to ensure action is taken to improve 
the quality of current programmes. They are also feeding into the position papers of the future 
policy.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/index_en.cfm#1  

112

 
 
 
 
 
 
Enterprise support: support to SMEs and large enterprises in Italy, including a 
comparison of grants and other financial instruments 

Evaluation related study

ABB activities: 

13 03 European Regional Development Fund and other 
regional operations 

Timing:  

 June 2012 

Background, scope and focus  

This study examined two cases of enterprise support – an investment grant available throughout 
Italy (Law 488) and various SME schemes in the region of Piemonte.  

A particular advantage of the study was the access to several detailed administrative data sources 
from  the  Statistical  Archive  of  Active  Enterprises  (ASIA)  assembled  by  the  Italian  Statistical 
Agency  (ISTAT).  This,  along  with  the  availability  of  a  clear  control  group  in  both  cases 
(narrowly rejected enterprises for Law 488, almost random allocation between different forms of 
support in Piemonte) gives a high degree of confidence in the results. 

In  addition,  a  beneficiary  survey  was  conducted  to  allow  a  comparison  with  more  traditional 
methods of estimating impact. 

Summary of findings and conclusions  

−  Both  monitoring  data  (reporting  82,000  jobs  supported)  and  the  beneficiary  survey 
(reporting  36,000  jobs  created)  proved  to  be  very  poor  estimators  of  actual  jobs  created  under 
Law 488 (12,000). This adds weight to the argument to use counterfactuals (not monitoring or 
beneficiary surveys) to assess impact. 

− 
Impacts under Law 488 are confined to SMEs – large enterprises  are using the  money for 
projects they would have carried out anyhow. Interestingly, this is where the size effect ends – 
results are remarkably consistent and positive over the subgroups of micro, small and medium-
sized enterprises. 

−  Even allowing for firm size, smaller grants are  much more cost-effective than larger ones: 
cost per jobs averaged €79,000 for the smallest grants (less than €125,000), rising to €489,000 
for the largest grants (above €500,000). 

−  The study provides a first indication of the relative merits of soft loans and grants: the loans 
had  a  cost  per  job  around  half  that  of  grants  plus  a  surprisingly  high  impact  on  investment  – 
EUR 5 per euro of gross grant equivalent.  

−  The quality of the jobs created (using productivity and payroll costs as a proxy) is usually 

113

 
 
 
 
 
 
 
 
 
similar to average jobs in the enterprises concerned. And, in the case of the loans, the quality is 
actually slightly higher than average. 

This  report  has  provided  the  evidence  base  for  the  Commission  proposal  to  exclude  large 
enterprises from general ERDF support schemes.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/impact_evaluation_en.cfm#1  

Counterfactual impact evaluation of cohesion policy 
Work package 2: Examples from Support to Innovation and Research 

ABB activities: 

13 04 Cohesion Fund 

Timing:  

 December 2011 

Background, scope and focus  

This  study  examined  publicly  available  beneficiary  and  commercial  databases  in  various 
Member States. The goal was to see where data allowed a counterfactual impact evaluation of 
innovation support. 

Of 10 cases examined, 5 were eliminated from the analysis because of small numbers of ERDF 
innovation projects and hence small sample size (Poland, Slovakia, Slovenia, Flanders, London 
and  Wales).  Spain  was  eliminated  because  of  poor  beneficiary  data,  France  was  eliminated 
because, although beneficiary data was otherwise complete, it was not possible to tell the year in 
which  the  support  had  been  given.  A  counterfactual  analysis  was  therefore  conducted  for  the 
Czech republic (impact on patents) and Germany (impact on R&D investment and a wide range 
of innovative behaviours, as measured by the Community Innovation Survey). 

Summary of findings and conclusions  

The  study  noted  a  range  of  problems  encountered  in  the  beneficiary  data.  It  therefore  made  a 
series of specific recommendations for reporting in the future programming period, including a 

114

 
 
 
 
 
 
 
 
 
 
 
 
 
clear and unique firm identification as well as a description of assistance which makes clear both 
purpose and timing. 

During  the  financial  crisis,  patent  applications  fell  by  63%  in  non-supported  enterprises  in  the 
Czech republic but only 14% in supported enterprises.  

The subsidized firms in the German sample show a median R&D intensity of 6.2%. Without a 
grant from the ERDF, they would have only achieved an estimated R&D intensity of 4.4% - an 
increase of just over 40%.  

In  fact,  the  representative  firm  would  have  had  R&D  expenditure  of  213,000  EUR  without 
ERDF. The grant increases R&D expenditure to about 300,000 EUR. Thus, the treatment effect 
in terms of EUR amounts to 87,000 EUR, on average, for a typical grant size of up to 51,000 
EUR.  Although  there  is  a  margin  of  uncertainty  here,  this  suggests  that  each  euro  of  public 
money is additional and levers in more than half a euro of extra private money.  

In  addition,  R&D  grants  in  Germany  impacted  on  a  wide  range  of  measures  from  the 
Community  Innovation  Survey,  from  product  and  process  innovation  to  initiating  new 
innovative projects (and not abandoning old ones).  

This  effect  was  mostly  additive  when  considering  national  grants,  however  there  was  only  a 
weak correlation with grant size (suggesting smaller grants, repeated in later years if necessary, 
would be particularly effective – further research would be necessary to validate this finding).  

This evaluation provides rigorous evidence on the performance of Cohesion Policy investments 
in  R&D.  It  further  tests  the  use  of  the  counterfactual  method  and  it  helped  define  the 
requirements for publication of beneficiary data in new regulation for 2014 – 2020. 

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/impact_evaluation_en.cfm#2  

Data-Linking and Impact Evaluation in Northern Ireland 

Evaluation related study

ABB activities: 

13 03 European Regional Development Fund and other 
regional operations  

Timing:  

 November  2011 

Background, scope and focus  

115

 
 
 
 
 
 
 
 
 
 
 
This study tracked 480 firms (253 Invest NI clients and 227 non-assisted firms) for which annual 
GVA, turnover and data were available. The goal was to estimate the impact of assistance using 
counterfactuals.  This  was  not  straightforward,  since  supported  firms  can  be  quite  different  to 
non-supported firms (making it more difficult to find similar "matches"). 

Summary of findings and conclusions  

The main findings and conclusions of the study were: 

−  There was a significant positive impact on GVA and turnover.  

− 
Impact  on  employment  was  not  statistically  significant,  but  positive  at  around  2%  per 
annum.  Over  the  period  2001-2007,  employment  in  non-assisted  manufacturing  firms  fell  by 
3.9% per annum - for assisted firms the drop was only 1.9%. There was a similar boost in the 
business service sector, where non-assisted firms grew by about 4.9% per annum, while assisted 
firms grew by 6.9%.  

This  study  provides  a  further  input  to  the  accumulation  of  evidence  in  relation  to  enterprise 
support.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/impact_evaluation_en.cfm#3  

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the 
Cohesion Fund  
Work Package B: Cost-benefit analysis of selected transport projects 

ABB activities: 

13 04 Cohesion Fund 

Timing:  

 20 June 2011 

Background, scope and focus  

This study contains an ex post evaluation of a selection of ten transport projects in EU countries 
benefitting from EU Cohesion and ISPA funding during the period 2000-2006. The method used 
was ex post cost benefit analysis. The study is one of three studies commissioned by DG REGIO 
to assess the effects of the Cohesion Fund and ISPA on economic and social cohesion and draw 

116

 
 
 
 
 
 
 
 
 
 
 
lessons for the future. These studies are package A (overall); package B (transport); and package 
C (environment).  

Summary of findings and conclusions  

All projects delivered value for money. 

The Cohesion Fund contribution was needed to unlock the economic benefits of these projects. 
Benefits  from  these  projects  come  from  8  categories  (travel  time  savings,  vehicle  operating 
costs,  safety  improvements,  carbon  emission,  air  and  noise  reduction  ad  other).  Generally,  the 
utilisation rates found are compatible with the objective to build in sufficient spare capacity to 
give room for growth over a project's lifetime. It was difficult to establish a direct causal link 
between  the  transport  infrastructure  investments  and  the  wider  socio-economic  impacts 
(especially relevant for GDP). 

Cost-benefit analysis (CBA) as a method: The CBA framework used for the appraisal of these 
projects is both consistent with DG REGIO's guidelines and with good practice. However,  the 
scope of the ex ante analysis has often been narrower than what would be ideal. Different EU 
Member States use different appraisal parameters. This in itself is not an issue, but there appears 
to  be  little  similarity  in  some  instances  between  the  values  used  and  other  sources  for  these 
values,  such  as  HEATCO.  The  study  identified  weaknesses  in  the  demand  modelling,  the 
definitions of the counterfactuals and the modelling of network effects.  

Effectiveness  of  CBA  in  decision-making  (supporting  the  project  generation  and  project 
decision of the Member States and the Commission). Whilst cost-benefit analysis is regularly 
used  worldwide,  it  forms  only  part  of  a  broader  decision-making  process  –  even  in  countries 
with a long history of using it. For many of the reviewed projects, the study found that CBA had 
been  used  mainly  to  confirm  that  the  project  offered  value  for  money  and  to  support  the 
application to access EU funding.  

Utility  of  ex  post  CBA  (from  the  point  of  view  of  the  project  promoters,  Member  States 
and  the  Commission).  Ex  post  CBA  can  add  value  to  the  planning  process.  It  adds 
transparency,  strengthens  the  evidence  base  and  provides  feedback  on  the  methods  and 
techniques used to design and appraise the infrastructure. For these reasons, active ex post CBA 
programmes have been in place for some time in the UK and in France, for example.  

Ex post CBA as a tool for evaluating project impacts. Ex post CBA's main strengths are its 
holistic  approach,  that  is,  its  ability  to  consider  a  project  as  a  whole,  as  well  as  its  ability  to 
deliver  an  unambiguous  indicator  of  a  project's  economic  worth.  A  further  strength  of  ex  post 
transport CBA is that it focuses on the direct impacts of the transport project and their associated 
externalities.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpb_en.htm  

117

 
 
 
 
Ex post evaluation of cohesion policy interventions 2000-2006 financed by the 
Cohesion Fund (including former ISPA) 
Work Package C – Cost benefit analysis of environment projects 

ABB activities: 

13 04 Cohesion Fund 

Timing:   May 2011 

Background, scope and focus  

The objective of the evaluation is to carry out ex post cost benefit analyses for 10 environment 
projects  co-financed  by  the  Cohesion  Fund  (including  former  ISPA)  and  to  learn  from  these 
analyses for the current and future programming periods. 

Summary of findings and conclusions  

General conclusions based on the ten Cost-Benefit Analyses: 

1.  Many of the environment infrastructure projects were carried out to meet legal requirements.  

–  Legal compliance in itself does not ensure positive impacts in economic welfare terms – in 
most cases this was achieved at costs that exceeded the benefits that could be monetised  

–  Achieving  positive  ENPV  may  be  impossible  on  project  level;  especially  where  the  main 
objective of projects were to meet regulatory requirements  

2.  In the two cases with objectives other than compliance, the environmental focus was found 
to be relevant to the needs of the region  

3.  Generally, the projects generated positive side effects in terms of environmental awareness 
and institutional learning  

4.  Distributional effects are also important to be considered:  

–  many  projects  benefit  low-income  regions,  indirectly  paid  by  higher  income  regions; 
however  

–  CBA as a tool does not capture such distributional effects.  

How to improve ex ante CBA: 

1.  CBA  thinking  should  apply  to  the  selection  of  alternatives,  prior  to  the  final  design  of  the 
project, and integrated into the decision process.  

2.  Do the CBAs  much earlier in the process. A  solid CBA should precede the final technical 
design of the project, and examine individual components of projects.  

118

 
 
 
 
 
 
 
 
 
 
3.  CBA to be related to the Master Plan context.  

4.  Cost efficiency analysis could be considered for 'need to have' projects.  

5.  Clearer distinction between the financial and the economic CBA.  

Findings on ex post CBA: 

1.  Timing matters: ex post analysis is too early right after project completion  

2.  Starting from individual components is the easiest way to identify the benefit elements  

3.  Concentrate on valorisation of the main benefit elements  

4.  Wider benefits are often important outcomes but which are difficult to quantify  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpc_en.htm  

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the 
Cohesion Fund (including former ISPA) 
Work Package D – Management and Implementation  

ABB activities: 

13 04 Cohesion Fund 

Timing:   April 2012 

Background, scope and focus  

The objective of the evaluation is to assess the building up of management capacity to deliver 
ISP A and Cohesion Fund projects in the 14 countries benefiting most from the Cohesion Fund 
and ISPA in the 2000-2006 programming period (not included are Malta, Cyprus and Croatia).  

Summary of findings and conclusions  

Capacity-building support has been long term and multi-layered. Technical Assistance approvals 
cover  a  broad  spectrum,  allowing  an  adaptive  use  of  resources.    The  impact  of  experience  on 

119

 
 
 
 
 
 
 
 
 
 
 
 
 
capacity is evident in a number of areas, including: administrative structures culture; tools and 
procedures;  managerial,  technical  and  financial  skills;  the  decentralisation  of  responsibilities; 
and better coordination. 

Important steps have already been taken to improve the performance of the CF delivery system, 
such  as  flexibility  in  the  period  for  eligible  expenditure  and  in  the  context  of  the  2007-13 
Regulations. In the context of the post-2013 reform of Cohesion policy, there is an opportunity 
to  review  procedures  for  the  Cohesion  Fund  in  the  light  of  the  reduced  risk  that  exists  in  the 
post-enlargement environment, and as a result of the experience accumulated since 2000. 

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpd_en.htm  

Study on the relevance and effectiveness of ERDF and Cohesion Fund support to 
regions with specific geographical features – islands, mountainous and sparsely 
populated areas 

Evaluation related study

ABB activities: 

13 04 Cohesion Fund 

Timing:   February 2012 

Background, scope and focus  

This study analyzed the extent to which cohesion policy interventions have been appropriate and 
effective in the special territories for the 2000-2006 and 2007-2013 periods. It was carried out 
for both periods 2000–2006 and 2007–2013 and was based on analysis of ERDF/CF impact on 
fifteen NUTS2 regions, and six NUTS3 areas. 

Summary of findings and conclusions  

Main findings: ERDF and CF are appropriate tools for the development of regions with specific 
geographical features, providing the necessary funding, flexibilities and focus to encourage the 
realisation of effective projects. 

A number of common characteristics emerged among these regions and areas, including:  

120

 
 
 
 
 
 
 
 
 
 
 
 
–  an outflow of young people, low natural birth rates and ageing local populations;  

reliance  on  particular  ‘assets’,  including  natural  resources,  climate,  landscapes  and  the 

– 
development of tourism or related activities; and   

–  a  lack  of  ‘critical  mass’  in  terms  of  the  size  of  domestic  markets,  the  number  of  skilled 
workers, access to training, and potential for inward investment.  

The study found that ERDF and CF offer significant added value by:  

–  providing a long-term, stable financial framework in which regions can develop a range of 
projects;  

–  acting as an important catalyst to attract domestic funding for important projects; 

–  providing  a  flexible  tool  to  meet  specific  needs  and  challenges,  while  remaining 
complementary to other domestic and EU Funds; and 

– 

improving strategic focus, stakeholder involvement, and  levels of ‘good governance’. 

The  study  concluded  that  the  two  financial  instruments  could  become  even  more  effective  by 
highlighting  geographical  specificities  at  each  stage  in  the  funding  process  and  not  perceiving 
them  as  ‘problems’.  A  further  finding  was  that  the  policy  should  move  beyond  infrastructure 
development  as  the  main  funding  objective,  instead  concentrating  on  a  limited  number  of 
priorities  that  have  already  proven  advantageous  for  the  region  –  namely  an  asset-based 
approach. In addition, more supple multi-level governance arrangements would be advantageous 
for the success of regional policy in these regions and areas. 

This study has provided evidence on the importance of the sub regional level in Cohesion policy. 
It has contributed to the decisions on the territorial dimension of the future regulation.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/index_en.cfm#3  

Study on the contribution of local development in delivering interventions co-financed 
by the European Regional Development Fund (ERDF) in the periods 2000-06 and 
2007-13 

Evaluation related study

ABB activities: 

13  03  European  Regional  Development  Fund  and other 
regional operations 

Timing:  

 22/12/2011 

121

 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The study assesses the contribution of the local development approach to the effective delivery 
of  Cohesion  Policy.  It  covers  interventions  co-financed  by  the  ERDF  in  the  2000-2006  and 
2007-13 programming periods. 

Summary of findings and conclusions  

Findings from the review of 38 Operational Programmes and the in-depth analysis of 5 regional 
cases  –  in  Andalusia  Region,  Berlin  Metropolitan  Area,  Czech  Regional  Northwest,  Puglia 
Region and West Wales and the Valleys – serve as a base for operational recommendations on 
how and when local development could be used to deliver Cohesion Policy and how to monitor 
and  evaluate  the  effects  of  local  development  interventions  on  economic,  social  and  territorial 
cohesion. 

–  The main contribution of the Local Development Approach in the delivery of interventions 
co-financed by the ERDF is the dimension of policy integration.  

– 
In  adopting  the  local  development  approach,  member  states  and  managing  authorities 
accepted and sometimes encouraged a large amount of variation. LDA comes in all shapes and 
ways and it is not possible to reduce it to a single model. 

–  Main  models  of  Local  development  approach  may  be  sorted  out  in  terms  of  pure  LDA  -  
characterized by small territorial focus,  integrated thematic approach and partnership as a goal -  
LDA  as  a  corrective  in  sectoral  policies  -  characterized  by  variable    territorial  focus,  single 
thematic focus, partnership both as a tool and as a goal - LDA in regional policy - characterized 
by    wide(r)  territorial  focus,    integrated  thematic  approach,  partnership  as  a  tool  and 
selective/strategic partnership (including multi-level governance). 

–  Common features for successful LDA have been showed to be the availability of time and 
the  capacity  issue.  The  continuity  of  the  programmes  from  the  territorial,  thematic  and 
governance  points  of  view  is  of  paramount  importance.  Pure  LDA  approaches  are  likely  to 
deliver  better  results  if  capacity  building  –  particularly  administrative  capacity  –  is  one  of  its 
goals and one of its main types of intervention.  

–  Positive results achieved through the LDA model are mostly in the field of socio-economic 
development,  improving  the  quality  of  life  and  the  attractiveness  of  the  territories,  fighting 
unemployment, creating new business opportunities and new jobs. 

–  The main challenge facing the LDA approach is the aspect of multi-level governance. As 
no clear universal institutional models are available this is one of the main fields in which further 
study seems needed. 

This  study  is  providing  an  input  to  the  policy  discussions  on  local  developments  in  the  new 
regulation for 2014 – 2020.  

Availability of the report on Europa  

http://ec.europa.eu/regional_policy/information/evaluations/index_en.cfm#4  

122

 
 
 
 
14 – Taxation and Customs Union 

Mid-term Evaluation of Fiscalis 2013 

ABB activities: 

14 05 Taxation policy 

Timing:  

 14/07/2011  

Background, scope and focus  

The  Fiscalis  2013  programme  is  the  main  instrument  to  facilitate  the  cooperation  between  tax 
authorities  in  the  EU.  It  seeks,  in  particular,  to  improve  the  proper  functioning  of  the  taxation 
systems in the Internal Market by increasing cooperation between participating countries, their 
administrations and officials.  

The  programme  finances  different  types  of  activities,  notably  (i)  developing  and  operating 
centrally  deployed  and  trans-European  IT  systems  (ii)  supporting  joint  actions  between  tax 
officials (such as seminars, working visits, working groups, steering groups, etc) to facilitate the 
exchange of good practice and (iii) developing common training tools for the use of tax officials 
across Europe. 

The programme primarily targets tax officials of EU Member States, but candidate countries and 
potential  candidate  countries  can  also  request  participation  in  the  programme.  The  financial 
envelope for the programme's duration (2008-2013) is EUR 156.9 million. At present, around 4 
000 officials participate annually in 250 to 275 events organised per year. 

The subject of the mid-term evaluation of the Fiscalis 2013 programme was to provide a review 
of  the  results  obtained  at  the  mid-term  of  the  duration  of  the  programme  in  terms  of 
effectiveness and efficiency, relevance and EU added value. 

Summary of findings and conclusions  

The  evaluation  findings  indicate  that  the  [programme  is  performing  very  well  overall.  The 
contribution of Fiscalis to improve the proper functioning of the taxation systems in the internal 
market is significant. The mid-term evaluation concluded that:  

–  Fiscalis  2013  has  contributed  to  a  large  extent  to  improving  information  exchange  and 
administrative  cooperation.  During  the  period  2008-2011,  Fiscalis  2013  has  supported  the 
existing IT systems. According to the monitoring data on the use of the system and the findings 
on their utility in the midterm evaluation, IT systems have contributed to significantly improve 
the exchange of information in all tax areas which is an important instrument in the fight against 

123

 
 
 
 
 
 
 
 
 
 
 
 
tax  fraud.  The  joint  actions  also  enabled  the  exchange  of  information  and  knowledge  between 
national administrations in different taxation areas: indirect and direct taxes and mutual recovery 
assistance.  

–  Fiscalis was instrumental to ensure a high standard of understanding of the EU law and to 
enable  the  participating  countries  to  acquire  a  better  understanding  of  the  practices  and 
procedures in the different tax systems of the EU and to share their experiences in applying EU 
law.  

–  Fiscalis 2013 has contributed to the development and dissemination of good administrative 
practice between national administrations; 

–  Fiscalis  has  indirectly  contributed  to  taking  the  necessary  measures  for  accession  by  the 
candidate countries and potential candidates; 

–  All  findings  indicate  that  the  programme  management  has  been  optimal  for  achieving  the 
desired results with limited resources; 

–  Fiscalis  offers  high  value for  money.  The  combined  outcomes  of  the  Multilateral  Controls 
and  the  communication  and  information  exchange  systems  generate  revenues  that  are 
significantly higher than the programme’s expenditures; 

–  The  specific  objectives  mentioned  in  the  Fiscalis  Decision  correspond  to  the  current 
challenges  faced  by  the  national  administrations.  The  different  types  of  activities  and  tools 
offered by Fiscalis correspond to existing needs to a high extent; 

–  The  evaluation  has  also  clearly  shown  that  Fiscalis  offers  high  EU  added  value.  Findings 
indicate that the contribution of the programme to the functioning of the tax systems could not 
be  sustained  if  the  programme  was  to  be  discontinued.  The  stakeholders  of  the  programme 
confirmed that many activities necessary to achieve progress in taxation cooperation would not 
have happened at all, or would have only happened much later and/or at a higher cost and with 
less optimal results if the cooperation framework of the programme had not existed;  

–  Another  important  value  added,  despite  the  fact  that  implementing  tax  policies  remains 
predominantly a nationally reserved policy area, is that the programme has been instrumental in 
generating cooperation spirit and networking among Member States and between Member States 
and the Commission.  

The  evaluation  data  has  been  gathered  via  general  surveys  with  the  participants  in  the 
programme  activities,  targeted  surveys  with  stakeholders  with  expertise  in  certain  areas  of  the 
programme, interviews with programme managers in DG TAXUD and national administrations, 
case  studies  and  desk  research.  About  2 000  former  participants  of  the  programme  were 
surveyed.  Furthermore,  targeted  surveys  of  the  programme  coordinators  and  tax  experts  in  the 
participating  countries  were  conducted.  In  addition  the  consultant  used  the  results  of  a  survey 
issued  to  tax  and  customs  officials  in  Member  States  measuring  their  awareness  of  the 
programme and its perceived relevance for their daily work. 

Availability of the report on Europa  

http://ec.europa.eu/taxation_customs/resources/documents/common/publications/studies/fiscalis2
013_mid_term_report_en.pdf  

124

 
 
 
 
Mid-term Evaluation of Customs 2013 

ABB activities: 

14 04 Customs policy 

Timing:  

 14/07/2011  

Background, scope and focus  

The  Customs  2013  programme  was  established  by  Decision  624/2007/EC  as  a  multiannual 
action  programme  for  customs  in  the  EU  to  support  and  complement  action  undertaken  by 
Member States in ensuring the effective functioning of the internal market in the customs field. 
The  programme  finances  different  types  of  activities,  notably  (i)  developing  and  operating 
centrally deployed and trans-European IT systems (ii) supporting joint actions between customs 
officials (such as seminars, working visits, working groups, steering groups, etc.) to facilitate the 
exchange  of  good  practice  and  (iii)  developing  common  training  tools  for  the  use  of  customs 
officials across Europe. 

The  financial  envelope  for  the  programme's  duration  (2008-2013)  is  EUR  323.8  million.  The 
programme  Decision  required  Commission  to  conduct  a  mid-term  evaluation  assessing 
effectiveness and efficiency of the programmes activities, as well as the continued relevance of 
the initial objectives.  

The  mid-term  evaluation  covered  the  activities  carried  out  under  the  programme  from  its  start 
(2008) up to the half-way point of its duration. 

Summary of findings and conclusions  

Based on the positive feedback from its target audiences (both national customs administrations 
and on an in-depth assessment of a sample of activities in selected thematic areas, the evaluation 
concludes that Customs 2013 has been very helpful with a view to the effective functioning of 
the internal market in the customs field:  

–  all  of  the  programme's  objectives  are  relevant  and  important,  corresponding  closely  to  the 
needs  and  problems  of  the  national  customs  administrations,  and  in  particular  the  objective  of 
supporting customs administrations to act as one;  

–  Customs 2013 is very helpful with a view to the effective functioning of the internal market 
in  the  customs  field,  by  contributing  directly  and  significantly  to  support  the  uniform 
implementation  and  application  of  the  EU  customs  legislation.  In  particular,  Customs  2013 
contributed  to  the  harmonisation  of  the  working  methods  of  national  customs  administrations, 
thanks  to  the  joint  actions  and  IT  systems  that  standardise  approaches,  bring  together  national 
customs officials and facilitate the exchange of information and best practices; 

125

 
 
 
 
 
 
 
 
 
 
–  Customs  2013  helps  Member  States  to  protect  the  economic  and  financial  interests  of  the 
EU; 

– 
the programme helps Member States' customs administration to facilitate trade and improve 
competitiveness,  by,  amongst  others,  involving  trade  community  in  joint  actions  or  supporting 
introduction of the Authorised Economic Operator scheme; 

–  Customs 2013 contributes to helping national administrations to increasingly act as if they 
were one administration, in particular through activities aiming to harmonise working methods, 
development  of  IT  systems  and  common  training.  Harmonisation  effects  are  generated  by  the 
vast majority of programme activities whether this is an explicit goal or not; 

– 
the  programme  contributes  significantly  to  strengthening  safety  and  security,  for  example 
through  supporting  development  of  common  risk  criteria  and  security  risk  rules  as  well  as 
facilitating coordination and exchange of information and best practice; 

–  Customs 2013 provides support to candidate countries and potential candidates, in particular 
by providing four of them the opportunity to participate fully in the programme, which supports 
adjustments of their customs legislation, working methods and IT systems to the ones practised 
in the EU; 

– 

the programme is managed well and in a transparent way; 

–  with  regard  to  joint  actions,  their  generally  high  effectiveness  and  the  fact  that  costs  are 
mostly limited to participants’ travel and subsistence expenses mean that the efficiency is very 
high. Further efficiency gains could be achieved by maximising the dissemination of the results 
of C2013 actions within national customs administrations; 

– 
the  programme  provides  significant  EU  added  value  by  supporting  a  number  of  activities 
that  are  necessary  to  ensure  the  proper  functioning  of  the  Customs  Union,  and  that  could  not 
have taken place (or would have been significantly more resource and/or time-intensive) without 
EU coordination and financial support. 

Availability of the report on Europa  

http://ec.europa.eu/taxation_customs/resources/documents/common/publications/studies/customs2
013_mid_term_report_en.pdf 

A retrospective evaluation of the elements of the VAT system 

ABB activities: 

14 05 Taxation policy 

Timing:  

 15/12/2011  

126

 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The evaluation focused on the consequences, in economic terms, of the functioning of the most 
pertinent elements of the current EU VAT system. The need for this assessment was triggered by 
the on-going revision of the EU VAT framework and in particularly by the 'Green Paper on the 
future  of  VAT  –  Towards  a  simpler,  more  robust  and  efficient  VAT  system'  published  in 
December  2010.  This  evaluation  looked  into  the  design  and  implementation  of  certain  VAT 
arrangements, assessing their effectiveness and efficiency in terms of results (meeting objectives 
they were serving) and impacts (direct, indirect, expected and unexpected) they had created. It 
examined their relevance and their coherence with the smooth functioning of the single market 
and the requirement to avoid distortion of competition specified in Article 113 of the TFEU. 

Summary of findings and conclusions  

1.  VAT  exemptions  are  the  most  obvious  and  probably  the  most  economically  damaging, 
creating  significant  distortion  and  deflection  of  trade  of  goods  and  services,  increasing  the 
compliance  and  administrative  costs  while  reducing  productivity,  output  and  international 
competitiveness and generally impeding the Single Market;  

2.  The extensive use of reduced rates creates little desirable effects at a high fiscal cost and 
they  add  to  the  complexity  of  the  system  –  thereby  increasing  administrative  costs,  litigation 
costs  and  compliance  cost.  Zero  and  reduced  rates  can  be  progressive  and  can  be  used  to 
encourage  use  of  socially  desirable  goods  and  services.  But  because  of  the  structure  of  VAT, 
they are rarely well targeted tools to use for either of those aims. They also distort households’ 
spending  patterns,  reducing  welfare.  The  evaluation  showed  that  it  would  be  possible,  in 
principle, to abolish zero and reduced rates of VAT and compensate all households and still have 
revenue left over; 

3.  There  are  far  too  many  differences  in  VAT  procedures  across  the  EU  Member  States, 
which  form  a  source  of  trade  costs  that  hamper  the  development  of  the  internal  market  and 
discourage  cross-border  trade.  It  is  estimated  that  a  10%  reduction  in  differences  in  VAT 
procedures could boost intra-EU trade by as much as 3.7% and GDP by up to 0.4%; 

4.  VAT  requirements  generate  high  administrative  and  compliance  cost  for  businesses  and 
administrations, in particular for the small businesses; 

5.  The level of VAT evasion and fraud is still worrisome - The ‘VAT gap’ – the gap between 
actual VAT revenues and what they would have been with full compliance – is big, estimated at 
an average of 12% of liabilities in 2006. Besides tax avoidance and non-payment arising from 
insolvencies, most VAT is lost through transactions in the shadow economy that are fraudulently 
not  reported,  followed  by  frauds  based  on  reducing  the  reported  level  of  taxable  sales  or  on 
exaggerating claims for refunds of VAT paid on business inputs.  

6.  Contrived  insolvency  fraud  is  also  a  problem,  although  recent  measures  have  reduced  its 
extent. It is unlikely that shadow economy fraud will be reduced by applying lower rates to the 
transactions that are not reported, because it remains attractive to evade the associated income 
tax.  

7.  While most VAT fraud is domestic, cross-border trade is associated with particular forms of 
fraud,  notably  missing  trader  intra-Community  (MTIC)  fraud.  This  arises  because  of  the 
break in the VAT ‘audit trail’ that occurs at the border, and the zero-rating of exports; 

127

 
 
 
8.  A  good  measure  of  the  extent  of  departures  from  a  simple  uniform  VAT  is  the  VAT 
revenue ratio, which tells us that actual VAT revenues in 2008 were only 58.1% of what they 
would  have  been  if  all  consumption  had  been  successfully  taxed  at  the  standard  VAT  rate,  or 
85.4%  of  what  they  would  have  been  if  all  consumption  by  households  had  been  successfully 
taxed  and  no  government  consumption  had  been  taxed.  This  shows  that,  taken  together, 
exemptions, reduced rates and various forms of non-compliance significantly reduce the amount 
of VAT that is raised; 

9.  Harmonising procedures and limiting differences in VAT rates, and more generally reducing 
compliance costs were considered to be desirable and recommended. 

Availability of the report on Europa  

http://ec.europa.eu/taxation_customs/resources/documents/common/publications/studies/report_e
valuation_vat.pdf 

128

 
 
 
 
 
15 – Education and Culture   

Ex-post evaluation of 2010 European Capitals of Culture and Interim evaluation of selection 
and monitoring procedures of ECOC 2010-2016 

ABB activities: 

15 04 Developing cultural and audiovisual cooperation in 
Europe 

Timing:  

 12/08/2011 

Background, scope and focus  

1. Evaluations subject 

The general objectives of the European Capital of Culture (ECOC) are to: "highlight the richness 
and  diversity  of  European  cultures  and  the  features  they  share,  as  well  as  to  promote  greater 
mutual acquaintance between European citizens".1  

The  current  evaluation  covers  two  subjects:  1)  Ex  post  evaluation  of  2010  ECOC;  2)  Interim 
evaluation of selection and monitoring procedures of ECOC 2010-2016 

2. The scope of evaluations  

Subject 1 covered the three cities designated as ECOCs in 2010: Essen for the Ruhr, Pecs and 
Istanbul.  Each  of  the  designated  cities  created  a  specific  cultural  programme  for  the year. The 
Cultural Programmes were required to have a European dimension, based principally on cultural 
co-operation. Each of the cities received Melina Mercouri Prize of up to €1.5m from Strand 1.3 
of the Culture Programme (2007-13). 

Subject  2  assessed  the  implementation  of  new  selection  and  monitoring procedures  introduced 
by the current Decision. It therefore  covered the selection procedures that had been applied to 
the  designation  of  the  2013-16  titles  and  the  monitoring  and  EU  co-financing  procedures  that 
had been applied to the 2010-15 titles. 

Summary of findings and conclusions  

For subject 1 the following results are to be mentioned: 

Relevance: The ECoC Action has been consistent with and relevant to the objectives of Article 
167 of the Treaty.   

Effectiveness: All three ECoC were successful in implementing cultural programmes that were 

1 Decision No 1622/2006/EC of the European Parliament and of the Council of 24 October 2006, establishing 
a Community action for the European Capital of Culture event for the years 2007 to 2109. 

129

 
 
 
 
 
 
 
 
 
 
                                                 
more extensive, innovative, diverse and high-profile than would have been their cultural offering 
in  the  absence  of  ECoC  designation.  All  three  cultural  programmes  were  of  European 
dimension, through highlighting European themes, including events featuring artists of European 
significance  or  transnational  co-operation,  and  attracting  European  tourists  and  audiences.  All 
three ECoC stimulated investments in cultural and other infrastructure that would otherwise not 
have taken place. 

Efficiency:  The  2010  ECoC  featured  different  governance  arrangements  and  experienced 
different degrees of efficiency. In Essen for the Ruhr, the delivery agency was largely successful 
in  balancing  the  requirements  for  high-profile,  international  events  with  regional  development 
objectives. In Pécs, some delays occurred due to a lack of clarity between the relative roles and 
responsibilities of different organisations, as well as by changes in key personnel; however this 
didn't  affect  the  overall  implementation  of  the  programme.  In  Istanbul,  whilst  the  agency 
succeeded  in  implementing  a  very  extensive  cultural  programme  and  marketing  campaign, 
certain  features  of  the  governance  arrangements  proved  problematic  and  led  to  the  overall 
impact of the ECoC being smaller than originally anticipated. 

The EU funding did not significantly influence the cities decision to apply for ECoC designation 
and formed a modest proportion of the total expenditure on the cultural programme in all three 
cities.  In view of the modest amount provided from the EU budget, the ECoC designation has a 
very effective leverage effect. 

Sustainability:  Across  all  three  ECoC,  there  is  evidence  of  new  cultural  activities  that  will 
continue  beyond  the  title  year.  However,  the  evidence  for  sustained  improvements  in  cultural 
governance varies; in Essen for the Ruhr, a number of ambitious long-term goals have been set 
and responsibilities have been transferred to regional partners. In Pécs, two legacy bodies have 
been  created  to  manage  new  cultural  facilities,  though  the  municipality  has  yet  to  create  its 
overarching  “umbrella”  structure  to  support  cultural  operators  across  the  city.  In  Istanbul,  the 
governance  model  introduced  by  the  ECoC  will  come  to  an  end  once  the  agency  ceases 
operation and it is not certain that stakeholders will coalesce around a shared strategy. 

The purpose of the selection and monitoring procedures (scope of evaluation for the subject 2) 
can be seen as being, first, to enable the selection of ECoC in an efficient manner and, second, to 
strengthen the quality of the applications and of the eventual cultural programmes. Therefore the 
main results of this evaluation are presented in two main headings relating to the “efficiency of 
the processes” and the “effect on the quality of ECoC”. 

Efficiency of the processes 

1.  The  overall  efficiency  of  the  application  and  selection  process  is  satisfactory.  There  are 
inevitable variations between the experiences in different countries, but no significant problems 
or delays were identified.  

2.  The documentary guidance provided to candidate cities, Member States and panel members 
is thorough and strong on administrative process. 

3.  The application form, whilst enabling the ECoC to provide the necessary information for the 
selection panel to make its decision, does not allow applicants to present their bids in the clearest 
and  most  attractive  format  and  leads  to  the  submission  of  an  unnecessarily  large  volume  of 
material. 

4.  The ECoC criteria are clear in principle and have generally been understood and applied by 
applicants  and  the  Panel,  though  some  selected  ECoC  have  not  performed  particularly  well  at 
application stage against one or other of the categories of criteria and many cities still struggle to 

1 Compared to the previous procedure under which it considered only those applications nominated by 
the Member State in question – usually only one from each Member State. 

130

 
 
                                                                                                                                            
achieve an appropriate balance between local and European components.  

5.  The  Panel  is  mostly  operating  effectively  and  has  been  strengthened  by  the  mixture  of 
national and European appointees and by the fact that the European appointees hold a majority, 
with the chair being one of the European members; its role has also been reinforced by virtue of 
the fact that it considers all applications.1 However, its credibility has been weakened by a small 
number of EU members either failing to attend meetings or giving the impression of being ill-
prepared. 

6.  Providing  EU  funding  in  the  form  of  a  prize  rather  than  a  traditional  grant  has  been 
welcomed by ECoC due to the reduced administrative burden and increased flexibility in the use 
of the funding. The "motivational effect" of the Prize could be increased by requiring cities to 
develop their applications and cultural programmes (up to the second monitoring point) without 
reference to the Prize. 

Effect on the quality of ECoC 

1.  Candidate cities value the ECoC brand highly and recognise the benefits of applying as well 
as winning. As a result they are prepared to invest significant resources in their applications. 

2.  The  open  competition  (within  Member  States  listed  in  the  Order  of  Entitlement)  has 
increased interest in the ECoC, ensured an equitable distribution of ECoC across Member States 
and helped generate a high number of credible applications. However, the sustainability of the 
process beyond 2019 in its current format needs to be given careful thought. 

3.  The  selection  process  is  generally  held  to  be  fair  and  transparent  and  has  enabled  the 
selection  of  credible  candidates  in  every  Member  State,  in  part  due  to  the  two-stage  process 
which  enables  applicants  to  improve  their  applications  on  the  basis  of  expert  advice  received 
from the Panel. 

4.  DG  EAC  provides  guidance,  advice  and  support  that  is  highly  valued  by  panel  members, 
Ministries  and  designated  ECoC.  Ministries  and  cities  would  welcome  additional  informal 
support  and  there  may  be  practical  ways  by  which  this  can  be  provided  without  posing  a 
substantial additional burden on DG EAC. 

5.  The monitoring process has played a key part in strengthening the ECoC, in particular their 
focus  on  the  European  dimension.  The  new  informal  "post-designation  meeting"  and  informal 
visits by panel members have proved a valuable complement to the formal monitoring meetings 
and reports. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/education_culture/evalreports/index_en.htm 

131

 
 
 
 
 
External Evaluation of the European Institute of Innovation and Technology 

ABB activities: 

15 02 Lifelong learning, including multilingualism 

Timing:  

 May 2011 

Background, scope and focus  

1. Evaluation subject 

Article 16 of the EIT Regulation asks for an examination of "…how the EIT fulfils its mission, 
covering  all  activities  of  the  EIT  and  the  KICs  and  assessing  its  added  value,  impact, 
effectiveness,  sustainability,  efficiency  and  relevance  of  the  activities  pursued  and  their 
relationship  and/or  complementarity  with  existing  national  and  Community policies  to  support 
higher education, research and innovation". However, the EIT was set up only in 2008, which 
clearly limited the scope for this first EIT evaluation. The ToR requested the evaluator to “cover 
the set up of the EIT, its operationalisation and initial stages during the period 2008–10, and [to] 
seek  mainly  to  assess  the  preliminary  impact  of  the  EIT  and  reflect  on  “lessons  learned”  to 
support  the  implementation  of  the  future  activities  of  the  EIT”.  In  2011  the  contract  was 
amended to include an impact assessment study and a detailed assessment of the effectiveness 
and  efficiency  of  the  negotiations  between  the  EIT  and  the  Knowledge  and  Innovation 
Communities  leading  to  the  signing  of  Framework  Partnership  Agreements  and  Annual  Grant 
Agreements.  It  was  foreseen  that  the  evaluation  results  would  provide  an  input  into  the 
development  of  the  EIT’s  Strategic  Innovation  Agenda  (SIA).  The  overall  budget  was  EUR 
268,556.77 (+ EUR 25,750.00 on reimbursable expenses). Further evaluations will follow every 
5 years after the entry into force of a new financial framework. 

2. Scope of evaluation 

Due  to  the  timing  of  the  evaluation  its  coverage  could  not  fully  meet  the  requirements  set  in 
Article 16 of the EIT Regulation, for details see point 1. 

Summary of findings and conclusions  

The  EIT  has  broadly  met  its  key  operational  objectives.  In  terms  of  headline  activities,  the 
implementation of the EIT remains on track and was even ahead of schedule in the case of the 
selection  of  the  first  KICs.  Given  the  constrained  timescale  and  the  novelty  of  the  EIT  this 
represents  a  significant  achievement.  Overall,  the  application  and  selection  process  for 
designating  the  first  KICs  was  appropriate,  fair  and  transparent.  The  efficiency  of  the  process 
was broadly satisfactory, led to a good number of bids of a satisfactory standard. Negotiations 
with the designated KICs were a protracted process which took longer than anticipated, largely 

132

 
 
 
 
 
 
 
 
 
 
 
because the complexity of the process had been underestimated. The negotiation process created 
tensions between the parties involved and was not efficiently undertaken. A number of tensions 
and  challenges  have  been  evident  also  in  the  process  of  establishing  the  EIT.  Inter  alia,  these 
relate to the ‘learning by doing’ approach adopted by EIT staff, to the different expectations of 
partners in the process as to the balance between flexibility and guidance, to the approach taken 
to  the  ‘simplification’  of  EU  rules  and  regulations  and  to  relationship  between  the  involved 
partners. The Governing Board has taken a particularly operational role in the initial stages of 
the EIT. Ambitious timescales and perceptions of shifting requirements have contributed to the 
tensions reported.   

Availability of the report on Europa  

http://ec.europa.eu/dgs/education_culture/evalreports/education/2011/eitreport_en.pdf  

Interim Evaluation of External Cooperation Agreements in Higher Education, 
Training and Youth with the US and Canada 

ABB activities: 

15 02 Lifelong learning, including multilingualism 

Timing:  

 April 2011 

Background, scope and focus  

The general objectives of both Agreements are to: 

promote mutual understanding between the peoples of the European Union and the United 

- 
States / Canada including broader knowledge of their languages, cultures and institutions; 

- 
improve  the  quality  of  human  resources  [US]  /  human  resource  development  [Canada]  in 
both the European Community and the United States / Canada, including [US] / by facilitating 
[Canada] the acquisition of skills required to meet the challenges of the global knowledge-based 
economy. 

The  agreements  span  the  period  of  2006-13  and  support  (a)  Joint/double  degree  projects,  (b) 
Joint mobility projects, (c) Policy oriented dialogue and projects, (d) an Alumni Association, and 
(e) the Schuman-Fulbright. 

The EU-Canada Agreement also provides support to Youth Actions. 

The actions funded under both Agreements are open to the 27 EU Member States, and Canada 
and the United States respectively. 

133

 
 
 
 
 
 
 
 
 
 
 
 
Financing  of  bilateral  EU-Canada  and  EU-US  co-operation  activities  has  to  be,  to  the  extent 
possible, on the basis of an overall matching of funds between the Parties. 

For the eight-year period 2006-13, the European Commission allocate 15,6M€ to the EU-Canada 
Agreement, and 43,4M€ of operational credits to the EU-US Agreement. 

The previous evaluation was delivered in March 2005. 

The  evaluation  covered  the  first  four  years  of  implementation  of  all  actions  included  in  the 
current cooperation agreements with the US and Canada (2006-2013). 

Summary of findings and conclusions  

According  to  the  evaluation,  the  Agreements  are  generally  speaking  relevant  to  the  needs, 
effective in terms of the objectives set, and highly cost-effective. Main points of improvement 
are  opportunities  for  policy  discussion  and  exchange,  the  Youth  cooperation  (Canada). 
Furthermore,  the  publicity  for  the  Agreements  and  the  dissemination  of  the  results  could  be 
improved, especially in the EU and in Canada. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/education_culture/evalreports/education/2011/extreport_en.pdf  

Interim Evaluation of the Lifelong Learning Programme (LLP) 2007-2013 

ABB activities: 

15 02 Lifelong learning, including multilingualism 

Timing:  

 18 February 2011 

Background, scope and focus  

The Lifelong Learning Programme (LLP or the Programme) is based on articles 149 and 150 of 
the Treaty establishing the European Community. These articles stipulate that "The Community 
shall  contribute  to  the  development  of  quality  education  by  encouraging  cooperation  between 
Member  States  and,  if  necessary,  by  supporting  and  supplementing  their  action,  while  fully 
respecting  the  responsibility  of  the  Member  States  for  the  content  of  teaching  and  the 

134

 
 
 
 
 
 
 
 
 
 
 
 
 
organisation  of  education  systems  and  their  cultural  and  linguistic  diversity"  and  "The 
Community shall implement a vocational training policy which shall support and supplement the 
action of the Member States, while fully respecting the responsibility of the Member States for 
the content and organisation of vocational training." 

The evaluation is carried out mid-term and covers the first three years of implementation (2007 – 
2009). It covers all aspects of the evaluation subject. 

Summary of findings and conclusions  

–  Specific objectives are highly relevant to the EU strategic objectives, to policy developments 
in  the  MS,  and  to  the  needs  of  the  beneficiaries.  However,  there  are  too  many  objectives,  and 
often missing links between the different levels of objectives. 

–  The policy instrument chosen (i.e. a spending programme) is the most appropriate and cost-
effective instrument to address the objectives as defined for the EU intervention. 

–  The  integration  of  the  three  predecessor  programmes  into  LLP  increased  efficiency  and 
effectiveness  of  the  programmes,  as  important  synergies  were  exploited  –  particularly  in  the 
areas  of  administration  and  promotion  of  the  programme.  Furthermore,  the  integration  made 
possible  to  avoid  overlaps  between  the  different  sectors.  However,  the  programme  is  highly 
complex with its around 70 actions, and the successor programme should be better focused. 

–  The Programme is well achieving its objectives. 

Judging from the results of the evaluated period, there is a certain risk – mainly relating to 

– 
under-funding – that some of the programme's quantified targets will not be reached. 

–  The management structure works well. 

–  While the implementation of dissemination activities is satisfactory, less attention is paid to 
exploitation  of  results  and  mainstreaming  into  policy,  which  puts  a  limit  to  effectiveness.  To 
address this issue the creation of an ‘observatory’ for the successor programme is recommended. 

–  Further efforts should be made to strengthen the links between the different Education and 
training sectors, in order to improve the Programme's contribution to making lifelong learning a 
reality. 

It  is  recommended  to  provide  a  better  link  between  formal,  informal  and  non-formal 

– 
learning. 

–  The actions of mobility as well as the Grundtvig programme have insufficient funding. Their 
financial expansion would allow more people to benefit from mobility. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/education_culture/evalreports/education/2011/llpreport_en.pdf  

135

 
 
 
 
 
 
Evaluation of Preparatory Actions and special events in the field of sport 

ABB activities: 

15 05 Encouraging and promoting cooperation in the field of 
youth and sports 

Timing:  

 July 2011  

Background, scope and focus  

The  entry  into  force  of  the  Lisbon  Treaty  (TFEU)  in  December  2009  conferred  direct 
competence to the EU in sport for the first time. Following from this, a budget line was granted 
for three years of Preparatory Actions (from 2009-2011) and special events in the field of sport, 
with  the  general  objective  of  preparing  for  future  incentive  measures  in  this  area.  The  total 
budget for 2009-2011 amounted to EUR 25.5m and was used to fund transnational collaborative 
projects and non-commercial sport events of major importance (EUR 8.5m each) in addition to 
several studies, surveys and conferences on relevant topics (EUR 2.1m).  

The  evaluation  assesses  the  relevance,  effectiveness,  efficiency  and  EU  added  value  of  the 
activities carried out in 2009 and (to a minor extent) 2010. 

Summary of findings and conclusions  

The 30 transnational projects funded in 2009 and 2010 facilitated co-operation between a wide 
array of sport organisations, local authorities, universities / research institutes and other actors. 
At  both  programme  and  project  level,  support  for  transnational  projects  can  be  considered 
relevant  to  the  sport  provisions  of  the  Lisbon  Treaty  and  other  policies.  The  projects 
demonstrated EU added value on a number of levels, in particular through addressing issues with 
a  cross-border  element  and  /  or  challenges  for  which  no  one  Member  State  has  a  complete 
solution.  The  projects  achieved  considerable  success  in  meeting  their  own  objectives  and 
developing the European dimension in sport (a key aim of the Lisbon Treaty), in particular with 
regard  to  building  and  strengthening  networks  and  kick-starting  co-operation  between 
organisations working on sport around Europe. This co-operation appears set to continue. 

Unfortunately,  the  one-year  duration  of  the  projects,  while  ideally  suited  to  the  experimental 
nature  of  Preparatory  Actions,  somewhat  limited  their  ability  to  translate  project  outputs  (e.g. 
best  practice  guides)  into  tangible  lessons  for  policy  makers.  Increasing  the  duration  would 
likely  increase  the  impact  that  individual  projects  are  able  to  make.  Similarly,  all  contractual 
responsibility for each network was placed upon one co-ordinating organisation. While simpler 
from an administrative view, this sometimes incentivised co-ordinating organisations to play too 
prominent  a  role  in  designing  and  rolling  out  network  activities,  reducing  the  potential  for 
synergies and the benefits of funding to be spread across a network. The Education, Audiovisual 
and Culture Executive Agency, which is likely to administrate future incentive measures, should 

136

 
 
 
 
 
 
 
 
 
 
use its competence and expertise to encourage network partners to spread responsibilities more 
evenly across networks and Member States.  

The  budget  available  for  the  transnational  projects  was  sufficient  to  test  a  limited  number  of 
network  themes,  types  and  sizes  while  employing  a  robust,  but  not  overly  restrictive  selection 
process. In future, a larger budget would enable funding to be spread more effectively across a 
wider number of subject areas. 

Support  for  five  non-commercial  sport  events,  which  spanned  competitions  aimed  at  youth, 
disabled athletes and mainstream sport, made considerable progress in developing the European 
dimension in sport, in addition to making strides in health-enhancing physical activity and social 
inclusion. Clear evidence of progress in these areas was available for specific activities that took 
place alongside the main sport competitions (e.g. promoting sport among school children), but 
the  Commission  was  unable  to  use  a  selective  tendering  process  in  order  to  privilege  such 
activities, and to ensure that event organisers emphasise the EU’s involvement. For future event 
support,  it  is  therefore  recommended  that  the  Commission  set  up  a  more  robust  tendering 
process, with specific award criteria and competitive tendering.  

The  eight  studies,  surveys  and  conferences  which  received  funding  have  largely  fulfilled  their 
role in providing the Commission and other actors with needed policy support in across a range 
of cross-cutting issues with a transnational element. Support for such activities should continue. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/education culture/evalreports/indexen.htm  

Youth in Action – Interim evaluation 

ABB activities: 

15 05 Encouraging and promoting cooperation in the field of 
youth and sports 

Timing:  

 February 2011  

Background, scope and focus  

The Youth in Action Programme (YiA) aims at promoting young people's citizenship (including 
European),  solidarity,  mutual  understanding,  tolerance,  mobility,  non-formal  learning  and 
intercultural  dialogue;  and  at  developing  the  quality  of  youth  work  in  Europe.  Furthermore,  it 
encourages the inclusion of all young people regardless of their educational, social and cultural 
background. The Programme is implemented in the 27 EU member states and beyond, and has a 

137

 
 
 
 
 
 
 
 
 
 
 
 
budget of 885 MioEUR (EUR 27) over a period of 7 years (2007-2013). 

The evaluation assesses all Actions and sub-Actions of YiA, with a focus on the decentralised 
Actions  (which  represent  81%  of  the  budget  of  the  Programme).  It  takes  into  account 
management and content-related aspects over the period 2007–2009.  

Summary of findings and conclusions  

1.  PROGRAMME DESIGN, RELEVANCE, ADDED VALUE 

The  intervention  logic  has  improved  in  comparison  to  the  previous  Youth  Programme,  but  it 
would be easier to understand if it was simplified by grouping and revising the objectives. The 
YiA  Programme  is  well  embedded  in  the  wider  EU  strategies  and  policy  objectives  and  is 
complementary to other EU programmes and to national initiatives.  YiA is relevant to the needs 
of young people, youth workers and youth organisations. Increasing its already existing focus on 
employability would further improve a future Youth programme's contribution to EU strategies 
and its relevance to young people's needs. The added value of the Programme is mainly seen in 
that it facilitates international cooperation in the field of Youth. 

2.  EFFECTIVENESS 

YiA is successfully achieving its objectives addressed at young people and youth workers, and it 
actively  targets  young  people  with  fewer  opportunities,  which  is  a  unique  feature  of  the 
Programme compared to other programmes. Although YiA only reaches a limited proportion of 
the total youth population, and there is room for improving its visibility, the reach is wider due 
to  the  support  to  youth  support  systems  and  youth  workers,  who  will  then  apply  the  methods 
acquired also when working with young people outside a YiA project. YiA also contributes to 
the  achievement  of  EU  Youth  policy  objectives.  The  influence  of  the  Programme  on  national 
legislation  is  limited,  but  is  more  promising  in  relation  to  national  youth  policy  and  practice. 
Furthermore,  the  Programme  has  a  sustainable  impact  on  participants  and  youth  workers  in 
terms  of  participation  in  events  after  YiA,  continued  work  for  NGOs,  and  increased  mobility 
afterwards.  If  the  EU  did  not  have  a  Youth  programme,  it  is  felt  that  the  type  of  activities  it 
supports would probably not be financially sustained by other sources. The Youthpass certificate 
is a useful tool in defining and scoping learning activities and objectives. 

3.  EFFICIENCY 

The Programme has a high absorption rate. The management costs of the Programme as a whole 
appear relatively high but are justified by the character of the Programme. The division of the 
budget  over  the  actions  is  appropriate.  Youth  organisations  and  National  Agencies  both 
experience  a  relatively  high  administrative  burden,  although  management  tools  have  been 
considerably improved. There is high participant satisfaction (which compares very positively to 
the  level  of  satisfaction  of  participants  in  other  (national)  youth  programmes  and  the 
implementation structures are considered to be efficient.   

Availability of the report on Europa  

http://ec.europa.eu/dgs/education_culture/evalreports/index en.htm  

138

 
 
 
 
 
16 – Communication   

Interim evaluation of Euranet – The European Radio Network 

ABB activities: 

16 02 Communication and the media 

Timing:  

 21/06/2011  

Background, scope and focus  

A  mid-term  external  evaluation  was  launched  in  November  2010  on  the  initiative  of  the 
operational unit in DG COMM. The main goal of this study was to assess the efficiency of the 
network  by  identifying  strengths  and  weaknesses  of  the  implementation  of  Euranet  and  to 
provide  an  overview  of  the  results  and  the  impact  obtained  by  Euranet  until  now  as  well  its 
effectiveness in reaching its operational objectives. 

Summary of findings and conclusions  

The evaluation report showed that a main added value of the network was the increased level of 
production and broadcasting of news related to Europe and the production of new programmes, 
which  is  unlikely  to  be  sustainable  without  funding.  It  was  also  found  that  Euranet  has  led  to 
enrichment  of  editorial  coverage  –  more  in-depth  coverage  of  EU  affairs,  comparative 
approaches, journalistic capacity-building, and mutual learning. 

However, the evaluators identified a number of issues related in particular to (i) the development 
of  the network;  (ii)  the  content  relevance  of  a  significant  share  of  the  production,  and  (iii) the 
uneven  distribution  of  production  and  broadcast  from  a  language  and  audience  point  of  view. 
The report also pointed out the governance problems that impacted the overall co-ordination of 
the network and its consolidation in the period 2009-2010, and caused one of the network co-
leaders  to  leave  in  March  2009,  while  noting  some  recent  improvements  as  reported  by  the 
members. 

Development of the network – The evaluators criticised the lack of dynamism of the network 
and the small increase in the number of languages covered: 12 languages used in air broadcast at 
the end of 2010, instead of 17 indicated in the contractor's offer, and a slow progression in the 
number  of  members,  with  only  two  more  members  compared  to  the  initial  composition  of  the 
network. 

Volume  and  content  relevance  of  the  production  –  the  evaluators  identified  eligibility 
problems  as  regards  almost  two  thirds  of  the  production  of  one  member  (representing  around 
one third of the network's total production). Due to consistent over-production – not subject to 
additional funding though – the network as a whole was found to meet the contract's objective of 
production when excluding the not-eligible production. According to the mapping drawn up on 

139

 
 
 
 
 
 
 
 
 
 
the basis of one week production, at least one  half of the programmes, not-eligible production 
excluded, was found to have a very limited link, if any, with the EU, and only 7% of the content 
was found to respond to the 'compared-views' approach – confronting points of view of different 
EU Member States, citizens, etc. – which was a major focus of the project. 

Production  ratio  by  language,  member  radio,  and  country  –  the  ratio  production/language 
and production/member was found imbalanced compared to the number of listeners speaking the 
respective  language  and  listening  to  the  respective  radios.    For  example,  a  significant  share  of 
the  production  was  in  English  whereas  there  was  no  member  radio  from  English-speaking 
member states until March 2011.  

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/results.do  

Evaluation of the Management Partnership’s Actions within the scope of the 2008-
2010 Communication Plans in Portugal 

ABB activities: 

16 03 "Going Local" communication 

Timing:  

 22/09/2011  

Background, scope and focus  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication  COM(2004)196  and  Commission  Decision  C(2004)4841.  Guidelines  for  the 
evaluation of MP set the evaluation goal to assess: 

its effectiveness in terms of increased impact of the Commission's communication priorities 

– 
at Member State level and achieved goals in reaching the intended public and yielding the 
desired results; 

– 
its added value and its efficiency in bringing about synergies and an increased coordination 
between the Member State and the Commission as well as in its mechanisms and procedures of 
functioning. 

Summary of findings and conclusions  

140

 
 
 
 
 
 
 
 
 
 
 
 
 
The results of the evaluation show that 2008-2010 Management Partnership in Portugal reached 
very positive results. Positive and very positive assessments are predominant in Communication 
Plan of each evaluated year.  

Target  group  -  The  initial  goals  of  the  Management  Partnership  as  to  target  audiences  (in 
quantity and diversity) have been met very positively. The number of citizens reached (directly 
and / or indirectly) by the Actions (around 1.8 million) is considered excellent. 

The diversity of audiences reached during the term of the Management Partnership is also to be 
considered  as  very  positive,  although  with  areas  for  improvement  with  regard  to  the  common 
citizen. The impact on the school community (including teachers) is considered excellent.  

Results and Added value - Quantitative and qualitative results of the 2008-2010 Management 
Partnership  are  considered  net  gains  in  terms  of  value  generated  in  comparison  with  the 
activities undertaken by the partners before its existence. 

In the absence of the Management Partnership, it would not have been possible to run most of 
the 36 Actions reported here. This is due to the fact that many contractors submitted proposals 
only because they relied on the institutional framework of the Management Partnership as a way 
of  valuing  their  image,  their  skills,  their  capacity  for  innovation,  exploring  new  markets  and 
audiences, etc.  

The  expected  outcomes  of  interventions  among  target  audiences  are  judged  to  have  been 
attained. The average quality level of the Actions is very high. Only very few Actions registered 
levels  of  implementation  below  expectations,  and  were  subjected  to  corrective  actions  by  the 
Intermediate Body. 

Cooperation  between  MS  and  EC  and  functioning  of  MP  -  The  2008-2010  Management 
Partnership  proved  very  efficient  in  generating  synergy  and  increased  levels  of  coordination 
between the European Commission and Portugal. 

The  Management  Partnership  corresponded  to  a  new  reality  on  its  creation,  mechanisms  were 
established  (which  proved  to  be  effective)  to  coordinate  the  activities  of  the  different  partners 
involved to avoid redundancy in their performances. 

The coordination between partners generated Communication Plans, Operations and Actions that 
led  up  to  their  target-groups  content/activities/initiatives  of  great  value  and  quality  without 
redundancy  to  the  other  activities  of  European  information  usually  carried  out  by  the  each 
partner. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/results.do  

141

 
 
 
 
 
Ex-post Evaluation of the Management Partnership 
concluded between the Republic of Austria and the European Commission 
for the period of 2008-2010 

ABB activities: 

16 03 "Going Local" communication 

Timing:  

 30/12/2011 

Background, scope and focus  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication  COM(2004)196  and  Commission  Decision  C(2004)4841.  Guidelines  for  the 
evaluation of MP set the evaluation goal to assess: 

its effectiveness in terms of increased impact of the Commission's communication priorities 

– 
at Member State level and achieved goals in reaching the intended public and yielding the 
desired results; 

– 
its added value and its efficiency in bringing about synergies and an increased coordination 
between the Member State and the Commission as well as in its mechanisms and procedures of 
functioning. 

Summary of findings and conclusions  

The  results  of  the  evaluation  support  the  conclusion  that  the  activities  launched  by  the 
Management Partnership have made a substantial contribution to improve EU communication in 
Austria.  

Target group - Individual projects were able to address special interest groups and multipliers 
such  as  teachers  and  local  councillors  and  in  this  way  provide  interlinking  and  qualified 
information services. The projects realised in the period 2008-2010 and beyond reached a large 
number of people and raised interest in EU topics. Without the Management Partnership these 
projects would never have taken place.  

Results  and  Added  value  -  If  the  Management  Partnership  were  to  disappear  in  Austria,  a 
major  share  of  EU  communication  activities  would  not  take  place.  Besides,  the  Management 
Partnership attempts to incentivise organisations and implement projects in co-operation in the 
hope that some activities will then be taken over by these organisations. In this sense, one can 
speak of a clear additional value of the Management Partnership. 

Three  projects  were  evaluated  in  more  detail,  and  for  this  purpose  the  participants  of  these 
projects  were  interviewed.  Their  feedback  suggests  that  the  respective  activities  (teachers’ 
conference, visit of the local councillors to Brussels, trips of young journalists) have generated a 

142

 
 
 
 
 
 
 
 
 
 
considerable  increase  in  knowledge.  Particularly  projects  with  a  dialogue-oriented  design 
succeeded  in  involving  the  participants  and  in  providing  an  authentic  image  of  the  European 
Union. 

Cooperation between MS and EC and functioning of MP - The interviews show that the co-
operation  within  the  Management  Partnership  (representatives  of  the  EC,  the  EP,  the  Federal 
Chancellery, the Federal Ministry of European and International Affairs) was going very well. 
All  persons  interviewed  stated  that  the  Management  Partnership  with  its  concrete  projects 
constitutes  a  communication  platform  that  has  resulted  in  an  intensive  information  exchange 
which  also  involves  EU  projects  and  co-operations  outside  the  Management  Partnership.  As 
meetings  are  held  regularly,  a  continuous  information  flow  is  guaranteed,  and  important 
synergies are created between the representatives of the European Commission and the Austrian 
institutions. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/results.do  

143

 
 
 
 
 
 
 
17 – Health and Consumer Protection   

Evaluation of the EU policy on animal welfare (EUPAW) 

ABB activities: 

17 04 Food and feed safety, animal health, animal welfare 
and plant health 

Timing:  

 May 2011  

Background, scope and focus  

The  evaluation  was  conducted  by  an  external  contractor:  GHK  Consulting  of  the  Food  Chain 
Evaluation  Consortium.  It  covers  all  the  main  EU  animal  welfare  initiatives  performed  by  the 
Commission  including  the  ones  under  the  responsibility  of  other  DGs  (mainly  DG  ENV  for 
experimental and wild animals and DG AGRI for the animal welfare aspects of the CAP).  

Summary of findings and conclusions  

The evaluation of the EU policy on animal welfare  provided a good description of the current 
policy and its effects. It confirms a number of directions for policy reflections suggested in May 
2010 by the European Parliament on the previous Community Action Plan (on animal welfare in 
2006) like the need to increase our emphasis of our future strategy on enforcement, international 
activities and research. 

Key messages: 

1.  Targeted EU animal welfare legislation has improved the welfare of many of Europe’s farm 
and  experimental  animals,  but  more  could  be  achieved  with  stronger  and  more  consistent 
enforcement of existing rules. By extending the scope of EU welfare legislation, other groups of 
animals could benefit from higher welfare standards.  

2.  EU legislation to protect animals has, in general, helped to reduce competitive distortions in 
the internal market caused by differences in national standards, but in certain areas further action 
on enforcement and harmonisation is required.  

3.  EU funding for research and scientific advice on animal welfare, totalling about €15 million 
annually,  has  made  a  positive  contribution  to  policy.  Most  funding  has  been  for  farm  animals 
and the development of techniques to facilitate the replacement of in vivo animal testing.  

4.  The  extent  to  which  EU  communication  actions  have  raised  stakeholder  and  public 
awareness and responsibility towards animal welfare is unclear. To maximise the impact of the 
limited  resources  available,  a  clear  communications  strategy  and  stronger  monitoring  and 
evaluation are needed.  

5.  The  EU’s  international  initiatives  have  helped  to  raise  awareness  and  create  a  shared 

144

 
 
 
 
 
 
 
 
 
 
international  understanding  of  animal  welfare  issues  and  standards,  particularly  with  trading 
partners in markets for food products, but there is much more to do.  

6.  The  establishment  of  equivalent  market  conditions  between  EU  businesses  and  those  from 
third  countries  exporting  to  the  EU  is  a  long  term  project  on  which  the  Commission  has  been 
working via bilateral and multilateral channels. Foundations are being laid but there is more to 
do.  

7.  The  financial  resources  and  instruments  at  EU  level  have  grown  to  meet  the  increasing 
resource needs of the EUPAW, though there is a need for further growth in funding as the policy 
continues to develop in the years ahead.  

8.  EU animal welfare policy appears to have succeeded in striking a balance between the varied 
needs  and  expectations  of  citizens,  industry  and  other  groups  on  an  issue  for  which  ambitions 
differ  across  Europe.  There  are  widespread  calls  for  more  consistent  enforcement  but  less 
appetite  for  a  new  wave  of  standards,  suggesting  an  agenda  defined  by  evolution  rather  than 
revolution for the next few years.  

9.  The various components of EU animal welfare policy are broadly complementary, mutually 
support and consistent, and have (thus far) avoided major conflict with other EU policies, such 
as on competitiveness, trade and the environment.  

10. EU  welfare  standards  have  imposed  additional  costs  on  the  livestock  and  experimental 
sectors, estimated at around 2% of the overall value of livestock output and a similar proportion 
of  the  annual  costs  of  experiments  using  animals.  There  is  no  evidence  that  this  has  so  far 
threatened the economic sustainability of these sectors.  

11. Management of the EUPAW costs the Member States’ public administrations an estimated 
total  of  around  €105  million  a  year,  with  about  €53  million  on  farm  inspections,  about  €0.5 
million for regulating welfare of experimental animals and about €13 million for administrative 
costs.  

Availability of the report on Europa  

http://ec.europa.eu/food/animal/welfare/actionplan/actionplan_en.htm 

1st interim evaluation of the Public Health Executive Agency (Executive Agency) 
(EAHC since July 2008) 

ABB activities: 

17 03 Public Health  

Timing:  

 December 2010 

145

 
 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The overall purpose of the recently finalised evaluation of the Executive Agency for Health and 
Consumers  (EAHC)  for  the  period  considered  (spring  2006  to  mid-2009)  was  to  assess  the 
management  of  the  Public  Health  Programme  (PHP)  whilst  implementation  of  the  Consumer 
Programme and Better Training for Safer Food Programme fell outside the window period and 
scope retained. Overall, no major shortcomings were identified as regards the final deliverable. 
However,  it  was  noted  that  the  contractor  had  difficulties  with  taking  guidance  from  the 
Commission  on  board  in  good  time.  It  must  be  noted  that  this  evaluation  comes  to  complete 
previous  cost-benefit  analyses  of  the  EAHC,  carried  out  by  other  companies,  such  as  Agra 
CEAS. 

Summary of findings and conclusions  

The current public health programme runs until 2013 and the mandate of the Agency until 2015. 
The  evaluation  concluded  against  an  early  closure  of  the  EAHC  before  expiry  of  its  current 
mandate,  as  such  an  approach  could  result  in  programme  continuity  issues  and  in  higher 
monetary costs than those that would be accrued  by keeping the Executive Agency operational 
until 2015. 

The main message from the evaluation is that the Agency has managed to improve procedures 
and increase the level of service provided to beneficiaries. Overall, no major shortcomings were 
identified  as  regards  effective  management  of  the  PHP,  though  the  increase  in  quality  is  not 
attempted  to  be  measured  in  monetary  terms.  A  relatively  high  level  of  staff  turnover  has 
nevertheless  been  noted.  Another  significant  finding  refers  to  communication  between  DG 
SANCO  and  EAHC  regarding  project  result  assessment  and  incorporation  into  future  Annual 
Work  Programmes.  The  evaluators  therefore  recommend  that  the  latter  be  addressed  and 
improved. The evaluation highlighted areas 

where progress had been made – the cost benefit analysis shows a positive monetary benefit in 
favour  of  the  EAHC  when  comparing  programme  management  of  the  PHP  in  the  Agency  to 
programme  management  within  the  Commission  –  and  aspects  where  room  for  improvement 
remains: 

Recommendations for improvement of the EAHC 

1.  Reduction of the number of programme priorities and funding of a lower number of projects 
with higher volumes 

Currently, the Executive Agency is asked to fund projects in a wide range of topics. Reducing 
the  number  of  programme  priorities  and  funding  of  a  lower  number  of  projects  with  higher 
project  funding  could  have  a  positive  impact  on  the  number  of  projects  allocated  per  project 
officer as well as on the impact and sustainability of the project itself.  

2.  Communication between DG SANCO and EAHC  

The evaluators concluded on the following: 
–  Communication between DG SANCO and the EAHC regarding project result assessment 
and incorporation into future AWP could be improved by I) common terminology and common 
document formats on outcome assessment, utilisation and integration into policy development, 
II) working sessions for outcome assessment, utilisation and integration into policy development 
with the participation of DG 

146

 
 
 
SANCO policy officers and EAHC project officers. 
–  Another complementary analysis of projects would be to assess project results. This 
would be initiated by the Agency with the help of grant holders in order to inform 
SANCO on the value of projects. 

3.  Allocation of project monitoring to external experts  

The  involvement  of  external  experts  under  the  supervision  of  the  Executive  Agency  project 
managers  in  project  monitoring  could  facilitate  managing  periods  of  heavy  workload  in  the 
Agency. It is important to note that the above mentioned solution should be applied selectively 
and only when there is no alternative as it might have an impact on the operating budget of the 
programme  and  requires  checks  of  conflict  of  interest  and  allocation  of  time  for  supervision 
from the EAHC project officers. It is finally unclear from the experiences in other DGs whether 
this option really saves time and resources. 

4.  Overcoming the challenges of staff retention 

It  is  concluded  that  the  Agency  faces  challenges  of  staff  retention,  mainly  attributed  to  the 
relatively  unfavourable  employment  conditions  for  contractual  agents  in  the  Agency. 
Regrettably, these conditions such as the local purchasing power of the CAs income in relation 
with typical Luxembourg salaries have not been analysed further. Also reasons for leaving other 
than low income have not been mustered. Taking into account that other executive agencies are 
facing  similar  but  less  severe  staff  retention  challenges  and  that  employment  conditions  for 
contractual  agents  should  be  identical  in  all  executive  agencies,  a  possible  option  is  that  the 
Commission revises employment conditions in all executive agencies. 

5.  Enhancement of the capacity of the NFP network  

Currently,  the  national  focal  points  are  an  “unofficial”  network  linking  Member  States  to  the 
European health policy. There is still space for improving this network via a boost of capabilities 
and capacities for augmenting its programme literacy in relation to outputs at national level and 
interaction  with  potential  applicants.  Much  of  this  is  however  not  in  EAHC  hands  but  in 
Member States'. 

Availability of the report on Europa  

http://ec.europa.eu/health/programme/docs/eahc_eval_dec2010_en.pdf 

Evaluation of the European Consumer Centres' Network (ECC-Net) 

ABB activities: 

17 02 Consumer policy 

Timing:  

 February 2011  

147

 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The  evaluation  in  subject  started  in  February  2010  and  was  conducted  by  a  Consumer  Policy 
Evaluation Consortium. 

It  analyses  the  functioning  of  the  ECC-Net  since  its  creation  in  2005  and  recommending 
improvements for the future.  

The  European  Consumer  Centres  Network  (ECC-Net)  was  created  in  2005  by  merging  two 
previously existing networks: the Euroguichets and the European Extra-Judicial Network (EEJ-
Net).  The  Network  comprises  29  centres  which  deal  with  cross-border  business-to-consumer 
(B2C) issues in 27 EU Member States plus Iceland and Norway. The overall aim of the ECC-
Net is to promote consumer confidence in the internal market. 

Summary of findings and conclusions  

The evaluation shows that services provided by the ECC-Net are very relevant to the needs of 
the  individual  European  consumers  shopping  cross-border.  It  improves  their  confidence  in  the 
internal market as well as their welfare. It presents a positive cost benefice balance: 1,77 times 
its cost to tax payers. The demand for the services is constantly growing. It uses a wide variety 
of promotional tools and reaches hundreds of thousands of consumers across Europe each year. 
While the network is the best known of all EU networks, the overall awareness of the existence 
of the network remains low and efforts to increase its visibility are needed.  

The evaluation stresses the importance of consumer education as well as consumer and business 
awareness  campaigns  highlighting  the  need  for  a  more  systematic  coordination  of  the  ECCs' 
promotional activities with relevant European and national stakeholders.    

The evaluation points out the limited resources of some of the centres – not necessarily matching 
the  demand  and  the  funding  instability  that  might  limit  the  scope  of  action  of  the  ECCs  (not 
possible to plan the resources allocation over a one year period). 

The quality of the services has been found to differ from one centre to another and evaluators 
proposed  an  effective  case  handling  protocol  as  a  management  tool  to  handle  issues  such  as 
overly long delays in handling cases. 

The  limited  ECC-Net’s  authority  to  force  traders  to  cooperate  has  been  highlighted.  Major 
constraints  outside  the  centres'  control  have  been  listed  as:  lack  of  a  well  functioning  ADR 
system across Europe; and a lack of willingness on the part of some traders to engage with the 
ECCs in a resolution of consumer complaints effort.  

In respect to redress issues, the contractor is proposing the establishment of formal protocols and 
procedures formalising and improving cooperation between the ECCs and enforcement bodies.  

Removal  of  an  obligation  to  develop  ADR  functions  from  the  objectives  of  the  ECCs  is  also 
proposed – as it is a national competence.  

On  the  other  hand,  stronger  competence  for  the  ECCs  is  proposed  in  what  comes  to  guide 
consumers to the appropriate judicial action. 

Apart from the financial redress benefits for the consumers, the centres bring further significant 
non-quantifiable benefits such as consumer detriment avoided and increased confidence in cross 
border shopping. 

148

 
 
 
 
From a contractual and financial point of view, the evaluators proposed making the continuation 
of  funding  conditional  upon  individual  ECCs  demonstrating  a  positive  benefits-cost  ratio. 
Replacement  of  the  current  system  of  annual  grants  with  a  system  of  framework  partnership 
agreements was also recommended. 

Availability of the report on Europa  

http://ec.europa.eu/consumers/ecc/docs/final_report_cpec_en.pdf 

Evaluation of Consumer Education, Information and Capacity building actions 

ABB activities: 

17 02 Consumer policy 

Timing:  

 October 2011  

Background, scope and focus  

The evaluation started in April 2011 and was conducted by ECORYS UK Ltd.  

Its aim was to inform decision making on future EU actions in consumer education and capacity 
building  

It  examined  four  DG  SANCO  actions  developed  over  the  past  10  years,  namely  DOLCETA, 
Europa  Diary,  TRACE  and  European  Integrated  Masters  programme,  in  total  representing  an 
expenditure of around € 27 M€.  

Summary of findings and conclusions  

The  evaluation  concludes  that  the  consumer  landscape  in  the  EU  is  quite  dynamic  and 
heterogeneous,  and  requires  interventions  with  variable  geometry,  as  opposed  to  the  "one-size 
fits all models" of Dolceta, Europa Diary and, to a lesser extent, TRACE. It advocates to refocus 
actions  towards  building  interactive  community  resources  to  facilitate  best  practice  exchange 
between Member States, rather than passive resources built by the EC for the Member States. 

The  evaluation  found  that  Dolceta,  the  Europa  Diary  and  Trace  had  resulted  in  separate  but 
partly  overlapping  information/education  resources  without  sharing  distribution  channels  and 

149

 
 
 
 
 
 
 
 
 
 
 
 
intellectual  capital,  which  hindered  the  overall  coherence  and  efficiency  of  the  EU  consumer 
information and education effort. 

In  the  case  of  the  Diary  and  Dolceta,  the  evaluation  further  concluded  that  both  actions  may 
duplicate national resources in some Member States, and that they are not responsive enough to 
policy developments in relation to the frequency of updating. 

In more detail, the evaluation concluded that: 

The  3  year  start-up  subsidies  for  setting  up  European  Integrated  Masters  programme  courses 
resulted in three consumer policy masters courses. As the funding from DG SANCO was limited 
in  time,  the  sustainability  of  the  courses  was  dependent  on  their  capacity  to  find  subsequent 
alternative  funding  sources,  notably  the  Erasmus  Mundus  programme.  The  evaluation  also 
recommends  to  apply  for  further  funding  from  Erasmus  Mundus,  while  DG  SANCO  should 
continue to give political support to the Masters courses that continue their offer to students. . 

The evaluators found that the TRACE training courses for national level consumer organisations 
are  very  positively  viewed  by  the  participants  and  stakeholders  and  support  EU  action  to 
contribute to build capacity of staff in consumer organisations. However action should be taken 
to address the high cost per participant (€ 1,665.98 per person per day), to improve outreach to   
national audience and overcome language barriers. In addition,, the high turnover in the staff of 
consumer  organisations,  and  the  need  to  further  disseminate  the  knowledge  of  trained  staff 
within  organisations,  and  the  need  to  make  the  training  materials  publicly  accessible  were 
identified  as  issues  for  future  action.  The  evaluation  recommends  increasing  Trace  impact 
significantly  by  going  on-line  and  multilingual,  as  well  as  broadening  the  courses  to  a  wider, 
more active  and participatory training community, to facilitate  sharing good practices between 
national organisations 

The evaluation found that DOLCETA as set up in 2003 was not suitable to the current internet 
environment. Lack of baseline data made an assessment of progress over the years difficult. But 
Dolceta appears not to fulfil its objectives towards its multiple targets, neither as information nor 
as an education tool.  

The  evaluators  considered  Dolceta's  usage  level  to  low  to  justify  the  expenditure,  its 
technological  design  obsolete  and  its  content  of  very  uneven  quality  and  in  many  cases 
duplicating information that is already available at national level. Dolceta was further found to 
mix  up  different  target  groups,  to  overlap  with  the  Europa  Diary  in  the  secondary  education 
school group, to incur high translation costs, and to lack regular updating. Its cost-efficciency, 
with  a  cost  per  hit  above  4  €,  was  found  insatisfactory  and  the  contractor  receommended  to 
discontinue this action as redesigning the site would require excessive resources.   

The evaluators found that while the Europa Diary resources have clear value, the real use of the 
printed  material  for  consumer  education  purposes  is  uneven  across  Member  States,  and  the 
printed form does not address the needs of teachers across Europe.  

The  evaluation  at  the  same  time  underlined  the  significant  and  critical  role  of  teachers  as 
strategic  intermediaries  for  consumer  education  of  young  people.  As  embedding  consumer 
issues in school curricula is a priority, the evaluation suggests that DG SANCO should continue 
to provide consumer education materials for teachers to use in classroom. The evaluators suggest 
to  undertake  research  to  identify  the  needs  of  teachers  in  each  Member  State  and  to  find  out 
which delivery channel (print, on-line or other) would be appropriate to satisfy such needs.  

The  evaluation  recommends  that  the  Diary  content  and  accompanying  teacher  guide  should  in 
the  meantime  be  made  available  on-line  so  as  to  stay  in  touch  with  the  user  community  of 
teachers and not severing ties with them.  

150

 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/consumers/strategy/docs/evaluation_consumer_education_report_en.pdf 

Evaluation of the EU-RLs in the field of food and feed safety and animal health and 
live animals 

ABB activities: 

17 04 Food and feed safety, animal health, animal welfare 
and plant health 

Timing:  

 11/04/2011 

Background, scope and focus  

There are currently 45 EURLs (27 in the feed and food area and 18 in the animal health and live 
animals area). This evaluation continued and finalised the assessment process of the EURLs and 
their  network  which  began  with  the  evaluation  of  twelve  EURLs  in  the  field  of  animal  health 
and live animals completed in 2009. The ad hoc Inter-Service Steering Group for this evaluation 
was managed and chaired by SANCO Unit E5. Unit 01 participated in this group and assisted 
with contractual, methodological and operational issues in the support and steering given to the 
contractors. The evaluation assessed the functioning and performance of each EURL in the food 
and feed safety area and of their network as a whole (as well as of two remaining EURLs in the 
field of animal health), the effectiveness and efficiency of EU funding, the possible overlaps or 
synergy potentials between EURLs in  particular  fields and the appropriateness of their current 
mandates. 

Summary of findings and conclusions  

It was found, based on a review method developed by the contractor and an assessment grading 
sheet  adopted  for  this  exercise,  that  the  EURLs  performed  in  general  adequately  and  partly 
excellently.  In  spite  of  overall  positive  results,  19  EURLs  underperformed  on  at  least  one 
evaluation indicator in the assessment grid. Most notably, a significant number of EURLs do not 
collect  nor  summarise  feedback  provided  by  participants  in  workshops  or  in  ad  hoc  training 
activities.  Some  EURLs  do  not  provide  corrective  actions  and  follow-up  to  national  reference 
laboratories  (NRLs)  that  underperformed  during  proficiency  tests  (PTs)  and  the  tools  used  by 
some  EURLs  to  share  information  with  NRLs  could  be  improved  (up-to-date  websites,  user-
friendliness of tools, etc). 

151

 
 
 
 
 
 
 
 
 
 
 
The evaluation furthermore concluded that the EURLs' assistance to NRLs had been adequate in 
order to improve analytical methods and the quality of analytical data generated in the EU, that 
the analytical methods and techniques developed, validated, or assessed could be considered as 
responding to state-of-the art standards and being appropriate to ensure food and feed safety, and 
that  the  coordination  and  training  activities  carried  out  such  as  PTs  and  workshops  had  been 
satisfactory. 

The  evaluators  also  noted  that  a  large  majority  of  EURLs  had  been  able  to  deliver  good 
scientific advice and expertise in a very timely manner and that there was a common agreement 
that  EURLs  fulfilled  the  requirements  laid  down  in  Article  32  (4)  of  Regulation  (EC)  No 
882/2004  (suitably  qualified  staff  with  adequate  training,  presence  of  equipment  and  products 
needed to carry out tasks assigned to them, sufficient knowledge of international standards and 
practices,  etc.)  and  that  they  contributed  to  the  achievement  of  the  objectives  pursued  by  EU 
legislation and the improvement of food and feed safety in the EU. 

The evaluation identified clear added value in the network of EURLs over the (counterfactual) 
situation without an EU system for developing new analytical/diagnostic methods, contributing 
to  the  harmonisation  and  improvement  of  analysis/diagnosis  and  providing  confirmatory 
diagnosis in emergency situations as well as over a (hypothetical) central EURL responsible for 
all  tasks  currently  performed  by  the  EURLs  in  the  various  Member  States  (MS)  (since  the 
current  EURL  network  builds  on  long-standing  expertise  of  existing  laboratories).  The  EU 
funding  appeared  furthermore  to  be  used  in  a  cost  efficient  manner  when  compared  to  the 
benchmark of a (hypothetical) centralised approach and other benchmarks (no EURL(s), limited 
network of laboratories in Canada) as economies of scale and scope and learning curve effects 
stemming from the combination with existing laboratories would be lost.  

It  also  underlined  however  that  at  the  network  level,  some  potential  efficiency  gains  resulting 
from the increase of collaboration between EURLs, and the reduction of overlaps between them 
and the tapping of potential synergies identified, did exist.  

Availability of the report on Europa  

http://ec.europa.eu/food/food/controls/reference_laboratories/evaluation_en.htm 

152

 
 
 
 
 
Ex-post evaluation of the Public Health Programme (PHP) 2003-2008 

ABB activities: 

17 03 Public health 

Timing:  

 March 2011 

Background, scope and focus  

Overall purpose of this evaluation: assess the effectiveness, efficiency and utility of the Public 
Health Programme (PHP) which means that it is to be ascertained whether the achievements of 
the  programme  correspond  with  its  objectives,  are  whether  these  are  reached  at  reasonable 
resource  use/costs  and  correspond  with  needs,  problems  and  issues  of  relevance  (to 
stakeholders). 

Furthermore,  the  impact  of  the  programmes,  projects,  and  activities  on  the  improvement  of 
public  health  policies  in  the  Member  States  and  at  EU  level  is  assessed.  This  is  done  by 
evaluating  the  extent  to  which  the  programme  has  achieved  the  intended  outcomes/impacts, 
delivered inputs to policy, ensured consistent and  complementary implementation with respect 
to the Member States' expected achievements in the field of public health, and been implemented 
in  accordance  with  the  international  public  health  aims.  All  this  is  undertaken  with  a  view  to 
examining European added value. 

Summary of findings and conclusions  

It was found that very many projects funded by the Public Health Programme rely primarily on 
networking, best practice promotion and exchange of views and experiences, making it difficult 
to  measure  its  effectiveness  or  impact  in  quantitative  terms  in  a  fully  comprehensive  way. 
Generally, one was left to recognise difficulties encountered by our external contractors, COWI 
A/S,  in  the  execution  of  this  evaluation,  but  acknowledge  its  contribution,  inter  alia,  as  one 
material for discussion in the conception of the next Programme 2013+. 

Irrespective of the scope and future of a Commission Health Strategy to which the programme 
would  refer,  the  main  message  from  the  evaluation  pinpoints  the  need  for  a  reduction  in  the 
number  of  priority  areas  in  the  Annual  Work  Programmes  (AWP).  Concretely,  it  is 
recommended that this be done by allowing a maximum of five priority areas in each of the three 
strands to increase the impact within the priority areas, bringing them to not more than 15 per 
yearly call. The number here is more guessed than elaborated and the concept of priority in the 
context of the PHP has not been further defined and not aligned with the findings of the Court of 
Auditors report. 

A  second  relevant  significant  finding  refers  to  policy  applicability  of  funded  projects  –  DG 
SANCO  is  invited    to  ensure  that  the  priority  areas  in  the  AWP  are  focused  and  based  on  a 

153

 
 
 
 
 
 
 
 
 
 
thorough analysis of needs and EU added value (carried out by staff versed in these issues on the 
basis of criteria to be developed by the Executive Agency for Health and Consumers (EAHC), 
which  should  also  compile  brief  descriptions  of  project  results,  compatible  with  the  existing 
database,  including  considerations  about  use  potential  and  policy  recommendations,  and 
disseminate these to Commission staff and national stakeholders at the political level). 

A third remarkable message refers to the need to reach Member States which appear inactive in 
the  PHP,  typically  countries  with  a  relatively  low  GDP/capita.  The  evaluators  therefore 
recommend  that  inclusion  be  pursued  by  providing  technical  assistance  to  write  proposals 
(EAHC) or by increasing the EC financial contribution (DG SANCO), possibly on the basis of 
an  alternative  cost  model  since  the  need  to  reach  high  levels  of  co-funding  in  the  projects  is 
prohibitive for those countries facing lower levels in public spending. 

In  the  absence  of  objectively  formulated,  clear  performance  indicators  at  programme  level 
(success  criteria  based  on  a  thorough  elaboration  of  the  intervention  logic  underpinning  the 
different  areas  and  priorities)  to  facilitate  follow-up  and  evaluation  of  achievements,  the 
evaluation  stressed  the  need  to  clearly  define  these  and  highlighted  areas  where  progress  had 
been made and aspects that could still be strengthened going forward: 

Availability of the report on Europa  

http://ec.europa.eu/health/programme/docs/ex_post_evaluation_en.pdf 

Mid-Term Evaluation of the EU Health Strategy 2008-2013 

ABB activities: 

17 03 Public Health  

Timing:  

 August 2011 

Background, scope and focus  

The mid-term evaluation of the EU Health Strategy (HS) – covering the period 2008 to 2011 – 
was undertaken from October 2010 to August 2011 not as a legal obligation as is most often the 
case but as a policy review opportunity endorsed by Management. Its aim was to take stock of 
the strategic policy document in a joint exercise with the mid-term evaluation of the Public 
Health Programme (PHP) 2008-2013. The two evaluations proceeded in parallel, with different 
external expert teams assigned to each individual file.  

It must be noted that the original need for an EU Health Strategy has been largely defined by a 

154

 
 
 
 
 
 
 
 
 
 
 
 
 
sharp  increase  of  policy  domains  eligible  for  EU-level  actions  where  MS  often  requested 
initiative  or  assistance  from  the  Commission.  These  domains  became  evident  in  relation  to 
growing  health  challenges  in  MS,  the  new  Treaty  responsibilities  in  health  and  also  a  relative 
shift of the centre of gravity of the EU health policy debate away from WHO. It is in this context 
that a sound, high quality evaluation of the Health Strategy represents clearly an indispensible 
instrument for further policy development.  

Summary of findings and conclusions  

1.  Equivocal nature of the Strategy document 

Of particular interest in terms of the key findings of the evaluation are the status of the Strategy 
as  a  document  and  its  dual  but  equivocal  nature  representing  on  one  hand  a  "shared  vision" 
framework  paper  and,  on  the  other  hand,  a  more  directing  framework  aiming  to  align  a  set  of 
common  objectives,  values  and  actions  and  calling  for  a  concerted  response  to  increasingly 
diverse and mounting challenges. This duality is largely a legacy of the Strategy's development 
process at its inception but is equally a feature impacting on the current implementation of the 
Health Strategy. Moreover, the reception of the Health Strategy is different between one end of 
the stakeholder spectrum (the Senior Level Working Party consisting of MSs) and the other (the 
EU Health Policy Forum captured by some vocal NGOs). As a result, the mobilisation capacity 
is hence overall reduced and this represents a significant barrier for implementation. This aspect 
could  have  been  elicited  and  described  in  more  detail  by  the  external  contractors  as  this  task 
would have been feasible given the means at hand.   

2.  Weak links with other EU and MS strategies and actions 

No solid evidence has been found as to the directive influence of the HS on what MS undertake 
in  terms  of  actions,  policies  and  programmes.  The  occasional  good  level  of  compatibility 
identified by the contractors between the EU HS and national MS strategies in terms of thematic 
and  structural  ideas  does  not  conclusively  establish  a  direct  causal  link.  Stakeholders  in 
particular do not see the EU HS as being the driving force behind action at national level, despite 
some partly unfounded expectations in this regard. Therefore, the HS is not recognized as yet to 
be  a  strong  determinant  of  health  policy  in  the  EU: firstly because  of  its  equivocal  nature  and 
secondly perhaps due to the fact that the HS's two coordination mechanisms, the SLWP and the 
EUHPF, are not living up to their full potential.  

In  addition,  the  external  assessment  gives  much  more  prominence  to  the  fact  that  the  HS's 
impact on other EU policies and activities/funding programmes appears to be relatively low. Not 
only has there been no discernible direct impact of the EU HS on these policies, but the recent 
evaluation  of  the  Health  Programme  confirmed  the  difficulty  in    this  area,  i.e.  the  absence  of 
significant means mobilised to realise an alignment of the health policy as developed by this DG 
with  Structural  Funds  expenditure1.  In  the  absence  of  an  established,  solid  political 
Commission–wide  reference  framework  for  policy  integration,  any  future  strategic  action  as  a 
follow-up to the Health Strategy would have to establish the process of integration by its own 
means  and  point  to  those  activities  which  could  help  to  reach  its  objectives  and  realise  its 
principles. 

3.  Are the coordination mechanisms the drivers of implementation? 

1 Few projects such as EUREGIO under the Health Programme were funded and these preferentially 

covered relatively more affluent regions.  

155

 
 
 
                                                 
For the Senior Level Working Party ( SLWP) does the evidence not only point in the direction of 
a dwindling level of seniority in its attendance over time, but Council presidencies priorities are 
not  influenced  by  the  agenda  of  the  SLWP  or  vice  versa.  Engagement  with  this  coordination 
mechanism has been fading, though it is understood that at present the SLWP itself is likely to 
tackle  this  via  its  new  mandate.  Moreover,  according  to  the  views  of  EUHPF  and  SLWP 
representatives, the coordination mechanisms have contributed to health policy developments at 
MS level to a very limited extent1. Although this might never have been expected by some from 
the start, it might also partially be due to the equivocal nature of the HS. If a strategy is generally 
understood as a framework that aligns principles and objectives with actions and means, such a 
definition is nowhere met, neither in the White Paper nor in the evaluation document. 

An  easy  to  reach    assessment  of  the  main  operational  conclusions  of  the  evaluation  is  that 
irrespective of the scope and direction of a future Commission Health Strategy, the real need for 
an ex-post evaluation of the present Health Strategy in 2 years' time cannot be retained. It is very 
unlikely that such an exercise would retrieve any measure of further intelligence and supporting 
data  for  evidence-based  policy-making.  One  could  possibly  draw  more  input  from    a  set  of 
targeted follow-on studies Supplementing on e of the weaknesses of the current exercise would 
be  to  obtain  more  refined  policy  implementation  indicators  which  could  assist  in  a  future 
monitoring exercise of the implementation of a renewed Health Strategy.  

Availability of the report on Europa  

http://ec.europa.eu/health/strategy/docs/midtermevaluation_euhealthstrategy_2011_report_en.pdf 

Mid-Term Evaluation of the Health Programme (2008-2013) 

ABB activities: 

17 03 Public Health  

Timing:  

 October 2011 

Background, scope and focus  

The  purpose  of  this  assignment  was  to  carry  out  an  interim  evaluation  of  the  (2nd)  Health 
Programme  2008-2013  in  order  to  provide  an  overview  of  the  implementation  of  the  Health 

1 Few if any PHP projects can be considered as spin-offs of the stakeholder community under the 

EUHPF, showing an issue with access to the implementation of the Strategy by the stakeholders 
(especially for those who have in principle little alternative funding source for their actions) 

156

 
 
 
 
 
 
 
 
 
 
 
                                                                                                                                            
Programme in the first three years. The overview should provide a quantitative and qualitative 
description  of  priorities  set,  financial  mechanisms  used  (grants/operating  grants/joint 
actions/tenders  etc),  beneficiaries,  actions  funded,  and  intended  results.  In  view  of  recent 
evaluations of the previous Public Health Programme (2003-2007), the new evaluation exercise 
checks where and how improvement has been achieved and whether the same problems persist. 
The  conclusions  and  recommendations  produced  will  feed  into  the  ongoing  implementation  of 
the Health Programme up until its termination on 31st December 2013 and into the preparation 
and design of the post-2013 programming period.  

Summary of findings and conclusions  

This evaluation builds on previous efforts to take stock of the Public Health Programme and of 
its  forerunners1.  Much  as  in  the  previous  exercise,  we  have  again  to  recognise  the  difficulties 
encountered by our external contractors, TEP (The Evaluation Partnership), in the execution of 
this evaluation. We also acknowledge the contribution material made available for the discussion 
towards the next Programme 2013+.  

Focus of the present exercise was to be the  impact and added value of the PHP, as it had not 
been  possible  to  demonstrate  these  aspects  clearly  in  the  previous  evaluations  by  RAND  and 
COWI  for  the  2003-2008  Programme.  This  was  due  either  to  the  absence  of  an  intervention 
logic as pointed out by the Court of Auditors or to the over-reliance on network-building among 
public health actors as a main activity of the Programme up to the point that these networks and 
their maintenance was to become an objective per se for the Health Programme, while little if 
any  work  was  channelled  into  verifying  what  the  impact  of  this  capacity-building  would 
pragmatically be.  

The significant step forward achieved by this evaluation was the identification of projects where 
the added value could be verified thanks to predefined assessment criteria. 

KEY FINDINGS 
Firstly,  even  though  the  Health  Programme  funding  volume  is  relatively  minor2,  it  impacts 
significantly  on  the  work  done  by  public  health  practitioners  across  the  EU.  It  contributes  to 
creating and maintaining a professional public health community at European level fostering the 
exchange of knowledge and experiences. 

Secondly,  activities  such  as  on  health  determinants  and  comparable  health  data  could  be 

1 Originally, the PHP has been growing out of a small number of singular, empirically managed actions 
in response to calls from Council and Parliament such as actions on HIV/AIDS, health information 
etc.  Inevitably,  the  number  of  priorities  increased  to  meet  the  different  expectations  that  were 
developing up to a level well above what could remain manageable and deliver sufficient impact. 
Nevertheless, the PHP evaluation recognizes a strong potential in contributing to the preparation, 
development  and  implementation  of  EU  public  health  policies.  While  in  the  initial  period  from 
2003-2007,  a  rather  large  spectrum  of  health  priorities  were  attempted  to  be  covered  and  this  is 
somewhat expectable with a programme in its formative stage, the experience now gained offers 
the  future  programme  the  opportunity  to  benefit  directly  from  the  setting-up  of  an  intervention 
logic encompassing SMART objectives, the designation of concrete actions directly linked to the 
instruments  at  hand  and  the  recognition  of  clear  performance  indicators.  Additionally,  at  the 
present  stage,  stakeholders  already  expressed  their  views  for  prioritising  the  needs  and 
concentrating the efforts on actions meeting well defined, proven EU added value criteria. 

2 The HP is endowed with a total budget of 321.5 million EUR whereas FP7 consists of 6.1 billion EUR. 

157

 
 
 
                                                 
developed in new Member States thanks to the support of the Health programme in a situation 
where the current budget restrictions would not allow this to become a priority. 

Thirdly,  the  current  Health  Programme  has  promoted  important  issues  at  the  level  of  EU  and 
national  political  agendas  i.a.  on  rare  diseases  and  cancer  screening  guidelines  and  has 
influenced  policy  development  and  implementation  at  national  levels.  Dissemination  of  PHP 
results is in this context clearly an additional field for improvement and is directly linked to the 
issue  of  the  intervention  logic:  the  outcomes  of  the  financed  actions  that  target  health  policy 
making at EU level as well as at national or regional level are insufficiently known and therefore 
not  recognized  by  national  stakeholders  and  policy  makers..  More  dissemination  would  entail 
more feedback from end users and would also help to monitor impact generated by the actions of 
the programme.  

On the programme management level, there is a significant improvement in the delivery of the 
Programme following the first five-year cycle mainly due to the outsourcing of management to 
the Executive Agency for Health and Consumers. The new financial mechanisms have generally 
been received positively and all mechanisms have been utilised.  

MAIN CONCLUSIONS 

Firstly: The Public Health programme post-2013 should be much more focused on priorities and 
concentrate financial support on a fewer number of actions in key priority areas, bringing up the 
largest EU added value as was already underlined by the Court of Auditors in the report "The 
European  Union's  Public  Health  Programme  (2003-2007):  an  effective  way  to  improve 
health?"1. This focusing and priority reduction cannot be realised unless a specific methodology 
based on the agreed EU AV criteria2 is adopted.  

Secondly: The Public Health programme post-2013 should be enabled to better serve and further 
involve  all  EU  Member  States,  especially  those  with  a  relatively  low  GDP  per  capita  where 
stakeholders  encounter  numerous  cultural,  procedural  and  financial  barriers  that  reduce  their 
opportunities  for  participation.  Emphasis  should  be  placed  on  actions  in  areas  where  Member 
States cannot act in isolation in a cost-effective manner,.  

Thirdly:  Evaluators  made  recommendations  for  developing  more  tangible  and  focused 
Programme  objectives  set  in  a  better  defined  strategic  framework  with  long-term  targets. 
Limiting the number of actions to those health issues that are of most concern to Member States 
and where there is real value identified in intervening at EU level could increase efficiency gains 
and  maximise  the  impact  of  the  Programme.  Actions  and  their  results  need  to  be  built  into  a 
regular  reporting  system  and  shared  more  effectively  between  Commission  services  and  with 
stakeholders and national policy makers. This assumes the development of credible and accepted 
intermediate  milestones  associated  with  a  targeted  iterative  review  process.  The  second 
recommendation  meets  also  the  outcome  of  a  recent  audit  on  the  usage  of  the  Programme 
management Executive Agency. 

Fourth  Main  Conclusion:  The  exercise  validated  the  use  of  the  7  EU  added  value  criteria  as 
devised  by  the  Executive  Agency  for  rendering  the projects  funded  more  effective  in  terms  of 
impact.  Use  of  these  added  value  criteria  needs  to  be  further  analysed  and  complemented  by 
reflections on devising a way to operationalise also the leverage effect3of the projects, a matter 
which would benefit from further attention.   

1 http://eca.europa.eu/portal/pls/portal/docs/1/2838313.PDF 

2 These 7 EU AV criteria are: promotion of best practice, networking, economies of scale, implementation of EU 

legislation, benchmarking for decision-making, cross-border threats and free movement of people. 

3  By  "leverage  effect"  it  is  meant  the  strategic  advantage  and  traction  that  a  project  can  bring  to  a  stakeholder 

group or population, much broader than the one originally directly involved in the project.  

158

 
 
                                                                                                                                            
By  emphasising  the  need  to  develop  a  solid  intervention  logic  for  the  entire  programme,  the 
evaluation  also  upgrades  the  urgency  of  the  finding  in  the  Court  of  Auditors'  report  as  to  the 
need for an intervention logic at project level.  

Finally, the problematic level of dissemination of this Programme 2008-2013 as established by 
the current evaluation echoes the low level of notoriety of the Programme verified by the former 
evaluation.  

Availability of the report on Europa  

http://ec.europa.eu/health/programme/docs/mthp_final_report_oct2011_en.pdf 

Evaluation of the Community Plant Variety Right Acquis 

ABB activities: 

17 04 Food and feed safety, animal health, animal welfare 
and plant health 

Timing:  

 Date of the report: April 2011 

Background, scope and focus  

The  CPVR  regime  is  based  on  Council  Regulation  (EC)  N°  2100/94  of  27  July  1994  on 
Community Plant Variety Rights and its implementing rules. Community Plant Variety Rights 
are granted to breeders on request and the breeder’s rights cover in a uniform and harmonised 
manner  the  entire  territory  of  the  European  Union  (EU).  The  Regulation  sets  out  rules  for  the 
application,  examination  of  new  plant  varieties,  while  satisfying  requirements  of  the 
TRIPS1/WTO2 agreement and the 1991 Act of the International Union for the Protection of New 
Varieties of Plants (UPOV) and providing plant variety protection that becomes effective in all 
Member States through a single application process. It coexists with national plant variety right 
systems  in  23  Member  States  (MS).3  In  1995  an  EU  agency  with  legal  personality,  the 
Community  Plant  Variety  Office  (CPVO),  located  in  Angers  in  France,  was  established.  This 
self-financed agency is responsible for the implementation of the Regulation. The CPVO adopts 
individual  decisions  with  regard  to  granting  plant  variety  protection  titles  which  are  legally 
binding for third parties. 

The  evaluation  of  the  Community  Plant  Variety  Rights-  acquis  covered  the  period  that  the 

1 TRIPS: Agreement on Trade-Related Aspects of Intellectual Property Rights 
2 WTO: World Trade Organisation 
3 Cyprus, Greece, Luxemburg and Malta do not have a national plant variety rights system. 

159

 
 
 
 
 
 
 
 
 
 
 
                                                 
CPVR legislation has been in effect (1995 - present). The main focus was to assess how well the 
CPVR  acquis  has  met  its  original  objectives  assess  current  strengths  and  weaknesses,  and 
propose options to resolve deficiencies identified in the system.  

Summary of findings and conclusions  

The  evaluation  has  concluded  that  the  CPVR  system  functions  well  overall,  that 
stakeholders are happy with it and that they wish to retain the system in its current form, 
albeit with some carefully targeted adjustments.   

Key findings: 

The evaluation found that there is broad support for the stated objectives of the legislation. The 
CPVR system functions well overall, that stakeholders are happy with it and that they wish to 
retain the system in its current form, albeit with some carefully targeted adjustments.  

Harmonisation:  The  CPVR  acquis  provides  for  a  harmonised  intellectual  property  regime  for 
plant varieties at EU level, but practical 'enforcement' varies in the Member States (MS). CPVR 
holders  consider  this  to  be  one  of  the  biggest  problems  inhibiting  an  effective  EU-wide  plant 
variety rights system. Enforcement in this evaluation refers to the practical management of rules 
set out by this regulation and not to the transposition of the legal requirements of this regulation. 
The practical management is regulated by national or EU general legislation and consequently 
can  not  be  the  subject  of  the  current  CPVR  acquis.  Possible  solutions  must  be  carefully 
explored, especially as regards proportionality.    

Appropriateness  and  effectiveness:  The  CPVR  acquis  has  enabled  the  granting  of  intellectual 
property rights valid throughout the EU. It coexists effectively with national plant variety rights 
regimes. The acts to which only CPVR holders are entitled and the rights conferred on CPVR 
holders are appropriate in general, although there is scope to improve the provisions extending to 
harvested material in the case of unauthorised use or for material in transit.  

Basic requirements for plant variety protection: A plant variety must be new, distinct, uniform 
and  stable  (i.e.  conform  to  DUS  criteria),  and  have  an  appropriate  variety  denomination  to 
qualify  for  CPVR  protection.  The  application  of  these  criteria  is  generally  appropriate  and 
effective,  though  the  quality  of  testing  centres  could  be  improved  in  few  Member  States.  The 
CPVO has recently implemented an audit and entrustment process to assess the capabilities of 
all examination offices across the EU. DUS criteria examinations are generally satisfactory. Few 
stakeholders  consider  that  the  criteria  themselves  could  be  more  flexible  to  reflect  varieties 
adapted to local conditions. The CPVO has developed a database to check vatiety denomination, 
but,  due  to  the  lack  of  its  utilisation  by  some  Member  States,  the  CPVR  system  for  variety 
denomination is complex and unsatisfactory and needs to be changed. 

Duration  of  protection:  The  duration  of  protection  is  in  general  appropriate  to  the  system’s 
needs. CPVR protection durations are comparable to national plant variety rights systems in MS. 
On  average,  a  CPVR  ‘lifespan’  is  much  shorter  than  the  protection  period  provided  under 
existing legislation. Nonetheless many breeders would still like to see the duration of protection 

160

 
 
 
extended to 30 years for all plant varieties. 

Exemption  for  plant  breeders:  The  CPVR  acquis  provides  an  appropriate  exemption  for  plant 
breeders. The breeders’ exemption is the cornerstone of effective plant variety rights, and central 
to  what  makes  plant  variety  rights  an  important  and  useful  system  of  intellectual  property 
protection.  The  breeders'  exemption  is  an  open  access  system  to  the  new  varieties  for  further 
breeding activities and therefore it encourages competition in the breeding sector and facilitates 
rapid advancement and innovation in variety development. 

Essentially  derived  varieties:  The  provision  for  ‘essentially  derived  varieties’  (EDV)  is 
appropriate and helps to protect against plagiarism of plant varieties that are too similar to one 
another.  Nevertheless,  the  definition  is  unclear,  however,  both  in  the  CPVR  Basic  Regulation 
and  in  UPOV  and  there  are  few  established  protocols  for  making  EDV  determinations. 
Procedures  to  determine  ‘essential  derivation’  are  not  well-established,  therefore  technical 
determinations do not produce clear results. There is scope for improvement in this area. 

Exemption to farmers: The acquis also provides an limited exemption to farmers to save part of 
their harvest and sow this material on their own farm, providing for payment of an appropriate 
royalty  on  farm  saved  seed  (FSS).  The  small  farrmers  are  exempted  from  this  obligation. 
Implementation of the ‘agriculture exemption’ has been problematic and some stakeholders are 
widely dissatisfied with it. In particular Court of Justice of the European Union (CJEU)  had to 
address several rulings on the legal interpretation of farmers’ obligation to report FSS use. The 
operation of this exemption could be improved and more effective royalty collection systems for 
FSS established across the EU. 

Definition of a ‘small farmer’: The appropriate definition of a ‘small farmer’ in the context of 
the  farm  saved  seed  measures  is  also  contentious  and  there  is  disagreement  about  whether  the 
provision  should  remain  in  the  Basic  Regulation.  Any  decision  should  be  taken  in  agreement 
with  the  EU  Agricultural  Policy  as  revised  to  ensure  that  changes  lead  to  an  enhanced 
consistency between the various legal acts that apply to agriculture, starting of course with the 
legislation on production and marketing of seed and propagating material. 

Definitions of ‘own holding’ and ‘equitable remuneration’: The ‘own holding’ definition in the 
context of the farm saved seed measures could be clarified to better reflect farming practices as 
defined by the EU Agricultural Policy, and there is scope to clarify the definition of ‘equitable 
remuneration’ . 

List  of  species  covered:  The  list  of  species  covered  by  the  agriculture  exemption  is  generally 
satisfactory though there is a case to reconsider potatoes on the list of exempt species because of 
the phytosanitary risks associated with farm saved potatoes as mentioned by some stakeholders 
and some Member States. 

Proposed options for the future: 

The evaluator does not provide for straightforward recommendations but rather discusses several 
directions with their pros and cons. 

A large part of the options are related to improvements of some requirements to permit an easier 
implementation of the current legislative framework. The most relevant ones are the following: 

1.  Useful  adjustments  could  be  made  where  the  acquis  interacts  with  other  EU  legislative 
frameworks: 

161

 
 
–  The  Seed  and  plant  propagating  Marketing  legislation-  implementing  a  ‘one  key,  several 
doors’  approach,  supervised  by  the  CPVO,  could  create  greater  efficiencies  and  avoid 
duplication, where only one procedure is used to grant CPVR and to market  varieties of species 
covered by those Directives.  

–  The  legislative  framework  for  patents  (Directive  98/44/EC)  -  CPVO  could  provide  more 
information regarding plant-related patents and their implications for particular plant varieties to 
help breeders determine when a patent overlaps with plant variety rights.  

–  The EU Enforcement Directive (2004/48/EC) - Articles 94 and 97 could be amended so that 
the  two  frameworks  are  aligned  on  compensation,  damages,  and  restitution,  and  to  ensure that 
the language in the CPVR Regulation does not preclude use of Directive 2004/48/EC. 

2.  Extending the CPVR acquis to EFTA countries would help to bring the EU and the EFTA 
trade polices into closer alignment and may improve EU breeding industry competitiveness. 

3.  The  provision  for  essentially  derived  varieties  (EDV)  is  appropriate  but  the  definition  of 
EDV  needs  clarification.  EDV  enforcement  could  be  improved  by  developing  standardised 
protocols  for  the  most  economically  important  species.  CPVO  could  play  a  greater  role  in 
developing these protocols. 

4.  The provisions for the farm saved seed (FSS) need to be revised. Following the decision the 
of  the  Court  of  Justice  of  the  European  Union  (CJEU)  on  several  rulings,  CPVR  holders 
currently  find  it  difficult  to  obtain  royalties  for  FSS  use  due  to  their  limited  ability  to  request 
information  from  farmers  without  prior  evidence  of  such  use.  Amending  the  CPVR  Basic 
Regulation  to  introduce  new  definitions  and  requirements  such  as  obligate  farmers  to  answer 
‘yes’ or ‘no’ when asked whether they have used farm saved seed would relieve the burden on 
breeders to discover its use and is consistent with the CJEU decisions. 

The duration of the protection should be revised on a case-by-case basis to ensure that the 
plant  breeders  can  obtain  a  return  on  investment  where  the  breeding  programme  and  variety 
development are particularly long. 

Availability of the report on Europa  

http://ec.europa.eu/food/plant/propertyrights/docs/cpvr_evaluation_final_report.pdf  

Evaluation of the EU legislative framework in the field of cultivation of GMOs under 
Directive 2001/18/EC and Regulation (EC) No 1829/2003, and the placing on the 
market of GMOs as or in products under Directive 2001/18/EC 

ABB activities: 

17 04 Food and feed safety, animal health, animal welfare 
and plant health 

162

 
 
 
 
 
 
 
 
 
 
 
Timing:  

 March 2011  

Background, scope and focus  

This  evaluation  examined  the  issues  that  have  arisen  in  the  implementation  of  the  EU’s 
legislative framework governing cultivation of genetically modified organisms.  
The main focus was  
– 
to  gather  information  on  the  issues  faced  by  those  involved  in  the  operation  of  the  EU’s 
legislative framework on the cultivation of genetically modified organisms and the marketing of 
their other uses; and  
– 

to generate options and ideas on how any problems that are identified might be addressed. 

Summary of findings and conclusions  

At time of adoption, the Directive and Regulation constituted an attempt at a new and improved 
legislative framework governing what had proven to be a difficult area of EU policy. Yet, from 
the time it came into force until March 2010 the EU did not adopt a single decision, positive or 
negative,  on  an  application  to  cultivate  a  GMO.  The  Amflora  potato  is  the  first  cultivation 
approval  in  Europe  since  1998.  Applications  cycle  within  the  system  are  stalled,  inch  forward 
and  then  cycle  again  at  the  next  stage  of  the  process.  Dissatisfaction  and  frustration  is 
widespread in many quarters. 

This  evaluation  was  asked  to  consider  whether  the  legislative  framework  in  the  field  of 
cultivation  of  GMOs  under  Directive  2001/18/EC  and  Regulation  (EC)  1829/2003  and 
marketing  of  their  other  uses  under  2001/18/EC  is  fit  for  purpose.  The  empirical  evidence  is 
clear  –  as  currently  implemented,  it  is  not.  Despite  the  best  effort  of  many  hard-working 
individuals and ongoing efforts to improve it, the system is not working as envisaged and is not, 
in aggregate, meeting its objectives. As the analysis in this report and earlier outputs from this 
evaluation have suggested, the ‘dysfunction’ arises as a consequence of a complex set of factors, 
both external and internal to the authorisation process. 

The  external  socio-political  environment  undoubtedly  affects  the  way  actors  engage  with  and 
use  the  authorisation  process.  The  EU  is  attempting  to  operate  an  approval  system  based  on  a 
science-based  safety  assessment  for  products  that  many  in  Europe,  including  Member  State 
governments, object to on socio-economic and ethical grounds. And while extensive efforts have 
been made to ensure that the appraisal systems are rigorous, they struggle to accommodate the 
particular  assumptions,  perceptions  of  risk  and  local  concerns  of  different  actors  The  resulting 
frustration triggers objections, which result in requests for further analysis, which increases the 
workload on the system, which in a world of finite resource leads to more delays, which further 
increases frustration. 

The evaluation has identified a number of options that could help to fine-tune (through changes 
in procedure and process) and lubricate (through greater resources) the authorisation machinery. 
These changes are worthwhile: each could make a small contribution towards creating a system 
that is more efficient, time-limited and transparent. 

None  of  these  modifications,  however,  directly  address  the  gap  between  the  scope  of  some 
actors’ concerns with GMO products and the scope of the EU’s authorisation process as set out 
in the legislation. The Commission’s initiative to bring forward options that allow more choice 

163

 
 
 
 
 
to  Member  States  in  deciding  whether  to  cultivate  GMOs  does,  however,  reach  into  this  more 
difficult  territory.  Consideration  of  these  more  fundamental  reforms  should  not  delay  or 
substitute  for  efforts  to  improve  the  authorisation  system  as  it  exists  today  –  improving  the 
specification  of  processes  where  needed  and  looking  at  whether  injection  of  more  resources 
would help those processes work faster and better. 

Availability of the report on Europa  

http://ec.europa.eu/food/food/biotechnology/evaluation/docs/gmo_cultivation_report_en.pdf  

164

 
 
 
 
 
 
18 – Home affairs 

Report from the Commission to the European Parliament and the Council on the 
application of Directive 2003/109/EC concerning the status of third-country nationals 
who are long-term residents 

Evaluation related study

ABB activities: 

18 02 Solidarity - External borders, return, visa policy and 
free movement of people  

Timing:  

 28/09/2011; COM(2011) 585 final 

Background, scope and focus  

Directive 2003/109/EC establishes common rules on granting third country nationals an EU long 
term residence permit after they have resided legally in the EU for five years.  

This new status created by the Directive was called for by the European Council in 1999 with a 
view  to  fostering  the  integration  of  long  term  residents  in  the  Member  States  and  promoting 
economic and social cohesion. 

To  achieve  this  aim,  the  third-country  nationals  that  are  granted  an  EU  long-term  residence 
permit are granted a secure residence status, including a set of uniform rights which are as close 
as possible to those enjoyed by the citizens of the EU. They enjoy equal treatment with regard to 
access  to  employment  and  self-employed  activities,  education  and  vocational  training, 
recognition of professional diplomas, social security, social assistance and social protection, tax 
benefits, access to goods and services. Under certain conditions they also have the right to reside 
in another Member State. 

When applying for the EU long-term residence status third-country nationals have to prove that 
they  comply  with  certain  conditions  (stable  and  regular  resources,  sickness  insurance,  and 
integration measures). Once they achieve this status, it is a permanent one and it can be lost only 
in very few cases. 

Summary of findings and conclusions  

According to the findings of the report, there is still a weak impact of the Long Term Residents 
Directive in many Member States. There are four Member States having around four fifths of all 
long term residents (EE, AT, CZ, IT). And, Member States with a large number of third-country 
nationals, like DE or FR, issuing so far only around 2 000 EU long-term resident permits. The 
available data also shows that only a very small number of the long term residents are using their 
mobility rights under the Directive.  

On  one  hand,  the  report  reveals  many  deficiencies  in  the  transposition  of  the  Directive  – 

165

 
 
 
 
 
 
 
 
 
restrictive interpretation of the scope of the Directive, additional conditions for admission, very 
high fees for the application in granting the permit, illegal obstacles to intra-EU mobility, not a 
proper application of the equal treatment clause. On the other hand, a real problem seems to be 
the general lack of information among many third-country nationals about the existing status as 
an EU long-term resident and the rights attached to it. 

The Commission intends therefore to increase its efforts to ensure that the Directive is correctly 
transposed  and  implemented  across  the  EU.  In  order  to  achieve  this  result,  the  Commission  is 
considering  launching  further  infringement  proceedings  when  necessary  (a  ruling  of  the 
European Court of Justice on fees under the Directive was issued on 26 April 2012, C-508/10). 
The Commission will also increase its efforts to better inform the third-country nationals on their 
rights  under  the  Directive.  For  this  purpose,  it  will  make  use  of  the  tools  already  available 
(Immigration Portal, Europa website) and support Member States in launching awareness-raising 
campaigns. 

Availability of the report on Europa  

http://ec.europa.eu/home-affairs/news/intro/docs/20110928/1_EN_ACT_part1_v62.pdf  

Report from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions  
on the results achieved and on qualitative and quantitative aspects of implementation 
of the European Refugee Fund for the period 2005-2007 
(COM(2011)2) 

Evaluation related study

ABB activities: 

18 03 Migration flows - Common immigration and asylum 
policies 

Timing:  

 14/01/2011  

Background, scope and focus  

Article 54(5) of Decision No 573/2007/EC of the European Parliament and of the Council lays 
down that the Commission shall submit to the European Parliament, the Council, the European 
Economic  and  Social  Committee  and  the  Committee  of  the  Regions  a  report  on  the  results 
achieved and on the qualitative and quantitative aspects of the implementation of the European 

166

 
 
 
 
 
 
 
 
 
 
 
Refugee Fund for the period 2005 to 2007.  

Summary of findings and conclusions  

Actions  supported  by  the  Fund  covered  one  or  more  of  the  three  following  areas:  reception 
conditions of asylum seekers and asylum procedures; integration of target group persons whose 
stay in the Member States is of a lasting or stable nature; and the voluntary return of target group 
persons  to  their  country  of  origin.  Ninety  three  per  cent  of  the  budget  appropriations  were 
allocated to the Member States for carrying out national programmes under shared management, 
while the rest was available for Community actions implemented directly by the Commission to 
finance transnational actions or actions of interest to the European Union as a whole. 

The  total  costs  of  all  operations  funded  through  the  national  programmes  amounted  to  almost 
€300 million. The Fund’s contribution accounted for slightly more than half of that amount. In 
total, 1 403 projects were funded. The majority of project beneficiaries were non-governmental 
organisations  and  local  authorities  (82  %).  From  the  three  Actions  eligible  for  support,  the 
Action  'Reception  conditions  and  asylum  procedures'  was  the  most  important  in  terms  of  the 
number of operations funded, followed by the Actions 'Integration' and 'Voluntary return' (in this 
order). 

In most of the Member States the results achieved met the planned targets. Integration operations 
proved  to  be  the  most  successful  ones,  followed  by  those  targeting  the  reception  of  asylum 
seekers, whereas operations on voluntary return produced the least satisfying results. The main 
reasons why certain operations achieved results in excess of those planned, or did not achieve 
them,  were  connected  not  to  the  type  of  activity  concerned,  but  rather  to  external  factors, 
especially the unpredictability of asylum flows. 

The costs incurred to achieve the projects’ outputs seemed reasonable. The resources needed to 
implement the projects were assessed as sufficient in general. The timeliness of fund availability 
was  less  satisfactory.  Funds  were  available  to  implementing  organisations  later  than  expected; 
this was due to the fact that not all EU provisions were in place until one year after the beginning 
of the programming period, but national delays were also a factor. 

The  use  of  the  Fund  proved  to  be  additional  to  national  funding.  There  were  only  very  few 
instances  where  projects  could  have  been  funded  through  national  support.  Moreover,  in  no 
single case did the Fund replace financing from other European funds. As a result, few, if any, of 
the projects funded would have been implemented in the absence of the Fund. 

The  projects  provided  direct  benefit  to  a  total  target  group  of  more  than  350  000  persons.  Of 
these,  26  200  persons  returned  to  their  country  of  origin  under  voluntary  return  schemes.  In 
addition, more than 6 500 persons belonging to the project implementing organisations received 
support from the Fund, e.g. for training or as a result of recruiting additional staff. 

Significant  impacts  were  identified  mainly  in  the  improvement  of  reception  conditions  and 
asylum procedures in the Member States. In particular, in the Member States which joined the 
European  Union  from  2004,  the  projects  provided  a  major  contribution  to  the  renovation  or 
extension of reception centres for asylum seekers or to the quality of the services they provide to 
asylum seekers, thus enabling meeting the EU standards. Impacts on the integration of refugees 
and  on  the  voluntary  return  of  target  group  persons  were  less  pronounced  at  EU  level;  they 
appeared to be specific to some Member States. 

Finally, projects produced tangible, lasting effects for the organisations implementing projects in 
the  form  of  an  increase  in  experience,  qualification  and  institutional  capacity,  as  well  as  in 
relation to fund and project management; cooperation and networking with other organisations; 

167

 
 
 
and recognition from national authorities. 

Availability of the report on Europa  

http://ec.europa.eu/transparency/regdoc/rep/eims/HOME/136663/COM%282011%292.pdf  

Report from the Commission to the European Parliament and the Council on the 
application of Directive 2005/71/EC on a specific procedure for admitting third 
country nationals for the purposes of scientific research 

Evaluation related study

ABB activities: 

18 02 Solidarity - External borders, return, visa policy and 
free movement of people 

Timing:  

20/12/2011; COM(2011) 901 final 

Background, scope and focus  

Directive  2005/71/EC  establishes  a  special  admission  procedure  and  conditions  of  entry  and 
residence  applicable  to  third  country  nationals  for  stays  of  more  than  three  months  in  the 
Member States for the purposes of conducting a research project under a hosting agreement with 
a research organisation. 

The Directive has created a common entry scheme for researchers who will work for a research 
organisation of  a  Member  State.  Its  objectives  are  twofold:  on  the  one  hand,  to  simplify entry 
procedures for third-country researchers and on the other hand, to give responsibility to research 
organisations in Member States and give them a leading role in the procedure. This is done using 
the  instrument  of  'hosting  agreements'  which  are  agreed  between  a  researcher  and  a  research 
organization specifically recognised for this purpose. 

The  Directive  makes  the  EU  more  attractive  to  third-country  researchers,  by  providing  them 
with a set of rights (mainly on family reunification, teaching, equal treatment with nationals in 
areas  such  as  diploma  and  professional  qualifications  recognition,  working  conditions,  social 
security  benefits,  tax  benefits  and  access  to  goods  and  services).  Importantly,  it  also  contains 
provisions on intra-EU mobility.  

Summary of findings and conclusions  

168

 
 
 
 
 
 
 
 
 
 
 
 
The legal analysis showed that Member States have transposed most of the key elements of the 
Directive,  be  it  the  approval  of  research  organisations,  hosting  agreements  or  application 
procedures.  Yet  there  is  a  clear  need  for  improvements,  highlighted  by  the  low  numbers  of 
researchers  admitted  under  the  Directive  (6,945  in  2010,  many  coming  from  India,  China,  the 
United States, the Russian Federation and Japan). This is in stark contrast to  the challenges at 
stake - in order to meet the Europe 2020 target of increasing R&D investment to 3% of GDP, 
one million additional researchers will be needed. 

Member  States'  effective  implementation  of  the  Directive  is  paramount,  in  particular  by 
providing  clear  and  unambiguous  definitions  of  researchers'  rights,  safeguarding  them  in  the 
event of the withdrawal of a research organisation's approval, and by making clear distinctions 
between permits for researchers and other types of permits. Basic definitions such as 'researcher' 
and 'research organisation' need to gain a uniform understanding in all Member States bound by 
the Directive. The report also points to a number of areas in which the Directive may need to be 
amended.  These  include  clear  definitions  regarding  the  legal  quality  and  format  of  hosting 
agreements,  a  more  uniform  way  of  updating  and  publishing  the  list  of  research  organisations 
and possibly also laying down time limits for deciding on an application. 

Therefore,  the  Commission  intends  to  propose  amendments  to  the  Directive  during  2012. 
Moreover, the Commission will continue to ensure that the Directive is correctly transposed and 
implemented across the EU, including by launching infringement proceedings where necessary. 
The Commission will also continue working at the technical level with the Member States and 
intends to better inform and assist both the Member States and third-country nationals, making 
the best use of the internet, in particular the EU Immigration Portal. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0901:FIN:EN:PDF 

Mid-term evaluation of the Framework Programme “Security and Safeguarding 
Liberties (2007-2013) (SSL)”  

ABB activities: 

18 05 Security and safeguarding liberties 

Timing:  

June 2011 

Background, scope and focus  

169

 
 
 
 
 
 
 
 
 
 
 
 
The Framework Programme “Security and Safeguarding Liberties (2007-2013) (SSL)”, consists 
of  two  Programmes:  “Prevention  and  Fight  against  Crime  (ISEC)”  and  “Prevention, 
Preparedness  and  Consequence  Management  of  Terrorism  and  other  Security  Related  Risks 
(CIPS)”.  

According to their respective legal basis (Decision 2007/125/JHA, article 15.3(c) and Decision 
2007/124/EC, Euratom, article 14.3(c), OJ L58 of 24.2.2007), the Commission has submitted to 
the European Parliament and the Council an interim evaluation report on the results obtained and 
the qualitative and quantitative aspects of the programmes. The report covered the 2007 – 2009 
period as well as initial data related to 2010. 

Summary of findings and conclusions  

During the 2007 – 2009 period, the CIPS and ISEC Programmes cumulatively supported nearly 
400  projects  and  financed  approximately  150  procurement  contracts,  worth  a  total  of  €  213 
million. ISEC allocations amounted to € 167 million, for CIPS to € 46 million. The bulk of the 
funding  went  to  280  action  grants,  accounting  for  61%  of  total  resources,  followed  by 
approximately 100 FPA grants, accounting for 24% of funds. Procurement accounted for about 
10% of the funds. 

The  Programmes  provided  support  for;  a  wide  range  of  activities,  from  training  to  the 
purchasing of equipment, and from the preparation of technical publications to the assistance of 
cross border police operations (including the Joint Investigation Teams).  

Both  Programmes  presented  a  high  level  of  trans-nationality  which  was  generally  highly 
appreciated by stakeholders. The Programmes are aimed at a varied audience, ranging from law 
enforcement  officers  to  members  of  civil  society  organisations,  and  from  researchers  to  legal 
practitioners. Law enforcement officers were the main applicant group in both ISEC and CIPS. 
In  general,  the  geographical  distribution  of  projects  was  fairly  skewed,  with  a  relatively  small 
number of countries playing a leading role. 

For the ISEC programme, during the period covered, the programme's activities were found to 
correspond  well  with  the  EU’s  priorities  and  policies.  The  initial  priorities  of  the  ISEC 
programme were reconfirmed in the Commission’s Communication of 22 November 2010 on 
the  Internal  Security  Strategy  in  Action  ("Internal  Security  Strategy")  as  part  of  the  future 
strategic  objectives  and  priority  actions  in  the  field  of  Internal  Security.  For  the  CIPS 
programme, the level of intervention among the various domains corresponded to the priorities 
and policies agreed in the field of Critical Infrastructure Protection, which were confirmed in the 
Stockholm  Programme.  The  Commission’s  Communication  of  22  November  2010  has 
reconfirmed the overall importance of Critical Infrastructure Protection.  

In  general,  projects  supported  by  the  Programmes  were  largely  effective  in  delivering  their 
planned  outputs.  However,  about  two  thirds  of  the  projects  were  not  expected  to  use  all  the 
funds  available,  mostly  because  there  was  an  initial  overestimation  of  costs.  In  addition, 
administrative application procedures were considered relatively lengthy.  

Nevertheless, the results sought and achieved reflected the varied nature of the projects funded, 
ranging  from  operational  outcomes  (e.g.  arrests,  seizures  etc.)  to  less  tangible  results  such  as 
awareness-raising  and  development  of  new  approaches.  The  most  common  types  of  results 
achieved  were  the  development  and  adoption  of  new  tools  and  methodologies,  such  as 
databases,  and  the  spreading  of  best  practices,  but  many  projects  also  contributed  to 
furthering  the  knowledge  of  specific  issues  and/or  solutions.  Overall,  the  level  of 
achievement of results was broadly satisfactory. 

170

 
 
 
As  a  general  conclusion,  it  was  considered  that  ISEC  and  CIPS  programmes  delivered  the 
desired outcomes reasonably well and that implementation of both programmes was reasonably 
successful. 

Availability of the report on Europa  

http://ec.europa.eu/home-affairs/funding/security/funding_intro_en.htm  

Direct link to the document: http://ec.europa.eu/home-
affairs/funding/docs/COM2011318final16062011.pdf  

Report from the Commission to the European Parliament and the Council on the 
Application of Directive 2004/114/EC on the conditions of admission of third country 
nationals for the purposes of studies, pupil exchange, unremunerated training or 
voluntary service 

Evaluation related study

ABB activities: 

18 02 Solidarity - External borders, return, visa policy and 
free movement of people 

Timing:  

28/09/2011; COM(2011) 587 final 

Background, scope and focus  

Directive 2004/114/EC establishes common rules of admission for non EU nationals to an EU 
Member State for the purposes of studying, pupil exchange, unremunerated training or voluntary 
service.  

The overall objective of the Directive is to promote Europe as a world centre of excellence for 
studies. As part of this effort, the Directive aims to set up a common legal framework, making it 
easier  for  third-country  nationals  to  enter  and  stay  in  the  EU  for  the  purpose  of  studying. The 
Directive  also  lays  down  a  set  of  basic  rules  on  the  conditions  under  which  school  pupils, 
unremunerated trainees and volunteers may enter and stay in the EU. As opposed to the rules on 
students,  Member  States  may  choose  whether  to  apply  the  Directive  to  these  three  groups  of 
migrants.  

The  report  meets  the  Commission’s  reporting obligation  under  Article  21  of  the  Directive  and 
evaluates  the  state  of  transposition  of  the  Directive,  identifies  problematic  issues  which  are 
impeding  the  full  potential  of  this  EU  instrument  and  proposes  a  way  forward  to  address  the 
identified shortcomings and further reinforce the attainment of its objectives.  

171

 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

According  to  a  report  issued  by  the  Commission,  foreign  students  still  experience  difficulties 
coming to the EU to study, due to an uneven implementation of the relevant EU legislation. The 
first  report  on  how  Member  States  have  implemented  EU  rules  on  the  entry  and  residence  of 
students,  pupils,  unremunerated  trainees  and  volunteers  from  third  countries  suggests  that  the 
use of these forms of temporary migration could be further improved. In particular, the level of 
harmonisation  achieved  by  the  Directive  and  the  rights  it  grants  could  be  increased  and 
strengthened. 

The  report  reveals  a  crucial  need  for  amendments  to  the  Directive,  especially  regarding:  the 
reinforcement of procedural guarantees; the strengthening of mobility clauses; the stimulation of 
synergies  with  EU  programmes  that  facilitate  third-country  nationals'  mobility  into  the  EU;  or 
the improvement of the level of harmonisation as regards volunteers, unremunerated trainees and 
school pupils.  

Therefore,  the  Commission  intends  to  propose  the  amendments  to  the  Directive  during  2012. 
Moreover, the Commission will continue to ensure that the Directive is correctly transposed and 
implemented  across  the  EU,  including  by  launching  further  infringement  proceedings  where 
necessary. The Commission will also continue working at the technical level with the Member 
States  and  intends  to  better  inform  and  assist  both  the  Member  States  and  third-country 
nationals, making the best use of the internet, in particular the EU Immigration Portal. 

Availability of the report on Europa  

http://ec.europa.eu/home-affairs/news/intro/docs/20110928/1_EN_ACT_part1_v6.pdf  

Report from the Commission to the European Parliament, the Council, The European 
Economic and Social Committee and the Committee of the Regions 
On the results achieved and on the quantitative and qualitative aspects of 
implementation of the External Borders Fund for the period 2007-2009 (report 
submitted in accordance with Article 52(3)(b) of Council Decision 574/2007/EV of 23 
May 2007) 

Evaluation related study

ABB activities: 

18 02 Solidarity - External borders, return, visa policy and free movement of 
people  

Timing:  

 30/06/2010 

172

 
 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

One  of  the  four  Funds  of  the  2007-2013  General  Programme  "Solidarity  and  Management  of 
Migration  Flows"  with  a  total  allocation  of  4.032,23  million  Euros,    is  the  External  Borders 
Fund1, established for the period 2007 – 2013 with a total envelope of 1.820 million Euros. This 
amount  is  supplemented  by  the  contributions  from  the  Schengen  Associated  States,  Norway, 
Iceland, Switzerland and Lichtenstein, participating in the Fund as of 2010. 

The basic act establishing the Fund requires the Commission to submit an intermediate report on 
the results achieved and on the qualitative and quantitative aspects of the implementation of the 
Fund2.  

The  report  presents  results  achieved  by  the  annual  programmes  2007-2009  as collected  on  the 
basis of reports from the Member States in the second half of 20103, completed with information 
available  with  the  Commission  in  the  first  quarter  of  2011.  These  results  are  intermediate 
pending the approval of the final reports on the implementation of the programmes4.  

The report also provided input in the reflection process on the future instruments.  

Summary of findings and conclusions  

Having  reviewed  the  reports  from  the  Member  States,  and  also  taking  into  account  other 
information  (including  from  monitoring  missions),  the  Commission  finds  that,  despite  starting 
difficulties and some specific issues with the rules and implementation, the Fund lived up to its 
objectives. For 2007-2009 it clearly fulfilled its purpose as a specific vehicle to channel EU aid 
to boost investments at borders and in consulates in the interest of the Schengen area as a whole.  

As illustrated in section 5.3.3, the Fund actively leveraged important EU policy initiatives and 
projects.  As  such,  it  contributed  significantly  to  furthering  integrated  border  management  and 
the  use  of  new  technologies  to  facilitate  legitimate  travel.  Besides  the  development  of  the 
Schengen  Information  Systems  and  the  Visa  Information  System,  the  Fund  supported  pilot 
projects  on  the  introduction  of  automated  border  controls  and  preparation  for  the  registered 
travellers' programme. Thanks to the financial contribution of the Fund to these measures, even 
if  relatively small, valuable  experiences  were gained  through  the  development  of  such 
innovative  systems which  will  be an 
the 
important 
further preparations of the 'smart borders concept' for the future, notably with regard to making it 
possible to use them (for third-country nationals as well). In that respect, the Member States are 
encouraged to include fingerprint readers in their ABC projects, in order to make them future-
proof  in  relation  to  the  EES  and  RTP  (Smart  Borders  initiative).  The  Fund  proved  also  very 
useful  to  boost,  in  a  very  short  period  of  time,  investments  in  vulnerable  areas,  such  as  the 
Mediterranean. 

the  Commission 

input 

for 

in 

It has helped improve the management of the EU external borders, fighting irregular migration 
as well as enhancing Member State cooperation, as testified by the following results: 

1  Decision No 574/2007/EC, OJ L144, 6.6.2007, p.22 
2  Article 52(3)(b) of Decision No 574/2007/EC 
3  A  compilation  and 

the  country 

reports  are  available  at  http://ec.europa.eu/home-

affairs/funding/borders/funding_borders_en.htm   

4  Article 51 of the basic act  

173

 
 
 
 
                                                 
–  Firstly,  thanks  to  the  investments  in  national  communication  systems,  border  crossing 
infrastructure, state-of-the-art operating equipment for border checks and surveillance, means of 
transport  and  training  for  border  guards,  Member  States  have  benefited  from  significant 
technical  improvements  at  the  external  borders  and  at  border  crossing  points,  resulting  in 
increased  safety  at  EU  external  borders,  more  effective  border  traffic  control  and  improved 
coordination between the various law enforcement authorities at national level. Thus, the Fund 
has  contributed  to  the  national  efforts  to  handling  efficiently  the  external  borders  and  in 
particular  to  fighting  illegal  entry  into  the  Schengen  area,  completing  past  investments 
(Schengen  Facility)  and  strengthening  further  the integrated  border  management  system  of  the 
EU.  

–  Secondly,  under  the  2007-2009  annual  programmes,  the  Member  States  purchased  and 
upgraded the necessary equipment for the VIS and SIS II, enabling a smooth connection of the 
national  platforms  to  the  central  systems.  Given  that  all  participating  Member  States  must  be 
ready  with  development  of  their  national  systems,  in  order  for  these  European  projects  to 
function properly as a whole, the Fund, as a long term framework for complementing national 
funding, has proven to be of vital importance for the implementation these common IT projects.  

–  Lastly,  the  Member  States  made  use  of  the  additional  resources  provided  by  the  Fund  for 
measures  to  improve  the  visa  handling  process  by  investing  in  modern  technology  equipment 
(including biometric capture devices) and strengthening the security of consular offices. These 
measures  have  contributed  to  a  swifter  processing  of  visa  applications,  enhancement  in  the 
quality of services and an equal and fair treatment of visa applicants.  

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52011DC0857:EN:NOT 

External Evaluation of the European Migration Network 

ABB activities: 

18 03 Migration flows - Common immigration and asylum 
policies 

Timing:  

13/11/2011  

Background, scope and focus  

174

 
 
 
 
 
 
 
 
 
 
 
 
 
As indicated in Article 13 of Council Decision 2008/381/EC establishing the EMN, the purpose 
was  to  perform  an  external  and  independent  evaluation  on  the  development  of  the  EMN  since 
the  entry  into  force  of  the  decision,  i.e.  since  21st  May  2008.  It  covered  the  following  broad 
aspects: 

– 

the extent to which the EMN meets its objectives and how/what can be improved; 

the  functioning  of  the  EMN,  including  of  its  various  members  (Commission,  Service 

– 
Providers, EMN NCPs); 

–  how  the  EMN  might  further  contribute  to  the  more  effective  use  of  existing  information 
sources and networks to ensure the availability of comparable data on migration issues, with a 
view  to  better  informing  policy  choices  (in  accordance  with  Section  6.1.3  of  the  Stockholm 
Programme). 

Moreover,  the  evaluation  explored  whether  and  to  what  extent  the  Council  Decision  might  be 
amended. 

Summary of findings and conclusions  

The Table below gives an overall summary assessment of the EMN's performance. 

Table 1: Overall Summary – EMN Performance 

Positive 

Less Positive 

•  Quality of EMN outputs (mostly up-to-date, objective, reliable and comparable) with Ad-Hoc 

Queries especially highly valued 
Positive influence on policy-making, especially at the EU level, but also in relation to many 
Member States 
Intensity of networking between EMN NCPs and their commitment to the EMN 

• 

• 

• Quality and quantity of outputs compared with level of financial and human resources Time-lags in 
producing information and limited comparability, especially when not all EMN NCPs make an 
input 

•In relation to the wider public, very limited impact and uncertainty as to  the EMN’s role 
•Relatively low visibility of EMN (e.g. low number of website hits and newsletter registrations) and 

• 

weak national networks in some countries 
Lack of resources at EU level to support EMN’s development. 

Overall, the EMN was considered to be performing well, providing useful information to support 
policymaking  at  the  EU  and  Member  State  level.  Less  positive  were  the  limited  readability of 
the  information  it  produced  (e.g.  too  long)  for  policymakers  and  the  lack  of  visibility  or 
awareness of the EMN amongst other practitioners as well as policymakers, particularly within 
the EU institutions.  

Some  thirty  recommendations  to  improve  further  the  functioning  of  the  EMN  were  identified. 
Indeed,  even  whilst  the  evaluation  was  underway,  the  EMN  took  on  board  several 
recommendations  in  the  development  of  its  annual  work  programme  for  2012,  in  particular  in 
relation  to  ensuring  that  information  provided  was  more  relevant,  up-to-date  and  with  concise 
summaries of findings for policymakers in particular. 

175

 
 
 
 
 
 
 
Availability of the report on Europa  

Report not yet available on Europa, but is publicly available at: 

http://emn.intrasoft-intl.com/Downloads/prepareShowFiles.do?directoryID=249.  

Report from the Commission to the European Parliament, the Council, The European 
Economic and Social Committee and the Committee of the Regions 
on the results achieved and on the quantitative and qualitative aspects of 
implementation of the European Return Fund for the period 2008-2009 (report 
submitted in accordance with Article 50(3)(b) of Council Decision 575/2007/EV of 23 
May 2007) 

Evaluation related study

ABB activities: 

18 02 Solidarity - External borders, return, visa policy and free movement of 
people 

Timing:  

30/06/2010 

Background, scope and focus  

One  of  the  four  Funds  of  the  2007-2013  General  Programme  "Solidarity  and  Management  of 
Migration  Flows"  with  a  total  allocation  of  4.032,23  million  Euros,  is  the  European  Return 
Fund1, established for the period 2008 – 2013 with a total envelope of 676 million Euros.  

The basic act establishing the Fund requires the Commission to submit an intermediate report on 
the results achieved and on the qualitative and quantitative aspects of the implementation of the 
Fund2.  

The  report  presents  results  achieved  by  the  annual  programmes  2008-2009  as collected  on  the 
basis  of  reports  from  Member  States  in  the  second  half  of  20103,  completed  with  information 
available  with  the  Commission  in  the  first  quarter  of  2011.  These  results  are  intermediate 
pending the approval of the final reports on the implementation of the programmes4.  

1  Decision No 575/2007/EC, OJ L144, 6.6.2007, p.45 
2  Article 50(3)(a) of the basic act 
3  A  compilation  and 

the  country 

reports  are  available  at  http://ec.europa.eu/home-

affairs/funding/return/funding_return_en.htm  

4  Article 51 of the basic act  

176

 
 
 
 
 
 
 
 
 
 
 
 
                                                 
Summary of findings and conclusions  

Having  reviewed  the  reports  from  Member  States,  and  taking  into  account  other  information, 
including from monitoring missions, the Commission finds that, despite starting difficulties and 
some specific issues with the rules and implementation, the Fund is living up to its promise to 
boost the implementation of integrated return management in Member States.  

– 
In a number of Member States thanks to specific information campaigns, further information 
on return possibilities was made available and disseminated widely to migrant communities in 
Member States. In some cases, these outreach efforts were based on the practical experiences of 
persons who had returned and are enjoying a successful reintegration in their countries of origin.  

–  The Fund supported the introduction or consolidation of return management practices which 
had  been  almost  non-existent  in  many  Member  States  beforehand.  Projects  successfully 
enhanced  the  capabilities  of  government  services,  in  co-operation  with  non-governmental 
organisations and/or the IOM, to encourage the voluntary return of migrants and asylum seekers. 
Assisted  voluntary  return  programmes  were  consolidated  or  extended  to  cater  for  aid  to  more 
third-country  nationals.  Several  projects  on  counselling  and  reintegration  supported  the 
development of innovative and more tailor-made tools for return management that can be rolled 
out more comprehensively in the coming years. 

– 
In addition, thanks to the Fund, Member States were able to offer more or higher financial 
incentives and/or support to returnees for setting up income-generating activities, thus resulting 
in sustainable returns.  

– 
In  the  area  of  forced  return,  the  Fund  was  consistent  with  the  activities  of  the  Frontex 
Agency on the coordination of joint return operations. Member States have successfully used the 
resources  under  the  Fund  for  national  and  joint  return  flights  which  complemented  the  efforts 
made by the Agency.  

–  Moreover,  the  cooperation  projects  between  Member  States  and  countries  of  return  have 
resulted in an increase in the number of removals by improving working arrangements with third 
countries on the identification and issuing of travel documents.  

–  Finally,  the  Fund  has  also  generated  opportunities  for  further  exchange  of  experience 
between  Member  States.  Less  experienced  Member  States  are  making  use  of  this  funding  to 
learn from the Member States that have long standing experience.  

In  terms  of  results,  the  Fund  has  therefore  already  undoubtedly  led  to  the  optimisation  of  the 
measures  for  assisted  voluntary  return,  a  reduction  in  the  length  of  stay  in  reception  and/or 
detention  centres,  improved  dissemination  of  information  to  the  target  population  on  return 
options and an increase in the number of third-country nationals actually taking advantage of the 
opportunity  of  voluntary  return,  benefiting  from  more  dignity  and  better  prospects  than  they 
would have encountered, if they had been subject to a forced removal.  

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52011DC0858:EN:NOT 

177

 
 
 
 
 
 
19 - External Relations 

Evaluation de la coopération de l'UE avec la Tunisie de 1995 à 2008  

ABB activities: 

19 02 Cooperation with third countries in the area of 
migration and asylum 

Timing:   May 2011 

Background, scope and focus  

Le  rapport  présente  les  résultats  de  l'évaluation  de  la  stratégie  de  coopération  de  l'Union 
européenne (UE) avec la Tunisie entre 1995 et 2008.Les objectifs sont de fournir aux services 
concernés  de  l'UE  ainsi  qu'au  grand  public  une  appréciation  globale  et  indépendante  de  la 
coopération passée et présente de l'UE avec la Tunisie et d'en tirer le principaux enseignements 
en vue d'améliorer les stratégies et programmes actuels et futurs. 

Par rapport à une évaluation géographique classique, la présente étude a bénéficié des résultats 
d'une étude menée en parallèle par le même bureau d'études sur les opérations d'appui budgétaire 
dont  la  Tunisie  a  bénéficié,  L'évaluation  est  structurée  autour  de  10  questions  évaluatives,  qui 
couvrent  les  différents  critères  d'évaluation  du  CAD  de  l'OCDE  et  de  la  Commission 
européenne.  

Summary of findings and conclusions  

La  coopération  a  largement  contribué  à  la  mise  en  œuvre  de  réformes  institutionnelles  et 
sectorielles  dans  les  secteurs  clefs  de  l'économie  et  de  l'éducation  .Le  soutien  continu  au 
développement  des  ressources  humaines  a  contribué  à  atteindre  des  résultats  quantitatifs 
remarquables, un défi restant toutefois à approfondir, l'employabilité des diplômés. Le soutien 
au secteur de la gouvernance et des  processus démocratiques s'est caractérisé par un dialogue 
difficile  avec  le  gouvernement  sur  le  renforcement  de  l'état  de  droit  et  sur  les  modalités 
d'ouverture  à  la  société  civile.  Au  niveau  de  la  gestion  des  programmes,  il  y  a  eu  une  bonne 
articulation entre les différents instruments de coopération 

Principales conclusions 

1.  Un  partenariat  solide  et  à  long  terme  entre  l'UE  et  la  Tunisie  caractérisé  par  une  forte 
cohérence  par  rapport  aux  axes  prioritaires  de  l'Accord  d'Association  puis  au  Plan  d'Action 
Voisinage,  embrasse  les  domaines  politiques,  économiques  et  culturels.  La  pertinence  de  la 
coopération est reflétée par sa correspondance avec les programmes inscrits aux plans nationaux 
de  développement  et  avec  les  besoins  de  croissance  économique  et  d'intégration  internationale 
du pays; 

178

 
 
 
 
 
 
 
 
 
 
2.  La  cohérence  et  la  complémentarité  entre  les  différentes  interventions  de  la  CE  sont 
significatives, mais l'approche sectorielle est faible; 

3.  La coopération de la CE a appuyé les réformes du gouvernement au niveau national/central 
mais  n'a  pas  mis  en  œuvre  des  mesures  et  des  programmes  spécifiques  d'appui  au 
développement décentralisé; 

4.  La  CE  a  contribué  dès  le  début  de  la  période  aux  réformes  dans  le  système  d'éducation 
nationale  aux  différents  niveaux  ainsi  que  dans  le  secteur  de  la  formation  professionnelle 
aidant  à  atteindre  les  objectifs  du  millénaire.  L'intégration  des  ressources  humaines  dans  le 
marché du travail représente encore un défi majeur; 

Sur les thèmes de la justice, de l'Etat de droit et des libertés d'expression et d'association 
ainsi que sur les modalités d'ouverture à la société civile, le dialogue avec le gouvernement 
a été souvent difficile.  

Principales recommandations: 

1.  Renforcer  et  relancer  le  partenariat  –  Le  dialogue  politique  sur  le  "statut  avancé"  pourrait 
représenter  pour  l'UE  et  la  Tunisie  une  occasion  pour  fixer  de  nouveaux  objectifs  stratégiques 
qui  puissent  élargir  et  relancer  le  cadre  d'opportunités  représenté  jusqu'à  présent  par  l'Accord 
d'Association.  Il  convient  notamment  dans  ce  cadre  de  donner  une  nouvelle  impulsion  aux 
politiques de compétitivité et d'emploi; 

2.  Appui aux politique sectorielles du gouvernement tunisien – Cet appui devrait se poursuivre 
à  tous  les  niveaux  et  s'ouvrir  davantage  à  la  concertation  préalable  avec  les  intervenants 
nationaux et avec les autres bailleurs de fonds; 

3.  Cohésion  territoriale  –  Afin  de  consolider  les  résultats  des  réformes  sectorielles  soutenues 
par la CE et de favoriser leur impact sur le développement décentralisé, il convient d'envisager 
un soutien à la mise en œuvre de véritables politiques de cohésion territoriale, selon les principes 
de  bonne  gouvernance  multi  niveau  et  les  modèles  de  référence  existant  dans  le  contexte 
européen; 

4.  Développement  des  ressources  humaines  –  L'appui  dans  ce  domaine  doit  être  poursuivi  et 
consolidé en veillant à une meilleure intégration entre l'éducation et la formation professionnelle, 
les politiques sectorielles et de décentralisation ainsi que les exigences du marché du travail; 

5.  Gouvernance démocratique – Les acteurs de la gouvernance tels que la justice, les droits de 
l'homme ou les médias devraient être pris en compte à tous les niveaux dans le futur, dès la 
programmation,  en  impliquant  les  acteurs  aux  niveaux  politique  et  institutionnel  ainsi  que  la 
société civile. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1287_docs_en.htm 

179

 
 
 
 
Evaluation of the European Commission's co-operation with the Philippines  

ABB activities: 

19 10 Relations with Asia, central Asia and Middle East 
(Iraq, Iran, Yemen) 

Timing:  

June 2011 

Background, scope and focus  

The  evaluation  covers  co-operation  strategies  and  implementation  including  a  mix  of  all 
activities  and  modalities  during  the  period  2002-2009.  During  that  period  EU  support  co-
operation with the Philippines has focused on combating poverty and raising standards of living 
of  the  poorest  groups  in  the  country.  This  has  included  support  for  rural  development  and  for 
agrarian reform; for the health sector; for decentralisation, local governance and governance as a 
cross-cutting  issue;  for  the  promotion  of  human  rights,  democracy  and  the  rule  of  law;  and  a 
contribution  to  peace-building.  Since  2005,  the  EU  co-operation  programme  has  primarily 
supported equitable access to social services, with particular attention to quality health services 
through a sector-wide approach and budget support mechanism. 

Summary of findings and conclusions  

The  overall  assessment  of  EU  strategy  in  the  Philippines,  during  the  period  2002-2009,  is 
positive.  The  EU  cooperation  programme  has  been  aligned  with  the  Philippine  Government’s 
policies  and  adequately  designed  and  implemented.  The  Health  Sector  Policy  Support 
Programme (HSPSP) has emphasised coordination and harmonisation of aid. Through HSPSP, 
technical assistance has contributed in reinforcing capacities. EU support in rural development 
has  contributed  to  improved  living  standards  for  the  poor,  however  the  main  projects  did  not 
address  mechanisms  perpetuating  poverty.  The  cooperation  programme  has  given  prominent 
attention to the role of trade in the development process; Philippines now is in a better position 
to participate in international trade. However, synergies between the bilateral EU-Cooperation 
programme  and  regional  EU-ASEAN  projects  are  limited  and  coordination  is  not  formally 
institutionalised.  The  EU  managed  to  establish  high  visibility  in  human  rights  and  the  anti-
corruption  agenda.  The  EU’s  mix  of  financing  instruments  and  aid  modalities  has  been 
appropriate  to  the  national  context  and  EU  cooperation  objectives.  Coordination  mechanisms 
exist  in  all  cooperation  sectors;  however,  the  specific  dynamics  of  Member  States’  national 
interests  and  the  changing  financial  volume  of  bilateral  programmes  limit  strategic  planning 
and division of labour. 

Main conclusions 

1.  The  EU  programme  has  been  highly  relevant  to  poverty  reduction  in  the  Philippines. 

180

 
 
 
 
 
 
 
 
 
 
However, several nuances or points of attenuation have emerged. For example, the main projects 
on  rural  development  did  not  engage  in  analysing  structures  and  mechanisms  perpetuating 
poverty.  

2.  Although the start of the EU support to HSPSP was slow and it took time to match sector 
budget support with the context of decentralisation, the Technical Assistance (TA) has invested 
capacity  and  succeeded  in  several  provinces.  In  a  similar  vein,  the  MHSPSP  has  contributed 
fully to the EU’s poverty reduction objective.  

3.  The  EU  enjoys  high  visibility  and  credibility  with  regard  to  governance  particularly 
concerning to the anti-corruption and human rights agenda.  

4.  EU  dialogue  with  civil  society  on  strategy  and  CS  involvement  in  the  cooperation 
programme itself is seen as exemplary. The series of MTR consultations that the EU Delegation 
held with CSOs in 2009 is a case in point.  

5.  Projects as part of both bilateral and regional programmes as well as thematic budget lines 
have  had  a  strong  focus  on  training  and  capacity-building  and  have  performed  strongly  at  the 
output 
insufficient  attempts  at  monitoring  and  assessing 
outcomes/result  of  capacity-building.  In  many  cases  capacity-building  seems  to  be  seen  as  an 
end in itself.  

level.  However, 

there  are 

6.  Overall the Philippines is in a better and stronger position to participate in international trade 
than at the beginning of the evaluation period in 2002. However, synergies between and among 
the  bilateral  EU-Cooperation  cooperation  programme  and  regional  EU-ASEAN  projects  only 
exist to a limited extent and coordination mechanisms are not formally institutionalised.  

Main Recommendations 

1.  Projects aiming at reducing poverty should carefully address who they are actually reaching 
and  with  what  effect,  and  devise  well  elaborated  systems  that  are  able  to  monitor  progress  in 
terms of poverty reduction on a regular basis.  

2.  The  EU  should  consider  assisting  GoP  in  improving  governance  in  the  health  sector  and 
increasing  participation  of  civil  society  and  private  sector  involvement.  Both  can  be  attained 
through more efficient local health systems.  

3.  Both projects in direct support of governance and the practise of mainstreaming governance 
in all cooperation sectors should continue.  

4.  More effort should be put into monitoring the effectiveness of training and capacity-building 
components of projects in all sectors. In many cases, it is not clear how trained stakeholders use 
the acquired knowledge within their respective organisations and, thus, how capacity building is 
related to overall project outcomes/results.  

5.  The  formalising  and  institutionalisation  of  project  coordination  between  bilateral  EU-
Philippines  and  regional  EU-ASEAN  programmes/projects  should  be  considered  to  achieve 
more synergies between bilateral and regional interventions that address similar sectors.  

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1299_docs_en.htm  

181

 
 
 
 
 
Evaluation des opérations d'aide budgétaire de la Commission européenne à la 
Tunisie entre 1996 et 2008 

ABB activities: 

19  08  European  Neighbourhood  Policy  and  relations  with 
Russia 

Timing:   March 2011 

Background, scope and focus  

Les  objectifs  de  l'évaluation  étaient,  d'une  part,  de  tester  la  nouvelle  approche  d'évaluation  de 
l'aide budgétaire et, d'autre part, de fournir une appréciation globale et indépendante des résultats 
de l'aide budgétaire fournie par la Commission Européenne à la Tunisie entre 1996 et 2008 afin 
d'en tirer les principaux enseignements et d'améliorer les futurs programmes d'appui budgétaire. 

Le  champ  de  l'évaluation  a  porté  sur  7  opérations  d'aide  budgétaire,  sur  les  14  financées  par 
l'UE, comportant les appuis budgétaires généraux (gérés conjointement par la Banque Mondiale 
et  la  Banque  Africaine  de  Développement)  et  les  appuis  sectoriels  dans  les  domaines  de 
l'éducation secondaire et supérieure et de la formation professionnelle. Le montant total transféré 
a  été  de  400  M€,  soit  environ  60%  de  la  totalité  de  l'aide  budgétaire  apportée  par  l'UE  sur  la 
période. 

Les  résultats  de  l'évaluation  ont  alimenté  l'évaluation  de  la  stratégie  pays  de  l'UE  réalisée 
simultanément. 

Summary of findings and conclusions  

Grâce  à  l'approche  évaluative  adoptée,  l'évaluation  a  pu  souligner  l'importance  des  appuis 
budgétaires  de  l'UE  pour  l'atteinte  des  résultats  de  développement  enregistrés  par  la  Tunisie 
entre 1996 et 2008. L'appui budgétaire a fourni un cadre de dialogue, des appuis techniques et 
des appuis financiers qui ont renforcé la mise en œuvre des stratégies du gouvernement. Celles-
ci, profitant du cadre d'opportunités offert par l'Accord d'Association, ont permis la réalisation 
de réformes économiques et sociales importantes, qui ont été à la base des succès en matière de 
croissance économique et de bien-être social enregistrés par le pays. De nombreux défis restent 
toutefois à relever. 

Principales conclusions 

En  Tunisie,  l'aide  budgétaire  générale  a  été  un  succès  dans  la  mesure  où  elle  a  contribué  à  la 
croissance de l'économie tunisienne. 

–  L'instrument a été bien adapté au contexte tunisien, à savoir un Gouvernement engagé, sur le 
long terme, dans des réformes en profondeur. 

182

 
 
 
 
 
 
 
 
 
 
–  L'Accord d'Association (AA), partenariat global euro-tunisien, a été le moteur du succès de 
la coopération car il a permis à la Commission européenne d'avoir un dialogue permanent sur les 
politiques  nationales.  Grâce  à  l'aide  budgétaire,  les  passages  cruciaux  nécessaires  à  la  réussite 
des  grandes  réformes  économiques,  mais  non  identifiés  dans  l'AA,  ont  été  définis,  discutés  et 
suivis.  L'arrivée  de  la  Commission  européenne  comme  nouvel  acteur  fournissant  de  l'aide 
budgétaire, en plus de l'appui budgétaire fourni par la Banque Mondiale depuis 1986, a permis 
d'aller plus loin qu'une approche purement technique et d'obtenir des résultats durables. 

–  Ainsi, la Tunisie a su conjuguer stabilité macro-économique et croissance. La période a été 
caractérisée  par  un  fort  dynamisme  du  secteur  privé,  notamment  grâce  à  la  réforme  de 
l'environnement  des  affaires  et  à  la  libéralisation  du  commerce  extérieur,  soutenues  par  l'aide 
budgétaire. Des points de blocages subsistent cependant, principalement liés à la rigidité et à la 
faible compétitivité du marché intérieur. 

–  Les ABG ont contribué spécifiquement à l'amélioration de la gouvernance économique, et à 
la  légalité  dans  le  monde  des  affaires,  qui  ont  enregistré  des  progrès,  bien  que  les  standards 
atteints soient encore insuffisants. 

–  Les résultats obtenus dans le secteur de l'éducation sont plus mitigés. 

–  Des  avancées  quantitatives  en  termes  d'accès  à  l'enseignement  secondaire  et  supérieur  ont 
été constatées. 

–  Cependant,  l'aide  budgétaire  "Education"  a  été  handicapée  par  un  manque  d'approche 
sectorielle, tous bailleurs de fonds confondus, de la part de la Tunisie. 

–  Les réformes de l'éducation (secondaire et supérieure) et de la formation professionnelle ont 
été abordées avec détermination, mais les résultats, en termes de qualité et d'employabilité, sont 
encore faibles. 

–  Par rapport à l'appui budgétaire sectoriel, l'appui budgétaire général s'est ainsi révélé comme 
le  moyen  le  plus  approprié  pour  appuyer  les  grandes  réformes  économiques.  Il  a  été  le  seul 
instrument  qui  a  permis  aux  partenaires  de  créer  un  espace  de  dialogue  rapproché  sur  les 
réformes à mettre en œuvre. 

Principales recommandations 

–  Relancer le partenariat euro-tunisien pour un nouveau bond stratégique de la coopération; 

–  Renforcer  l'approche  sectorielle  de  la  part  du  gouvernement  tunisien  grâce  à  un  dialogue 
stratégique ouvert aux principaux partenaires internationaux dans les macro-secteurs prioritaires; 

–  Utiliser  l'appui  budgétaire  pour  parachever  les  grandes  réformes    et  les  étendre  à  de 
nouveaux  secteurs  (par  ex.  services  et  agriculture)  dans  une  perspective  d'accroissement  des 
investissements et de création d'emplois; 

–  Conjuguer, de façon équilibrée, les différents inputs des aides budgétaires : fonds, dialogue 
sur les politiques et conditionnalités, assistance technique, harmonisation et alignement. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1286_docs_en.htm  

183

 
 
 
 
 
 
Evaluation of the crisis response and preparedness components of the European 
Union’s Instrument for Stability (IfS) 

ABB activities: 

19 06 Crisis response and global threats to security 

Timing:  

 July 2011 

Background, scope and focus  

In line with Article 21 of the Regulation of the European Union’s Instrument for Stability (IfS), 
the European Commission authorised an independent evaluation of the IfS’ crisis response and 
preparedness  components  —  Articles  3  and  4(3)  of  the  IfS  Regulation,  respectively.  Of  the 
€2.062  billion  originally  allocated  to  the  IfS  in  2007-2013,  the  crisis  response  component 
(including conflict prevention measures) was to receive no less than 72% (€1.49 billion), while 
no  more  than  5%  was  to  be  allocated  to  the  component  on  crisis  preparedness  (Article  4(3)), 
including capacity building. The allocation of funding to the different areas of focus reflects the 
intention  of  the  Council  and  European Parliament  to  make  the  IfS  primarily  an  instrument  for 
crisis response (including prevention). 

As  the  first  horizontal  appraisal  of  these  components  since  the  IfS  inception  in  2007,  this 
evaluation assesses the relevance of the two components across various dimensions, with a view 
to  contributing  to  upcoming  decision-making  about  the  future  of  the  IfS  after  its  current  legal 
basis expires in 2012. In line with the terms of reference, the main objective of the evaluation is 
“…to ascertain the results to date of the Instrument for Stability crisis response and preparedness 
components  and  help  enhance  IfS  approaches  towards  future  crisis  response  measures  and 
strategies for future preparedness programming.” 

The key legal and policy documents related to the IfS’ creation and functioning, as well as key 
policy  documents  of  the  European  Union  in  the  area  of  peace  and  security,  constitute  the 
baseline of the evaluation. In addition, a range of programme documents related to selected case 
studies were examined. As this is an overall programme-level evaluation, its scope and process 
did  not  include  specific  in-depth  project-level  reviews;  hence  this  evaluation  is  not  meant  to 
draw conclusions on the effectiveness and impact of specific IfS crisis actions on the ground, nor 
on  capacity-strengthening  activities  at  project  level.  However,  documentation  on  five  case 
studies  was  reviewed  to  bolster  the  conclusions  made  here.  These  comprised  four  Exceptional 
Assistance  Measures  (EAMs)  in  Georgia,  Sudan,  Somalia  and  Indonesia,  and  support  to  Post 
Conflict Needs Assessment (PCNA) and Post Disaster Needs Assessment (PDNA) methodology 
development and training. 

In order to refine findings, the evaluators elaborated a set of ‘qualifiers’ used to review findings 
as they emerged from the evaluation process. These include: 

Institutional  opportunities/constraints,  ranging  from  access  to  high  quality  information  on 

– 
crises, to a small team managing a large budget; 

184

 
 
 
 
 
 
 
 
 
 
–  The inherently political nature of the IfS, which enables the EU to play an enhanced role as a 
foreign and security policy actor; 

–  The fact that the IfS operates across the security and development nexus and is designed to 
complement,  and  often  to  prepare  the  ground  for  the  EU’s  mainstream  external  action 
instruments; 

–  The high risk and highly volatile environments in which crisis response instruments like the 
IfS often operate; and 

–  The  complex  nature  of  the  international  crisis  preparedness,  response  and  peace-building 
architecture. 

Summary of findings and conclusions  

The  evaluation  concludes  that  the  IfS  has  significantly  contributed  to  enhancing  the  overall 
relevance,  effectiveness  and  efficiency  of  EU  crisis  response  and  preparedness  actions.  It  also 
makes  a  significant  contribution  to  the  coherence  of  the  EU  peace,  security  and  development 
architecture  –  and  to  global  peace  and  stability.  Critical  to  its  contributions  is  the  IfS’ 
demonstrated capacity to provide quick, timely and catalytical responses in situations of crisis. 

The design and implementation of crisis response and preparedness actions and strategies funded 
under Articles 3 and 4(3) have been found to be in line with the Regulation, and contribute to 
achieving  the  objectives  stated  therein.  Decision-making  processes  are  sufficiently  robust  and 
inclusive; they allow an important exchange between the technical and political decision-makers 
of the EU peace, security and development architecture. The management and disbursement of 
funding  is  considered  effective  and  timely  once  IfS  measures  have  been  identified.  The 
evaluators  consider  the  IfS  to  be  a  unique  Instrument  within  the  EU  peace,  security  and 
development architecture. Articles 3 and 4(3), in particular, fill an important strategic, funding 
and  capacity  gap,  and  allow  the  EU  to  support  a  broad  range  of  critical  crisis  prevention  and 
response initiatives, as well as capacity building for crisis preparedness. 

Initiatives funded by the crisis response and crisis preparedness components have contributed to: 

–  Expanding  and  scaling  up  EU  capacity  to  intervene  in  crisis  situations  from  a  geographic, 
thematic, funding and time-horizon viewpoint, particularly through Article 3 conflict prevention 
and crisis response measures; 

–  Leveraging  and  enhancing  the  coherence  and  effectiveness  of  the  EU  peace,  security  and 
development instruments and initiatives (including the Common Security and Defense Policy) in 
support of critical crisis prevention, stabilisation and peace-building objectives;  

–  Providing relevant policy input to high level EU decision-making and helping to kick-start 
important reforms, while also providing EU visibility and political leverage in third countries; 

–  Enhancing  the  capacities  of  potential  implementing  partners  to  prepare  for  and  respond  to 
crises; and 

–  Promoting inclusive approaches to crisis prevention and response by involving civil society, 
regional and other international partner actors. 

The  evaluators  make  the  following  recommendations  to  further  bolster  the  already  significant 
impact of the IfS and build on the contributions made by the EU through the Instrument: 

185

 
 
 
 
1.  Increase  financial  resources  for  the  IfS  in  order  to  extend  impact  to  additional  crisis 
contexts;  

2.  In the Regulation, include further flexibility in the overall timeframe for the use of the IfS 
for crisis response measures and strengthen linkages with other EU Instruments; 

3.  Further strengthen the design and focus of interventions; 

4.  Further balance political and technical aspects of decision-making; 

5.  Diversify choices of — and specify criteria for — the selection of implementing partners; 

6.  Bolster monitoring and evaluation processes; 

7.  Seek to more consistently integrate an enhanced risk management strategy into IfS project 
design; 

8.  Increase Service for Foreign Policy Instruments staff resources at HQ and country level 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

Evaluation of the European Commission's cooperation with overseas countries and 
territories  

ABB activities: 

19 11 Policy strategy and coordination for ‘External 
relations’ policy area 

Timing:  

June 2011 

Background, scope and focus  

This  evaluation  covers  the  strategies  of  the  European  Union  (EU)  for  cooperation  with  the 
Overseas Countries and Territories (OCTs) and their implementation over the period 1999-2009, 
including the intended effects under the current programming cycle 2008-2013. The objectives 
of  this  study  are  to  provide  an  independent  assessment  and  to  identify  key  lessons  learned  in 
order to improve future strategies and programmes.  

The evaluation is based on 8 evaluation questions, relating to the five DAC evaluation criteria 
plus coherence and EU value added. This study was supervised by a Reference Group composed 
by  representatives  of  the  4  EU  Member  States  directly  concerned,  by  the  Association  of  the 

186

 
 
 
 
 
 
 
 
 
 
 
OCTs and by members of the EU services. 

Summary of findings and conclusions  

The EU has been consistent with the EU policy objectives of promoting the economic and social 
development in the OCTs and bringing these islands economically closer to the EU. Regarding 
trade, the EU preference regime did not sufficiently counterbalance the economic diversification 
constraints. The involvement of the OCTs in  regional programmes was minimised by isolation 
from regional networking, high participation costs and sometimes by the language barrier. Few 
results have been achieved on climate change and the disaster crisis management. EU support 
management  and  delivery  mechanisms  have  been  less  conductive  to  the  achievements  of  the 
cooperation's objectives.  

Conclusions 

9.  EU support to the Overseas Countries and Territories (OCTs) has served its purpose well by 
being consistent with the EU's policy objectives of promoting economic & social development in 
the OCTs and bringing the OCTs economically closer to the EU. EU support was also consistent 
with the priorities set out by the OCT governments and the needs of their populations.  

10.  The  EU  trade  preference  regime  did  not  sufficiently  counterbalance  the  OCTs’  economic 
diversification  constraints.  At  the  same  time,  the  EU-ACP  EPA  negotiations  represent  a  threat 
for the OCTs by levelling trade advantages to ACP-countries (free of duty and quota) more in 
line with the OCTs preferential terms.  

11.  In  spite  of  the  efforts  towards  regional  integration  and  cooperation  supported  under  the 
EDFs,  the  involvement  of  the  OCTs  in  regional  programmes  was  minimised  due  to  isolation 
from  regional  networking,  high  participation-costs  and  in  the  South  Pacific,  by  the  language 
barrier.  

12.  Despite recognising the importance of the environment, the reality of climate change and the 
importance of crisis management, few results from the EU-OCT cooperation were found in this 
field so far. However, prospects for future impact from interventions that have recently started 
hold promise.  

13.  The  delivery  mechanisms  of  EU  support  and  its  management  have  been  less  conducive  to 
the overall achievement of the cooperation objectives. Significant delays have occurred in some 
cases and there have been cases where the Commission’s approach was inconsistent.  

Recommendations 

1.  In future, the framework should be revised in order to respond better to new emerging OCT 
priorities,  such  as  energy  and  climate  change  challenges  and  opportu¬nities,  and  to  new  EU 
objectives  such  as  sustainable  marine  management.  It  should  also  facilitate  OCT  access  to  all 
relevant EU agencies, programmes and thematic budget lines. 

2.  The  EU  should  move  from  the  present  trade  regimes  to  a  framework  of  positive 
discri¬mination. The trade incentive framework should be integrated in a GBS policy dialogue 
platform on trade and economic diversification. The incentive framework should counterbalance 
any  further  erosion  of  trade  preference  by  estab¬lishing  an  EU-funded  scheme  designed  to 
promote internationalisation and innovation among OCT SMEs. More resources should also be 
committed to technical assistance. 

3.  The EU should increasingly consider the OCTs as EU stakeholders in regional programmes 
than as ACP countries. In order to build a closer relationship with OCTs’ neighbours, the OCTs 

187

 
 
 
and  neighbouring  ACP  countries  should  reach  an  agreement,  closely  coordinated  with  EU 
bilateral National Indicative Programmes (NIPs) on economic co-operation and trade focused on 
services, environment and scientific and cultural co-operation.  

4.  OCTs  should  be  supported  in  developing  "centres  of  excellence",  capable  of  creating  and 
disseminating  knowledge,  in  particular  with  respect  to  climate  change,  renewable  energy  and 
environment  issues  that  apply  to  their  regions.  Centres  of  excellence  could  be  shared  between 
different OCTs, thus promoting their co-operation. 

5.  A  mix  of  co-operation-instruments  (project  versus  budget  support)  should  reflect  the 
individual  OCT  circumstances  and  needs.  When  project  modalities  are  applied,  it  should  be 
ensured that they are part of a sector approach and that the territorial capacities are adapted to the 
chosen modality. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1294_docs_en.htm  

Thematic evaluation of European Commission support to conflict prevention and 
peace building (CPPB) 

ABB activities: 

19 11 Policy strategy and coordination for ‘External 
relations’ policy area 

Timing:   October 2011 

Background, scope and focus  

This evaluation assesses European Union support to CPPB, as defined by the 2001 Commission 
Communication on Conflict Prevention (COM(2001)21) and subsequent documents. It covers all 
support provided during the period 2001-2010 and in all relevant regions where EU support is 
implemented. It concerns a total of €7.7bn of funds contracted by the European Union over the 
period covered. 

188

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The EU has significantly increased its focus on CPPB (reflected in a total of €7.7bn of funding 
over the period) and strengthened its policy framework. However, the conceptual orientations at 
policy level have not always been appropriated at operational level. The support often remained 
in a developmental perspective. The EU needs to clarify its role and focus on crisis management 
and causes of conflict. The EU should consolidate and further develop its support for CPPB and 
ensure  that  its  financial  support  is  sufficiently  complemented  and  leveraged  by  non-financial 
support.   

Main conclusions: 

1.Since 2001, the EU has operated a substantial shift towards support to CPPB, but there is a gap 
between the EU’s policy commitments and the actual implementation. 

2.Conceptual  orientations  at  policy  level  have  generally  not  been  appropriated  at  operational 
level and were not always univocal and shared at strategic level. 

3.The support of the EU often remained in a developmental perspective rather than operating a 
shift towards a genuine CPPB perspective with a clear and prioritised strategy. 

4.The conflict (-prone) or post-conflict context challenged the relevance of the alignment of EU 
support on the strategies and policies of national authorities: 

–  National development strategies did not always exist; 

– 

It was not always clear with whom to align; 

–  The national authorities were sometimes a major party to the conflict; 

5.The EU provided value added that differentiated it from most other actors: 

–  Stronger neutrality; 

–  Reliability (continued presence); 

–  Critical mass in terms of financial support; 

–  Ability to draw on a wide array of instruments; 

–  Long-term thematic experience in fields potentially impacting on CPPB; 

–  Credibility in terms of promoting democracy, peace and human rights. 

Main recommendations: 

The EU and, where appropriate the High Representative, should: 

1.  Clarify  the  role  the  EU  wants  to  play  in  conflict  (prone)  and  post  conflict  contexts  by 
focusing both on crisis management efforts and on tackling the root causes of conflict through a 
clear  and  prioritised  strategy  geared  to  CPPB  rather  than  to  “classic”  development.  The 
institutional  changes  brought  by  the  Lisbon  Treaty  aim  at  providing  positive  developments  in 
that regard. 

2.  Strengthen  its  position  as  a  key  player  in  terms  of  CPPB  by  consolidating  and  further 
developing its support for CPPB (operational recommendations are provided in the report. 

3.  Ensure that its financial support is sufficiently complemented and leveraged by non-financial 

189

 
 
 
 
 
support at global, regional and country levels. 

4.  Ensure  clarification  and  common  understanding  of  concepts  among  its  staff,  including  at 
operational level. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1291_docs_en.htm  

Thematic evaluation of the European Commission support to Justice and Security 
System Reform (JSSR) 

ABB activities: 

19 11 Policy strategy and coordination for ‘External 
relations’ policy area 

Timing:   November 2011 

Background, scope and focus  

This  evaluation  assesses  the  European  Union  support  to  JSSR  during  the  period  2001-2009, 
considering only those aspects of justice reform which most directly fall under Security System 
Reform:  strengthening  of  criminal  justice  systems  and  the  legal  institutions  involved  in  the 
oversight of security institutions. 

The evaluation provides an overall independent assessment of the EU's past support to JSSR and 
identifies  key  lessons  with  a  view  to  improving  current  and  future  Commission  strategies  and 
programmes. 

Summary of findings and conclusions  

The EU substantially increased its funding, amounting to €1bn, developed a concept to support 
JSSR  and  engaged  in  a  wide  range  of  assistance  activities.  However,  the  EU  support  has 
generally  not  been  geared  towards  enhancing  the  delivery  of  services  in  a  manner  that  was 
responsive to people's needs. The weaknesses of the EU Policy Framework and internal capacity 

190

 
 
 
 
 
 
 
 
 
 
 
 
 
 
limitations hampered the support's impact and prevented the EU from bringing all its potential 
added value. 

Main conclusions: 

On policy commitment and framework 

Since  2001,  the  EU  has  substantially  increased  its  engagement  in  JSSR  globally  through 
funding,  development  of  its  concept  and  utilisation  of  a  wide  range  of  financial  and  non-
financial instruments. However, the EU did not generally adopt a strategic, political approach to 
supporting  JSSR  in  partner  countries  due  to  weaknesses  in  the  EU  Policy  Framework  and  the 
limitations of its instruments. 

On strategy and implementation 

–  The  EU  focused  on  building  institutional  capacity  within  state  security  and  justice  bodies 
rather than on addressing the constraints to service delivery from the perspective of the intended 
beneficiaries. 

–  The  EU  placed  strong  emphasis  on  national  ownership  at  both  policy  and  programming 
levels, but its focus on state institutions and its inflexible programming procedures significantly 
constrained its ability to tailor JSSR assistance effectively  

On results and impact 

EU  assistance  contributed  to  enhanced  institutional  capacities  within  state  security  and  justice 
bodies to deliver public services, but the overall impact on people's security and access to justice 
has  been  difficult  to  measure  and  was  limited  by  its  overall  strategy  for  supporting  JSSR 
processes. 

On institutional capacity and partnerships 

–  The EU institutional set up, human resource capacity and programming tools and guidance 
were not commensurate with its policy commitment and its level of funding for JSSR 

–  No  clear  division  of  labour  or  enhanced  coordination  and  complementarity  in  JSSR 
assistance programmes between the EU and its partners, either within the EU or outside  

Main recommendations: 

On policy commitment and framework 

The Commission should work with the EEAS to strengthen the EU JSSR Policy Framework and 
develop complementary operational guidelines  

On strategy and implementation 

–  The EU should adopt a more strategic, political approach to JSSR assistance  

–  Greater emphasis on securing the local knowledge and inputs at the design stage of the JSSR 
programmes  

–  A  longer  timeframe  for  the  JSSR  programming,  combined  with  more  flexible  and  rapid 
approval procedures  

–  Service delivery outcomes should drive the EU overall approach to JSSR  

On results and impact 

The  EU  should  incorporate  the  mechanisms  required  to  measure  results  and,  in  particular,  the 
impact of its assistance on the lives of people 

191

 
 
 
 
On institutional capacity and partnerships 

–  The Commission with the EEAS should work to develop a stronger pool of EU JSSR experts 
by improving training and ensuring appropriate support 

–  The EU should exploit the comparative advantage offered by its supranational character and 
more effectively harness Member State capabilities  

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1295_docs_en.htm  

Thematic evaluation of the EC support in employment and social inclusion sectors in 
partner countries 1999-2009 

ABB activities: 

19 11 Policy strategy and coordination for ‘External 
relations’ policy area 

Timing:   September 2011 

Background, scope and focus  

The  objective  of  this  evaluation  is  to  assess  to  what  extend  the  EU  sustained  relevant  and 
efficient  development  support  at  various  levels  of  Employment  and  Social  Inclusion  (ESI)  to 
eradicate poverty and achieve social cohesion in partner countries during the period 1999-2008. 

The evaluation is based on 7 evaluation questions, relating to the five DAC evaluation criteria 
(relevance,  effectiveness,  efficiency,  sustainability  and  impact)  plus  coherence  and  EU  value 
added. At the occasion of this study, it has been notably produced a global worldwide inventory 
of EU/ESI interventions, a reconstruction of the ex-ante intervention logic of EU support and a 
large  survey  to  EU  Delegations  and  national  counterparts’  staff  on  the  assessment  of  EU/ESI 
programmes and their features 

Summary of findings and conclusions  

Employment  and  Social  Inclusion  (ESI)  is  a  significant  priority  in  development  policy  and  in 
actual programmes. Nevertheless, there has been a weak mainstreaming of ESI in EU supported 

192

 
 
 
 
 
 
 
 
 
 
 
programmes and a poor sector or thematic approach including a lack of ESI-related indicators. 
Moreover,  informal  economy  was  often  overlooked.  EU  support  to  trade  development, 
macroeconomic  stability  and  growth  contributed  to  creating  new  employment  opportunities. 
However,  few  of  these  interventions  focused  explicitly  on  employment  creation.  Technical  and 
Vocational  Education  and  Training  support  policies  contributed  to  improving  "employability". 
The EU has focused positively on the social cohesion of vulnerable groups and on labour market 
governance. 

Main conclusions 

1.  Regarding policy and response strategy, ESI represents a significant priority in development 
policy  and  actual  programmes.  Nevertheless,  there  was  weak  mainstreaming  of  ESI  in  EU 
programmes during the period covered by the evaluation and a poor sector or thematic approach 
including  a  lack  of  ESI-related  indicators.  It  is  also  evident  that  informal  economy  is  often 
overlooked. 

2.  EU  support  of  trade  development,  macro-economic  stability  and  growth  contributed  to 
creating new sustainable employment opportunities. However, few of these interventions focus 
explicitly on employment creation. 

3.  TVET  (Technical  and  Vocational  Education  and  Training)  and  other  active  labour  market 
support  policies  contributed  to  improving  employability  which  is  a  major  bottleneck  in 
employment. Their importance has been growing but they have rarely been combined with other 
interventions to a fully integrated and comprehensive thematic approach. 

4.  The  EU  has  focused  positively  on  the  social  cohesion  of  vulnerable  groups  and  on  labour 
market governance such as social dialogue or labour disputes but with only limited weight. 

Main Recommendations 

1.  On policy and strategy, it is recommended to mainstream ESI focus in economic and social 
development  support,  to  foresee  a  specific  focus  on  informal  economy  and  to  further  enhance 
country  ownership  of  the  ESI  agendas.  Regarding  programming  and  implementation,  it  is 
recommended to promote government-led sector approaches (SWAPs) comprising an increasing 
and improvement of the use of budget support and an establishment of targets and indicators as 
well as an improvement of Monitoring and Evaluation systems. 

2.  On  increasing  employment  opportunities,  it  is  suggested  to  enhance  employment 
consequences  of  economic  reform  and  private  sector  support  programmes  through  the 
introduction  of  specific  measures  and  monitoring  and  evaluation  tools,  mainly  via  the 
introduction of adequate employment indicators. 

3.  On  improving  employability  via  TVET  support,  it  is  recommended  to  monitor  more 
systematically  the  effect  of  support  to  TVET  on  employment  and  income  and  support  the 
establishment  of  TVET  partnerships  with  the  involvement  of  employers  and  workers 
organizations in the design and implementation of TVET policies and programmes. 

4.  Social  inclusion  and  protection  should  be  systematically  linked  to  support  for  growth  and 
employment-related  interventions.  Marginalized  and  vulnerable  groups  should  be  taken  into 
account in economic growth programmes and active labour market policies. 

Availability of the report on Europa  

193

 
 
 
 
 
 
http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1296_docs_en.htm  

Study on legal instrument and lessons learned from the evaluations managed by the 
Joint Evaluation Unit  

Evaluation related study

ABB activities: 

19  11  Policy  strategy  and  coordination  for  ‘External 
relations’ policy area 

Timing:  

 July 2011 

Background, scope and focus  

The  current  European  multi-annual  budget  runs  from  2007  to  2013  and  includes  a  set  of  10 
external action budget lines or ‘legal instruments’ as they are known. Out of the current 10 legal 
instruments, 6 were analysed in this study: DCI, ENPI, INSC, IfS, EIDHR and ICI. Proposals for 
Europe’s  next  multi-annual  financial  framework  (MFF)  from  2014  are  discussed  during  2011-
2012. 

The  overall  purpose  of  this  study  was  to  see  what  lessons  could  be  learnt  from  the  past 
experience of EU external action programmes that might be useful in the design and formulation 
of the EU’s next multi-annual budget.  

The study was conducted in two steps: 

–  Ex-ante analysis: what did the 6 legal instruments set out to achieve? 

–  Ex-post analysis: what can be learned from 57 evaluation reports finalised between 2006 and 
2010? 

Summary of findings and conclusions  

The ex-ante analysis carried out under the first step of the study (analysis of the 6 regulations) 
led to the following key conclusions:  

–  Having  a  single  legal  base  for  each  instrument  helps  improve  the  clarity  of  what  is 
intended in the legal instrument as well as the governance and accountability provisions. 

–  The objective to have policy-driven instruments remains relevant and useful as the policies 
help clarify and explain intentions in greater detail. 

–  Formulating clear logic chains in regulations is key. This can be encouraged by avoiding 

194

 
 
 
 
 
 
 
 
 
 
unstructured  lists  of  activities  and  unclear  links  between  activities,  outputs  and  their  intended 
impacts.  The  relationships  between  each  step  in  the  chain  should  be  clear  and  logical  so  as  to 
avoid gaps in the logic and ‘orphan’ outputs or impacts. 

–  A division of tasks between the regulation and the policy document can be a way to go into 
greater  detail  without  overloading  the  regulation.  The  policies  can  be  used  to  provide  a  more 
detailed  framework  than  is  found  in  the  regulations  so  as  to  ensure  a  common-basis  of 
understanding  and  guide  the  choices  made  in  country/regional  strategy  papers  as  well  as  help 
ensure alignment with the regulation. The instrument should not just be seen as a random cluster 
of activities that the EC wishes to conduct, but rather as an agreed intervention framework with a 
thought through logic. 

–  Prioritisation  to  guide  execution.  Several  regulations  contain  long  lists  of  identified 
activities. Prioritising these in relation to the objectives of the instrument can be useful to clarify 
the intervention logic. 

–  The  complementarity  between  thematic  and  geographic  instruments  needs  to  be  clearer. 
Complementarity that was ‘required’ by the regulations did not always emerge in practice. This 
points to a need to (a) specify what type of complementarity is intended and to (b) clarify further 
the  division  of  tasks  and  relationship  between  thematic  and  geographic  instruments  and 
programmes. 

The  ex-post  analysis  carried  out  under  the  second  step  of  the  study  (analysis  of  evaluation 
reports) led to the following key conclusions:  

–  Lack of clarity in intervention logics leaves room for interpretation. Linking the evaluation 
results  to  the  interventions  logics  is  made  more  difficult  when  the  latter  are  not  clear  in  their 
logic chain and overloaded at the result level. This obscures both learning and accountability on 
whether  the  results  as  found  in  evaluations  contributed  to  the  impact  and  global  impact 
objectives as stated in the legal instruments. 

–  A fair degree of convergence was evident between what was expected and what took place, 
but not all results are positive. The sizeable proportion of mixed results (i.e. which did not make 
a difference) indicates room for improvement in both the programming and implementation, but 
also in the formulation of the guidance the instruments themselves provide for these actions. 

–  Missing links. There was a lack of evidence on linkages and interrelations between the six 
legal  instruments  being  made  in  practice  in  their  use.  None  of  the  evaluations  addressed  this 
explicitly. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1292_docs_en.htm  

195

 
 
 
 
 
 
20 – Trade  

Evaluation of the Commission's Market Access Database (MADB) 

ABB activities: 

20 02 Trade policy 

Timing:  

 October 2011  

Background, scope and focus  

The  evaluation  of  the  MADB  followed  up  on  a  previous  evaluation  undertaken  in  2006,  and 
assessed  the  current  state  of  the  MADB  –  including  improvements  implemented  since  the 
previous study - and the context in which it operates. The work ran from February to October 
2011. 

The purpose of the evaluation was to assess the contribution of the MADB towards achieving its 
main  goals  (effectiveness);  to  consider  the  financial  and  human  resources  committed  to  its 
operation  (efficiency);  and  to  provide  recommendations  for  future  orientations  and  eventual 
improvements  of  the  service.  DG  TRADE  had  already  received  feedback  that  the  MADB  in 
general is a useful tool for stakeholders, but wanted to learn: about how to further improve the 
service;  about  the  utility  and  reliability  of  each  of  the  core  sections  of  the  MADB;  about 
possibilities to make it more user-friendly; and about ways to reach a broader audience as the 
level of use was considered to be rather low.  

Summary of findings and conclusions  

Overall,  the  MADB  is  well  targeted  to  address  the  needs  of  its  four  main  target  groups.  It 
supplies reliable, comprehensive and up-to-date information which cannot be found in the same 
quality and depth through other, freely available services.  

The level of usage of the MADB has remained relatively stable over the last years at around 
1,700  “daily  unique  visitors”  on  an  average  weekday  (excluding  users  from  within  the  EC).  
However,  although  the  MADB  ought  to  be  especially  beneficial  to  smaller  sized  enterprises  – 
which often lack the resources, networks and critical mass to procure trade-related information 
efficiently through other means – SMEs seem to be underrepresented among those visitors. 

Stakeholders  generally  hold  a  positive  view  over  the  overall  quality  and  usefulness  of  the 
MADB.  It  combines  different  pieces  of  information  into  one  integrated  system  –  making  it 
unique among comparable services. 

Overall,  the  evaluation  found  that:  the  performance  of  the  MADB  with  regard  to  its  first 
function  –  ie,  providing  basic  information  of  interest  to  EU  exporters  --  is  excellent;  the 

196

 
 
 
 
 
 
 
 
 
 
 
second function (ie, listing all the trade barriers affecting EU exports) is less well implemented; 
and the approach taken to fulfil the third function (ie, making available an interactive means of 
communication between business and the EU authorities) has not yet proved successful. 

User feedback regarding the user-friendliness of the MADB is positive: 71% of respondents 
find  the  MADB  as  a  whole  rather  or  very  convenient  to  use.  The  MADB  could  however  be 
made  more  convenient  to  use  by:  enabling  multi-country  or  multi-tariff  line  searches; 
providing  the  option  to  download  multiple  information  and  in  different  formats;  creating  a 
“printer friendly” alternative for documents and results; and, by providing more explanation for 
novice users. 

More  than  half  (57%)  of  MADB  users  report  also  using  other  online  databases  or  web 
services: mostly those that are free of charge. In comparison with the most popular alternative 
sites,  the  MADB  is  relatively  user-friendly,  more  up-to-date  and  more  comprehensive  as 
regards to tariffs and import formalities (the Applied Tariffs Database and Exporters' Guide to 
Import  Formalities  sections).  However,  certain  competitor  websites  have  more  complete  trade 
barriers  and  SPS  measures  databases,  and  some  of  them  offer  information  in  more  than  one 
language. 

The awareness and take-up of the MADB amongst target groups is still limited, especially 
amongst (smaller) exporting companies. The number of companies visiting the MADB is much 
lower than the overall number of European exporters. Information on the MADB does not reach 
them sufficiently: authorities and organisations on the ground do not seem to be very active in 
advertising the MABD. Only 251 web pages link to the MADB, and – within the framework of 
a  “mystery  shopping”  exercise  –  only  two  out  of  14  trade  promotion  agencies  or  business 
representations  recommended  using  it.  Furthermore,  finding  the  MADB  through  a  Google 
search is very difficult if one does not know its full name, using common trade-related search 
terms. 

The  arrangements  for  day-to-day  management  of  the  MADB  are  adequate.  However,  the 
swiftness of uploading data to ATD and EGIF sections and especially feedback to complainants 
could  be  improved.  The  resources  spent  on  the  MADB  are  more  than  justified  by  the 
monetised  benefits  to  users.  Total  costs  for  the  service  amount  to  ca.  €1.7  million, 
corresponding  to  a  per-unit  cost  of  €2.8  per  single  query  –  this  is  significantly  lower  than  the 
price for a single query on the most comparable alternative service. 

Availability of the report on Europa  

http://madb.europa.eu/mkaccdb2/evaluation2011.htm 

197

 
 
 
 
 
 
21 – Development and Relations with African, Caribbean and Pacific 
(ACP) States  

Evaluation of the European Commission's co-operation with the Dominican Republic 

ABB activities: 

21 06 Geographical cooperation with African, Caribbean 
and Pacific (ACP) States 

Timing:   November 2011 

Background, scope and focus  

The  evaluation  covers  co-operation  strategies  and  implementation  of  all  activities  during  the 
period  2001-2009,  primarily  of  the  European  Development  Fund  (EDF8  &  EDF9,  and 
programming  of  EDF10).  During  that  period  EU  support  co-operation  areas  included  focal 
sectors (education / health, water & sanitation, governance, and macroeconomic support). 

Summary of findings and conclusions  

The EU aid contributed to reducing poverty in key sectors, but has not specifically targeted the 
socio-equity gap. The sustainability of some results remains weak, primarily, but not only, due to 
the  inadequate  Government's  financial  support  to  most  sectors  in  which  the  EC  has  operated 
(education,  natural  resources  management  and  the  DR-Haiti  border  zone  development). 
Therefore,  the  EU  should  continue  with  the  current  strategy  while  focusing  more  on  capacity 
development of the various local stakeholders and investing in robust monitoring and evaluation 
systems. 

Main conclusions 

1.  The EU supported the development priorities of the Government of the Dominican Republic 
and the needs of the people of the country, but did not specifically address the socio-equity gap 
that is critical in the country. 

2.  The EU’s final choice of focal sectors was appropriate and proved to be effective, given the 
needs of the country and the comparative advantages of the EU. 

3.  The EU and the Government of the Dominican Republic have not been able to put into place 
a  framework  for  mutual  accountability  founded  on  policy  dialogue,  results-based  performance 
and the commitment to sustainability. 

4.  The  EU  has  been  relatively  successful  in  transitioning  from  a  project-based  approach  to  a 
budget support-based approach. The decision to use Budget support was warranted from policy 

198

 
 
 
 
 
 
 
 
 
 
 
and  bilateral  relationship  perspectives  but  the  mutual  accountability  base  and  the  efficiency  of 
that mechanism needs improvement. 

5.  In  implementing  its  strategic  vision  concerning  the  role  that  Civil  Society  and  Non-State 
Actors  should  play  in  Dominican  Republic  Society,  the  EU  has  been  successful  in  helping  to 
generate policy frameworks concerning Civil Society and in supporting Civil Society capacity to 
participate in specific policy debates. 

6.  The EUD is not sufficiently equipped with resources and management systems (design and 
assessment,  risk  management,  monitoring,  capacity  development  expertise)  to  effectively 
support the strategic objectives of the EU in the DR. 

7.  Although capacity development is a central tenet of the EU cooperation policy generally and 
in  the  Dominican  Republic  specifically,  capacity  development  design  in  the  country  does  not 
focus on sustainable and evidence-based competencies. 

8.  The  EU  is  not  systematically  mainstreaming  governance  improvement  and  corruption 
elimination into its interventions, and following up with adequate monitoring systems. 

9.  The  EU’s  programming  and  its  subsequent  implementation  followed  EU  guidelines  on  the 
cross-cutting issues of environment and gender. 

Main Recommendations 

1.  Continue  with  poverty  reduction  as  the  key  programme  impact  objective  and  focus  on  the 
reduction  of  the  socio-economic  equity  gap  in  the  country,  ensuring  that  the  management 
emphasis is on verifiable results and clear milestones of policy reform. 

2.  Ensure  that  capacity  development  is  at  the  heart  of  every  EU-supported  intervention,  and 
manage  the  capacity  focus  in  keeping  with  the  EU’s  Backbone  Strategy,  while  focussing  on 
evidence-based competencies and on-going monitoring and evaluation. 

3.  Invest in monitoring and evaluation processes and systems to be able to 1) define baselines, 
2) follow progress on results, 3) enable faster adjustments to be made, 4) manage risks and 5) 
plan strategically based on a business eco-system approach. 

4.  Expand support to the education through a comprehensive multi-modality strategy to resolve 
key issues such as quality of education and management of resources. 

5.  Continue to use budget support as the main modality for supporting the Government of the 
Dominican  Republic  but  tighten  up  the  performance  assessment  framework.  Focus  the  policy 
dialogue and the performance framework on results and significantly reinforce policy dialogue 
mechanisms. 

6.  Formalize  a  Civil  Society  Organisation  Support  Strategy  based  on  a  focussed  approach  to 
the capacity required within civil society and ensure that all EU modalities and instruments as 
well as the efforts of Member States leverage each other. 

7.  Continue to increase the competitiveness of the Dominican Republic both internally and for 
trade, by selecting a small number business clusters for a comprehensive support programme for 
competitiveness capacity. 

8.  Support to the  mining sector should be continued so as to consolidate capacity and ensure 
sustainability. 

Availability of the report on Europa  

199

 
 
 
 
http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1293_docs_en.htm  

Evaluation of the European Commission's co-operation with the Republic of Malawi 

ABB activities: 

21  06  Geographical  cooperation  with  African,  Caribbean 
and Pacific (ACP) States 

Timing:   November 2011 

Background, scope and focus  

The  evaluation  covers  co-operation  strategies  and  implementation  including  a  mix  of  all 
activities  and  modalities  during  the  period  2003-2010.    During  that  period  EU  support  co-
operation  with  Malawi  included  focal  sectors  (regional  development,  agriculture  and  food 
security, regional interconnection – local and regional road transport), non-focal sectors (support 
to trade and investment, fight against HIV/AIDS and governance – civic education, justice and 
support to Non-State Actors) and cross-cutting issues (including environment, sustainability and 
gender issues). 

Summary of findings and conclusions  

Although  EU  cooperation  has  well  responded  to  the  country's  enduring  needs,  certain  aid 
modalities  could  be  improved  to  adapt  to  evolving  needs.  The  support  has  contributed  to 
improved  access  and  availability  to  food.  It  has  been  successful  in  reducing  malnutrition  in 
children  under  5.  EU  aid  has  positively  contributed  to  the  maintenance  of  the  existing  road 
network, yet little has been achieved in terms of wider regional connectivity.  Budget support has 
proved to be a powerful tool which has contributed to macro-economic stabilisation. 

Main conclusions 

Budget  support  is  a  powerful  tool  with  an  excellent  absorption  capacity  for  the  beneficiary 
country,  and  sector-wide  approaches  (SWAps)  are  the  preferred  modalities  of  both  the 
Government  of  Malawi  and  the  Commission.  Progress  towards  SWAps  has  nevertheless  been 
impeded in many sectors by institutional capacity issues. 

Certain  modalities  (such  as  programme  estimates  and  EDF  procurement  regulations)  are  very 

200

 
 
  
 
 
 
 
 
 
 
 
 
 
 
demanding in terms of administrative and financial management and have hence led to delays, 
major under-spending and lack of achievement of support objectives.  This is not the case for all 
modalities:  budget  support,  pooled  funding,  co-operation  and  administration  agreements  and 
division of labour in compliance with the Paris Declaration are recognised as effective. 

Capacity  constraints  and  institutional  weakness  continue  after  many  years  of  major  technical 
assistance  by  the  Commission  and  other  donors.  These  continuing  weaknesses  impact  upon 
broader  governance,  communication  and  dialogue,  leadership  and  sector  management  at  all 
levels. There are persistent significant needs for continued EU support to institutional capacity 
and governance. 

The focal sector support of the EU to continuing national needs has been correctly focused and 
appropriate in terms of national absorption capacities and the capacity of the EU Delegation. In 
most support sectors sustainability prospects are better at 'grass roots' levels where beneficiaries 
have  a  more  immediate  and  personal  appreciation  of  the  benefits  but  this  sustainability  is  not 
assured. Sustainability prospects at higher levels are poor. However, needs are still there and, for 
focal sectors, there are no exit strategies in sight, whilst affordability and sustainability of some 
support  interventions  are  in  doubt.  Further,  "emerging  risks"  of  population  growth  and  as-yet 
undetermined effects of climate change potentially add pressure on development prospects. 

Support  of  the  EU  for  regional  integration  and  connectivity  has  not  been  effective.  Regarding 
infrastructure,  Malawi  is  well  placed  to  take  advantage  of  a  paradigm  shift  in  regional 
connectivity  that,  if  correctly  leveraged  by  Malawi,  could  have  significant  impacts  on  quality 
and costs of transport services and on the competitiveness of Malawian products. 

Main Recommendations 

Maximising the impacts of sector-wide approaches and budget support 

Launch support for the sector-wide approach in the Road Transport (sub) sector under the Road 
Transport Sector Policy Support Programme (RTSPSP) subject to confirmation of credible and 
convincing evidence of resolution of issues regarding "arrears".   

Management of modalities 

Critically  examine  the  Commission's  and  EDF  procedures  with  a  view  to  making  them  more 
practical and more user-friendly. 

Regarding  visibility  actions,  the  report  questions  how  positive  perceptions  weigh  up  against 
negative perceptions.  The report suggests that the positive perceptions offset the negative ones 
but also highlights the need to assess what the value and/or purpose of visibility is. 

Aspects of governance 

During  the  course  of  implementation  of  the  proposed  EDF10  Democratic  Governance 
Programme  consider  additional  aspects  regarding  the  National  Initiative  for  Civic  Education, 
support to formal and informal justice sub-sectors and capacity building for non-state actors. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1297_docs_en.htm  

201

 
 
 
 
Evaluation conjointe des opérations d'aide budgétaire au Mali entre 2003 et 2009 

ABB activities: 

21  06  Geographical  cooperation  with  African,  Caribbean 
and Pacific (ACP) States 

Timing:   September 2011 

Background, scope and focus  

Les  objectifs  de  l'évaluation  étaient,  d'une  part,  de  tester  la  nouvelle  approche  d'évaluation  de 
l'aide budgétaire et, d'autre part, de porter une appréciation globale et indépendante des résultats 
de  l'aide  budgétaire  fournie  par  tous  les  partenaires  techniques  et  financiers  au  Mali  afin  d'en 
tirer les principaux enseignements pour améliorer l'aide budgétaire future. 

Le champ de l'évaluation a porté sur 32 opérations d'aide budgétaire, sur les 33 financées par dix 
partenaires  techniques  et  financiers  (PTF)*.  L'évaluation  a  porté  sur  les  opérations  d'appui 
budgétaire général (ABG) et d'appui budgétaire sectoriel (ABS) (Education, Santé, Gestion des 
finances publiques et Décentralisation) pour un montant total transféré de 966 M€. 

Summary of findings and conclusions  

L'aide  budgétaire  a  fortement  contribué  au  financement  de  la  politique  nationale  de 
développement.  La  gestion  des  finances  publiques  a  été  améliorée.  Grâce  à  la  gestion 
macroéconomique prudente des autorités maliennes et à l'aide budgétaire, les conditions de vie 
des populations maliennes se sont globalement améliorées pendant la période sous évaluation. 
De  nombreux  défis  restent  toutefois  à  relever,  notamment  en  matière  de  gestion  des  finances 
publiques  et  d'amélioration  du  climat  des  affaires  de  façon  à  accroître  le  développement 
économique  et  social.  L'aide  budgétaire  devrait  être  poursuivie  en  cherchant  à  améliorer  son 
efficacité.  

Principales conclusions générales 

–  Grâce à son importance pour le budget national (environ 10% des dépenses de l'Etat sur la 
période),  l’aide  budgétaire  a  permis  de  financer  la  stratégie  nationale  de  développement  pour 
laquelle  les  autorités  maliennes  avaient  un  fort  engagement.  Par  contre,  l'aide  budgétaire  n'a 
globalement pas eu d'effet sur l'évolution des politiques nationales. 

202

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
–  L'aide budgétaire a contribué aux avancées enregistrées en matière de gestion des finances 
publiques qui nécessite toutefois d'être encore améliorée. 

–  En raison de sa «masse critique», de sa flexibilité comme moyen de financement et de son 
suivi  de  la  stratégie  nationale,  l’AB  a  aidé  à  améliorer  l’efficacité  globale  de  l’affectation  des 
ressources et à faire face aux besoins stratégiques des secteurs prioritaires. 

–  Dans  le  contexte  d’une  gestion  macroéconomique  prudente,  la  contribution  financière  des 
AB  a  été  importante  pour  permettre  au  Mali  d’atteindre  un  taux  moyen  de  croissance 
économique réelle de 5% par an. 

Principales conclusions sectorielles 

–  L'appui  budgétaire  sectoriel  (ABS)  a  permis  d'accélérer  la  mise  en  œuvre  des  réformes  en 
matière de décentralisation et déconcentration. 

–  Sans le financement additionnel de l’ABS et de l’ABG, il aurait été impossible de financer 
les réalisations dans les secteurs de l'éducation et de la santé et donc les résultats positifs pour les 
populations maliennes. 

–  Toutefois, 
d’investissement privé sur lequel l'aide budgétaire n'a pas eu d'influence. 

la  croissance  économique  est 

le  frein  principal  à 

lié  au  faible 

taux 

Principales recommandations 

–  Renforcer l’impact des transferts financiers en agissant sur les indicateurs, la prévisibilité, la 
flexibilité et la date des décaissements des aides budgétaires. 

–  Redéfinir les objectifs du dialogue sur les politiques afin d'améliorer la contribution de l'aide 
budgétaire  aux  processus  nationaux  d’élaboration  et  de  suivi  des  politiques,  ainsi  qu’à  la 
formulation et au suivi du budget national. 

–  Eliminer les freins à la mise en œuvre de la stratégie nationale de développement et mettre 
en place un programme réaliste d’amélioration du climat d’investissements. 

Availability of the report on Europa  

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1290_docs_en.htm  

203

 
 
 
 
 
 
 
22 – Enlargement  

Strategic/Interim Evaluation of EU IPA Pre-Accession Assistance to Albania 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 February 2011  

Background, scope and focus  

It covers European Union assistance under Component 1 starting from the 2007 IPA programme 
and  the  intervention  logic  followed  in  the  Multi-Annual  Indicative  Planning  Documents 
(MIPDs) of 2007-2009, 2008-2010 and 2009-2010. The purpose is to provide an assessment of 
the  intervention  logic  of  IPA  assistance  to  Albania  including  the  extent  to  which  assistance 
should  be  programmed  and  implemented  through  a  sectoral  approach.  The  evaluators  also 
provide  a  judgement  on  the  performance  (actual  or  expected)  of  assistance,  particularly  as 
regards its relevance, efficiency, effectiveness, impact and sustainability. 

Summary of findings and conclusions  

Key findings 

–  The overall quality of the intervention logic is good and has improved over time although it 
is not sufficiently focused and measurable.  

–  A  common  weakness  in  the  intervention  logic  is  that  objectives  are  wide  and/or  not 
sufficiently precise. MIPDs do not include sufficient indicators to measure results and indicators 
in project fiches are not sufficiently SMART, particularly as regards timing. 

– 
IPA  financing  is  of  limited  capacity  to  address  Albania's  needs  or  to  achieve  the  priority 
objectives  in  MIPDs  as  they  are  currently  formulated.  Effectiveness  can  be  enhanced  by 
focusing available resources on selected priority sectors. 

–  There  has  been  relatively  limited  sequencing  within  prioritised  sectors  in  Albania, 
particularly with regards to infrastructure projects. 

–  A sector-based approach is a feasible and operational option for future IPA programming.  

–  The  involvement  of  line  institutions  in  project  implementation  can  be  improved.  This, 
together with high staff turnover, has led to some important implementation delays. 

–  The  Ministry  for  European  Integration  (MEI),  which  is  the  National  IPA  Co-ordinator 
(NIPAC),  prepares  to  set  up  sector  monitoring  sub-committees.  This  coincides  with  the 
finalisation  of  the  Integrated  Planning  System  (IPS)  monitoring  system.  On  the  one  hand,  this 

204

 
 
 
 
 
 
 
 
 
 
 
provides an opportunity for developing a unified sector monitoring system. On the other, there is 
a  risk  of  confusion  and  overload  in  line  institutions  and  of  duplication  of  procedures  and 
conflicting reporting time schedules.  

–  Contracting and disbursement has been low to date as compared to plan and there is a risk 
that  the  funds  may  not  be  fully  contracted  on  time.  The  lack  of  capacity  and  engagement  by 
beneficiary institutions is an important factor for this. 

–  Based on result-oriented monitoring (ROM) reports, expected impact and sustainability were 
judged  as  positive.  Nevertheless,  the  evaluators  reported  some  risks  related  to  insufficient 
institutional support and post-implementation arrangements, including financing. 

Key recommendations 

–  The number of priorities and objectives in the MIPDs should be reduced and better focused. 
Annual  programming  should  focus  consistently,  in  successive  years,  on  a  limited  number  of 
prioritised sectors. 

–  The quality of objectives of project fiches should be more focused and measurable as well 
as better linked to MIPD objectives. 

–  More  flexible  financial  allocations  per  priority  axis  and/or  annual  allocations  are 
recommended to improve prioritisation and sequencing of assistance. 

–  There is a need to formulate minimum quality standards for sector strategies to assess their 
use for a sector-based approach. 

–  The  Albanian  authorities  should  decide  which  sectors  would  be  appropriate  for  a  sector-
based  approach,  based  on  three  criteria:  (a)  relevance  to  EU  integration/accession;  (b)  the 
quality  of  strategies;  (c)  administrative  capacity  to  implement  and  monitor  strategies.  The 
evaluator is of the view that (a) should be the main criteria for the selection of sectors. 

–  The Albanian government should ensure that there is sufficient capacity in line institutions 
to manage the implementation of a sector-based approach for selected sectors. 

–  The  MEI  (NIPAC)  should  lead  the  development  of  a  single  monitoring  system  using 
common data inputs from line institutions and common reporting formats. 

–  There is a need to increase beneficiaries commitment in the activities financed with the aim 
to improve impact and sustainability of assistance. 

Availability of the report on Europa  

For the internal use of the Commission.  

205

 
 
 
 
 
 
Interim Evaluation of Cross-Border Programmes between Candidate/Potential 
Candidate Countries (Intra-Western Balkan Borders) under the Cross-Border 
Cooperation Component of IPA; Report II – Performance of assistance 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 02/06/2011 

Background, scope and focus  

This  evaluation  is  the  second  part  of  the  Interim  Evaluation  of  Cross-Border  Programmes 
between  Candidate/Potential  Candidate  Countries  (Intra-Western  Balkan  Borders)  under  the 
Cross-Border  Cooperation  (CBC)  Component  of  IPA;  and  focuses  on  the  CBC  programme 
implementation.  The  evaluation  covers  the  eight  Intra  Western  Balkan  IPA  CBC  programmes 
with an emphasis on the overall instrument level and, where relevant, at the programme or grant 
scheme  level.  The  main  objective  of  the  evaluation  is  to  identify  lessons  learned  and 
recommendations for the revision of the current CBC programmes. 

Summary of findings and conclusions  

Key findings 

–  Stakeholders  perceive  that  the  programme  has  contributed  to  improve  good  neighbourly 
relations. 

–  Projects are expected to deliver expected outputs; some projects may need an extension; the 
limited budget available for some programmes may pose challenges to their implementation.  

–  The long process of evaluating project applications has resulted in the need to change some 
projects and has generated frustration of the stakeholders involved.  

–  There are no unified contract templates which could be used by partners on either side of 
the border, nor formal procedures for secondary procurement.  

–  Monitoring  systems  are  not  in  place  yet  and  this  results  in  a  lack  of  data  collection  and 
weak risks mitigation mechanisms. 

–  Sustainability  is  uncertain  since  systems  and  institutions  which  could  secure  impact  after 
the end of projects have not been planned or designed yet.  

Key recommendations:  

–  Programme  allocations  between  the  two  sides  should  be  more  balanced  I  order  to  reduce 

206

 
 
 
 
 
 
 
 
 
 
 
leftover amounts.  

–  The evaluation and selection process should be shortened by merging steps and checks and 
by delegating more tasks and responsibilities to the JTSs.  

–  Rules  and  procedures  should  be  harmonised  and  streamlined,  and  the  capacity  of 
administrative structures strengthened. 

–  A monitoring system including common data collection and assessment of risks should be 
put in place.  

–  Adequate  advice  should  be  provided  to  future  applicants  to  avoid  the  recurrent  use  of 
extensions.  

–  The  capacity  of  stakeholders  should  be  improved  through  more  focused  training  and 
programme guidelines in order to promote sustainability.  

–  Action  should  be  taken  to  promote  applications  from  groups  which  have  not  been  well 
represented (i.e. municipalities and other public bodies). 

Availability of the report on Europa  

http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/20120304_2cbc_wb_2_
en.pdf  

Retrospective evaluation of EU pre-accession assistance to CBC programmes in 
Bulgaria and Romania 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 01/2011 

Background, scope and focus  

The objective of the evaluation is to evaluate the relevance, efficiency, effectiveness, impact and 
sustainability of PHARE CBC programmes. The scope includes 2004–2006 CBC programmes i) 
between Bulgaria and Romania and ii) between them and neighbouring IPA countries.  

Summary of findings and conclusions  

207

 
 
 
 
 
 
 
 
 
 
 
 
 
Key findings 

–  The  CBC  programmes  show  satisfactory 
logic  although  with  some 
weaknesses.  The  intervention  logic  of  programmes  is  sound,  with  in  general  SMART 
objectives.  However,  there  is  not  sufficient  coherence  between  indicators  at  different 
programming levels and between Joint Programming Documents and Project Fiches. 

intervention 

–  Relevance  of  CBC  programmes  has  been  satisfactory.  The  interventions  and  grant  sizes 
have  been  adequate  to  support  and  complement  the  Accession  Strategy.  Furthermore,  CBC 
provided the opportunity (the first for some countries) for stimulating cooperation among local 
stakeholders.   

–  Local and regional authorities have been consulted during the programming phase, though 
the level of involvement has not always been satisfactory, particularly in case of border region 
municipalities. 

–  Project  selection  has  been  based  on  clear,  coherent  and  homogenous  criteria,  though  the 
level of transparency could be improved.  

– 
In  terms  of  efficiency  and  effectiveness  CBC  has  only  been  partially  satisfactory.  In 
general,  the  administrative  procedures  and  organisational  structures  in  place  have  been 
adequate.  However,  in  certain  programmes  some  inefficiencies  have  undermined  the  efficient 
and  effective  delivery  of  interventions.  These  include:  insufficient  cooperation  between 
National Aid Co-ordinator (NAC) services; insufficient resources to manage a large amount of 
grants;  uncertainties  regarding  transition  from  PRAG1  procurement  to  national  procurement 
rules,  problems  with  delegation  of  responsibilities;  disproportionate  burden  in  terms  of 
documents to be submitted, a mostly paper-based selection process and an excessive focus on 
tight controls rather than a real attention towards programme results. 

–  Programme and project monitoring has only been partially adequate. Indicators defined at 
programme  level  have  not  in  all  cases  been  adequate  and  consistent  with  those  defined  at 
project level, and a formal monitoring has not been carried out at programme level, but only at 
project  level.  The  overall  monitoring  system  and  its  reports  have  been  focused  on  the 
implementation of the programme, with limited focus on results/impacts. Another weakness in 
the monitoring process has been the lack of a comprehensive web-based information system.  

–  The  main  impacts  have  been  the  improvement  of  institutional  capacity  and  improved 
cooperation  although  the  implementation  period  has  not  been  appropriate  in  some  cases. 
Assistance has contributed to improve management skills of beneficiaries; the development of 
positive  attitude  towards  cross-border  cooperation  and  the  creation  of  sustainable  networks 
between institutions and local authorities as well as between grants beneficiaries. In many of the 
programmes, 
investment 
component/small-scale infrastructure projects may have not been appropriate.  

implementation  period 

schemes  with  an 

for  grant 

the 

Availability of the report on Europa  

For internal use of the Commission.  

1 Practical Guide to Contract Procedures for EU External Actions 

208

 
 
 
 
 
 
                                                 
Thematic evaluation of EU's support to civil society in the Western Balkans and 
Turkey 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 26/06/2011 

Background, scope and focus  

The thematic evaluation of EU's support to civil society (CS) in the Western Balkans and Turkey 
(WBT) consists of two parts. It covers EU assistance to the WBT funded under the IPA 2007-
2009  national  programs  and  the  IPA  multi-beneficiary  programmes  under  Component  I.  This 
includes  the  multi-beneficiary  programmes  within  the  Civil  Society  Facility  (CSF):  Technical 
Assistance for Civil Society Organisations (TACSO), the People 2 People programme (P2P) and 
Partnership Actions. Part I of the evaluation provides an assessment of the intervention logic of 
EU assistance to support civil society in the WBT. 

Summary of findings and conclusions  

1.  Interventions  are  relevant  for  the  objectives  identified  in  the  Multi-Annual  Indicative 
Documents  (MIPDs).  However,  measurement  of  progress  is  challenged  by  the  absence  of 
SMART indicators and the wide definition of strategic objectives. 

2.  CS needs are formulated with reference to enlargement political objectives. However, these 
objectives may not always reflect WBT citizens’ socio-economic priorities such as employment, 
service provision or environmental concerns.  

3.  Involvement  of  CS  organisations  and  other  key  stakeholders  in  needs  assessment  and  the 
accuracy of this assessment could be enhanced throughout the programming cycle. 

4.  While programming is well prioritised, sequencing of projects is less regular and its practice 
depends on national sector priorities and project context. 

5.  Lengthy calls for proposals significantly delay the implementation of the programmes. 

6.  Key  terminology  such  as  “Civil  Society”,  “Civil  Society  Development”,  “Civil  Society 
Dialogue”, and “Civil Society Facility” is not sufficiently clear among stakeholders and does not 
correspond to common perceptions of CS organisations/NGOs in the WBT.  

Key recommendations 

209

 
 
 
 
 
 
 
 
 
 
 
 
 
–  Strategic  objectives  should  be  more  operational  and  supported  by  relevant  SMART 
indicators.  

–  The  EC  and beneficiaries  should  agree on  operational  definitions of  key  concepts,  such  as 
“Civil Society”, “Civil Society Development”, and “Civil Society Dialogue”.  

–  The  beneficiary  selection  procedure  should  be  reviewed  to  assess  the  adequacy  of  the 
coverage of different CS players, and to focus support on beneficiaries with good sustainability 
prospects. 

–  Participation  of  CS  and  other  stakeholders  in  programming  and  project  selection  could  be 
improved by adopting and implementing the EC “Minimum Standards”1 for consultation of CS 
on EU related matters. 

–  The  time-span  from  programming  to  implementation  should  be  reduced,  particularly  by 
speeding up the call for proposal process.  

– 

Impact could be improved by increasing the use of long-term grants. 

–  Sustainability  of  interventions  may  be  enhanced  through  increasing  partnerships  with 
stakeholders with a potential for income generation and local visibility (e.g. local authorities). 

Availability of the report on Europa  

For internal use of the Commission.  

Thematic Evaluation of IPA-Funded Information and Communication Activities 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 07/06/2011 

Background, scope and focus  

The  evaluation  covers  EU  assistance  to  the  Western  Balkans  and  Turkey  in  the  field  of 
information  and  communication  (IC)  activities  funded  under  the  IPA  Multi-Beneficiary 

1  Commission  of  the  European  Communities,  “Towards  a  reinforced  culture  of  consultation  and  dialogue  – 
General  principles  and  minimum  standards  for  consultation  of  interested  parties  by  the  Commission”, 
COM(2002) 704 final, 11 December 2002 

210

 
 
 
 
 
 
 
 
 
 
 
 
                                                 
Programme  during  2007-2011.  The  purpose  was  to  provide  an  assessment  of  the  intervention 
logic  of  IC  programmes  and  actions  running  within  the  EU's  communication  strategy  for  the 
candidate and potential candidate countries.  The evaluators were asked to provide a judgement 
on the performance of assistance, particularly as regards its relevance, efficiency, effectiveness, 
impact and sustainability.  

Summary of findings and conclusions  

1.  While overall objectives of IPA-funded IC are clear and coherent at Headquarters (HQs) and 
EU Delegation (EUD) level, specific objectives are too broad and do not meet SMART criteria. 

2.  IC  activities  are  particularly  relevant  in  a  context  where  beneficiaries  place  relatively  low 
importance on communicating enlargement.  

3.  While  complementarity  of  activities  and  coordination  with  key  stakeholders  (i.e. 
beneficiaries' administration, Member  States  and international organisations) are good, there is 
scope for increased coordination and planning at the strategic level. 

4.  IPA  IC  funds  have  generally  been  deployed  in  an  efficient  manner.  However,  efficiency 
could  be  increased  by  downscaling  activities  with  limited  effectiveness  (e.g.  publications)  and 
greater concentration of resources on fewer activities. 

5.  At an operational level, human resources within EUDs have been deployed in an effective 
manner and EUD staff show a high level of commitment towards their tasks. However, with the 
exception of the former  Yugoslav Republic of Macedonia and Kosovo1, all EUDs lack human 
resources to effectively and efficiently implement IC activities. 

6.  IC  activities  are  clearly  visible  and  effective  for  audiences  with  an  interest  in  enlargement 
but less so for “non-informed” audiences and the broader public. 

7.  EUDs are effective in providing independent and objective information on enlargement but 
less effective in timely addressing public misconceptions and concerns.  

8.  Overall, IPA IC activities are likely to have a soft and indirect impact on public knowledge 
through involvement of multipliers and media coverage of activities.  

9.  A significant positive impact on public opinion is unlikely, since the communication efforts 
are too marginal and there exist other important factors influencing public opinion. 

10.  Sustainability of activities without continued EU funding is likely to be low as funding from 
beneficiaries or other sources is uncertain. 

Key recommendations 

–  While  general  objectives  may  remain  broad,  specific  objectives  should  be  more  focussed, 
realistic and time- bound. 

–  DG  ELARG  should  provide  more  operational  support  to  EUDs  in  the  planning  and 

1 This designation is without prejudice to positions on status and is in line with UNSCR 1244/1999 and 

the ICJ Opinion on the Kosovo Declaration of Indepndence. 

211

 
 
 
                                                 
implementation of IC activities. 

–  Consultation  with  EUDs  on  the  definitions  of  activities  to  be  undertaken  at  central  level 
should be strengthened. 

–  Current staff allocation and organisation of communication within EUDs should be reviewed 
and where possible additional human resources should be allocated to IC management.  

–  EUDs should elaborate guidelines for the effective management of monitoring/measurement 
activities undertaken by contractors. Evaluations of previous activities should be used as one of 
the baselines for selection and implementation of future activities. 

–  Conditions  for  sustainability  of  funded  activities  should  be  better  addressed  during  the 
selection process at EUDs. 

Availability of the report on Europa  

For internal use of the Commission.  

Thematic evaluation of EU pre-accession multi-beneficiary assistance to Western 
Balkans and Turkey in the fields of environment and disaster risk reduction 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 31/05/2011 

Background, scope and focus  

The  evaluation  covers  EU  assistance  to  the  Western  Balkans  and  Turkey  in  the  fields  of 
environment  and  disaster  risk  reduction  (DRR)  under  the  multi-beneficiary  (MB)  programmes 
financed under CARDS during 2002-2006 and under IPA - Component I during 2007-2009. The 
purpose  is  to  provide  judgement  on  the  performance  (actual  or  expected)  of  assistance, 
particularly  as  regards  its  relevance,  efficiency,  effectiveness,  impact  and  sustainability.  The 
evaluators  were  asked  also  to  provide  relevant  lessons  learned  and  recommendations  for  the 
programming  of  future  multi-beneficiary  pre-accession  assistance  in  the  areas  covered  by  the 
evaluation. 

212

 
 
 
 
 
 
 
 
 
 
    
 
 
 
Summary of findings and conclusions  

1.  The  intervention  logic  under  IPA  is  clearer  than  the  one  under  CARDS.  However,  the 
objectives are too broad and not sufficiently linked across the different programming stages. 

2.  Objectives at project level are relevant and financial assistance follows the priorities outlined 
in key strategic documents.  

3.  While  programming  is  lead  by  the  European  Commission  (EC)  in  close  consultation  with 
“institutional” stakeholders (namely EC, national authorities, International Financial Institutions 
(IFIs), there is limited involvement of NGOs. 

4.  There are no formally defined criteria to support project selection and the selection process 
is perceived by some stakeholders as insufficiently transparent. 

5.  Programmes  and  projects  are  not  systematically  supported  by  assessments  in  terms  of 
funding needs, added value of the MB/regional approach and response to national needs.   

6.  A  key  contribution  of  the  MB  programmes  has  been  the  promotion  of  beneficiaries' 
institutional change towards working together on environment and disaster-risk reduction (DRR) 
issues by encouraging networking and sharing experiences.  

7.  Existing monitoring is not effective as it does not provide a comprehensive sector overview. 
This limits its potential use for improving programme design. 

8.  Impact  and  sustainability  of  programmes  is  hampered  by  uneven  commitment  and 
ownership of the beneficiary countries which dedicate limited human and financial resources and 
often do not plan follow-up actions beyond EU assistance. 

9.  Multi-annual  long-term  projects  are  effective  in  creating  sustainable  links  between 
participants beyond the projects implementation.  

10.  Dissemination and visibility activities are not fully effective in promoting sustainability.  

Key recommendations 

–  Programming should be more focused and prioritized. 

–  Capacity and participation of national institutions to should be reinforced.  

–  Consultation  with  stakeholders,  including  NGOs,  should  be  strengthened  and  more 
transparent. 

Impact  and  sustainability  of  assistance  could  be  enhanced  by  favouring  multi-annual 

– 
projects. 

–  Post-project  management  procedures  should  be  defined  in  the  programme  and  project 
design, including a strategy to use project outputs and disseminate results.  

–  Monitoring should be reinforced at programme level.  

Availability of the report on Europa  

For internal use of the Commission.  

213

 
 
 
 
 
 
Strategic/Interim Evaluation of EU IPA Pre-Accession Assistance to Kosovo1  

ABB activities: 

22 AWBL-03 Enlargement pre-accession negotiations 

Timing:  

 December 2010/Dissemination February 2011  

Background, scope and focus  

This  evaluation  was  completed  in  December  2010.  It  covers  European  Union  assistance  under 
Component 1 starting from the 2007 IPA programme and the intervention logic followed in the 
MIPDs of 2007-2009, 2008-2010 and 2009-2011. The purpose is to provide an assessment of the 
intervention logic of IPA assistance to Kosovo including the extent to which assistance should 
be  programmed  and  implemented  through  a  sectoral  approach.  The  evaluators  also  provide  a 
judgement  on  the  performance  (actual  or  expected)  of  assistance,  particularly  as  regards  its 
relevance, efficiency, effectiveness, impact and sustainability. The results of this evaluation have 
been  taken  into  consideration  within  the  Mid-term  Meta-evaluation  of  IPA  which  will  be 
completed shortly. 

Summary of findings and conclusions  

Key findings 

–  The  overall  quality  of  the intervention  logic  is  good although not sufficiently  targeted  and 
measurable. 

–  Sequencing  for  infrastructure  projects  could  be  improved,  taking  into  account  the 
beneficiary's capacity and the availability of co-financing.  

–  There are indications that a sector-based approach to IPA programming in Kosovo could be 
implemented in the near future.  

–  National  strategies  do  not  cover  all  acquis-related  sectors  and  MIPD  priorities,  there  are 
significant gaps in coverage. The quality of existing strategies varies significantly.  

–  The  alignment  of  IPA  and  other  donors'  assistance  is  key,  as  there  is  significant  donor 
activity in Kosovo. 

–  Adequate administrative and organisational structures have been established, but their role is 
not well-defined and the involvement of officials in project preparation is limited.  

1 This designation is without prejudice to positions on status and is in line with UNSCR 1244/1999 and 

the ICJ Opinion on the Kosovo Declaration of Indepndence. 

214

 
 
 
 
 
 
 
 
 
 
 
                                                 
–  An examination of contracting data indicates that the procurement stage of implementation 
is well managed by the European Commission Liaison Office (ECLO) to Kosovo.  

Increased contracting needs in PFs are often underestimated and this offsets efficiency gains 

– 
deriving from larger projects. 

–  Based  on  ROM  monitoring  reports,  expected  impact  and  sustainability  were  judged  as 
positive.  Nevertheless,  the  evaluators  reported  some  risks  related  to  insufficient  institutional 
support and post-implementation arrangements, including financing. 

Key recommendations 

–  The number of priorities and objectives in the MIPDs should be reduced and better focused. 

–  The  quality  of  objectives  of  NPs/PFs  should  be  more  targeted  and  measurable  and  better 
linked to MIPD objectives. 

Infrastructure  projects  should  be  sequenced  according  to  beneficiary  capacity  and 

– 
availability of co-financing.  

–  National  sector  strategies  should  cover  all  acquis-related  sectors  and  MIPD  priorities.  The 
quality of most existing strategies needs to be improved by developing realistic action plans and 
adding indicators which can be used for performance monitoring. 

–  The  Ministry  of  European  Integration  should  take  a  leadership  role  in  the  sector  strategic 
planning process.  

–  Action to limit the number of contracts per project fiche is recommended. 

–  Mechanisms  to  improve  prospects  for  impact  and  sustainability  of  assistance  should  be 
introduced, including formal post-implementation arrangements.  

Availability of the report on Europa  

For internal use of the Commission.  

Strategic/Interim evaluation of IPA pre-accession assistance to Serbia 

ABB activities: 

 22 AWBL-03 Enlargement pre-accession negotiations 

Timing:  

 January 2011 

Background, scope and focus  

215

 
 
 
 
 
 
 
 
 
 
 
 
This  evaluation  was  part  of  a  project  completed  in  January  2011.  It  covers  European  Union 
assistance under Component 1 starting from the 2007 IPA programme and the intervention logic 
followed in the MIPDs of 2007-2009, 2008-2010 and 2009-2010. The purpose was to provide an 
assessment  of  the  intervention  logic  of  IPA  assistance  to  Serbia  including  the  extent  to  which 
assistance should be programmed and implemented through a sectoral approach. The evaluators 
were also asked to provide a judgement on the performance (actual or expected) of assistance, 
particularly  as  regards  its  relevance,  efficiency,  effectiveness,  impact  and  sustainability.  The 
evaluation was also intended to provide input for the Mid-term Meta-evaluation of IPA. 

Summary of findings and conclusions  

Key findings 

–  The harmonisation between IPA objectives and the beneficiary’s policies is good although 
some adjustment would be suitable upon the introduction of the sector-based approach. 

–  Donor-coordination works well; donors are participating in the inter-sector working groups 
and the process is supported by an online database. 

In 

the 

– 
comprehensiveness as compared to previous programmes.  

last  National  Porgrammes  (NPs)  projects  have  gained 

in  coherence  and 

–  There  is  a  well-functioning  project-selection  mechanism  in  place,  with  proper  sequencing 
and prioritisation.  

–  Administrative  and  monitoring  structures  are  in  place,  though  adequate  human  capacity 
remains a major problem.  

–  The performance in terms of contracting and disbursement is appropriate.  

–  Measuring  progress  is  difficult,  as  the  main  programming  documents  do  not  include 
SMART indicators/objectives.  

–  More than seventy strategies were mapped by the consultants. Overall, their quality in terms 
of  identification  of  needs  (including  financial)  is  limited  and  they  are  not  systematically 
monitored.  

Key recommendations 

–  Reduction  of  the  number  of  strategic  objectives  in  the  MIPDs  would  be  advisable,  with 
SMARTer indicators to measure progress.  

–  Solving  the  staffing  problems  within  the  Ministry  of  Finance  would  allow  the  gradual 
withdrawal of programming technical assistance. 

–  Monitoring structures need to be strengthened, especially by introducing formal mechanisms 
to make better use of the monitoring results.  

–  Continuous efforts should be placed in development of Serbian leadership in programming 
and implementation of assistance.  

Availability of the report on Europa  

216

 
 
 
 
 
For internal use of the Commission.  

Evaluation on stakeholder participation in programming and implementation of pre-
accession assistance to Turkey 

ABB activities: 

22 AWBL-03 Enlargement pre-accession negotiations 

Timing:  

January 2011  

Background, scope and focus  

The overall objective of the evaluation was to assess stakeholder involvement in programming 
and implementation of pre-accession assistance to Turkey, as well as providing lessons learned 
and  recommendations  on  how 
in  future  programming  and 
implementation under the Instrument for Pre-Accession (IPA) in the Turkish context. 

involve  stakeholders 

to 

Summary of findings and conclusions  

Key findings 

–  The  principle  of  stakeholder  participation  is  generally  accepted  and  there  is  a  growing 
interest and preparedness to ensure wider stakeholder participation in IPA I programming.   

–  Stakeholders  involved  in  the  programming  and  implementation  of  IPA  I  included  a  large 
category of institutions (development partners, EU organisations, governmental institutions and 
civil  society  organisations  (CSOs).  The  involvement  of  CSOs  (especially  trade  unions  and 
NGOs) is relatively limited due to lack of capacity and the high level of fragmentation. Interest 
representation is highly centralised. 

–  A  long  term  perspective,  preferably  supported  by  sectoral  strategies,  is  a  key  factor  for 
effective  stakeholder  participation.  Some  sectors  (political  criteria,  civil  society,  and 
environment)  show  larger  stakeholder  participation,  compared  to  others  (justice,  freedom  and 
security, taxation).  

–  MIPD  preparation  and  revision  offer  limited  scope  for  wide  stakeholder  participation. 
Inviting stakeholders to provide written comments on the draft MIPD appears  ineffective. The 
introduction of sector strategies in 2011-2013 MIPD may facilitate stakeholder participation.    

–  The  preparation  process  of  the  Annual  National  Programme  is  the  main  bottleneck  for 
stakeholder involvement. External stakeholder participation in Project Identification Sheet (PIS) 
and Project Fiche (PF) formulation remains limited. 

217

 
 
 
 
 
 
 
 
 
 
 
–  The  phase  with  the  widest  involvement  of  stakeholders  is  implementation.  Governmental 
bodies represent the largest share of direct and indirect beneficiaries. The Senior Project Officer 
(SPO) has a key role in terms of stakeholder participation as they are responsible for deciding on 
the members of the Steering Committee.  

–  Of the different contract forms, grant schemes were assessed as offering the largest scope for 
stakeholder participation. 

–  Monitoring  activities  are  performed  by  SPOs  and  IPA  management  partners  with  very 
limited involvement of external stakeholders. External stakeholders are usually not involved in 
monitoring activities, which contrasts with other IPA components.  

Key recommendations 

–  Beneficiaries need to develop a strategic approach to stakeholder involvement. This should 
include:  (a)  an  assessment  of  relevant  actors  and  stakeholders  and  (b)  a  roadmap  for  the 
development and implementation of a capacity building strategy. 

–  The Secretary General for EU Affairs (EUSG) should exert stronger leadership in providing 
operational  support  and  guidance  to  beneficiaries,  monitoring  progress  and  enforcing  the 
strategic approach. 

–  The EU Delegation should improve consultation with development partners, MS embassies 
and  CSOs  during  the  preparation  of  sector  strategies,  MIPDs  and  project  pipeline.  This  may 
include  increasing  the  use  of  workshops  and  sectoral  working  groups  to  complement  written 
consultation.  

–  EUSG should adjust the guidelines and formats of the PIS and PF templates and assessment 
forms. This may include making stakeholder participation a selection criterion for PIS and PF. 

–  SPOs  should  extend  the  participation  of  stakeholders  in  consultative  and  decision-making 
bodies, such as Steering Committees.  

–  EUSG  should  consider  systematically  inviting  representatives  of  CSOs  and  other  external 
stakeholders  as  observers  to  the  Joint  Monitoring  Committee/Transition  Assistance  and 
Institution-Building  Component  (JMC/TAIB)  and  Sectoral  Monitoring  Sub-Committees 
(SMSCs),  in  order  to  improve  visibility  and  accountability  to  the  wider  public  and  foster 
capacity building. 

Availability of the report on Europa  

For internal use of the Commission.  

218

 
 
 
 
 
 
Review of Twinning in Turkey 

Evaluation related study

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 23/05/2011  

Background, scope and focus  

The  evaluation  was  intended  to  provide  a  retrospective  assessment  of  Twinning  in  Turkey  in 
terms  of  its  relevance,  efficiency,  effectiveness,  impact  and  sustainability.    Secondly,  it  was 
expected  to  provide  recommendations  for  programming  future  Twinning  assistance  in  Turkey.  
The period covered by the evaluation was 2002 to 2009. 

Summary of findings and conclusions  

Key findings 

–  The vast majority of Twinning projects in Turkey achieved their objectives.  (Based on the 
positive average score in the Interim Evaluation Reports (IER), and the confirmation of the IER 
scores in the case studies.)  

–  The most common reasons for Twinning projects to fail to deliver included: lack of political 
commitment,  problems  with  procurement,  inadequate  expert  mobilization  from  the  Member 
State institution, and an outdated needs assessment. 

–  The  presence  of  synergies  between  Twinning  projects  and  other  EU  and/or  donor  funded 
projects was a key factor supporting the success of Twinning projects. 

–  Results  and  impacts  could  be  achieved  more  cost-effectively  by  reducing  the  delay  in  the 
delivery of the activities. 

Institutional and/or legal alignment to the Acquis is in most cases still pending, and will be 

– 
dependent on the political adoption of the relevant legislative or organisational reforms. 

–  Twinning  has  had  significantly  more  impact  in  the  Environment  sector  than  in  the 
Agriculture and Fisheries sector. Its role in the Justice and Home Affairs sector has been mixed 
and requires more sustained efforts. 

–  Despite  mostly  good  intentions,  ongoing  cooperation  between  the  beneficiary  and  the 
member state institution after the end of the Twinning project is not widespread in practice. 

Key Recommendations  

– 

In line with the Twinning manual, political commitment should be assessed at the start of the 

219

 
 
 
 
 
 
 
 
 
 
project.    The  criteria  used  to  support  a  new  Twinning  project  should  be  more  objectively 
verifiable. 

–  Twinning should be used in more sectors.  Training on the Twinning instrument, including 
information  on  Twinning  projects  in  other  government  sectors  and  other  accession  countries, 
should be provided. 

–  Programming and contracting procedures should be shortened, so that the time between the 
needs assessment and the actual start of the Twinning project is reduced. 

–  The beneficiary should ensure that the staff members directly involved in the management of 
the Twinning project are fully available. The Secretary General for EU Affairs (EUSG) should 
monitor and interfere in case such availability is not guaranteed. 

–  The  selection  of  the  Member  State  institution  should  be  carefully  based  on  Resident 
Twinning  Adviser  (RTA)  availability  and  competences,  as  well  as  the  institutions’  capacity  to 
endow short term experts to the project.  

Availability of the report on Europa  

For internal use of the Commission.  

Evaluation of Twinning versus Technical Assistance 

ABB activities: 

22 02 Enlargement process and strategy 

Timing:  

 26/01/2011 

Background, scope and focus  

The  evaluation  covers  European  Union  assistance  delivered  through  twinning  (standard  and 
light) and technical assistance. The purpose is to assess the contribution of these instruments to 
the  overall  accession  process  in  the  Western  Balkans  and  Turkey.  The  evaluators  undertook  a 
comparative  analysis  of  the  relative  advantage  of  each  instrument  as  well  as  of  the  choice 
considerations.  The  report  presents  lessons  learned  and  guidance  on  the  programming  and 
planning  of  future  assistance.  In  this  context,  the  evaluation  team  developed  a  cost-benefit 
template to support decisions on when to use twinning and when technical assistance. 

Summary of findings and conclusions  

220

 
 
 
 
 
 
 
 
 
 
 
 
–  Twinning  is  suitable  for  support  of  public  bodies,  acquis  related  assignments  and  for 
organisations  with  sufficient  capacity  and  awareness  to  handle  twinning  projects.  For 
organisations  less  able  to  make  full  use  of  twinning,  technical  assistance  (TA)  is  a  suitable 
alternative.  

–  TA is nominally more expensive than twinning. The cost-difference is, however, offset by 
flexibility/efficiency gains. The time necessary to start operations or to adjust the work plan are 
more cumbersome under twinning. 

– 
Important assumptions as regards twinning are proven valid: its possibility to foster relations 
between institutions in the EU and in the (potential) candidates, as well as changes in working 
culture. In contrast, TA appears to be more flexible and easier to steer. 

–  The  sampled  projects  scored  above  average  on  impact,  effectiveness,  efficiency  and 
sustainability. There was no significant difference in this regard between projects using twinning 
or TA. 

–  Some  of  the  factors  for  a  successful  project  are:  the  'chemistry'  between  providers  and 
beneficiaries; a shared understanding of the issue; clear division of responsibilities; no imposed 
solutions; detailed knowledge of the issue and its background by the provider.  

–  The selection of providers is equally important as the selection of an instrument. Providers 
from institutions detached from mainstream public administration would normally have less of a 
benefit in terms of developing relations by the beneficiary with ‘sister institutions’ in the EU. 

Key recommendations 

–  Twinning  is  suitable  for  acquis  related  assignments  in  acquis  related  beneficiary 
organisations that have sufficient capacity to absorb twinning. For other assignments, TA  may 
be more effective. 

–  Developing lasting relations with a member state is important. This, however, should not be 
a reason for selecting twinning if the beneficiary is not able to make full use of it.  

– 
Insisting on more detailed twinning proposals would facilitate the selection of the provider. 
A  presentation  from  TA  providers,  as  part  of  the  tender  procedure,  would  also  facilitate 
selection. 

–  The  lead  time  for  twinning  projects  could  be  shortened.  The  twinning  manual  should  be 
more explicit on the roles and responsibilities of beneficiary and provider. 

–  The  preparatory  workshop  for  providers  should  be  longer  (i.e.  more  than  the  current  1.5 
days). It needs to focus on communication skills, flexibility and pro-active approach during the 
assignment. 

Availability of the report on Europa 

http://ec.europa.eu/enlargement/pdf/financial_assistance/institution_building/2012/20120525_twi
nning_vs_ta_final_report.pdf  

221

 
 
 
 
 
 
23 – Humanitarian Aid 

Evaluation and Strategy Orientation of DG ECHO-Funded Health Sector Activities in 
Burmese Refugee Camps in Thailand (2004-2009) 

ABB activities: 

23 02 Humanitarian aid including aid to uprooted people, 
food aid and disaster preparedness 

Timing:  

 February 2011 

Background, scope and focus  

The camps for Burmese refugees in Thailand are the result of the ongoing conflict between the 
central Burmese regime and some ethnic minorities. The Directorate-General for Humanitarian 
Aid and Civil Protection of the European Commission has been supporting food assistance to the 
refugees  in  Thailand  since  1995,  and  since  2004  it  has  funded  the  provision  of  basic  health 
assistance  in  six  of  the  nine  camps  along  the  border.  The  scope  of  the  evaluation  covered  the 
actions funded through the humanitarian policy of the EU in the health sector between 2004 and 
2009.  The  first  objective  of  the  evaluation  is  to  provide  a  retrospective  assessment  of  the 
activities in the health sector and the second one is to provide a prospective strategic assessment 
with  a  view  to  identifying  practical  options  and  recommendations  in  the  context  of  the  EU 
current strategy.  

Summary of findings and conclusions  

The  health  conditions  in  all  the  refugee  camps  over  the  period  were  satisfactory,  compared  to 
standard reference indicators and even more so to indicators for Burma/Myanmar. This situation 
logically reflects the effectiveness of health structures, which have been organized to the best of 
the  partners’  abilities  with  adequate  funding  over  a  protracted  period,  despite  a  number  of 
constraints.  All  major  health  risks  have  been  addressed,  resulting  in  low  mortality  rates  and 
appropriate  outbreak  and  disease  control.  Nevertheless,  in  the  absence  of  involvement  of  the 
Thai  authorities,  the  currently  controlled  situation  is  entirely  based  on  external  assistance, 
including  public  health  and  medical  support,  and  these  figures  would  undoubtedly  increase 
rapidly  again  to  unacceptable  levels  without  such  assistance.  As  a  consequence  of  the  lack  of 
integration  perspectives,  the  continued  EU  assistance  to  all  health  subsectors  is  similarly  still 
appropriate, although it should be increasingly focused on a public health approach rather than a 
purely medical one. 

Regarding  future  strategy  orientation  about  possible  transfers  of  Health  services  to  the  Thai 
Health  System,  as 
to 
NGO/international  community  provision  of  health  care  would  be  the  provision  of  this  care  by 
the  Thai  authorities.  However  it  is  highly  unlikely  that  this  will  happen.  Therefore,  under  the 

long  as  refugees  reside 

the  only  alternative 

the  camps, 

in 

222

 
 
 
 
 
 
 
 
 
 
present conditions, there does not seem to be any alternative for the financing of health care for 
the  refugees  other  than  by  the  international  community.  This  includes  the  various  health 
activities inside the camps and the payment for cases referred to the Thai hospitals. 

Availability of the report on Europa  

http://ec.europa.eu/echo/files/evaluation/2011/Burma_Final_Report.pdf  

Evaluation of DG ECHO's Actions in the Water Sanitation/Public Health Sector in 
Zimbabwe 

ABB activities: 

23 02 Humanitarian aid including aid to uprooted people, 
food aid and disaster preparedness 

Timing:  

 June 2011 

Background, scope and focus  

In 2008, cholera outbreaks in Zimbabwe infected local communities and the water and sanitation 
networks. At that time the Government was unable to respond to the epidemic in any meaningful 
way.  In  order  to  address  some  of  the  underlying  factors  contributing  to  water  borne  disease 
outbreaks,  Water,  Sanitation  and  Hygiene  (WASH)  partners  established  an  Emergency 
Rehabilitation and Risk Reduction programme. Since 2008 and until 2011 the European Union 
humanitarian  services  have  mainly  provided  support  in  the  areas  of  Water  and  Sanitation  and 
health  (under  the  acronym  WERU,  standing  for  Water,  sanitation  and  health  Emergency 
Response  Unit)  and  food  security.  The  scope  of  the  evaluation  covered  the  implementation  of 
EU-funded humanitarian actions between 2008 and 2010 (3 years). Furthermore, it focused on 
the water and sanitation components of the overall action, with the links to public health. 

Summary of findings and conclusions  

The response to the cholera emergency in Zimbabwe was not a carefully planned programme 
which  was  established  in  advance.  The  WERU  approach  that  emerged  from  the  emergency 
response  of  several  separate  agencies  and  actors  in  the  sector  was  interactive  in  nature  and 

223

 
 
 
 
 
 
 
 
 
 
 
 
 
 
organic  in  its  growth  and  development.  The  evaluation  team  had  difficulties  to  assemble 
verifiable  indicators  of  past  performance  for  comparison  against  present  targets.  Moreover, 
there was insufficient documentation available specifically explaining the formulation of the 
WERU  approach  and  detailing  its  main  features  and  implementation  modality.  Despite  the 
lack of formalized documentation it is clear that the collective understanding of the WERU 
approach  was  a  voluntary  agreement  by  the  four  organizations  who  had  signed  partnership 
agreements with UNICEF.  

It is considered that the response to the cholera was well developed and entirely relevant to 
the dramatic evolution of the cholera epidemic in the country. The overall impression is that 
the  work  done  by  WERU  partners  during  and  shortly  after  the  peak  of  the  cholera  crisis  in 
2008 and 2009 was a  major contributing factor to the prevention and control of the disease 
and  was  greatly  appreciated  by  the  Government  of  Zimbabwe.  The  EU  role  in  this  rapidly 
developing situation was pivotal as the Harare ECHO office was able to respond to funding 
requests and dispense those funds with alacrity and flexibility. Nevertheless, it is recognised 
that  efficiency  was  better  achieved  during  the  emergency  phase  of  the  WERU  programme 
between 2008 and 2010 than during the less stressed post-emergency phase during and after 
2010.  In  fact,  the  coordination  mechanisms  operated  during  the  latter  stages  of  WERU 
activity are not considered to be ideal in that they are entirely voluntary and do not have any 
executive powers to enforce technical or health sector standards, which are essential for the 
sustainability  of  the  results.  Finally,  even  if  WERU  was  originally  an  emergency  response 
unit, it shifted its focus slightly towards developing new capacity in emergency preparedness. 
This  is  itself  a  quasi-transition  activity  and  a  good  starting  point  for  developing  a  more 
coherent transition strategy. 

Availability of the report on Europa  

http://ec.europa.eu/echo/files/evaluation/2011/Zimbabwe_Final_Report.pdf  

Evaluation of the DG ECHO action in Uganda 

ABB activities: 

23 02 Humanitarian aid including aid to uprooted people, 
food aid and disaster preparedness 

Timing:   May 2011 

Background, scope and focus  

The  reduction  in  humanitarian  aid  to  the  north-central  region  of  Uganda,  which  has  been  the 
focus  of  the  humanitarian  community  for  more  than  10  years,  is  a  result  of  the  perceived 
reduction of humanitarian needs in the country.  

224

 
 
 
 
 
 
 
 
 
 
 
The  scope  of  the  evaluation  covers  the  implementation  of  the  EU  funded  humanitarian  action 
between  2006  and  2010,  and  focuses  on:  food  assistance,  protection  of  Internally  Displaced 
Persons  (IDPs),  water,  sanitation  and  hygiene  promotion  (WASH),  health/nutrition,  and 
especially  on  Linking  Relief  Rehabilitation  and  Development  keeping  in  mind  the  plan  of 
phasing out by December 2011. 

Summary of findings and conclusions  

In general terms, the EU funded activities, mainly multi-sectoral programmes, were relevant and 
appropriate to needs. Projects were implemented within agreed budgets; sometimes more direct 
beneficiaries were registered than foreseen previously (for instance 89,529 people could benefit 
from the Oxfam project in WASH sector compared to 77,357 previously foreseen). 

Regarding  the  WASH  sector,  Uganda  was  a  pilot  country  for  the  introduction  of  the  cluster 
approach,  which  has  permitted  (in  particular  the  Food  Security  and  Agricultural  Livelihoods) 
important achievements, for example coverage improved for thematic areas of child protection. 
However  not  all  sectors  worked  well  under  clusters,  local  actors  were  largely  excluded, 
ownership remaining low.  

In  addition,  the  EU  WASH  interventions  focused  on  rehabilitation  of  existing  infrastructure 
rather than construction of new water points.  

Regarding  sanitation  and  hygiene  promotion,  there  has  been  an  increase  in  access  to  water  at 
places  of  return,  and  general  reduction  of  distance  to  water  points;  there  was  an  increase  in 
latrine  coverage;  however  sanitation  structure  was  not  adapted  to  the  persons  with  physical 
disabilities.  The  hygiene  promotion  was  not  included  in  the  budget,  resulting  in  lack  of 
sensitising the community about the safe water chain and household sanitation. 

Health interventions were highly relevant and well aligned with the Health Sector Strategic Plan 
Framework. However the capacity building was not fully efficient.  

Nutrition interventions regarding capacity building of district staff to be able to manage feeding 
centres and of health service staff brought variable results. 

Regarding humanitarian protection, the EU has maintained a consistent support in this area, with 
a particularly well focused programme reducing number of camps.  

LRRD  is  constrained  by  lack  of  appropriate  instruments.  However  DG  ECHO  livelihood 
interventions discussed earlier with DG Development permitted a coherent approach. They were 
considered complementart with a view to longer term development focus. Sectors not considered 
as  focal  in  Country  Strategy  Papers  remains  of  concern  for  example  WASH,  health  and 
education. 

As a conclusion, overall the relevance  and efffectiveness of EU funded interventions has been 
good with an appropriate focus on multi-sectoral programmes, which was highly adapted to the 
needs.  

The EU has significantly contributed to health facility rehabilitation and health service delivery. 

Availability of the report on Europa  

225

 
 
 
 
 
http://ec.europa.eu/echo/files/evaluation/2011/Uganda_Evaluation_report.pdf  

Real-time evaluation of humanitarian action supported by DG ECHO in Haiti 2009-
2011 

ABB activities: 

23 02 Humanitarian aid including aid to uprooted people, 
food aid and disaster preparedness 

Timing:  

 April 2011  

Background, scope and focus  

This evaluation covers the EU response, delivered through its department for Humanitarian Aid 
and Civil protection (DG ECHO), to the earthquake of 12 January 2010 and the cholera crisis, 
the work of the DIPECHO programme and the Global Plan (implemented in 2009 for a period of 
18 months). The objectives of the evaluation were: 

–  To analyse the appropriateness  and effectiveness of DG ECHO’s action in response to the 
Haitian crisis with a special focus on the earthquake response;  

–  To assess the current situation of DG ECHO's action in Haiti, in order to identify relevant 
opportunities, challenges and threats; 

–  To provide recommendations within a larger comprehensive strategic framework. 

–  To  assess  the  effectiveness  of  DG  ECHO's  coordination  with  the  different  stakeholders  of 
the crisis (Member States, UN agencies, other donors, partners, etc) at different levels; 

–  From  a  LRRD  perspective,  to  analyse  the  coherence  and  complementarity  of  EU  funded 
actions with present and future interventions of other EC services. 

Summary of findings and conclusions  

In  general  the  programmes  supported  and  the  strategies  adopted  by  DG  ECHO  have  been 
relevant, as well as the presence of the DIPECHO programme. DG ECHO's advocacy for DRR 
is found to be appropriate. DG ECHO’s presence in Haiti and its network of partners capable of 
carrying out operations in different humanitarian sectors contributed to the effectiveness of the 
European response.  

The  decision  to  fund  a  broad  range  of  partners  was  essential  in  Haiti.  This  variety  made  it 
possible to ensure that aid was delivered on a large scale. Some difficulties occurred in LRRD 
(linking relief rehabilitation and development) despite the shared will to advance in this sense, 

226

 
 
 
 
 
 
 
 
 
 
 
due to the time frames for projects (limited to 18 months) and the fact that funding mechanisms 
for  the  transition  have  yet  to  be  developed.  Cross  cutting  issues  as  gender,  protection  and 
environment  were  only  marginally  taken  into  account  and  rarely  transformed  into  programme 
content, except when it was linked to questions of DRR.  

DG ECHO’s 3-pillared strategy for 2011 which involves three main areas (withdrawal from the 
earthquake  response,  withdrawal  from  the  Global  Plan  and  maintenance  of  a  strong  DRR 
component) responds to the challenges posed by the context of the crisis. ECHO should pursue 
the withdrawal during 2011-2012.  Furthermore the efforts in LRRD, including consideration of 
the  links  between  humanitarian  action  and  poverty  reduction  should  be  pursued.  The  Haitian 
disaster  management  and  resilience  building  mechanisms  should  be  strengthened  at  all  levels, 
placing  the  risk  reduction  need  at  the  heart  of  development  policies  in  Haiti.  Funds  should  be 
allocated  progressively  to  deal  with  the  evolution  of  the  contexts.  The  efforts  to  improve 
working methods in urban contexts should be supported. ECHO should continue to advocate for 
better coordination in the field, both at the strategic and operational levels and both at the sector 
and multi-sector levels.  

Availability of the report on Europa  

http://ec.europa.eu/echo/files/evaluation/2011/Groupe-URD_evaluation_Haiti_en.pdf 

Commission evaluation of the Civil Protection Mechanism and the CP Financial 
Instrument 2007-2009 

ABB activities: 

23 03 Civil protection financial instrument 

Timing:  

 November 2011 (COM(2011)696) 

Background, scope and focus  

The evaluation was undertaken in compliance with article 14 of the Civil Protection Mechanism 
Decision, which requires the Commission to evaluate the application of the Decision by the end 
of  2010,  and  transmit  the  conclusions  of  that  evaluation  to  the  European  Parliament  and  the 
Council.  Moreover,  Article  15(2)(b)  CPFI  requires  the  Commission  to  evaluate,  on  an  interim 
basis, the results obtained and the qualitative and quantitative aspects of the implementation of 
the Civil Protection Financial Instrument (CPFI). 

The results of the evaluation provide the Commission and other stakeholders with key findings 
on  and  lessons  to  be  drawn  from  the  experience  gained  in  the  implementation  of  the  above-

227

 
 
 
 
 
 
 
 
 
 
 
 
mentioned  actions  in  the  field  of  civil  protection.  These  will  help  the  Commission  in  the 
continued  effective  implementation  of  the  Mechanism  and  the  Financial  Instrument,  and  also 
provided important input to the preparation of the revision of Civil Protection legislation by the 
Commission in December 2011 (COM(2011)934. 

Summary of findings and conclusions  

The  conclusions  of  the  external  evaluators  are  overall  positive  in  that  they  recognise  and 
emphasise the usefulness and relevance of EU Civil Protection activities during the period under 
review. In particular, the Monitoring and Information Centre has been widely acknowledged as 
providing useful services that are relevant to Participating States when civil protection assistance 
interventions are deployed within or outside the EU. 

Regarding the ad hoc nature of the current EU disaster response, there appears to be a need to 
shift  to  a  system  where  advance  planning  allows  core  assets  to  be  available  for  immediate 
deployment.  The  planning  of  EU  civil  protection  operations  would  be  improved  through 
developing  reference  scenarios,  mapping  Member  States'  assets  and  developing  contingency 
plans, establishing a pool of Member States' assets pre-committed on a voluntary basis to the EU 
operations,  streamlining  and  reinforcing  provisions  on  transport  support,  as  well  as  other 
measures outlined in the October 2010 Communication on disaster response (COM(2010)600. 

The training courses have proved to be a valuable asset in terms of preparing national experts for 
civil  protection  assistance  interventions,  thus  improving  the  overall  response  capability  of  the 
Mechanism.  Nonetheless,  the  evaluation  also  showed  that  the  further  evolution  of  the  training 
arrangements is limited due to a lack of an overall policy framework. Similar conclusions have 
been drawn in respect of the exercises programme, which has received overall support but has 
experienced a lack of a general policy framework. To better serve their ultimate purpose, i.e. the 
improvement of operations, the establishment of an integrated training and exercises policy will 
need to be considered. 

The transport assistance provisions now seem to be firmly embedded in the Mechanism, and a 
highly significant use of the pooling and co-financing arrangements has been noticed in the last 
two  years,  even  though  calls  for  streamlining  the  procedures  have  been  voiced  and  are  duly 
taken  note  of.  Beyond  mere  simplification  of  rules  and  administrative  procedures,  important 
considerations  arising  from  the  evaluation  lead  to  a  need  to  investigate  different  levels  of  co-
financing depending on the urgency or priority of delivery for certain relief resources, as well as 
improving the access to transport assets/options. 

The modular approach was met with great interest and success among Participating States, and 
should be further developed, including through specialised exercises and developing 

Innovative arrangements  seeking to enhance the  availability of key relief assets tested through 
pilot projects and preparatory actions proved to be viable and should be built upon. It is to be 
noted, however, that by their very nature pilot projects and preparatory actions are time-limited 
and cannot substitute a more permanent policy and regulatory framework. 

The European cooperation and coordination in the field of civil protection has seen substantial 
progress  but  there  is  still  also  unused  potential.  One  important  area  which  has  received 
increasing awareness and acknowledgment is the policy need in the field of disaster prevention 
and  disaster  management.  An  enhanced  EU  prevention  policy  framework  would  be  able  to 
address  the  various  prevention  aspects  in  different  EU  policy  fields  (environment,  security, 
health and regional policies) and facilitate further cooperation among Participating States. 

228

 
 
 
 
Availability of the report on Europa  

Hyperlink to the evaluation report on Europa 

http://ec.europa.eu/echo/evaluation/thematic_en.htm#cp  

229

 
 
 
 
 
 
26 – Commission's administration 

Evaluation of Internal Communication and Staff Engagement Strategy of the 
European Commission  

ABB activities: 

26 01 Administrative expenditure of ‘Commission’s 
administration’ policy area 

Timing:  

 31/3/2011 

Background, scope and focus  

The evaluation of Internal Communication and Staff Engagement is a follow-up to the previous 
evaluation report on internal communication of April 2007. It specifically reviewed the progress 
made  on  internal  communication  and  staff  engagement  based  on  the  objectives  and  actions 
included  in  the  "Communication  on  the  Internal  Communication  and  Staff  Engagement", 
assessing  to  what  extent  the  objectives  have  been  met,  identifying  the  gaps  and  recommend 
measures to enhance internal communication and staff engagement in the Commission.  

The  scope  of  the  evaluation  covers  internal  communication  and  staff  engagement  at  both 
corporate and local (DG/directorate/unit) level.  

Summary of findings and conclusions  

The overall conclusion is that, while the infrastructure for internal communication is in place, it 
is not obvious with the current approach to explicitly show the impact of internal communication 
and  staff  engagement  on  the  organisational  performance.  More  synergies  between  the  internal 
communication  function  and  other  organisational  functions  need  to  be  created;  the  active 
participation  of  middle  and  senior  managers  and  interactive  communication  needs  to  be 
enhanced. 

Main results of the evaluation 

–  The fact that a strategy for Internal Communication and Staff Engagement was adopted by 
the Commission in 2007 has given these areas an official standing and assisted in bringing them 
to the attention of senior and middle management 

–  The  engagement  score  at  the  Commission  increased  between  2006  and  2008,  and  it  is 
comparable to other equivalent organizations  

–  The Commission’s approach to internal communication, a combination of a centralised and 
decentralised approach, is still deemed appropriate 

230

 
 
 
 
 
 
 
 
 
 
 
Internal communication experience and expertise in the Commission needs to be enhanced 

– 
in order to raise the profile of internal communication within the organisation 

–  A lack of support and buy-in from senior management for internal communication and staff 
engagement issues seems to exist, mainly due to a limited understanding of their full potential 
for driving employee performance 

–  Links between DG HR and DG COMM have improved over the last few years through joint 
initiatives but there is still scope for further collaboration  

–  A  continuing  commitment  to  supporting  tools  and  activities  aimed  at  facilitating  internal 
communication at the Commission and DG levels  exists, but they are  mainly used for passing 
information  rather  than  actively  engaging  staff;  the  recent  increased  promotion  and  use  of 
participatory methods is rated positively  

–  Numerous monitoring and evaluation tools are in place at Commission level and in the DGs. 
However,  a  more  integrated  and  systematic  approach  to  collating  data  and  reporting  on  it 
periodically is likely to result in a better ongoing view of the internal communication and staff 
engagement situation across the Commission 

–  A  little  progress  with  developing  and  implementing  indicators  on  internal  communication 
and staff engagement performance for managers has been achieved 

The evaluation report proposes a set of recommendations, among which:  

Identification  of  a  high  level  owner  for  the  strategy  and  establishment  of  an  Internal 

– 
Communication & Staff Engagement Committee made up of senior managers;  

– 
Internal  communication  &  staff  engagement  competencies  to  be  included  and  consistently 
used  in  the  recruitment  and  appraisal  of  middle  and  senior  managers;  senior  managers  to  set 
expectations for internal communication and staff engagement for their middle managers; 

–  Reinforcing the links with the HR community;  

–  Aligning internal communication with strategic objectives of the DG and Commission; 

–  Further  coordination  of  external  and  internal  communications  –  communicate  to  staff 
before/simultaneously to announcing any important decision externally; 

–  More two-way communication and opportunities for staff to provide feedback; and 

–  Systematic measuring, evaluation and reporting. 

Availability of the report on Europa  

For internal use of the Commission.  

231

 
 
 
 
 
Ex-ante evaluation of the contract "Processing of public procurement notices for publication 
in the Supplement to the Official Journal of the European Union, hereafter OJ S" 

Evaluation related study

ABB activities: 

26 02 Multimedia production 

Timing:  

 25/08/2011  

Background, scope and focus  

It's an ex-ante evaluation of the contract for the production and translation of public procurement 
notices  to  be  published  in  the  Official  Journal  S,  namely  on  TED  website  (Tenders  Electronic 
Daily).  

This  contract  covers  preparation,  inputting,  translation,  proofreading,  codification  (encoding, 
metadata  production)  and  administrative  monitoring  of  public  procurement  and  other  notices 
published  in  the  OJ  S.  This  corresponds  to  notices'  lifecycle  from  their  reception  by  the 
Publications  Office  until  their  readiness  for  publication,  or,  in  case  of  certain  alternative 
scenarios, until their deletion. 

Summary of findings and conclusions  

The findings of the evaluation showed that the current contract, restructured on the occasion of 
the last call for tender, is functioning well. Thus, there wouldn't be any benefits from the changes 
in  the  production  contract  which  would  justify  additional  development  costs  and  risks  to  the 
proper final implementation of TED-Monitor. Therefore, it appears reasonable to keep the basic 
structure of the contract as it is.  

The first years of the implementation of the contract proved that some modifications are needed 
and it was possible to identify weaknesses which should be addressed in the future contract. 

Managing  technical  and  organisational  risks  was  a  centrepiece  in  defining  the  changes  to  be 
implemented.  As  outlined  above,  a  change  in  the  contract  structure  could  not  bring  serious 
advantages in this area either. 

There are some changes expected (accession, publication of additional defence related notices) 
but they have no significant impact on the volumes and costs, except for the need of translation 
into additional language(s). 

232

 
 
 
 
 
 
 
 
 
 
 
Availability of the report on Europa  

For internal use of the Commission.  

Ex-ante evaluation of the contract "Performance of publication services" 

Evaluation related study

ABB activities: 

26 02 Multimedia production 

Timing:  

 14/12/2011  

Background, scope and focus  

It's  an  ex-ante  evaluation  of  a  multiple  framework  contract  for  the  provision  of  the  general 
publication  services.  It's  a  complex  contract  with  several  lots,  multiple  contractors  and  large 
variety of publishing services required.  

This  evaluation  was  in-deep  analysis  of  the  contract  structure,  price  schedule,  technological 
requirements  that  should  adapt  the  contract  to  quick  technological  changes  (digitalization  of 
printing,  electronic  distribution,  multiplication  of  technologies),  low  profitability  of  the 
publishing industry and the OP's limited human resources. 

Summary of findings and conclusions  

The findings of the evaluation showed that, although the current contract is functioning well, the 
future one should be restructured in order to adapt to the evolution of technology, the trend to 
limit paper production and go towards electronic publications and to fulfil the requirements of 
the Transformation Programme of the Publications Office. 

The next contract should offer a contractual framework for more efficient multi-purposing (by 
media) of texts as well as an updated approach to e-publishing. 

This  can  be  ensured  by  a  central  role  for  the  extensible  Markup  Language  (XML),  a  format 
permitting separation of content from the medium employed, development of an electronic-first 
workflow capability and pure XML publishing. To achieve this, prepress and printing operations 
should be separated. 

This  evaluation  confirmed  the  necessity  to  shorten  production  time  and  to  simplify 
administrative and technical work. The Cross-media Publishing Unit is analysing the processes 
in order to see how to achieve this objective. Some of the solutions investigated were: possibility 

233

 
 
 
 
 
 
 
 
 
of  dissemination  by  the  printers  and  fast  digital  printing  for  small  print-runs  of  urgent 
publications. 

Availability of the report on Europa  

For internal use of the Commission.  

234

 
 
 
 
29 – Statistics  

Report from the Commission to the European Parliament and the Council 
on the quality of fiscal data reported by Member States in 2010 

Evaluation related study

ABB activities: 29.02 

29 02 Production of statistical information 

Timing:  

11/04/2011 

Background, scope and focus  

Article 8(3) of Council Regulation (EC) No 479/2009 on the application of the Protocol on the 
excessive  deficit  procedure  annexed  to  the  Treaty  establishing  the  European  Community1 
requires  the  Commission  (Eurostat)  to  report  regularly  to  the  European  Parliament  and  to  the 
Council  on  the  quality  of  the  actual  data  reported  by  Member  States.  The  report  provides  an 
overall assessment of compliance with accounting rules, completeness, reliability, timeliness and 
consistency of the data. 

Summary of findings and conclusions  

Eurostat  concludes  that  progress  on  the  quality  of  fiscal  data  continued  in  2010.  In  general, 
Member  States  have  provided  better  information,  both  in  EDP  notification  tables  and  in  other 
relevant  statistical  returns.  Overall  consistency  of  EDP  data  with  the  reported  ESA95 
government  accounts  is  satisfactory  and  is  still  improving,  particularly  on  the  financial  side, 
compared with the situation in earlier years. 

No  reservations  on  the  quality  of  the  reported  data  were  expressed  in  2010  apart  from  the 
reservation  on  the  quality  of  the  data  reported  by  Greece  in  the  April  2010  notification.  This 
reservation  was  lifted  in  November  2010.  Closing  a  long-lasting  and  critical  process  since 
October 2009, Eurostat concluded on 22 November that the latest revised data for 2006-2009 for 
Greece were sufficiently reliable for EDP purpose, in line with the quality of the data of other 
EU  Member  states.  A  specific  information  note  on Greek  fiscal  data  is  available  on  Eurostat's 
web site. 

In  spite  of  the  recognised  improvements,  some  issues  nevertheless  persist  linked  to  the 
compliance  with  accounting  rules  and  the  completeness  and  quality  of  some  of  the  statistical 
information provided. Eurostat has therefore invited Member States to continue investing in the 
quality of government finance statistics with a view to achieving the desired level of quality as 
regards the compliance with the relevant accounting rules, completeness, reliability, timeliness 
and consistency of government data. 

Furthermore, drawing upon the experience in the case of Greece  and upon observations of the 
European Statistical Governance Advisory Board, the Commission will issue a Communication 

235

 
 
 
 
 
 
 
 
 
addressing the issues of systemic risk in national statistical systems and of the deployment of a 
risk-based  approach  for  Excessive  Deficit  Procedure  data,  notably  for  upstream  source  data, 
implementing Eurostat’s strengthened powers under Regulation (EC) 479/2009, as amended. 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Fiscal%20data.pdf  

Report from the Commission to the European Parliament and the Council on the 
implementation of Regulation (EC) No 450/2003 of the European Parliament and of 
the Council concerning the labour cost index (LCI) 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 08.03.2011 

Background, scope and focus  

Each  quarter,  The  Commission  (Eurostat)  releases  an  update  on  the  hourly  labour  cost  index, 
which is an important tool for analysing short- and medium-term economic development in the 
EU.  

Article 13 of Regulation (EC) No 450/2003 states that the Commission must submit a report on 
the quality of labour cost data to the European Parliament and the Council every two years.  

More particularly, the 2011 report focuses on accuracy, timeliness and coherence of the data. 

Summary of findings and conclusions  

Overall, the degree of compliance with the LCI Regulation has continued to increase since the 
previous report in 2008. Most Member States are now compliant. Member States have continued 
to  channel  resources  into  implementing  actions  to  achieve  more  comparable  and  timely  index 
series. This has clearly raised the overall quality and thus increased the usefulness of the data. 
The timeliness of data delivery has improved, and the number of Member States with accuracy 
problems has decreased. 

236

 
 
 
 
 
 
 
 
 
 
 
 
In recent years, the Commission (Eurostat) has regularly urged Member States to step up their 
improvement  efforts.  The  Commission  will  be  monitoring  the  remaining  non-compliance  and 
quality issues regularly through the data delivered and other national documentation. Where the 
desired  or  planned  improvements  are  not  advancing  appropriately,  the  relevant  national 
authorities  will  be  approached,  and  the  Commission  will  take  the  necessary  action  to  enforce 
compliance. 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Labour%20cost%20index.p
df  

Report from the Commission to the European Parliament and the Council on the 
implementation of Decision (EC) No 1297/2008/EC of the European Parliament 
and of the Council of 16 December 2008 on a Programme for the Modernisation of 

European Enterprise and Trade Statistics (MEETS) 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

11/01/2011 

Background, scope and focus  

This report documents the activities carried out by Member States under the Modernisation of 
European  Enterprise  and  Trade  Statistics  (MEETS)  Programme  together  with  the  Commission 
(Eurostat),  with  respect  to  the  provisions  of  the  MEETS  Decision  and  the  associated 
Commission  Decisions  on  the  annual  work  programmes  for  2009  and  2010,  both  adopted  in 
2009.  The  MEETS  project  follows  very  closely  the  main  orientations  set  out  in  the 
Communication from the Commission to the European Parliament and the Council of 10 August 
2009 on the production method of EU statistics: a vision for the next decade. 

Summary of findings and conclusions  

Since the adoption of Decision No 1297/2008/EC and the adoption of the first work programme 

237

 
 
 
 
 
 
 
 
 
 
 
 
in April 2009, more than 25 projects have been launched for the four MEETS objectives. Major 
progress has been made in setting up the collaborative networks that are undertaking activities in 
the following six main areas: consistency of concepts and methods, profiling large and complex 
in  statistical 
multinational  enterprise  groups,  microdata-linking  and  data  warehousing 
production,  linking  micro-data  on  ICT  usage,  methodology  for  modern  business  statistics  and 
the  use  of  administrative  and  accounting  data.  In  addition,  several  projects  were  launched  to 
support the development and implementation of the EuroGroups Register in the Member States. 
It  should  be  mentioned,  however,  that  Member  States  have  a  limited  absorption  capacity  as 
regards the additional co-ordination activities involved in the networks. 

All  the  studies  that  were  carried  out  and  finalised  will  be  evaluated  together  with  the  main 
stakeholders from Commission services and the Member States. Developed methodologies will 
be  piloted.  The  identification  of  areas  of  lesser  importance  will  be  further  integrated  into  the 
general  overall  strategy  of  Eurostat  on  priority  setting.  This  strategy  is  based  on  the 
Communication from the  Commission to the European Parliament and the Council on priority 
setting, simplification and reduction of response burden in the field of Community statistics. 

Adequate  EU  funding  has  been  guaranteed  by  two  annual  work  programmes  adopted  by  the 
Commission  in  order  to  help  Member  States  to  achieve  the  goals  in  respect  of  reducing  the 
burden on businesses and to help the European Statistical System in responding to new needs for 
statistical  information.  Thirty-one  grants  totalling  around  €13.5m  were  co-financed  and  13 
external studies for a total of around €1.5m were subcontracted. In addition, workshops and task 
forces  were  financed  to  develop  the  actions.  Owing  to  their  complexity,  all  actions  will  be 
continued  and  more  detailed  information  on  the  progress  made  will  be  provided  in  the  annual 
reports, which will continue to be produced until 2013. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0813:FIN:EN:PDF  

Rolling review of the Education Statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 10/06/2011 

Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 

238

 
 
 
 
 
 
 
 
 
 
 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

Eurostat conducted a rolling review of Education Statistics, in order to evaluate the quality of the 
outputs  and  the  processes  in  this  area.  This  Rolling  Review  covers  the  following  three 
collections  in  the  area  of  education  statistics:  UNESCO  Institute  of  Statistics/OECD/Eurostat 
(UOE)  data  collection  on  education  systems,  Continuous  Vocational  Training  Survey  (CVTS) 
and Adult Education survey (AES). The three collections constitute the main instruments for the 
data collection in regard with the EU (and sometimes Candidate and EFTA countries’) Lifelong 
Learning  statistics.  This  Rolling  Review  focuses  mainly  on  the  UOE  data  collection  and  the 
CVTS.  

Summary of findings and conclusions  

Education  statistics  are  a  statistical  product  of  high  relevance  for  users  and  for  the  national 
authorities that produce them. However, the results of this rolling review reveal that depending 
on  the  collection  there  are  particular  quality  elements  of  the  produced  statistics  or  particular 
elements in various developments in this area with which users and partners are not satisfied. In 
most of the cases Eurostat is well aware about this; however, mainly resource constrains hinder 
improvements from the side of Eurostat.  

Altogether,  16  recommendations  for  improvement  were  made  with  8  concerning  data 
production, 3 on data and metadata dissemination, 2 on data quality and 3 on other matters.  

13 of those recommendations were based on partners' suggestions, 6 on self-assessment checklist 
and 3 on user survey results (with some recommendations being based on multiple sources). 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Executive%20Summary_ED
UC%20Rolling%20Review.pdf  

Rolling review of the Information Society (IS) Statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

07/04/2011 

239

 
 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

Rolling  review  of  Information  Society  (IS)  Statistics  was  carried  out  between  June  2010  and 
March  2011  collecting  information  through  three  instruments:  a  user  satisfaction  survey,  a 
partner satisfaction survey and the completion of a self-assessment checklist by the professional 
staff within Eurostat, assessing all major steps within the data production cycle. 

Summary of findings and conclusions  

IS statistics are a statistical product of high relevance for users and their overall quality is highly 
rated by both users and by the national authorities that produce them. 

Depending  on  the  IS  dataset,  the  collection  of  these  data  is  either  based  on  legislation  or 
gentleman’s agreement or compiled from the data produced by other Eurostat units. 

Therefore  particularities  apply  to  each  individual  dataset  and  their  production.  For  instance, 
partners do not perceive as a problem of the Telecommunication services statistics the absence 
of European statistical legal basis for their production. While users of ICT usage statistics do not 
advocate  the  change  of  the  current  flexibility  to  a  more  stable  set  of  IS  indicators,  partners  of 
these  statistics  are  almost  equally  split  between  the  two  options.  Both  users  and  partners 
welcome the collection of micro-data on ICT usage in households and by individuals from 2011 
onwards. 

Altogether,  10  recommendations  for  improvement  were  made,  out  of  which  4  on  data 
productions, 2 concerning dissemination, 2 on data quality and 2 on other matters.  

7  were  based  on  partner  survey,  7  on  self-assessment  checklist  and  5  on  user  survey  results 
(some recommendations were based on multiple sources). 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Executive%20Summary_IS
%20Rolling%20Review_0.pdf  

240

 
 
 
 
 
 
Rolling Review on Inland Waterways Transport 
Statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information   

Timing:  

 March 2011 

Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

This evaluation covers the collection of annual and quarterly statistics for goods transported in 
tonnes and tonne-km and the shares by nationality of vessels by type of vessels and by type of 
goods.  Eurostat  collects  data  on  Inland  Waterways  transport  under  Regulation  (EC)  No 
1365/2006  of  the  European  Parliament  and  of  the  Council  on  Inland  Waterways  transport 
statistics.  

Summary of findings and conclusions  

While  users  and  partners  share  a  positive  appreciation  of  the  quality  of  the  statistics,  they  can 
take very different views about individual qualities. Partners were generally more positive about 
quality of Eurostat’s Inland Waterways statistics than users. One major exception was coherence 
where  users  were  much  more  strongly  positive  than  partners.  This  may  reflect  partners’  better 
knowledge of the mirror statistics. At the opposite extreme, partners were much more positive 
about timeliness than users. Perhaps again, their understanding of what is involved in preparing 
the statistics within the current timetable is a factor in this case. Completeness is another quality 
where  partners  outscored  uses.  For  accuracy,  accessibility  and  clarity,  partner  and  user  scores 
were relatively close. 

A  key  issue  were  the  problems  revealed  by  the  mirror  statistics,  illustrating  how  well  the 
international transport between two partners was matched in the statistics supplied. These show 
disturbing  differences  and  make  quality  improvement  urgent.  Better  cooperation  between 
partners could help to provide this.  

The level of product detail collected was not meeting the needs of some important users but no 
easy solution was at hand. Timeliness is another issue but improvement would be difficult. 

There still remain a number of quality issues, for example in the mirror statistics and in transit 
traffic.  These  will  be  the  subject  of  further  discussion  with  partners  in  the  normal  round  of 

241

 
 
 
 
 
 
 
 
 
 
consultation in the Inland Waterways transport Statistics Working Group. Even so, there is now 
a need to look again at the collection process to see how well it matches important user needs 
and  how  best  to  attempt  to  meet  these.  The  issues  that  stand  out  are  product  detail  and 
comparability across countries. 

Altogether, 9 recommendations for improvement were made with 2 on improving dissemination, 
4 on data collection, 2 on improving data quality and 1 on partner/user engagement. 

3  were  based  on  self-assessment  check-list,  2  on  partner  survey  results  and  5  on  user  survey 
results (with some recommendations based on multiple sources). 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Executive%20summary%20
Final.pdf  

Rolling Review - Labour Force Survey Statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 December 2011 

Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

This rolling review focuses on EU labour force statistics and the Labour Force Survey (LFS) in 
particular. The LFS is the most authoritative source for labour market statistics in the EU; it also 
provides  consistent  information  about  the  demographic  and  household  characteristics  of 
individuals to a level of precision not matched by any other household survey. In recent years, 
the  LFS  data  has  become  more  important  because  of  the  increased  political  interest  in  the 
development of the European labour market. 

Summary of findings and conclusions  

242

 
 
 
 
 
 
 
 
 
 
 
 
The EU LFS is unique due to the range of topics it covers (more than 100 variables), the sample 
size and the length of the time series of quarterly data (from 2005 Q1 onwards). It also provides 
information allowing Europe-wide household analyses of the labour market. The existence of the 
legal  basis  of  the  EU  LFS  ensures  cross-country  comparability  of  Europewide  labour  force 
information, which is independent of the national administrative and legislative framework. The 
LFS legal basis includes provisions on quality reporting and this is instrumental in ensuring the 
quality of the EU LFS. At the same time, while the legal basis is a foundation of the EU LFS 
framework, it cannot currently be regarded as being indisputably perfect. 

Problems  that  have  occurred  while  applying  the  Council  Regulation  have  been  mentioned  by 
partner  NSIs,  particularly  in  relation  to  the  interpretation  of  the  precision  requirements,  the 
measurement of some variables and the delays in data transmission. Partner NSIs also recognize 
that national survey practices vary across Member States, raising some doubts about the cross-
national  comparability  of  the  data.  Eurostat  has  recognised  the  importance  of  this  issue  and 
consultation  on  the  review  of  the  legal  basis  of  the  EU  LFS  has  started  between  Eurostat  and 
NSIs. 

Altogether,  11  recommendations  for  improvement  were  made,  with  4  on  improving  survey 
characteristics,  1  on  cooperation  between  Eurostat  and  NSIs,  4  on  improving  quality,  2  on 
efficiency. 

6 were based on partner survey results, 7 on user survey and 1 on self-assessment checklist (with 
some recommendations being based on multiple sources).  

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20LFS%20Statistics%
20Executive%20Summary%20FINAL.pdf  

Rolling Review - Maritime Transport Statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

18/06/2011 

Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 

243

 
 
 
 
 
 
 
 
 
 
 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

This rolling review deals with European maritime transport statistics, covering the collection of 
annual and quarterly statistics of goods and passengers moved through ports, in terms of tonnes 
of cargo, numbers of transport units, numbers of passengers, and numbers and sizes of vessels 
used. Eurostat collects data on maritime transport under Directive 2009/42/EC together with the 
implementing  and  amending  acts  Regulation  1090/2010,  Decision  2010/216/EU,  Decision 
2008/861/EC, and Decision 2001/423/EC. 

Summary of findings and conclusions  

The survey of users of European maritime transport statistics had a reasonable response from a 
good  spread  of  users  in  terms  of  their  affiliation,  interests,  and  their  use  of  the  data  and 
frequency of access. Accuracy and timeliness are considered by users to be the most important 
quality categories. In the context of the recession, up-to-date information such as the provisional 
estimates is of more relevance than quarterly data three or more months after the event. 

Partners  gave  a  positive  assessment  on  Eurostat’s  role  in  the  management  and  production  of 
maritime statistics. With regard to organisation of meetings, publications, data processing, these 
are all considered to be very good.  

When  considering  possible  changes  and  extensions  of  the  data  collection,  partners  appear  to 
recognise the benefits of these, but consider it difficult or impossible to implement them at this 
stage. Hazardous goods statistics are seen to have potential value, given environmental concerns 
and policy. There is a significant lack of knowledge about EMSA data collection, although those 
who  did  know  about  it  felt  that  it  was  useful  source  that  should  be  exploited.  There  is  strong 
support for a centralised ship database. However those who buy or maintain such a database give 
strict  warning  about  the  effort  and  resources  that  are  required  to  maintain  it.  There  was  some 
feeling on the partners' side that the resources available to Eurostat for maritime statistics should 
be increased. 

According to the checklist prepared by Eurostat staff, the strengths of the system are its sound 
legal  basis,  clearly  defined  meta-data,  and  an  established  and  well-understood  methodology. 
Interaction  and  communication  through  the  Working  Group  and Task  Force  are  excellent,  and 
much  appreciated  by  partners.  There  are  good  relationships  with  key  users.  The  timeliness  of 
available information has been much improved by the development of quarterly publication of 
results. 

Weaknesses are recognised as the level of commodity detail, the absence of intermodal linkages 
and  information,  and  the  lack  of  statistics  on  hazardous  cargoes  and  emissions.  There  is  a 
shortage of resources to enable rapid progress in tackling these issues. 

Altogether,  11  recommendations  for  improvement  were  made,  with  3  on  improving 
dissemination,  5  on  extending  data  collection,  2  on  data  quality,  1  on  engagement  with 
partners/users. 

5 were based on partner survey, 3 on self-assessment checklist, 1 on user survey results and 2 
were made by experts conducting the rolling review. 

244

 
 
 
 
 
Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Executive%20Summary%20
Maritime%20Statistics.pdf  

Rolling Review - EU National Accounts 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 November 2011  

Background, scope and focus  

Rolling  reviews  represent  the  layer  of  evaluation  in  the  overall  Quality  Assurance  Framework 
and as such correspond to the requirements of the Commissions'  Internal Control Standards to 
undertake  evaluations  of  major  activities  on  a  regular  basis.  The  results  are  used  to  formulate 
recommendations for improvements. 

The  review  on  National  Accounts  statistics  was  implemented  from  the  end  of  March  2010  to 
November 2011. Conducted in three phases, the review comprised a user satisfaction survey, a 
partner satisfaction survey and the completion of a self-assessment checklist by the professional 
staff within Eurostat, assessing all major steps within the data production cycle. 

Summary of findings and conclusions  

75% of respondents consider this data as essential or important for their work. Users' satisfaction 
was highest with data accuracy and coherence, lowest with accessibility.  

The  main  issues  with  the  data  delivery  from  providers  (NSIs)  are  that  less  than  half  of  the 
countries have timeliness problems, most countries present missing data and less than half of the 
countries  use  inappropriate  transmission  means.  Nevertheless,  almost  all  data  transmitted 
correspond to requirements. 

Altogether,  12  recommendations  for  improvement  with  regard  to  dissemination  of  results, 
improvement of data quality and engagement with partners were put forward. Priority levels and 
ownership was assigned to each recommendation. 

More  particularly,  among  other 
comparability and clarity of data on National Accounts and relevant metadata. 

things  suggestions  concerned 

timeliness,  accessibility 

245

 
 
 
 
 
 
 
 
 
 
Altogether,  12  recommendations  for  improvement  were  made,  with  4  on  dissemination,  4  on 
data quality and 4 on engagement with users/partners. 

5 were based on partner survey, 4 on user survey and 4 on self-assessment checklist (with some 
recommendations being based on multiple sources). 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20National%20Accoun
ts%20Executive%20Summary.pdf  

Report from the Commission to the European Parliament and the Council on the 
implementation of Decision No 1608/2003/EC of the European Parliament and of 

the Council on science and technology statistics 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 11.04.2011 

Background, scope and focus  

Official statistics on science, technology and innovation in the European Union are largely based 
on Decision No 1608/2003/EC of the European Parliament and of the Council of 22 July 2003 
concerning the production and development of Community statistics on science and technology. 
This report evaluates the implementation of the individual statistical actions listed in Article 2 of 
the Decision. 

The first part of the report focuses on the implementation of the measures provided for in Article 
2 of the Decision. It is followed by chapters on data quality, costs and the statistical burden. The 
final  chapter  of  the  report looks  ahead  to  strategic  actions  that  should  be taken in  the years  to 
come. 

Summary of findings and conclusions  

The  main  achievements  in  the  various  domains  of  STI  statistics  have  been  increased  data 

246

 
 
 
 
 
 
 
 
 
 
 
 
production volume and data quality and harmonisation of the data collection questionnaire and 
time series with the OECD; agreement reached on the treatment of ‘R&D funded from abroad’; 
initiation  of  measuring  trans-nationally  coordinated  research  in  Europe  (GBAORD  and  trans-
national public R&D performers); and improvement of the quality and harmonisation of the data 
through the established quality reporting and initiation of improvement actions on this reporting 
basis. 

To  meet  the  requirements  of  Regulation  (EC)  No  753/2004,  many  countries  adapted  their 
national questionnaires and data collection from 2002/2003 onwards. A number of countries had 
problems  with  transmission  of  R&D  and  GBAORD  data  for  the  first  two  obligatory  reference 
years (2003 and 2004), with special reference to the completeness of the data and meeting the 
deadlines  imposed  by  the  above  Commission  Regulation.  Subsequently,  data  availability  and 
deadline compliance improved considerably. 

Regarding  data  quality, 
the 
implementation  of  Regulation  (EC)  No  753/2004.  The  data  users  have  not  raised  criticisms 
concerning the relevance, accuracy or comparability of the data. However, some coverage and 
measurement-related improvements are desired. 

the  quality  of  European  R&D  statistics 

improved  with 

When  it  comes  to  innovation  statistics,  the  timeliness,  completeness  and  comparability  of  the 
national  data  sets  have  improved  thanks  to  the  shorter,  clearer  questionnaires,  the  improved 
production  and  implementation  process  at  national  level  and  greater  familiarity  with  the 
innovation concepts on the part of respondents. A harmonised model questionnaire prepared in 
line with Regulation (EC) No 1450/2004 is considered to have made a positive contribution to 
the comparability of the results. 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Report%20to%20the%20EP
%20and%20the%20Council%20-%20Science%20and%20technology  

Report from the Commission to the European Parliament and the Council concerning 
short-term statistics as required by Council Regulation (EC) No 1165/1998 of 19 May 
1998 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

08/06/2011 

247

 
 
 
 
 
 
 
 
 
 
 
 
Background, scope and focus  

The  report  follows  up  on  the  report  that  was  transmitted  in  June  2008  pursuant  to  the  article 
14(2) of Council regulation 1165/1998 concerning short-term statistics.  

The report outlines the uses of short-term statistics and their relevance for key European policies 
and  the  steering  of  European  monetary  policy.  It  also  indicates  major  developments  in  short-
term statistics since the last quality report; describes in greater detail the different quality aspects 
of  short-term  statistics;  and  presents  the  cost  and  the  burden  imposed  by  the  collection  and 
processing of data for the statistics. 

Summary of findings and conclusions  

Users declared themselves to be satisfied with the quality of short-term statistics and the related 
services. 95% of the users who formulated a judgement on the quality found short-term statistics 
to be adequate, and 60% found it to be good or very good.  

Member  States  show  a  high  level  of  compliance  with  the  short-term  statistics  Regulation  in 
terms of reliability, timeliness, coherence and comparability. The EU27 average score was  8.9 
(out of 10) as of 1st October 2010, as compared to 8.5 for 1st April 2005 and 6.6 for 1st January 
2004. Most Member States are now close to full compliance with the Regulation. 

In  general,  the  timeliness  of  short-term  statistics  can  be  considered  to  be  very  good  and  any 
delays usually correspond to the target delivery day falling on a weekend or a public holiday. In 
2010, all dates announced in the release calendar were met.  

Regarding the statistical burden, the available data for the European Union as a whole indicate a 
reduction  in  both  the  production  cost  and  the  burden  on  enterprises  between  2006  and  2009. 
Short-term statistics already use a variety of instruments to limit the burden on enterprises. 

For some indicators European sample schemes have been introduced. Countries participating in 
such a scheme only have to compile data for those industries or products for which their national 
contributions to the aggregate at European level make a significant difference. 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Short-
term%20statistics_0.pdf  

248

 
 
 
 
 
 
 
Report from the Commission to the European Parliament and the Council on the 
implementation of Council Regulation (EC, Euratom) No 58/97 and of the Regulation 
(EC) No 295/2008 of the European Parliament and of the Council concerning 
structural business statistics (recast) which repealed and replaced Regulation No 
58/97 

Evaluation related study

ABB activities:  

29 02 Production of statistical information 

Timing:  

 04/05/2011 

Background, scope and focus  

Structural business statistics (SBS) cover industry, construction, trade and services. This report 
covers  the  implementation  of  Regulation  No  58/97  that  has  been  repealed  and  replaced  by 
Regulation  (EC)  No  295/2008  of  the European  Parliament  and  of  the  Council.  The  aim  of  the 
report  is  to  provide  information  on  the  actions  undertaken  by  the  European  Commission  to 
ensure that high quality European structural business statistics are made available to users, and 
also  on  the  manner  and  extent  to  which  each  of  the  Member  States  has  implemented  the  SBS 
Regulations.  Moreover,  it  provides  information  on  the  burden  on  businesses  and  the  actions 
taken by Member States in order to reduce that burden. 

Summary of findings and conclusions  

The report concluded that the compliance level for 2007 data has been increased relative to the 
compliance level recorded in the previous report. Also, punctuality has improved generally over 
the  years,  although  some  countries  continue  to  send  their  data  with  some  delay,  which  has 
influenced the punctuality of the dissemination of EU aggregates. Compared with the previous 
report,  the  most  notable  improvements  from  the  punctuality  point  of  view  were  made  by 
Belgium, Ireland, Greece and Slovenia. However, Belgium and Greece sent their data with some 
delay. 

As far as the completeness of the datasets transmitted by the countries is concerned, all datasets 
received  make  up  90%  of  the  required  data.  This  represents  a  significant  improvement  in 
comparison  with  the  situation  described  in  the  previous  report  provided  to  the  European 
Parliament  and  to  the  Council  in  2007,  but  it  is  still  insufficient.  Several  countries  are  taking 
action to improve data availability. 

Eurostat and the Member States have also taken steps in order to reduce the costs and the burden 
on business. Some of the actions taken by Member States were related to their way of organising 
the  SBS  data  collection.  Sample  surveys  were  frequently  used  by  the  majority  of  the  Member 
States. In many Member States there is a strong and growing preference for using administrative 

249

 
 
 
 
 
 
 
 
 
data  sources  instead  of  carrying  out  surveys.  However,  there  are  several  obstacles  to  using 
administrative data sources alone, due to the fact that administrative data normally differ from 
statistical  data  in  terms  of  definitions,  formats,  codes,  transmission  protocols,  etc.  For  this 
reason,  political  support  is  needed  to  establish  agreements  between  the  National  Statistical 
Institutes  and  the  relevant  governmental  organisations  on  producing  administrative  data  which 
would serve the relevant purposes. 

Availability of the report on Europa  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/Structural%20business%20s
tatistics.pdf  

250

 
 
 
 
 
 
32 – Energy 

Mid-term evaluation on the European Energy Programme for Recovery (EERP) 

ABB activities: 

32 04 Conventional and renewable energies 

Timing:  

 December 2011 

Background, scope and focus  

This evaluation on the EERP programme had to assess its contribution to the effectiveness of the 
use  made  for  its  appropriations.  The  report  had  to  cover  3  original  sectors  of  the  EERP 
Programme which are Infrastructure, Offshore Wind Energy and Carbon Capture Storage. The 
general  objective  of  this  evaluation  was  to  analyse  the  relevance,  the  efficiency  and  the 
effectiveness of the Programme. The Specific  objectives are to evaluate  

– 

the state of play in the implementation of the EERP Projects; 

–  evaluate,  as  far  as  possible,  at  this  early  stage  of  the  implementation  of  the  EERP,  the 
impacts  of  the  programme  in  terms  of  economic  recovery,  both  at  microeconomic  and 
macroeconomic levels; 

–  evaluate the contribution that the programme is giving to the achievement of the EU energy 
policy objectives.  

Summary of findings and conclusions  

Main findings of the evaluators  

–  Overall,  the  objectives  and  the  resources  of  the  EEPR  Regulation  were  of  relevance  to 
economic recovery and of significant relevance to energy policy. 

–  The  financial  resources  made  available  are  being  used  but  not  always  as  rapidly  as 
envisaged. The economic crisis has been more protracted than had been hoped. 

–  Nine electricity and gas infrastructure projects had already been completed by mid-2011 – 
two  electricity  interconnector  projects,  three  gas  interconnectors  and  four  reverse  flow/storage 
projects; most other projects have begun construction and/or begun tendering/placing orders for 
major  items  of  capital  expenditures.  A  handful  of  electricity  and  gas  infrastructure  projects, 
mainly 6 large gas infrastructure projects, have not yet taken their Investment Decision, but this 
is not a bar to preparatory investment.  

–  One EEPR offshore wind farm began feeding power to the grid in December 2010; another 
began installing foundations in 2011. The orders placed by two offshore wind farm projects had 
by  May  2011  contributed  significantly  to  the  development  of  industrial  clusters  in  innovative 

251

 
 
 
 
 
 
 
 
 
 
technologies.  

–  No  CCS  demonstration  project  has  yet  taken  the  final  investment  decision.  At  least  one 
project appeared likely to do so some time in 2012.  

–  Where projects are taking longer to get off the ground than had been hoped, this is consistent 
with the pattern of financing and permitting difficulty frequently found in programmes involving 
large  infrastructure,  particular  where  projects  cross  borders  or  a  technology  is  moving  from 
demonstration to industrial-scale deployment. 

–  Utilisation  of  the  EEPR  funds  measured  solely  on  the  basis  of  payments  to  promoters 
understates  the  impact  of  the  programme,  since  this  has  leveraged  substantial  additional 
expenditure in terms of orders placed. 

–  The  EEPR  had  a  catalytic  effect  on  other  funders  in  a  significant  number  of  cases  and  on 
regulatory  authorities  in  a  few  cases,  and  therefore  has  already  contributed  to  achieving  the 
programme objectives to a perceptible extent. 

–  There  appear  to  be  short-term  employment  benefits  in  electricity  and  gas  infrastructure, 
offshore wind and to a lesser extent CCS. Long-term employment and competitiveness benefits 
from offshore wind, and with less certainty at this stage, CCS appear likely. 

– 
It  is  already  possible  to  identify  infrastructure  completed  projects  in  which  the  EEPR  is 
contributing to security of supply, competitiveness, greenhouse gas reduction and completion of 
the single market. 

Main recommendations by the evaluators 

The contractor failed to include recommendations to improve the performance of the programme 
in the upcoming years. An earlier version of the report included two sets of recommendations  

1.  Related to the programme design, and hence not relevant for a mid- term review and  

2.  Projects management tools to facilitate future programme reviews and reporting. Regarding 
the 2. key recommendations were:  

–  Better  use  of  standard  reporting  templates  across  the  three  sub-programmes  to  facilitate 
comparisons. 

–  Set up relational databases for the management of projects, preferably at EEPR level but as a 
minimum for each sub programme 

–  Define  and  collect  performance  indicators  (budgetary  objectives,  project  progress  and 
expenditure,  location  of  suppliers/  manufacturers,  job  creation  and  skills,  value  of  the  orders, 
socio-economic indicators, energy policy indicators) 

–  Report progress based on the above mentioned indicators 

Availability of the report on Europa  

http://ec.europa.eu/energy/evaluations/annual_en.htm 

252

 
 
 
 
 
 
Mid –term evaluation on horizontal issues concerning energy savings in the EU 

ABB activities: 

32 04 Conventional and renewable energies 

Timing:  

 October 2011 

Background, scope and focus  

The mid-term evaluation reviewed the cross-cutting provisions of the energy services Directive 
(Directive  2006/32/EC).  The  sectorial  provisions  of  this  Directive  are  evaluated  in  the 
framework of a parallel "Background study for Energy Supply Side Efficiency Framework".  

The evaluation was conducted in preparation for the new energy efficiency Directive (proposed 
in June 2011) which will replace Directive 2006/32/EC once adopted. 

The evaluation period covers the years 2006 until present. 

The main objective of the evaluation was to get feedback for the revision of major cross-cutting 
provisions of Directive 2006/32/EC.  

Summary of findings and conclusions  

The main findings of the evaluators were  

The 9% end use energy target of Directive 2006/32/EC has not contributed to extra savings for 
the majority of Member States. 

1.  The National Energy Efficiency Action Plans play a major role for Member States' energy 
efficiency policy coordination. However, the monitoring of individual measures through the so-
called "bottom-up"methodology is extremely costly. 

2.  The  development  of  national  ESCO  (energy  service  company)  markets  is  still  highly 
different from Member State to Member State. 

Main recommendations by the evaluators 

3.  Transparency on the available ESCO offers should be increased. 

4.  Reporting through National Energy Efficiency Action Plans should continue after 2016 and 
should take a larger scope. 

5.  The exemplary role of the public sector as formulated in Directive 2006/32/EC is too weak 
and would need to be stepped up. 

The  findings  of  the  mid-term  evaluation  have  been  incorporated  into  the  draft  for  the  present 
proposal for an Energy Efficiency Directive. 

253

 
 
 
 
 
 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/energy/evaluations/annual_en.htm 

Report of the Mid-term Evaluation on the Background study for energy supply side 
efficiency framework 

ABB activities: 

32 04 Conventional and renewable energies 

Timing:  

 December 2011 

Background, scope and focus  

The  mid-term  evaluation  part  of  this  study  reviewed  the  implementation  of  the  cogeneration 
Directive  (2004/8/EC)  and  the  supply  side  related  provisions  of  the  energy  services  Directive 
(Directive 2006/32/EC).  

The  evaluation  was  conducted  as  a  preparation  for  the  new  energy  efficiency  Directive  which 
will recast Directive 2004/8/EC and Directive 2006/32/EC once adopted. 

The evaluation period covers the years of 2004-2011 for Directive 2004/8/EC and 2006-2011 for 
Directive 2006/32/EC. No previous evaluation of this legislation has been conducted. 

The main objective of the evaluation was to get feedback for the revision of Directive 2004/8/EC 
and the supply side provisions of Directive 2006/32/EC.  

Summary of findings and conclusions  

The main findings of the evaluators were in relation to the cogeneration Directive 

1.  The effect of the cogeneration Directive on the uptake of high-efficiency cogeneration has 
remained  limited.  Its  impact  on  national  policy  and  market  developments  varies  greatly  in 
different Member States, especially in support schemes and electricity grid system rules.  

2.  Little binding provisions were provided to help realise national potentials; instead the focus 
is on the analysis of potentials and evaluation of barriers. This helped develop policy experience, 
but contributed to the fact that growth in combined heat and power (CHP) has been significantly 
less than anticipated and often below economic potentials.  

254

 
 
 
 
 
 
 
 
 
 
 
 
 
3.  The  Directive  successfully  established  a  common  framework  for  defining  high-efficiency 
cogeneration  and  calculating  electricity  from  cogeneration.  The  Guarantees  of  Origin  systems 
based  on  this  framework  proved  however  not  fully  operational  everywhere  to  drive  market 
differentiation or better value.   

4.  The Directive is silent on district heating and cooling, which acts as enabler creating viable 
heat  loads  for  CHP  plants  and  waste  heat  and  is  a  vital  element  for  achieving  energy  savings. 
The development of these networks remained uneven and overall at a low level of 10% of total 
energy supply in the EU (2009). 

The  main  findings  of  the  evaluators  were  in  relation  to  the  energy  services  Directive 
(Articles 6, 7.2, 10, 13)  

1.  The impact of provisions on energy companies (Article 6) has been moderate. The market 
for  energy  efficiency  services  did  not  develop  much  and  energy  efficiency  still  plays  a  minor 
role in most energy companies' business model. 

2.  The provisions on information and advice on energy efficiency for final consumers (Article 
7.2)  were  implemented  in  general  through  soft  measures,  such  as  information  campaigns  to 
general public mainly by government agencies.  

3.  The role of network tariffs and regulations in enabling energy efficiency measures remained 
limited. There was no consensus on the interpretation of the relevant provisions (Article 10) and 
tariff methodologies failed to recognise the value of energy conservation and efficiency.  

4.  The  broad  general  wording  of  the  provisions  on  metering  and  informative  billing  (Article 
13) left too much room for interpretation allowing Member States to justify not taking action. 

Main recommendations by the evaluators 

1.  Focus  should  be  given  to  a  common  policy  framework  for  the  effective  promotion  of 
cogeneration to stimulate national policies, to give cogeneration and district heating & cooling 
equal  basis  to  compete  in  the  internal  energy  market  and  to  realise  cost-effective  national 
potentials 

2.  Policy  measures  are  needed  to  enhance  the  market  for  energy  efficiency  services,  such  as 
energy  saving  obligations,  support  for  ESCOs  and  energy  performance  contracting  and 
providing for dedicated funds and programmes,  

3.  The potential for energy savings through network should be better pursued by using network 
tariffs and regulations and, inter alia focus on enabling demand response. 

4.  Feedback, information and advice on actual energy consumptions to consumers are needed 
to  enable  savings  by  end-users.  This  can  be  done  by  e.g.  more  frequent  billing,  real-time 
information in conjunction with feedback, such as on visual displays, offering financial reward 
for consumption reduction and load shifting. 

The  findings  of  the  mid-term  evaluation  have  been  incorporated  into  the  draft  for  the 
present proposal for an Energy Efficiency Directive. 

Availability of the report on Europa  

http://ec.europa.eu/energy/evaluations/annual_en.htm 

255

 
 
 
 
 
 
Report on the final evaluation of the Intelligent Energy Europe II 

ABB activities: 

32 04 Conventional and renewable energies 

Timing:  

 September 2011 

Background, scope and focus  

The Intelligent Energy – Europe (IEE) Programme supports deployment of sustainable energy. 
The  Programme  focuses  on  the  removal  of  market  barriers  and  creating  a  more  favourable 
business environment for increasing energy efficiency and renewable energy markets (including 
clean  transport),  changing  behaviour,  raising  awareness,  and  making  EU  energy  policy  better 
understood and implemented in Europe's cities and regions. 

Intelligent Energy – Europe (2007-2013), with the total budget of EUR 730 million, builds on 
the  experience  gained  from  its  predecessor,  the  first  Intelligent  Energy  -  Europe  (2003-2006) 
Programme.  Since  2007,  Intelligent  Energy  –  Europe  has  been  included  in  the  overall 
Competitiveness and Innovation Framework Programme (CIP)1 in order to better contribute to 
achieving the objectives of EU energy policy and to implementing the Lisbon Agenda. 

The  IEE  Programme  objective  is  to  support  sustainable  development  and  to  contribute  to  the 
achievement  of  the  general  goals  of  environmental  protection,  security  of  supply  and 
competitiveness, by providing for action in the fields of: 

– 

fostering energy efficiency and the rational use of energy resources ('SAVE'); 

–  promoting  new  and  renewable  energy  sources  and  to  support  energy  diversification 
('ALTENER'); 

–  promoting energy efficiency and the use of new and renewable energy sources in transport 
('STEER'). 

The  main  aim  of  the  final  evaluation  was  to  analyse  the  relevance,  effectiveness,  efficiency, 
coherence and synergies, sustainability, utility, and impact of the actions supported and financed 
by the programme. 

Summary of findings and conclusions  

Main findings: 

1   Decision  No  1639/2006/EC  of  the  European  Parliament  and  of  the  Council  of  24  October  2006 
establishing a Competitiveness and Innovation Framework Programme (2007 to 2013),  Besides of 
IEE II, the following two programmes are also included in the CIP: (1) The Entrepreneurship and 
Innovation  Programme  (EIP)  (2)  The  Information  and  Communications  Technologies  Policy 
Support Programme (ICT-PSP) 

256

 
 
 
 
 
 
 
 
 
                                                 
Relevance:  the  programme’s  objectives  and  funding  priorities  are  perceived  as  relevant  to  the 
needs,  barriers  and  issues  it  was  designed  to  address.  Flexibility  and  adaptability  of  the 
programme  objectives  are  reinforced  by  the  prioritisation  process  (annual  work  programmes) 
that  allows  the  programme  to  evolve  over  time  and  adapt  to  policy  developments  and  budget 
increases. 

Effectiveness:  the  legal  framework  establishing  the  IEE  II  programme  is  clear,  understandable 
and  effective.  Based  on  the  progress  of  the  activities  supported  by  the  programme,  it  is  likely 
that  they  will  achieve  their  objectives.  The  efforts  made  by  the  EACI  to  simplify  the 
management process might increase the effectiveness of the projects. However there is room for 
improvement  in  the  effectiveness  of  programme  level  communication  and  it  is  difficult  for 
project coordinators to quantify the objectives and their related indicators and to collect data to 
feed them.  

Efficiency:  The  adequacy  of  the  level  of  funding  under  IEE  II  to  achieve  the  programme’s 
objectives is difficult to determine at the overall programme level (macro level) given the fact 
that  impacts  cannot  easily  be  identified  for  IEE  within  the  overall  EU  energy  framework. 
Nonetheless,  the  activities  funded  by  the  programme  are  perceived  as  better  value  for  money 
than  alternatives  where  these  exist.  The  means  put  in  place  are  not  excessive,  and  could  be 
increased  to  better  facilitate  achievement  of  the  overarching  objectives  of  the  programme. 
Moreover the EACI is overall perceived as efficient. 

Coherence  and  synergies:  there  is  evidence  of  interactions  and  synergies  between  IEE  II  and 
other EU initiatives in the field of sustainable energy development. 

Sustainability: the actions co-funded/funded by the programme should generate impacts which 
are  likely  to  have  a  lasting  effects.  However,  the  expected  (lasting)  impacts  of  the  actions 
supported are unlikely to be quantified or directly imputable to the sole action of the programme.  

Utility: the programme is perceived as bringing added-value. The main added-value reported are 
the transnational dimension of the action supported, the transfer of knowledge and best practices 
from  more  advanced  Member  States  in  energy  issues  to  less  advanced  Member  States  helping 
them preparing the path to achieve European objectives and its adequate combination of actions.  

Impact: it is likely that the actions supported by the programme will have an impact on both EU 
and national level policy development and implementation.  

Main recommendations: 

1.Programme-level  communication  on  the  IEE  II  programme  should  be  consolidated  for  all 
programme components to ensure sufficient visibility and consistency.  

2.A  single  consolidated  overview  of  target  groups  for  the  different  programme  components 
should  be  created,  and  used  as  the  basis  for  follow-up  of  the  programme  and  project 
communication. 

3.The training for National Contact Points should be developed further, and they should receive 
more programme support.  

4.An upfront indicative prioritisation for the two remaining work programmes (2012 and 2013) 
should be considered. 

5.Market  replication  projects:  target  groups  for  the  dissemination  of  the  experience  should  be 
clearly  defined  and  an  alternative  selection  method  should  be  envisaged  once  it  is  considered 
that a sufficient number of “pilot projects” have been established. 

257

 
 
6.The  national  specificities/needs  should  continue  to  be  taken  into  account  when  prioritising 
projects. 

7.There  should  be  further  follow  up  of  the  management  costs  per  programme  component  and 
reporting on this to the programme committee. 

8.There should be continued close monitoring of the potential overlaps between IEE II and the 
SF/CF INTERREG IVC, as well the LIFE+ programmes; 

9.The synergies of IEE II with other EU programmes should also be continued. 

Availability of the report on Europa  

http://ec.europa.eu/energy/evaluations/annual_en.htm 

258

 
 
 
 
 
 
33 – Justice 

COM(2011)254 REPORT FROM THE COMMISSION TO THE COUNCIL AND 
THE EUROPEAN PARLIAMENT Report on the interim evaluation of the 
"DAPHNE III  Programme  2007-2013"  

ABB activities: 

33 02 Fundamental rights and citizenship 

Timing:  

 11/05/2011 

Background, scope and focus  

As  required  by  Article  15  of  Decision  n°  779/2007/EC  of  the  European  Parliament  and  of  the 
Council1,  this  report  presents  the  results  obtained  in  the  course  of  the  implementation  of  the 
DAPHNE III programme since its approval on 3 July 2007 until 2010.  

The  evaluation  of  the  programme  assessed:  (1)  The  relevance  in  terms  of  the  programme's 
objectives and approach including its EU added value; (2) The effectiveness of the programme; 
(3) The efficiency of the programme. 

The results of the evaluation will be used:  

1.  To improve the implementation during the reminder of the current programme; and, 

2.  To improve the design of the programme in view of the "successor programme". 

Summary of findings and conclusions  

In terms of relevance, the evaluation concludes that overall the Daphne III programme is highly 
relevant to the problems it was designed to address. The level of violence is still prevalent and 
the demand for the programme is high. 

In terms of effectiveness, the Daphne programme has a considerable impact and has managed to 
make  progress  towards  achieving  its  objectives.  However,  a  number  of  procedural  and 
substantive challenges have been identified which affect progress. 

In  terms  of  efficiency,  the  financial  resources  currently  provided  by  the  programme  to 

1  Decision  No  779/2007/EC  of  the  European  Parliament  and  of  the  Council  of  20  June  2007 
establishing  for  the  period  2007-2013  the  specific  programme  to  prevent  and  combat  violence 
against children, young people and women and to protect victims and groups at risk (Daphne III 
programme), as part of the General Programme Fundamental Rights and Justice.  

259

 
 
 
 
 
 
 
 
 
 
 
                                                 
implement  actions  are  appropriate  in  that  they  allow  high  quality  projects  to  be  funded,  while 
greater use could be made of Commission initiatives to support the programme administration 
and the dissemination of results. 

On  the  basis  of  the  main  conclusions/recommendations  of  the  evaluation,  the  improvement  of 
the programme should focus on the following aspects: 

- 

Improved focus on EU policy priorities, 

-  Better dissemination and sustainability of projects' results, 

-  Balanced participation in the Daphne programme, 

-  More efficient management of the Daphne programme. 

In order to improve the reach and impact of the programme the Commission will strengthen the 
links  of  the  programme  with  policy  developments,  ensure  balanced  participation  and  improve 
the  dissemination  of  the  results  of  the  projects  and  the  management  of  the  selection  process. 
Lessons learned during this evaluation will be integrated into the reflection and preparatory work 
for the next multiannual financial framework. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0254:FIN:EN:PDF  

COM(2011)249 REPORT FROM THE COMMISSION TO THE COUNCIL AND 
THE EUROPEAN PARLIAMENT Report on the interim evaluation of the 
"Fundamental Rights and Citizenship  Programme  2007-2013"  

ABB activities: 

33 02 Fundamental rights and citizenship 

Timing:  

 05/05/2011 

Background, scope and focus  

260

 
 
 
 
 
 
 
 
 
 
 
 
 
As required by Article 15 of the Council Decision 2007/252/EC1, this report presents the results 
obtained in the course of the implementation of the Fundamental Rights and Citizenship (FRC) 
Programme. It is based on data from 2007 to 2010.  

The overall aims and objectives of the evaluation were: 
−  To provide an overview of the results obtained in the first three years of the programme; 
−  To  provide  an  assessment  of  qualitative  and  quantitative  aspects  of  the  implementation  in 
terms of relevance, effectiveness and efficiency; 
−  To provide recommendations and guidance on how implementation in the remaining years 
of the programme could be improved. 

Summary of findings and conclusions  

In terms of relevance, the evaluation concludes that the FRC programme is highly relevant to the 
needs it was set out to address. It has contributed to the development and strengthening of EU 
actions  in  the  areas  of  freedom,  security  and  justice  and  specifically  responded  to  the  need  to 
protect fundamental rights and promote EU citizenship. 

In  terms  of  effectiveness,  at  the  stage  of  the  interim  evaluation  any  far-reaching  conclusions 
regarding  whether  the  programme  is  reaching  its  objectives  would  be  somewhat  premature,  in 
view  of  the  limited  number  of  finalised  projects.  Furthermore,  the  programme,  due  to  its 
structure,  varied  priorities,  and  thematic  fields,  range  of  target  groups  and  breadth  of 
stakeholders makes the setting of meaningful and useable indicators complex and the extraction 
of  revealing  information  somewhat  illusive.  Overall,  no  major  obstacles  that  would  be 
considered as a detriment  to the progress in  implementing the activities were  identified by the 
beneficiaries.  It  remains  evident  that  the  projects  have  as  much  led  to  the  deepening  of 
collaboration between existing partners as to the creation of new partnerships. 

In  terms  of  efficiency,  the  Commission’s  general  management  ability  was  considered 
appropriate. Some problems were identified concerning the efficiency of the delivery in terms of 
selection  procedures,  including  aspects  related  to  the  plethora  of  documents  required  for 
submission and the lengthy evaluation procedure that follows. 

On  the  basis  of  the  main  conclusions/recommendations  of  the  evaluation,  the  improvement  of 
the programme should focus on the following aspects: 

- 

Improved focus on EU policy priorities 

-  Concentration on projects with a strong European dimension 

-  Balance of participation in the FRC Programme 

-  More efficient management of the FRC Programme 

- 

Improved dissemination and exploitation of results 

Although  the  evaluation  also  highlighted  some  challenges  in  the  implementation  of  the  FRC 
programme and identified a number of improvements that could be made, there is no denying of 
the intrinsic quality of the programme, its relevance to current European policy priorities, to the 
needs of its chosen target groups and stake-holders, as well as its added value.  

1  Council Decision 2007/252/EC of 19 April 2007 establishing for the period 2007-2013 the specific 
programme ‘Fundamental Rights and Citizenship' as part of the General Programme 'Fundamental 
Rights and Justice'.  

261

 
 
 
                                                 
Therefore,  in  order  to  further  improve  the  impact  and  effectiveness  of  the  programme,  the 
Commission  will  further  focus  the  programme,  ensure  a  stronger  coherence  between  result 
driven  projects  and  policy  developments,  spread  the  reach  of  the  programme  to  the  EU12 
Member  States,  rationalise  the  management  processes  of  the  programme  and  emphasise  the 
dissemination of its achievements. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0249:FIN:EN:PDF   

The Mid-term Evaluation of PROGRESS - Final Report 

ABB activities: 

33 06 Equality 

Timing:  

 22/12/2011 

Background, scope and focus  

The PROGRESS was adopted by Decision No. 1672/2006/EC of the European Parliament and 
of  the  Council  on  the  24th  October  2006  and  it  intervenes  in  five  areas:  1.  Employment,  2. 
Social protection and inclusion, 3. Working conditions, 4. Anti-discrimination and diversity, 5. 
Gender equality. The PROGRESS Programme is managed by DG EMPL and since 01.01.2011 
the strands Anti-discrimination and diversity and Gender equality are managed by DG Justice.  

The  objective  of  the  mid-term  evaluation  is  to  measure  the  degree  to  which  PROGRESS  is 
meeting its objectives, the efficiency of its use of resources and its European added value. The 
evaluation criteria of relevance, effectiveness, efficiency and European added value were used. 
The period covered by the evaluation is from 2007 until 2009. 

The lessons, conclusions and recommendations will be used to:  

− 

Improve the on-going implementation of PROGRESS until 2013;  

−  Assist in preparing and designing the approach for the post-2013 programming period.  

262

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The  conclusions  of  the  mid-term  evaluation  in  the  field  of  Anti-discrimination  and  Gender 
Equality are the following: 

Antidiscrimination  

The evaluation showed a reasonable to good overall score for antidiscrimination. The quality of 
information  and  communication  activities  is  high,  not  least  through  the  FDAD  campaign, 
although the dissemination of some activity products could be improved or amended. There is a 
good achievement of both Immediate and Intermediate Outcomes, and the quality of outputs is 
good.  There  is  room  to  improve  some  aspects  of  stakeholder  engagement  however,  as  well  as 
aspects of added value such as agenda setting. 

Gender equality  

The overall assessment scores for this policy field are average to good. The needs clearly remain 
relevant. The quantity of outputs is overall satisfactory: particularly high for monitoring outputs 
and  lowest  for  statistical  tools.  High  quality  outputs  were  found  particularly  in  training, 
monitoring,  policy  advice  and  NGOs/networks,  but  were  of  a  more  mixed  level  for 
information/communication and identification/dissemination of good practices. The quantity and 
quality of outputs from both the expert networks and NGO were particularly high. Evidence of 
effective partnerships was more limited. 

Availability of the report on Europa  

http://ec.europa.eu/social/keyDocuments.jsp?type=0&policyArea=86&subCategory=987&countr
y=0&year=0&advSearchKey=progressevaluation&mode=advancedSubmit&langId=en   

Evaluation of the impact of the EU instruments affecting children’s rights with a view 
to assessing the level of protection and promotion of children’s rights in the EU 

ABB activities: 

33 02 Fundamental rights and citizenship 

Timing:  

 June 2011  

Background, scope and focus  

The main objective of this evaluation was to identify EU actions in the area of children's rights 

263

 
 
 
 
 
 
 
 
 
 
 
 
and  to  assess  whether  the  EU  had  taken  effective  account  of  the  children  and  their  rights  in 
developing internal and external actions that affect them. 

Summary of findings and conclusions  

Several  shortcomings  were  identified  in  particular  due  to  the  fact  that  children's  rights  policy 
cuts across many policy areas. There were problems within each responsible DG, and between 
the  different  DG  concerned  as  the  roles  were  not  clearly  defined,  there  was  insufficient 
communication with relevant stakeholders.  

Several recommendations were made: development of a comprehensive Action Plan, including 
guidelines  and  capacity  building  for  the  DG,  better  monitoring  in  particular  regarding  the 
funding programmes, and improvement of the stakeholder's consultation. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

Communication from the COM to the EP and the Council 
Consular Protection for EU citizens in third countries: state of play and way forward

Evaluation related study

ABB activities: 

33 02 Fundamental rights and citizenship 

Timing:  

 23/03/2011  

Background, scope and focus  

The right to consular protection adds an external dimension to the concept of Union citizenship. 
Due  to  several  reasons  the  number  of  Member  States'  citizens  living  in  third  countries  is  ever 
increasing and only in the USA, Russia and China are all Member States represented. The Treaty 
of Lisbon reinforced this right and the competence of the Union to act. 

Summary of findings and conclusions  

264

 
 
 
 
 
 
 
 
 
 
 
 
 
Citizens are not sufficiently aware of this right. Furthermore, implementation of this right was 
not correctly ensured by the Member States services. 

The  COM  proposes  to  act  under  three  pillars:  increase  awareness  through  targeted 
communication  measures,  draft  proposals  under  the  new  legal  framework  of  the  Treaty  of 
Lisbon, improve burden sharing and optimise the use of resources, including in crisis situations. 

Availability of the report on Europa  

http://ec.europa.eu/justice/policies/citizenship/diplomatic/docs/communication_consular_protectio
n.pdf 

Study for an external, independent assessment of the European Action on Drugs 

Evaluation related study

ABB activities: 

33 04 Drugs prevention and information 

Timing:  

 31/08/2011  

Background, scope and focus  

In 2010, drug use levels reached historically high levels. While this affects some Member States 
more  than  others,  drug  use  generates  adverse  impacts  on  EU  society  with  regards  to  public 
health, education levels attained, employments, and crime levels. 

Fight against cross-border crimes, including illicit drug trafficking, is one of the main priorities 
identified in the Stockholm Programme. 

Summary of findings and conclusions  

The European Action on Drugs (EAD) responds to the need to mobilise civil society around the 
prevention of drug use and abuse. 

Stakeholders  found  that  the  objectives  of  the  EAD  were  relevant  to  their  needs  (awareness 
raising,  mobilise  civil  society),  were  effectively  achieved  and  were  seen  as  cost  effective  by 
most  stakeholders.  The  impacts  of  the  EAD  on  behavioural  change  regarding  the  drugs  were 

265

 
 
 
 
 
 
 
 
 
 
 
 
more limited even if the EAD was a catalyst to the creation of national networks of civil society 
organisations. 

The recommendations aimed at setting clear and attainable objectives for the EAD, improving 
its  management  and  governance  arrangements  and  improving  information  and  communication 
technologies. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

Report from the COM to the EP and the Council on the implementation since 2007 of 
the Council Framework decision of 13 June 2002 on the European arrest warrant and 
the surrender procedures between Member States (COM(2011) 175 final) 

Evaluation related study

ABB activities: 

33 03 Justice in criminal and civil matters 

Timing:  

 11/4/2011  

Background, scope and focus  

This is the third implementation report since the entering into force of the Framework Decision 
(FWD). This report is the first to be drafted after the entering into force of the Treaty of Lisbon 
and the legally binding nature of the Charter of Fundamental rights of the EU. Being a ex third 
pillar legal instrument, these implementation reports are the only way the Com has to check the 
correct implementation of the FWD as no infringement procedures were allowed. 

Summary of findings and conclusions  

Despite  operational  success,  the  system  still  has  shortcomings.  Transposition  of  the  FWD  in 
some  Member  States  is  still  not  complete  or  correct  in  spite  of  the  fact  that  the  situation  has 
improved as other Member States followed the recommendations made by the COM is previous 
reports. 

The implementation of the FWD at national level presents some problems from the fundamental 

266

 
 
 
 
 
 
 
 
 
 
 
 
rights point of view which led the COM to reinforce the ongoing work on the strengthening of 
the procedural rights of the suspected or accused persons in criminal proceedings. 

Other  measures  were  also  proposed  as  work  ahead  such  as  training,  better  definition  of  the 
proportionality on when to issue an European Arrest Warrant, reinforcement of the transposition 
and implementation of the FWD. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

Report from the COM on the functioning of Council Decision 2005/387/JHA on the 
information exchange, risk assessment and control of new psychoactive substances 
(COM(2011) 430 final) 

Evaluation related study

ABB activities: 

33 04 Drugs prevention and information 

Timing:  

 11/7/2011  

Background, scope and focus  

The assessment is a follow up of the recommendations of the EU Drugs Action Plan 2009-2012. 

It focuses on the functioning of the Council decision to address the new changing reality of new 
psychoactive substances. 

Summary of findings and conclusions  

The report indicates that the market for new psychoactive substances has changed dramatically: 
significant  increase  in  the  number  of  substances  detected,  variety  and  diversification  of  their 
distribution channels in the Member States. The Council Decision does not provide for a system 
that is quick enough to deal with the changing reality.  It concludes therefore that the Council 
Decision is not an adequate instrument to address the new challenges. 

267

 
 
 
 
 
 
 
 
 
 
 
 
 
Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

COM(2011)246 REPORT FROM THE COMMISSION TO THE COUNCIL AND 
THE EUROPEAN PARLIAMENT Report on the interim evaluation of the specific 
Programme "Drug prevention and information"; (DPIP) 2007-2013  

ABB activities: 

33 04 Drugs prevention and information 

Timing:  

 5/5/2011 

Background, scope and focus  

As required by Article 15 (3) (b) of Decision No 1150/2007/EC of the European Parliament and 
of the Council1 (hereinafter referred to as the basic act), this report presents the results obtained 
in  the  course  of  the  implementation  of  the  Drug  prevention  and  information  programme 
(hereinafter referred to as DPIP or DPI Programme) since its approval on 25 September 2007 
till  2010.  Based  on  the  findings  of  the  qualitative  and  quantitative  analysis  of  the  DPIP's 
implementation,  on  its  main  achievements  while  taking  into  account  the  main  challenges,  the 
report provides recommendations concerning the remaining period of implementation.  

Summary of findings and conclusions  

The Interim evaluation confirmed the relevance of the DPIP. Its assessment has proven that both 
the  general  and  the  specific  objectives  are  pertinent  to  the  needs  and  problems  of  the  target 
groups. It also concluded that during the first 3 years of its implementation, the DPI Programme 
could  boast  certain  achievements  in  terms  of  reaching  its  originally-set  objectives,  and  it  has 
shown the potential to achieve good results in the field, despite its limited financial resources.  

1  Decision No 1150/2007/EC of the European Parliament and of the Council of 25 September 2007 
establishing for the period 2007-2013 the Specific Programme Drug prevention and information as 
part of the General Programme Fundamental Rights and Justice  

268

 
 
 
 
 
 
 
 
 
 
 
 
 
                                                 
The  limited  budget  however  is  preventing  the  achievement  of  an  impact  at  a  large  scale  with 
considerable EU dimension which has also limited the number of participating organisations.  

The  findings  of  the  evaluation  confirmed  that  there  are  no  major  difficulties  hindering  the 
successful implementation of the programme. However, in order to maximise the potential of the 
DPIP within its existing framework, to strengthen its impact and to optimize the implementation 
there  is  a  need  for  certain  adjustments.  The  Commission  proposes  the  following  measures  to 
address these:  

Increased EU dimension and EU added value and impact 

In order to gain visibility for the programmes and to ensure that the projects achieve an impact 
not  limited  to  local  or  regional  level,  broader  and  better  elaborated  projects  with  a  higher  EU 
added value and sustainability should be favoured. To deliver activities with a broad European 
dimension,  coverage  and  sustainability,  the  financing  possibilities  within  the  DPIP  should  be 
streamlined  and  the  limited  available  funding  should  be  concentrated  on  multiannual  actions 
implemented by several partners. To this end the Commission will assess the added value of the 
operating grants provided for organisations only for one budgetary year which have had limited 
impact sofar. 

The  Commission  should  further  strengthen  the  synergies  between  the  financial  instruments 
supporting the objectives of the EU Drugs Strategy. The cooperation and communication among 
the  programmes  should  be  enhanced  (in  particular  between  ISEC  and  DPIP  and  DPIP  and  the 
Public  Health  Programme)  at  the  level  of  programming  during  the  formulation  of  the  annual 
work  programmes  in  order  to  avoid  duplications  and  to  allow  potential  beneficiaries  to  target 
their applications more efficiently.  

Increased efficiency of management of DPIP 

The lengthy and administratively demanding procedures have been identified as major obstacle 
to success. To this end the Commission will take measures to reduce the time spent between the 
publication of the calls for proposals and the actual conclusion of the grant agreement to quicken 
the provision of funding. Further consideration should be to address the procedural obligations 
imposed  by  the  basic  act,  namely  the  consultations  on  the  outcome  of  the  selection  of  grant 
beneficiaries  with  the  DPIP  committee  and  the  European  Parliament,  which  should  also  be 
viewed in the context of the future financial perspectives.  

It was confirmed that abolishing of the paper-based application process had a positive impact on 
the  management,  and  was  perceived  positively  by  the  target  audience  of  the  DPIP.  The  new 
PRIAMOS system has proven its usefulness.  

Improved dissemination and better use of results  

The  evaluation  identified  certain  deficits  in  relation  to  the  exploitation  of  the  result  of  the 
projects financed under the DPIP. It was indicated that making the results centrally available for 
the public would not only directly contribute to one of the specific objectives of the DPIP but 
also  would  allow  the  generation  of  better  projects.  This  would  help  the  current  and  future 
beneficiaries  as  well  as  the  Commission  to  avoid  duplications  and  allow  to  build  on  already 
existing achievements. 

269

 
 
 
 
 
Availability of the report on Europa  

http://eur-
lex.europa.eu/Result.do?T1=V5&T2=2011&T3=246&RechType=RECH_naturel&Submit=Searc
h 

COM(2011)351 REPORT FROM THE COMMISSION TO THE COUNCIL AND 
THE EUROPEAN PARLIAMENT Report on the interim evaluation of the specific 
Programme "CIVIL Justice"; (JCIV) 2007-2013  

ABB activities: 

33 03 Justice in criminal and civil matters 

Timing:  

15/6/2011 

Background, scope and focus  

As required by Article 16 (3b) of Decision No 1149/2007/EC of the European Parliament and of 
the Council1 (hereinafter referred to as the basic act), this report presents the results obtained in 
the  course  of  the  implementation  of  the  Civil  Justice  programme  (hereinafter  referred  to  as 
JCIV  or  JCIV  Programme)  since  its  approval  on  25  September  2007  till  2010.  Based  on  the 
findings  of  the  qualitative  and  quantitative  analysis  of  the  JCIV's  implementation,  on  its  main 
achievements  while 
report  provides 
recommendations concerning the remaining period of implementation.  

the  main  challenges, 

into  account 

taking 

the 

Summary of findings and conclusions  

The  programme  started  up  little  more  than  three  years  ago  and  few  of  the  initiatives  financed 
have been completed. It is thus still too early to make an in-depth analysis of the programme's 
impact  and  results.  However,  it  can  be  seen  already  that  the  projects  financed,  both  those 
undertaken  at  the  initiative  of  the  Commission  and  specific  projects,  are  contributing  to  the 
fulfilment of its objectives and, more generally, respond to the Commission's desire to promote 
knowledge  of  European  law  in  the  field  of  civil  justice  and  its  correct  implementation.  Major 

1  Decision of the European Parliament and of the Council of 25 September 2007 establishing for the 
period  2007-2013  the  Specific  Programme  ‘Civil  Justice  as  part  of  the  General  Programme 
Fundamental Rights and Justice  

270

 
 
 
 
 
 
 
 
 
 
 
                                                 
Commission initiatives already under way (the Judicial Atlas, database, information campaigns, 
conferences and studies) are designed to keep legal professionals, and even the general public, 
informed  about  recent  developments  in  European  civil  law  in  various  complementary  ways. 
Particular emphasis is laid on practical and relevant information that can be accessed using new 
communication technologies. 

The possibility of cofinancing NGOs' work programmes, which has a more limited share of the 
budget,  attracted  only  a  few  applications  in  the  first  three  years  and  of  those  only  six 
organisations were selected. There is no doubt about the relevance of the objectives of the Civil 
Justice  Programme,  which  extends  the  impact  of  the  legislative  work  of  the  Commission  and 
sometimes  provides  inputs  upstream.  Like  previous  programmes,  the  Civil  Justice  Programme 
plays an important role in supporting European justice policy. The activities financed with quite 
modest  grants  help  civil  society  organisations  and  Member  States  to  make  an  effective 
contribution  to  the  formulation  and  smooth  implementation  of  European  law.  In  addition,  the 
programme  is  efficiently  managed,  despite  the  limited  human  resources,  as  the  Commission's 
partners generally agree. 

There are still some recommendations that could be made to make the programme even more 
effective, however. 

How can the programme be made more attractive? While the quality of the projects financed 
appears  indisputable,  it  must  be  admitted  that  not  many  proposals  are  received  and  their 
geographic  spread  is  not  very  wide.  The  Commission  has  received  just  over  150  proposals  in 
four years whereas it was expecting between 100 and 150 in the first year. The difficulties were 
identified as follows: • the very technical nature of the topics covered by the programme, which 
means that only the 'initiated' submit proposals; • small organisations have difficulty in finding 
partners and their own sources of cofinancing; • the programme is little known outside the inner 
circle of 'clients'; • potential candidates' fear of complex procedures. 

While the first of these difficulties is intrinsic to the very nature of the programme, it is possible 
to  deal  with  the  three  others  by  doing  two  things:  -  better  publicising  the  programme, 
particularly through presentations in the Member States to reach more grassroots organisations 
and  tell  them  how  to  apply,  how  to  design  a  good  project  and  how  to  find  partners.  The 
Commission works in partnership with the national authorities concerned, which are responsible 
for  bringing  together  the  relevant  associations.  -  simplifying  procedures.  Desired  by  all 
stakeholders,  simplification  is  an  exercise  that  must  unite  the  requirements  of  transparency, 
sound  financial  management  and  equal  treatment  of  partners.  Current  procedures  lead  to 
unacceptably  long  delays  between  the  publication  of  calls  for  proposals  and  the  start-up  of 
projects. They are caused  by a plethora of administrative steps that are disproportionate to the 
amounts involved. 

Better defined priorities that are more in line with the Commission's policy priorities: The 
setting of priorities should focus more on political activities and promote projects that meet these 
priorities. More attention should be paid to the dissemination of project results in order to ensure 
better visibility of both the results and the programme as a whole. 

More  European  added  value  Financing  should  be  concentrated  on  projects  that  have  a  real 
European  dimension  and  offer  the  most  significant  European  added  value.  To  achieve  this 
objective  we  should  finance  larger  projects.  To  do  this  the  necessary  provisions  should  be 
incorporated  in  the  annual  work  programmes  and  the  next  basic  decision.  The  Commission's 
proposed  basic  decision  for  the  current  programme  required  the  participation  of  at  least  three 
States, while the final text provides for no more than two.  

Operating  grants    -  The  operating  grants  component  of  the  programme  has  so  far  produced 
disappointing results, which may be attributable to declining enthusiasm for this type of grant, 
but  another  reason  is  the  fact  that  most  of  the  proposals  received  tend  to  be  closer  to  specific 
projects than annual work programmes. In addition, the European dimension is very difficult to 

271

 
 
identify  in  this  type  of  activity  (except  for  the  true  European  networks  which  were  the  main 
target  of  the  Commission's  proposal).  Given  the  workload  created  by  the  publication  of  an 
annual call for proposals for operating grants and the results obtained, the Commission should 
consider discontinuing this in 2014. 

Links  between  the  programmes:  should  they  be  merged?  Various  ways  of  reforming  the 
programme may be considered with an eye to the next financial programming period starting in 
2014, including a merger with the twin Criminal Justice programme. The programmes were kept 
separate  in  2007  because  of  their  different  legal  bases,  which  called  for  different  adoption 
procedures,  but  this  is  no  longer  the  case  since  the  entry  into  force  of  the  Lisbon  Treaty. 
Accordingly, the Commission might consider proposing a 'Justice' programme bringing together 
the current Civil and Criminal Justice programmes. 

Availability of the report on Europa  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0351:FIN:EN:PDF 

COM(2011)255 EVALUATION REPORT FROM THE COMMISSION TO  THE 
EUROPEAN PARLIAMENT AND THE COUNCIL "Interim evaluation report on 
the Criminal Justice Programme"  

ABB activities: 

33 03 Justice in criminal and civil matters 

Timing:  

11/5/2011 

Background, scope and focus  

As  required  by  Article  16  (3b)  of  Decision  No  2007/126/JHA  of  the  Council1  (hereinafter 
referred to as the basic act), this evaluation report provided an overview of the results obtained 
in th eperiod 2007-2010 and presents the qualitative and quantitative aspects of implementation 
of  the  programme.  The  evaluation  was  carried  out  by  the  Commission,  which  analysed  the 
strengths and weaknesses of the programme by trying to answer the following questions: 

2.  What  improvements  can  be  made  to  implementation  of  the  programme  for  the  period 

1  Decision No 2007/126/JHA of the European Parliament and of the Council of the European Union 
dated  12 February 2007 establishing for the period 2007-2013 the Specific Programme Criminal 
Justice as part of the General Programme Fundamental Rights and Justice  

272

 
 
 
 
 
 
 
 
 
 
 
 
                                                 
remaining? 

3.  What improvements can be made to the design of the programme with a view to drawing up 
the succeeding programme(s)? 

The evaluation examined: :  

the  relevance  of  the  programme,  assessing  how  far  the  programme  objectives  succeed  in 

− 
meeting needs and resolving problems; 

the  effectiveness  of  the  programme,  determining  how  far  the  programme  objectives  have 

− 
been achieved; 

− 

its efficiency, verifying whether the effects were obtained at a reasonable cost.  

Summary of findings and conclusions  

The evaluation of the programme concludes that there is a need to continue this funding, insofar 
as the actions supported tackle specific and genuine problems and make a positive contribution 
to and complement the national measures implemented in this field. However,it  also identified 
several  weaknesses  that  need  to  be  addressed  if  European  actions  in  this  field  are  to  be  even 
more effective. 

However, it has also identified several weaknesses that need to be addressed if European actions 
are to be even more effective. Separate implementation of the civil justice, criminal justice and 
other  programmes  means  that  they  do  not  maximise  real  synergies  and  may  even  create 
unnecessary overlaps. 

The  evaluation  report  made  a  series  of  recommendations  that  it  felt  would  further  enhance  its 
effectiveness, namely: 

PRIORITY SETTING 

Monitoring  of  annual  priorities  should  be  improved  by  increased  use  of  the  PRIAMOS  online 
application  registration  and  evaluation  system  which  enables  proposals  to  be  classified  by 
priority and specific needs on the ground to be identified as a result. This statistical information 
can be used to help set annual priorities.  

EUROPEAN ADDED VALUE 

The evaluation showed that there are a growing number of projects with European added value 
in the field of criminal justice. However, this criterion is still one of the hardest for participating 
organisations to satisfy. It is not interpreted in the same way throughout DG Justice, and it has a 
low weighting compared with other award criteria (15%).  

The  Commission  plans  to  encourage  a  debate  to  define  this  criterion  more  clearly  and  to 
determine  how  much  of  a  role  it  should  play  in  relation  to  national  funding  instruments.  The 
European added value criterion could be given greater weight in the selection procedure.  

SIMPLIFICATION OF PROCEDURES 

In  order  to  make  the  programme  more  effective  without  any  extra  human  resources,  the 
Commission  plans  to  continue  to  harmonise  and  simplify  the  guidelines  (a  practical  guide  for 
applicants will be updated annually), the grant application forms and the evaluation criteria.  

The  provisional  timetables  of  calls  for  proposals  should  be  published  sufficiently  well  in 
advance,  and  the  Commission  should  ensure  that  all  the  deadlines  are  met.  Grant  applicants 
should  also  be  informed  clearly  and  in  good  time  about  the  current  procedures  and  given 

273

 
 
 
sufficient time to prepare their proposals.  

In order to increase the number of projects funded per call for proposals, the Commission should 
plan to publish only one call for proposals for grants for actions every two years.  

It should consider no longer using operating grants because they have not proved effective. 

In the interests of economies of scale, the Commission should try to channel its financial support 
towards larger projects. In the interests of harmonisation with other DG Justice programmes and 
efficient management, certain tasks should be outsourced (e.g. evaluation of the award criteria). 

PUBLICITY AND VISIBILITY 

The programme’s internet site  should be improved to give it greater visibility, for example by 
sending  messages  to  alert  the  programme's  natural  partners  (former  beneficiaries,  members  of 
the programme committee, active networks, etc.). This would be a way of disseminating more 
widely the results of calls for proposals, announcements of new calls and the list of organisations 
eligible to take part in the project.  

Special efforts (e.g. information sessions, FAQs) should be targeted at Member States that are 
under-represented in the programme (CY, DK, IR, LT, SK, SE, EE, FI, GR, LU and PL), to raise 
awareness about the programme and help improve the quality of the proposals.  

In order to promote the emergence and resilience of a civil society in the field of criminal law, 
this awareness-raising campaign should focus on NGOs, which are not sufficiently involved in 
the programme.  

Closer  links  with  other  DG  Justice  programmes  should  be  sought  in  order  to  ensure  a  bigger 
critical mass and hence greater visibility. 

Other types of cooperation through partnership or joint management should be envisaged with 
organisations  with  an  established  reputation  in  the  field,  such  as  the  Council  of  Europe  and 
OECD. 

Availability of the report on Europa  

http://ec.europa.eu/justice/funding/jpen/interim_evaluation_report_2011_en.pdf  

274

 
 
 
 
 
SECTION 3: PROSPECTIVE EVALUATIONS 

This section provides fact sheets for 6 prospective evaluations in support of proposals for new 
or renewed EU interventions. 

 
 
 
04 – Employment and social affairs 

Support for carrying out ex-ante evaluation for the new PROGRESS Programme 
post-2013 

Evaluation related study

ABB activities: 

04 04 Employment, social solidarity and gender equality 

Timing:  

 October 2011 

Background, scope and focus  

A  composite  impact  assessment  of  the  financial  instruments  under  the  responsibility  of  the 
Directorate-General ‛Employment, Social Affairs and Inclusion’, covering the European Social 
Fund, the European Globalisation Adjustment Fund, the Progress Programme, EURES and the 
Progress Microfinance Facility was carried out. The impact assessment considered three options: 

–  Option  1:  no  policy  change:  under  this  option,  the  Progress  programme,  EURES  and  the 
Progress  micro-finance  facility,  would  continue  to  exist  as  separate  instruments  running 
alongside the European Social Fund.  

–  Option  2:  a  new  integrated  programme  for  social  change  and  innovation:  the  new 
programme is made up of three separate albeit complementary sections: Progress, EURES and 
microfinance and social entrepreneurship. 

–  Option 3: a single instrument for employment and social affairs: such an instrument would 
have a shared management strand (the European Social Fund) and a direct management strand.  

The  impact  assessment  has  concluded  that  option  2  is  the  preferred  option  that  would  bring 
about the most benefits in terms of efficiency gains, critical mass, coherence and effectiveness, 
while avoiding political and institutional risks. 

As  part  of  the  review  of  the  PROGRESS  programme,  stakeholder  consultations  and  a  needs 
assessment  were  also  carried  out  to  prepare  the  Commission's  legislative  proposal  and  ex-ante 
evaluation for a new PROGRESS programme (psot-2013). 

Summary of findings and conclusions  

The  Progress  Programme  has  contributed  to  delivering  effective  policy  responses.  The 
strengthened  policy  framework  under  the  new  Europe  2020  strategy  implies  an  even  stronger 
need for evidence-based policy-making so that Union policies and legislation are responsive to 
the  socio-economic  challenges.  The  successor  to  the  Progress  Programme  should  assist  the 
Commission in implementing its tasks: 

276

 
 
 
 
 
 
 
 
 
 
 
– 

fact-finding and evidence-gathering about relevant policy developments; 

–  monitoring  and  reporting  on  progress  made  by  Member  States  towards  Union  common 
priorities and objectives; 

–  ensuring effective and uniform application of Union law;  

–  –modernising Union law on working conditions in line with the Smart Regulation principles. 

EURES aims to improve labour market transparency by making job vacancies available on the 
EURES  Job  Mobility  Portal  and  to  provide  support  for  information,  advice  and  guidance 
services  at  national  and  cross-border  level.  At  the  same  time,  the  role  of  public  employment 
services (PES) has changed as a result of the recent economic crisis and the need for more tailor-
made  services.  PES  should  become  life-long  learning  providers,  offering  a  wide  range  of 
services (such as skills assessment, training, career guidance, matching jobs and profiles, client 
counselling (workers and employers) and catering for the needs of those furthest from the labour 
market.  In  addition,  EURES  should  promote  new  working  methods  with  private  employment 
services. 

One  of  the  major  obstacles  to  business  creation  is  lack  of  access  to  finance,  and  especially 
micro-credit, which has been exacerbated by the recent economic crisis. Europe's micro-finance 
sector  has  not  yet  reached  maturity.  In  order  to  grow,  Union  micro-finance  institutions  would 
need  to  build  and  maintain  adequate  funding  models.  Therefore,  there  is  a  clear  need  to 
strengthen  institutional  capacity-building  (especially  of  non-banking  microfinance  institutions) 
in order to cover start-up costs and funding for lending to high-risk target groups. 

The  European  Union  Programme  Social  Change  and  Innovation  should  aim  to  increase 
coherence of EU action in the employment and social areas by bringing together and building on 
the  implementation  of  the  Progress  Programme,  EURES  and  the  European  Progress 
Microfinance  facility.  In  addition,  the  Programme  provides  for  an  opportunity  to  simplify 
implementation  through  common  provisions  covering,  inter  alia,  common  general  objectives, 
common typology of actions and rationalisation of reporting and evaluation. At the same time, 
the  Programme  provides  for  a  limited  set  of  specific  provisions  applying  to  the  three  sections 
(Progress,  EURES  and  Microfinance  and  Social  Entrepreneurship)  in  order  to  take  account  of 
legal  requirements  (including  comitology  rules  which  only  applies  to  the  Progress  section, 
geographical  coverage  and  specific  reporting  and  evaluation  requirements  in  the  case  of  the 
Microfinance and Social Entrepreneurship section). 

Availability of the report on Europa  

http://eur-
lex.europa.eu/Result.do?direct=yes&lang=en&col=GRP_CITATION&value=52011PC0609&wh
ereihm=Instruments%20cited:%2052011PC0609 

277

 
 
 
 
 
08 – Research  

State of the Art and Forward-Looking Analysis of Environmental Research and 
Innovation 

Evaluation related study

ABB activities: 

08 06 Cooperation - Environment (including climate 
change) 

Timing:  

 12/2011 

Background, scope and focus  

This  is  the  second  of  the  two  studies  required  to  support  the  preparation  of  the  impact 
assessment report for Horizon 2020. 

The aim of the second group of experts (GoE2) on "State of the art and forward looking analysis 
of environmental research and innovation" is to provide the rationale for the EU to intervene and 
to identify future challenges and avenues for environmental research and innovation. This study 
is based on results of the Group of Experts (GoE1) on "Stock-taking of results and Impacts of 
EU-funded environmental research". 

Summary of findings and conclusions  

This  report  identifies  and  appraises  future  challenges  and  options  for  environmental  research 
under the future Horizon 2020. 

The  GoE2  identified  eight  grand  challenges:  Climate  change,  Loss  of  biodiversity,  Resource 
constraints and loss of ecosystem services, Urbanization, Ageing society, Environmental health, 
Social preparedness, and Eco-innovative solutions and their dissemination.  

These include cross cutting themes such as the importance of better monitoring, modelling and 
forecasting  methods  to  design  policies  and  measures,  the  strong  linkages  between  the 
environmental  social  systems,  the  importance  of  inter-disciplinary  research  and  the  regional 
nature of many of the challenges, which stress the need for trans-boundary research and policy 
initiatives.  

– 
International  comparisons  reveal  that  the  EU27  has  an  overall  lower  share  of  government 
expenditure on R&D than the US or Japan, but it does allocate a larger share of that R&D budget 
to environmental categories.  

–  The GoE2 discussed four possible scenarios for the future Horizon 2020. From the analysis 
of those scenarios, it appears that a specific, holistic, cooperative environmental programme is 

278

 
 
 
 
 
 
 
 
 
 
 
needed  to  support  the  increasingly  crucial  environmental  policies,  harvesting  innovative  ideas 
for business purposes and providing a clear interface to address the above mentioned challenges.  

–  At  the  same  time  the  programme  places  itself  firmly  at  the  interface  of  both  ideas-driven 
environmental research and industry-driven research programmes.  

–  Regardless of the policy option selected, it is recommended that the environmental research 
programme  is  implemented  in  a  manner  that  protects  the  competitive  nature  and  equal 
opportunities principle that have applied to previous framework programmes. 

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

Stock-Taking of Results and Impacts of EU-Funded Environmental Research 

Evaluation related study

ABB activities: 

08 06 Cooperation – Environment (including climate 
change) 

Timing:  

 12/2011 

Background, scope and focus  

The aim of this study was to support the preparation of the impact assessment report for Horizon 
2020. 

The  aim  of  the  first  group  of  experts  (GoE1)  on  "Stock-taking  of  results  and  Impacts  of  EU-
funded environmental research" is to provide evidence of outputs, results and impact of FP6 and 
FP7 environmental research. 

Summary of findings and conclusions  

1.  FP Environment research is mainly relevant to impacts related to policy support and research 
excellence. Innovation is not at the top of the agenda/priorities of environmental research funded 
in FPs. The expectations of direct economic benefits cannot therefore be very high, although the 

279

 
 
 
 
 
 
 
 
 
 
 
 
indirect and long-term economic impacts could be huge in several areas. 

2.  Overall, it appears that FP Environment projects achieve their most positive contributions in 
research  excellence  with  a  very  positive  contribution  to  high  quality  publications  significantly 
supporting  leading  European  countries  to  compete  internationally  in  terms  of  publications  and 
citations  as  well  as  an  upward  trend  in  collaborative  publications  both  within  the  EU  and 
beyond, mainly with China, Russia and India. 

3.  Regarding policy impacts, the value of FP environment research is unquestionable especially 
in  terms  of  EU  policy  making,  and  international  agreements  and  conventions.  There  is  ample 
evidence proving the use of FP environment research results in EU policy documents and for the 
development of certain EU directives and standards. 

4.  From  the  business  perspective,  FP  Environment  has  brought  positive  and  sustainable 
changes in the way research is carried out and proved to be a valuable means to access additional 
funding and thus expand research objectives. FP7 Environment research played a significant role 
in  establishing  durable  networks  and  its  value  in  this  regard  is  highly  appreciated.  Although 
direct economic impacts are quite limited, there are indications based on participants’ judgments 
of  positive  economic  impacts  associated  with  certain  environmental  research  areas  in  terms  of 
patents, prototypes and new products and processes. Yet the issue remains that participation by 
businesses, and especially SMEs, stays at a relatively low level. 

5.  These results constituted useful background information for the GoE-2.  

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm?pg=archive#results  

Fusion Energy - State of Development and Future Role 

Evaluation related study

ABB activities: 

08 20 "Euratom – Fusion energy" 

Timing:  

 06/2011 

Background, scope and focus  

The aim of this study was to present the current state of development and future role of fusion 
energy  and  furthermore  analyse  socio-economic  risks  and  opportunities  related  to  funding  of 
fusion research. The study was commissioned by DG RTD in the context of preparations of an 

280

 
 
 
 
 
 
 
 
 
 
 
impact assessment for next Euratom Research and Training Programme. 

Summary of findings and conclusions  

1.  Fusion  research  has  made  considerable  progress  towards  overcoming  the  scientific 
challenges, in line with the research resources made available.  

2.  An increase in resources would speed up the commercialisation of fusion but the prospects 
are too long term and high risk for commercial companies to fund research yet.  

3.  The  physics  which  makes  fusion  power  a  viable  possibility  has  been  demonstrated  at  the 
JET facility in 1997 which produced 16MW of fusion power while being driven by 25MW of 
input  power.  The  challenge  now  is  to  demonstrate  that  fusion  works  in  an  integrated  power 
station  like  facility  and  to  make  the  reaction  stable,  produce  a  net  amount  of  electricity    and 
commercially viable.  

4.  Fusion  offers  genuine  benefits  in  comparison  to  other  energy  sources  that,  given  the 
seemingly  inevitable  growth  in  energy  demand  predicted  should  ensure  it  a  role  in  the  future 
energy mix.  

5.  The  research  and  development  costs  associated  with  developing  fusion  are  very  large  but 
given the scale and value of the global energy market and the benefits fusion could bring, the on-
going  resources  allocated  to  other  alternative  sources  of  energy  (e.g.  renewables)  fusion  is 
worthy of public support.  

Availability of the report on Europa  

http://ec.europa.eu/research/evaluations/index_en.cfm  

281

 
 
 
 
 
 
 
11 – Maritime Affairs and Fisheries  

An ex-ante evaluation for the conclusion of a new Fisheries Partnership Agreement (FPA) 
and Protocol between the European Union and Mauritius 

ABB activities:  

11 03 International fisheries and law of the sea 

Timing:  

 October 2011 

Background, scope and focus  

This evaluation provides a prospective ex-ante evaluation for the conclusion of a new Fisheries 
Partnership Agreement (FPA) between the EU and Mauritius. 

Summary of findings and conclusions  

The impact assessment of a new fishing agreement  and of its protocol of application confirms 
that  the  need  exists  for  the  European  and  Mauritius  fishing  industries,  but  also  to  create  the 
conditions  for  a  bilateral  sectoral  policy  dialogue  between  the  EU  and  the  Authorities  of 
Mauritius.  The  EU  has  frameworks  for  policy  dialogue  at  the  regional  level  (IOTC,  Indian 
Ocean  Commission)  and  at  a  bilateral  level  with  third  countries  having  concluded  bilateral 
fishing agreements with the EU (Comoros, Madagascar, Mozambique and Seychelles). 

Without  an  agreement  with  Mauritius,  the  geographic  coverage  is  incomplete.  In  addition,  the 
impact  assessment  verifies  that  the  conclusion  of  a  fisheries  agreement  with  Mauritius  is 
coherent with other EU interventions in the region which all have the common general objective 
to  promote  responsible  and  sustainable  fishing  practices.  There  are  no  alternative  options  to  a 
fishing agreement to satisfy the needs of the two parties. The other option for the EU would be 
not to act, similar to status quo, with as main disadvantage the lack of framework for bilateral 
sectoral  policy  dialogue.  Without  an  agreement,  the  EU  vessels  could  continue  to  exploit  the 
Mauritian fishing zone but without the legal security and the visibility provided under a fishing 
agreement. 

The  impact  analysis  does  not  identify  adverse  economic,  social  or  environmental  incidences. 
Concerning the economic and social impacts, an agreement will have positive consequences i) 
by  strengthening  the  viability  of  the  European  and  Mauritius  fishing  industries,  and  ii)  by 
providing  the  authorities  in  charge  of  fisheries  with  additional  funding  to  support  the 
implementation  of  the  fisheries  policy  with  benefits  for  the  local  target  populations.  Fishing 
opportunities available for the European tuna vessels in the fishing zone of Mauritius does not 
mean  an  increase  of  fishing  effort  as  the  fishing  vessels  concerned  are  already  active  in  the 
Indian  Ocean.  An  agreement  will  not  result  in  additional  fishing  capacities,  but  will  support  a 
better  spatial  distribution  of  fishing  effort.  Additionally,  the  conclusion  of  an  agreement  will 

282

 
 
 
 
 
 
 
 
 
 
give  to  the  EU  an  instrument  to  manage  its  fishing  fleets,  should  emergency  measures  be 
required. 

The  fishing  zone  of  Mauritius  does  not  show  any  particular  biological  or  environmental 
sensitivities. Environmental problems in relation with the activities of the fishing fleets must be 
tackled at the multilateral level of IOTC. 

Availability of the report on Europa  

http://ec.europa.eu/fisheries/documentation/studies/mauritius/index_en.htm 

16 – Communication 

Ex-Ante Evaluation of the European Year of Citizens 2013 

ABB activities: 

16 05 Fostering European citizenship 

Timing:  

 11/08/2011 

Background, scope and focus  

The  ex-ante  evaluation  was  part  of  a  proposal  for  the  decision  on  the  European  Year  being 
prepared by DG COMM. The aim of this ex-ante evaluation was to ascertain if, how and to what 
extend  the  European  Year  of  Citizens  (EYC)  in  2013  will  enhance  awareness  of  the  rights 
attached to Union citizenship, as well as to gather information and carry out analyses that would 
help to define objectives, to ensure that these objectives can be met, that the instruments used are 
cost-effective and that reliable later evaluation will be possible. 

Summary of findings and conclusions  

The  results  of  the  evaluation  suggest  that  EYC  focus  on  enhancing  awareness  of  Union 
citizenship rights has the potential to contribute to facilitating the exercise of the rights available 
under EU law in a cross-border context, to the benefit both of the individuals concerned and of 
European societies and economies. These rights are relevant in economic, societal and political 

283

 
 
 
 
 
 
 
 
 
 
 
 
 
terms and have the potential to enhance citizens' sense of belonging to the EU. This would also 
be  beneficial  for  Union  citizenship  itself,  given  the  positive  association  between  Union 
citizenship and free movement as one of its core benefits.  

Objectives - The general objective of the EYC would be a greater sense of belonging of citizens 
to the EU through the promotion of rights of Union citizens, in particular the right to move and 
reside  freely  within  the  territory  of  the  Member  States.  There  were  three  specific  policy 
objectives proposed. 

Approach  -  The  EYC  would  combine  an  awareness  raising  campaign  directed  at  the  general 
public as well as at specific target groups (with a focus on multipliers); events targeted at public 
authorities,  practitioners  and  other  key  stakeholders;  a  horizontal  strategy  for  the  use  of 
internet/social media in this context; events related to citizens' rights and citizens' participation 
(that  could  serve  as  a  run  up  to  the  2014  European  elections)  organised  with  the  two 
Presidencies  and  in  close  cooperation  with  the  EP  and  possibly  the  EESC/CoR).  Moreover, 
innovative  –  e.g.  internet/social  media-based  –  forms  of  dialogue  with  citizens  could  be 
organised. 

Benefits (strengths) of this approach are that the EYC would have a high outreach to the general 
public  and  to  specific  target  audiences.  It  would  deliver  clear  and  coherent  messages  and 
mobilise stakeholders at both the EU and national level. Member States as well as civil society 
would  be  empowered  to  assume  ownership  of  the  theme.  This  EYC  would  be  a  concrete 
expression  of  the  Commission's  strong  commitment  to  putting  citizens  at  the  heart  of  the  EU 
political agenda. 

The EYC would be implemented at the EU level and carried out in close cooperation with the 
other  EU  Institutions.  The  Representations  in  the  Member  States  and  the  involvement  of 
Member States through a Steering Committee and the relevant Council formations would ensure 
the  linking  of  the  EU  initiative  with  the  national  level.  Civil  society  would  be  involved  in 
preparing and implementing the EYC.  

Cost - No additional funding is sought as the proposal for the EYC aims at activating existing 
structures. The flexibility for annual priority-setting based on the budget lines and programmes 
of the DG COMM provides sufficient financial margin for running the EYC on a EUR 1 million 
budget.  

The organisation of the EYC will be based on the lessons learned until now from other years, 
existing reports on the policy area as well as stakeholder consultation and other, on-going or to-
be-launched  studies  identified  with  subjects  linked  to  the  objectives  of  the  EYC.  The  findings 
will be used to identify and promote good practices and, as a basis for recommendations. 

There will be an internal evaluation carried out. 

Availability of the report on Europa  

http://ec.europa.eu/dgs/secretariat_general/evaluation/search/search.do  

284

 
 
 
 
 
20 – Trade  

Trade SIA relating to the negotiation of a comprehensive economic and trade agreement 
between the EU and Canada 

ABB activities: 

20 02 Trade policy 

Timing:  

 June 2011 

Background, scope and focus  

The  EU‐Canada  SIA  was  carried  out  with  the  intention  of  assessing  how  a  Comprehensive 
Economic  and Trade Agreement (CETA) between the two sides  might affect  economic, social 
and  environmental  issues  in  the  EU  and  Canada,  as  well  as  in  other  relevant  countries.  Its 
sources of evidence include formal modelling (CGE, E3MG and investment gravity modelling), 
and  quantitative  and  qualitative  evidence  collected  from  desk  research  and  consultations  with 
stakeholders.  Causal  chain  analysis  is  applied  to  the  evidence  base  to  identify  the  significant 
cause‐effect links been the Agreement and its economic, social and environmental impacts. 

The  assessment  is  based  upon  a  modelled  analysis  of  the  likely  impact  of  four  different 
liberalisation scenarios, of differing degrees of ambition; and was undertaken at three levels: (i) 
macro‐economic assessment; (ii) sectoral assessment; and (iii) cross‐cutting assessment. 

Stakeholder  consultation  is  a  core  element  of  the  SIA  method.  The  study  website,  email  and 
phone updates, and the arranging of civil society meetings in Brussels and a local workshop in 
Canada were used to create awareness of the SIA and elicit feedback from stakeholders. In‐depth 
consultations  were  undertaken  via  interviews,  soliciting  written  comments  on  drafts  of  study 
reports and the implementation of steering committee meetings. 

Summary of findings and conclusions  

The analysis finds that the CETA will lead to overall gains in welfare, real GDP, total exports 
and real wages in both Canada and the EU over the long‐term. While these gains are expected 
under  the  four  scenarios  modelled  in  the  economic  assessment,  the  gains  are  expected  to  be 
higher under an agreement that offers the highest degree of tariff and services liberalisation. 

At the sectoral level, the greatest gains in output and trade appear to be stimulated by services 
liberalisation  and  by  the  removal  of  tariffs  applied  on  sensitive  agricultural  products.  Both 
Canada and the EU would benefit from a CETA that provided a high degree of liberalisation in 
the services sector, particularly with respect to transport, telecom and business services. 

The  CETA  could  have  a  positive  social  impact  where  it  includes  text  devoted  to  better 
implementation and ratification of the ILO’s Core Labour Standards and Decent Work Agenda. 

285

 
 
 
 
 
 
 
 
 
 
Canada,  specifically,  could  see  its  standards  and  rights  improved  with  respect  to  collective 
bargaining  and  freedom  of  association  with  provisions  that  require  ratification  of  the  ILO’s 
Convention 98, which provides legally binding measures on such rights. 

In terms of the environment, increased agricultural production could lead to a greater degree of 
intensification and use of chemical inputs, while increased beef production could lead to greater 
herd  size  and  production  of  methane.  The  environmental  impact  associated  with  energy  and 
extractive industries is likely to be limited, though it could be exacerbated if the agreement leads 
to  significant  increases  in  FDI  in  Canada’s  oil  sands  and  mining  industries.  Increased  trade 
would  likely  increase  the  GHG  emissions  associated  with  transport,  though  this  could  be 
mitigated should the CETA replace land transport (notably between Canada and the U.S.) with 
maritime transport and facilitate the development of Canada’s short sea shipping industry. 

Availability of the report on Europa  

http://trade.ec.europa.eu/doclib/docs/2011/september/tradoc_148201.pdf 

http://ec.europa.eu/trade/analysis/sustainability-impact-assessments/assessments/ 

286

 
 
 
 
 
22 – Enlargement  

(Ex-ante) Evaluation to support the preparation of pre-accession financial 
instruments beyond 2013  

ABB activities: 

22 AWBL-03 Enlargement pre-accession negotiations 

Timing:  

 13/06/2011 

Background, scope and focus  

The  main  purpose  of  the  report  was  to  assist  the  Commission  in  preparing  the  future  pre-
accession assistance instrument. The evaluation also helped to inform an Impact Assessment on 
the  successor  instrument,  which  was  prepared  by  DG  ELARG  after  the  publication  of  the 
evaluation. 

Summary of findings and conclusions  

There is a strong rationale for a future pre-accession financial instrument beyond 2013. Taking 
account  of  the  relative  merits  and  impacts  of  the  four  policy  options  considered,  the  preferred 
option was the continuation of the current programme with similar levels of EU divided amongst 
seven  beneficiaries  (Albania,  Bosnia  and  Herzegovina,  former  Yugoslav  Republic  of 
Macedonia,  Montenegro,  Kosovo,  Serbia  and  Turkey).  The  evaluation  also  concluded  that  the 
allocation  of  resources  between  components  and  intervention  areas  should  be  similar  to  that 
envisaged during the current financing period. 

The  evaluation  concluded  that  the  economic  and  wider  benefits  to  the  EU  of  enlargement 
involving the current beneficiaries would more than offset the cost of a new financial instrument 
to the EU.  Also, having an EU instrument enables the realisation of European added value, for 
example:  (i)  an  opportunity  to  closely  link  financial  support  to  progress  with  political  criteria; 
(ii) the means for the EU to act as a catalyst for supranational and interregional cooperation; and, 
(iii)  the  means  for  the  EU  to  act  as  a  broker  for  drawing  on  the  expertise  of  different 
administrations and agencies within the EU (e.g. twinning). 

Availability of the report on Europa  

For internal use of the Commission. 

287

 
 
 
 
 
 
 
 
 
 
 
 
32 – Energy 

Report on Ex-ante Evaluation/impact assessment of a possible successor to the 
intelligent Energy II Programme 

ABB activities: 

32 04 Conventional and renewable energies 

Timing:  

 October 2011 

Background, scope and focus  

Background, scope and focus of the evaluation 

The Directorate General for Energy is in charge of designing and preparing the proposal of the 
Commission for the successor of the multiannual Community programme in the field of energy, 
“Intelligent Energy – Europe II” (IEE II) for the next Multiannual Financial Framework  2014-
2020. This proposal will be made on the basis of Article 192 of the Treaty. 

The ex-ante evaluation report:  

1.  Provides  arguments,  and  supporting  evidence  –  quantified  wherever  possible  –based  on 
elements  of  a  public  consultation  -  as  input  to  the  forthcoming  impact  assessment  by  the 
Commission. 

2.  Gives a view on what the potential IEEII successor programme could look like taking into 
account  that  the  barriers  to  adoption  of  energy  efficiency  and  renewable  energy  sources  that 
were  the  basis  of  the  adoption  of  IEE  II,  and  the  corresponding  EU  policy  position,  may  no 
longer be the same as they were for IEE II. A shift of focus in actions for a successor to IEE II 
could therefore be necessary. 

The  report  is  based  on  data  collected  through  four  surveys  and  several  interviews  with 
the  Executive  Agency  for 
programme  stakeholders  as  well  as  data  collected  from 
Competitiveness  and  Innovation  (EACI)  and  European  Investment  Bank  (EIB)  project 
management databases. It is moreover based on research and review of existing analyses within 
the area of sustainable energy development in the EU. These inputs were combined in qualitative 
and  quantitative  analyses  of  the  defined  policy  options  for  a  potential  successor  to  the  IEE  II 
programme. As is the case with all forward-looking assessments, this assessment of a potential 
successor to IEE II takes departure in existing experiences with the current programme (as the 
final  evaluation  of  IEE  II  was  carried  out  in  parallel  it  has  not  been  possible  to  leverage  all 
material  coming  from  this  evaluation  but  efforts  were  made  to  maximise  this).  The  analysis 
projects these experiences into the future by taking into account expected evolutions of the IEE 
II context by 2014. 

288

 
 
 
 
 
 
 
 
 
 
 
Summary of findings and conclusions  

The evaluation started from the main findings of the final evaluation of IEE II, namely that the 
current  programme  (2007-2013)  is  relevant  and  useful.  The  ex-ante  report  assessed  three 
different policy options which diverged from the status quo addressed in the final evaluation of 
IEE  II:  non  continuation,  a  capacity  building  oriented  programme  and  an  investment  oriented 
programme.  

The report looked at the marginal benefits and costs of these three policy options. It concluded 
that  continuing  the  programme  in  its  current  form  entailed  higher  net  benefits  than  not 
continuing  the  programme.  It  did  not  conclude  whether  a  capacity-building  oriented  or  an 
investment oriented programme would be the most cost-effective option. It stated however that 
both of these options would imply a higher budget than the status quo but the additional benefits 
would clearly outweigh the costs in both cases. These two options would be the best from a cost-
benefit point of view.  

The  study  suggested  that  a  combination  of  them  could  be  envisaged,  i.e.  focusing  on  both 
capacity-building  and  investment  mobilisation.  Furthermore,  given  the  difficulties  Member 
States face in implementing sustainable energy policies, including EU legislation in the field, the 
study  concluded  that  there  is  still  a  great  need  for  projects  that  focus  on  facilitating  policy 
implementation.  Projects  of  this  type  have  been  considered  as  successful  in  the  past,  and  are 
expected to provide high marginal returns in the future. 

On the opposite, the study showed that the marginal returns from awareness raising activities are 
probably  not  as  high  as  in  the  beginning  of  the  programme  although  there  may  still  be  areas 
where it is relevant, and in particular some activities may be more relevant in the new Member 
States than in the old ones. 

Availability of the report on Europa  

http://ec.europa.eu/energy/evaluations/annual_en.htm 

289

 
 
  
 
 
 
Ex-Ante Evaluation preparing a 
Euratom Council Regulation on Union support for the nuclear decommissioning 
assistance programmes in Bulgaria, Lithuania and Slovakia 

ABB activities: 

32 05 Nuclear energy 

Timing:  

 24/11/2011  

Background, scope and focus  

The  three  countries,  Lithuania,  Slovakia  and  Bulgaria,  operated  old  soviet  design  nuclear 
reactors  which  the  international  community  concluded  could  not  be  upgraded  to  meet  the 
minimum  required  safety  standards,  at  an  economically  acceptable  cost.  As  such  these  plants 
required to be closed earlier than their foreseen end-of-lifetime dates. 

In  the  context  of  the  negotiations  for  accession  to  the  European  Union,  the  three  candidate 
countries took the commitment to close and subsequently decommission these nuclear reactors 
by a commonly agreed date. This early closure represented an exceptional financial burden for 
the  Member  States  which  was  not  commensurate  with  the  economic  strength  of  the  countries 
concerned.  In  recognition  of  this  fact  and  as  act  of  solidarity  the  European  Union  committed 
itself  to  continue  to  provide  adequate  additional  financial  assistance  for  decommissioning  of 
these reactors. The closure commitment of the three Member States as well as the commitment 
of the EU to provide financial EU support was foreseen in the corresponding Accession Treaties. 

This  document  covers  the  assessment  of  the  proposal  for  further  financial  EU  support  to 
Lithuania, Slovakia and Bulgaria, originating in the corresponding Accession Treaties. It fulfils 
both the requirements for an ex ante evaluation and an impact assessment.  

Summary of findings and conclusions  

In  order  to  allow  for  safe  decommissioning,  adequate  financial  resources  should  be  available 
when required1. Although all three Member States have established national funds to set aside 
financial resources for decommissioning, those resources are for historical reasons insufficient. 

1  Commission recommendation on the management of financial resources for the decommissioning 

of nuclear installations, spent fuel and radioactive waste. OJ L 330, 28.11.2006, p.31 

290

 
 
 
 
 
 
 
 
 
 
                                                 
Therefore the immediate problem to be addressed is the funding shortfall to progress with safe 
decommissioning  of  the  nuclear  power  plants,  to  ensure  that  the  closure  becomes  irreversible1 
and  at  the  same  time  stimulate  the  beneficiary  Member  States  to  gradually  take  over  the 
responsibility  with  respect  to  the  full  financial  cover  and  ownership.  Public  intervention  is 
required to assist the Member States with additional financial EU support. 

Funding shortfall would jeopardize the safe maintenance of the shut down reactors until they are 
completely defueled and the seamless continuation of safe decommissioning, because of the risk 
that  further  decommissioning  steps  would  be  postponed  to  an  undefined  date,  awaiting  the 
availability of funds and transferring this liability and responsibility to future generations. It also 
bears  the  risk  of  reopening  of  the  nuclear  power  plants.  At  most  of  the  reactor  units  no  major 
irreversible  dismantling  steps  have  been  implemented.  In  case  of  an  incident  or  accident  this 
would inevitably also lead to an environmental degradation. 

The necessity for the EU intervention is the fact that adequate funds required for continuing safe 
decommissioning  cannot  be  made  available  in  due  time  through  the  respective  national  funds. 
Unlike other Member States in a similar situation but without being confronted to early closure 
of their plants, it was not possible for them to accumulate sufficient funds from operation of the 
plants. 

It is in the interests of the European Union to ensure that the concerned reactors remain closed 
and that they are defueled and dismantled in order to reduce the risk of negative consequences 
for the EU citizen and for the environment. The EU added value lies in the support of measures 
targeted  to  reach  an  irreversible  state  within  the  decommissioning  process  of  the  concerned 
nuclear reactor units, in accordance with their respective decommissioning plans, while keeping 
the highest level of safety. This will contribute to provide substantial and durable support for the 
health  of  workers  and  the  general  public,  preventing  environmental  degradation  and  providing 
for real progress in nuclear safety and security. 

No further EU support under the baseline option would stop the decommissioning programmes 
and consequently jeopardize nuclear safety. The business as usual option would result in a much 
higher financial EU support with reduced added value. Further support to projects in the energy 
sector would lead to a distortion of competition and the continuing high level of financial Union 
support would not be a sufficient incentive for the Member States to take over the full financial 
responsibility for the completion of decommissioning. 

EU  partial  financing  for  decommissioning  only  (option  3)  is  considered  the  most  appropriate 
solution  following  a  clear  expressed  political  will.  This  option  maximises  the  EU  added  value 
and  supports  the  transition  towards  full  Member  State  funding  of  the  safe  completion  of 
decommissioning beyond the next multiannual financial framework. Option 3 provides for real 
improvement in nuclear safety and will help to achieve timely real physical progress in defueling 
and decommissioning and ensure that the closure is irreversible. 

Option 3 combined with the implementation mechanisms B (for Bulgaria and Slovakia) and C 
(for Lithuania) will provide for a continued strengthening of the Commission’s management for 
the  effective,  efficient  and  economical  use  of  EU  funds.  The  specific  objectives  being  aligned 
with  the  proposed  budget  and  based  on  revised/updated  decommissioning  plans  (needs 
expressed  by  the  Member  States)  together  with  the  meaningful  performance  indicators  are  the 
basis for achieving the expected benefits from the further financial EU support after 2013. This 
approach is in full accordance with the recommendations from the European Court of Auditors. 

1  Closure to become irreversible means that decommissioning has progressed so far on a technical 
level,  that  it  would  economically  no  longer  be  advantageous  to  consider  the  re-opening  of  the 
concerned reactor units. 

291

 
 
 
                                                                                                                                            
Availability of the report on Europa  

http://ec.europa.eu/energy/nuclear/decommissioning/decommissioning_en.htm 

http://ec.europa.eu/energy/nuclear/decommissioning/doc/sec_2011_1387.pdf 

292

 
 
 
 
 
 
SECTION 4: REFERENCES TO REPORTING ON PERFORMANCE IN THE ACTIVITY STATEMENTS 
(AS) TO THE DRAFT BUDGET 2013 

- The AAR 2011 (PART 1): http://ec.europa.eu/atwork/synthesis/aar/index_en.htm  
- The Activity statement PDB 2013: 
http://ec.europa.eu/budget/library/biblio/documents/2013/DB2013/DB2013_WD_I_%20Activity_S
tatements.pdf  

293