COMMISSION STAFF WORKING DOCUMENTAccompanying the documentREPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT ANDTHE COUNCILon the evaluation of the Union's finances based on the results achieved
Date: 2013-06-26
Institutions: European Commission
Subjects: EU expenditure, financial management, multiannual financial framework

EUROPEAN 
COMMISSION 

Brussels, 26.06.2013 
SWD(2013) 228 final 

COMMISSION STAFF WORKING DOCUMENT 

Accompanying the document 

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND 
THE COUNCIL 

on the evaluation of the Union's finances based on the results achieved 

{COM(2013) 461 final} 
{SWD(2013) 229 final}

EN

 
 
 
 
 
 
Table of contents 
SECTION 1: OVERVIEW OF EVALUATIONS COMPLETED IN 2012.......................................3 
SECTION 2: RETROSPECTIVE EVALUATIONS ..................................................................29 
ABB 01 – Economic and Financial Affairs .............................................................30 
ABB 02 – Enterprise................................................................................................37 
ABB 03 – Competition ............................................................................................48 
ABB 04 – Employment and social affairs ...............................................................50 
ABB 05 – Agriculture and Rural Development.......................................................71 
ABB 06 – Mobility and Transport ...........................................................................80 
ABB 08 – Research..................................................................................................87 
ABB 09 – Information Society and Media ............................................................100 
ABB 11 – Maritime Affairs and Fisheries.............................................................114 
ABB 13 – Regional Policy.....................................................................................125 
ABB 15 – Education and Culture ..........................................................................133 
ABB 16 – Communication.....................................................................................142 
ABB 17 – Health and Consumer Protection ..........................................................155 
ABB 18 – Home affairs .........................................................................................162 
ABB 19 - External Relations .................................................................................164 
ABB 20 – Trade .....................................................................................................178 
ABB 21 – Development and Relations with African, Caribbean and Pacific (ACP) 
States ......................................................................................................................188 
ABB 22 – Enlargement..........................................................................................205 
ABB 23 – Humanitarian Aid .................................................................................211 
ABB 26 – Commission's administration................................................................224 
ABB 29 – Statistics................................................................................................228 
ABB 32 – Energy...................................................................................................234 

SECTION 3: REFERENCES TO REPORTING ON PERFORMANCE IN THE ANNUAL ACTIVITY 
REPORTS 2012 ............................................................................................................236 

EN 2 

Error! Unknown document property name.  EN 

 
 
 
SECTION 1: OVERVIEW OF EVALUATIONS COMPLETED IN 2012 

This overview lists the titles of the available evaluations in order of Activity Based Budgeting 
Activity within each Policy Area.  

Of  all  evaluations,  61%  concerned  expenditure  programmes;  10%  covered  regulatory 
activities,  linked  to  regulations,  directives,  regulatory  policy  communications  and  'soft  law'; 
18%  concerned  communication,  information  and  coordination  activities;  6%  internal 
administrative processes of the EU Institutions with the remaining 5% in other categories. 

EN 3 

Error! Unknown document property name.  EN 

 
 
ABB-ACTIVITY  

N° 

HEADING 

01 03 

International 
economic and 
financial affairs 

01 – Economic and Financial Affairs 

TITLE OF EVALUATION 

• 

 Ex-post evaluation of Macro Financial Assistance (MFA) operations to 

Lebanon 

•  Ex-post evaluation of Macro Financial Assistance (MFA) operations to 

Georgia 

•  Ex-post evaluation of Exceptional Financial Assistance (EFA) 

operations to Kosovo 

01 04 

Financial operations 
and instruments  

•  Evaluation of EIF own resources activity 

ABB-ACTIVITY  

N° 

HEADING 

02 – Enterprise  

TITLE OF EVALUATION 

EN 4 

Error! Unknown document property name.  EN 

 
 
 
 
•  Error! Reference source not found. 

Evaluation of Member State Policies to Facilitate Access to Finance for S

1) ABB activities concerned: 02 02 - 

2) Timing: June 2012, (duration 2010-2012) 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

The  objective  of  the  study  is  to  evaluate  the  efficiency,  utility,  sustaina
effectiveness and impact of existing Member States programmes to facilitate the a
to finance of SMEs. This is important to provide recommendations on which (ty
programmes work best and which elements could also be used in other countries.

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study 

In the context of the current crisis, it is increasingly difficult and expensive for SM
access private sources of finance. Member States and the European Commission
taken comprehensive measures to address the current tensions in financial marke
to safeguard the flow of credit to the real economy. To facilitate access to financ
study provides information on how to access over EUR 50 billion of public finan
the  27  Member  States.  The  study  presents  over  120  national  or  regional  fina
programmes  and  provides  key  information  helping  SMEs  to  apply  for  the  dif
programmes in terms of characteristics, terms, conditions and contact information.

The  study  also  provided  an  evaluation  of  public  financing  programmes  in  5  Me
States (Germany, France, the UK, Poland and Sweden) to exchange good practic
assess  which  programmes  work  best  and  could  be  used  in  other  countries.  The 
stresses  the  importance  of  having  efficient  and  flexible  financing  programm
correspond to changing market conditions and to be well integrated in the local bu
environment.  In  terms  of  operation,  public  programmes  need  quick  and  eff
decision  marking  processes  and  a  strong  collaboration  with  the  private  sector.  P
financing programmes should also be aligned with business advice services.  

6) Availability of the report on Europa:  
 http://ec.europa.eu/enterprise/policies/finance/guide-to-funding/indirect-
funding/files/evaluation-of-national-financing-programmes-2012_en.pdf 

• 

• 

Evaluation of EU Member State Business Angel Markets and Policies

1) ABB activities concerned: 02 02  

2) Timing: October 2012.  

02 02 

Competitiveness, 
industrial policy, 
innovation and 
entrepreneurship  

EN 5 

Error! Unknown document property name.  EN 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

Business Angels play an important role in the economy and in many countries co

 
 
 
 
 
 
•  Evaluation of the European Chemical Agency 

•  Error! Reference source not found.Error! Reference source not 
found. 

• 

Evaluation of the Impact of European Space Policy on European Spac
Manufacturing and the Services Industry 

1) ABB activities concerned: 02 03  

2) Timing: August 2012 (Period covered by the report: 2007-2011). 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

The evaluation aims were to: 

to  strengthening  European  space 

−  Assess  the  impacts  to  date  of  the  European  Space  Policy  adopted  in  2
strengthening space industrial competitiveness (in manufacturing and services); 
−  Assess the main challenges ahead for the European Space Industry in manufa
and services in maintaining and strengthening space industrial competitiveness; 
−  Review  the  strategic  importance  and  specific  characteristics  of  the  European
Industry to assess whether a dedicated European Space Industrial Policy is appropri
−  Develop quantitative and qualitative indicators to measure the future contribu
the  ESP 
in 
manufacturing  and  services  and  in  wider  sectors  where  space-based  technologi
applications are a key competitiveness driver; 
− 
competitiveness issues facing the upstream and downstream space sectors; 
− 
Inform  the  preparation  of  a  European  Space  Industrial  Policy  (and  p
accompanying Regulations) in accordance with the Lisbon Treaty provisions (TFEU
173 and 189); 
−  Analyse the potential role of EU regulation in future as an enabler in facilitat
development of new and emerging markets in space services, such as in the distribu
high-resolution satellite data and its commercial distribution.  

Identify  current  and  possible  future  developments  in  the  space  industry 

industrial  competitiveness 

5) Summary of findings and conclusions of the evaluation or evaluation related  
    study: 

The overall conclusions were that: 

−  The  adoption  of  the  ESP  in  2007  has  provided  an  important  framework  t
which  critical  precursor  steps  have  been  taken  towards  the  development  of  the
European  Space  Industrial  Policy.  It  is  highly  doubtful  without  the  existence  of  th
whether it would have been possible to launch a dedicated industrial policy for th
sector. 
−  Through  the  frame  of  the  ESP,  progress  has  also  been  made  in  strengthening
governance even if some of the achievements predate the ESP’s adoption, such as th
EC-ESA Framework Agreement and the holding of regular Space Council meetings
− 
Following the adoption of Article 189 of the TFEU, considerable political an
progress has been made. This gives the EU joint competence, together with ESA 
member states, to pursue a formal European Space Industrial Policy. 
− 

Following  the  Lisbon  Treaty’s  adoption,  space  is  now  formally  an  EU  polic

02 03 

Internal market for 
goods and sectoral 
policies 

EN 6 

Error! Unknown document property name.  EN 

 
 
EN 7 

Error! Unknown document property name.  EN 

 
 
ABB ACTIVITY  

N° 

HEADING 

03 
AWBL 
02 

03 
AWBL-
03 

Policy coordination, 
European competition 
network and 
international 
cooperation 

Control of State aid 

03 – Competition 

TITLE OF EVALUATION 

•  Study on co(re)insurance and on ad-hoc co(re)insurance agreements on 
the subscription market 

•  Study on the Implementation of the existing Broadband Guidelines 

• 

ABB ACTIVITY  

N° 

HEADING 

04 – Employment and social affairs 

TITLE OF EVALUATION 

•  Evaluation of the reaction of the ESF to the economic and financial 
crisis 

•  Evaluation of the ESF support to Lifelong Learning 

04 02 

European Social Fund 

•  ESF Expert Evaluation Network: Access to Employment and Social 
Inclusion 

•  Analysis of costs and benefits of active vs passive labour market 
measures 

• 

04 03 

Working in Europe - 
Social dialogue and 
mobility 

•  ENEA preparatory action on active ageing and mobility of elderly 
people 

•  Evaluation of Public Employment Services performance measurement 
systems and corresponding recommendations on geographical labour 
mobility indicators 

• 

EN 8 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
•  Development of a methodology for the systematic evaluation of Health 
and Safety at Work Directives and evaluation of the practical 
implementation of Directive 89/654/EEC "Work Places" testing the new 
evaluation methodology - Legislation evaluation 

•  PROGRESS mid-term evaluation 

•  Evaluation of flexicurity 2007-2010 

• 

•  Mid-term evaluation of the European globalisation fund (EGF) 

04 04 

Employment, social 
solidarity and gender 
equality 

04 05 

European 
Globalisation 
Adjustment Fund 
(EGF) 

05 – Agriculture and Rural Development  

ABB ACTIVITY  

N° 

HEADING 

TITLE OF EVALUATION 

•  Evaluation of the Common Agricultural Policy measures applied to the 
cereals sector 

05 02 

Interventions in 
agricultural markets 

•  Evaluation of the Common Agricultural Policy measures applicable to 
the wine sector 

•  Evaluation of the European School Fruit Scheme 

• 

•  Evaluation of the Common Agricultural Policy measures applied to the 
cereals sector 

05 03 

Direct aids 

•  Evaluation of the Common Agricultural Policy measures applicable to 
the wine sector 

05 04 

Rural development 

• 

•  Ex-post evaluation of Rural Development Programmes 2000-2006 

•  Synthesis of mid-term evaluations of rural development programmes 
2007-2013 

•  Ex-post evaluation of the EU Forest Action Plan 

• 

EN 9 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

06 02 

Inland, air and 
maritime transport 

06 – Mobility and Transport  

TITLE OF EVALUATION 

•  Evaluation of Regulation 1371/2007 on rail passengers’ rights and 
obligations 

• 

 Mid-term evaluation of TEN-TEA 

• 
(The Trans-European Transport Network Executive Agency) 

06 03 

Trans-European 
networks 

•  Ex-post evaluation of technical follow-up of TEN-T funded ERTMS 
projects carried out under Service Framework Contract TREN/E2/322-2008 
(Lots 1, 2 and 3) 

• 

ABB ACTIVITY  

N° 

HEADING 

07 03 

Development and 
implementation of 
Union environmental 
policy and legislation 

07 – Environment  

EVALUATION PROJECTS 

•  Final evaluation of LIFE+ 

ABB ACTIVITY 

N° 

HEADING 

08 – Research  

TITLE OF EVALUATION 

08 02 

Cooperation — 
Health 

•  Review of S&T cooperation between the European Union and India 

•  Review of S&T cooperation between the European Union and Brazil 

•  SME Participation in FP7 - Spring report2011 Monitoring Report of FP7 

• 

EN 10 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
08 03 

08 04 

08 13 

08 19 

Cooperation — 
Food, agriculture and 
fisheries, and 
biotechnology 

•  Overview of International Science, Technology and Innovation 
cooperation between Member States and countries outside the EU and the 
development of a future monitoring mechanism 

International cooperation in Science, Technology and Innovation: 

• 
Strategies for a Changing World 

• 

Cooperation — 
Nanosciences, 
nanotechnologies, 
materials and new 
production 
technologies 

Capacities — 
Research for the 
benefit of small and 
medium-sized 
enterprises (SMEs) 

Capacities — 
Support for coherent 
development of 
research policies 

•  Strategy definition and road mapping for industrial technologies to 
address grand challenges 

• 

• 

 SME Participation in FP7 - Spring report 

 Review of main activities and deliverables of ERAWATCH 

• 

• 

• 
 Review of main activities and deliverables of IRMA Administrative 
Arrangements with IPTS (Industrial Research Monitoring and Analysis) 

• 

08 03  

AWBL 

European Research 
Area Development 

Interim Evaluation of Art. 185 European Metrology Research 

• 
Programme (EMRP) 

ABB ACTIVITY  

N° 

HEADING 

09 – Information Society and Media 

TITLE OF EVALUATION 

EN 11 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
09 02 

Regulatory 
framework for the 
Digital Agenda 

•  Study on the evaluation of BEREC and the BEREC Office - SMART 
2011/0009 

Inventory and Review of Spectrum Use: Assessment of the EU potential 

• 
for improving spectrum efficiency - SMART 2011/0016 - PC 

Implementation of the DAE - Action of the DAE - Actions under the 

• 
responsibility of member states - SMART 2011/0023 – IAV 

• 

09 03 

Information and 
communication 
technologies take-up 

•  SMART 2010/0048: assessment of the economic and social benefits 
of digitisation of cultural heritage  

09 04 

Cooperation - 
Information and 
communication 
technologies (ICTs) 

•  Report on Communication and Recommendation on mobilising 
Information and Communication Technologies to facilitate the transition to an 
energy-efficient, low-carbon economy COM (2009) 111; C(2009) 7604 

•  Future impact of ENIAC and ARTEMIS - SMART 2012/0050 - IAV 

Interim Assessment of the Future Internet Public-Private Partnership 

• 

• 

•  Study into the impact of FP6 IST 

09 05 

Capacities — 
Research 
infrastructures 

•  Development of impact measures for e-Infrastructures - SMART 
2010/0051 

• 

ABB ACTIVITY  

N° 

HEADING 

11 02 

Fisheries markets 

11 03 

International 
fisheries and law of 
the sea 

11 – Maritime Affairs and Fisheries  

TITLE OF EVALUATION 

 Ex-post evaluation on the implementation of the compensation regime 

• 
the additional costs incurred in the marketing of certain fishery products 
from the outermost regions the Azores, Madeira, the Canary Islands, French 
Guiana and Réunion 

• 
 Ex-post evaluation of the current Protocol to the Fisheries Partnership 
Agreement between the European Union and Republic of Kiribati, and ex-
ante evaluation including an analysis of the impact of the future Protocol on 
sustainability 

•  Ex-post evaluation of the current Protocol to the Fisheries Partnership 
Agreement between the European Union and Solomon Islands, and ex-ante 
evaluation including an analysis of the impact of the future Protocol on 

EN 12 

Error! Unknown document property name.  EN 

 
 
 
 
 
sustainability 

•  Evaluation ex-post du protocole de l'accord de partenariat dans le 
domaine de la pêche entre l'Union européenne et la Côte-d'Ivoire 

11 06 

European Fisheries 
Fund (EFF) 

•  Synthesis of the 26 national evaluation reports 

• 

Interim evaluation of the European Fisheries Fund (2007-2013) 

• 

11 09 

Maritime policy 

Interim evaluation of marine knowledge-related preparatory actions and 

• 
pilot projects. (internal evaluation)  

• 

EN 13 

Error! Unknown document property name.  EN 

 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

13 03 

European Regional 
Development Fund 
and other regional 
operations 

13 – Regional Policy  

TITLE OF EVALUATION 

•  Ex post evaluation of projects co-financed by ERDF and Cohesion Fund 
in the period 1994-1999 

•  The use of the ERDF to support Financial engineering instruments 
(FEIs) 

Jaspers Evaluation 

• 

• 

13 04 

Cohesion Fund 

•  Ex post evaluation of the Cohesion Fund (Incl. former ISPA) 2000 - 
2006 

•  Country reports on the performance of Cohesion Policy 2007 - 2013 

• 

ABB ACTIVITY  

N° 

HEADING 

15 02 

Lifelong learning, 
including 
multilingualism 

15 04 

15 07 

Developing cultural 
and audiovisual 
cooperation in 
Europe 

People — 
Programme for the 
mobility of 
researchers 

15 – Education and Culture   

TITLE OF EVALUATION 

• 

 Interim evaluation of Erasmus Mundus II (2009 – 2013) 

•  External evaluation of the European Training Foundation (ETF) 

• 

•  Ex-post Evaluation of 2011 European Capitals 

•  of Culture 

•  Ex-post evaluation of the Preparatory Action Media International 2008-
2010 

• 

•  FP7 Marie Curie Life-long Training and Career Development 
Evaluation: Individual Fellowships and Co-funding Mechanism 

• 

EN 14 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

16 – Communication 

TITLE OF EVALUATION 

16 02 

Communication and 
the media 

• 

Interim evaluation of PressEurop 

16 03 

"Going Local" 
communication 

•  Mid-term evaluation of Europe direct information centres (2009-2012) 

•  Evaluation of the management partnership's actions within the scope of 
the 2008-2010 communication plans in Italy 

•  2009-2012 communication plans in Sweden 

•  Evaluation of the management partnership's actions within the scope of 
the 2009-2012 communication plans in Finland 

•  Evaluation of the management partnership's actions within the scope of 
the 2009-2011 communication plans in Malta 

•  Evaluation of the management partnership's actions within the scope of 
the 2009-2012 communication plans in Greece 

•  Evaluation of the management partnership's actions within the scope of 
the 2009-2012 communication plans in Lithuania 

•  Evaluation of the management partnership's actions within the scope of 
the 2008-2010 communication plans in Belgium 

•  Evaluation des opérations de communication menées dans le cadre du 
partenariat de gestion 2008-2010 en France 

•  Evaluation of the management partnership's actions within the scope of 
the 2009-2012 communication plans in Poland 

• 

16 05 

Fostering European 
citizenship 

•  Evaluation of the European year of volunteering 2011 

• 

ABB ACTIVITY  

N° 

HEADING 

17 – Health and Consumer Protection   

TITLE OF EVALUATION 

EN 15 

Error! Unknown document property name.  EN 

 
 
 
 
 
• 

External evaluation of the Consumer Protection Cooperation Regul

1)  ABB activities concerned: 17 02 

2)  Timing: 17 December 2012, (Period covered by the report: 2007 – 2011)  

3)  Budget: not applicable 

4)  Background, scope and focus:  

At  the  end  of  2010  the  Commission  initiated  an  assessment  of  the  enforcemen
mechanisms established by the CPC Regulation. This process was inserted as a new r
the  CPC  Regulation  in  the  framework  of  an  amendment  to  the  legal  text  (Regula
954/2011).  The evaluation was part of this process. The main focus was put on the C
its  implementation  in  the  Member  States  and  the  infrastructural  and  budgetary  supp
evaluation  addressed  both  (1)  the  cooperation  arrangements  in  the  framework  of  the
established  by  the  CPC  Regulation  in  the  context  of  the  enforcement  of  the  legislati
Annex to the Regulation and (2) the broader forms of cooperation established by the C
under its Chapter IV. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation 
Relevance of the CPC Regulation:  

The  report  concluded  that  the  rationale  for  intervention  is  as  valid  and  appropriate  t
when  the  CPC  Regulation  was  introduced  in  2004.  It  may  even  be  said  that  the  C
framework  is  even  more  needed  than  before  given  developments  in  technology  an
patterns of households (i.e. in digital and online purchasing) and the complexity of tr
aspects such as advertising, bundling of products and trading internationally have rein
to provide tools to deter rogue traders and to enforce the law. Even though, the curren
the  CPC  Regulation  are  appropriate  and  relevant,  none  of  the  specific  objective
Regulation have been fully achieved. Therefore it would be appropriate to improve th
text within the CPC Regulation in order to ensure more uniform understanding among n

Scope of the CPC Regulation: The analysis concluded that the current scope of the C
annex is broadly correct in terms of the sensitivity of the legislation to public enforcem
collective  consumer  interest,  cross-border  relevance  and  consistency.  To  maintain 
consumer  protection  the  evaluation  report  suggested  the  possible  inclusion  of  the  R
Rights  Regulation,  the  Roaming  Regulation  and  the  Mortgage  Credit  Directive  in  th
CPC Regulation. Also it was recommended to examine further the possible inclusion o
provisions within certain legislative acts, such as the Air Services Regulation. 

       Legal barriers to cooperation under the CPC Regulation  

Applicable substantive law 

The  CPC  Regulation  does  not  regulate  which  applicable  substantive  law  app
assistance  requests.  This  issue  has  been  raised  as  a  source  of  difficulties 
effectiveness  of  cross-border  enforcement  cooperation.  The  evaluation  offe
recommendations to alleviate constraints arising due to differences in substantive l

EN 16 

Error! Unknown document property name.  EN 

•  Provide guidance on applicable law in relation to the handling of mutual assi
under  the  CPC  Regulation  in  order  to  promote  better  awareness  and  compli
consumer legislation; 

•  Foster a common understanding in relation to the application of EU consumer
through  guidance,  on-line  legal  resources,  and  interactive  forums  to  also  fa
knowledge exchange;

 
 
 
17 03 

Public health 

•  Assessment of the added value of the EU alcohol strategy and its 
implementation to enhancing action, cooperation and coordination to reduce 
alcohol related harm 

• 

EN 17 

Error! Unknown document property name.  EN 

 
 
• 

• 

Evaluation of the EU rapid response network, crisis management and com

capacity regarding certain transmissible animal diseases 

1) 

ABB activities concerned: 17 04 

2)  Timing:    1  August  2012  (duration  of  the  evaluation  09.2012-07.2012,  Period 
report: 1998 – 2009) 

3)  Budget: At the EU level the activity has no directly allocated budget. The releva
the  Member  States  (MS)  are  covered  by  national  arrangements.  Following  outbr
diseases on the territory of MS, specific costs directly related to the emergency me
them (e.g. compensating to owners for the slaughter, destruction of animals, costs 
cleaning, disinfection of holdings and equipment, cost of supply of the vaccine e
financial  contributions  met  by  the  EU,  adopted  as  Commission  Decisions  in 
Council Decision 2009/470/EC. 

4)  Background, scope and focus:   The  evaluation  was  carried  out  in  the  context
Health  Strategy  for  the  European  Union  "Prevention  is  better  than  cure"
Communication  from  September  2007  COM(2007)539  final)  and  was  required 
Action  Plan  (Commission  Communication  from  September  2008-COM  (2008) 
focus was put on assessing the current legislative and non-legislative environmen
state  of  preparedness  and  the  capacity  of  the  EU  rapid  response  network,  w
encompasses  the  Commission  Services,  MS  (especially  the  veterinary  Compete
and  stakeholders,  such  as  private  sector  veterinarians  and  the  economic  operat
During its work, the contractor involved all Member States and major stakeholders

5) 
Summary  of  findings  and  conclusions  of  the  evaluation  or  evalu
     study: Effectiveness, efficiency and added value of the EU rapid response network: 

The EU rapid response system is considered generally to work effectively and ef
have improved very significantly over the last decade.  The availability of well de
and  up  to  date  Contingency  Plans  (CPs)  as  an  indicator  of  preparedness  can  h
emergency  from  becoming  a  crisis.  The  effectiveness  of  the  response  also 
cooperation  and  coordination  within  the  overall  rapid  response  network,  includin
Commission and MS, regular and timely exchange of information (including scien
and  advice)  between  laboratories  and  with  stakeholders,  and  the  building  and  m
confidence  and  trust  between  all  parties.  The  evolution  of  the  EU  animal  heal
indicates  a  decrease  in  the  amount  of  EU  co-funding  for  emergency  veterinary 
some EUR65 million in 2000 to EUR30 million in 2011. Over the last five years E
has  averaged  EUR37  million,  well  below  the  average  over  the  whole  period  (E
2000-2011 in terms of outturn payments, i.e. the sum of credits generated by a M
year). This points to the more efficient use of funds to achieve longer term objecti
reinstating  of  disease  free  status  for  major  diseases  in  the  EU.  The  compre
legislation now in place (including Contingency Plans and the EU emergency ne
components) can be considered as a valuable shield against traditional contagious 
and appears to be quite effective in terms of triggering the relevant steps and cont
fight against emerging diseases or new "profiles" of known diseases. As a result o
evaluation period, out of a significant number of outbreaks, relatively few have d
crisis. On the basis of the criteria of financial cost and economic impact, the follow
identified: Classical Swine Fever (1997 DE); Avian Influenza-AI (1999/2000 IT);
H5N1 Influenza (2005-06); Foot and Mouth Disease (2001, UK); Bovine Tubercu

17 04 

Food and feed safety, 
animal health, 
animal welfare and 
plant health 

EN 18 

Error! Unknown document property name.  EN 

 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

18 05 

Security and 
safeguarding 
liberties 

ABB ACTIVITY  

N° 

HEADING 

19 04 

European Instrument 
for Democracy and 
Human Rights 
(EIDHR) 

18 – Home affairs 

TITLE OF EVALUATION 

•  Evaluation on the European Union Crime Prevention Network 

• 

19 – External Relations 

TITLE OF EVALUATION 

•  Thematic evaluation of the European Commission support to respect of 
Human Rights and Fundamental Freedoms (including solidarity with victims 
of repression) 

• 

EN 19 

Error! Unknown document property name.  EN 

 
 
 
 
 
• 

Evaluation of the EU cooperation with Colombia 

1) ABB activities: 19 09:  

2) Timing: October 2012, (period covered by the report: 2002-2011) 

3) Budget :Disbursements over the evaluation period: EUR196.61 million 

4) Background, scope and focus: 

The evaluation provides an overall independent assessment of the Commission’s pa
current cooperation with Colombia over the period 2002-2011 and identifies key les
order  to  improve  the  current  and  future  strategies  and  programmes  of  the  Comm
particularly considering the new development policy set in the “Agenda for Change”

The evaluation focuses on the following areas of investigation: Peace, Stability and 
economic  Development;  Rule  of  Law  and  Justice;  Human  Rights  and  Victims 
Armed  Conflict;  Good  Governance;  Productivity  and  Competitiveness;  Environ
Sustainability  and  the  Linkage  between  Relief,  Rehabilitation  and  Development. 
analyses the instruments and aid modalities made available by the Commission, as w
the  coherence,  coordination  and  complementarity  ("3Cs")  of  EU  cooperation  a
European added value. 

5) Summary of findings and conclusions of the evaluation  

The  EU  cooperation  strategy  with  Colombia  from  2002  to  2010  is  characterized 
core objective of peace building. In general terms, the EU strategy was coherent w
Colombian policies in relation to peace-building by contributing to address the root 
for conflict. 

Key conclusions: 

•  Although  the  EU  strategy  has  supported  transition  from  relief  to  rehabilitatio
highly  sensitive  conflict  situation,  it  did  not  sufficiently  succeed  to  link  the  lon
objective of development to these medium term objectives. Determining factors w
lack  of  a  comprehensive  and  structured  strategic  framework  to  mainstream
governance principles as well as a non-adaptive capacity development scheme, not
regards  the  territorial  development.  Weaknesses  have  also  been  identified  as  r
coordination  and  complementarity  with  other  donors.  The  distinctive  policy  strate
active EU Member States hampered visibility of EU added value. 

Conflict prevention and resolution: The EU contributed in strengthening civil s
• 
creating  social  networks  and  promoting  productive  alternatives  and  reactivation  in
most affected by conflict. However, sustainability remains a major concern. 

Rule of law, justice and human rights: By supporting the new Penal Oral Accu
• 
System, the EU has helped to improve the application of the Justice and Peace Law
provided  political  and  technical  support  for  the  protection  of  victims'  rights  a
promotion  of  human  rights.  However,  impunity  in  relation  to  human  rights  vio
remains  a  key  problem  and  specific  improvements  can  not  conclusively  be  attribu
EU actions. 

19 09 

Relations with Latin 
America 

EN 20 

Error! Unknown document property name.  EN 

Trade  and  competitiveness:  By  supporting  the  Local  Economic  Develo
• 
approach, the EU has contributed to create the basis for development and competiti
notably  in  rural  areas.  The  next  step  needs  to  facilitate  participation  in  nation
international markets. 

•

Environmental sustainability: Albeit not being a priority in its strategy and no

 
 
 
19 10 

19 11 

Relations with Asia, 
Central Asia and 
Middle East (Iraq, 
Iran, Yemen) 

Policy strategy and 
coordination for 
‘External relations’ 
policy area 

•  Evaluation of the EU cooperation with Nepal 

• 

•  Evaluation of Visibility of EU external action 

•  Thematic evaluation of EC support to Decentralisation processes 

• 

ABB ACTIVITY  

N° 

HEADING 

20 – Trade  

TITLE OF EVALUATION 

•  Evaluation of the economic impact of the trade pillar  

•  of the EU-Chile association agreement 

•  Evaluation of the EU’s trade defence instruments (TDIs) 

•  Evaluation of the Market Access Partnership (MAP) 

20 02 

Trade policy 

•  Trade sustainability impact assessment (Trade SIA) in support of 

•  EU negotiations for deep and comprehensive free trade agreements 
(DCFTAs) 

•  with Georgia and the Republic of Moldova 

• 

21 – Development and Relations with African, Caribbean and Pacific (ACP) States  

ABB ACTIVITY  

N° 

HEADING 

TITLE OF EVALUATION 

21 02 

Food security 

•  Final evaluation of the EU Food facility  

21 05 

Human and social 
development 

•  Evaluation of the EU Support to the Health sector  

EN 21 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
21 06 

21 07 

21 08 

Geographical 
cooperation with 
African, Caribbean 
and Pacific (ACP) 
States 

Development 
cooperation actions 
and ad hoc 
programmes 

Policy strategy and 
coordination for 
"Development and 
relations with ACP 
States" policy area 

ABB ACTIVITY  

N° 

HEADING 

22 02 

Enlargement process 
and strategy 

•  Evaluation of the EU cooperation with the Caribbean Region Error! 
Reference source not found.Evaluation of the EU cooperation with the 
Republic of Congo Error! Reference source not found.Evaluation of the 
EU cooperation with Djibouti Evaluation of the EU cooperation with 
EthiopiaError! Reference source not found.Evaluation of the EU 
cooperation with Jamaica  

•  Evaluation of the EU cooperation with Zambia  

•  Evaluation of the Commission's cooperation with the Council of Europe 
(focused on the joint programmes)  

• 

22 – Enlargement  

TITLE OF EVALUATION 

•  Evaluation of governance, rule of law, judiciary reform and fight against 
corruption and organised crime in the Western Balkans (Lot 2) 

•  Thematic Evaluation of Rule of Law, Judicial Reform and Fight against 
Corruption and Organised Crime in the Western Balkans - Lot 3 

•  Thematic evaluation on judiciary and fundamental rights in Turkey 

•  Thematic Evaluation of EU's Support to Civil Society in the Western 
Balkans  and Turkey 

•  Strategic/interim evaluation of SIGMA programme 

• 

ABB ACTIVITY  

N° 

HEADING 

23 – Humanitarian Aid 

TITLE OF EVALUATION 

EN 22 

Error! Unknown document property name.  EN 

 
 
 
 
 
23 02 

Humanitarian aid 
including aid to 
uprooted people, food 
aid and disaster 
preparedness 

•  Evaluation and review of humanitarian access strategies in DG ECHO 
funded interventions 

•  The evaluation of dg echo's disaster preparedness and disaster risk 
reduction actions in southern Africa & the Indian ocean 

•  Evaluation of the European Commission in Ethiopia 

•  Evaluation of the European Commission's Humanitarian Activities in 
Bangladesh 

•  Evaluation of the European Commission's Humanitarian Activities in 
Colombia 

•  Evaluation and Review of the DG ECHO financed Livelihood 
Interventions in Humanitarian Crises 

•  The evaluation of echo's intervention in the occupied Palestinian 
territory and Lebanon 

•  Evaluation of the European Commission supported humanitarian aid in 
urban settings 

•  Evaluation of the DG ECHO Legal Framework for Funding of 
Humanitarian Actions (FPA 2008) 

• 

ABB ACTIVITY  

N° 

HEADING 

26 01 

26 03 

Administrative 
expenditure of 
‘Commission’s 
administration’ 
policy area 

Services to public 
administrations, 
businesses and 
citizens 

26 – Commission's administration 

TITLE OF EVALUATION 

•  Evaluation of the language training organised in Brussels by DG HR 

• 

• 

• 

Interim evaluation of the ISA programme 

29 – Statistics  

EN 23 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
ABB ACTIVITY  

N° 

HEADING 

TITLE OF EVALUATION 

•  Rolling Review for Regional Statistics 

• 

Rolling Review for GISCO 

1) ABB activities: 29 02  

2) Timing: October 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus: 

GISCO  is  a  Eurostat  service  which  promotes  and  stimulates  the  use  of 
Geographical Information Systems (GIS) within the European Statistical 
System and the Directorates-General of the Commission. Rolling Review 
for  GISCO  is  a  part  of  the  overall  Quality  Assurance  Framework.  This 
review  was  implemented  between  May  2011  and  August  2012,  and 
conducted in four parts: 

• 

Two user satisfaction surveys (for data and maps). 

29 02 

Production of 
statistical 
information  

Two  checklists,  completed  by  the  GIS  staff,  assessing  major  steps 
• 
within  the  provision  of  geographical  reference  data  and  production  of 
maps.  

The results of these assessments are used to formulate recommendations 
for 
these 
improvements. 

improvements  and 

identify  ways  for 

implementing 

to 

5) Summary of findings and conclusions of the evaluation:  

An 
increasing  relevance  of  geographically  oriented  analysis  for 
administrative and political decisions was recognised in the last years. A 
importance  and  even 
great  majority  of  respondents  stressed 
essentiality  of  georeference  data.  Characteristics  such  as  accessibility, 
quality,  clarity  and  comparability  were  generally  assessed  as  good. 
Positive results of user satisfaction are also found with quality of Eurostat 
maps. 

the 

What emerged from the review is the normal tension between the needs 
of the users  of the georeference data and those who buy and supply the 
data. Users want a wider range of geographic data, while GISCO is more 
concerned  with  spending  the  limited  resources  to  buy  data  which  are 
required 
to  support  Eurostat’s  publications  (thematic  maps)  and 
geographical oriented applications of the Commission (projects). 

A  total  of  18  recommendations  were  provided  with  highest  priorities 

EN 24 

Error! Unknown document property name.  EN 

 
 
 
concentrating on the fields of future trend for GISCO responsibilities and 
strengthening geographical data production and dissemination according 
to user demands. 

6) Availability of the report on Europa 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%2
0GISCO%20Executive%20Summary%20FINAL.pdf 

• 

Rolling Review for Harmonised Indices of Consumer Prices and 
related price statistics 

1) ABB activities: 29 02  

2) Timing: April 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4)  Background,  scope  and  focus:  As  part  of  Eurostat’s  quality  assurance 
framework,  a  rolling  review  of  Harmonised  Indices  of  Consumer  Prices 
(HICPs)  and  related  price  statistics  was  undertaken  starting  in  the  late 
Summer  of  2011  to  assess  the  quality  of  HICP  data  and  the  statistical 
processes used to produce them. The survey was conducted in three parts: 

•  User survey. 

• 

• 

Partner survey. 

Self-assessment performed by the staff of Eurostat Unit C5. 

A number of recommendations for improvement were made based on the 
views, comments and concerns of users, partners and HICP staff. 

5) Summary of findings and conclusions of the evaluation: The HICP and 
related  price  statistics  domain  is  a  well-established  and  well  regarded 
statistical  domain  by  users,  partners  and  Eurostat  alike.  Data  quality, 
assessed  under  the  criteria  of  accuracy,  timeliness,  accessibility, 
completeness,  clarity,  comparability  and  coherence  is  rated  as  very 
good or good by users, partners and the staff, with all criteria being of 
importance.    Partners  are  very  positive  about  Eurostat’s  role  in  the 
organisation and functioning of the HICP programme and its direction 
of  change.  Data  treatment,  validation  and  dissemination,  publications, 
data confidentiality and information exchange tools are all very highly 
rated. User responses also point out a good support service provided by 
Eurostat.  This  is  consistent  with  staff  evaluation  of  systems  and 
processes for data production which are well established and work well. 
While  the  rolling  review  assessments  positively  endorse  the  areas 
above,  14  sets  of  recommendation  are  provided  with  highest  priorities 
assigned to: 

•  Compiling  of  detailed  inventories  /  technical  manuals  of  national 
HICP implementation practices and procedures. •  Reviewing 

EN 25 

Error! Unknown document property name.  EN 

 
 
 
compliance monitoring. 

•  Adopting  a  formal  risk  management  approach  for  the  HICP 
production processes at Unit level, coherent with Eurostat’s overall risk 
management strategy. 

• 

• 

Improving to the on-going development of the HICP. 

Improving bilateral communications with key stakeholders 

6) Availability of the report on Europa:  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%2
0HICP%20Executive%20Summary.pdf 

• 

Rolling Review for Postal Statistics 

1) ABB activities: 29 02  

2) Timing: August 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus : 

Rolling  Review  for  Postal  Statistics  is  part  of  the  Quality  Assurance 
Framework  related  to  streamlining  the  existing  quality  activities  in 
Eurostat  and  to  position  them  in  the  wider  framework  of  the  European 
Statistics Code of Practice and Total Quality Management. The focus of 
this review is assessment  of postal statistics which  supports the purpose 
of  Community  policy  in  the  postal  sector.  The  importance  of  postal 
services  both  for  the  economic  prosperity  and  social  well-being  and 
cohesion of the EU make this a priority area for Community action and 
emphasises 
importance  of  postal  statistics.  The  review  was 
implemented during September 2011 to May 2012 and conducted in three 
parts: 

the 

•  User satisfaction survey. 

• 

Partner satisfaction survey. 

•  Assessment  of  the  production  process  by  the  professional  staff 
within Eurostat. 

Results are used to formulate the actions to improve the performance of 
the European Statistical System in the field. 

5) Summary of findings and conclusions of the evaluation:  

The  results  of  user  survey  indicate  there  is  a  major  demand  for  postal 
statistics.  

Users, in general, perceive the quality of postal data as good or adequate. 
timeliness  and 
However,  concerns  were  raised  with  regards 

to 

EN 26 

Error! Unknown document property name.  EN 

 
 
 
confidentiality. 

Partners had a favourable opinion of Eurostat’s work and were also very 
positive about the direction of change in Eurostat’s performance over the 
past 3 years. Concerning data quality, partners had a positive appreciation 
of the Eurostat postal statistics, aside from timeliness. 

A  total  of  14  recommendations  were  provided  as  the  outcome  of  the 
survey.  Importance  was  highlighted  to  the  possible  collection  of 
European  postal  statistics  by  DG  MARKT  and  processes  related  to  the 
transfer as well as improvement of data collection 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%2
0Postal%20Statistics%20Executive%20Summary%20FINAL%20VERSION
_0.pdf 

• 

Rolling Review for Innovation and R&D Statistics 

1) ABB activities: 29 02  

2) Timing: March 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus: Rolling Review for Innovation and R&D 
Statistics  is  a  part  of  the  Quality  Assurance  Framework  aimed  at 
streamlining  the  existing  quality  control  activities  in  Eurostat.  A  review 
was undertaken on Innovation statistics, covering both innovation surveys 
and R&D statistics. 

The  main  components  of  the  review,  which  were  implemented  between 
February and November 2011, were: 

•  User satisfaction survey. 

• 

Partner satisfaction survey. 

Self-assessment checklist implemented by the staff of Eurostat Unit 

• 
G6. 

The survey aimed to identify and give recommendations for improvement 
which have been made available for general publication. 

5) Summary of findings and conclusions of the evaluation : 

The  results  of  user  survey  indicate  that  both  R&D  and  Innovation 
statistics  are  essential  tools  for  majority  of  respondents.  The  data  are 
mainly used for research, policy preparation or trend analysis. The quality 
was considered to be quite good with respect to accuracy, coherence and 
relevance;  however  criteria,  such  as  accessibility,  completeness  and 

EN 27 

Error! Unknown document property name.  EN 

 
 
 
 
comparability  needed  further  enhancements.  Self-assessment  responses 
indicate that information seems to be rather complete concerning both the 
data and metadata. The R&D figures published by Eurostat are regarded 
as  being  accurate  and  the  timeliness  of  dissemination  considered 
satisfactory. There seems to be general partner satisfaction with the way 
the system of R&D and Innovation statistics works within the European 
Statistical  System.  Two  thirds  of  countries  think  that  data  quality  has 
increased as a result of the introduction of the 

Regulation for R&D statistics. However, several suggestions were made 
for simplification of the statistics. 

A total of 8 recommendations were given, with improvement of Eurostat 
internal processes given the highest priority. 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%2
0Innovation%20and%20RD%20Statistics%20Executive%20Summary_0.pdf 

•  Programme for the Modernisation of European Enterprise and Trade 
Statistics (MEETS) 

• 

ABB ACTIVITY  

N° 

HEADING 

32 03 

Trans-European 
networks 

32 – Energy 

TITLE OF EVALUATION 

•  Evaluation of 5 TEN-E completed projects of European interest 

• 

EN 28 

Error! Unknown document property name.  EN 

 
 
 
 
SECTION 2: RETROSPECTIVE EVALUATIONS 

This section provides fact sheets for 130 retrospective evaluations (i.e. interim and ex 
post  evaluations)  and  evaluation-related  studies  (e.g.  focusing  on  monitoring  or 
implementation), with information on their findings and conclusions. 

EN 29 

Error! Unknown document property name.  EN 

 
ABB 01 – Economic and Financial Affairs 

 Evaluation of EIF own resources activity 

1) ABB activities: 01 04 

2) Timing: 19/6/2012 (duration: 19/10/2011-19/6/2012) 

3) Budget (annual): EUR 972,157,195 

4) Background, scope and focus: 

The European Investment Fund (EIF or the Fund) has been set up in 1994 to "provide a 
significant  contribution  to  the  development  of  trans-European  networks  in  the  areas  of 
transport, telecommunications and energy infrastructures and to the development of small 
and  medium-sized  enterprises  (SMEs)".  It  was  intended  to  "stimulate  sustained  and 
balanced growth within the Community". EIF's own resources are used for venture capital 
and  private  equity  investments  as  well  as  for  securitisation  transactions.  The  EU 
participated in the capital increase in order to maintain its 30% shareholding in the Fund to 
support the Fund's continued focus on EU policies. 

The evaluation analyses whether the original objectives of the EU as a member of the EIF 
as  well  as  for  the  participation  in  the  capital  increase  have  been  achieved.  Moreover,  it 
should  evaluate  the  added  value  of  EIF's  own  resources  activity  compared  to  other  EU 
actions supporting innovation and SMEs. 

5) Summary of findings and conclusions of the evaluation:  

The Added Value achieved by EIF Own Resources Activity 

The added value of EIF own resources activity to the European Commission is achieved 
through  the  delivery  of  SME  financing  policy  objectives  from  a  unique  market  position 
and within a ‘self-sufficient’ Financial Envelope. 

Added Value of EU Shareholding 

Whilst  own  resources  activity  provides  a  specific  ‘blended  return’  (reflecting  the  EIF’s 
market orientated approach to delivery of policy impacts), the further added value of EU 
shareholding  in  the  EIF  can  be  identified  in  terms  of  maintaining  focus  on  EU  policy 
objectives; the working relationship with EIB and other shareholders: presence in a ‘post 
crisis world’ and supporting enhanced leverage and policy impacts through rating stability. 

Effectiveness 

The  evaluation  has  demonstrated  the  added  value  of  EIF  own  resources  activity  in 
delivering  European  SME  finance  activity  (risk  capital  and  debt  finance)  including 
investment volumes, development of innovative financial instruments, and the building of 
investment infrastructure and ecosystems based upon partnership and shared knowledge. 

Nevertheless,  the  business  processes  within  the  EIF  that  might  provide  comprehensive 
assessment of market failures across SME finance markets and reporting of policy impacts 
should be developed further to better measure and evaluate own resources policy impact.  

EN 30 

Error! Unknown document property name.  EN 

 
 
 
Efficiency  

EU  shareholding  is  efficient  (the  current  value  of  EU  shareholding  largely  exceeds  the 
cash  contribution  provided  by  the  EU  budget),  but  can  be  enhanced.  Efficiency  includes 
the added value achieved by the limited budgetary commitment currently supporting EIF 
own  resources  activity  undertaken  on  a  ‘blended  returns’  investment  approach  (for 
example, leverage, innovation and market development) and the particular additional value 
achieved by a shareholding approach (for example, enhanced leverage and credibility and 
position amongst financial institutions, markets and partners). 

Overall,  any  calculation  of  policy  benefits  and  impacts  which  might  be  realised  through 
the re-allocation of EU shareholding capital to alternative delivery mechanisms for SME 
financing would need to strongly consider the impact of a reduction in the scale and scope 
of  added  value  of  own  resources  activity  not  only  within  the  remit  of  own  resource 
activities  but,  additionally,  across  the  broader  activities  of  EIF  including,  principally, 
mandate activity. 

Recommendations 

In summary, the EU shareholding in EIF and EIF own resources activity remains relevant 
in pursuit of SME financing policy objectives and generally effective in delivering policy 
impacts  –  notwithstanding  limitations  in  the  EIF’s  ability  to  demonstrate  optimal  policy 
impacts. 

Added  value  and  impact  of  EIF  own  resources  activity  for  the  European  Commission 
comprises  the  value  achieved  by  policy  activity  undertaken  within  a  ‘blended  returns’ 
framework,  activity  supported  by  specific  shareholding  positions  and  additional  benefits 
resulting  from  other  (mandate)  activity  of  the  EIF  which  supports  EU  policy  objectives. 
Given this, other more efficient policy mechanisms are not immediately evident.  

6) Availability of the report on Europa: 

http://ec.europa.eu/economy_finance/evaluation/pdf/final_reprt_eval_eif.pdf 

Ex-post evaluation of Macro Financial Assistance (MFA) operations to 

Lebanon 

1) ABB activities: 01 03  

2) Timing: 7/5/2012 (duration: 6/7/2011-7/5/2012) 

3) Budget: EUR 80 million (total amount of support) 

4) Background, scope and focus 

Macro-Financial  Assistance  (MFA)  is  a  policy-based  financial  instrument  of  untied  and 
undesignated balance-of-payments support to partner third countries. It takes the form of 
medium/long-term loans or grants, or a combination of these, and generally complements 
financing provided in the context of an International Monetary Fund's reform programme. 

Ex-post evaluations focus on assessing the contribution of EU Macro Financial Assistance 

EN 31 

Error! Unknown document property name.  EN 

 
 
 
to macro-economic and structural adjustment of the recipient country. 

The exercise covers three main areas of analysis: 

•  Economic  impact  of  the  MFA  assistance  operation  on  the  economy  of  the  recipient 
country (e.g. GDP growth, Balance of Payments, exchange rates, fiscal balances); with and 
without IMF involvement. 

•  Value  added  of  EU  intervention  (stand-alone  and/or  in  combination  with  IMF 
intervention) provided through the operation. 

• Sustainability of the country’s external position as a result of the assistance. 

5) Summary of findings and conclusions of the evaluation  

Impact on macroeconomic stabilisation 

The  analysis  of  macroeconomic  developments  in  Lebanon  suggests  an  improvement  in 
areas  to  which  the  objectives  of  the  MFA  operation  were  related,  i.e.  (i)  to  support 
Lebanon’s domestic efforts of post-war reconstruction and sustainable economic recovery; 
(ii)  to  alleviate  the  financial  constraints  on  the  implementation  of  the  government’s 
economic  programme;  and  to  support  Lebanon’s  efforts  to  reduce  public  debt  to  a 
sustainable level. Specifically, Lebanon has clearly made substantial progress in post-war 
reconstruction and experienced economic recovery that was sustained over a period of four 
years,  despite  the  particularly  unfavourable  global  economic  environment.  While 
implementation  of  the  government’s  economic  programme  defined  before  the  Paris  III 
conference  has  faced  several  obstacles,  financial  constraints  on  the  implementation  have 
eased  and  were  not  among  major  problematic  issues.  Since  early 2011  the  economy  lost 
momentum  largely  owing  to  rising  domestic  political  uncertainty  and  regional  turmoil. 
Issues  related  to  the  third  identified  macroeconomic  objective  of  supporting  Lebanon’s 
efforts to reduce public debt to a sustainable level are discussed in more detail in the next 
chapter. 

The  conclusion  is  that  the  two  macroeconomic  objectives  of  the  MFA  assistance  were 
largely achieved. 

Impact on external sustainability 

The high public debt in Lebanon cannot be described as sustainable. Sensitivity analysis of 
the  baseline  scenario  to  adverse  shock  confirms  the  existence  of  substantial  downside 
risks.  Slowdown  in  GDP  growth,  higher  interest  rates  or  a  weaker  primary  government 
surplus would result in a quite substantial increase of the debt to GDP ratio. The risk to the 
public debt outlook is therefore considered to be medium. 

Impact on structural reforms 

The  selection  and  formulation  of  structural  conditions  played  a  prominent  role  in  the 
design  of  the  MFA  operation.  The  decision  making  was  informed  in  particular  by  the 
government  reform  philosophy,  the  challenges  identified  in  a  number  of  reviews  which 
were carried out prior to MFA, as well as consultations with various stakeholders. 

Value added of MFA 

Two particular aspects can be emphasised, namely (i) the confidence building effect at the 
time of announcement of the MFA as part of the comprehensive EU package, and (ii) the 
positive signal that was provided by inclusion of MFA in the EU package which advanced 

EN 32 

Error! Unknown document property name.  EN 

 
the dialogue between the EU and the government. 

Recommendations 

• Value the announcement effect of MFA assistance as part of larger support packages and 
align conditions to this by agreeing on quick disbursing conditions. 

• Value the additional value MFA may have on strengthening relations with the respective 
country, but do not see this as an additional objective of MFA. 

6) Availability of the report on Europa: 

http://ec.europa.eu/economy_finance/evaluation/pdf/evaluation_lebanon_en.pdf 

Ex-post evaluation of Macro Financial Assistance (MFA) operations to 

Georgia 

1) ABB activities: 01 03 

2) Timing: 11/4/2012 (duration: 9/8/2011-11/4/2012) 

3) Budget: EUR 46 million  (total amount of support) 

4) Background, scope and focus:  

Macro-Financial  Assistance  (MFA)  is  a  policy-based  financial  instrument  of  untied  and 
undesignated balance-of-payments support to partner third countries. It takes the form of 
medium/long-term loans or grants, or a combination of these, and generally complements 
financing provided in the context of an International Monetary Fund's reform programme. 

Ex-post evaluations focus on assessing the contribution of EU Macro Financial Assistance 
to macro-economic and structural adjustment of the recipient country. 

The exercise covers three main areas of analysis: 

•  Economic  impact  of  the  MFA  assistance  operation  on  the  economy  of  the  recipient 
country (e.g. GDP growth, Balance of Payments, exchange rates, fiscal balances); with and 
without IMF involvement. 

•  Value  added  of  EU  intervention  (stand-alone  and/or  in  combination  with  IMF 
intervention) provided through the operation. 

• Sustainability of the country’s external position as a result of the assistance. 

5) Summary of findings and conclusions of the evaluation:  

How would the economy of Georgia have evolved in the absence of MFA assistance? 

In  the  absence  of  the  MFA,  it  is  highly  likely  that one  of  the  following  scenarios  would 
have occurred:  

Scenario  A:  reduced  public  spending  by  the  Georgian  government  in  2009  and  2010 
(lower expenditure matches the lower revenue); 

Scenario  B:  reduced  public  spending  in  2009  but  no  change  in  spending  in  2010  (the 

EN 33 

Error! Unknown document property name.  EN 

 
 
 
revenue gap in 2010 in the absence of MFA is replaced with additional deficit financing). 

In both cases, Georgia would have experienced a greater fall in GDP in 2009 as compared 
to  what  actually  happened  (i.e.  a  4  per  cent  reduction  in  GDP  instead  of  3.8  per  cent). 
Moreover,  under  Scenario  A,  real  GDP  would  have grown  by  a  smaller  amount  in 2010 
(6.2  per  cent  instead  of  the  actual  observed  growth  of  6.3  per  cent).  On  the  other  hand, 
under Scenario B, real GDP would have grown at a slightly higher rate in 2010 (6.5 per 
cent)  –  as  the  economy  would  have  grown  from  a  much  smaller  base  in  2009  when 
compared to the baseline scenario.  

The  unemployment  rate  would  have  been  slightly  higher  in  2009  (+0.1  per  cent)  under 
both  counterfactual  scenarios.  Under  scenario  A,  a  slightly  higher  unemployment  rate 
(+0.3 per cent) would also have been observed in 2010. 

To  what  extent  has  the  MFA  assistance  been  effective  in  terms  of  the  short-term 
macroeconomic stabilisation of Georgia?  

The  modelling  results  suggest  that  the  MFA  had  the  following  impact  on  Georgia’s 
economy, although the size of these impacts is small (less than 0.5 per cent): Overall, the 
results of the modelling exercise show that the net effect of MFA on Georgia’s economy 
was  marginal:  the  impact  on  real  GDP  ranging from  -0.2  per  cent  to  +0.2  per  cent.  The 
MFA  had  a  bigger  impact  in  conjunction  with  the  IMF-SBA:  the  impact  on  real  GDP 
ranging from -1.0 per cent to -1.3 per cent. 

To  what  extent  has  the  MFA  contributed  to  returning  the  external  financial  situation  of 
Georgia to a sustainable path over the medium to longer-term? 

The  MFA,  which  was  provided  in  grant  form,  has  contributed  to  improving  Georgia’s 
external financial situation in the short term. However, given the small size of the effects 
described above, relative to Georgia’s economy and its balance sheet items, it is difficult to 
argue convincingly from the model results that MFA contributed substantially to averting 
a crisis through improved external sustainability. However, the modelling fails to capture 
confidence effects and conditionality that may well have longer-term implications for the 
country’s economic trajectory.  

To  what  extent  has  the  MFA  assistance  been  effective  in  terms  of  supporting  structural 
reform in Georgia?  

Overall, the MFA has made a positive but marginal difference in promoting PFM reforms 
in  Georgia.  The  scope  and  impact  of  the  MFA  conditions  was  limited.  The  MFA 
conditions  were  heavily  concentrated  on  the  simpler  reform  activities  or  ‘softer  and  low 
lying fruit’ such as action plans, capacity development plans, manuals, and training.  This 
narrow focus of conditionality on simpler reforms was relevant and appropriate, given the 
need for quick disbursement of support and the scale of the assistance. 

The added value and reinforcing effect of EU support was most clearly evident in the case 
of internal audit reforms. In other areas, the added value of EU support was less obvious 
due to existence of strong domestic ownership and/or other donor support.  

How has the way in which the MFA operation was designed and implemented conditioned 
its effectiveness and efficiency?  

The Commission should be more specific in formulating conditionality and be clear about 
its  expectations,  especially  in  the  case  of  softer  actions  such  as  development  of  action 

EN 34 

Error! Unknown document property name.  EN 

 
plans. 

The Commission should take steps to improve the visibility of the MFA instrument in the 
recipient country through, for example, local press releases. 

6) Availability of the report on Europa: 

http://ec.europa.eu/economy_finance/evaluation/pdf/mfa_georgia_final_report-apr_en.pdf 

Ex-post evaluation of Exceptional Financial Assistance (EFA) 

operations to Kosovo 

1) ABB activities: 01 03 

2) Timing: 17/10/2012 (duration: 9/3/2012-17/10/2012) 

3) Budget: EUR 50 million  (total amount of support) 

4) Background, scope and focus: 

Macro-Financial  Assistance  (MFA)  is  a  policy-based  financial  instrument  of  untied  and 
undesignated balance-of-payments support to partner third countries. It takes the form of 
medium/long-term loans or grants, or a combination of these, and generally complements 
financing provided in the context of an International Monetary Fund's reform programme. 

Ex-post evaluations focus on assessing the contribution of EU Macro Financial Assistance 
to macro-economic and structural adjustment of the recipient country. 

The exercise covers three main areas of analysis: 

•  Economic  impact  of  the  MFA  assistance  operation  on  the  economy  of  the  recipient 
country (e.g. GDP growth, Balance of Payments, exchange rates, fiscal balances); with and 
without IMF involvement. 

•  Value  added  of  EU  intervention  (stand-alone  and/or  in  combination  with  IMF 
intervention) provided through the operation. 

• Sustainability of the country’s external position as a result of the assistance 

5) Summary of findings and conclusions of the evaluation:  

The main findings and conclusions of this evaluation are as follows: 

Macro-Economic Stabilization and External Sustainability 

The  macro-economic  performance  of  Kosovo  between  2006  and  2011  was  positive, 
overall. Fiscal deficits fluctuated during the review period, and Kosovo’s public finances 
remain fragile and exposed to high-probability, high-incidence risks. 

The EFA helped alleviate the financial situation of Kosovo. The disbursement of the first 
tranche of EFA in 2010 provided a useful contribution to the stabilization of bank balances 
following the steep reduction observed in 2009 and 2010. Some recovery of bank balances 
is visible in the 2012 data, and their level is expected to fully recover in 2013 through the 
receipts  from  the  expected  privatisation  of  PTK.  Paired  with  the  intended  reduction  in 
fiscal  deficit  from  2014  onwards,  these  recent  developments  point  to  the  fact  that  the 

EN 35 

Error! Unknown document property name.  EN 

 
 
 
financial  situation  of  Kosovo  has  been  supported  by  EFA,  also  by  exercising  some  soft 
pressure on GoK towards more prudent spending. 

The  EFA  supported  the  development  of  a  sound  economic  and  fiscal  framework.  The 
study concluded that progress was made on the reform items included as EFA conditions 
due  to  a  variety  of  factors,  including  the  commitments  made  in  the  context  of  the  EFA 
agreement itself.  

Kosovo’s  level  of  public  debt  is  low  and  the  risk of  debt  distress  is  moderate.  Kosovo’s 
current  account  deficit  was  consistently  high  but  the  structure  of  Kosovo’s  imports  and 
exports has improved over the period. 

Structural Reforms 

The structural conditions attached to the EFA operation were relevant in light of Kosovo’s 
situation  in  2007.  The  gross  effect  of  EFA  conditionality  was  highest  on  “quick  win” 
technical issues and more problematic on contentious issues. 

The political reinforcing effect of EFA’s conditionality is its most important legacy. 

Design and Implementation 

The special situation of Kosovo and the strong political commitment of the EC to support 
it  resulted  in  a  number  of  design  peculiarities  of  the  EFA  operation  which  played  an 
important – but not exclusive – role in the disbursement delays. 

EU Value Added 

The value added of the EFA was highest through its impact on: 

•  lifting  the  credibility  of  the  country  to  the  international  community  and  to  potential 
investors; and 

• tilting the internal balance of power in the GoK in favour of the MoF. The MoF focused 
on macroeconomic and fiscal stability and was willing to reform along the lines suggested 
by international community. The EFA conditions helped to put pressure on other members 
in the cabinet who were less eager to implement reforms. 

6) Availability of the report on Europa: 

http://ec.europa.eu/economy_finance/evaluation/pdf/efa_kosovo_eval_final_report_oct_2012.
pdf 

EN 36 

Error! Unknown document property name.  EN 

 
 
ABB 02 – Enterprise  

Evaluation of the European Chemical Agency 

1) ABB activities concerned: 02 03  

2) Timing: March 2012. (Period covered by the report: 2007-2010). 

3) Budget: ECHA budget 2008-2010: EUR 212,329,000;  
                   EU subsidy 2008-2010: EUR 165,071,000 

4) Background, scope and focus: 

The European Chemicals Agency (ECHA) created on 1 June 2007 and located in Helsinki 
plays  a  central  role  in  the  implementation  of  REACH  and  CLP.  It  manages  the  main 
processes of both Regulations and its Scientific Committees adopt opinions which serve as 
a  basis  for  the  Commission  to  take  decisions  in  the  context  of  the  restriction  and 
authorisation  of  hazardous  chemicals  and  with  regard  to  the  harmonization  of 
classification and labelling of substances. The Agency also takes decisions, notably in the 
areas of registration and evaluation under REACH. 

The purpose of the study was to assess the Agency's efficiency, effectiveness and economy 
in  building  up  its  capacities  and  in  managing  its  operations  as  well  as  assessing  the 
Agency’s role/added value, its acceptability and location.  

This evaluation was part of a set of studies and reports to be delivered by Member States, 
the  European  Chemicals  Agency  and  the  European  Commission  and  contributing  to  the 
Commission’s review required by the REACH Regulation (REACH review, adopted on 5 
February 2013). 

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study: 

The quality of ECHA’s goal setting improved as the Agency developed; ECHA has  met 
most of its key objectives in terms of outputs but could have improved its performance at 
the level of outcomes. 

ECHA’s  overall  efficiency  was  reduced  by  unforeseeable  circumstances  and  by 
complementary activities that often increased effectiveness. 

It has responded with flexibility to changing external circumstances and has demonstrated 
flexibility in the allocation of its resources. 

The  Agency  performed  well  in  most  work  areas  during  the  review  period,  although 
dissemination  left  room  for  improvement.  ECHA  has  had  a  good  start-up  as  an 
organization  and  implemented  most  of  its  REACH  and  CLP  tasks  effectively.  It  has 
successfully involved all stakeholders, but it could have done more to ensure its perceived 
independence. 

In addition to recommendations aimed at drawing lessons from the ECHA experience for 

EN 37 

Error! Unknown document property name.  EN 

 
 
other and future agencies, the evaluation report recommends EACH to be more transparent 
about  its  contribution  to  REACH  and  CLP  objectives,  to  develop  scenarios  to  ensure 
efficiency  and  economy  in  the  future,  to  broaden  management  activities  to  prevent 
disengagement (with a specific focus on SMEs and increasing the number of organizations 
allowed to attend Committee meetings). The main recommendation to the Commission is 
to maintain the current role of ECHA under REACH and CLP. 

The results of this evaluation were further analysed and screened together with the results 
of other eleven studies and evaluations for the purpose of overall evaluation of Reach in 
the REACH Review adopted on 5 February 2013 (COM(2013)49). 

6) Availability of the report on Europa:  

 http://ec.europa.eu/enterprise/dg/files/evaluation/201203-final-report-echa_en.pdf 

Evaluation of the Functioning of the European chemical market 

after the introduction of REACH 

1) ABB activities concerned: 02 03  

2) Timing: March 2012. (Period covered by the report: 2007-2010). 

3) Budget: not applicable (evaluation of legislation) 

4) Background, scope and focus: 

The objective of the study was to evaluate the implementation of the REACH Regulation 
(that came into force on 1 June 2007) in relation to its impact on the operation of the single 
market and the competitiveness of the European chemicals industry. The evaluation aimed 
to evaluate the relevance, coherence, efficiency, effectiveness, sustainability and impact of 
the REACH Regulation in relation to the operation of the market of chemical substances. 
It  also  aimed  to  identify  strengths  and  weaknesses  of  the  REACH  implementation  with 
respect  to  the  dynamics  of  the  market,  consumer  choice  and  prosperity,  costs  of 
compliance  and  administrative  procedures.  The  impact  on  SMEs  was  given  particular 
focus given the presence of relevant provision in REACH Regulation and the expectation 
that any discrimination against them is avoided. This evaluation was part of a set of studies 
and reports to be delivered by Member States, the European Chemicals Agency, and the 
European  Commission  and  contributing  to  the  Commission’s  review  required  by  the 
REACH Regulation (REACH review, adopted on 5 February 2013). 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The  evaluation  concludes  that  the  REACH  Regulation  is  relevant  in  enhancing  the 
competitiveness  of  the  EU  chemical  industry  and  for  the  development  of  a  harmonized 
market  for  chemicals.  REACH  is  clearly  relevant  to  the  development  of  a  level  playing 
field and a harmonized market. The costs of compliance are sizeable and appear to have 
some  impact  on  the  profitability  of  firms  and  the  competitiveness  of  certain  sectors  in 
relation  to  their  access  to  non-EU  markets.  The  negative  impacts  are  expected  to  be 
reduced in the future. The typical cost of registration per substance by one firm has been 

EN 38 

Error! Unknown document property name.  EN 

 
 
about EUR70 000; representing about 1% of firms’ total annual turnover. Additional costs 
arise from the information exchange requirements and the handling of Safety Data Sheets 
and affect all firms in the chemicals supply chain. It is still too early to identify long term 
impacts of REACH on the firms’ financial position. Firms active in basic chemicals and 
metals  absorb  the  costs  while  in  the  specialty  chemicals  markets  firms  have  greater 
capacity  to  pass  costs  down  the  supply  chain.  REACH  costs  and  the  introduction  of 
substances in the candidate list have led to the withdrawal of some chemical substances; 
there are only limited cases where this has become problematic for the access of firms to 
critical  raw  materials.  There  are  indications  that  REACH  leads  to  a  reduction  in  the 
number  of  suppliers  of  certain  substances  increasing  concentration  in  some  chemicals 
markets. The scale of this effect is so far rather limited.  

On  the  basis  of  the  limited  information  available,  the  overall  conclusion  is  that  REACH 
has not had a sizeable impact on the prices of final consumer products.  

There is no supportive evidence as to the effect of REACH on consumer confidence or the 
development  of  new  substances  and  creation  of  new  business  opportunities  at  this  stage. 
REACH  contributes  to  the  strengthening  of  communication  along  the  chemicals  supply 
chain even though this has yet to materialize.  

On  REACH  mechanisms,  the  operation  of  consortia  appears  to  be  much  more  effective 
than  Substance  Information  Exchange  Fora.  There  is  no  evidence  that  REACH 
mechanisms have led to the loss of confidential information. However, the participation in 
SIEFs  provides  information  that  can  present  important  business  intelligence.  There  is 
generally a positive view on the role of ECHA and national helpdesks and the respective 
tools  developed.  Still,  firms  also  need  the  assistance  of  Trade  Associations  and  private 
consultants for more practical and tailored guidance. Overall Only Representatives appear 
to have a positive role in supporting non-EU firms and facilitating their access to the EU 
chemicals market. 

Main recommendations:  

Avoid, to the extent possible, changes to any of the key or provisions of the Regulation; 
Aim  to  clarify  any  unclear  requirements,  increase  predictability  and  improve  tools  and 
implementation structures to reduce or eliminate unnecessary costs;  

Prepare for the following registration periods with focus on the needs of SMEs;  

Expand  the  awareness  raising  and  information  provision  tools.  Continue  monitoring  of 
market and industry developments;  

Support industry in the use of knowledge. 

The results of this evaluation were further analysed and screened together with the results 
of other eleven studies and evaluations for the purpose of overall evolution of Reach in the 
REACH Review adopted on 5 February 2014 (COM(2013)49). 

6) Availability of the report on Europa: 

 http://ec.europa.eu/enterprise/dg/files/evaluation/201203-final-report-chemical-

EN 39 

Error! Unknown document property name.  EN 

 
market_en.pdf 

Evaluation of the Impact of the REACH regulation on the innovativeness of EU 
chemical industry 

1) ABB activities concerned: 02 03  

2) Timing: June 2012. (Period covered by the report: 2007-2010). 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

The overarching objective of the study was to evaluate the changes in the dynamics of the 
chemicals  market  after  the  introduction  of  the  REACH  regulation.  Specifically  the 
evaluation aimed to: 

− 
Identify,  test  and  apply  methodologies  for  evaluating  the  relevance,  coherence, 
efficiency, effectiveness, sustainability and impact of the REACH regulation in relation to 
the ability to innovate by EU chemical industry.  
− 
provide  recommendations  to  improve  the  mechanisms  introduced  by  the  REACH 
regulation  to  stimulate  innovation  (by  e.g.  encouraging  private  investments,  increasing 
patenting  or  trade  marking,  speeding  time-to-market,  increase  communication  in  the 
supply chain, secure financing for innovation, etc.); 
This evaluation was part of a set of studies and reports to be delivered by Member States, 
the  European  Chemicals  Agency  and  the  European  Commission  and  contributing  to  the 
Commission’s review required by the REACH Regulation (REACH review, adopted on 5 
February 2013). 

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study: 

The report concluded that the Regulation's objectives regarding the innovativeness of the 
EU chemical industry are highly relevant in terms of EU industrial and innovation policies. 
The industrial information creation, capture and dissemination mechanisms created by the 
Regulation have acted as stimuli to product conception or innovation to varying degrees. 
Some 72% of companies surveyed thought it had led to an increase in access and scrutiny 
of  information  about  chemical  substances  and  24%  indicated  that  they  had  been  able 
benefit from this through increased knowledge of substances and properties. However, this 
has come at a significant cost to the industry.  More positive results may be apparent in the 
longer term, as companies reorient their R&D and innovation programmes. 

The registration process has had an impact on innovation, but the candidate list is currently 
creating  the  greatest  deal  of  innovative  activity  (with  the  SIN  list,  and  more  recently, 
possibly  the  CoRAP).  Authorisation  and  restriction  are  affecting  a  smaller  part  of  the 
industry.  It  has  not  been  possible  to  discern  overall  unequivocal  benefits  for  consumers, 
the market and society at this stage of implementation. 

SMEs,  and  particularly  small  firms,  seem  especially  affected  by  the  Regulation  due  to 

EN 40 

Error! Unknown document property name.  EN 

 
 
issues surrounding access to input markets such as costs of specialised staff, training, and 
Letters of Access for formulators; and also access to finance in the view of increased risks 
and  uncertainties  associated  with  innovation.  Highly  innovative  SMEs  have  a  more 
negative  view  of  the  overall  effect  of  REACH  on  innovation  than  large  firms  for  the 
present and the future.   

Main recommendations: 
−  Enablers:  increase  awareness  of  REACH-related  innovation,  identify  and  promote  
research  funding  into  substitution  of  substances  on  the  authorisation  list  and  encourage 
private and public funding for REACH-related innovation 
− 
Firm  activities:  address  uncertainties  surrounding  the  candidate  list,  increase 
encouragement  of  the  development  of  QSARS  and  similar  alternative  testing  methods, 
promote  the  use  of  the  PPORD    (and  consider  a  smaller  firms  friendly  version  of  the 
PPORD), consider raising the volume exemption to a level that would provide for higher 
volume testing and piloting, consider lengthening the consultation times for animal testing,  
identify bottlenecks in supply at the level of GLP labs, identify ways to make it easier and 
cheaper for firms to protect IP and CBI within REACH. 
−  Outputs: increase predictability about timing and costs and reduce expected times for 
registration for phase-in substances, consider developing a new category in REACH, that 
of a “small chemical company” which could, for example, be subject to reduced REACH 
registration obligations, or be able to claim reduced costs for Letters of Access. 

The results of this evaluation were further analysed and screened together with the results 
of other eleven studies and evaluations for the purpose of overall evaluation of REACH in 
the REACH Review adopted on 5 February 2013 (COM(2013)49). 

6) Availability of the report on Europa: 

http://ec.europa.eu/enterprise/dg/files/evaluation/final-report-reach-june-2012_en.pdf 

Evaluation of the Ecodesign directive (2009/125/EC) 

1) ABB activities concerned 02 02  

2) Timing: March 2012 (Period covered by the report: 2009-2011). 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

Under Article 21 of the Ecodesign Directive, the Commission was required to review, no 
later  than  2012,  the  effectiveness  of  the  Ecodesign  Directive  and  of  its  implementing 
measures and to assess the appropriateness of extending the scope of the Directive to non-
energy related products.  

The  evaluation  was  aiming  at  the  review  of  the  effectiveness  of  (i)  the  Directive  and  its 
implementing  measures,  (ii)  the  Ecodesign  methodology,  (iii)  the  threshold  for 
implementing  measures  as  described  in  Article  15  of  the  Directive,  (iv)  market 

EN 41 

Error! Unknown document property name.  EN 

 
 
 
surveillance, and (v) self-regulation measures. 

5) Summary of findings and conclusions of the evaluation or evaluation related  
    study: 

The evaluation study concluded that, in general, the Ecodesign Directive is achieving its 
policy  objectives  (free  movement  of  goods  and  environmental  protection)  and  that  no 
revision of the Directive is deemed appropriate at the moment or necessary to increase its 
effectiveness and that of its implementing measures. 

The study has, in particular, pointed out that: 

− 
In  principle,  the  Directive  is  achieving  its  policy  objectives.  Since  2005  the  main 
focus  of  the  implementing  measures  has  been  on  energy  efficiency.  The  available  data 
illustrates  a  move  towards  energy  efficiency  for  all  products  regulated  by  the  Ecodesign 
implementing measures. 
− 
It  is  too  early  to  correctly  evaluate  the  full  effect  of  the  Directive  and  of  the 
implementing mandatory and self-regulation measures because of the insufficient period of 
their application. For one out of the twelve ecodesign Regulations adopted at the time of 
the  evaluation,  Tier-1  requirements  had  not  yet  entered  into  force  and  for  eight 
implementing measures, Tier-2 requirements had not yet entered into force. Furthermore, 
out of two proposed voluntary agreements, none has so far been officially endorsed by the 
Commission. 
− 
measures, defined in Article 15 of the Ecodesign Directive, remain appropriate. 
As  pointed  out,  there  have  been  some  limitations  on  the  conduct  of  the  evaluation:  the 
early timing of the evaluation and data related issue, due to the official statistical sources 
referring to the period before the Implementing Measures were adopted. These limitations 
need to be taken into account in reading the evaluation report. 

It  is  considered  that  the  indicative  criteria  for  adopting  implementing  ecodesign 

Based on the findings of the evaluation, the Commission adopted, on 17 December 2012, a 
Report to the European Parliament and the Council: ‘Review of Directive 2009/125/EC of 
the European Parliament and of the Council of 21 October 2009 establishing a framework 
for 
the  setting  of  ecodesign  requirements  for  energy-related  products  (recast)’ 
[COM(2012) 765].  

The report confirmed that no legislative changes were necessary in the Directive, as it is 
achieving  its  policy  objectives  reasonably  well.  It  also  concluded  that  it  would  be 
premature to enlarge the scope of the Directive, but referred to the forthcoming review of 
the  Energy  Labelling  Directive  in  2014  for  a  wider  assessment  of  the  EU's  sustainable 
product policy.  

6) Availability of the report on Europa: 

 http://ec.europa.eu/enterprise/dg/files/evaluation/cses_ecodesign_finalreport_en.pdf 

EN 42 

Error! Unknown document property name.  EN 

 
 
 
Evaluation of Member State Policies to Facilitate Access to Finance for SME 

1) ABB activities concerned: 02 02 - 

2) Timing: June 2012, (duration 2010-2012) 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

The  objective  of  the  study  is  to  evaluate  the  efficiency,  utility,  sustainability, 
effectiveness and impact of existing Member States programmes to facilitate the access 
to finance of SMEs. This is important to provide recommendations on which (type of) 
programmes work best and which elements could also be used in other countries. 

5) Summary of findings and conclusions of the evaluation or evaluation related  
     study 

In the context of the current crisis, it is increasingly difficult and expensive for SMEs to 
access private sources of finance. Member States and the European Commission have 
taken comprehensive measures to address the current tensions in financial markets and 
to safeguard the flow of credit to the real economy. To facilitate access to finance, the 
study provides information on how to access over EUR 50 billion of public finance in 
the  27  Member  States.  The  study  presents  over  120  national  or  regional  financing 
programmes  and  provides  key  information  helping  SMEs  to  apply  for  the  different 
programmes in terms of characteristics, terms, conditions and contact information.  

The  study  also  provided  an  evaluation  of  public  financing  programmes  in  5  Member 
States (Germany, France, the UK, Poland and Sweden) to exchange good practice and 
assess  which  programmes  work  best  and  could  be  used  in  other  countries.  The  study 
stresses  the  importance  of  having  efficient  and  flexible  financing  programmes  to 
correspond to changing market conditions and to be well integrated in the local business 
environment.  In  terms  of  operation,  public  programmes  need  quick  and  efficient 
decision  marking  processes  and  a  strong  collaboration  with  the  private  sector.  Public 
financing programmes should also be aligned with business advice services.  

6) Availability of the report on Europa:  
 http://ec.europa.eu/enterprise/policies/finance/guide-to-funding/indirect-
funding/files/evaluation-of-national-financing-programmes-2012_en.pdf 

EN 43 

Error! Unknown document property name.  EN 

 
 
 
 
 
Evaluation of the Impact of European Space Policy on European Space 
Manufacturing and the Services Industry 

1) ABB activities concerned: 02 03  

2) Timing: August 2012 (Period covered by the report: 2007-2011). 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

The evaluation aims were to: 

to  strengthening  European  space 

−  Assess  the  impacts  to  date  of  the  European  Space  Policy  adopted  in  2007  in 
strengthening space industrial competitiveness (in manufacturing and services); 
−  Assess the main challenges ahead for the European Space Industry in manufacturing 
and services in maintaining and strengthening space industrial competitiveness; 
−  Review  the  strategic  importance  and  specific  characteristics  of  the  European  Space 
Industry to assess whether a dedicated European Space Industrial Policy is appropriate; 
−  Develop quantitative and qualitative indicators to measure the future contribution of 
the  ESP 
in  space 
manufacturing  and  services  and  in  wider  sectors  where  space-based  technologies  and 
applications are a key competitiveness driver; 
− 
competitiveness issues facing the upstream and downstream space sectors; 
− 
Inform  the  preparation  of  a  European  Space  Industrial  Policy  (and  possible 
accompanying Regulations) in accordance with the Lisbon Treaty provisions (TFEU – Art. 
173 and 189); 
−  Analyse the potential role of EU regulation in future as an enabler in facilitating the 
development of new and emerging markets in space services, such as in the distribution of 
high-resolution satellite data and its commercial distribution.  

Identify  current  and  possible  future  developments  in  the  space  industry  and  on 

industrial  competitiveness 

5) Summary of findings and conclusions of the evaluation or evaluation related  
    study: 

The overall conclusions were that: 

−  The  adoption  of  the  ESP  in  2007  has  provided  an  important  framework  through 
which  critical  precursor  steps  have  been  taken  towards  the  development  of  the  future 
European  Space  Industrial  Policy.  It  is  highly  doubtful  without  the  existence  of  the  ESP 
whether it would have been possible to launch a dedicated industrial policy for the space 
sector. 
−  Through  the  frame  of  the  ESP,  progress  has  also  been  made  in  strengthening  space 
governance even if some of the achievements predate the ESP’s adoption, such as the 2003 
EC-ESA Framework Agreement and the holding of regular Space Council meetings. 
− 
Following the adoption of Article 189 of the TFEU, considerable political and legal 
progress has been made. This gives the EU joint competence, together with ESA and the 
member states, to pursue a formal European Space Industrial Policy. 
− 

Following  the  Lisbon  Treaty’s  adoption,  space  is  now  formally  an  EU  policy  in  its 

EN 44 

Error! Unknown document property name.  EN 

 
own right, rather than part of the broader industrial policy landscape. 
−  Moreover, the Competitiveness Council’s legal jurisdiction has been extended to the 
space  sector.  Gaining  wider  acceptance  that  space  is  a  key  strategic  sector  and  of  vital 
importance in promoting growth and jobs is a major step forward. 
−  Although more could be done to strengthen the ESP’s visibility among industry and 
wider space  stakeholders, it has played an important role in providing strategic direction 
and a focal point for national space agencies, national authorities and wider stakeholders 
across the EU. 
− 
Progress  has  also  been  made  through  the  frame  of  the  ESP  in  promoting  industrial 
competitiveness  through  EU  expenditure  programmes  and  specific  policy  initiatives. 
While the ESP has not had a dedicated budget of its own, seen through the ESP’s prism, 
various EU research and innovation and operational programmes have contributed to the 
achievement of objectives relating to industrial competitiveness. 

The evaluation’s recommendations deal with: 

the definition of the European Space Industry Policy, 
the prerequisites for the adoption of a  EU regulatory framework,  
the need for a stable industrial base with balanced SME participation,  

− 
− 
− 
−  ways to ensure technological non-dependence and independent access to space, 
− 
− 
European space industry. 

the promotion of the development of new markets for space applications and services, 
the  need  for  defining  indicators  and  monitoring  the  general  performance  of  the 

The  results  of  this  evaluation  have  fed  into  the  Communication  on  EU  Space  industrial 
policy, adopted on 28 February 2013 (COM (2013)108). The recommendations on the EU 
regulatory  framework  will  be  further  analysed  in  an impact  assessment  and  submitted  to 
extensive consultation in 2013.  

6) Availability of the report on Europa:  
http://ec.europa.eu/enterprise/dg/files/evaluation/space_final_report_evaluation_of_the_impac
t_of_esp_en.pdf 

Evaluation of the Pressure Equipment Directive 

1) ABB activities concerned 02 03 

2) Timing: October 2012 (Period covered by the report: 2002-2012- 10 years of 
implementation of PED Directive). 

3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

EN 45 

Error! Unknown document property name.  EN 

 
 
 
 
 
After ten years of implementation, the evaluation aimed to assess the performance of the 
Pressure  Equipment  Directive  since  its  introduction  and  to  make  a  judgement  of  the 
Directive’s  continuing  relevance.  The  purpose  of  the  evaluation  has  been  to  assess  how, 
and to what extent, the Directive has met its objective of guaranteeing free circulation of 
stationary pressure equipment within the EU while ensuring a high degree of safety.  

5) Summary of findings and conclusions of the evaluation or evaluation related  
     study: 

The  introduction  of  the  PED  has  proven  relevant  for  the  improved  functioning  of  the 
Internal  Market  and  successfully  combined  market  integration  with  maintaining  the  high 
levels of product safety that Europe was already accustomed to. One of the core objectives 
of  the  Directive,  the  removal  of  barriers  to  trade  and  improving  the  effectiveness  and 
efficiency  of the  Internal  Market,  has  been  realised  to  a  significant  extent.  The  PED  has 
opened  up  trading  opportunities  across  the  Internal  Market  with  the  benefit  of  providing 
users with greater levels of choice. The conclusion of the report is that no urgent revision 
of  the  Pressure  Equipment  Directive  is  needed.   This  gives  time  to  focus  on  better 
implementation  and  prepare  gradually  for  full  revision  later  on.  The  PED  has  been  well 
formulated  and  meets  the  needs  of  the  target  beneficiaries.  In  relation  to  other  pieces  of 
European legislation, the scope of the PED is generally clear and the Directive fits into a 
relatively  stable  and  coherent  regulatory  environment.  Overall,  the  PED  has  achieved 
European  added  value.  The  largely  harmonised  legislative  framework  has  had  a  positive 
impact  on  the  European  pressure  equipment  sector.  This  outcome  could  not  have  been 
achieved  through  Member  States  acting  alone  or  bilaterally.  The  issue  of  market 
surveillance at national level has been raised as a scope for improvement. The report calls 
for stronger and more consistent enforcement of the rules with a view to ensuring that the 
same conformity assessment procedures are correctly applied to the same types of pressure 
equipment. This and other evaluation recommendations have been taken into account for 
the subsequent actions in 2013, mainly through the foreseen alignment of the Directive to 
the  New  Legislative  Framework.  The  Commission  proposal  on  strengthening  market 
surveillance  in  the  Member  States  (Product  Safety  and  Market  Surveillance  Package, 
adopted in February 2013) is also going to contribute to address weaknesses highlighted in 
the report.  

6) Availability of the report on Europa: 
http://ec.europa.eu/enterprise/dg/files/evaluation/evaluation-of-the-pressure-equipment-
directive_en.pdf 

Evaluation of EU Member State Business Angel Markets and Policies 

1) ABB activities concerned: 02 02  

2) Timing: October 2012.  

EN 46 

Error! Unknown document property name.  EN 

 
 
 
 
3) Budget: not applicable (evaluation of policy/legislation) 

4) Background, scope and focus: 

Business Angels play an important role in the economy, and in many countries constitute 
the  largest  source  of  external  funding,  after  family  and  friends,  in  newly  established 
ventures. Given the importance of informal investors for the creation and maintenance of 
an  entrepreneurial  economy,  more  data  is  needed  from  the  angel  investment  market  in 
order to make rational and well-grounded policy decisions. The objective of the evaluation 
was to improve policy-making regarding SMEs’ access to seed and early stage capital by 
increasing the knowledge of the characteristics of the Business Angels market and sharing 
EU Member States best practices of policies and programmes supporting Business Angel 
financing. The project aimed to review the existing public policies and to gather evidence 
concerning the effectiveness of public support. 

5) Summary of findings and conclusions of the evaluation or evaluation 
     related study 

The project provided an overview of the key characteristics and results of programmes or 
polices  aimed  at  increasing  and  supporting  Business  Angel  investment  in  the  selected 
Member  States.  It  provided  interesting  details  on  how  Business  Angels  operate,  their 
number and volume of investment, public policies and support for business angels, as well 
as  providing  recommendations  on  further  data  collection  needs  and  issues  for  public 
policy.  The review of the evidence on public policy schemes supporting Business Angels 
suggested  a  rather  mixed  picture.  Each  of  the  various  types  of  public  policy  tool  has  its 
own  advantages  and  disadvantages  and  a  combination  of  tools  may  be  needed  in  many 
cases. Given their possible contribution of business angels as a source of finance for both 
mainstream and technology intensive early stage and new companies, public policy must 
take  into  consideration  the  needs  of  business  angels  when  new  legislation  or  regulations 
are  introduced.  Public  support  measures  that  directly  provide  risk  capital  to  early  stage 
business  should  be  designed  in  a  way  to  attract  investment  by  business  angels, 
complementing their role and making use of the non-financial aspect of their involvement 
including management expertise and networking. It was also suggested that existing data 
collection  procedures  be  continued,  and  continually 
to  create  more 
comparability,  consistency  and  completeness,  as  the  existing  data  sources  regarding 
Business  Angels’  market.  These  should  be  supplemented  with  a  series,  or  well-designed 
programme,  of  specific  studies  aimed  at  meeting  gaps  in  existing  knowledge  which  are 
quite fundamental to the notion and support of Business Angels. The Commission's policy 
is to identify and spread good practices that can help improve the conditions for business 
angel  investment  across  the  EU.  The  evaluation  contributes  to  the  specific  objective  of 
facilitating access to finance for the creation and growth of SMEs. 

improved 

6) Availability of the report on Europa: 
     http://ec.europa.eu/enterprise/dg/files/ba-rep_en.pdf 

EN 47 

Error! Unknown document property name.  EN 

 
ABB 03 – Competition 

Study on the Implementation of the existing Broadband Guidelines 

1) ABB activities concerned: 03 AWBL 03  

2) Timing: 7 December 2011, (Duration:  6/2011 to 11/2011) 

3) Budget: not applicable 

4) Background, scope and focus:  

The  Commission  prepared  a  review  of  the  Broadband  Guidelines  of  2009  by  December 
2012.  The  core  objective  of  the  study  was  to  support  the  revision  process  by  drawing 
lessons  from  the  implementation  of  past  decisions  and  by  clarifying  a  given  set  of 
technological issues. A sample of 10 broadband projects (9 in the EU, 1 in New Zealand) 
was  selected.  Five  of  the  selected  EU  case  studies  were  approved  before  the  Guidelines 
were adopted in 2009 (between 2005 and 2007) and four were approved during the validity 
of the Guidelines (in 2010). Based on the case studies and the expert opinion on technical 
issues  the  contractor  identified  several  challenges  associated  with  the  Guidelines:  basic 
principles (definition of NGA, harmonisation of SMP regulation and State aid rules), State 
aid  and  broadband  projects  in  general  (compatibility  conditions  in  point  51  of  the 
Guidelines,  amendments  with  regard  to  broadband  networks  in  general),  State  aid  and 
NGA  projects  (distinction  of  NGA  areas,  additional  conditions  for  NGA  networks, 
amendments with regard to NGA networks) and separation. 

5) Summary of findings: 

The Guidelines provide appropriate demarcations of the respective terms, distinct between 
basic, NGA and ultra-fast broadband and request a proper reporting to the Commission on 
aid projects.  

Time lag between the implementation of the projects and the evaluation:  

The  case  studies  did  not  provide  the  same  level  of  detail  regarding  their  empirical 
information. Quantity and quality of the information received from the case studies depend 
on  the  concrete  implementation  of  the  measures  (e.g.  internal  monitoring/reporting 
procedures, status/progress of implementation). 

6) Availability of the report on Europa:  

http://ec.europa.eu/competition/consultations/2011_broadband_guidelines/final_report_en.pdf 

EN 48 

Error! Unknown document property name.  EN 

 
 
 
Study on co(re)insurance and on ad-hoc co(re)insurance agreements on the 
subscription market 

1) ABB activities concerned: 03 AWBL 02  

2) Timing: 15 November 2012. 

3) Budget: not applicable 

4) Background, scope and focus:  

The study constitutes a first step in the process to prepare for the potential review of the 
BER, which will expire on 31 March 2017, as well as for the report which the Commission 
will have to submit to the European Parliament by 31 March 2016. It is also a follow up to 
the Commission's report on the Business Insurance Sector Inquiry ("BISI"), published on 
25 September 2007.  

The Commission performed the study in order to collect empirical knowledge concerning 
the  functioning  of  co(re)insurance  pools  and  co(re)insurance  agreements  on  the 
subscription markets and to gather information on the application of the BER. Its aim was 
also  to  contribute  to  the  Commission`s  market  monitoring  of  the  operation  of  these 
co(re)insurance schemes.  

5) Summary of findings : 

The  study  provides  an  overview  of  the  functioning  of  co(re)insurance  pools  and  ad-hoc 
co(re)insurance  agreements  in  the  EEA.  It  identifies  the  types  of risks  which  are  usually 
insured by the above mentioned structures and explains the functioning and specificities of 
co(re)insurance  pools  and  ad-hoc  co(re)insurance  agreements.  Therefore  it  constitutes  a 
key basis to undertake the work necessary to review the functioning of the existing BER 
and make appropriate proposals for its potential modification.  
A  number  of  respondents  were  difficult  to  identify.  Some  respondents  declined  to  reply 
and others provided incomplete information, in particular with regards to questions of the 
functioning of the BER.  

6) Availability of the report on Europa:  

http://ec.europa.eu/competition/sectors/financial_services/insurance.html 

EN 49 

Error! Unknown document property name.  EN 

 
 
 
 
ABB 04 – Employment and social affairs 

Analysis of costs and benefits of active vs passive labour market measures 

1) ABB activities concerned: 04 02   

2) Timing: March 2012 (duration: December 2010 - March 2012; Period covered 1990- 2010 
with a focus on the recent crisis period (2008-10).  

3) Budget: not applicable 

4) Background, scope and focus:  

The  study  analysed  the  use  of  active  and  passive  Labour  Market  Policies  (LMP)  since 
1990  to  help  formulate  LMP  in  the  framework  of  the  Europe  2020  strategy  and  the  exit 
strategy from the 2008 crisis. It covers the following elements:  

• 

• 

• 

• 

• 

• 

the rationales and theoretical impacts of active and passive measures (APM);  

trends in public expenditure on active and passive LMP;  

the origin of funding and the  main institutions responsible for key LMP delivery in 
the EU as well as the implications of the different funding mechanisms.  

an analysis the effectiveness of active labour market measures 

conclusions on the priorities, pitfalls and timing of the implementation of a mixture of 
active and passive measures.  

identification  of  gaps  in  the  knowledge  of  the  measures  effectiveness  and  the 
implications of the gaps for future policy measure design and research. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
    study 

The study was not an evaluation of an existing Programme. It was a study (funded under 
the  PROGRESS  Programme)  analysing  the  efficiency  of  labour  market  measures  and 
expenditures (active as well as passives), notably on the basis of a review of the literature 
of existing evaluations. 

The final recommendations are structured around the following points:  

•  Mobilizing labour: integration of benefits to avoid unemployment trap; extensions of 
job  search  and  acceptance  requirements  for  other  benefits  such  as  social  assistance; 
reduction of the implicit benefit of a non-working spouse; sanctions; continuous job search 
assistance  oriented  on  employer  demand;  training  for  all  groups  that  lack  specific  skills 
needed  in  the  labour  market;  controlled  expansion  of  training  activities;  mid-term  job 
placement targets. 
• 
Supporting  employment  especially  of  young  people:  reducing  legal  differences 
between temporary and permanent contracts; rewarding the actor investing in activation of 
hard-to-place (young) workers; promote quality apprenticeships and traineeship contracts; 

EN 50 

Error! Unknown document property name.  EN 

 
discontinue/decrease  employment  incentives  for  work  experience  when  the  economy 
recovers. 
• 
Protecting  the  vulnerable:  extend  coverage  and  eligibility  of  social  protection; 
organise  coverage  on  the  basis  on  "apparent  employment  relationship"  rather  than  on 
specific  forms  of  contracts;  more  generous  social  assistance  with  strict  job  search  and 
acceptance  criteria;  apply  active  measures  from  the  start  for  vulnerable  groups;  focus 
employment incentives on long-term unemployed 
• 
Improving  effectiveness  of  measures  and  funding:  anticipate  crisis  period  by 
raising tax/contribution rate sufficiently in advance; address potential risks associated with 
combination  central  funding  and  local  implementation;  build  in  flexible  arrangements  to 
facilitate  budget  shifts;  if  budget  cuts  ,  focus  on  fewer  costlier  measures  to  assist  larger 
number of unemployed; offer public sector job at min wage to tackle recipient of benefits 
having informal job; make continuation of measures contingent on evaluation results 
Finally the last point was more specifically about evaluations :  

• 
Improving quality of evaluations: include ex ante evaluations of policy proposals; 
experiments  in  specific  regions  to  measure  results  and  compare  different  options;  take 
inspiration  from  US  evaluations;  distinguish  initial  and  long-term  effect;  attempt  to 
estimate deadweight loss, substitution and displacement effects 

6) Availability of the report on Europa 

    http://ec.europa.eu/social/BlobServlet?docId=7602&langId=en# 

Evaluation of the reaction of the ESF to the economic and financial crisis 

1) ABB activities concerned: 04 02  

2) Timing: March 2012 (duration: December 2010- March 2012).  

3) Budget: whole budget for the 2007-2013 programming period is   EUR 76.6 billion. 

4) Background, scope and focus: 
the  purpose  of 

this  assignment 

the  crisis  was  –  on 

the  basis  of 
For 
 GDP  and  employment  growth  figures  –  defined  as  the  period  between  IIIQ  2008 
 and IIIQ 2009. This evaluation analysed how ESF management and support was adjusted 
in reaction to the crisis, whether the changes helped mitigating the effects of the crisis and 
how  the  ESF  could  be  made  more  responsive  to  the  needs  of  post-crisis  recovery.  The 
changes to the ESF implementation were analysed in the context of the labour market and 
policy  developments  in  the  Member  States.  The  focus  of  the  evaluation  was  on 
governance.  It  has  not  dealt  with  the  effectiveness  or  impacts  of  the  specific  measures 
undertaken,  for  which  the  delays  are  still  too  short.  An  analysis  of  the  effectiveness  of 
different labour market instruments has recently been undertaken in another study.1 The 2 
main criteria were: 1) Coherence: "Have the changes in the use of the ESF been 'coherent' 

1 Analysis of costs and benefits of active compared to passive measures, ECORYS and IZA, 2012. 

EN 51 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
                                                            
with the short term needs of the labour markets in the Member States?" and 2) Rationality: 
"Have  the  changes  also  helped  foster  the  recovery  from  the  crisis  in  a  longer  term 
perspective?" 

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study 

The ESF proved to be a flexible instrument which has been used for quick reaction 
against the crisis. By the end of 2010 84 out of 117 OPs had been adapted in response to 
the  crisis.  In  only  27  cases  a  formal  change  of  OP  was  required.  Most  of  the  changes 
concerned  broadening  of  scope,  launching  specific  calls  and  including  further  target 
groups in the OP. 

17 Member States introduced at least one of the simplification measures proposed by the 
EC: 
• 
The most utilized measure was the extension of the use of the 2000-6 funds; used by 
BE, CZ, DE, ES, HU, PT, SK and SI. Some MS did not take advantage of this option for 
fear that this upfront liquidity would delay the implementation of the funds for the current 
programming period.  
•  HU, IE, MT, PT, RO, SI and UK took advantage of advanced payments.  
• 
'Front loading', i.e. a temporary reimbursement of 100%, was reported to be used only 
by  ES,  LV,  SI  and  UK,  as  it  has  to  be  counterbalanced  later  in  the  implementation  and 
disbursement period by additional national funding.  
• 
were introduced in AT, BE, DE, FR, IT, LV, MT, NL, PT, SI and UK. 

Further  elements  of  simplification  (such  as  flat-rate  costs,  standard  unit  costs,  etc.) 

In an emergency situation like the economic crisis MS preferred to stick to already tested 
solutions. The introduction of any kind of modification to the financial management 
or  the  delivery  system  was  considered  time-consuming  and  seemed  difficult  in  the 
middle of the implementation. Especially in MS with limited administrative capacity the 
proposed  simplification  measures  could  not  be  really  used  because  the  national  and 
regional administrations were not sufficiently flexible to change their procedures. 

The role of the ESF in mitigating the crisis varied considerably according to the impact 
of  the  crisis  on  the  respective  economy  and  to  the  volume  of  the  ESF  in  active  labour 
market  policy.  While  in  the  'better  off'  countries,  such  as  AT,  BE,  DE,  SE,  UK,  it  is 
possible to orient the ESF towards anticipation and innovation, in the other countries the 
immediate  needs  and  governance  issues  remain  of  utmost  importance.  The  evaluation 
furthermore identifies two groups of Member States where the ESF plays a major role in 
active  labour  market  policy,  but  where  governance  structures  will  need  further  attention 
(BG, EL, RO, where various problems of implementation occurred, and CZ, HU, SK, SL, 
where the absorption rates are extremely low so far).  

The reaction of the ESF was coherent with the immediate needs of the labour market 
in the crisis. The focus of the ESF shifted temporarily from the traditional target groups 
(young, low skilled, women, disabled, migrants) to the groups that were hit the most in the 
first  phase  of  the  crisis  (i.e.  prime-aged  men  having  worked  in  construction  or 
manufacturing),  and  from  more  strategic  interventions  (e.g.  means-tested  training)  to 
interventions  aimed  at  directly  maintaining  or  creating  employment  (e.g.  combination  of 

EN 52 

Error! Unknown document property name.  EN 

 
 
short time work with training, supported employment). 

The  ESF  improved  the  consistency  of  the  anti-crisis  measures  with  the  long-term 
perspective requiring structural adaptation. The ESF was broadly used to finance the 
training schemes offered in the short time working arrangements adding thus a structural/ 
active component to the otherwise passive measures.  

6) Availability of the report on Europa 

http://ec.europa.eu/social/BlobServlet?docId=7671&langId=en 

PROGRESS mid-term evaluation 

1) ABB activities concerned: 04 04 

2)  Timing:  January  2012  (Duration:  June  2010-December  2011.  Period  covered  2007-mid 
2010). 

3)  Budget:  The  overall  multi-annual  budget  of  PROGRESS  is:  EUR  683  million  (Progress 
Microfinance budget excluded). 

4) Background, scope and focus  
The mid-term evaluation of PROGRESS was performed by ECORYS. The evaluation covers 
the five policy sections of the programme: employment, social protection and social inclusion, 
working  conditions,  anti-discrimination  and  gender  equality.  It  focuses  on  the  relevance, 
effectiveness,  efficiency  and  EU  added  value  of  the  programme  for  the  period  2007-mid 
2010.  In  addition,  it  reviews  horizontal  issues,  notably  the  programme  management, 
partnership  strategies  and  gender  mainstreaming.  It  is  based  on  the  review  of  relevant 
literature,  operational  reports,  interviews  with  PROGRESS  stakeholders,  and  a  survey  of 
grant beneficiaries. One case study for each policy section illustrates key success factors and 
obstacles  and  provides  illustrative  examples  of  the  EU  added  value.  The  evaluation  also 
benefited from the data and reports produced in the framework of the on-going PROGRESS 
monitoring contract. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 
On  the  whole  PROGRESS  has  delivered  positive  results.  The  nature  of  PROGRESS-
funded activities is often indirect, in the sense that the activities try to improve the quality of 
debate, processes, cooperation or sharing of best practices, rather than directly aiming at core 
policy final outcomes (such as employment targets). As a consequence the achievements are 
also often indirect and dependent on further actions at the Member State level and sometimes 
causality is difficult to establish. But study findings point to overall positives results; 
The shift to a single programme has had advantages in terms of reduction of administrative 
burden and, to some extent, economies of scale, but the development of cross-cutting issues 
could be improved, as the programme still has silo characteristics between the policy fields; 
Improvements could be made to the functioning of the PROGRESS Committee. Whilst 
the Committee is considered a valuable forum for discussions and coordination, there is some 
frustration among Member States on its working. More substantial and thorough discussions 

EN 53 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
and earlier involvement on the design of the priorities would be desirable; 
PROGRESS-funded  activities  have  influenced  EU  policies  and  legislation  (e.g.  Europe 
2020  anti-poverty  targets,  reform  of  pension  systems,  implementation  of  law  on  health  and 
safety at work, labour law). They have also influenced emerging/changing policy areas – the 
anti-discrimination theme features strongly here; 
PROGRESS  has  generated  value  by  operating  as  a  complement  to  other  policies  and 
programmes (for instance the European Social Fund, the European social dialogue). Another 
strong  point  is  the  production  of  statistical  tools/indicators  to  provide  an  evidence  base  for 
policy making; 
The  findings  show  that  the  communication  and  dissemination  of  results  is  an  area  that 
needs  to  be  clearly  further  developed  to  support  awareness  raising  and  mutual  learning. 
Whilst the European Commission is very active in making brochures, newsletters and leaflets 
to spread the word, on other aspects of dissemination (e.g. the availability of study reports, the 
use of websites etc.) more could be done; 
Financial and administrative monitoring is normally well executed. At the same time it is 
desirable that monitoring becomes more substantive in nature; 
A  particular  area  where  there  is  substantial  room  for  improvement  is  gender 
mainstreaming. There is little evidence of strong support for the concrete implementation of 
this  principle  in  programme  management.  This  view  is  further  confirmed  by  poor  results in 
terms of numbers of activities which carry out a gender analysis prior to implementation or 
which disaggregate project data by gender, and limited examples of gender mainstreaming in 
activities funded in policy sections other than gender equality.  

Recommendations 

As for the different policy sections: 
Employment:  the  budget  for  policy  advice,  research  and  analysis  and  policy  debate  events 
should be increased. Provisions should be introduced that allow for more flexible responses to 
information needs. 
Social  protection  and  inclusion:  the  link  between  PROGRESS  and  the  Flagship  Initiative 
'European  platform  against  poverty  and  social  exclusion'  should  be  strengthened.  The 
methods  for  peer  reviews  used  in  the  SPSI  and  employment  sections  should  be  shared  and 
compared; the involvement of MS officials from public authorities in specialised communities 
of practice should be more targeted; the co-operation with social partners should be increased. 
Working  conditions:  it  should  be  clarified  how  PROGRESS  outputs  match  the  need  for 
initiating preventive actions and fostering a prevention culture in the field of health and safety 
at  work.  Tasks  on  evidence  production  should  be  further  streamlined  and  divided  with  EU-
OSHA  and  Eurofound;  external  access  to  studies  and  reports  should  be  made  easier; 
timeliness  of  report  availability  should  be  improved  by  streamlining  procurement  and 
improving  the  use  of  networks  of  experts;  more  attention  should  be  paid  to  gender  and 
discrimination  on  various  grounds  and  other  cross-cutting  issues  that  affect  working 
conditions. 
Antidiscrimination  and  diversity:    action  should  focus  on  the  development  of  comparable 
data and statistics for all EU27. Even in absence of new legislation, awareness-raising should 
be continued where necessary in relevant Member States; awareness around legal rights and 
obligations should be improved. 

EN 54 

Error! Unknown document property name.  EN 

 
 
 
 
Gender equality: focus on producing statistical tools/indicators should be increased, in order 
to provide an evidence base; support for the expert networks and the NGOs should continue; 
the  identification  and  dissemination  of  good  practice  should  be  strengthened,  with  more 
structured  follow-up  to  events  and  better  dissemination  of  successful  PROGRESS  outputs 
from other policy sections should contribute more to achieving a better integration of gender 
equality as a cross-cutting issue. 
As for horizontal themes: 
Programme  management:  a  multi-annual,  strategic  perspective  should  be  added  to  the 
annual cycle. Monitoring should be kept distinct from communication and should better trace 
back  outputs  to  activities  and  people.  A  central  outputs  storage  system  would  facilitate 
monitoring.  Additional  instruments  could  also  be  produced  to  communicate  PROGRESS` 
good  results,  such  as  thematic  best  practice  directories.  The  role  of  the  PROGRESS 
Committee  should  be  rethought:  the  policy  and  expert  committees  could  be  more  deeply 
closely  involved  as  this  would  allow  reinstating  the  discussion  on  policy  themes  that  was 
possible when separate funding programmes were attached to policy strands.  
Increasing dissemination of results is recommended for most policy fields, together with the 
recommendations  to:  develop  good  practice  dissemination  tools;  ensure  greater  diversity  in 
participation in events; widen stakeholder involvement in policy debate, and strengthen role 
of social partners; increase involvement of local and regional government as stakeholders.  
Gender mainstreaming: It is necessary to ensure that gender equality is mainstreamed in a 
meaningful  way  across  all  PROGRESS-supported  activity.  The  most  effective  way  to 
mainstream  gender  and  antidiscrimination  is  to  explicitly  incorporate  them  in  policy 
objectives, not just as a cross-cutting issue. 

6) Availability of the report on Europa 

http://ec.europa.eu/social/BlobServlet?docId=7430&langId=en 

Evaluation of the ESF support to Lifelong Learning 

1) ABB activities concerned: 04 02  

2)  Timing:  July  2012  (Duration:  October  2010-July  2012.  Period  covered:    programming 

period 2000-06 and the first four years of the period 2007-13). 

3) Budget:   

Overall  multi-annual  budget  of  the  ESF  for  the  2007-2013  programming  period  is  EUR  
76.6  billion 

4) Background, scope and focus:  

The purpose of the evaluation was to analyse the reach and impact of the ESF supported 
LLL  activities  on  young  jobseekers,  low  skilled  workers  and  older  workers.  The  report 
identified  activities  which  are  more  successful  in  reaching  these  target  groups  and  other 
factors of success. Furthermore, the evaluation looked for European Added Value which 
could be attributed to the ESF support to LLL activities. This information could improve 
the  content  and  delivery  of  the  ESF  supported  LLL  activities  aimed  at  the  three  target 

EN 55 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
groups in the 2014-2020 programming period. The work covers the EU27 and comprises 
an  EU-wide  overview  as  well  as  more  detailed  case  studies  on  eight  Member  States 
(Austria, Italy, Lithuania, Netherlands, Poland, Spain, Sweden and the UK).   

Methodology:  The evaluation is on the one hand based on the secondary analysis of the 
operational programmes, evaluations from these two programming periods, SFC 2007 as 
well as ESTAT data (such as the LFS) and LLL research, especially concerning the three 
target groups. On the other hand the consultants undertook own empirical work, mainly in 
the  form  of  interviews  with  ESF  managing  authorities  and  implementing  bodies  in  the 
Member States. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study 

ESF  is  a  major  funder  of  LLL  across  the  EU.  Approx.  EUR  55.3  billion  is  allocated  to 
expenditure categories related to LLL for 2000-13. An estimated 25 million young people, 
18 million individuals with low skills, and 1.9 million older people benefitted from ESF 
supported LLL activity across the EU between 2000 and 2010.  
Without  the  ESF  many  of  these  interventions  would  not  have  taken  place.  The  ESF  has 
allowed  more people to participate in LLL and has ensured that groups which otherwise 
would  not  have  been  reached  (in  particular  subgroups  among  the  low  skilled  and  the 
young) benefit from targeted interventions.  
While  all  MS  have  some  sort  of  LLL  strategy  in  place,  so  with  considerable  qualitative 
differences,  the  participation  of  the  adult  population  in  LLL  differs  between  1.2  %  in 
Bulgaria and 32.8% in Denmark.  
ESF support to LLL is closely linked with national policy priorities (such as high attention 
towards  transition  of  young  people  into  the  labour  market  and  a  much  higher  social 
acceptance  of  older  workers  leaving  the  labour  market  prematurely),  traditions  and  the 
skills level of the active population (e.g. in Italy and Spain more than 40% of the working 
age  population  have  at  most  ISCED  level  2,  while  in  Poland  or  Lithuania  the  figure  is 
below 15%).  
 The  report  shows  that  it  is  possible  to  reach  these  specific  groups,  but  they  need  to  be 
targeted and prioritised at the level of OPs and during the actual delivery.  
Delivery  arrangements  and  mechanisms  such  as:  managing  and  tracking  progression  of 
participants between ESF  activities, referral systems between different LLL activities, or 
long term strategies are important to achieve an impact in particular for the more difficult 
groups. Output targets linked to LLL are important for the success of the interventions.  
Young people  
Young people are in general overrepresented in ESF-supported LLL activities compared to 
their  proportion  in  the  working  age  and  active  population,  which  shows  that  the  ESF  is 
overall  well  equipped  to  reach  this  target  group.  They  account  for  more  than  40%  of  all 
ESF participations in Germany, Hungary and France and 30% or more in Spain, the UK, 
Poland, Malta, Belgium and Latvia. In contrast to that, it is 15% or less in Italy, Portugal, 
Cyprus and Greece, reflecting national policy decisions. Thereby e.g. the low participation 
in  Italy  contrasts  with  low  educational  attainment  of  a  large  group  of  young  people  and 
high youth unemployment.  

EN 56 

Error! Unknown document property name.  EN 

 
 
 
 
 
Activities  which  are  particularly  successful  in  attracting  and  helping  young  people  are: 
promoting  work  experience  (internships,  traineeships),  activities  to  support  progression 
into higher education (providing a taste of what further education is like), career guidance 
and  personal  action  plans.  Informal  approaches  and  social  activities  are  also  important 
elements.  
The  main  impacts  for  this  group  are  transition  into  employment,  to  encourage  further 
learning and to develop softer skills linked including increased self-confidence.  
Older workers   
With  less  than  5%  of  all  participations,  older  workers  are  underrepresented  in  ESF 
supported LLL. This is in line with their general participation in LLL. The proportion of 
older workers in all participations ranges from 12% or more in Estonia and Sweden to less 
than  2%  in  Greece  in  2007-10.  Attracting  older  workers  to  LLL  remains  a  rather  low 
priority  in  the  MS.  Even  in  cases  where  older  workers  are  targeted,  these  targets  are 
frequently not met and revised downward. While LLL is important also for this age group, 
relatively  low  levels  of  unemployment  amongst  this  age  group  provided  a  welcomed 
reason  to  not  strongly  focus  on  this  group.  Updating  skills  so  to  increase  job-security  is 
particularly  important  for  this  target  group.  It  also  needs  specific  motivation  and 
encouragement  for  older  workers  to  (actively)  participate  in  training  events,  as  their 
confidence in the ability to learn is often low.  
Low-skilled  
Low-skilled  workers  are  under  particular  pressure  in  the  labour  market,  they  count  for 
about  one  third  of  the  training  related  participations.  There  are  huge  variations  between 
Member States, ranging from more than 50% in Greece, France, Germany and Malta and 
Luxemburg to less than 15% in Cyprus, Slovenia and Estonia. Low skilled workers are not 
at  all  a  homogenous  target  group.  They  are  often  structured  into  subgroups  –  such  as 
people with a migration background or long-term unemployed with outdated skill sets, or 
as single mothers (with little professional experience).  
Furthermore,  well  qualified  people  become  low-skilled  when  they  have  been  out  of  an 
adequate  job  for  some  time.  Thus  this  group  is  rather  difficult  to  grasp  and  this  makes 
personalised instruments (guidance, job matching activities, up skilling in specific sectors) 
particularly important.  
Positive  impacts  of  ESF  interventions  for  this  group  are  finding  a  new  job  but  also 
increased job-security and quality (by achieving a formally acknowledged qualification) as 
well  as  an  increased  willingness  to  undertake  further  training  and  –  more  generally  – 
increased confidence and self-esteem. 

6) Availability of the report on Europa: 

http://ec.europa.eu/social/BlobServlet?docId=7922&langId=en 

EN 57 

Error! Unknown document property name.  EN 

 
 
Development of a methodology for the systematic evaluation of Health and 
Safety at Work Directives and evaluation of the practical implementation of 
Directive 89/654/EEC "Work Places" testing the new evaluation methodology - 
Legislation evaluation 

1) ABB activities concerned: 04 04 

2) Timing: 29.04.2012 

3) Budget: not applicable 

4) Background, scope and focus: 

The objective of the contract was to develop a methodology for the systematic evaluation 
of OSH Directives and testing it in a pilot evaluation of Directive 89/654/EEC concerning 
the minimum safety and health requirements for the workplace.  

The  evaluation  methodology  and  the  test  on  Workplace  Directive  89/654/EEC  will 
contribute to the upcoming ex-post evaluation of the implementation of the EU legislation 
on Health and Safety at Work in the EU Member States. 

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study 

Development  of  common  methodology  for  future  evaluations  of  OSH  legislation 
(including the ex-post evaluation ending 2015). The Generic Methodology report presents 
a  viable  generic  evaluation  approach  based  on  a  comprehensive  set  of  questions 
(indicators)  using  available  documents  accompanied  by  focused  surveys/interviews. 
Although no "turn key" method was provided that can be imposed as the only one and sole 
approach for the evaluation of health and safety at work directives, it provides one possible 
and workable evaluation approach as well as many useful components which can inform 
future  evaluation  efforts.  The  second  objective,  namely  the  cross-nationally  comparative 
evaluation of the Workplace Directive 89/654/EEC in all 27 Member States and a sample 
of EFTA-EEA countries, has also been achieved. The report contains a fully-fledged and 
comprehensive evaluation of the impact of Directive 89/654/EEC. The main findings are: 

Initial relevance 
The  goal  of  the  Directive  was  clearly  to  progressively  improve  the  level  of  occupational 
safety  and  health  through  the  harmonisation  of  already  existing  national  regulations.  In  a 
broader  sense,  the  respondents  agreed  on  the  high  initial  relevance  with  regard  to  the 
importance of the requirements of the WPD. Those requirements provide a fundamental basis 
for guaranteeing occupational safety and health at the workplace. 

Quality of the preparation 
On a general level, the good quality of the preparation of the WPD is widely accepted by the 
stakeholders.  The  majority  of  stakeholders  agree  that  the  WPD,  in  general,  targets  relevant 
and basic OSH aspects and that the requirements are clearly formulated. The level of detail is 
sometimes considered as not sufficient but some countries already had details in their national 
regulation  or  added  details  when  transposing  the  WPD,  or  published  practical  guides  and 

EN 58 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
recommendations for the implementation at company level. 

Implementation 
In many countries, the legal implementation did not change the national regulations to a large 
extent. Many pre-existing legislative frameworks already covered the legislative scope of the 
WPD. In some cases, the transposition made it possible to modernise the existing legislation 
and  add  some  missing  provisions.  Most  of  the  stakeholders  mentioned  that  the  transposed 
requirements were relevant for their national regulation. Overall, the transposition was not the 
subject of a controversial national debate. 

The  practical  implementation  (compliance  with)  of  WPD-related  OSH  obligations  can  be 
considered  as  ‘good’.  According  to  the  analysis  of  the  collected  data,  the  level  of 
implementation  of  the  specific  WPD-related  requirements  was  better  than  the  level  of 
implementation  of  the  general  provisions  (such  as  risk  assessment,  information,  workers’ 
consultation and training). The global compliance seems therefore to lead to lesser results. 

Impact 
OSH  results  are  very  difficult  to  measure  in  a  quantitative  way,  but  the  perception  of  the 
results  among  stakeholders  tends  to  indicate  a  slightly  positive  result.  A  number  of 
stakeholders were convinced that the WPD generally contributed to the improvement of the 
working conditions. 

Effectiveness 

If  we  compare  the  initial  relevance  with  the  overall  impact,  we  can  conclude  that  the 
Directive has proven its effectiveness. However, it may vary from one country to another 
as this is related to the corresponding pre-existing legal framework. 

Current and future relevance 
There were few suggestions for changes to the WPD in the future. An important part of the 
stakeholders,  employers  and  workers  argued  that  no  changes  were  needed.  The  practical 
improvements on specific topics were linked to the level of detail and concreteness of the 
provisions.  Suggestions  to  include  additional  provisions  relate  to  psychosocial  risks, 
ergonomic design, indoor air quality or specific types of mobile workplaces (referring to 
specific work situations such as telework, transportation). 

6) Availability of the report on Europa 

http://ec.europa.eu/social/BlobServlet?docId=8761&langId=en 

Evaluation of flexicurity 2007-2010 

1) ABB activities concerned: 04 01  

2) Timing: December 2012 (Period covered 2007-2010). 

3) Budget: not applicable 

EN 59 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
4) Background, scope and focus:  

The objectives of the evaluation were twofold:  

To  assess,  more  than  three  years  after  their  publication/adoption  the  impact  of  the 
• 
Communication “Towards Common Principles of flexicurity” and the Common Principles 
on national policy making in the EU Member States; and 

To  analyse  the  impact  of  the  implemented  integrated  flexicurity  policies  on  labour 
• 
market outcomes at EU and national level in terms of strengthening employment security, 
facilitating transitions and reducing labour market segmentation. 

The  evaluation  covered  the  27  Member  States  looking  at  the  time  frame  between  the 
publication  of  the  two  key  Commission  Communications  on  Flexicurity  so  far  i.e. 
“Towards Common Principles of flexicurity”, adopted in June 2007, and “An Agenda for 
new  skills  and  jobs”,  adopted  in  November 2010. However,  the  evaluation  also  includes 
developments occurring after November 2010 when they resulted in significant changes in 
national policy making, such as Spain’s February 2012 labour market reforms. 12 Member 
States have been subject to in-depth case study research.  

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study 

The evaluation recommends the Commission to: 

−  Bring the debate and lessons on the implementation of flexicurity from a conceptual 
to  a  practical  level,  by  better  exploring  ways  to  replicate  evaluated  best  practices  in  the 
area of flexicurity; 

− 
Stress  balanced  approaches  of  policies  promoting  flexibility  and  security  in  labour 
markets, by shifting the emphasis from flexicurity (which is the outputs / component parts) 
to  ‘mutual  responsibility’  and  to  the  concept  of  policies  aimed  at  restoring  "dynamic 
equilibria"; 

−  Emphasise  the  link  between  a  balanced  approach  to  flexibility  and  security  and  the 
wider  economic  policy  framework  by  giving  greater  attention  to  the  interaction  between 
economic  and  labour  market  policy  (although  this  is  highlighted  in  the  processes  and 
outputs  of  the  European  semester  at  EU  level,  the  link  is  currently  less  evident  in  the 
National Reform Programmes and ought to be emphasised further); 

−  Make better use of the Open Method of Co-ordination to emphasise flexicurity related 
policy recommendations by using OMC better in the process of the European Semester to 
address implementation of country specific recommendations; 

−  Demonstrate the added value of the strong involvement of social partners by further 
action at Member State level to foster and achieve greater involvement, including through 
capacity building as well as the establishment of suitable avenues for interaction; 

−  More  “telling”  and  monitoring  of  indicators  by  selecting  and  suggesting  most 
“telling”  indicators  to  be  monitored  more  regularly  to  provide  assessment  of  the  key 

EN 60 

Error! Unknown document property name.  EN 

 
policies and their impact on labour market outcomes. 

One observation to be made on the evaluation is that the time covered would fall short of 
important reforms that have indeed been undertaken by a number of Member States in the 
flexicurity area after 2010. The report acknowledges that. But a thorough assessment felt 
outside the mandate of the evaluation. 

6) Availability of the report on Europa 

http://ec.europa.eu/social/BlobServlet?docId=10018&langId=en 

 Mid-term evaluation of the European globalisation fund (EGF)  

1) ABB activities concerned: 04 05  

2) Timing: 8 December 2011 (duration: April 2011-November 2011. period covered January 
2007- April 2009). 

3) Budget: overall multi-annual budget of the evaluated Programme:  500 million EUR per 
year (7 years). 

4) Background, scope and focus:  

The purpose of the mid-term evaluation was to assess the effectiveness and sustainability 
of the EGF through analysis at two levels: (a) the EGF instrument (b) the level of the 15 
individual  cases  implemented  between  January  2007  and  June  2009.  The  latter  also 
included an assessment at the level of the individual measures funded by the EGF in each 
of these cases and their beneficiaries. The mid-term evaluation was required by Article 17 
of  the  EGF  Regulation  (Regulation  (EC)  No  1927/2006),  which  stipulates  that  an 
evaluation on the effectiveness and sustainability of the results obtained by the EGF must 
be delivered by 31 December 2011. 

5) Summary of findings and conclusions of the evaluation or evaluation related 
     study: 

The  headline  findings  and  conclusions  on  the  effectiveness  and  sustainability  both  the 
EGF as an instrument and the performance of the individual cases are: 

(i)  The  average  gross  re-employment  rate  was  41.8%  and  ranged  from  78%  to  4%  in 
individual  cases.  In  the  longer  term,  re-employment  rates  tended  to  rise  after  12  months 
after the closure of EGF assistance. 

(ii)  Stronger  performing  cases  tend  to  be  associated  with  the  provision  of  personalized 
support  for  workers  and  tend  to  be  complemented  by  strong  further  national  support 
measures.  

EN 61 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
 
 
(iii)  Unsurprisingly,  the  subjective  assessment  of  the  value  of  EGF  support  is  relatively 
positive among participants, however, it is noteworthy that independent experts also have a 
quite positive qualitative assessment of the value of EGF action.  

(iv)  Apart  from  the  direct  re-employment,  EGF  assistance  also  generated  other  positive 
effects,  in  particular  in  the  form  of  distance  travelled  towards  the  labour  market  for 
individuals (increased confidence, improved job search and other skills). 
(v) EGF budget absorption varies widely across the cases and it is not possible to 
generalize about the contributing factors. 

In interpreting the findings it should be borne in mind that: 
(i) EGF support can sometimes arrive in a locality a significant time after job-losses occur. 
In  such  cases,  some  redundant  workers  had  already  re-entered  the  labour  market  while 
those  remaining  tend  to  have  a  'harder-to-help'  profile.  The  theme  of  more  responsive 
procedures is reflected in the recommendations. 

(ii)  The  evaluators  found  that  lower  educational  levels  were  associated  with  those  cases 
where  re-employment  rates  were  lowest,  while  higher  re-employment  rates  tended  to  be 
associated  with  areas  with  lower  general  unemployment  and  more  favourable  economic 
conditions. 

The report makes a number of recommendations on EGF 'architecture' and processes at 
the level of the instrument: 

(i) Maintain the current triggering threshold of 500 redundancies 

(ii) Promote awareness that workers made redundant outside the 'reference period' can be 
considered as eligible. 

(iii) Increase flexibility of implementation 

(iv) Review possibilities to speed up the application process 

(v) Strengthen the capacity for submitting applications at national and regional/local level, 
and provide further opportunities for sharing lessons and good practices  

(vi) Clearer monitoring requirements, including follow-up of beneficiaries, and financial 
means to carry out this monitoring. 

(vii)  Provide  practical  advice  on  monitoring  and  reporting,  and  support  exchange  of 
good practice 

(viii) Greater local dissemination efforts to raise the visibility of the EGF 

(ix) Ensure communication between institutional stakeholders   

6) Availability of the report on Europa:  

EN 62 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
 
 
 
 
http://ec.europa.eu/social/BlobServlet?docId=7326&langId=en 

Evaluation of Public Employment Services performance measurement systems 
and corresponding recommendations on geographical labour mobility indicators 

1) ABB activities concerned: 04 03   

2) Timing: 15.05.2012, (Period covered by the report: December 2010 – March 2012,) 

3)  Budget:  Multi-annual  budget  for  EURES  activity  plans  (2006-2013):  EUR  100,000,000 
(ca. EUR 14,200,000 on a yearly basis) 

4) Background, scope and focus:  

In the framework of the preparation of the EURES reform to contribute to the objectives of 
the EURES 2020 strategy, it was deemed necessary to obtain a clear picture of the Public 
Employment Services (PES) performance measurement systems and indicators to monitor 
transnational and cross border geographical  labour  mobility. The PES being members of 
the EURES network, there is a considerable degree of diversity in their monitoring tools.  

The study was thus aimed at a) carrying out a comparative analysis PES monitoring and 
performance  systems  and  b)  mapping,  compiling  and  proposing  suitable  qualitative  and 
quantitative indicators. The scope has been the transnational and cross border geographical 
labour  mobility  activities  implemented  by  the  PES  in  the  framework  of  the  2010-2013 
EURES guidelines.  

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study: 

EURES is undergoing a reform with a view to transform the network into an economic and 
result-oriented  EU  job  mobility  tool,  with  the  potential  to  contribute  to  achieving  the 
employment  and  economic  growth  milestones  of  the  Europe  2020  strategy.  Apart  from 
information  and  advisory  activities  related  to  the  free  movement  of  workers  in  the  EU, 
EURES is deemed to enhance its performance when it comes to recruitment, job matching 
and job placement, i.e. it should help jobseekers to find work in other EEA countries and 
employers to find workers across borders. 

In  this  context  the  study  gave  an  EU-wide  insight  in  the  existing  PES  performance 
measurement systems; it examined their strong and weak aspects as well as the areas for 
further  development.  The  main  deliverable  of  the  final  study  report  has  been  a  toolbox 
with  commonly  accepted  output  and  result  indicators,  tailored  to  enhance  employment 
services  effectiveness  and  efficiency  when  it  comes  to  support  intra-EU  job  mobility. 
Some of these indicators will be included in the EURES business model as from 2014.  

EURES will be more accountable under the coming 2014-2020 MFF namely in terms of 
quantitative results.  Consequently, the study findings are timely appropriate to feed into 

EN 63 

Error! Unknown document property name.  EN 

 
 
 
 
  
 
 
 
 
 
 
 
the on-going design of the post-2014 EURES operational set up. This will of course have 
an impact on the PES data sources and management tools to be used across the network. 

6) Availability of the report on Europa 

http://ec.europa.eu/social/BlobServlet?docId=7876&langId=en 

ENEA preparatory action on active ageing and mobility of elderly people 

1) ABB activities concerned: 04 03 

2) Timing: Period covered by the report: 2006-2011 

3) Budget: The total sum of grants for all ENEA projects amounts to EUR 3.6 million. 

4) Background, scope and focus: 

The  preparatory  action  was  launched  at  the  initiative  of  the  European  Parliament.  The 
action  intended  to  finance  measures  to  "encourage  the  establishment  of  exchange 
programmes for the elderly through specialised organisations tasked with developing, inter 
alias,  resources  for  mobility  and  the  adaptation  of  infrastructures".  These  exchange 
programmes  should  also  promote  the  European  mobility  and  freedom  of  movement  of 
older  persons,  enabling  them  to  develop  themselves  culturally  and  helping  them  to  fulfil 
their role vis-à-vis other people and other generations. 

The evaluation of the ENEA projects aimed at assessing the feasibility and usefulness of 
mainstreaming  of  this  preparatory  action  into  an  EU-programme,  on  the  basis  of  the 
knowledge, know-how and experiences generated in the ENEA projects.  

The scope of the evaluation focussed on the effectiveness, efficiency and added value of 
these  ENEA  cooperation  and  exchange  projects.  It  also  assessed  how  sustainable  the 
realised  impacts  were  and  what  were  the  main  factors  behind  success  or  failure  of  each 
project.  

The following target groups were addressed: 
• In terms of age groups: 55 years and older (55+), 50 years and older (50+) 
•  In  terms  of  social  status:  unemployed  persons,  elder  workers  /  employees,  retired 
persons, 
educationally 
disadvantaged, low financial means, socially excluded), persons with low language skills, 
elder migrants 
• Stakeholders such as employers, authorities/ decision makers and organisations 
as well as people living around migrants. 

senior  volunteers,  disadvantaged  persons 

(low-skilled, 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study 

Those ENEA projects which aimed at increasing employment struggled with a number of 
(systemic) obstacles: The different labour market and social security regulations hampered 

EN 64 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
 
 
the possibilities of working abroad and this has not always been anticipated in the project 
planning.  Also  the  geographic  differences  in  work  cultures  have  been  underestimated. 
These  problems  were  reinforced  by  the  economic  crisis  making  companies  reluctant  to 
offer  internships  and  recruiting  employees.  Thus,  the  projects’  effectiveness  in  terms  of 
enhancing  employability  was  rather  low  and  even  weaker  in  terms  of  a  concrete  re-
integration into the labour market. The attempt of a transnational job matching for elderly 
unemployed turned out to be too ambitious. 

By comparison, the promotion of volunteering among elderly persons was more successful 
and  effective.  A  higher  involvement  of  the  target  group  in  voluntary  activities  could  be: 
not  only  the  participants  were  willing  to  engage  in  voluntary  but  also  the  volunteer 
organisations  have  become  aware  of  the  potential  of  elderly  persons  as  volunteers.  In 
addition, ENEA seems to have contributed to initiating a change of mind-set in favour of 
voluntary  work  in  the  Eastern  European  countries  and  resulted  in  a  know-how  transfer 
from the old Member States’ third sector organisations to the new Member States’ NGOs. 

The  projects  promoting  the  integration  of  older  migrants  dedicated  their  work  to  a 
relatively new topic: Both the fact that migrants often stay in their host country beyond the 
retirement  age  and  people  migrate  after  retiring  to  another  Member  State  have  not  been 
considered in many policy schemes up to now. Thus, ENEA functioned to bring this issue 
on  the  agenda.  Additionally,  the  project  which  worked  with  elderly  migrants  arrived  at 
activating  the  target  group  in  general  terms  (integration  in  social  life,  take  up  of  LLL 
activities). The other project - which did not involve the target group directly -contributed 
to raising the awareness of the stakeholders. 

The using of cross-border mobility in the form of international exchanges can only be 
recommended  under  certain  conditions:  A  successful 
international  exchange 
programme  (especially  for  older  people)  requires  a  detailed  preparation  of  the  target 
group. Intercultural training, workshops for personal development, and a preparation in 
practical  issues  are  to  be  mentioned.  Detailed  planning  is  a  prerequisite  and  refers  to 
travel costs and cost of living abroad but also the management of the running expenses 
of the participants in their home country during the exchange.  

The ENEA programme addressed a wide range of issues regarding active ageing. This 
goes in line with the character of the programme as preparatory action, but implied a 
set  of  targets  where  rather  different  approaches  were  required.  Future  programme 
designs should ensure that projects focus on a more specific (or homogeneous) target 
group and respectively on a specific target.  

The evaluators suggested that focussing on the border regions seems to be most promising 
in terms of the international exchange programmes and cross-border mobility projects. 

6) Availability of the report on Europa: 

     Not yet available. 

EN 65 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
ESF in the future: Measuring the impact of changing regulatory requirements 
on administrative cost and administrative burden of managing the European 
Social Fund 

1) ABB activities concerned: 04 02  

2) Timing: 12 June 2012 (Period covered by the report: 2000-2012) 

3) Budget:  

Overall budget of the ESF for the 2007-2013 programming period is EUR  76.6 billion 

4) Background, scope and focus:  

The aim of the study was to analyse the administrative costs (costs for personnel, costs 
for  external  services  which  are  bought,  and  overhead  costs,  including  costs  arising  from 
Information  Obligations).  Moreover,  it  studied  the  administrative  burden,  which  is  the 
cost  of  administrative  activities  that  organisations  conduct  only  because  of  ESF.  In 
addition, the study analysed the impact of the introduction of the Simplified Cost Option. 

The  study  had  to  provide  overall  conclusions  and  suggestions  for  future  2014-2020  ESF 
regulations based on the analysis of major drivers of administrative costs of the ESF in the 
2000-2006 and 2007-2013 programming period. 

The study surveyed 12 ESF 2007-2013 Operational Programme Authorities in seven case 
study Member States: Belgium, Czech Republic, Finland, Italy, Malta, Poland and UK 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study: 

• 
Total  administration  costs  across  the  seven  MSs  amount  to  EUR1,227,242,000  or 
4.8%  of  the  total  ESF  2007-2013  budgets  (however,  it  ranges  from  2%  to  13%  of  total 
ESF 2007-13 programme budgets across the seven MSs); 

•  When compared to similar programmes run by the World Bank, IMF, United Nations 
and other programmes, levels of ESF administrative costs appear reasonable. The range in 
the level of administrative costs of these comparable programmes is from 3% (in the case 
of  the  ERDF/CF)  to  12%,  with  other  national  public  sector  programmes  having  much 
higher  levels  (for  example  the  administration  of  healthcare).  The  administrative  costs  of 
ESF, therefore, are towards the lower end of the spectrum; 

• 
ESF  tends  to  have  higher  administrative  costs  than  ERDF/CF  (Table  1),  spending 
however  less  on  National  Strategic  Policy  Level  ,  Programme  Certification  and 
Programme Audit; 

• 
ESF administration costs are two percentage points higher than of ERDF/CF (4.8% to 
2.6%)  reflecting  ESF’s  large  numbers  of  projects,  beneficiaries  and  participants,  and  the 
average smaller size of projects. 

• 
National  Strategic  Policy  Level 

39  specific  administrative  tasks  were  organised  into  five  administrative  groups: 
(NSPL);  Programme  Preparation;  Programme 

EN 66 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
Management; Programme Certification; and; Programme Audit (Table 2). 

• 
The  large  majority  of  costs  (85%)  were  within  the  Programme  Management  tasks  - 
Information  and  publicity  requirements  (18.5%);  Verification  of  deliverables  and 
compliance  (10.2%);  Ensuring  a  system  for  data  recording  (7.9%),  and;  Selection  of 
operations (5.3%). Guiding the work of the monitoring committee falls just below the 5% 
threshold (4.8%);  

• 

Beneficiary administrative costs ranged between 4% and 8% of total ESF support;  

• 
Based  on  the  definition  of  administrative  burden  used  in  the  Study  74%  of 
administrative  costs  are  administrative  burden.  However,  levels  of  administrative  burden 
are highly sensitive to how burden is defined and measured. If a strict measure of the term 
is used (i.e. any activity defined in the regulations, excluding ‘other’ administrative tasks, 
overheads  and  national/regional  regulations)  levels  of  burden  would  appear  to  be  even 
85%. Similarly, if the measure takes into account ‘business as usual’, which is expected to 
be variable across Member States depending on the degree to which their national systems 
either mirror or diverge from EU practices, levels of administrative burden would be lower 
(41%); 

Larger  OPs  lead  to  lower  administrative  costs  and  workload.  However,  the 
• 
relationships  between 
level  of  administrative  costs  and  geographical  coverage, 
management experience, governance and OP complexity were not found by this Study to 
be  significant.  Levels  of  administrative  costs  for  Audit  Authorities  and  Certifying 
Authority,  despite  having  similar  administrative  functions,  vary  considerably  between 
OPs. The same for the levels of external, overhead and national regulation administrative 
costs; 

• 
The  availability  of  SCOs  had  generally  been  positively  received  by  OP  Authority 
pointing  out  both  their  advantages  and  disadvantages.  There  is  also  evidence  to  suggest 
that  the  flat  rate  and  unit  cost  SCOs  have  reduced  administrative  costs,  in  some  OPs. 
Respondents believed that the overall level and costs of ESF administration had increased 
compared to the 2000-2006 ESF Programme. 

The study has been completed in the middle of 2012, after the cohesion policy legislative 
package had been presented by the Commission. However, the outcomes of the study will 
be used during the negotiation process as well as during the preparation of the Operational 
Programmes.  

The analysis has proved that the overall administration costs are reasonable, but indicated 
that further efforts to reduce administrative costs and burdens should be targeted on those 
administrative  tasks  which  account  for  the  largest  proportion  of  administrative  costs  and 
workload, and therefore are likely to have a bigger impact (e.g. information and publicity 
requirements  (18.5%  of  administrative  costs)).  However,  further  reductions 
in 
administrative  costs  and  burdens  must  not  be  at  the  expense  of  positive  aspects  of 
programme administration. For example, selection of operations, is designed to ensure the 
effective  targeting  of  resources  where  need  is  greatest,  and  influences  the  quality  of 
operations. Reducing administrative costs may therefore reduce the cost effectiveness of a 
programme.  

It has not been possible for this Study to undertake a full assessment of the impact of the 
SCOs on administrative costs in the 2007-2013 ESF programme. At the time of the Study, 

EN 67 

Error! Unknown document property name.  EN 

 
for  many  OPs,  SCOs  had  been  implemented  partially  or  belatedly,  with  only  limited 
impact  measurable  within  the  timeframe  of  the  Study.  However,  according  to  the  study 
SCO  alone  have  not,  and  are  unlikely  to,  reduce  administrative  costs  and  burdens 
significantly. 

6) Availability of the report on Europa: 

http://ec.europa.eu/social/BlobServlet?docId=9119&langId=en 

ESF Expert Evaluation Network: Access to Employment and Social Inclusion 

1) ABB activities concerned: 04 02  

2) Timing: October 2012, (Period covered by the evaluation: 2007-2013) 

3) Budget: Overall budget of ESF for 2007-2013 programming period is EUR 76.6 billion. 

4) Background, scope and focus:  

The  Expert  Evaluation  Network  (EEN)  involves  one/two  national  expert(s)  in  each 
Member  State.  They  collect  and  analyse  all  evaluations  related  to  ESF  Operational 
Programmes  carried  out  by  the  MS  in  the  2007-2013  programming  period  to  produce 
Inventories  of  evaluations,  Country  Reports  for  all  27  MS,  and  Synthesis  Reports  at  EU 
level.  The  ESF  EEN  was  launched  in  2011,  and  lasts  until  end  of  2013.  In  2012  two 
thematic semesters were conducted. The first on “Access to Employment”, the second on 
“Social Inclusion”. The reports cover ESF related evaluations conducted so far in the 27 
MS, related to the programming period 2007-2013. 

5) Summary of findings and conclusions of the evaluation or evaluation related  
    study: 

Report on Access to Employment: 

Key Figures: 
•  Around EUR23 billion committed  
•  Over 12.5 million final recipients supported  
•  Over 2.4 million final recipients achieved an employment result 
• 

but little evidence on sustainability of employment  

Good practices identified in some MS: 
•  Designing interventions to reflect needs and assets of individuals and priority groups  
• 
•  Building strong partnerships between agencies and beneficiary organisations 

Engaging effectively with employers  

Factors helping effective implementation: 
•  Opportunity to refocus OPs after recession  
•  Good quality staffing, systems and management arrangements  
•  National Employment Service with strong capacity 

EN 68 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
 
 
 
 
 
People with multiple disadvantages need more intensive and/or lengthy interventions 
Important value of key worker/personal adviser working closely with final recipients 

Impact of economic crisis on labour market opportunities  
Poor design of some key areas of intervention (no baseline, no quantification of 

Learning Points: 
• 
• 
on one to one basis 
•  More support needed for final recipients after they enter employment, particularly 
given weak labour market 
•  Main constraints: 
• 
• 
targets, etc…) 
•  Despite its importance, evaluation evidence on is often fragmented and lacking 
robustness 
• 
beneficiary organisations (not always reliable and correct) 
•  With rising numbers of unemployed and greater pressure on budgets, big challenge 
for ESF around balance between helping those closer to and those further from labour 
market 
•  More evaluation evidence needed around this specific issue to guide policy makers 
•  Counterfactual evaluations are very rare. Need for more impact evaluations 

Simple indicators (i.e. initial job entry) too much dependent on exit data supplied by 

Report on Social Inclusion: 

Key figures: 
•  Nearly EUR24 billion committed.  
•  Over 14.5 million final recipients engaged, although this might involve double 
counting as SI final recipients engage in different interventions.  
•  Broad range of target groups: over 2.4 million unemployed, over 1.3 million 
migrants, and over 1 million young people have been supported by SI interventions.  

Particularly effective interventions: 
•  Working in schools with young people at risk of early leaving or at the point of 
transition from school to work 
• 
Interventions with families from Roma communities 
•  Activities to bring adults back into the education system 

Intelligent design of interventions using evaluation evidence but also involving final 

Factors supporting effective implementation:  
• 
Focussing on the needs of individuals.  
•  Well integrated interventions and services. 
• 
recipients themselves.  
•  More emphasis on early interventions, for example to reduce early drop out from 
school. 
•  Responding flexibly to changing economic circumstances.  
•  Building the capacity of the organisations designing and delivering interventions.  
•  Key support services (drug counselling, money advice, etc.) 

EN 69 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
•  Raising awareness and changing attitudes and behaviours (e.g. towards disabled 
people) among employers and key agencies. 

The impact of the economic crisis is pushing socially excluded further back in the 

Little robust evidence. In particular on soft results (recognizing the difficulty of 

Main constraints:  
• 
measuring these) this is particularly lacking.  
• 
queue for job opportunities 
•  Difficulties in engaging with hardest target groups to reach 
• 
targets, etc…) 
• 
•  Counterfactual evaluations are very rare. Need for more impact evaluations  
Need to know what types of interventions produce significant net effects 

Poor design and delivery of interventions 

Poor design of some key areas of intervention (no baseline, no quantification of 

6) Availability of the report on Europa: 

http://ec.europa.eu/social/BlobServlet?docId=8954&langId=en 

EN 70 

Error! Unknown document property name.  EN 

 
 
ABB 05 – Agriculture and Rural Development 

Evaluation of the Common Agricultural Policy measures applied to the cereals 
sector 

1) ABB activities concerned: 05 02 and 05 03 

2) Timing: December 2012 (covered the period from 2000 onwards)  

3) Budget:  

This evaluation covered several instruments and their related budget, as it focuses on the 
combined  impact  of  policies  in  one  particular  sector  within  agriculture  (cereals).  The 
evaluation  covered  the  phasing  in  of  the  single  payment  scheme  (SPS)  and  single  area 
payment scheme (SAPS), which removed the link between the obligation of cultivating a 
particular  crop  and  payment.  For  the  whole  of  the  agricultural  sector,  in  2010  SPS 
payments amounted to EUR 29.07 billion, SAPS payments to EUR 4.46 billion. In the last 
decade, cereals accounted for between 31% and 33.5 % of the Utilised Agricultural Area 
in the EU.  

4) Background, scope and focus:  

The evaluation, which was carried out in the context of the 6 year cycle stipulated by the 
Financial Regulation, covered all direct support schemes laid down in Council Regulation 
(EC) No 1782/2003 and the changes introduced by Council Regulation (EC) No 73/2009 
where these could already be observed. It also covered Regulation (EC) No 1234/2007 on 
the  Single  CMO  in  so  far  as  it  affected  the  cereals  sector,  and  its  predecessor,  Council 
Regulation (EC) No 1784/2003 on the common organisation of the market in cereals. The 
changes introduced by Council Regulation (EC) No 72/2009 were examined as far as the 
effects of these changes could already be observed. 

In practice for most data the year 2010 was the last for which information was available. 
Thus, it covers the period before the introduction of decoupled direct payments (decided 
in  the  2003  'mid-  term  review'  CAP  reform),  the  phasing  in  (2004-2006)  and  the  post-
reform period. 

The evaluation covered all 27 Member States and focussed on the period from 1 January 
2005 onwards. However, for analytical reasons, data going back to 2000 were used. The 
analysis  focussed  on  the  following  crops:  common  wheat,  durum  wheat,  maize,  barley, 
rye, oats and triticale. 

5) Summary of findings and conclusions of the evaluation: 

The  evaluation  concluded  that  taken  as  a  whole,  the  reforms  have  encouraged  an 

EN 71 

Error! Unknown document property name.  EN 

 
 
 
increasingly  market  oriented  approach  to  cereal  growing,  without  reducing  support  to 
producers,  endangering  the  supply  for  users,  or  placing  an  increased  burden  on  the 
environment. The reforms have been relatively efficient, in that the cost of cereal- specific 
measures has declined, while national payment agencies and, to a lesser extent, producers 
see the administrative burden as manageable. Increasingly, support for producers and for 
protection of the environment is driven by Pillar II schemes, which was outside the direct 
scope of this evaluation. 
The measures have promoted the development of cereal crops and end-uses in which the 
EU-27  has  a  comparative  advantage.  Common  wheat  is  the  cereal  in  which  this 
comparative  advantage,  judged  by  international  cost  competitiveness,  is  greatest.  The 
reforms  included  the  decision  not  to  grant  export  refunds,  which  meant  that  WTO 
Uruguay Round limits upon subsidised exports were less relevant. 

The reforms promoted efficiency and the outcomes were both coherent and relevant. The 
exceptions  tended  to  be  in  sectors  where  measures  worked  against  the  emergence  of 
comparative advantage. The retention of some durum wheat coupled aids (including those 
via Article 68 payments) created deadweight and failed to prevent a substantial decline in 
output  in  traditional  areas.  The  other  notable  exception  affecting  cereals  was  a 
consequence  of  non-CAP  measures,  namely  the  excessive  promotion  of  silage  maize 
cultivation in some Member States in response to national incentives and the Renewable 
Energy Directive. 

6) Availability of the report on Europa: 

http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/2012/cereals-
2012/fulltext_en.pdf 

EN 72 

Error! Unknown document property name.  EN 

 
Evaluation of the Common Agricultural Policy measures applicable to the wine 
sector 

1) ABB activities concerned: 05 02 and 05 03 

2) Timing: October 2012 (period covered 2001-2011 with a focus on the impact of the 2008 
reform) 

3) Budget:  

From 2004 until 2008, year of the reform, the spending for the common organisation of 
the  market  for  wine  had  increased  from  around  EUR  1.1  billion  to  EUR  1.5  billion  per 
year. Since then, it has decreased to around EUR 1.3 billion per year. 

4) Background, scope and focus:  

The  evaluation  assesses  the  effects  of  the  2008  reform  of  the  wine  sector  .  The  reform 
represents a shift from a support system based on intervention and measures for limiting 
production  to  a  system  aimed  at  decoupling  and  with  a  reduced  number  of  regulatory 
instruments. 
The evaluation covers the years 2001-2011, distinguishing between the period before and 
after  the  implementation  of  the  2008  reform.  It  covers  the  18  wine  producing  Member 
States  and  was  based  on  detailed  case  studies  of  wine  producing  regions  and  the  UK 
market. 

5) Summary of findings and conclusions of the evaluation: 

This  evaluation  concluded  that  there  is  overall  coherence  between  the  objectives  of  the 
reformed  wine  CMO  and  the  principles  of  the  2003  CAP  reform,  as  well  as  the  overall 
objectives  of  EU  agricultural  policy.  However,  it  observed  some  inconsistency  between 
the  measure  for  promotion  on  third  country  markets  (promotion  of  private  brands)  and 
those of Regulation (EC) No 3/2008 (allowing generic promotion).The implementation of 
measures  in  the  form  of  national  support  programmes,  allowing  the  flexibility  and 
adaptability  of  support  measures  to  local  needs  of  wine  sectors,  has  been  effective. 
However,  some  problems  limiting  the  effectiveness  (and  efficiency)  have  appeared,  but 
these are related to the policy management. The evaluation concluded that the objective of 
simplification seems not have to been achieved with the current programming approach. 

The evaluation concluded that for the stabilisation of the wine market grubbing-up is more 
efficient than the distillation measures, in particular crisis distillation. Moreover, in Sicilia, 
crisis distillation is more efficient than green harvesting. 
Regarding the objective of stabilising producer incomes, in Spain aid for potable alcohol 
distillation is efficient. In the same MS, the single payment, which substitutes the aid to 
potable  alcohol  distillation,  is  efficient  (but  not  sufficient)  for  growers  who  continue 
producing  for  potable  alcohol  distillation,  whereas  it  is  not  efficient  if  they  have  re-
orientated production towards non-protected designations of origin wines. The latter case 

EN 73 

Error! Unknown document property name.  EN 

 
 
 
is  likely  to  distort  competition  between  wine  growers  receiving  the  single  payment  and 
those who are not entitled. 

Regarding the objective of strengthening the competitiveness of EU producers and wines, 
the  measures for "conversion/restructuring of  vineyards" and "investments" are efficient, 
but the terms of implementation in some Member States/Regions have reduced efficiency. 

A judgment on the efficiency of the measure "promotion on third country markets" is less 
straightforward,  due  to  factors  related  to  implementation  strategies  and  rigidity  of 
procedures adopted by some MS/regions. 

6) Availability of the report on Europa: 

http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/wine-2012_fr.htm 

Evaluation of the European School Fruit Scheme 

1) ABB activities concerned: 05 02 

2) Timing: October 2012, (period covered: school years 2009/2010 and 2010/2011.  

3) Budget:  

The maximum EU contribution to the financing of the scheme is EUR 90 million per 
school year. 

4) Background, scope and focus:  

In  recent years,  the  consumption  of  fruit  and vegetables  has  been  falling  in  the  EU.  The 
majority of Europeans and in particular children fail to meet the minimum intake of 400 g 
per day recommended by the World Health Organisation. An estimated 22 million children 
in the EU are overweight, and 5.1 million of them are obese. Developing healthier eating 
habits  by  consuming  more  fruit  and  vegetables  can  play  an  important  role  in  combating 
health problems related to poor nutrition, such as child obesity and other risk factors and 
disorders. 

In November 2008 the Agriculture Council of Ministers agreed on a Commission proposal 
for  an  EU-wide  scheme  to  provide  fruit  and  vegetables  to  school  children.  The  EU 
provides  co-financing  for  the  scheme  (50%  or  75%  for  convergence  regions),  with  EU 
funds being matched by national or private funding. 

The evaluation was intended to contribute to the reporting obligations laid down in Article 
184  of  Council  Regulation  (EC)  No  1234/2007  which  provides  that:  "The  Commission 
shall present a report before 31 August 2012 to the European Parliament and the Council 
on  the  application  of  the  School  Fruit  Scheme,  accompanied,  if  necessary,  by  new 
proposals.  The  report  shall  in  particular  address  the  issues  of  the  extent  to  which  the 
scheme  has  promoted  the  establishment  of  well-  functioning  School  Fruit  Schemes  in 
Member  States  and  the  impact  of  the  Scheme  on  the  improvement  of  children's  eating 
habits". 

EN 74 

Error! Unknown document property name.  EN 

 
 
 
 
The School Fruit Scheme started in the school year 2009/2010 and it is not limited in time. 
The Scheme has a one year policy cycle. 

5) Summary of findings and conclusions of the evaluation: 

The evaluation has found that the scheme was effective in increasing children's fruit and 
vegetables consumption. The majority of Member States has observed a positive impact of 
the  scheme  on  children's  fruit  and  vegetables  consumption  meaning  an  increase  of 
consumption  beyond  the  fruit  and  vegetables  distributed  to  the  children.  Regarding  the 
influence of the EU funding on the scheme's effectiveness, the evaluation suggests that the 
EU aid has a positive or even essential impact. The scheme is found to be pivotal for the 
implementation of nation (or region) wide School Fruit Schemes in nearly all participating 
Member States. The analysis of potential success factors of the scheme has identified the 
following factors: a wide range of products, a high frequency and long duration of offering 
fruit and vegetables, free distribution. 

A systematic comparison by the evaluation study of the original National Strategies sent to 
the Commission on the one hand and their factual implementation, as documented in the 
annual monitoring reports on the other, showed that many changes occurred between the 
plan in the strategy and its implementation. 

The  ex  post  evaluation  suggests  that  an  increase  of  the  EU  funding  share,  provided  that 
other funding remains constant, leads to a higher uptake and a larger scale of the scheme. 
The scheme was also found to improve the image of the EU and increase public awareness 
of the importance of the work of the EU. 

The results of the evaluation and the specific recommendations were taken into account in 
the changes proposed to the scheme in the context of the CAP 2020 reform proposals. 

The EU evaluation complemented the evaluations carried out by the Member States. The 
EU evaluation proved useful to put the national and regional implementation data, such as 
frequency  of  distribution,  in  a  common  format  for  all  participating  Member  States  and 
regions. 

6) Availability of the report on Europa: 

http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/2012/school-fruit-
scheme/fulltext_en.pdf 

 Ex-post evaluation of Rural Development Programmes 2000-2006 

1) ABB activities concerned: 05 04 

2) Timing: February 2012, (period covered 2000-2006) 

3) Budget:  

The overall envelope for the rural development programmes 2000-2006 was EUR 59.189 
billion. 

EN 75 

Error! Unknown document property name.  EN 

 
 
 
 
4) Background, scope and focus:  

For the programming period 2000-2006, the main instrument of rural development policy 
was  Council  Regulation  (EC)  No  1257/99,  which  originally  provided  a  menu  of  22 
measures.  The  2003  CAP  reform  added  a  further  four  measures,  through  Council 
Regulation  (EC)  No  1783/2003.  In  addition,  the  accession  of  the  EU10  on  1  May  2004 
added  a  further  seven  measures  available  to  the  new  Member  States.  The  evaluation 
covered  all  15  Member  States  benefiting  from  EU  co-financed  rural  development 
programmes  (RDPs)  during  the  2000-2006  period,  plus  the  10  Member  States  which 
joined  the  EU  on  1  May  2004  and  whose  RDPs  covered  the  years  2004-2006.  The 
evaluation examined the relevance, coherence, effectiveness and efficiency of the different 
rural  development  measures  and  programmes.  In  addition,  the  evaluation  also  sought  to 
identify  factors  influencing  the  contribution  of  rural  development  measures  to  the  new 
challenges under the CAP Health Check and to EU priorities for the post-2013 period. 

5) Summary of findings and conclusions of the evaluation: 

At the overall policy level, the objectives reflected identified needs: to change and access 
markets;  reverse  the  trends  of  economic  and  social  decline  and  depopulation  of  the 
countryside; remove inequalities and promote equal opportunities; improve environmental 
conditions and protect/preserve the environment and ensure the viability of farming. In the 
new Member States, policy objectives addressed the need to restructure the farming sector 
and reduce dependency on semi-subsistence farming, as well as to meet EU standards and 
diversify the rural economy and improve the rural infrastructure. 

As  to  complementarity  between  RDPs  and  other  support  instruments,  the  evaluation 
indicates  that  while  good  examples  of  complementarity  at  programme  level  can  be 
identified, lack of efficient coordination between authorities designing and implementing 
development  interventions  in  rural  areas,  seems  to  be  the  main  reason  for  low  levels  of 
complementarity. 

Overall,  RDPs  demonstrate  a  capacity  to  generate  positive  economic  and  environmental 
impacts, although most measures have more impact at the individual beneficiary level than 
at the level of the whole rural economy and population 

Income  effects  seem  to  be  highly  correlated  with  efficient  targeting  of  measure-  specific 
support  and  the  combined  application  of  measures.  In  addition,  rural  development 
measures  maintained  and/or  generated  employment  in  rural  areas,  especially  at  the  local 
and regional level. Net employment effects of RDP measures seem to be generally lower 
than  net  income  impacts,  though  it  is  difficult  to  distinguish  the  impact  of  RDP 
interventions from that of other factors. 

Evidence of environmental impacts is mainly of a qualitative nature due to a lack of robust 
baseline and monitoring data. The greatest environmental impacts were linked to the agri-
environment  and  forestry  measures,  and  were  highly  positively  correlated  with  the 
financial  weight  of  these  measures  in  the  RDPs.  Effective  use  of  training  and  advisory 
services improved the overall environmental impact of programmes. 

As to impacts on the quality of life and maintenance of working and living conditions in 
rural areas, the adaptation measures seem to have produced positive effects on the quality 
of life and sustainability of rural communities. 

Based  on  the  evidence  collected  and  subsequent  analysis,  the  evaluators  formulated  the 

EN 76 

Error! Unknown document property name.  EN 

 
 
following general recommendations in relation to rural development policy: 

(1)  Improve  targeting:  Better  targeting  is  key  to  improving  the  efficiency  of  rural 
development measures. Targeting should be programme-specific to meet specific needs of 
the  territory.  Effective  targeting  is  based  on  the  identification  of  priority  areas, 
beneficiaries,  types  of  activity  or  achievement.  Therefore  improved  strategy  making  is  a 
prerequisite for improved targeting. 

(2)  Creating  synergy:  Complementarity  between  measures  should  be  enhanced  within 
future rural development policy. Synergetic groups of measures should be identified, and 
links  between  them  strengthened,  either  in  the  legal  framework  and/or  in  the 
implementation process. 

(3)  Unlock potential of underperforming measures. Action should be taken in relation to 
measures with high but unrealised potential to contribute to objectives e.g. training. 

(4)  Consider  wider  rural  development  needs  and  opportunities:  A  better  balance  should 
be  struck  within  RDPs,  with  a  stronger  focus  on  the  wider  rural  economy  and 
environment, compared to the agricultural sector. 

(5)  Develop  complementarity  with  other 
complementarity between RDPs and other programmes and funds. 

funds  and  programmes:  Strengthen 

(6) Focus on territory rather than individual beneficiaries: Moving the vision of efficiency 
and  competitiveness  from  the  farm  and  business  level  to  the  rural  area  level  as  a  whole, 
would improve the impact on rural economies and populations.  

6) Availability of the report on Europa:  

http://ec.europa.eu/agriculture/evaluation/rural-development-reports/2012/ex-post-evaluation-
rdp-2000-2006/fulltext_en.pdf 

Synthesis of mid-term evaluations of rural development programmes 2007-2013 

1) ABB activities concerned: 05 04 

2) Timing: November 2012, (period covered 2007- 2009) 

3) Budget:  

The overall envelope of the rural development programmes 2007-2013 amounts to EUR 
96.241 billion. 

4) Background, scope and focus:  

Council Regulation (EC) No 1698/2005 on support for rural development by the European 
Agricultural Fund for Rural Development provides the legal framework for the preparation 
and  the  implementation  of  rural  development programmes  (RDPs)  in  the  Member  States 
for the period 2007-2013. RDPs were subject to a mid-term evaluation (MTE) organised 
under the responsibility of Member States and carried out by independent evaluators 

5) Summary of findings and conclusions of the evaluation: 

EN 77 

Error! Unknown document property name.  EN 

 
  
 
 
The synthesis of the MTE reports was undertaken by the Commission  with the purpose of 
summarizing and analysing the reports focusing on results and impacts achieved so far by 
the  2007-2013  RDPs  and  on  the  effectiveness  of  the  monitoring  and  evaluation 
framework.  The  evaluation  covers  all  27  Member  States.  It  is  based  mainly  on  the  88 
national  and  regional  RDPs  and  4  Network  Programmes  2007-2013  but  considers  also 
other supporting documents and data sources, such as the RDPs, Annual Progress Reports 
and  National  Strategy  Plans  for  Rural  Development.  As  the  MTEs  are  carried  out 
relatively early in the programming period when many programmes/measures are not yet 
very  advanced  in  their  implementation,  conclusions  from  the  synthesis  can  only  give 
preliminary  results  on  the  performance  of  the  RDPs.  These  preliminary  findings  are 
presented below, structured according to the different types of effects on rural areas. 

- Economic impacts (growth, employment, labour productivity): 69% of the MTEs report a 
net positive impact of the RDPs on growth while a modest positive impact on employment 
was observed in 62% of the reports. As far as growth is concerned, some MTEs that do not 
state any positive impacts underline that at the stage of the MTE the scale of RDP funding 
is still too small to have a notable impact on the rural economy. In terms of employment 
creation, there is evidence that programmes have funded projects which have created new 
jobs,  sometimes  in  significant  numbers,  and  that  they  have  helped  to  sustain  jobs  in 
farming.  As  for  labour  productivity,  mixed  outcomes  are  noted.  While  some  positive 
impacts are stated, some programmes are also evaluated as having prioritised job creation 
at the expense of labour productivity. 

- 
Environmental  impacts  (biodiversity,  water  quality,  climate  change):  Some  positive 
environmental  impacts  were  inferred  in  the  MTEs,  mostly  on  the  basis  of  expert 
judgments. It is apparent that there has been a significant increase in the area of land under 
agri-environmental  management  in  a  number  of  Member  States  and  this  should  help  to 
ensure  positive  programme  outcomes  for  the  environment,  insofar  as  the  management 
obligations are tailored carefully to the environmental strengths and opportunities of each 
territorial context. 

- 
Impacts  on  quality  of  life  in  rural  areas:  Qualitative  benefits  to  social  capital  and 
reduced social inequalities are cited as RDP benefits in several MTE reports. Furthermore, 
improved  rural  services,  including  waste  and  water  management,  as  well  as  tourism 
benefits,  were  noted  in  some  cases.  These  benefits  seem  to  be  valued  elements  for 
successful and sustainable rural development in the local communities. The synthesis also 
looked  at  various  aspects  of  the  Common  Monitoring  and  Evaluation  Framework  for 
RDPs,  such  as  the  overall  functioning  of  the  system  and  the  performance  of  the  output, 
result  and  impact  indicators  used  in  the  evaluations.  The  MTEs  assess  the  monitoring 
system as good overall (58%) and as ensuring a relevant set of data in general. However, 
the  system  is  often  regarded  as  being  too  complex.  On  the  basis  of  the  analysis,  the 
evaluation recommends a simplification and slimming down of the common indicator set 
as well  as improved guidance and explanation of the calculation, use and aggregation of 
the indicators. These recommendations are being used in the design of the monitoring and 
evaluation system for rural development for the period 2014-2020. 

6) Availability of the report on Europa: 

http://ec.europa.eu/agriculture/evaluation/rural-development-reports/2012/synthesis-mte-
2007-2013/fulltext_en.pdf 

EN 78 

Error! Unknown document property name.  EN 

 
Ex-post evaluation of the EU Forest Action Plan 

1) ABB activities: 05 04 

2) Timing: August 2012, (period covered 2007-2011 with a focus however on the period not 
studied by the mid-term evaluation published in November 2009) 

3) Budget: Not applicable 

4) Background, scope and focus:  

The  EU  Forest  Action  Plan  (FAP)  was  adopted  by  the  Commission  on  15  June  2006. 
Building on the principles of the EU Forestry Strategy (sustainability and subsidiarity), it 
provides a framework for forest-related actions at EU and Member State level. It aims at 
serving as an instrument of coordination between Union actions and the forest policies of 
Member States. 

The evaluation covered the 18 key actions and activities as well as additional activities of 
the  Member  States  carried  out  in  the  context  of  the  Action  Plan.  It  also  considered  the 
other EU instruments relevant to the implementation of the key actions of the EU Forest 
Action Plan.  

5) Summary of findings and conclusions of the evaluation:  

The evaluation reported the implementation of the activities foreseen in the context of the 
4  objectives  of  the  forest  action  plan  (to  improve  the  long-term  competitiveness  of  EU 
forestry, to improve and protect the environment, to contribute to the quality of life, and to 
foster coordination and communication). 

The evaluation concluded that EU Forest Action Plan has had a beneficial impact on the 
implementation  of  the  EU  Forestry  Strategy.  It  indicated  that  the  FAP  and  its 
organisational setup was adequate for its purpose at the time, taking into account the core 
principles  defined  in  the  EU  Forestry  Strategy  from  1998.  However,  there  were  higher 
expectations for the implementation of the Action Plan. Yet, fulfilling those expectations 
would have required a higher level of commitment from the parties involved. With respect 
to  the  needs  the  Action  Plan  was  intended  to  address,  international  policy  developments 
have caused and are causing shifts in priorities which were not foreseeable to a full extent 
when  preparing  the  Action  Plan.  The  EU  Forest  Action  Plan  was  able  to  respond  to 
changing  needs  to  a  certain  extent,  but  its  ability  to  facilitate  open  dialogue  between 
different interests related to forests was limited. 

The  conclusions  and  recommendations  of  the  evaluation  provide  a  main  input  in  the 
review of any future EU Forest Strategy.  

6) Availability of the report on Europa: 

http://ec.europa.eu/agriculture/evaluation/market-and-income-reports/2012/forest-action-
plan/fulltext_en.pdf 

EN 79 

Error! Unknown document property name.  EN 

 
 
 
 
ABB 06 – Mobility and Transport  

 Mid-term evaluation of TEN-TEA 
(The Trans-European Transport Network Executive Agency) 

1) ABB activities concerned: 06 03  

2) Timing: Completed in July 2012 (period covered 15 April 2008 -14 April 2011) 

3)  Budget:  The  Administrative  costs  of  the  Agency  for  the  period  2008-2011  amounted  to 
EUR 36.06 million. 

The TEN-T Programme budget for the period 2007-2013 amounts to EUR8.13billion 

4) Background, scope and focus: 

The evaluation was required under article 25 §1 of Council Regulation (EC) N° 58/2003 
that lays down the status for Executive Agencies and stipulates that an external evaluation 
report for the first three years of operation shall be prepared by the Commission. 

The main objectives of the evaluation were 

• 
To  provide  a  mid-term  evaluation  of  the  TEN-T  EA,  including  a  Cost-Benefit 
analysis,  and  to  determine  whether  the  Agency  continues  to  be  the  most  cost-effective 
solution  for  the  management  of  the  European  Union's  financial  support  of  the  TEN-T 
Network. 

• 

To identify any problems in the systems and processes used by the Agency.  

To  identify  permanent  monitoring  (of  the  agency)  requirements  by  DG  MOVE 

• 
(periodicity, type etc.).   

The scope of this mid-term evaluation included the overall organisation of the Executive 
Agency,  the  effectiveness  and  efficiency  of  the  Agency's  operations,  the  reliability  of 
financial and non-financial data provided by the Executive Agency and the relations of the 
Executive Agency with other Commission services and with external service providers. 

5) Summary of findings and conclusions of the evaluation study: 

The overall conclusion of the evaluation is that the TEN-T EA is a well-run organisation 
that  has  successfully  met  the  targets  that  have  been  set.  The  evaluation  shows  that  the 
Agency  is  performing  its  mandated  tasks  in  an  effective  and  efficient  way.  It has  a  very 
high performance in relation to project  management tasks. The Agency performance has 
indirectly  facilitated  an  improvement  in  the  operational  implementation  of  the  TEN-T 
Programme. Using an Agency to manage the projects financed by the TEN-T programme 
is the most cost-effective option and, as such, the relevance of the TEN-T EA continues to 
be  high.  The  cost  and  delivery  of  the  Agency  fulfils  the  objectives  and  expectations  of 

EN 80 

Error! Unknown document property name.  EN 

 
 
improved implementation of the TEN-T programme, as sought at the time of establishing 
the Agency. 

The  tasks  carried  out  by  the  Agency  are  still  consistent  with  the  ones  defined  in  the 
Commission Decision to establish the Agency and the Act of Delegation. The outsourcing 
to  the  Executive  Agency  remains  the  most  cost-effective  manner  to  ensure  the 
management  of  the  EU's  support  to  the  TEN-T  network.  There  are  no  factors  in  the 
organisational  environment  that  have  changed  to  the  extent  that  the  merits  of  the 
outsourcing  option  should  be  questioned.  If  anything,  given  the  more  pronounced 
constraints on the EU budget, the need to maximise efficiency gains through arrangements 
such as outsourcing has become even more pressing. 

Overall,  the  TEN-T  EA  is  operating  effectively  and  in  compliance  with  Council 
Regulation  58/2003,  the  Financial  Regulation  and  the  legal  framework  by  which  it  was 
established.  The  Agency's  objectives  have  been  achieved  to  a  high  degree  and  overall 
target-achievement  has  improved  continuously  from  2008  to  2011.  The  creation  of  the 
Agency also offered the possibility of increasing the number of staff dealing with TEN-T 
project management compared to the level of staff available in DG MOVE.  Furthermore, 
the  level  of  satisfaction  of  the  stakeholders  with  the  services  provided  by  the  Agency  is 
high.  

The  organisational  structure  of  the  Agency  is  well  suited  for  managing  its  tasks  and 
objectives.  The  Agency  possesses  a  high  level  of  expertise  in  project  and  financial 
management, but could acquire additional skills in some areas of transport expertise such 
as  traffic-flow  forecasting,  cost-benefit  assessment  of  transport  infrastructure  and  cost 
engineering. 

6) Availability of the report on Europa: 

http://ec.europa.eu/transport/facts-fundings/evaluations/doc/2012-no95-ten-tea-report.pdf 

Evaluation of Regulation 1371/2007 on rail passengers’ rights and obligations 

1) ABB activities concerned: 06 02  

2)  Timing:  July  2012,  (period  covered:  from  the  entry  into  force  of  the  Regulation  in 
December 2009 until mid-2012) 

3) Budget: Not applicable. 

4) Background, scope and focus: 

Under  Article  36  of  the  Regulation,  the  Commission  needs  to  report  to  the  European 
Parliament and Council on the implementation and the results of this Regulation, notably 

EN 81 

Error! Unknown document property name.  EN 

 
 
 
 
on the service quality standards. The evaluation study was carried out in order to provide 
input into that report. 

The main objective of the evaluation was to gather information on the implementation of 
the Regulation and to assess whether there is evidence of non-compliance. The evaluation 
also  provides  recommendations  on  how  to  deal  with  shortcomings  and  improve  the 
application. 

5) Summary of findings and conclusions of the evaluation study: 

Overall, the evaluator has not identified any single major problem with the implementation 
of  the  Regulation,  either  by  Member  States  or  by  railway  undertakings.  There  is  no 
evidence of systematic non-compliance with the Regulation, or a major requirement which 
is so unclear that it cannot be implemented. However, it has been signalled in the report 
that some provisions of the Regulation are not fully clear and that the implementation of 
some requirements by certain Member States and railways undertakings is lagging behind. 
Some  of  the  objectives  of  the  Regulation  have  only  been  partially  achieved.  Main 
limitations are the following: 

The  fact  that  Member  States  were  allowed  to  derogate  certain  services  from  the 

• 
application of the Regulation has severely limited its impact.  

• 
Several Member States either have national laws providing for rights similar to those 
in  the  Regulation,  or  they  impose  similar  rights  and  policies  through  other  means  (eg. 
public service contracts or government decisions).   

•  Most  Member  States  showed  little  pro-active  enforcement  actions.  Some  specific 
problems with enforcement have been detected. 

•  Compliance of railway undertakings was rather poor with the requirement to publish 
service quality reports. 

• 

There are also some interpretative issues with the text of the Regulation.  

Based  on  the  above  findings,  main  recommendations  of  the  evaluation  report  are  the 
following: 

Exemptions:  The  Commission  should  consider  whether  exemptions  should  still  be 

• 
permitted for domestic long distance services.  

•  Actions  to  improve  awareness  of  passenger  rights:  There  could  be  a  requirement  to 
display  notices,  in  a  prominent  position,  at  rail  stations  and  on  board  trains,  informing 
passengers  about  their  rights.  The  Commission  and  NEBs  should  also  take  measures  to 
promote passenger awareness of the rights defined in the Regulation.  

•  Actions  to  improve  enforcement:  The  Commission  should  develop  guidelines  for 
NEBs, including also the development of an appropriate complaint handling procedure, the 
provision of information to passengers, and (subject to national law) the circumstances in 
which  sanctions  should  be  considered.  NEBs  should  take  a  more  pro-active  approach  to 

EN 82 

Error! Unknown document property name.  EN 

 
enforcement, in particular through inspections.  

• 
Service  quality  reports:  NEBs  should  verify  that  the  reports  are  published  and  that 
their content is compliant with the Regulation, and take enforcement action if this is not 
the case.  

Issues  requiring  clarification:  Some  elements  of  the  Regulation,  which  could  be 

• 
subject to more than one interpretation, should be clarified.  

• 
Issues which are not fully covered by the current Regulation: A number of issues are 
addressed by passenger rights’ Regulations in other transport sectors but not fully by the 
rail Regulation. In some cases, but not always, this reflects objective differences between 
transport modes. The Regulation could be extended in the some areas. 

6) Availability of the report on Europa: 

http://ec.europa.eu/transport/facts-fundings/evaluations/doc/2012_reg1371_07.pdf 

Ex-post evaluation of technical follow-up of TEN-T funded ERTMS projects 
carried out under Service Framework Contract TREN/E2/322-2008 (Lots 1, 2 
and 3) 

1) ABB activities concerned: 06 03  

2) Timing: Completed in December 2012, covering the period 2008-2012 

3) Budget: Framework contract with a maximum total budget of EUR2.2 million. 

4) Background, scope and focus: 

In  2007,  the  European  Commission  entered  into  a  4-year  Service  Framework  Contract 
TREN/E2/322-2008  (comprising  several  lots)  to  strengthen  its  capacity  to  monitor  EU 
funded projects for the deployment of the ERTMS programme. The framework contracts 
for  the  technical  follow-up  and  monitoring  of  ERTMS  projects  came  to  an  end  in 
December 2012.  

The  evaluation  was  undertaken  to  assess  the  effectiveness  of  the  Service  Framework 
Contract  in  delivering  the  technical  expertise  which  enabled  the  mitigation  and 
minimisation  of  interoperability  threats  within  TEN-T  funded  ERTMS  projects.  The 
Commission  aimed  to  determine  whether  technical  follow-up  of  such  projects  (and 
possible  future  projects)  should  be  continued  from  2013  and  if  so,  evaluate  the  most 
appropriate  way  to  ensure  that  these  projects  (and  future  ERTMS  projects)  are  fully 

EN 83 

Error! Unknown document property name.  EN 

 
 
 
 
 
interoperable taking into account resource constraints faced by the Commission. 

5) Summary of findings and conclusions of the evaluation study: 

The  technical  follow-up  of  ERTMS  projects  carried  out  between  2008  and  2012  was 
effective and the experts were able to raise awareness of the importance of interoperability 
and detect interoperability risks. Experts also identified solutions to mitigate interoperable 
risks  and  in  some  cases,  the  intervention  of  experts  led  to  solutions  that  would  not 
otherwise have been implemented in order to mitigate risks.  

Given this demonstrated added-value there is still a serious and on-going need for the level 
of  support  that  has  been  provided  by  the  technical  follow-up  of  TEN-T  funded  ERTMS 
projects  to  date.  The  risk  of  funding  "non-interoperable  projects"  is  still  high  because 
ERTMS  implementation  is  still  in  a  critical  phase  and  because  there  are  still  some 
concerns  with  Notified  Body  certificates.  The  report  also  underlines  that  the  cost  of  the 
follow-up compared to the amount of funding provided to ERTMS projects is very small. 

The evaluation recommends that technical follow-up should be extended beyond 2013 and 
2014 until after 2020 and that the European Railway Agency is the best-placed entity to be 
in charge of the technical follow-up in the future. An adequate level of resources for the 
technical  follow-up  is  estimated  by  the  evaluators  to  be  approximately  530  annual  man-
days. Continued follow-up by ERA would bring highest benefits in terms of effectiveness 
and efficiency compared to other options involving EU entities. This option also coincides 
with the DG MOVE agenda for externalisation from 2014. 

Key recommendations: 

Technical follow-up should be conducted as early as possible on projects, even in the 

• 
pre-tender phase; 

The  buy-in  from  stakeholders  should  be  enhanced  by  making  participation  in  the 

• 
technical follow-up a compulsory requirement; 

•  All projects benefitting from EU funding should be subject to technical follow-up. 

6) Availability of the report on Europa: 

http://ec.europa.eu/transport/facts-fundings/evaluations/doc/2012-no98-ten-t-funded-ertms-
projects.pdf 

EN 84 

Error! Unknown document property name.  EN 

 
 
ABB 07 – Environment  

Final evaluation of LIFE+ 

1) ABB activities concerned: 07 03 

2) Timing: 17.12.2012 (period covered 2007-2013) 

3) Budget:  

The  on-going  Financial  Instrument  for  the  Environment  (LIFE+)  has  a  budget  of 
EUR2.143  billion  covering  both  the  operational  expenditure  of  the  DG  and  the  co-
financing of projects. 

4) Background, scope and focus  

The LIFE Programme is the EU's main financial instrument to support the implementation 
of Community policy and legislation in the environment sector.  It was launched in 1992 
and is now in its fourth programming period. LIFE+ (2007-2013). New elements will be 
added  to  LIFE  in  the  next  programming  period,  including  the  further  expansion  of  the 
programme scope 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study: 

This final evaluation of LIFE+ comes just over two years after the Mid Term Evaluation 
and  one  year  after  the  proposal  for  a  new  LIFE  programme.  From  the  evidence  and 
opinions collected there have not been any major changes to the nature of the programme, 
or  the  quality  of  its  outputs,  in  the  last  two  years.  However  there  is  evidence  of  the 
continual improvements in the programme that have been noted in previous evaluations. 

The presence of EU added value has been given more weight in bid selection and is more 
extensively  explained  in  the  guidance.  Though  it  is  recognised  that  there  is  a  need  to 
continue reminding projects of this objective and encouraging applicants to look for more. 

External monitoring is working well and beneficiaries as well as LIFE units are satisfied 
with the services provided by external monitors. 

The  expenditure  on  public  procurement  is  appropriate  and  responds  to  the  needs  of  the 
policy cycle. There is some high profile communication and outreach supported, but it is 
hard to assess its strategic impact. Nevertheless it is delivered in line with quality criteria 
and strategic need. 

The  administrative  burden  for  participants  is  slightly  higher  than  for  other  EC 
programmes.  The  participants  generally  find  this  acceptable  but  would  like  to  see  it 
reduce. 

The  electronic  application  process,  despite  some  teething  problems,  has  reduced  the 
administrative  burden.  There  is  potential  to  expand  the  use  of  electronic  reporting  in 
project monitoring. 

EN 85 

Error! Unknown document property name.  EN 

 
 
There are some impressive results; summing up the project output indicators shows 194m 
hectares  of  land  purchased,  49  600  actions  expected,  95  267  training  sessions  planned, 
6.1m  people  to  be  targeted  by  communications  and  1.2m  pupils  and  students  to  be 
engaged.  

There  has  been  an  improvement  to  feedback  to  NGOs  and  some  improvements  on 
payment timing, though this is limited by EC procedures. 

External  and  inter-project  communication  within  the  programme  (which  is  important  to 
dissemination) has improved, e.g. better website and project database, platform meetings, 
themed brochures and conferences and clustering of projects. 

6) Availability of the report on Europa:   

The report has not yet been published on Europa, but will become available soon. 

EN 86 

Error! Unknown document property name.  EN 

 
 
ABB 08 – Research  

Interim Evaluation of Art. 185 European Metrology Research Programme (EMRP) 

1) ABB activities concerned:  

2) Timing: 30 November 2011 , (duration June to November 2011) 

3)  Budget:  Total  budget  Seventh  Framework  Programme  (FP7)  around  EUR  55  billion  (Euratom 
included) for 2007-2013. 

4) Background, scope and focus: 

Aim of the evaluation study 

1.  Assess the progress towards the objectives of the EMRP.  

2.  Assess  the  level  of  financial  contributions  to  the  Programme  by  participating  countries 
compared to initial commitments. 

3.  Assess  the  performance  of  EMRP  as  an  integration  of  national  metrology  research 
programmes in the spirit of Article 169 of the EC Treaty (new Article 185 of the TEU) and assess 
scientific,  management  and  financial  integration.  In  this  context,  the  role  of  the  European 
Commission should also be addressed.  

4.  Assess, with a focus on operations and results, the EMRP as operational structure, taking into 
account  the  quality  and  the  efficiency  of  the  implementation.  In  this  context,  the  role  of  the 
European Commission should also be addressed. 

5.  Assess  the  European  added  value  of  EMRP,  using  Article  185  TFEU,  compared  to  other 
forms  of  support  to  R&D  (via  the  Framework  Programme,  via  ERA-NETs,  via  National  & 
Regional programmes). 

6.  On the basis of this assessment what is the impact of EMRP and draw possible lessons to be 
learnt and recommendations for adjustments as appropriate of the current EMRP. 

7.  On  the  basis  of  this  assessment,  make  recommendations  for  a  possible  continuation  of 
EMRP (if applicable).  

8.  On the basis of lessons learnt on the case of EMRP, provide possible recommendations for 
future joint programmes involving Member States and the European Commission using Article 
185. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

Panel Conclusions 

The  overall  opinion  of  the  Panel  is  that  the  EMRP  is  a  well  managed  joint  European  research 
programme that has already achieved a relatively high level of scientific, management and financial 
integration.    After  only  two  years  of  implementation  it  is  too  early  to  assess  the  quantitative 
impacts and so the Panel has concluded on four main questions: how well is the EMRP performing 

EN 87 

Error! Unknown document property name.  EN 

 
 
compared with the original expectations, how can it be improved, how can its impact be increased 
if there is a successor programme and what lessons have been learned for other Joint Programming 
Initiatives.  

Qualitative Impact of the EMRP 

The 12 point framework of the ex-ante impact assessment was also used by the Panel to structure 
its  qualitative  conclusions  on  the  impact  of  the  EMRP.    This  had  two  advantages.    It  allowed  a 
direct comparison with the expectations and minimised the risk of appearing to be over-critical in 
areas where the potential impact was not expected to be so great.   

6) Availability of the report on Europa: 
http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/mtr_report
_final.pdf 

Review of S&T cooperation between the European Union and India 

1) ABB activities concerned: 08 02 

2) Timing: June 2012, (period covered 2007-2011) 

3) Budget: FP7 total budget around EUR 55 billion (Euratom included) for 2007-2013. 

4) Background, scope and focus: 

Aim of the evaluation study Perform an analysis of the impact of cooperation with countries with an 
S&T Agreement, comprising an in-depth desk research on the achievements and scope of the S&T 
agreements as well as on-site visit in the targeted country. 

5) Summary of findings and conclusions of the evaluation or evaluation related study: Overall 
the  Agreement  establishes  that  EU-India  S&T  cooperation  relies  on  reciprocity  and  symmetry. 
Cooperative activities are many and include: the participation of Indian institutions and scholars in 
the  projects  funded  by  the  Framework  Programme,  the  pooling  of  projects,  the  mobility  of 
researchers  and  exchange  of  information.  However,  the  report  argues  that  the  Agreement  is  only 
partially adequate to enhance EU-India S&T partnership and that internal and context weaknesses 
need to be addressed including: 

1. Mutual knowledge may be improved by means of several types of courses (including e-learning) 
directed to the actors of the EU-India S&T partnership; 

2.  Solid  and  rigorous  monitoring  and  ex-post  evaluation  of  EU-India  projects  in  S&T  should  be 
considered as a priority; 3. Coordinated calls might be enhanced by creating joint funding schemes 
with the participation of the EU and MS. The action of the India Group of the Strategic Forum for 
International S&T Cooperation should be supported to enhance EU-MS coordination; 

4.  The  EU  and  India  should  mobilize  stakeholders  to  produce  viable  innovations  and  new 
technologies through joint projects. EU, MS and India’s public funding can play a facilitating but 
subsidiary role, leaving the private sector as main driver; 

5. The EU and India may join in the promotion of inclusive technology in order to find common 

EN 88 

Error! Unknown document property name.  EN 

 
 
solutions to sustainable development needs, both in the EU and in India; 

6. The gap between science and industry – and between research and innovation – can be addressed 
by enhancing the involvement of the private sector in EU-India S&T partnership. Field interviews 
have  provided  evidence  of  interesting  initiatives  which  might  promote  the  collaboration  between 
India and EU’s enterprises.  

6) Availability of the report on Europa: 

http://ec.europa.eu/research/iscp/pdf/india-review-brochure.pdf 

Review of S&T cooperation between the European Union and Brazil 

1) ABB activities concerned: 08 02  

2) Timing: 30 June 2012 (period covered 2007-2012) 

3) Budget: FP7 total budget around EUR 55 billion (Euratom included) for 2007-2013. 

4) Background, scope and focus  

Aim of the evaluation study: Review the first five years of the S&T Cooperation Agreement between 
the EU and Brazil 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

Over and above the participation of Brazil in the European Framework Programmes for Research 
and  Technological  Development  (RTD),  the  Agreement  had  a  positive  impact  in  moving  the 
cooperation  from  a  bottom-up  to  a  more  programme-level  approach.  This  is  reflected  in  the 
roadmaps  developed  in  the  five-year  period.  However,  for  the  cooperation  to  achieve  a  real 
strategic shape and greater visibility, there is still a need for both sides to converge more strongly 
on the thematic areas in which to focus the cooperation. 

Both  sides  have  openings  for  participation  of  the  other  party  in  their  research  programmes, 
however, Brazilian programmes only allow for individual researcher participation, not institutions. 

The  Steering  Committee  has  met  annually  and  discharged  of  its  duties,  but  could  strengthen  its 
review of the efficient implementation and functioning of the agreement. S&T policy developments 
on  both  sides  are  regularly  reported  on  and  they  had  an  influence  on  the  development  of  the 
cooperation within the agreement. 

Two joint calls have been implemented in biofuels and ICT respectively; another one in ICT is on-
going while a joint call in health has not yet been implemented on the Brazilian side, 2 years after 
the European side has done so. 

The BILAT project in support of the agreement has carried out a lot of useful technical work and 
promoted cooperation, contributing to a rise in Brazilian participation in FP7 compared to FP6 in 
terms of numbers of participations and financial compensation. The link to S&T policy in Brazil is 
not ensured. Brazilian stakeholders interviewed consider collaboration in S&T with the EU and its 
Member States a priority. 

EN 89 

Error! Unknown document property name.  EN 

 
 
The  agreement  does  not  stipulate  alignment  or  interaction  with  existing  S&T  agreements  of  EU 
Member States with Brazil (agreements, which do not either stipulate any alignment with the EU-
Brazil  agreement).  Yet,  they  address  similar  objectives  and  thematic  areas  and  constitute  an 
important backbone of Brazilian partnerships in S&T with Europe. This duality in the cooperative 
approach  between  Brazil  and  Europe  needs  constructive  consideration  at  the  S&T  policy  level 
between the European Commission and EU Member States to avoid redundancies, create synergies 
and  make  European  cooperation  with  Brazil  more  coherent  and  effective.  In  the  interviews,  the 
recently  established  Strategic  Forum  for  International  Cooperation  (SFIC)  was  viewed  as  an 
interesting mechanism for coordination. 

The major recommendations are: 

• 

A more focused thematic orientation based on new strategies on both sides. 

Greater  participation  in  planning  and  execution  of  the  cooperation  of  relevant 

• 
stakeholders, especially from the private sector. 

• 
Improved functioning of the Agreement and institutionalisation (delegating technical tasks 
from  Itamaraty  (the  Brazilian  Foreign  Ministry  as  official  Brazilian  party  to  the  Agreement)  to 
relevant research policy and management bodies. 

• 

Alignment and coherence with agreements and activities of EU Member States. 

6) Availability of the report on Europa:  

http://ec.europa.eu/research/iscp/pdf/review_brazil_agreement_2007-2011.pdf  

Review of main activities and deliverables of ERAWATCH 

1) ABB activities concerned: 08 19  

2) Timing: May 2012 

3) Budget: Overall budget - EUR 5,359,200 

4) Background, scope and focus 

Aim of the study: 

to assess the extent to which the project is on track to achieve its objectives as stated in the 

• 
ERAWATCH2 Administrative Arrangement;  

• 
to  formulate  recommendations  in  view  of  the  development  of  the  future    Research  and 
Innovation  Observatory  that  should  monitor  research  and  innovation  policies  after  July  2013 
(follow-up of the current ERAWATCH and TrendChart contracts). 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The  evaluation  led  to  identify  six  main  clusters  of  key  challenges  faced  by  the  designers  and 
implementers of policy information systems such as  ERAWATCH and which should help guide the 
design  and  implementation  of  future  systems.  These  relate  to  management,  users,  content, 

EN 90 

Error! Unknown document property name.  EN 

 
 
infrastructure, quality and value. 

Management 

Ensuring an effective and efficient management at all levels of the project. 

Users 

Appropriately scoping the project and managing accordingly the expectations of 
what it can deliver in terms of different outputs.  

Engaging with clients and users on a continual basis.   

Satisfying the information needs of multiple users with varying requirements, 
including stratifying information according to its value in the policy cycle. 

Content 

Engaging appropriate expertise to provide the necessary content and covering the 
appropriate breadth of both research and innovation policy. 

Ensuring homogeneity and comparability in the information collected across 
countries. 

Adapting to changing information requirements. 

Respecting important differences between countries, e.g. in terms of size, economic 
profile, state of development of the policy system, etc.  

Infrastructure 

Building a content management system that can support a range of search / 
information analysis / synthesis functions.  

Ensuring interoperability with other existing platforms / information services (e.g. 
IUS, OECD) and striving for continuity across the system (retaining historical 
memory). 

Quality 

Maintaining a high quality system from data logging and manipulation all the way to 
dissemination and marketing and implementing appropriate quality procedures for 
products and processes.  

Being timely and up-to-date. 

Value 

Providing and demonstrating value added, nurturing a learning network and 
managing a constructive, changing and flexible system of information providers. 

Disseminating products and promoting and branding the information service. 

6) Availability of the report on Europa:  

Not yet available. 

SME Participation in FP7 - Spring report 

1) ABB activities concerned: 08 13  

2) Timing: 14.6.2011 (duration: March-June 2011, period covered: until the 1st of April 2011) 

3) Budget: FP total budget around EUR 55 billion (Euratom included) for 2007-2013 

4) Background, scope and focus 

Aim  of  the  evaluation  study:  This  "Spring  2011"  report  analyses  the  situation  of  the  FP7  Grant 

EN 91 

Error! Unknown document property name.  EN 

 
 
 
 
 
Agreements signed as of the 1st of April 2011. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

Focusing on the SME participation in the Thematic  Priorities, EUR 1.779,7 million, or 14.4% of 
the Cooperation budget, is going to SMEs. 

For  the  6,544  SME  participations  in  Thematic  Research  Projects  so  far,  the  average  EU 
contribution is 272,000EUR.  

Over  the  full  FP7  programme,  including  Research  for  the  Benefit  of  SMEs  and  the  Marie-Curie 
Actions,  10,127  of  the  participations  are  by  SMEs,  receiving  an  average  EU  contribution  of 
249,000EUR. 

The  Cooperation  Programme  still  has  EUR19,893  million  available  to  spend  (61.7%  of  the  total 
budget  in  the  remaining  years  of  the  Framework  Programme,  until  2013).  From  this  remaining 
budget, 15.4% should go to SMEs in order to reach the 15% target for the whole period.  

The SME strengthening measures for the Work Programmes 2011 are not yet visible in this report. 
The  budgetary  share  of  SMEs  is  forecasted  to  grow  from  the  current  14.4  %  to  15.7  %  of  the 
Cooperation Programme, slightly above the 15% target in the Framework Programme legislation. 
As a specific feature, this report provides a more detailed analysis of the SME support by some of 
the  Joint  Technology  Initiatives  (JTIs).  Overall,  21%  of  the  EU  contribution  to  the  JTIs  goes  to 
SMEs. 

This  report  also  includes  maps  with  a  regional  analysis  of  the  15%  target  in  signed  Grant 
Agreements within the Cooperation Programme.  Tables with the core data per call and on regional 
participation are enclosed in the annexes of the full report.  

6) Availability of the report on Europa: 

 http://ec.europa.eu/research/sme-techweb/pdf/smes-in-fp7-spring-2011_full-rep_en.pdf 

Review of main activities and deliverables of IRMA Administrative Arrangements with 
IPTS (Industrial Research Monitoring and Analysis) 

1) ABB activities concerned: 08 19  

2) Timing: 2012 

3) Budget: Overall budget -EUR 6,487,300 (over 2008 -2011) 

4) Background, scope and focus 

Aim of the study: To provide judgment and advice on:  

(cid:131) 
attainment of objectives as set in the technical specifications of the project. 

The  general  progress  of  the  IRMA  project:  implementation  of  tasks  and  degree  of 

EN 92 

Error! Unknown document property name.  EN 

 
 
(cid:131) 

The quality of the "products", in terms of both academic and policy relevance. 

(cid:131) 
regard to the released "IRMA products" as well as the feed-back received from relevant stakeholders. 

The  impact  of  the  project,  thus  considering  the  dissemination  activities  performed  with 

(cid:131) 
a successful implementation of the next phase of activities? 

What can be learnt for the IRMA follow up: Recommendations and guidance with regard to 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The IRMA project needs to be more focussed, more policy oriented with more integration between the 
various  work  packages  so  they  can  reinforce  each  other.  The  combination  of  the  various  work 
packages should lead to cumulative and stronger ‘stories’ with more political impact. The risk is that 
each  individual  work  package  (Scoreboard,  Working  papers,  Survey,  Case-studies)  has  its  limits, 
without the other work package filling the gap, e.g. if the Survey is too general, the case studies should 
be  chosen  to  deepen  the  issues  that  need  more  depth.  That  is,  they  need  to  inform  each  other.  The 
research papers could be structured by a more programmatic approach focussing on a limited number 
of  topics.  Regarding  the  complementary  relation  between  BERD  and  scoreboard  data,  additional 
analysis  comparing/  combining  the  two  complementary  perspectives  are  recommended.  Additional 
analyses on fast-growing companies and other companies "below the radar" (e.g. new entrants to the 
scoreboard sample) are also recommended.  

6) Availability of the report on Europa: not yet published 

Overview of International Science, Technology and Innovation cooperation between 
Member States and countries outside the EU and the development of a future 
monitoring mechanism 

1) ABB activities concerned: 08 03  

2) Timing: 08.12.2012 

3) Budget: FP7 total budget around EUR 55 billion (Euratom included) for 2007-013 period. 

4) Background, scope and focus: 

The objectives of this study were to provide: 

i)  an overview of EU Member States international STI policies and policy implementation; 

ii)  an  analysis  of  the  evolution  and  trends  in  the  international  STI  cooperation  policies  of  EU 
Member States and their implementation of over the last 10 years; 

iii)  recommendations  for  a  practical  and  cost  effective  methodology  for  monitoring  the 
implementation  of  EU  Member  States'  STI  cooperation  policies  with  international  partner 
countries. 

The study therefore included the following elements: 

• 

Literature review and synthesis of recent studies/reports on international Science, Technology 

EN 93 

Error! Unknown document property name.  EN 

 
 
and Innovation policies, policy expectations and cooperation activities of EU Members States with 
countries outside the EU; 

• 
Preliminary  screening  review  of  international  STI  cooperation  expenditure  by  EU  Member 
States  to  identify  the  most  active  Member  States  in  order  to  focus  the  subsequent,  more  detailed 
data/information gathering activities; 

•  Collation  and  overview  of  the  policy  objectives  of  EU  Member  States  for  international  STI 
cooperation; 

•  Collation  and  overview  of  STI  programmes  of  EU  Member  States  to  support  international 
cooperation; 

•  Collection  of  data  of  EU  Member  States  public  sector  expenditure  on  international  STI 
cooperation; 

Identification  and  analysis  of  EU  Member  States  funding  instruments  for  international  STI 

• 
cooperation; 

Identification  and  analysis  of  EU  Member  States  non-funding  policy  instruments  for 

• 
international STI cooperation; 

• 

Identification of trends in support to EU Member States international STI cooperation; 

•  Analysis  of  the  impact  (financial  and  non-financial)  of  the  implementation  of  public  sector 
international STI cooperation policy of the EU Member States; 

Identification and overview of EU Member States national monitoring and evaluation systems 

• 
for international STI cooperation; 

Preparation of practical recommendations for Member States and the European Commission 

• 
for the monitoring the implementation of international STI cooperation policies. 

5) Summary of findings and conclusions of the evaluation or evaluation related study: 

Data and information relating to MS international cooperation activities exhibit a large variation in 
terms  of  availability  and  level  of  detail  offered  and  a  range  of  instruments  are  employed  to 
implement internationalisation policy. 

Financial  data  on  international  cooperation  would  ideally  provide  a  strong  indicator  of  related 
activity  and  effort.  However,  the  study  has  identified  a  number  of  problems  which  makes  this  a 
difficult undertaking. Data on S&T expenditures at the national level, while available and relatively 
standardised,  is  rarely  disaggregated  into  activities  related  to  international  cooperation,  let  alone 
cooperation  with  third  countries.  The  same  is  valid  for  budgetary  data  for  individual  agencies  or 
councils. Many of the problems concerning the lack of disaggregation stem from the fact that much 
international cooperation in research is driven from the bottom up, particularly through responsive 
mode research funding programmes. The international aspects within research grants often remain 
‘hidden’ and are infrequently collected by research agencies. The increasing tendency for research 
agencies  to  ‘mainstream’  internationalisation  efforts  militates  against  the  collection  of  relevant 
budgetary data Nevertheless, based on the limited financial data that was available, it is possible to 
group  the  reviewed  MS  according  to:  overall  expenditure  on  international  S&T  cooperation  with 

EN 94 

Error! Unknown document property name.  EN 

 
third  countries  and  trends  in  expenditure  on  international  S&T  cooperation  activities  with  third 
countries. 

Bibliometric  data  appear  to  offer  a  very  useful  approach  to  assessing  levels  of  international 
cooperation between countries in terms of identifying major partner countries, research focus and 
trend information. They reflect the bottom up nature of international S&T collaboration very well; 
but  do  not  necessarily  reflect  top  down  policy  priorities.  They  have  a  series  of  disadvantages 
identified and rather surprisingly, their use among the reviewed MS was found to be rather limited 
to  international  benchmarking  exercises  and  little  routine  use  for  monitoring  purposes  was 
encountered. 

A  number  of  the  reviewed  MS  reported  the  use  of  assessments  of  national  internationalisation 
activities  in  S&T  although  few  reported  the  use  of  specific  indicators.  Only  one  of  the  MS 
reviewed  reported  the  use  of  indicators  to  monitor  the  progress  of  S&T  internationalisation 
objectives. Very few agency-level monitoring exercises on international S&T cooperation activities 
were reported. Evaluation of individual programmes relating to international S&T cooperation was 
reported as quite widely established. However, most of the examples tended to focus on issues of 
scientific quality, with publication data tending to form the most commonly applied indicator. 

Based on the analyses carried out the study has proposed a list of indicators that may be applied in 
the  context  of  monitoring  MS’  activities  in  S&T  international  cooperation,  both  with  regard  to 
intra-EU and third country activities. The study also identified a number of barriers to the use of 
indicators  for  monitoring  and  evaluation  purposes,  which  will  need  to  be  addressed.  Finally  the 
study came up with a set of recommendations for the design of a potential system of indicators for 
the  monitoring  of  MS  international  S&T  activities  with  third  countries:  1)  Derive  accepted 
definitions,  2)  Clarification  of  the  purpose  for  indicator  design  and  use,  2)  Prioritisation  of  key 
indicator  requirements,  3)  Systematic  Monitoring  arrangements,  4)  Allocation  of  responsibilities 
and oversight and 5) Derive a framework typology for instruments. 

6) Availability of the report on Europa: not yet published 

Strategy definition and road mapping for industrial technologies to address grand 
challenges 

1) ABB activities concerned: 08 04  

2) Timing: 27.4.2012 

3) Budget: FP total budget around EUR 55 billion (Euratom included) for 2007-013 period. 

4) Background, scope and focus: 

The  main  aim  of  this  project  is  to  assess  the  links  and  relevance  of  present  Nanosciences, 
Nanotechnologies,  Materials  and  new  production  technologies  (NMP)  activities  to  the  major 
technical issues and bottlenecks associated with Grand Challenges, providing a set of operational 
recommendations. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related  study:  The 

EN 95 

Error! Unknown document property name.  EN 

 
 
study identifies a list of critical bottlenecks, R&D, legal and market obstacles, some of which are 
political in nature. These underlying bottlenecks need to be addressed first, in order to be able to set 
out  constructive  development  processes  in  the  context  of  the  Grand  Challenges.  They  require 
concentrated cooperation with other Directorates of the Commission as well as policy initiatives on 
a higher level. 

There  is  a  shift  towards  priorities  addressing  the  Grand  Challenges  across  the  OECD  and  third 
countries. However these programmes vary in their focus on research, innovation and technologies; 
whilst a number of countries have developed programmes that support scientific and technological 
R&D in many or all of the Grand Challenges, far fewer countries have developed programmes of 
innovation support. 

The  study  recommends  encouraging  competitive  clusters  that  manage  and  implement  R&D 
projects  would  be  a  strong  strategy  to  address  the  needs  for  more  commercialization  and  higher 
competitiveness as a policy options. 

Recommendations for the European Commission: 

Building  the  design  of  policies  for  the  benefit  of  European  innovation  and  economic  growth  on 
more comprehensive and well-informed social and economic studies.   

Policy measures should aim at strengthening European corporate actors, and find ways to support 
decreasing  levels  of  R&D  funding  by  European  companies.  This  would  include  predictability  of 
regulatory regimes, tax credit schemes, and other investment incentives.  

The  European  Commission  should  focus  on  technologies  already  close  to  the  market  today, 
searching for demonstration and scaling up solutions. The EC should support actions for regulatory 
tools to implement existing technologies in need of a bigger market to become competitive.  

Undertake actions to create an open European VC market.  

Introducing  cluster-driven,  large  scale  regional  programmes  for  industrial  technologies  under 
Horizon 2020.  

Act more proactively as facilitator to attract and pool more national funds for joint activities in the 
area of key enabling technologies.  

Separate  projects  related  to  awareness  building,  testing  and  education  about  KETs  and  their 
possible influence on humans.  

European-wide  projects  oriented  towards  integrating  innovation  results  with  cultural  expression 
and social science investigation.  

Results  of  frontier  research  projects  should  undergo  screening  by  skilled  engineers  and  other 
relevant  professionals  in  the  relevant  field  before  publication,  as  there  is  a  risk  of  intellectual 
property leakage.  

Support  for  market-oriented  public-private  partnerships  should  be  specifically  implemented  in 
areas that show strong science-technology linkages, such as chemicals, drugs, instrumentation and 
electronics, or other that may surface during thorough assessments of different research fields.  

EN 96 

Error! Unknown document property name.  EN 

 
Partners  participating  in  EC  funded  collaborative  efforts  to,  e.g., solve  Grand  Challenges,  should 
also sign up to a detailed and committing exploitation plan before embarking on the project, all the 
way down to who will build pilot and implement the manufacturing process.  

6) Availability of the report on Europa: not yet published 

2011 Monitoring Report of FP7 

1) ABB activities concerned  08 02  

2) Timing: 29.08.2012, (period covered: 2007-2011) 

3) Budget: FP7 total budget around EUR 55 billion (Euratom included) for 2007-013 period.  

4) Background, scope and focus: 

The  Commission  has  a  legal  obligation  to  continually  and  systematically  monitor  the 
implementation  of  the  FP7  and  its  specific  programmes  and  regularly  report  and  disseminate  the 
results of this monitoring (FP7 Decisions (EC and Euratom, Articles 7(1) and 6(1)). 

FP7  monitoring  is  an  annual  exercise,  with  the  resulting  report  covering  the  year  preceding  the 
report's publication. The main objective of the 2011 Monitoring Report was to comply with these 
requirements and to report about the implementation of FP7 in 2011. 

5) Summary of findings and conclusions of the evaluation or evaluation related study: 

The 46 calls  for proposals with call closure date in  2011 recorded in CORDA by February 2012 
attracted  in  total  16.212  applications  for  funding.    In  February  2012  included  and  retained 
proposals involved a total of 59.955 and 12.932 applicants respectively with an overall success rate 
of 22%. The so-far recorded numbers of applicants in retained proposals are almost the same as in 
2010  (13.710),  but  significantly  lower  than  those  recorded  in  2009  (19.471),  while  their  success 
rates are lower to those of last year (23,9%) and is the average for the five years (22%). 

The aggregate figures for the period 2007-2011 show that for a total of 307 concluded calls, 95.862 
proposals  were  submitted,  out  of  which  79.145  –  involving  386.812  applicants  –  were  included, 
and 16.089 – involving 85.248 applicants – retained for negotiations. The average success rate for 
the five years period was 20% in terms of proposals and 22% in terms of applicants. 

Under  the  Seventh  Framework  Programme  (FP7)  Framework  Programme  on  Research,  in  2011, 
40% of participants in retained proposals came from the Higher and Secondary Education sector, 
so mainly from universities, 26% from industry, and 23% from research organisations.  

On  the  gender  dimension  of  FP7  participation,  it  is  estimated  that  20%  of  contact  persons  for 
scientific aspects in FP7 funded projects are female. SMEs received 16.3% of the funding for the 
Cooperation Specific Programme compared to the initial aim of at least 15% SME funding.  

The  significant  international  dimension  of  FP7  is  illustrated  by  the  fact  that  during  five  years  it 

EN 97 

Error! Unknown document property name.  EN 

 
 
 
funds projects with participant organisations from as many as 169 countries. Outside the group of 
EU  and  Associated  Countries  the  biggest  participants  are  the  USA,  China,  Russia,  Brazil,  South 
Africa, India, and Ukraine. 

On the redress and ethical review procedures, out of the 2.678 requests for redress received, only 
48  led  to  a  re-evaluation,  whereas  1.382  ethical  reviews  were  organised  so  far  with  no  project 
having been stopped. 

Up  to  February  2012,  over  14,000  grant  agreements  have  been  signed  and  over  1,000  projects 
completed. Based on the final reports of the completed projects, on average each project produced 
eight publications, four being 'open access'. On average, FP7 projects generated twenty-two direct 
full-time equivalent (FTE) jobs. 

6) Availability of the report on Europa:   

http://ec.europa.eu/research/evaluations/pdf/archive/fp7_monitoring_reports/fifth_fp7_monitoring_rep
ort.pdf#view=fit&pagemode=none 

International cooperation in Science, Technology and Innovation: Strategies for a 
Changing World 

1) ABB activities concerned 08 03 

2) Timing: September 2012 

3) Budget: FP7 total budget around EUR 55 billion (Euratom included) for 2007-013 period. 

4) Background, scope and focus: 

With the development of the European Research Area since 2000 a clearer focus on the need for 
more  synergy  and  effectiveness  in  European  STI  efforts  has  developed.    The  policy  context  of 
Horizon  2020  pays  increasing  attention  to  the  need  for  economic  growth,  competitiveness  and 
innovation.  Against  this  backdrop,  the  Commission  set  up  an  Expert  Group  on  international 
science,  technology  and  innovation  cooperation  to  provide  advice  for  the  further  development  of 
international cooperation policy and the international dimension of ERA. The EG was launched in 
parallel  with  the  preparation  of  two  important  policy  initiatives  from  the  Commission:  A 
Communication  on  the  ERA  Framework  and  a  Communication  on  a  European  Strategy  for 
international cooperation in STI to be published in the early summer and early fall respectively.  

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The main message coming from the Expert Group is that the EU urgently needs a collaborative and 
integrated strategy for international cooperation in STI. With this in mind several recommendations 
are launched: 

1. The strategy should focus on promoting European attractiveness as an international research and 
innovation hub and partner in order to strengthen European competitiveness and prosperity. 

2. Theme- and problem-oriented prioritization is needed rather than geographic; Grand Challenges 
as  a  clear  prioritization  tool  should  be  mainstreamed  also  in  the  international  dimension. 

EN 98 

Error! Unknown document property name.  EN 

 
 
Prioritization  of  international  collaboration  should  follow  closely  the  priorities  of  the  EU’s  core 
research  and  innovation  programmes,  while  the  geographical  approach  should  be  the  core  of  an 
implementation strategy. 

3. Make the Horizon 2020 truly open and attractive to the best and brightest in the world allowing 
European actors to work with the best brains wherever they are. 

4. The international perspective needs to be more fully integrated into ’regular’ programmes at EU 
level. 

5.  Variable  geometry  should  be  exploited  to  the  full,  with  flexible  arrangements  (within  EU  and 
with  countries  outside  EU)  including  multilateral  platforms  for  strategic  cooperation.  Variable 
geometry initiatives should also build on lead initiatives by individual Member States that expand 
their successful bilateral activities to several European partners. 

6. A strong focus on firms and innovation is needed. This has not been properly addressed before 
and  it  requires  a  new/different  approach;  there  are  fundamental  differences  in  drivers  of 
international cooperation between academia and industry and between research and innovation. 

7. Reinforce efforts to strengthen framework conditions for and removal of barriers to international 
cooperation. 

8. Design targeted initiatives for strengthening cooperation in selected (prioritized) areas: these can 
be multilateral, bilateral, and unilateral. The key criteria should be achieving benefits for European 
stakeholders,  effectively  address  global,  grand  challenges,  and  support  the  Union’s  external 
policies. 

9.  All  initiatives  must  be  based  on  more  evidence-  or  analysis-based  decision-making,  including 
forward  looking  analysis  to  inform  decision  making  about  likely  trends  and  future  changes  and 
systematic exchange of experiences. 

6) Availability of the report on Europa:  

http://ec.europa.eu/research/iscp/pdf/report-inco-web-4.pdf 

EN 99 

Error! Unknown document property name.  EN 

 
ABB 09 – Information Society and Media 

Future impact of ENIAC and ARTEMIS - SMART 2012/0050 - IAV 

1) ABB activities: 09 04 

2) Timing: 28.11.2012 (duration: 16/06/2012 and 14/12/2012) 

3) Budget (2010+2011): EUR 850 million (total of the ENIAC and ARTEMIS current  
    ceiling budgets) 

4) Background, scope and focus: 

The objective of the study was to support preparation of the Commission proposal for the 
future ENIAC/ARTEMIS JTI. The study was to  analyse the impact of potential synergies 
between the ARTEMIS and ENIAC JTIs in terms of industrial relevance (research agendas, 
stakeholders,  volume  of  activities)  and  efficiencies  /  economies  of  scale,  including  the 
relationship with relevant Eureka clusters. The objective of the study was also to analyse / 
benchmark  national  practices  in  terms  of  financial  commitments,  procedures  and  funding 
sources  and  rates.  Finally,  the  study  was  to  propose  key  performance  indicators  for  a 
potential future JTI, with respect to socio-economic data and strategic achievements. 

5) Summary of findings and conclusions of the evaluation:  

in 

the  global 

innovation  system 

The report provides some key findings and further two sets of recommendations related to 
the  strategic  research  agenda  and  the  implementation  of  the  future  Joint  Technology 
Initiative. According to the study team, JTIs definitely need to be continued. There is a need 
to continue to support the European electronic components and systems industry in view of 
strengthening  the  position  of  the  industry  in  global  value  chains  and  of  reinforcing  the 
European 
innovation  network.  There  are  clear 
complementarities and strong links between the various parts of the value chain covered by 
ENIAC and ARTEMIS. These links are set to grow in the future as the components industry 
seeks to capture the opportunities offered in the higher parts of the value chain. Embedded 
systems suppliers will also seek to exploit more the possibilities brought by the increasing 
performance  and  functionalities  of  components.  It  is  clear  though  that  progress  will  also 
require  focused  research  and  innovation  actions  within  the  various  parts  of  the  chain 
covered by the two JTIs and that needs to be led by appropriate stakeholders including not 
only  large  companies  but  also  SMEs.  The  study  recommends  setting  up  a  one  stop-shop 
efficient  and  effective  operational  unit  for  managing  programmes  for  the  European 
electronic components and systems industry, combining the current ENIAC and ARTEMIS 
JUs  in  a  single  entity.  And  it  involves  creating  room  for  cross-cutting  programmatic 
activities to better accommodate an already visible trend. The first set of recommendations 
relates to the development of the research and innovation agenda. First the study identifies 
that  opportunities  and  conditions  need  to  be  created  to  strengthen  the  focus  on  key  areas 
from  the  double  perspective  of  positioning  EU  industry  in  the  global  value  chain(s)  and 
strengthening the relevant parts of the European innovation ecosystem in global innovation 
networks. In view of a need for focus and critical mass, it is important to develop the new 
JTI as a limited number of dedicated and focused research agendas linked to the respective 

EN 100 

Error! Unknown document property name.  EN 

 
 
 
stakeholder communities. Especially a clear focus on the European value added will deliver 
more than the sum of national or single firm interests; the closer to market support may lead 
to a gradually growing involvement of users in the new JTI; and balancing short term and 
long  term  interests  potentially  becoming  a  key  issue,  applying  Triple  Helix  principles,  in 
which  each  stakeholder  group  has  its  own  role,  seems  to  lead  to  balanced  outcomes.  In 
addition,  there  are  significant  areas  of  synergy  and  some  overlap  between  the  JTIs  which 
underlines  the  importance  of  creating  room  for  cross-cutting  activities  between  the  JTIs. 
Furthermore,  agility  and  flexibility  are  needed  to  face  increased  development  speed  and 
nearing of technological barriers. It is thus advised to create room for identifying new needs 
and  opportunities  which  may  come  in  parallel  to  the  presently  envisioned  roadmaps.  And 
finally  the  participation  of  SMEs  is  better  than  generally  perceived,  but  still  needs 
strengthening. To this end it seems to be vital that networks in which SMEs are embedded 
(often building on regionalised clusters) are easily recognisable and accessible. The second 
group  of  recommendations  deals  with  issues  of  implementing  this  agenda  and  related 
governance and funding issues. Since the goal of this study was not intended to go into the 
detailed issues of European law and regulations, the focus was on key general issues which 
may give direction to the more detailed implementation. The analysis of available sources 
shows  a  strong  preference  for  a  “one-stop-shop”  organisation  while  at  the  same  time 
confirming  the  need  for  continuation  of  the  present  tri-partite  model  in  which  European 
Union  and  Member  States’  funding  comes  together  with  industry  funding.  Unified 
operations and making the one-stop-shop work require considerable effort on aligning the 
research and innovation agendas, even when a limited set of different agendas is foreseen. 
This task might become easier if industry can be incentivised to present stronger focus and 
priorities  which  may  lead  to  better  focused  proposals  and  better  identification  of  cross-
cutting issues, in turn expected to be a better basis for multilevel alignment of funding and 
other supporting actors. Moving the programmes closer to applied innovation requires also 
creating  opportunities  for  more  flexibility  and  agility  in  the  management  of  the 
programmes.  Inevitably  the  management  of  the  programmes  gets  deeper  entangled  in  the 
competitive  operations  of  firms  which  requires  recognition  of  the  role  of  the  programme 
managers.  

6) Availability of the report on Europa: 

http://ec.europa.eu/information_society/newsroom/cf/dae/itemdetail.cfm?item_id=9974 

Study on the evaluation of BEREC and the BEREC Office - SMART 2011/0009 

1) ABB activities: 09 02 

2) Timing: 21.12.2012 (Duration of the study: 9 months, Period covered by the report: January 
2010- December 2012) 

3) Budget (annual): not applicable 

4) Background, scope and focus 

According  to  Article  25  of  the  Regulation  establishing  the  BEREC  and  its  Office,  the 
Commission  needs  to  publish  a  Report  on  the  experience  acquired  as  a  result  of  the 
operation of BEREC and the Office within three years of the effective start of operations.  

EN 101 

Error! Unknown document property name.  EN 

 
 
 
The  evaluation  report  shall  be  forwarded  to  the  European  Parliament  and  to  the  Council. 
The European Parliament shall issue an opinion on the evaluation report. 

The  objectives  of  this  study  were  to  provide  an  assessment  of  the  results  achieved  by 
BEREC and the Office and their respective working methods, in relation to their objectives, 
mandates  and  tasks  defined  in  the  BEREC  Regulation  and  in  their  respective  work 
programmes. 

In  order  to  comply  with  the  above  obligation  and  to  ensure  an  impartial  evaluation,  DG 
CONNECT launched a call for tender for an external study and the contract was awarded to 
Price-Waterhouse  Coopers  (PWC).  The  work  of  the  contractor  has  been  coordinated  by  a 
Steering  Committee,  including  representatives  from  the  Commission,  BEREC,  market 
players  (ETNO  and  ECTA)  and  consumer  associations  (BEUC).  The  conclusions  of  the 
study of PWC were also submitted to a public workshop, which took place in October 2012. 
The evaluation Study was published by DG CONNECT on 21 December 2012, on Europa. 

5) Summary of findings and conclusions of the evaluation:  

The  key  findings  of  PWC  Evaluation  Report  regarding  the  evaluation  of  BEREC  and  the 
BEREC Office are the following: 

(1)  The  structure  of  BEREC  is  overall  relevant  and  efficient.  It  has  so  far  fulfilled  its 
functions rather successfully, in particular under Article 7/7a procedures, but there may be 
elements for improvement. Furthermore, when considering the effectiveness of the platform 
to achieve its requirements and objectives, BEREC may be considered, until now, a success. 

(2) BEREC can play a significant role to harmonise the electronic communications market. 
It  may  fulfil  this  activity  through  the  development  and  dissemination  among  NRAs  of 
regulatory  best  practices  on  the  implementation  of  the  regulatory  framework,  through  its 
advisory role and through reports and common positions which should serve as guidelines 
for  NRAs  towards  a  harmonised  approach.  However,  the  advisory  role  should  be  further 
defined  and  the  development  of  an  EU  approach  different  from  the  national  interests  is 
sometimes difficult. 

(3) The advisory role of BEREC is not sufficiently defined, despite the clear willingness of 
the  European  Commission  and  other  EU  institutions  to  employ  BEREC  as  such.  Some 
stakeholders  have  argued  that  BEREC  does  not  shed  enough  light  on  emerging  issues  or 
propose recommendations and/or guidelines to face them; it needs to be more proactive in 
choosing topics to tackle and in recommending clear solutions to the EU Institutions. The 
study indicates the difficulty of getting affirmed positions within BEREC - this will require 
a significant cultural change among NRAs. BEREC, being a bottom-up regulatory model, 
exemplifies  in  some  cases  more  national  considerations  than  a  pure  EU  single-market 
driven approach. 

(4) The independence of BEREC vis-à-vis the individual NRAs could be improved, in the 
sense of developing a collective European thinking different from the national interests of 
the NRAs which form it. BEREC, as a single entity, should be more focused on missions 
that concern the Single Market: harmonisation of the internal market and empowerment of 
EU  consumers.  Furthermore,  BEREC  has  to  be  independent  from  any  government  or 
stakeholder. In order to achieve this, it  is of utmost importance that, at the national level, 
each NRA composing BEREC carries out its functions independently. 

(5) Better ensure the accountability of BEREC towards its own objectives. BEREC should 

EN 102 

Error! Unknown document property name.  EN 

 
be more accountable for the tasks it chooses to tackle by itself, meaning the tasks included 
in its Work Programmes  and the Mid-Term Strategy. To do so, BEREC could indicate in 
each  Annual  Work  Programme  the  commitments  chosen  for  the  year  and  in  each  Annual 
Report detail what has been achieved with relation to these objectives, for example progress 
towards its long-term goals: In that perspective, BEREC should reflect on Key Performance 
Indicators to assess its own progress. However, BEREC's increased accountability shall not 
be at the detriment of its capacity to take up additional tasks on emerging issues, on its own 
initiative or upon request of the EU Institutions. If BEREC could express clearly the topics 
it  considers  as  key,  it  could  then  define  priorities  and  could  strengthen,  first,  its  role  of 
advisor to the EU institutions regarding the harmonisation of the Single Market and, second, 
its role with regard to NRAs in relation to benchmarking, snapshots sharing and exchange 
of best practices. 

(6) As to the efficiency of the organisation, the scope and the regular review of the BEREC 
Work  Programme  could  be  enhanced,  in  order  to  achieve  greater  prioritisation.  The 
decision-making process should be more top-down and provide more room to the Board of 
Regulators (BoR) to take strategic decisions. BEREC is currently working with a bottom-up 
approach, with the technical work done at the EWG level. Moreover, the heavy agenda of 
Plenary  Meetings  prevents  the  BoR  members  from  discussing  strategic  issues  and 
considering  the  future  role  of  BEREC  with  regards  to  the  evolution  of  the  electronic 
communications market. 

(7) The use of the BEREC Office needs to be clarified and improved. It is the responsibility 
of the whole BEREC platform to best utilise the BEREC Office for both administrative and 
professional  purposes.  As  per  the  professional  support,  the  expertise  of  the  Office  staff  is 
today not used as much as it could or should be. In that perspective, BEREC should decide, 
together with the Office and in line with the Regulation, on the exact tasks of the Office and 
on the role and responsibilities of each actor. 

6) Availability of the report on Europa: 

http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=1403 

Inventory and Review of Spectrum Use: Assessment of the EU potential for 
improving spectrum efficiency - SMART 2011/0016 - PC 

1) ABB activities: 09 02 

2) Timing: 11.09.2012 (Period covered by the report: 10 months) 

3) Budget (annual): not applicable 

4) Background, scope and focus:  

Spectrum is a scarce resource, but more and more wireless services need access in order to 
support important Union policy objectives. The need for a review and inventory of existing 
and  future  spectrum  use  was  clearly  accepted  at  the  Spectrum  Summit  and  in  the  RSPG 
Opinion in order to identify available spectrum but also to improve the efficiency of current 
use.  A  detailed  investigation  of  actual  spectrum  use  and  future  needs,  as  foreseen  in  the 
proposed  Radio  Spectrum  Policy  Programme,  require  expertise  and  resources  that  go 
beyond what is available in house.  

EN 103 

Error! Unknown document property name.  EN 

 
 
5) Summary of findings and conclusions of the evaluation:  

The  Study  included  a  small-scale  trial  run  of  an  inventory  which  was  simulated  by 
gathering as much spectrum usage information as possible on the supply side.  The aim was 
to  gain  experience  and  draw  conclusions  on  the  methodology,  and  to  identify  specific 
frequency bands where efficiency could be improved. 

The main results and messages of the Study are: 

•  Gathering  information:  Information  on  the  planned  or  intended  use  (allocation  and 
assignment)  information  is  generally  available  at  Member  States'  level,  partly  also 
aggregated at EU level. However, as for the real usage of the spectrum, Member States only 
partly collect data that is useful for assessing the situation, often only in selected bands. In 
general, Member States currently have little data on usage. The Study results have shown 
that  there  is  considerable  variation  in  the  scope  and  level  of  detail  of  information  on 
spectrum  usage  held  by  Member  States.  This  status  quo  points  towards  a  need  to  further 
build  up  our  knowledge  base  in  Europe  so  as  to  improve  efficient  use  of  spectrum  over 
time. 

•  Proposals  for  methods  to  assess  data:  A  certain  number  of  generic  criteria  forming 
multiple metrics are proposed to evaluate the efficiency of spectrum use. These efficiency 
criteria  were  developed  evolving  around  the  notion  of  utilisation  of  spectrum,  demand 
trends, technology inherent factors and geographic extent of deployment so as to evaluate 
technical  efficiency.  These  are  to  be  complemented  by  economic  and  social  value 
assessments  which  are  of  more  complex  nature  and  do  not  directly  result  from  the  data 
gathered, but need to be assessed in light of the specific measures being proposed to remedy 
the inefficient use. 

• First identification of "suspect" bands based on technical usage efficiency: In applying this 
method  the  Study  identified  a  first  set  of  bands  which  is  underutilised  or  not  used  at  all. 
While several bands confirm known problematic cases, these results should not be taken as 
final  results,  but  as  a  demonstration  how  the  proposed  method  leads  to  concrete 
deliverables. 

The results of the Study are a basis for the Commission's work on the spectrum inventory. 
They  also  guide  discussions  with  Member  States  on  a  bilateral  basis  and  in  the  Radio 
Spectrum Committee, in order to clarify the positions of the Member States with regard to a 
proposal from the Commission for an implementing act on spectrum inventory.  

A potential limitation can be the accuracy of data gathered by consultants. 

6) Availability of the report on Europa: 

http://emplaps1.empl.cec:7779/scc-upl-infso/67703.pdf 

Development of impact measures for e-Infrastructures - SMART 2010/0051 

1) ABB activities :09 05 

2) Timing: 7 March 2012 (Period covered by the report: 14 months) 

3) Budget (annual): The total EU contribution to the projects selected for the study 

EN 104 

Error! Unknown document property name.  EN 

 
 
 
     survey is EUR 211 million. 

4) Background, scope and focus: 

The  study  was  conceived  to  contribute  to  FP7  mid-term  and  final  assessment  with  the 
development of measures and indicators and the evaluation of results and impact of the e-
Infrastructures  part  of  the  Capacities  programme.  It  served  as  a  pilot  for  e-Infrastructure 
program impact assessments. Its main function was to gain knowledge of and insights into 
the direct and indirect impacts and achievement of objectives at the program level. This was 
approached  by  analysing  single  projects  as  components  of  the  program  to  understand  the 
program's impacts, pertinence and acceptability 

5) Summary of findings and conclusions of the evaluation:  

The study developed a set of five general characteristics that are representative of the goals 
of the e-Infrastructure program: 

•  Accessibility:  provide  access  to  scientific  data,  scientific  information  (e.g.  scientific 
publications)  and  infrastructure  capacity  based  on  interoperable,  standardized  platforms. 
Examples  of  this  are  access  to  unique  facilities  (EVALSO)  or  access  to  scientific 
information (OpenAIRE, Geo-Seas). 

• Efficiency: contribute to a more efficient way of working by providing improved or new 
problem  solving  capacities  (software  and  infrastructures).  Examples  of  this  are  EGI  and 
PRACE providing efficient computing capacities. 

•  Innovativeness:  contribute  to  fields  beyond  services  to  science  by  laying  down  new 
foundations in ICT and exploring usage potential in other areas. Examples of this are Géant 
activities  towards  Future  Internet  or  the  MMM@HPC  objectives  for  the  better  use  of 
infrastructures for industrial users. 

•  Sustainability:  contribute  to  interoperable  sustainable  availability  of  scientific  data, 
information and capacities. One example is the work of SIENA towards standardisation of 
data infrastructures. 

•  Transformative  Character:  contribute  to  the  process  of  transformation  in  scientific  work 
towards  e-science  through  increasing  collaboration  between  researchers  from  different 
locations  and  disciplines  and  enhancing  human  capital  for  e-science.  For  each  of  these 
general characteristics, intervention logic and relevant survey items and data analysis tools 
were  developed.  The  study  surveyed  21  projects  to  test  and  refine  survey  questions  and 
assessed outputs, outcomes and impacts based on the pilot survey results. In addition to the 
quantitative  analysis  of  the  survey  data,  case  studies  were  carried  out  to  gain  a  better 
understanding  of  the  interrelations  of  the  five  characteristics.  The  study  draws  two  main 
conclusions regarding the monitoring and evaluation of e- Infrastructures. A first conclusion 
is that continuous monitoring is needed to understand the evolution of e-Infrastructures and 
to support the e-Infrastructures program in fostering their development. However, there are 
limitations  in  using  survey  data  collection  as  a  single  source  for  evaluations.  Therefore, 
simple  standardised  surveys  should  be  complemented  with  more  rich  data  gathering  at 
regular periods, in terms of more extensive surveys, case studies or interviews, for example. 
The  second  main  conclusion  is  that  a  “one  size  fits  all”  approach  for  evaluation  is  not 
sufficient for the e-Infrastructures projects. There must be differentiated approaches to take 
into account the differences in the specific domain, size and objectives of projects under the 
programme. Further actions on this field should not rely on one single data gathering and 

EN 105 

Error! Unknown document property name.  EN 

 
analysis  approach  but  aim  to  find  the  best  combinations  of  approaches  to  cover  all  e-
Infrastructures  projects.  The  study  suggests  that  the  regular  monitoring  system  to  be 
embedded  in  yearly  reporting  of  e-Infrastructures  projects  should  contain  the  following 
items: 

1. Change of available resources 

2. Efficiency gains for users/user projects 

3. Cooperation and collaboration with other projects 

4. Number of user-reported innovations 

5. Number of announced patents 

6. Developed standards 

7. Origin of users 

8. Use of results in education or training 

9. Number of scientific or engineering disciplines addressed 

10. Training provided for participants and users. 

The study cooperated with the ERINA+ and the eNventory projects, and helped to identify 
better  impact  indicators.  The  study  also  contributed  to  the  definition  of  metrics  for  the 
eInfrastructure  unit  and  indirectly  helped  to  build  the  indicators  for  e-infrastructures 
projects  evaluation  and  assessment  at  programme;  also  for  future  Horizon  2020  Research 
Infrastructures activities. 

6) Availability of the report on Europa: 

http://emplaps1.empl.cec:7779/scc-upl-infso/65561.pdf 

Interim Assessment of the Future Internet Public-Private Partnership 

1) ABB activities : 09 04 

2) Timing:  May 2012, (Period covered: January 2011 – May 2012) 

3) Budget:  EUR 90 million for projects in place (the overall programme EUR 300 million) 

4) Background, scope and focus: 

The Future Internet Public Private Partnership Programme (FI-PPP) is an initiative launched 
under  the  7th  European  Commission  Framework  Programme  which  consists  of  public-
private research and development partnerships concerning the Internet of the future. 
It  aims  to  advance  Europe's  competitiveness  in  Future  Internet  technologies  and  systems 
and to support the emergence of Future Internet-enhanced applications of public and social 
relevance.  It  addresses  the  need  to  make  public  service  infrastructures  and  business 
processes  significantly  smarter  (i.e.  more  intelligent,  more  efficient,  more  sustainable) 
through tighter integration with Internet networking and computing capabilities. 

The Competitiveness Council of May 2012 acknowledged that Public Private Partnerships 
(PPPs)  are  crucial  for  addressing  European  socio-economic  challenges  and  requested  a 

EN 106 

Error! Unknown document property name.  EN 

 
 
 
mechanism  for  evaluating  the  long-term  socio-economic  impact  of  the  PPPs  and  their 
activities and the Council and European Parliament be informed on the results of the above 
analysis on the progress of PPPs.  

The objectives of the FI PPP Programme interim assessment were: 

• 

to evaluate the concept developed for the FI-PPP  

• 
to assess the overall progress achieved in the first year from the launch, including the 
cost-effectiveness  and  efficiency  of  the  PPP's  mechanisms  and  structures  as  well  as  the 
prospect for the PPP's reaching its objectives 

to come up with conclusions and recommendations to the European Commission and 

• 
to the participants in the FI PPP Programme 

• 

to bring forward proposals for how to further develop the FI-PPP 

to  contribute  to  the  preparatory  work  for  the  detailed  work  programmes  of  Horizon 

• 
2020, notably the guidelines for PPP 

5) Summary of findings and conclusions of the evaluation  

The Panel that performed this Interim Assessment of the FI-PPP finds that: 

• 
the FI-PPP has been a valuable experiment in attempting to achieve impact similar to 
that  of  Joint  Technology  Initiatives  (JTIs)  but  in  a  much  shorter  timescale  than  JTIs  by 
using the existing instruments of the 7th Framework Programme; 

• 
the decision to use the instruments and processes of the 7th Framework Programme to 
establish the programme was valid in enabling a rapid response by the EU to technological 
and market developments; 

• 
the market and technological situation is such that the aims of the FI-PPP to accelerate 
technological  development  and  take-up  by  engaging  early-adopter  users  to  identify  their 
needs for infrastructure enabled by the future internet are still valid. 

However the Panel also finds that: 

the industrial participants in the FI-PPP are not, in concert, fulfilling the role envisaged 

• 
for them in a public-private partnership; 

the projects supported within the FI-PPP are, generally, making progress toward their 

• 
own goals but not co-operating sufficiently so as to achieve the goals of the programme. 

6) Availability of the report on Europa: 

http://www.fi-ppp.eu/interim-assessment-of-the-future-internet-public-private-partnership-
may-2012/ 

Implementation of the DAE - Action of the DAE - Actions under the responsibility 
of member states - SMART 2011/0023 – IAV 

1) ABB activities: 09 02 

2) Timing: December 2012 (Period covered by the report: December 2010-September 2012) 

EN 107 

Error! Unknown document property name.  EN 

 
 
 
3) Budget (annual): N/A 

4) Background, scope and focus: 

The Digital Agenda for Europe (DAE) is the European Commission's flagship programme 
to boost the economy via ICT and digital technologies and consists of seven pillars and 101 
actions.  Research  was  undertaken  to  identify  progress  of  the  actions  for  which  Member 
States  are  directly  responsible,  highlight  success  factors  and  the  main  difficulties 
encountered in the implementation of the DAE at national level. The findings contributed to 
the  DAE  Review,  an  assessment  that  reported  on  progress  up  to  December  2012; 
furthermore, the Review identified new actions to provide renewed impetus to the original 
DAE actions. 

5) Summary of findings and conclusions of the evaluation:  

The study contributed greatly to the knowledge and understanding of DAE progress in the 
27  EU  Member  States  and  three  associated  countries.  The  framework  was  designed  to 
identify success factors and main difficulties occurred in the implementation of the Digital 
Agenda  at  national  level.  This  analysis  contributed  to  the  DAE  Review,  which  identified 
key policy areas and new actions to give impetus to the original DAE targets for the next 
two years (2013-2015), with particular reference to the achievement of the key performance 
targets .By creating a single, user-friendly portal, Member State representatives were able to 
upload the data required at their own convenience (the contractors also received automatic 
notification  that  updates  had  been  submitted).  This  allowed  for  a  simple  and  effective 
reporting  process.  Limitations:  the  data  was  largely  self-reported  by  Member  States,  with 
some  assessment  by  the  contractors;  in  cases  where  data  was  missing  or  incomplete,  the 
contractors undertook desk research. Best-practices reported by one Member State may not 
necessarily work in other Member States .Assessing findings and best-practices less than 24 
months after DAE implementation may be premature to draw firm conclusions. 

6) Availability of the report on Europa: 

Not yet available on Europa; currently available at 
http://daeimplementation.eu/misc/DG_CONNECT_iDAE_Final_Report.pdf 

 Study into the impact of FP6 IST 

1) ABB activities: 09 04 

2) Timing: July 2012, (Period covered by the report: 2002-2006) 

3) Budget (annual): around EUR 3.9 billion over 2002-2006 

4) Background, scope and focus:  

Consultants  Avedas,  Logotec  and  CWTS  carried  out  an  independent  study  for  the 
Commission  on  the  medium  to  long-term  impact  of  Framework  Programme  6  (FP6)  IST. 
The overall purpose of the study was to assess the impact of Community supported research 
in ICT in FP6, in view of refining the strategy and designing future policy in this field. The 

EN 108 

Error! Unknown document property name.  EN 

 
 
 
 
study  results  are  based  on  solid  evidence  gathered  (large  survey  with  more  than  1600 
returned questionnaires and 50 case studies). 

5) Summary of findings and conclusions of the evaluation: 

The  study  led  to  an  overall  positive  assessment  of  FP6  IST.  The  three  top-reasons  for 
participating  in  FP6-IST  projects,  virtually  recognized  by  all  organization  were  the 
following:  exploration  of  new  scientific/technological  areas,  strengthening  or  creation  of 
new  RTD  collaborations  and  increase  in  staff  capabilities  and  expertise.  As  part  of  the 
programme was based on specific industrial priorities, it was successful in creating network 
structures  around  leading  industrial  players  with  the  primary  aim  to  efficiently  solve 
technological/scientific  challenges.  In  some  areas  (e.g.  Components  and 
specific 
Microsystems) projects integrated the key players across different parts of the value chain, 
leading  to  medium  term  competitive  advantages  and  higher  wider  impact  on  innovation. 
Programme  activities  focused  on  longer  term  scientific  objectives  were  found  to  be 
successful in creating network structures around universities and research centres. This was 
the  case  of  networks  formed  in  e.g.  "knowledge  &  interface  technologies"  or  FET.  The 
study  also  found  that  FP6-IST  had  positively  affected  a  large  proportion  of  the  EU  R&D 
workforce  on  ICT  and  produced  major  spill-over  benefits  that  are  considered  to  largely 
exceed the direct benefits of projects to Programme participants. The important amount of 
the  knowledge  produced  in  FP6-IST  was  published  in  leading  scientific  and  technical 
journals  and  was  presented  in  high  profile  conferences  and  exhibitions.    Exploitation 
however was found to be weak. While output at the project level was generally estimated as 
effective  and  competitive,  when  estimating  outcome  and  impact  of  the  programme  after 
some  years,  the  positive  effects  were  downscaled.  The  main  underlying  mechanism  that 
explains  the  drop  down  is  the  degree  to  which  a  project  is  crucial  to  the  participating 
organizations. Most coordinators of virtually all organization types recognized SMEs as the 
main innovation actors in projects. In the case of commercialisation, SMEs often played the 
role of a catalyst or driver, driving exploitation towards marketable products and services. 
But very few SMEs tend to bet their company on the expected results of an FP project; the 
project  is  important  but  not  core  to  their  business  strategy  but  they  see  it  more  as  a 
complement to their activities.  

The study provides a number of ideas and recommendations on how to refine our strategy 
for more innovation and offering a better track to exploitation: 

- Enhancing the role of European SMEs as key innovation actors 

- Drawing attention and focus of Programme participants on market innovation 

- Reviewing the existing rules for IPR 

- Strengthening international cooperation targeting world leading scientists 

- Promoting a more innovation-oriented mentality at the level of programme design 

- A more entrepreneurial approach towards the Innovation Union 

The study results were an important input to H2020 programme preparations.  

The  focus  on  Innovation  has  been  a  primary  concern  in  designing  the  future  R&D&I 
programme  for  the  coming  programming  period  (2014-2020).  Horizon  2020  includes  not 
only  research  and  innovation  funding  currently  provided  through  the  Framework 
Programmes, but also the innovation activities previously funded under the Competitiveness 

EN 109 

Error! Unknown document property name.  EN 

 
and Innovation Programme (CIP). This concern is reflected also by the structure of Horizon 
2020,  as  the  emphasis  given  to  key  societal  challenges  is  expected  to  bring  together 
resources and knowledge across different sectors, technologies and scientific disciplines, in 
order  to  develop  breakthrough  solutions.    Moreover,  the  programme  will  cover  activities 
from  research  to  market  with  a  stronger  focus  on  innovation  related  activities,  such  as 
piloting,  demonstration,  test-beds  and  market  uptake  of  innovation.  Finally,  the  proposed 
financial  instruments  are  in  general  less  risk-adverse  and  more  open.  One  of  the  most 
prominent  examples  of  influence  on  policy  making  is  represented  by  the  recognition  of 
SMEs  as  innovation  drivers  in  Horizon  2020.  Concretely,  this  means  activities  more 
focused  towards  SMEs  with  reduced  red  tape  and  a  sizeable  dedicated  budget,  a  novel 
scheme  targeting  highly  innovative  SMEs  with  the  ambition  to  grow  and  internationalise, 
and  support  to  SME  clustering  to  access  the  technology  that  they  can't  afford  alone.  
Another example is the follow-up of the study's recommendation for a more entrepreneurial 
approach, which has been translated in a stronger focus on innovation (see above).  Finally, 
IPR  concerns  are  addressed  by  Horizon  2020  by  proposing,  among  others,  a  single  set  of 
participation rules, also on IPR. 

6) Availability of the report on Europa 

http://emplaps1.empl.cec:7779/scc-upl-infso/63031.pdf 

Assessment of the economic and social benefits of digitisation of cultural heritage - 
SMART 2010/0048 

1) ABB activities: 09 03 

2) Timing: January 2012 (Period covered by the report: 10 months) 

3) Budget (annual): not applicable. 

4) Background, scope and focus: 

The  study  was  commissioned  in  the  context  of  the  i2010  strategy  to  provide  information 
about the direct and indirect economic and social benefits of digitising cultural works and 
making  them  available  online,  focussing  on  benefits  such  as  new  markets  for  such 
materials,  and  potential  new  revenue  streams  for  the  right  holders.  It  aimed  at  paying 
particular  attention  to  the  digitisation  of  books  (especially  out-of-print  books),  and  the 
benefits  of  streamlined  rights  clearing  mechanisms  to  digitalise  books  and  making  them 
available online.   

5) Summary of findings and conclusions of the evaluation  

The contractor was asked to assess the direct and indirect, impact of digitisation of cultural 
heritage  on  economic,  social  and  environmental  variables.  The  report  proposes  a  basic 
logical  framework  (map  of  casual  relationships  and  relevant  indicators)  on  how  the 
digitisation of cultural heritage impacts on economy, society and the environment. 

The empirical evidence collected by the contractor is mainly based on a set of case studies 

EN 110 

Error! Unknown document property name.  EN 

 
 
 
 
of digitisation strategies and practices in eight different cultural institutions. 

These case studies were selected to cover different types of cultural organisations (libraries, 
archives,  museums  and  audiovisual  archives)  and  different  countries  within  the  European 
Union. One case concerns the United States (US Library of Congress). 

The case studies considered are:  

• 

• 

• 

• 

• 

• 

• 

• 

The British Library; 

The Hispanic Digital Library; 

The Dutch National Archive; 

The French National Audiovisual Institute; 

The Ghent University Library; 

The Parthenon Frieze; 

The Polish National Library; 

The US Library of Congress. 

For  each  of  these  case  studies,  the  contractor  used  interviews,  as  well  as  other  data 
gathering  techniques,  including  desk  research  and  available  scientific  literature.  Based  on 
these sources, the study includes a rather diverse set of quantitative and qualitative evidence 
concerning the impact of digitisation on the 3 domains concerned. It includes also statistics 
concerning the usage of digitised resources at level of cultural institutions. 

In several sections of the study, the contractor stresses the unavailability of solid key data 
and methodological limitations in the modelling as factors that make it particularly difficult 
to reach general quantitative conclusions about the impact of digitisation, in particular for 
the long term macro-economic impact. 

As for the financial return at micro level (cultural institution), the study makes wide use of 
assumptions. In particular, the contractor stresses that the information on costs and benefits 
is  not  currently  available  in  a  form  which  allows  for  the  return  on  investment  to  be 
calculated  in  an  accurate  manner.  A  Net  Present  Value  (NPV)  methodology  has  been 
applied  to  the  cases  studies  considered.  The  conclusions  indicate  that  the  NPV  of 
digitisation  is  almost  always  negative,  and  that  at  present  only  audiovisual  works  could 
offer opportunities for leveraging direct revenue for cultural institutions. 

As for the return on investment at macroeconomic level, the study estimates that each Euro 
invested  in  digitisation  can  produce  benefits  in  the  amount  of  EUR  1.20-3.00,  including 
social  and  education  benefits.  The  value  range  proposed  is  quite  wide,  which  limits  its 
relevance.  More  than  for  the  NPV  methodology,  the  contractor  points  to  the  severe 
limitations on the availability of data and the rather arbitrary assumptions on which such an 
estimate is based. 

Concerning  the  scenario  involving  a  significant  upgrade  of  the  financial  intervention  in 
support  to  the  digitisation  of  cultural  heritage,  the  study  suggests  a  linear  growth  of  the 
benefits associated to digitization. 

Results  fed  into  the  review  of  the  PSI  Directive  and  also  into  the  Commission’s 
Recommendation of 27 October 2011 on the digitisation and online accessibility of cultural 

EN 111 

Error! Unknown document property name.  EN 

 
material and digital preservation (OJ L 283, 29.10.2011, p. 39). 

6) Availability of the report on Europa 

Not yet available 

Report on Communication and Recommendation on mobilising Information and 
Communication Technologies to facilitate the transition to an energy-efficient, 
low-carbon economy COM (2009) 111; C(2009) 7604 

1) ABB activities: 09 04 

2) Timing: published in February 2013, (duration of the evaluation: December 2011-

September 2012) 

3) Budget (annual): Not applicable 

4) Background, scope and focus:  

According  to  Key  Action  12  of  the  DAE  the  Commission  was  to  assess  whether  the  ICT 
sector has complied with the timeline to adopt common measurement methodologies for the 
sector's own  energy performance and greenhouse gas (GHG) emissions and propose legal 
measures if appropriate. There are a myriad of standardisation initiatives worldwide on the 
energy/carbon footprint of the ICT sector. Those efforts are usually focused on subsectors 
of  the  industry  (e.g.  telecom)  and  it  is  not  possible  to  add  them  up  and  have  a  complete 
overview  of  the  industry's  total  footprint.  Therefore  there  was  a  need  for  a  consistent 
measurement framework. 

The evaluation of methodologies to assess the environmental footprint of ICT was twofold:  

1) the practical point of view of methodology usage was conducted through pilots run with 
the industry (http://www.ict-footprint.com/);  

2)  the  analysis  of  methodology  to  check  their  quality  and  completeness  was  conducted 
through  the  study  SMART  2011/0073  "Towards  an  overall  measurement  methodology  of 
the carbon and energy foot prints of the ICT sector".   

5) Summary of findings and conclusions of the evaluation:  

The overall conclusion regarding compatibility of footprint methodologies and standards for 
ICT  organisations,  products  and  services  is  that  on  the  one  hand  they  are  in  principle 
compatible and can deliver the same results, but on the other hand that the methodologies 
leave considerable freedom to make different methodological choices, potentially leading to 
different outcomes. Thus two footprint practitioners using different methodologies or even 
the  same  methodology  may  arrive  at  different  results  as  a  result  of  the  methodological 
changes  they  made.  It  was  also  noticed  that  the  emission  factors  used,  the  databases,  the 
calculation  tool,  the  quality  of  the  activity  data  and  the  person  who  is  conducting  the 
footprint  have  a  larger  impact  on  the  outcome  of  the  calculations  than  the  choice  of  the 
methodology.  

The  overall  conclusion  regarding  workability  is  that  in  principle  all  methodologies  are 

EN 112 

Error! Unknown document property name.  EN 

 
 
 
 
workable,  but  it  needs  to  be  kept  in  mind  that  foot-printing  in  general  requires  a 
considerable  amount  of  resources  in  terms  of  (out  of  pocket)  costs  and/or  staff.  The  most 
resource  intensive  part  in  any  foot-printing  study  is  the  collection  of  data.  The  needed 
resources for foot printing analyses depend on: 

• 

The complexity of the ICT organisation, product or service that is foot-printed; 

The  availability  of  existing  tools  (e.g.  semi-automatic  treatment  of  LCA  data)  and 

• 
outcomes of previous footprint studies that were developed and used in the past; 

• 

The level of knowledge and experience from previous work available internally.  

6) Availability of the report on Europa: 

http://ec.europa.eu/information_society/newsroom/cf//itemdetail.cfm?item_id=9689 

EN 113 

Error! Unknown document property name.  EN 

 
 
ABB 11 – Maritime Affairs and Fisheries  

Ex-post evaluation on the implementation of the compensation regime the 
additional costs incurred in the marketing of certain fishery products from the 
outermost regions the Azores, Madeira, the Canary Islands, French Guiana and 
Réunion 
(Evaluation of the measures in the EU’s outermost regions 
under Council Regulation (EC) N° 791/2007) 

1) ABB activities concerned: 11 02  

2) Timing: 1 August 2012 (Period covered by the report: 2007-2010) 

3) Budget: Maximum amount: EUR 14.996 million per year, EUR 105 million for 7-year period 

(Art. 5.4. of Council Regulation (EC) 791/2007) 

.4) Background, scope and focus: 

 Council Regulation (EC) 791/2007 introducing a scheme to compensate for additional costs 
incurred  in  the  marketing  of  certain  fishery  products  from  certain  outermost  regions  (the 
Azores, Madeira, the Canary Islands, French Guiana, and Réunion) stipulates in its Art. 8 that 
the  Commission  shall,  on  the  basis  of  an  independent  evaluation,  report  to  the  EP,  the 
Council  and  the  Economic  and  Social  Committee  on  the  implementation  of  the 
compensation, accompanied where necessary, by legislative proposals. The evaluation aimed 
at  measuring  the  performance  of  the  scheme  compared  to  its  objectives,  and  to  propose 
possible  ways  of  improvement.  This  evaluation  was  achieved  by  external  consultancy 
(Oceanic  Développement  (F)  in  association  with  MegaPesca  (P)  and  the  Fundación 
Universitaria  de  Las  Palmas  (E)).  The  work  included  an  analysis  of  the  documentation 
available, comprehensive  visits of the regions concerned (national and regional institutions, 
private sector of the industry and ancillary sectors), and review with the Commission services 
(steering  group).  The  approach  complied  with  Commission's  evaluation  guidelines  with 
regard to the evaluation criteria of effectiveness, efficiency, relevance, coherence and utility, 
with due consideration of the administrative burden. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study: 

The  evaluation  of  the  compensation  scheme  indicates  that  it  is  effective  toward  its  specific 
objective  of  contributing  to  maintain  the  competitiveness  of  the  fishing  industry  in  the 
outermost regions and toward its general objective of contributing to the development of the 
fishing industry in the outermost regions.  

•  The compensations scheme is also effective to contribute to the EU policy for consumers but 

with a limited impact.  

•  Overall,  the  implementation  of  the  scheme  is  evaluated  as  efficient.  Budget  envelopes 
granted  to  support  the  compensation  scheme  (ca.  EUR  15  million  annually)  have  been 

EN 114 

Error! Unknown document property name.  EN 

 
 
implemented at a level close to 100%. 

•  The  administrative  burden  for  its  management  is  considered  reasonable  at  national  and 

Commission levels. 

Recommendations:  

•  Define  and  adjust  the  notion  of  additional  costs  to  take  account  of  the  complexity  of  the 

calculation and to avoid potential over/under-compensations. 

•  Increase  the  financial  envelopes  granted  to  MS  to  fully  meet  the  objective  of  the 
compensation  system  and  compensate  the  totality  of  claims.    Indeed,  the  evaluation  shows 
that  the  quantities  of  products  potentially  eligible  to  the  compensation  mechanism,  and 
consequently  the  total  level  of  additional  costs,  are  higher  than  the  financial  envelopes 
provided for in the regulation;  

•  Specify implementation modalities of the scheme through an implementing act to avoid legal 

uncertainty.  

•  Improve the possibility of monitoring of the scheme by the Commission to better evaluate the 
contribution  to  the  objective  of  the  scheme  and  more  globally  to the  objectives  of  the  CFP 
and to those of the EU strategy for the outermost regions. 

It  should  be  noted  that  the  Regulation  stipulated  that  this  evaluation  and  the  relevant 
recommendations  could  lead  to  a  legislative  proposal.  This  was  the  legislator's  intention 
considering  that  the  compensation  regime  could  be  prolonged  under  the  form  of  an 
autonomous  legal  instrument.  However,  in  the  course  of  implementation,  the  Commission 
took the view that the compensation mechanism for outermost regions would be integrated in 
the future unique financial instrument: the European Maritime and Fisheries Fund (EMFF)1. 
Accordingly,  the  Commission  tabled  the  EMFF  proposal,  together  with  the  corresponding 
impact  assessment2  prior  to  the  finalisation  of  the  evaluation,  and  therefore  had  already 
fulfilled  this  possibility  to  submit  a  legislative  proposal.  Still,  the  evaluation  results  were 
known  soon  enough  to  be  impacted  in  the  EMFF  proposal  if  needed.    The  main 
recommendations  by  the  external  consultant  were  nevertheless  largely  consistent  with  the 
Commission's  proposal  made  in  the  context  of  the  EMFF.  The  only  difference  is  that 
although  the  consultant  recommends  an  increase  in  the  envelopes  granted  to  MS,  the 
Commission proposed to maintain the amount provided in the current regulation in the EMFF 
on the ground that they are sufficient to ensure that the mechanism functions and at the same 
time respond to the general need to  

1 COM(2011) 804 final Proposal for a Regulation of the European Parliament and of the Council on 
the European Maritime and Fisheries Fund [repealing Council Regulation (EC) No 1198/2006 and 
Council Regulation(EC) No 861/2006 and Council Regulation No XXX/2011 on integrated maritime 
policy 

2  SEC(2011)  1416  final  Commission  Staff  Working  Paper  Impact  Assessment  Accompanying  the 
document Proposal for a Regulation of the European Parliament and of the Council on the European 
Maritime  and  Fisheries  Fund  [repealing  Council  Regulation  (EC)  No  1198/2006  and  Council 
Regulation(EC) No 861/2006 and Council Regulation No XXX/2011 on integrated maritime policy 

EN 115 

Error! Unknown document property name.  EN 

 
                                                            
6) Availability of the report on Europa: 

http://ec.europa.eu/fisheries/documentation/studies/outermost-regions/index_en.htm 

Interim evaluation of the European Fisheries Fund (2007-2013) 

Synthesis of the 26 national evaluation reports 

1) ABB activities concerned: 11 06  

2) Timing: January 2012 (Period covered by the report: 2007-2010) 

3) Budget: The European Fisheries Fund (EFF) has a budget of EUR 4.3 billion for the period 

2007 to 2013. 

4) Background, scope and focus: 

 The purpose of the synthesis is to summarise the contents of the national interim evaluation 
reports on the EFF (2007-2013) and to highlight the main points and recommendations that 
emerge.  It  also  aims  to  provide  a  basis  for  the  organisation  of  a  "strategic  debate"  with 
member  states  to  be  undertaken  by  the  end  of  2011.  26  interim  evaluation  reports  were 
prepared in each member state based on a common methodological framework developed by 
the  European  Commission  prior  to  the  launch  of  the  evaluation,  which  addressed  minimal 
standards  and  provided  a  common  set  of  evaluation  questions,  criteria  and  indicators  to  be 
applied  for  the  assessment  of  all participating  Member  States.  The  interim  evaluations  thus 
focus on three main objectives: 

•  Determining if the programmes’ strategies are still relevant, taking into account changes 
in the common fisheries policy and socio-economic context. 

•  Assessing the quality and efficiency of the implementation and management in place. 

Assessing  effectiveness  and  appraising  progress  made  by  the  implemented  projects 

• 
towards the achievement of the Operational Programme (OP)'s set objectives. 

5) Summary of findings and conclusions of the evaluation or evaluation related 
    study: 

Synthesis of answers to the evaluation questions on programme management 

•  The partnership principle within the monitoring committees is respected to a varying degree 

across the different member states 

•  Most  evaluation  reports  reach  the  conclusion  that  the  management  and  control  systems  in 
place  are  rather  satisfactory,  with  clear  and  efficient  coordination  between  the  different 
structures  involved  in  programme  management,  despite  some  different  evolutions  and 
characteristics.  

•  Despite  a  general  improvement  compared  to  Financial  Instrument  for  Fisheries  Guidance 

EN 116 

Error! Unknown document property name.  EN 

 
 
(FIFG)  (1999-2006)  emphasised  in  several  reports,  the  administrative  burden  remains  a 
problem in achieving an effective implementation of the programme.  

•  Monitoring systems as a whole work well, but the quality and definitions of indicators should 
be improved. The usefulness of the indicators system in terms of evaluating the programmes’ 
results is not good. 

Synthesis of answers to the evaluation questions on programme effectiveness 

•  From a priority axis perspective, at 31 December 2010, the commitments by axes amounted 
to 47% of total EFF budget (EUR 575 million) for Axis 1, 43% (EUR 518 million) for Axis 
2, 40% (EUR 451 million) for Axis 3, 6% (EUR 34 million) for Axis 4 and 21% (EUR 35 
million) for Axis 5. Axis 1 projects thus account for the greatest proportion of commitments, 
due to their shorter term nature. Axis 4 achievement rate is the lowest, indicating that whilst 
groups have been formed, Fisheries Local Action Group (FLAG) projects are generally still 
in  the  very  early  stages  of  implementation.  The  main  reasons  explaining  delays  in 
implementation are twofold: 

−  Limited  co-financing:  the  unfavourable  economic  environment  and  uncertainties  regarding 

future developments have limited access to debt financing. 

−  Late launch  of programmes,  mostly due to delays in the validation of OP and management 
and  control  system,  which  have  adversely  affected  the  implementation  process  of  all 
measures under the OP. 

Conversely, some success factors are mentioned, and include efficient administration and greater 

interest from the sector. 

Priority axis 1: Measures for the adaptation of the Community fishing fleet 

•  There have not been any major implementation issues for measures 1.1 (permanent cessation 
of fishing activities) and 1.2 (temporary cessation of fishing activities) that were well known 
by  both  the  administrations  and  the  beneficiaries.  Except  in  Germany,  the  obligation  of 
implementing  a  Fishing  Effort  Adjustment  Plan  (FEAP)  has  not  prevented  or  delayed  the 
implementation  of  measures  1.1  and  1.2,  but  there  is  no  evidence  that  it  has  encouraged  a 
more strategic implementation either.  

•  The  implementation  of  measures  1.3  (investments  on  board  fishing  vessels  and  selectivity) 
and  1.4  (small-scale  coastal  fishing)  have  been  more  mitigated  for  the  former  and  rather 
difficult  for  the  latter,  in  part  because  of  the  economic  crisis  and  the  resulting  difficulty  to 
find co financing. 

Priority axis 2: Aquaculture, inland fishing, processing and marketing of fishery and aquaculture 

products 

•  The projects implemented in aquaculture under measure 2.1 have been dedicated primarily to 
productive  investments.  The  requirement  for  an  environmental  impact  assessment  has 
hampered the implementation of aquaculture projects in a limited number of member states. 
The implementation of aqua environmental measures did not experience major problems. On 
the  other  hand  the  implementation  of  animal  health  and  public  health  measures  has  been 

EN 117 

Error! Unknown document property name.  EN 

 
hampered by their unsuitability.  

•  Inland fishing measure 2.2 has been implemented only in a few  member states and did not 
have any noticeable impacts except in Finland (which is by far the main user of the measure). 

•  Measure  2.3  (fish  processing  and  marketing)  has  drawn  the  biggest  portion  of  the  funds 
programmed  for  Axis  2.  Projects  implemented  mostly  concern  increasing  production 
capacity,  improvement  of  production  systems  and  improvement  of  hygiene  and  working 
conditions in processing facilities. 

Priority axis 3: Measures of common interest 

•  Axis 3 is very diverse in nature, and is dedicated to supporting measures of common interest 
with a broader scope than measures normally undertaken by private enterprises. The spirit of 
this axis is to foster cooperation and organisation amongst stakeholders in order to contribute 
to  an  improved  competitiveness  and  productivity  of  organisations  as  well  as  to  the 
sustainability of the fisheries sector with projects that complement fisheries sector operations. 
Axis  3  projects  to  date  have  focused  predominantly  on  ports  infrastructure  (measure  3.3), 
collective actions (measure 3.1) and pilot operations (measure 3.5). 

Priority axis 4: Sustainable development of fisheries areas 

•  21  member  states  have  implemented  axis  4.  At  the  end  of  2010,  10  member  states  have 
approved FLAGs and projects, in 6 member states the selection process is still on-going for 
the  strategies  and  in  5  member  states  the  selection  process  of  the  FLAGs  themselves  is  in 
progress. 211 FLAGs and more than 1000 projects have been approved in 16 member states. 
The  main  positive  factors  in  the  implementation  process  are  the  strong  involvement  from 
local authorities and the LEADER (Community initiative for rural development) experience. 
The European Fisheries Areas Network (FARNET) is also considered as useful, nevertheless 
some  criticism  has  been  expressed  regarding  the  delays  in  the  publication  of  informative 
materials.  The  main  challenges  have  been  the  delays  in  setting  up  procedures,  access  to 
private financing and the lack of interest from the sector. The financial crisis has hampered 
the  good  development  of  axis  4  implementation  in  Ireland,  Greece  and  Denmark.  The 
impacts cannot be assessed sufficiently at this stage. 

Priority axis 5: Technical assistance 

Globally, the effectiveness of priority axis 5 implementation is moderate. The achievement of 
member states is very heterogeneous even if it is often quite high. Nonetheless, most of the 
evaluation reports highlight the usefulness of the technical assistance for OP implementation 
and information of the beneficiaries and potential beneficiaries. 

6) Availability of the report on Europa: 

 http://ec.europa.eu/fisheries/documentation/studies/eff_evaluation/index_en.htm 

EN 118 

Error! Unknown document property name.  EN 

 
 
 Ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement 
between the European Union and Republic of Kiribati, and ex-ante evaluation 
including an analysis of the impact of the future Protocol on sustainability 

1) ABB activities concerned: 11 03  

2)Timing: 12 June 2012 (Period covered by the report: 2006 – 2012) 

3) Budget:  

The  annual  financial  contribution  under  the  Protocol  evaluated  was  EUR  478.400  per  year 
from  which  30  %  was  dedicated  to  the  support  of  the  sectoral  fisheries  policy  in  Kiribati 
during  first  year  of  application.  The  second  year,  this  percentage  increased  to  40%,  and  to 
60% the following years. 

4) Background, scope and focus: 

This  evaluation  provides  an  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership Agreement (FPA) between the European Union and the Republic of Kiribati. It 
also provides an ex-ante evaluation and analysis of impacts of a potential future Protocol on 
sustainability.  The  evaluation  considers  the  Protocol  in  terms  of  its  relevance,  coherence, 
effectiveness, efficiency and sustainability. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The evaluation indicates that the Protocol is of a critical importance for the EU purse seine 
fleet operating in Kiribati. Catches in Kiribati represent around 50 % of vessels catches in the 
Western  and  Central  Pacific  Ocean  (WCPO).  The  utilisation  of  the  purse  seine  fishing 
authorisations  has  been  100  %  for  each  of  year  of  the  FPA,  and  utilisation  of  catch 
possibilities in excess of 100 % each year of the FPA. Concerning long-liners, except for two 
long-line fishing authorisations which were taken in 2007, there has been no interest by the 
EU  long-line  fleet  in  fishing  in  Kiribati.  The  Protocol  is  of  less  importance  for  the  EU 
upstream  sector,  and  for  the  EU-downstream  processing  sub-sector.  However,  the  Protocol 
generates some catches which are imported in loined form for canning in the EU. A total of 
EUR 6.4 million of value-added is accrued each year when considering the benefits to both 
the European Union and Kiribati. 75 % of this accrues to the EU and 25 % to Kiribati. EU 
value-added  is  most  prominent  in  the  catching  and  upstream  sectors,  and  less  marked  in 
downstream  processing.  In  Kiribati  value-added  is  almost  entirely  focused  in  the 
upstream/input sub-sector, principally in the form of payments made to the Government for 
access  and  sectoral  support,  but  also  from  vessel  support  services  i.e.  transhipment.  Some 
small value-added is made through the employment of Kiribati crew and observers. 

With  respect  to  effectiveness,  the  EU  fleet  plays  only  a  small  role  in  terms  of  overall  fleet 
activity  in  both  Kiribati  and  Western  and  Central  Pacific  area.  Its  ability  to  impact  on 
responsible fishing is therefore limited. The Protocol has been effective in supporting the EU 
catching  sub-sector  in  particular.  The  Protocol  has  also  been  effective  in  creating  some 
employment  in  both  the  EU  and  Kiribati.  The  employment  generated  by  the  Protocol  is 
estimated at 98 full-time equivalent (FTE) jobs. 

EN 119 

Error! Unknown document property name.  EN 

 
 
 
 
With respect to efficiency, the Protocol has been efficient for the EU in providing access at 
an affordable rate for EU vessels owners, and in generating a good benefit/cost ratio of 4.0, 
demonstrating that the Protocol provides good value for money with each every euro invested 
by  the  EU  and  ship-owners  generating  four  euros  of  benefit  in  terms  of  value-added.  The 
benefits  to  Kiribati  were  EUR  1.2  million  per  year  on  average  from  the  EU,  plus  smaller 
benefits in the form of value-added made from port calls by EU vessels and from local crew 
and observers on-board EU vessels. 

The Protocol has significantly contributed to the viability and sustainability of the EU purse 
seine catching sub-sector operating in the Pacific and its related employment, but only small 
contributions were made to the sustainability of the EU upstream and processing sub-sectors. 
The Protocol does not appear to threaten the sustainability of the Kiribati fishing sector.  

It is likely that a future Protocol would generate considerable levels of value-added to both 
the EU and to Kiribati. It would be in the interest of both the EU and Kiribati to have a new 
Protocol. For the EU fleet a ‘no Protocol’ would cause fishing rights as a proportion of sales 
revenues  to  rise  considerably,  while  for  Kiribati  a  failure  to  sign  a  new  Protocol  would 
eliminate  the  ear-marked  funds  for  special  sectorial  support  provided  by  the  EU  under  the 
Protocol. 

6) Availability of the report on Europa:  

http://ec.europa.eu/fisheries/documentation/studies/kiribati_2012/index_en.htm 

Ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement 
between the European Union and Solomon Islands, and ex-ante evaluation 
including an analysis of the impact of the future Protocol on sustainability 

1) ABB activities concerned: 11 03  

2) Timing: 19 July 2012 (Period covered by the report: 2009 -2012) 

3)  Budget:  The  annual  financial  contribution  under  the  Protocol  evaluated  was  EUR  400.000 
per  year  from  which  50  %  was  dedicated  to  the  support  of  the  sectoral  fisheries  policy  in 
Solomon Islands. 

4) Background, scope and focus: 

This  evaluation  provides  an  ex-post  evaluation  of  the  current  Protocol  to  the  Fisheries 
Partnership  Agreement  (FPA)  between  the  European  Union  and  Solomon  Islands.  It  also 
provides  an  ex-ante  evaluation  and  analysis  of  impacts  of  a  potential  future  Protocol  on 
sustainability.  The  evaluation  considers  the  Protocol  in  terms  of  its  relevance,  coherence, 
effectiveness, efficiency and sustainability 

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related
     study: 

The evaluations indicates that within the EU, the Protocol most strongly meets the interests 
and needs of the EU purse seine fleet although this is more in terms of catch possibilities. The 
Protocol could be described as relevant to the needs of EU consumers in that it provides some 

EN 120 

Error! Unknown document property name.  EN 

 
 
 
 
tuna  supplies  to  the  EU  market.  However,  the  quantities  are  very  small  compared  to  the 
overall EU market, so the relevance is also small. The Protocol is also relevant to the needs of 
EU  processors,  and  stakeholder  consultations  confirmed  that  some  catch  made  under  the 
Protocol is destined for processing in the EU, although not in significant amounts.  

The Protocol has been relatively efficient for the fishing industry given that access payments 
per  tonne  of  fish  caught  have  remained  constant  over  the  period  of  the  Protocol,  while  ex-
vessel fish prices rose by 49% and 32% for skipjack and yellow-fin tuna respectively in the 
period  2009/10  to  2011/12.  The  EU  fleet  has  spent  little  time  fishing  within  the  Solomon 
Islands' exclusive economic zone (EEZ). The Protocol has not been very significant in terms 
of the operations of the purse seine vessels authorized to fish in Solomon Islands waters. This 
coupled with provisional data for the 2011/12 season indicates vessels do not currently show 
a high dependency on catches made within the Solomon Islands EEZ.  

In terms of support for responsible fisheries the Protocol helps ensure that there is active EU 
involvement in regional fisheries issues and provides for direct sectoral support for Solomon 
Islands  that  potentially  helps  ensure  a  positive  contribution  of  the  sector  to  the  national 
economy. 

The financial and economic sustainability of the catching sub-sector has contributed directly 
only  a  small  amount  to  the  employment  of  15  EU  crew  working  (per  year)  in  the  catching 
sub-sector. It is estimated that around 404 EU jobs are dependent on the operations of the EU 
purse seine fleet and that the Protocol makes a contribution to securing these jobs in its role 
as part of a network of FPAs in the region. 

The  EU  purse  seine  fleet  is  only  a  minor  operator  in  the  Western  and  Central  Pacific  area, 
representing only 2% of the total purse seine fishery. The limited catches by EU vessels as a 
percentage  of  total  Western  and  Central  Pacific  Ocean  (WCPO)  catches  and  the  current 
under-utilisation  of  Solomon  Islands  vessel  days  mean  that  the  fishing  under  the  Protocol 
does not appear to threaten the sustainability of the Solomon Islands fishing sector.  

The  activities  and  visits  of  foreign  vessels,  including  those  from  the  EU,  contribute  to  the 
sustainability  of  the  domestic  upstream  sector  by  providing  revenues  and  employment 
opportunities, i.e. for transhipment services, agents, etc. 

Given  the  current  level  of  benefits  from  the  FPA,  a  new  protocol  should  seek  to  obtain  a 
better  balance  between  the  value  of  fishing  rights  negotiated  and  the  likely  utilisation  of 
fishing  opportunity  by  the  purse  seine  fleet.  At  the  same  time,  in  doing  so,  consideration 
should  also  be  given  to  how  the  Solomon  Islands  FPA  fits  within  a  wider  network  of 
agreements and the how the costs balance against the overall economic and social benefits as 
well as the strategic. 

Finally, the evaluation recommends the continued active participation by the EU within the 
Western  and  Central  Pacific  Fisheries  Commission  (WCPFC),  so  as  to  ensure  responsible 
fisheries. 

6) Availability of the report on Europa: 
http://ec.europa.eu/fisheries/documentation/studies/solomon_islands_2012/index_en.htm 

EN 121 

Error! Unknown document property name.  EN 

 
 
 
Evaluation ex-post du protocole de l'accord de partenariat dans le domaine de la 
pêche entre l'Union européenne et la Côte-d'Ivoire 
(Ex-post evaluation of the current Protocol to the Fisheries Partnership Agreement 
between the European Union and Ivory Coast) 

1) ABB activities concerned: 11 03  

2) Timing: 28 June 2012 (Period covered by the report: 2007 – 2012) 

3) Budget: EUR 595,000 per year for the support to the Ivorian fisheries policy 

4) Background, scope and focus: This evaluation provides a retrospective ex-post   evaluation 
for  the  Protocol  to  the  Fisheries  Partnership  Agreement  (FPA)  between  the  EU  and  Ivory 
Coast (2007-2013) and a prospective ex-ante evaluation for the negotiation of a new protocol. 

The  evaluation  considers  the  Protocol  in  terms  of  its  relevance,  coherence,  effectiveness, 
efficiency and sustainability.  

5)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related

     study: 

 In terms of overall effectiveness, the FPA with Ivory Coast meets the EU principles of good 
governance. Relevance: The FPA contributes, with all FPAs in West Africa, to the supply of 
canneries and fish market in Ivory Coast. In other terms, the FPA is relevant for both parties. 
For EU ship-owners, the FPA is not very significant in terms of catches, compared to catches 
in  high  seas  and  other  exclusive  economic  zones  (EEZs).  It  however  allows  continuity 
between  neighbouring  EEZs  and  an  easy  access  to  the  harbour  of  Abidjan,  which  remains 
central in the West Africa area, despite its deterioration.  

Effectiveness:  Even  though  it  was  implemented  in  a  delicate  political  context,  the 
implementation was satisfactory. Some provisions of the protocol could not be fulfilled, such 
as control and the boarding of observers; on the whole there is a good level of effectiveness. 
What  needed  to  be  done  was  well  executed.  However,  the  use  of  funds  of  the  financial 
contribution  for  sectoral  support  gives  uncertain  results  due  to  the  dispersal  of  actions  and 
therefore its limited impact.  

Efficiency:  The  implementation  of  the  FPA  contributes  to  the  creation  of  economic  wealth 
and employment, both for the EU and Ivory Coast, even if only 50% of the reference tonnage 
is  caught.  Hence,  each  euro  of  the  financial  compensation  generates  almost  three  euros  of 
gross  added  value,  of  which  around  80%  in  EU  and  20%  in  Ivory  Coast.  It  is  even  more 
important  for  Ivory  Coast  who  receives  215  EUR/tonnes,  i.e.  more  of  the  double  of  what 
constitutes  the  financial  basis  of  the  financial  compensation  and  of  ship-owners  fees  (100 
EUR/tonnes).  For  ship-owners,  real  administrative  costs  of  each  tonne  is  around  45 
EUR/tonnes, a number close to the profitability, estimated to be at 50 EUR/tonnes.  

Sustainability: By respecting good governance and responsible fisheries principles, the FPA 
participates to the effort of the European Commission and International Commission for the 
Conservation  of  Atlantic  Tunas  (ICCAT)  and  other  relevant  regional  fisheries  management 
organisations  (RFMOs)  to  act  for  a  sustainable  use  of  tuna  stocks  in  the  Atlantic  ocean. 
Indeed,  the  FPA  does  not  result  in  excess  fishing  capacities,  either  in  Ivorian  waters  or  in 

EN 122 

Error! Unknown document property name.  EN 

 
those  managed  by  ICCAT.  Fleets  should  however  continue  their  efforts  to  reduce  their 
catches of juveniles under fish aggregating devices (FADs). The contribution of purse seiners 
to the onshore processing industry and domestic market supply remains tangible.  

Nevertheless,  by  the  income  generated  and  the  employments  related  to  the  activities  in  the 
harbour  of  Abidjan,  and  in  canneries,  the  FPA,  and  more  generally,  the  presence  of  EU 
vessels during the period, contributes to the process of poverty reduction and globally, to the 
sustainable development by fitting engaged actions and effects within long term perspectives. 
Coherence:  Global  coherence  of  FPAs  is  satisfactory,  at  the  EU  level  (external  coherence, 
relations with ACP countries (African, Caribbean and Pacific Group of States), development 
policies,  external  policies,  etc.)  and  at  Ivory  Coast  level  (sectorial  policy  and  national 
development  policy).  Recommendations  made  are  both  general  and  technical.  On  general 
perspectives,  some  indications  related  to  the  FPA  itself  are  made,  related  to  fishing 
authorizations, the exclusivity clause and the use of funds of the financial compensation and 
others linked to it such as the refection of the harbour of Abidjan, the fisheries policy in Ivory 
Coast  and  the  monitoring  of  tuna  FPAs.  On  technical  issues,  recommendations  are  made 
regarding  the  delimitation  of  the  EEZ  of  Ivory  coast  and  on  all  the  aspects  regarding 
monitoring and control of vessels. 

6) Availability of the report on Europa:  

http://ec.europa.eu/fisheries/documentation/studies/cote_ivoire_2012/index_en.htm  

Interim evaluation of marine knowledge-related preparatory actions and pilot 
projects. (internal evaluation)  

1) ABB activities concerned: 11 09  

2) Timing: 29 August 2012 (Period covered by the report: 2008-2012) 

3) Budget: The budget was EUR 6,450,000 

4) Background, scope and focus: The aim of the European Marine Observation and Data 
Network (EMODnet) is to improve access to marine data and so reduce costs to users, 
stimulate innovation and reduce uncertainty in our understanding of the behaviour of the seas 
and oceans. To test how this could be done a number of preparatory actions were started in 
the period 2008-2010. Consortia of marine data organisations, selected through calls for 
tender and implemented through procurement contracts, set up portals that provide access to 
marine data, metadata and data products for six themes for whole sea-basins. 53 different 
organisations participated in the projects; largely public bodies responsible for managing 
marine data on a national scale but supported by some small private companies with expertise 
in managing distributed data. All the metadata and data products and most of the data are 
made available to users free of charge and free of restriction of use. On 29 August 2012, the 
commission adopted a Green Paper on marine knowledge1 

1 COM(2012) 473 final Marine Knowledge 2020: from seabed mapping to ocean forecasting, to which 

this evaluation was annexed. 

EN 123 

Error! Unknown document property name.  EN 

 
 
                                                            
5) Summary of findings and conclusions of the evaluation or evaluation related 
    study: 

Impact assessment suggests potentially over EUR500 million of benefits per year to EU 
maritime economy in ling run. Initiative would not have happened without EU.  

Independent assessment concluded that basic architecture chosen was appropriate and that 
information system was fit for purpose. Project costs were comparable to analogous EU 
initiatives and management overhead was low. As a result of the evaluation a secretariat has 
been set up to better monitor the final phase of the projects and the beginning of the next 
phase. Experience led to many modifications in terms of reference of follow-up projects 
under Financial Regulation of Integrated Maritime Policy. 

6) Availability of the report on Europa: SWD(2012) 250 final  
http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52012SC0250:EN:NOT 

EN 124 

Error! Unknown document property name.  EN 

 
 
ABB 13 – Regional Policy  

Ex post evaluation of projects co-financed by ERDF and Cohesion Fund in the 
period 1994-1999 

1) ABB activities concerned: 13 03 

2) Timing: November 2012 

3) Budget: EUR 4.8 billion. 

4) Background, scope and focus: 

The  objective  of  this  study  was  to  evaluate  the  type,  magnitude  and  timing  of  the 

long‐term effects of ten  major projects  implemented  during the 1994‐1999 programming 

period,  and  the  mechanisms  explaining  the  project  outcomes.  The  investment  projects 
analysed were in the transport (road, rail, seaports) and environment (water supply, waste 
water treatment, waste management) sectors, across five Member States: Greece, Ireland, 
Italy, Portugal and Spain, with a total value over EUR 50 million per project. By putting 
together  the  case  study  evidence,  the  evaluator  has  proposed  some  key  ideas  for  policy 
learning and recommendations for maximising long-term effects of investments. 

5) Summary of findings and conclusions of the evaluation:  

• 
The evaluation demonstrates that the EU financing produced positive overall effects 
along the different dimensions explored. Uncontroversial direct contributions are found to 
economic  growth  and  quality  of  life,  while  the  strength  of  each  type  of  effect  differs  by 
sector (transport or environment) or project. 

• 
The majority of effects detected have stabilised, either in the short-medium (i.e. from 
1 to 5 years after project completion) or in the long run (more than 6 years after project 
completion). However there are also significant factors which are not yet stabilised, mainly 
direct growth effects. 

• 
The infrastructural sector matters in particular as far as project design is concerned, 
with  positive  scores  recorded  on  average  for  transport  projects  and  negative  scores  for 
environmental  ones.  Project  design  is  a  weak  point  of  environment  projects,  which  was 
adversely affected by time constraints (deadlines imposed by EC Regulations), forecasting 

mistakes  as  well  as  social  pressure  (municipalities  of  the  Northern  Lisbon  sub‐region 

EN 125 

Error! Unknown document property name.  EN 

 
 
 
pushed  for  spreading  the  solid  waste  treatment  facilities  around  different  areas,  with 
negative  economic  and  environmental  consequences).  Transport  projects  are  generally 
characterised  by  more  efficient  and  flexible  designs,  reflecting  the  strategic  vision 
underpinning them. 

•  Generally, the evidence collected shows how the outcomes of development projects are the 

result  of  a  mixture  and  interaction  between  incidental  environmental,  socio‐economic, 

institutional  and  cultural  circumstances  and  the  managerial  responses  produced  by  the 
project. 

6) Availability of the report on Europa: 

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/9499_final_report_09111
2.pdf 

Ex post evaluation of the Cohesion Fund (Incl. former ISPA) 2000 - 2006 

1) ABB activities concerned: 13 04 

2) Timing: July 2012 

3) Budget: EUR 36.2 billion. 

4) Background, scope and focus: The main overall objective of the Cohesion Fund 

(including former ISP A) ex post evaluation was to assess the contribution of the Cohesion 
Fund and ISPA to the development of the EU transport system, to achieving the EU acquis 
in the field of environment and the effect of ISPA as a preparation for Structural Fund and 
Cohesion Fund programmes in 16 countries. A total of EUR 36.2 billion was made 
available from the Cohesion Fund and ISPA to the countries eligible for support. The 
evaluation was made up of five work packages: 

•  WPA – the contribution to EU transport and environmental policies 

•  WPB – cost-benefit analysis of selected transport projects 

•  WPC – cost-benefit analysis of selected environmental infrastructure projects 

•  WPD – the management and implementation of funding 

•  WPE - Synthesis report and expert review 

This fiche concerns WPE – the synthesis of all the previous work packages. 

5) Summary of findings and conclusions of the evaluation:  

•  The evaluation demonstrates that projects co-financed by the Cohesion Fund and ISPA for 
the  2000-2006  period  had  a  marked  effect  in  extending  and  improving  the  transport 

EN 126 

Error! Unknown document property name.  EN 

 
 
 
network  and  environmental  infrastructure  in  the  countries  receiving  support.  In  the 
transport sector, 1,286 km of new motorway were constructed in the 16 countries receiving 
support, 3,013 km of roads on the TENs were improved in the EU12 countries, 3,675 km 
of  railway  line  in  the  EU12  were  upgraded  etc.  In  the  environment  sector,  5.6  million 
people were connected to main water supply in Spain, 570,000 people were connected to 
main drainage in Portugal and around 1.1 million in Greece, 40% of non-compliant landfill 
sites was closed in Hungary and of around 60% in Lithuania, an additional 77,500 tonnes 
of waste a year were recovered in Slovenia etc. 

•  Transport  projects  generated  major  improvements  in  many  of  the  transit  routes  crossing 
the  countries  concerned  and,  accordingly,  in  the  transport  links  with  neighbouring 
countries. Equally, it simultaneously reduced journey times internally between centres of 
economic activity and population and, in many cases, took traffic away from city centres 
so relieving congestion and curbing pollution. 

• 
There is more uncertainty about the extent to which environmental projects generated 
benefits  which  exceeded  the  costs  of  undertaking  them,  but  there  is  no  question  that  the 
investment co-financed contributed significantly to the efforts of the countries concerned 
to  comply  with  EU  Directives  on  water  supply,  wastewater  treatment  and  solid  waste 
management.    They  had  an  equally  significant  effect  in  furthering  the  protection  of  the 
environment  and  reducing  pollution,  so  improving  both  the  sustainability  of  economic 
development and the quality of life across many parts of the countries receiving support.  

• 
It  is  difficult  to  assess  the  effects  on  ultimate  policy  objectives  –  on  economic 
development and cohesion – which the funding provided is intended to help achieve. This 
is partly because of the difficulties of disentangling the effects from other forces at work 
but  it  also  because  of  the  imprecise  nature  of  the  objectives  concerned,  especially  those 
relating to cohesion and with respect to environmental projects in particular. 

•  Given  the  long-term  nature  of  the  investment  carried  out,  for  both  transport  and 
environment, the actual effect is likely to take some time to become apparent, especially 
since it is likely to depend both on future investment in other parts of the network and on 
the policies implemented in other areas. 

• 
It  is  even  more  problematic  to  see  the  actual  effect  in  the  case  of  environmental 
projects  since  the  link  between  such  investment  and  economic  development  is  more 
tenuous. There is little evidence, that investing in environmental infrastructure will either 
stimulate  economic  growth  or  create  the  conditions  for  this  to  occur.  It  might  help  to 
strengthen  social  and  territorial  cohesion  by  improving  the  quality  of  life  in  the  areas 
where  the  investment  takes  place  and  by  reducing  disparities  in  living  conditions  across 
the EU, though this is extremely difficult to observe and measure.  

• 
The  evaluation  finds  significant  delays  in  implementing  projects,  especially 
environmental  projects.  The  delays  occurred  at  all  stages  of  the  project  implementation 
process:  strategic  planning  and  the  choice  of  projects  to  be  undertaken,  designing  and 
developing projects, consultation,  procurement and the work involved in the construction 
of the project itself. 

• 

 There  is  a  need  to  give  serious  consideration  to  ways  in  which  closure  procedures 

EN 127 

Error! Unknown document property name.  EN 

 
can  be  significantly  speeded  by  both  the  managing  authorities  and  the  European 
Commission.  

6) Availability of the report on Europa: 
http://ec.europa.eu/regional_policy/information/evaluations/archives_2000_2006_en.cfm#5 

EN 128 

Error! Unknown document property name.  EN 

 
 
Country reports on the performance of Cohesion Policy 2007 - 2013 

1) ABB activities concerned : 13 04 

2) Timing: January 2013 

3) Budget: EUR 270.1 billion  

4) Background, scope and focus: 

•  27  country  reports  and  a  synthesis  report  were  delivered  by  the  network  of  experts  on 
progress  in  implementing  the  programmes  co-financed  by  the  ERDF  and  the  Cohesion 
Fund over the 2007 – 2013 and their outcome up to the end of 2011. The reports look at 
the socio-economic context, changes in region disparities across the EU during the crisis, 
the  scale  of  support  from  the  ERDF  and  CF  for  the  present  period,  the  pace  of 
implementing  programmes,  the  achievements  and  analyses  evidence  coming  from 
evaluations undertaken in the Member States.5) Summary of findings and conclusions of 
the  evaluation  The  rate  of  implementation  of  programmes  in  many  countries  remains 
slow; this is especially the case in Romania, Bulgaria and the Czech Republic in the EU12 
and in Italy, France and Austria in the EU15. 

•  The amount of funding still to be absorbed over the remainder of the programming period 
is even larger than it appears since in some cases resources have been invested in financial 
engineering schemes and are yet to transfer to enterprises.  

•  Government  expenditure  on  capital  formation  has  been  hit  disproportionately  hard  by 
fiscal consolidation measures taken across the EU and has declined in most Member States 
over the past 2-3 years, in many cases substantially. 

•  Reductions in government spending on capital formation have been facilitated in a number 
of  cases  by  increases  in  EU  co-financing  rates  on  Cohesion  policy  programmes,  which 
have been reduced in size as a result. 

•  The  on-going  crisis  has  led  to  a  widespread  tendency  for  growth  and  jobs  to  become  a 
priority of national governments across the EU ahead of reducing regional disparities. 

•  The  focus  of  Cohesion  policy  appears  to  be  shifting  away  from  supporting  the 

development of weaker regions at a time when this seems to be more necessary.  

•  Despite the delays in the implementation of programmes, there is increasing evidence of 
the positive effects of the projects which have been carried out; this is particularly the case 
in respect of enterprise support, transport and environmental infrastructure. 

the  continuing  deficiency  of 

•  Assessing  the  outcome  of  programmes  and  the  achievements  from  EU  support  is,  made 
in  Annual 
difficult  by 
Implementation  Reports;  this  applies  to  the  quantitative  indicators,  which  too  often  bear 
little  relationship  to  the  aims  of  the  intervention  concerned  and  are  not  consistent  across 
programmes; it applies equally to the qualitative information, which in most cases does not 
enable  the  quantitative  outcome  to  be  meaningfully  interpreted  in  relation  to  policy 
objectives. 

information  published 

the 

EN 129 

Error! Unknown document property name.  EN 

 
 
 
There is a clear and urgent need for a major improvement in the indicators monitored 

• 
and their link to policy objectives. 

•  A  move  towards  a  results-oriented  policy  equally  requires  a  step  change  in  the 
number, nature and quality of evaluations carried out on EU co-financed programmes in 
order  better  to  be  able  to  assess  their  effects  on  the  pursuit  of  policy  goals  and, 
accordingly, to help select the measures likely to be most effective in this regard. 

• 
There is need for a common acceptance on the part of Member States that evaluations 
are an important part of formulating effective development policies and making the most 
productive  use  of  Cohesion  policy  funding  rather  than  simply  a  formal  obligation  of 
receiving the funding concerned. 

6) Availability of the report on Europa: 

http://ec.europa.eu/regional_policy/information/evaluations/index_en.cfm#1 

Jaspers Evaluation 

1) ABB activities concerned: 13 03 

2) Timing: December 2012 

3) Budget: EUR 167 million. 

 4) Background, scope and focus: 

The study evaluates the JASPERS initiative from its inception until the end of June 2011. 
Its main focus is to establish the impact of JASPERS on the quality and timeliness of the 
preparation, submission, approval and implementation of major projects, in the countries 
which  joined  the  European  Union  in  2004  and  2007.  These  include  Bulgaria,  the  Czech 
Republic, Cyprus, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Romania, Slovakia 
and Slovenia. 

5) Summary of findings and conclusions of the evaluation  

•  The  JASPERS  initiative  has  been  of  substantial  value  to  Member  States  in  the 
development  of  projects  for  funding  and  there  continues  to  be  strong  demand  for  its 
services.    There  is  sound  statistical  evidence  confirming  that  JASPERS  support 
significantly  reduces  the  duration  of  the  Commission's  approval  process,  as  well  as  the 
time taken to develop projects by the Member States. 

•  The  greatest  potential  to  improve  project  quality  is  when  JASPERS  is  involved  at  the 
strategic stage, and where JASPERS support helps Member States to develop their project 
planning capacity.   

•  JASPERS  should  have  an  explicit  objective  to  develop  the  project  planning  capacity  of 

EN 130 

Error! Unknown document property name.  EN 

 
 
 
 
Member  States  and  there  should  be  an  increased  focus  in  its  work  on  this  objective.    A 
three year work plan, agreed between Jaspers and each Member State, is proposed.  The 
work plans would be tailored to the needs and strengths of each Member State. 

•  JASPERS  intervention  should  come  as  early  as  possible  in  the  project  design  process, 

providing technical or specialist advice from the earliest stages of planning.    

•  Working arrangements between JASPERS, the Commission and the Member States should 
be  more  formalised.    This  should  clarify  roles  and  responsibilities  between  the 
Commission, Member States, and Jaspers. 

•  JASPERS advice should be routinely available to Member States for the development of 
sectoral strategies.  JASPERS involvement would be advisory in nature, and take place at 
the invitation of Member States. 

•  A  more  strategic  approach  is  required  to  improve  the  capacity  of  Member  States  to 
select  and  develop  high  quality  projects.    Based  on  an  analysis  of  individual  Member 
States’ project planning capacities, JASPERS could focus on specific activities that would 
have the greatest impact on project planning and project quality. 

• 
The  evidence  from  the  evaluation  confirms  the  need  for  and  the  importance  of 
improving  knowledge  transfer.    JASPERS  should  put  in  place  a  system  to  highlight 
technical  issues  that  have  been  addressed  and  resolved  in  individual  projects  where  they 
are considered to be of more general relevance.  Action by Member States is also required 
– for example by considering what structures are in place, and what actions are needed, to 
ensure that knowledge transfer happens effectively.  

6) Availability of the report on Europa: 

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/eval2007/jaspers_evaluat
ion/final_report_131212.pdf 

The use of the ERDF to support Financial engineering instruments (FEIs) 

1) ABB activities concerned: 13 03 

2) Timing: July 2012 

3) Budget: EUR 11.6 billion 

4) Background, scope and focus: 

27 policy papers delivered by the experts present an informed overview of the use of the 
ERDF  to  co-finance  FEIs,  the  rationale  for  Member  States  adopting  this  means  of 
investment  support  and  the  way  that  the  schemes  set  up  are  operating  in  practice.  The 
experts  were  asked  to  examine  and  report  on  these  issues  in  their  own  countries  on  the 
basis  of  published  information  and  interviews  with  Managing  Authorities  in  April-May 

EN 131 

Error! Unknown document property name.  EN 

 
 
 
 
2012.  

5) Summary of findings and conclusions of the evaluation  

•  In most countries, the use of the ERDF to finance FEIs is relatively recent and limited in 
size; the share going to FEIs is larger in the EU15 than EU12 and in Competitiveness than 
Convergence regions and takes the form more of support for loans and guarantees than VC 
funds. 

•  There was a very small overall increase across the EU in the share of the ERDF planned to 
go  to  FEIs  over  the  programming  period  combined  with  a  shift  from  VC  funds  to  loans 
and guarantees. 

•  The variation in use is broadly in line with expectations, given that FEIs require expertise 
to set up and operate and VC funds sufficient innovative high growth firms to be viable. 

•  There is a strong case for public support of loans and guarantees in many countries given 
the limited access of SMEs to finance and the large amount of collateral often demanded 
by  banks.  This  applies  particularly  to  countries  where  borrowing  from  banks  is  most 
problematic  and  where  the  support  to  businesses  they  provide  is  most  limited,  such  as 
Bulgaria, Hungary and Romania in the EU12 and Greece and Spain in the EU15. 

•  The case for support of VC funds is weaker and there are only a limited number of areas 
across  the  EU  which  has  a  sufficient  concentration of  small  high-growth  firms  to  justify 
public support of them. 

•  The main reasons for the use of FEIs according to Member States are closely in line with 
Commission statements, that they fill a gap in the financial market between the demand for 
funding from SMEs and the available supply in a ‘revolving’ way which means that more 
firms can be supported. 

•  While  the  use  of  the  ERDF  to  support  FEIs  may  be  justifiable,  there  is  not  enough 
evidence  to  determine  whether  the  scale  of  support  matches  the  size  of  the  gaps  in  the 
market  for  loans  and  equity  finance  and  how  far  the  sums  allocated  have  reduced  these 
gaps. 

• 
It is questionable whether the size of many of the VC funds set up with ERDF support 
is  large  enough  for  them  to  be  viable  given  the  high  fixed  costs  and  the  high  degree  of 
uncertainty attached to investments which makes it important to spread the risk. 

• 
The  complexity  of  FEIs  and  the  time  and  resources  needed  to  set  them  up  have 
reduced  their  use,  along  with  the  limited  extent  of  demand  for  them  perceived  by 
Managing Authorities and the preference for grants for many investments where the policy 
objectives extend beyond making a financial return. 

•  Very  little  data  exist  on  the  cost  of  setting  up  and  operating  FEIs  relative  to  non-
repayable grants but there is a widespread perception that they are higher and the period 
needed to set them up longer. 

•  Many  of  the  financial  engineering  schemes  set  up  with  ERDF  financing  to  support 
businesses operate in much the same way as privately-financed ones, except that they are 

EN 132 

Error! Unknown document property name.  EN 

 
limited  to  SMEs,  particular  sectors  and  firm  located  in  the  region;  many,  however,  also 
have  wider  objectives,  in  line  with  the  aims  of  Cohesion  policy,  and  impose  additional 
conditions on recipients of funding over and above purely financial ones. 

• 
The main problems of using the ERDF to co-finance FEIs stem from the complexity 
of the regulations and the uncertainty surrounding their interpretation because of their lack 
of clarity, which add to costs and time taken to set up schemes; this was especially so in 
the early part of the period but problems remain even after Commission attempts to clear 
up ambiguities, which deter the authorities from using FEIs. 

•  Very  few  evaluations  have  been  carried  out  on  ERDF-financed  FEIs  or  publically-
funded  schemes  generally;  those  that  have  been  undertaken  generally  indicate  positive 
effects  on  the  performance  of  the  firms  supported,  but  there  is  limited  evidence  on  the 
achievement  of  wider  objectives  –  on  the  competitiveness  of  the  business  sector  and 
regional development. 

6) Availability of the report on Europa: 

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/eval2007/expert_innovati
on/2012_evalnet_fei_synthesis_final.pdf 

EN 133 

Error! Unknown document property name.  EN 

 
 
ABB 15 – Education and Culture  

 Interim evaluation of Erasmus Mundus II (2009 – 2013) 

1) ABB activities: 15 02 

2) Timing: March 2012 (Period covered by the report: 2009 – 2011) 

3) Budget : Overall  budget of the evaluated Programme: EUR 963,690,000 

4) Background, scope and focus: 

This  interim  evaluation  was  launched  in  accordance  with  Article  13  of  the  Erasmus 
Mundus  Decision.  The  findings  should  provide  inputs  for  the  Commission’s  Interim 
Evaluation Report on the results achieved, and on the qualitative and quantitative aspects 
of the implementation of the programme. 

5) Summary of findings and conclusions of the evaluation:  

EU  Added  Value:  Generally,  international  academic  and  mobility  cooperation  entail 
transnational  aspects  which  by  reason  of  their  nature,  scale  or  effects  can  be  dealt  with 
better  at  EU  level.  The  implementation  of  European  Joint  Masters  and  Doctoral 
programmes  (with  a  common  Erasmus  Mundus  branding)  and  mobility  activities, 
involving  higher  education  institutions  from  all  over  the  world,  can  be  much  better 
managed  and  monitored  at  EU  level  by  the  Commission,  which  can  put  in  place  and 
effectively  run  the  necessary  management  structures  and  delivery  methods.  The  EU  can 
also  more  efficiently  address,  notably  through  dialogue  on  higher  education  policies, 
global  issues  such  as  obstacles  to  international  mobility,  recognition  of  degrees, 
attractiveness, brain drain or capacity building measures. 

The visibility of European higher education in the world can be better achieved through a 
coherent  EU  promotion  strategy,  involving  all  interested  Member  States.  EU  initiatives 
also act as a laboratory to test innovative ideas or set quality standards which then inspire 
the  national  level.  EU  achievements  in  the  field  (quality  assurance,  qualification 
frameworks,  European  Credit  Transfer  System,  key  competences,  tuning  of  higher-
education structures, etc.) also gain in being promoted in a visible and coherent manner at 
EU  level,  and  in  the  wider  world  as  collective  EU  achievements.  The  objectives  and 
actions  of  Erasmus  Mundus  II  were  defined  on  this  basis  and  the  current  evaluation 
provides evidence to support this logic and prove the programme's effectiveness. 

Specifically,  EM  II  provides  strong  support  and  makes  an  important  contribution  to  the 
internationalisation process of the European Higher Education Area. Furthermore, the joint 
masters  and  doctoral  programmes  funded  by  Action  1  had  considerable  added  value  by 
facilitating  the  success  of  graduates  when  looking  for  work  and/or  further  research 
positions. International experiences and intercultural competence could be regarded as the 
most important assets that distinguished Erasmus Mundus students from other graduates. 
Inclusion of doctoral and post-doctoral co-operation within Erasmus Mundus II has been 
one of the most successful innovations of phase II, attracting many very prestigious higher 

EN 134 

Error! Unknown document property name.  EN 

 
 
 
 
education institutions to participate. 

Effectiveness:  Action  1  joint  courses  produced  ambitious  graduates,  satisfied  with  their 
experience and strongly identifying with the Erasmus Mundus brand. Action 2 participants 
internationalised  their  teaching,  and  improved  their  institutional  capacities.  These 
partnerships were very responsive to development needs. Action 3 results were known to 
policy-makers and the National Structures. 

Although  the  programme  was  successful  in  increasing  the  involvement  of  European 
students,  the  introduction  of  scholarships  for  European  students  under  Action  1  did  not 
attract as much attention as expected. According to the results of a comparative analysis, 
the  stipend  and  other  scholarship  benefits  of  category  B  scholarships  were  lower  than 
those offered by other scholarship schemes. As a result, scholarships for European students 
were  not  competitive  enough  and  did  not  encourage  their  active  participation  in  the 
programme. 

Efficiency: The evaluation found that the programme was being implemented efficiently. 
First, most of the planned outputs of the programme were likely to be achieved by 2013 
with lower costs than initially anticipated. Second, outputs of the programme were being 
produced with analogous or even lower costs than those of scholarship schemes pursuing 
the  same  goal  (particularly  German  Academic  Exchange  Service  and  Fulbright).  Third, 
relevant  instruments  ensuring  low  administrative  costs,  such  as  large  size  consortia  and 
partnerships  or  calculation  of  incurred  costs  on  the  basis  of  lump  sum  amounts,  were  in 
place. Fourth, a significant number of institutional beneficiaries of the programme reported 
that their participation in Erasmus Mundus was a financial burden. Therefore, achievement 
of the same results with less funding would hardly have been possible. 

Management aspects: The monitoring of individual beneficiary selection and participation 
was  carried  out  to  some  extent,  with  the  universities  collecting  information  about 
applicants,  drop-outs  and  beneficiaries.  However,  it  could  be  standardised  and  more 
aligned with graduate tracking. Comparable information on candidate nationality, gender, 
ranking in pre-selection and selection, mobility track and duration, and results achieved (if 
applicable) would be useful for gathering data about the programme in the future. 

The  programme  beneficiaries  assessed  the  preparation  and  implementation  of  Erasmus 
Mundus  projects  positively,  except  for  the  extensive  administrative  workload.  Although 
the number of activity reports was reduced during the programme implementation, further 
simplifications were possible, including replacement of the annual reapplication approach. 
Although project monitoring and evaluation was primarily quantitative, one could exploit 
the  potential  of  the  Erasmus  Mundus  Quality  Assessment  project  to  better  assess  the 
quality of joint programmes with the involvement of field experts. 

Inputs  to  decision  making:  The  evaluation  results  have  been  used  to  support  the 
negotiations  of  the  2014  –  2020  Erasmus  for  All  programme,  e.g.  on  aspects  of 
strengthening the links to related programmes, and to further promote employability. 

Limitations to the evaluation: Due to the fact that the programme started up only in 2009 
this  interim  evaluation  could  not  be  organised  at  an  earlier  point  in  time.  Thus,  the 
evaluation  results  arrived  only  after  the  Commission's  proposal  for  the  successor 
programme was launched, which limited the possibilities of the evaluation to feed into the 
design of the successor programme. 

EN 135 

Error! Unknown document property name.  EN 

 
6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/education_culture/evalreports/education/2012/mundus_en.pdf 

FP7 Marie Curie Life-long Training and Career Development Evaluation: 
Individual Fellowships and Co-funding Mechanism 

1) ABB activities: 15 07 

2) Timing: February 2012 

3) Budget (annual): 

The indicative budget for the period 2007-2010 in respect of Activity 2 'Life-long Training 
and  Career  Development'  was  EUR448  million  for  the  Individual  Fellowships  (IF)  and 
EUR215  million  for  the  Co-funding  Mechanism  (COFUND).  In  addition,  the  two 
international  fellowship  programmes  within  Activity  4  'International  Dimension'  had  an 
indicative budget of EUR210 million. 

4) Background, scope and focus: 

The  scope  was  1)  the  two  implementation  modes  of  Activity  2  "Life-long  Training  and 
Career Development" in the FP7 People Specific Programme: Individual Fellowships and 
Co-funding; and 2) Activity 4 "International Dimension". 

The evaluation focused on relevance (including coherence and added value), effectiveness 
and efficiency 

5) Summary of findings and conclusions of the evaluation:  

EU ADDED VALUE 

Overall there is strong Added Value to the EU carrying out Marie Curie Actions (MCA) -
Individual Fellowships and COFUND- compared with Member States alone: this derives 
in  particular  from  the  cost  effectiveness  and  economies  of  scale  delivered  by  having  a 
common  strategic  approach  and  management  system  and  from  the  scheme's  openness  in 
terms  of  research  fields.  In  addition,  the  level  of  visibility  and  profile  that  MCA  has  is 
more likely to be achieved at a European scale. Measures to address fragmentation, in line 
with  the  development  of  European  Research  Area,  are  best  implemented  through  a 
Europe-wide scheme rather than via an equivalent series of national schemes. 

EFFECTIVENESS 

Both COFUND and Individual Fellowship Actions are achieving their objectives and are 
strengthening  the  research  capacity  of  the  EU  through  an  increase  in  the  quality  and 
quantity  of  researchers.  COFUND  Fellowships  are  contributing  both  quantitatively  and 
qualitatively  to  the  reinforcement  of  the  human  resource  potential  of  the  ERA;  the  main 
impact on host organisations is to expand research capacity and give institutions access to 
high-quality  researchers,  leading  to  stronger  institutional  research  outputs;  COFUND  is 
having a modest positive impact on the operational and administrative procedures of host 
institutions; and COFUND or Individual Fellowships have contributed to the development 

EN 136 

Error! Unknown document property name.  EN 

 
  
 
 
of researchers’ careers.  

EFFICIENCY 

Overall, the budget appears appropriate – the quantity and quality of outputs and results, 
and success rates for Individual Fellowships appear satisfactory. On average, remuneration 
for  Individual  Fellowships  is  slightly  higher  than  for  COFUND  fellowships  and 
employment conditions of Individual Fellows are also slightly better. Administrative costs 
for COFUND programme beneficiaries can be significant, but are justified by the benefits. 
The average duration between call deadlines and contract signature for COFUND projects 
needs to be reduced.  

The following are some examples of actions that have been or will be taken as a response 
to the results of the evaluation: 

Both COFUND and Individual Fellowships have been maintained as modalities of the 

• 
People Programme, and will be maintained in Horizon 2020. 

• 

COFUND's visibility is being increased through conferences and workshops. 

• 
The quality standards of Individual Fellowships will be strengthened by streamlining 
the  design  and  by  raising  their  visibility  outside  Europe  via  international  events, 
conferences and dissemination activities 

The  number  of  MCA  will  be  reduced  in  Horizon  2020,  and  current  labels  will  be 

• 
regrouped. 

Procedures  regarding  information  requirements  and  budget  negotiations  will  be 

• 
simplified in Horizon 2020. 

• 

COFUND will be opened to commercial bodies in Horizon 2020. 

The main limitation of the evaluation has to do with the fact that, for COFUND, it is too 
early to consider long-term impacts so the evaluation had to focus mainly on shorter-term 
results. However, an assessment of demand, motivations and expected impacts provides 
key insights into potential longer-term impacts, which are particularly relevant to 
considering the contribution of Marie Curie Actions (MCA) to the ERA for example. 

6) Availability of the report on Europa 

http://www.era.gv.at/attach/MarieCurieCOFUNDevaluationFinalReport_Feb2012.pdf 

EN 137 

Error! Unknown document property name.  EN 

 
 
 
Ex-post Evaluation of 2011 European Capitals 

of Culture 

1) ABB activities: 15.04  

2) Timing: August 2012 

3) Budget: EUR 3 million  

4) Background, scope and focus:   

The general objectives of the European Capital of Culture (ECOC) are to: "highlight the 
richness  and  diversity  of  European  cultures  and  the  features  they  share,  as  well  as  to 
promote greater mutual acquaintance between European citizens". The current evaluation 
covers two European Capitals of Culture in 2011: Tallinn and Turku.  

The evaluation looked at the ECoC discretely and considered how they performed against 
the requirements of the  Decision and their own objectives.   It also considered the ECoC 
Action as a whole, e.g. programme mechanisms operated by the European Commission. 

The methodology, combining a review of secondary data supplied by the ECoC as well as 
the  collation  of  primary  data  (e.g.  through  interviews,  site  visits  and  project  survey), 
allowed the evaluation to achieve the requested results.  

5) Summary of findings and conclusions of the evaluation:  

The  evaluators'  overall  conclusions  are  the  following:  Relevance:  ECoC  remains  of  key 
importance  and  thus  of  significant  relevance  for  the  Treaty  through  contributing  to  the 
flowering  of  the  Member  States  cultures,  highlighting  common  cultural  heritage  and 
increasing  cultural  co-operation  between  Member  States  and  internationally.  The  ECoC 
concept continues to be of relevance to the objectives of the EU and of local stakeholders, 
in  particular  in  promoting  the  European  dimension  of  culture;  in  the  development  of  the 
range  and  diversity  of  cities'  cultural  offerings;  in  strengthening  the  capacity  and 
governance of cities cultural sectors; in enhancing social development and citizenship; in 
promoting  the  international  profile  and  economic  development  of  cities.  It  remains 
complementary  to  other  EU  initiatives  and  programmes.  Efficiency:  The  current 
monitoring arrangements show a significant improvement to the previous years. However, 
they  do  not  ensure  that  all  cities  fulfil  all  their  commitments  made  at  application,  first 
monitoring  stage  and  second  monitoring  stage  (thus  also  in  respect  to  the  award  of  the 
Melina  Mercouri  Prize).  At  national  and  local  level,  the  governance  and  management  of 
ECoC is often challenging and political influences remain very significant.  

At European level, the ECoC Action continues to be very cost-effective when compared to 
other EU policy instruments and mechanisms. However, the share of the Melina Mercouri 
Prize  within  the  overall  budget  of  the  ECoC  programme  varied  widely  between  the  two 
cities and thus also its significance. For example, in Turku the Prize was relatively small in 
relation  to  the  overall  budget  and  was  thus  primarily  of  symbolic  importance,  i.e. 
providing recognition that the city had progressed significantly in its preparations to host 
the ECoC title. In contrast, the Prize represented more than 10% of the overall funding of 

EN 138 

Error! Unknown document property name.  EN 

 
 
 
Tallinn  and  was  thus  more  significant  in  terms  of  increasing  the  size  and  scale  of  the 
cultural  programme.  However,  in  neither  city  were  the  benefits  of  the  Prize  made 
particularly visible to cultural actors and audiences. 

innovative  and 

Effectiveness: The 2011 ECoC both succeeded in implementing cultural programmes that 
were  more  extensive, 
themes, 
artists/performers  and  audiences)  than  the  usual  cultural  offering  in  each  city.  They 
explored new themes, highlighted the richness and diversity of each city's cultural offering 
and used new or unusual venues. At the same time, it must be highlighted that the cultural 
programme  of  Turku  was  much  larger  in  scope  and  scale  than  that  of  Tallinn;  indeed, 
Tallinn 2011 was one of the least-extensive ECoC of recent years.  

international  (e.g. 

terms  of 

in 

European added-value: The European dimension of the cultural programme of both ECoC 
mostly  related  to  the  efforts  to  support  transnational  cultural  co-operation  and  to 
internationalise  the  cities'  cultural  sectors.  Whilst  European  themes  were  present  in  both 
ECoC, these tended to relate to specific projects rather than permeating the entire cultural 
programme. Whilst both cities presented very strong local narratives, those narratives can 
be  seen  as  containing  common  themes  that  are  essentially  European  in  nature.  For 
example, the cultural programme of both cities highlighted their common histories related 
to the role of Russia, the Baltic Sea and as well as the contemporary characteristic of being 
multicultural  societies.  The  experience  of  2011  demonstrates,  therefore,  that  all  "local" 
narratives  if  well  explained  can  be  considered  as  European  in  essence.    Both  ECoC 
strongly supported citizens' active participation and targeted people who traditionally tend 
to  participate  less  in  cultural  activities.  In  Tallinn,  significant  attention  was  given  to 
involvement  of  Russian  community  in  the  ECoC  programme  and  separate  projects  were 
devoted to it. In Turku, the Swedish-speaking community was encouraged to take part in 
the cultural activities through ensuring that all activities and materials were accessible, i.e. 
translated  into  the  Swedish  language.  However,  the  experience  of  both  ECoC  highlights 
the fact that very different (typically more intensive) approaches are required to widening 
the participation of citizens as creators or performers, as opposed to merely widening their 
participation as audiences. The evaluation delivered a set of recommendations, which will 
be the subject of an Action Plan to be elaborated in the coming months. 

6) Availability of the report on Europa 

http://ec.europa.eu/dgs/education_culture/evalreports/culture/2012/ecocreport_en.pdf 

Ex-post evaluation of the Preparatory Action Media International 2008-2010 

1) ABB activities : 15.04  

2) Timing: April 2012 

3) Budget (annual): EUR 8 million between 2008 and 2010 

4) Background, scope and focus  

MEDIA  International  was  a  preparatory  action  adopted  and  launched  by  the  European 

EN 139 

Error! Unknown document property name.  EN 

 
 
 
 
Parliament  to  strengthen  cooperation  between  the  audio-visual  industries  of  the  Member 
States of the European Union and those of third countries, and to encourage the circulation 
of cinematographic works between them. 

The objectives were to reinforce artistic and industrial cooperation between professionals 
of  the  audio-visual  markets  of  the  European  Union  and  those  of  third  countries;  to 
reinforce  the  exchange  of  information,  the  level  of  knowledge  on  foreign  audio-visual 
markets, and the international competencies of audio-visual professionals; and to improve 
the visibility and circulation of third-country audio-visual works on European markets, as 
well as European audio-visual works in those third countries. 

The actions implemented were to benefit the participants and works of European countries 
and  those  of  third  countries  in  a  balanced  way.  In  accordance  with  the  rules  of 
international  trade  and  specifically  WTO  regulations  ("most  favoured  nation"  principle), 
MEDIA International did not target any particular country; everyone had access, without 
discrimination. 

5) Summary of findings and conclusions of the evaluation:  

The evaluators' main conclusions are the following: 

Relevance:  The  purpose  of  the  preparatory  action  to  promote  the  dissemination  of 
European  works  in  third  countries  appropriately  addresses  the  challenge  to  open  new 
outlets for the European audio-visual industry.  

Efficiency and effectiveness: The choice of the preparatory action to target cinemas, rather 
than other channels such as television, reduced the capability of MEDIA International to 
increase the circulation of works, especially in less structured markets.  

MEDIA International has shown that it is possible to massively involve organizations from 
third countries in a European programme. This is evidenced in the wide  coverage of the 
programme (beneficiaries were from 48 countries, 31 of which were third countries).  

Even  though  it  was  on  an  insufficiently  large  scale  to  really  change  the  markets,  the 
preparatory  action  enabled  the  beneficiaries  to  improve  their  international  competencies 
and to form networks.  

The ETF was effective in providing and disseminating information and building capacity, 
but proved most effective in networking and knowledge transfer. 

European added value: The funded training and promotion actions enabled the participants 
to  acquire  knowledge  of  markets  in  foreign  countries,  and  to  establish  contacts  in  those 
countries. The beneficiaries were also able to strengthen their professional networks. 

MEDIA International had the greatest added value for organizations and participants from 
third countries where national support for training and export was less developed. 

Recommendations  are  quite  general  and  focus  more  on  the  condition  of  the  future 
preparatory  actions  and  the  modalities  of  an  international  Media  Programme,  given  the 
fact  that  the  moment  when  the  evaluation  has  been  finalised  the  Media  Mundus 
Programme has been already approved. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/education_culture/evalreports/culture/2012/media_fr.pdf 

EN 140 

Error! Unknown document property name.  EN 

 
External evaluation of the European Training Foundation (ETF) 

1) ABB activities: 15 02 

2) Timing: 8 February 2012 

3) Budget: Annual budget of about EUR 19 million 

4) Background, scope and focus:  

This evaluation addresses the activities of the ETF from 2006 to 2010, and focuses on its 
relevance, coherence, effectiveness and efficiency.  

This  is  the  4th  external  evaluation  of  the  ETF;  previous  evaluations  were  carried  out  in 
1998, 2002 and 2006. The next evaluation is planned for 2015. 

5) Summary of findings and conclusions of the evaluation:  

Overall, the ETF performs very well given its wide mandate and limited resources. There 
has  been  significant  effort  over  the  last  five  years  to  improve  ETF  governance  and 
operations, and this is already becoming evident in its actions and results. Despite its vast 
mandate,  large  geographical  area  and  modest  budget,  ETF  actions  were  relevant 
thematically and procedurally to beneficiaries and the ETF was flexible in addressing their 
needs. Regional activities were highly valued and should continue to be developed. ETF 
actions were coherent internally and with broader EU objectives. The ETF was effective in 
providing and disseminating information and building capacity, but proved most effective 
in networking and knowledge transfer. However, a clear hierarchy between strategic and 
operational  ETF  objectives  was  not  explicitly  developed  in  mid-term  perspectives  and 
annual work plans, and performance measurement of ETF lacked result-level indicators.  

The main actions proposed for improvement are the following: 

Information,  communication  and  networking:  The  ETF  should  be  proactive  in 
1. 
clarifying  further  its  role,  goals  and  ways  in  providing  support  to  its  partner  country 
beneficiaries.  New  approaches  developed  by  the  ETF  such  as  regional  and  thematic 
networks should be strengthened and expanded to other areas. This will help to harness the 
potential  of  both  ETF  to  beneficiary  and  beneficiary  to  beneficiary  communication  and 
policy  learning.  In  addition,  these  networks  should  include  short-term  feedback 
mechanisms to improve further day-to-day contact between the ETF and stakeholders. 

2.  Clear  hierarchy  of  objectives  and  performance  measurement:  Annual  work 
programmes  and  country  plans  should  demonstrate  a  clearer  link  between  actions, 
operational  and  strategic  objectives  of  the  ETF.  ETF  performance  indicators  should 
measure not only immediate outputs, but results as well. This would help to identify more 
accurately ETF progress towards the achievement of its objectives.  

3.  Sufficient  resources  and  long-term  engagement  by  the  ETF  at  the  partner  country 
level are absolutely essential in ensuring impact and added value of ETF actions. Broadly, 
this means that the ETF should maintain presence in all partner countries – and be given 
adequate resources to do so – even if this does not yield quick results or impacts. The EU 

EN 141 

Error! Unknown document property name.  EN 

 
 
 
 
should consider increasing ETF budget allocation to support EU priority partner countries 
when  specific  opportunities  for  increased  impact  are  presented  (such  as  democratic 
reforms and transition in the Southern and Eastern Mediterranean). 

An Action Plan has been established to address the above issues. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/education_culture/evalreports/training/2012/etf_en.pdf 

EN 142 

Error! Unknown document property name.  EN 

 
 
  
ABB 16 – Communication   

Evaluation of the European year of volunteering 2011 

1) ABB activities: 16.05  

2) Timing: 25/05/2012 

3) Budget (2010+2011): EUR 10.662 million 

4) Background, scope and focus: 

The  Council  of  the  European  Union  designated  2011  as  the  European  Year  of 
Volunteering (EYV) in order to encourage and support the efforts of the Community, the 
Member  States,  local  and  regional  authorities  to  create  the  conditions  for  civil  society 
conductive  to  volunteering  in  the  European  Union  (EU)  and  to  increase  the  visibility  of 
voluntary  activities  in  the  EU.  The  evaluation  both  formative  and  summative  assisted  in 
the implementation of the European Year and resulted in a final report on its impact and 
added value and drafted lessons that would be of use in future European Years. 

5) Summary of findings and conclusions of the evaluation:  

The evaluation report showed that overall the European Year of Volunteering (EYV) has 
had a positive impact on the world of volunteering, both at European and at national level. 
The objectives and activities of the Year were relevant, and the targeted, results-oriented 
approach was successful in reaching the objectives in all Member States, even though the 
impact varied according to specific national situations. 

The  European  Year  created  and  catalysed  changes  in  the  volunteering  environment  at 
European  and  national  level  and  led  to  the  adoption  or  modification  of  volunteering 
strategies  and  legislation  in  some  Member  States.  It  empowered  organisers  to  improve 
quality, including by focusing more attention on areas such as corporate volunteering and 
volunteering  as  a  non-formal  learning  experience.  It  increased  the  recognition  of 
volunteering through a wide range of initiatives and raised awareness of volunteering and 
its  value  to  society  through  the  media  and  the  European  communication  campaign.  The 
national activities had a valuable multiplier effect for these EU-level activities.  

The  European  Year  contributed  above  all  to  the  development  of  networks  and  new 
initiatives.  It  complemented  existing  activities  by  providing  additional  resources  that 
would not otherwise have been brought to bear, and it highlighted the European dimension 
of volunteering. 

The EYV 2011 left a legacy in the continuation of activities and structures which were put 
in place during 2011 and in the adoption of good practices that will bring changes in the 
years to come. 

Politically,  EYV  2011  achieved  sustainability  through  the  adoption  of  five  EU  policy 
documents  dealing  with  volunteering  in  the  European  Union,  i.e.  a  Commission 
Communication, two sets of Council Conclusions, an Opinion of the European Economic 
and Social Committee and a Report of the European Parliament.  

EN 143 

Error! Unknown document property name.  EN 

 
 
 
6) Availability of the report on Europa: 

    http://ec.europa.eu/citizenship/pdf/eyv_final_report_en.pdf 

Mid-term evaluation of Europe direct information centres (2009-2012) 

1) ABB activities: 16.03  

2) Timing: 27/01/2012 

3) Budget (annual): EUR 11.4 million 

4) Background, scope and focus: 

A mid-term external evaluation was lunch in June 2011. The aim of this evaluation was to 
examine  the  overall  performance  of  the  second  generation  of  Europe  Direct  Information 
Centres  (EDICs),  following  the  adjustments  of  the  mission  and  management  system 
implemented  in  2009.  In  this  sense  the  evaluation  has  provided  operational  conclusions 
and  practical  recommendations  to  be  used  as  reference  point  for  the  network's  future 
development. 

5) Summary of findings and conclusions of the evaluation:  

The  evaluation  report  showed  that  the  network  overall  complies  with  its  mission  of 
promoting  an  informed  and  active  citizenship.  However  the  network  remains  very 
heterogeneous, which has its strengths: maximises outputs with given inputs (at EU level) 
and  allows  tailoring  of  activities  to  host  structures’  interests  (at  local  level),  and 
weaknesses: medium to low levels of control (at EU level) and no assurance of the same 
service across the EU (at local level). 

Overall EDIC events have contributed significantly to an informed debate on EU issues: 
Nearly all EDICs (92%) have organised or participated in events (9.600 in 2010; 8.300 at 
the  end  of  2012).  The  four  types  of  main  events  were:  schools  (EU  lectures  and 
opportunities), conferences and debates; Europe days and participation to fairs. Education 
and training were by far the most frequent topics. Also media activities have – at least to 
some extent – contributed to promoting an informed debate. 

The organisation and management of the EDIC network, as implemented since 2009, had 
contributed to a more efficient and effective implementation of the network: Organisation 
of  the  network  is  adequate;  Guidance  and  assistance  are  provided  at  appropriate  levels; 
Management  is  decentralised  to  a  sufficient  degree;  Most  management  structures  –  the 
grant scheme, the module system, and the reporting system – demonstrate improvements if 
compared  to  the  previous  generation.  However,  formal  monitoring  practices  were  in 
decline (both by Representations and DG COMM). 

Requirements  as  defined  in  the  Management  guidelines,  Framework  and  Specific 
agreements are adequate and useful in guiding the implementation of the EDIC network. 
However the monitoring guidelines were little known, although when known and used – 
found very useful. The steering and coordination by Headquarters and Representations was 

EN 144 

Error! Unknown document property name.  EN 

 
 
 
 
perceived as adequate and the division of tasks optimal. 

Grant  scheme  based  on  lump  sum  contracts  as  implemented  since  2009  has  overall 
increase  the  efficiency  of  the  network.  It  has  significantly  reduced  the  administrative 
burden.  Despite  some  limitations,  the  module  system  was  perceived  as  superior  to  the 
previous system of global budget for the implementation of activities. It shifted the focus 
from  administration  of  costs  to  implementation  of  activities  It  also  encourages  forward 
planning  of  activities,  proactive  activities,  provides  better  structured  implementation  and 
allows better financial planning and predictability. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-europedirect-
infocentres2009-2012_en.pdf 

Interim evaluation of PressEurop 

1) ABB activities: 16 02  

2) Timing: 13/11/2012 

3) Budget (annual) : EUR 3 million  

4) Background, scope and focus:  

A mid-term external evaluation was launch in February 2012. The main goal of this study 
was to assess the impact and the use of PressEurop in order to improve it further and to 
assess the project's potential for the future.  

5) Summary of findings and conclusions of the evaluation:  

The evaluation report showed that PressEurop provided value for the money invested. It 
was  particular  perceived  as  relevant  to  its  objective  as  well  as  to  the  needs  of  its  target 
audience. Its "quality journalism" content and distribution measures have been deemed to 
"contribute  to  the  development  of  the  European  affairs  coverage".  However  this  project 
contributing to the building of a European public space could also be further improved as 
regards in particular the layout of the web portal and some communication measures of its 
language regime.        

Relevance: Regarding the quality of the coverage – both in terms of sources and countries 
– as well as the translation in 10 languages are relevant to the needs of the readers. The 
portal is functional and the current PressEurop users prefer to access the web portal via the 
Internet  and/or  receive  the  newsletter  rather  than  using  mobile  devices  and  smart/social 
features  (i.e.  Facebook,  Twitter,  etc.).  However  the  situation  is  always  evolving.  This 
project  is  complying  with  its  contractual  requirements  and  has  gone  beyond  them  in  a 
number of way: more content than required (at least 7 articles per day instead 3 as required 
in the contract); production of own content (such as blog and in brief which not required); 
development  of  an  original  integrated  comment  system  in  10  languages  (contractor's 
initiative); use of social media (contractor's initiative). 

EN 145 

Error! Unknown document property name.  EN 

 
 
 
 
Effectiveness:  PressEurop  contributes  to  improving  the  access  to  information  on  EU 
affairs  thanks  to  the  translation  of  quality  articles  from  newspapers  in  10  languages  and 
various  distribution  and  communication  measures.  These  measures  consist  in  displaying 
the content on different platforms (web portal, newsletter and mobile devices applications, 
social  media),  publication  on  partners'  websites  /  publications  and  participation  in  radio 
and TV programs. Nevertheless, the evaluators point out that the improvements in the lay-
out of the web portal and standardizing the partnerships regime could help to increase the 
audience of the project. 

Efficiency: the evaluation highlighted both strong assets and margins of improvement. 

- The scheme in terms of selection/translation clearly maximises the efficiency, ensuring 
the  selection  of  quality  articles  along  a  common  editorial  approach  within  limited  costs. 
The  wide  range  of  articles  and  viewpoints  is  appreciated  and  reflects  the  editorial 
independence foreseen in the contract. 

- The current 10 languages regime covers more than 87% of the EU population. The costs 
of translation, which amount to 25% of the PressEurop budget, appear proportionate to the 
potential impact.  

-  Both  the  social  media  and  the  mobile  applications  'audience  could  improve  their  cost-
efficiency. Indeed, it was innovative to offer such features (which were not required by the 
service contract) and it has nowadays become a clear necessity. However the investment 
realized  could  reach  more  people  and  hence  maximizes  the  cost-efficiency  of  the  all 
project. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-interim-evaluation-
presseurop_en.pdf 

Evaluation of the management partnership's actions within the scope of the 
2008-2010 communication plans in Italy 

1) ABB activities: 16 03  

2) Timing: May 2012 

3) Budget (annual): EUR 850,000 

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 

EN 146 

Error! Unknown document property name.  EN 

 
 
 
and procedures functioning. 

5) Summary of findings and conclusions of the evaluation:  

Efficiency:  In  evaluating  governance  and  managing  procedures  of  the  Partnership  some 
weaknesses  have  emerged.  Specific  and  relevant  weaknesses  have  been  encountered  in 
analysing  the  monitoring  process  and  system.  A  further  element  and  weak  point  that 
affected the execution of the evaluation consisted in the several changes and modifications 
of the Information and Communication Plans during their execution. In order to solve such 
critic  points  the  evaluator  has  proposed  some  actions  that  the  new  Intermediary  body  is 
putting in place.  

Effectiveness: The Partnership distinguished for its effectiveness in stimulating productive 
coordination  (previously  absent)  between  national  and  European  institutions  ensuring  a 
good level of synergy between them within a context of positive collaboration between the 
parties.  The  Partnership  has  "stimulated"  the  institutions  to  act  and  work  according  to  a 
different approach and "forcing" them in setting up overall governance, previously absent. 
The  main  contribution  and  added  value  of  the  Management  Partnership  has  been  that  of 
creating a common framework on the basis of precise communication themes which could 
be  "tailored"  according  to  the  specific  characteristics  of  the  national  context  and  of  the 
targets selected. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
italy2008-2010_en.pdf 

Evaluation of the management partnership's actions within the scope of the 
2009-2012 communication plans in Sweden 

1) ABB activities: 16 03  

2) Timing: June 2012 

3) Budget (annual): EUR 350,000 

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the  evaluation  of  MP  set  the  evaluation  goal  to  assess:  its  effectiveness  in  terms  of 
increased impact of the Commission's communication priorities at Member State level and 
achieved goals in reaching the intended public and yielding the desired results; its added 
value and its efficiency in bringing about synergies and an increased coordination between 
the  Member  State  and  the  Commission  as  well  as  in  its  mechanisms  and  procedures 
functioning. 

5) Summary of findings and conclusions of the evaluation:  

The organisation in the form of a Management Partnership has yield significant gains. The 
active parties all have the incentive to participate, and synergy effects have had a decisive 
impact  on  the  project's  processes.  The  project  management  has  been  efficient  and 
contributed  to  the  positive  result.  All  planned  educational  activities  have  been  carried 

EN 147 

Error! Unknown document property name.  EN 

 
 
through.  The  quantitative  targets  have  been  achieved.  Assessments  and  focus  interviews 
showed that training activities had resulted in increased knowledge of the EU among the 
participating teachers and school heads. The management partnership has also resulted in 
the creation of new structures which, in turn, might contribute to permanent effects even 
after the end of the management partnership. This has been achieved through the training 
school  ambassadors,  involving  school  leaders  and  creating  proper  educational  tools.  The 
effectiveness  of  the  Management  Partnership  has  been  judged  relatively  high.  It  has 
achieved  a  number  of  positive  results  when  compare  to  its  objectives.  The  effectiveness 
should  be  judged  against  the  fact  that  school  system  is  often  seen  as  important  for  the 
achievement  of  social  change.  During  the  period  the  management  partnership  has  been 
active, major reforms have taken place within the Swedish school system with particular 
focus  on  upper  secondary  schools  –  the  target  group  for  the  Swedish  management 
partnership.  It  resources  have  allowed  school,  individual  teachers  and  school  heads  to 
prioritise  teaching  about  the  EU  in  parallel  within  on-going  internal  developments  and 
changes. 

6) Availability of the report on Europa:  

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-ep-
sweden_en.pdf 

Evaluation of the management partnership's actions within the scope of the 
2009-2012 communication plans in Finland 

1) ABB activities: 16 03  

2) Timing: July 2012 

3) Budget (annual):  EUR 350,000    

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning.   

5) Summary of findings and conclusions of the evaluation  

In  Finland,  the  'School  in  Europe  –  Europe  in  School'  programme  has  been  evaluated. 
Most of the goals of this programme were achieved either with good or fair success.  

The  most  prominent  additional  value  the  Management  Partnership  has  created  for  the 
Commission's communications is the ability to more effectively reach two important target 
groups - teachers and pupils – improving geographical coverage and the ability to operate 

EN 148 

Error! Unknown document property name.  EN 

 
 
 
directly in terms of these two target groups. 

The  most  substantial  synergy  benefit  of  the  programme  has  been  utilising  a  "reserve"  of 
foreign  Erasmus  students  residing  in  Finland  in  the  service  of  the  comprehensive  and 
upper secondary schools.   

The  programme  has  strengthened  the  collaboration  between  the  Centre  for  International 
Mobility (CIMO) – which is responsible for several other EU programmes as well – and 
the  European  Commission,  European  Parliament,  European  Information,  and  the  Finnish 
National Board of Education.  

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
finland_en.pdf  

Evaluation of the management partnership's actions within the scope of the 
2009-2011 communication plans in Malta 

1) ABB activities: 16 03  

2) Timing: August 2012 

3) Budget (annual): EUR 250,000 

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the  evaluation  of  MP  set  the  evaluation  goal  to  assess:  its  effectiveness  in  terms  of 
increased impact of the Commission's communication priorities at Member State level and 
achieved goals in reaching the intended public and yielding the desired results; its added 
value and its efficiency in bringing about synergies and an increased coordination between 
the  Member  State  and  the  Commission  as  well  as  in  its  mechanisms  and  procedures 
functioning. 

5) Summary of findings and conclusions of the evaluation: 

the 

increase 

impact  of 

the  Communication's 
The  Management  Partnership  has 
communication  objectives.  Impact:  The  Management  Partnership  has  engaged  in 
innovative communication activities which are relevant and effective in delivering the core 
messages  to  target  audiences  in  Malta.  On  the  whole,  the  operations  which  were  carried 
out are considered effective and score an average of 3.6 (out of a maximum 5 points) in a 
multi-criteria  analysis  exercise  that  was  carried  out  to  analyse  the  20  operations  under 
evaluation. 

Efficiency  and  cost-effectiveness  of  operations:  In  evaluating  the  efficiency  and  cost-
effectiveness  of  operations,  the  multi-criteria  analysis  undertaken  in  respect  of  all 
communication operations resulted in a score of 3.7 (out of a maximum 5) for 2009, 3.2 
for  2010  and  3.8  for  2011.  This  scoring  would  indicate  that  the  efficiency  and  cost-
effectiveness  of  operations  are  generally  positive,  although  there  is  certainly  room  for 

EN 149 

Error! Unknown document property name.  EN 

 
 
 
further improvement.          

Added  value:  The  introduction  of  the  Management  Partnership  has  let  to  additional 
administrative  capacity  and  an  increase  level  of  communication  activities  aimed  at 
different target sectors of the local population. 

Synergies:  The  Management  Partnership  provides  a  structural  environment  for  synergy 
and coordination between the different partner institutions, which can now channel shared 
communication objectives through a central organisation.      

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
malta_en.pdf 

Evaluation of the management partnership's actions within the scope of the 
2009-2012 communication plans in Greece 

1) ABB activities: 16 03  

2) Timing: August 2012 

3) Budget (annual): EUR 350,000   

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning.   

5) Summary of findings and conclusions of the evaluation:  

Effectiveness: 
the  annual  communication  plans  appeared  exceptionally  ambitious 
compared  to  the  final  result.  It  should,  however,  be  acknowledged  that  since  late  2010, 
there was in Greek society an extremely negative political conjuncture, which undoubtedly 
affected the implementation of the planned activities of the Partnership.        

Efficiency:  The  realisation  of  the  Program  was  influenced  by  structural  reshuffles  at 
Minister level. As a result, the whole construction of the communication field between the 
two  parts  of  the  Program  had  to  restart  each  time  a  new  political  person  undertook  the 
position  of  the  Minister,  and  thus,  there  was  some  delay  both  in  the  planning  and  the 
realisation of the Program. 

EN 150 

Error! Unknown document property name.  EN 

 
 
 
 
6) Availability of the report on Europa 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
greece2009-2010_en.pdf 

Evaluation of the management partnership's actions within the scope of the 
2009-2012 communication plans in Lithuania 

1) ABB activities: 16 03  

2) Timing: September 2012 

3) Budget (annual): EUR 250,000    

4) Background, scope and focus: 

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C(2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning.   

5) Summary of findings and conclusions of the evaluation:  

The Management Partnership has been very positively evaluated by contractors.  

Numerous campaigns have been executed in Lithuania, which significantly contributed to 
effective  implementation  of  EC  communication  priorities,  reaching  foreseen  objectives 
and wide target audiences. Management Partnership program made significant impact on 
ability to communicate EU policies priorities and initiatives. 

The  Management  Partnership  contributed  an  added  value  (number  and  quality)  to  the 
communication activities undertaken by the partners. Management Partnership contributed 
to geographical extension of EU policies communication, as well to the content and quality 
of  communication.  Campaigns  became  larger,  number  of  campaigns  was  extended,  and 
more topics were covered. 

The highest level of synergy has been achieved within the Management Partnership.  

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
lithuania_en.pdf 

EN 151 

Error! Unknown document property name.  EN 

 
 
 
 
 
Evaluation of the management partnership's actions within the scope of the 
2008-2010 communication plans in Belgium 

1) ABB activities: 16 03  

2) Timing: September 2012 

3) Budget (annual): EUR 350,000    

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C(2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning. 

5) Summary of findings and conclusions of the evaluation:  

Sur la période évaluée, le Partenariat de gestion belge a réalisé 7 actions. 

De  manière  globale,  les  priorités  des  actions  développées  dans  ce  cadre  correspondaient 
aux priorités européennes annuelles de communication et aux objectifs stratégiques à plus 
long terme de la coopération entre l'Union européenne et les Etats membres.  

De manière globale, les actions furent bien adaptées à leur public. Ce fut notamment le cas 
des actions éducatives en direction des jeunes.  

L'impact  tant  qualitatif  que  quantitatif  des  actions  réalisées  a  été  difficile  à  mesurer  en 
raison de l'anonymat relatif de leur public et du manque de données recueillies.    

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
belgium2008-2010_en.pdf 

Evaluation des opérations de communication menées dans le cadre du 
partenariat de gestion 2008-2010 en France 

1) ABB activities: 16 03  

2) Timing: October 2012 

3) Budget (annual): EUR 850,000   

EN 152 

Error! Unknown document property name.  EN 

 
 
 
 
 
4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning.   

5) Summary of findings and conclusions of the evaluation:  

Les résultats de l'évaluation montrent que le partenariat de gestion 2008-2010 en France a 
atteint des résultats positifs. 

L'efficience:  le  partenariat  de  gestion  a  bien  été  créateur  de  synergie  et  de  coordination 
entre  la  Commission  européenne  et  le  Gouvernement  français  dans  la  définition  et  la 
les  priorités  de 
réalisation  des  actions.  Les  opérations  menées  ont  bien  servi 
communication dégagées par la Commission européenne et la France en ce qui concerne 
les thématiques exploitées et les cibles visées. La structure organisationnelle du partenariat 
a bien impliqué l'ensemble des acteurs concernés, permis une communication facile entre 
eux  et  fonctionné  selon  un  rythme  adapté  aux  exigences  de  la  réalisation  opérationnelle. 
Les  mécanismes  et  procédures  se  sont  avérés  efficients  et  ont  permis  un  usage  plus 
efficace  des  fonds  européens.  Ils  ont  bénéficié  du  coefficient  multiplicateur  des 
cofinancements  qui  seuls  ont  permis  d'obtenir  de  forts  impacts  sur  les  campagnes 
majeures.  L'investissement  du  partenariat  a  bénéficié  d'un  levier  de  coefficient  5  par 
rapport aux sommes dépensées. 

L'efficacité: le partenariat a permis d'améliorer l'impact des priorités de communication de 
la  Commission  européenne  en  France  grâce  à  l'effet  de  levier  susmentionné.  Les  actions 
menées ont bien atteint les publics prioritairement visés. Cependant, la trace laissée dans la 
durée  est  probablement  faible,  comme  l'absence  de  souvenir,  même  des  campagnes 
majeures parmi les interviewés non directement concernés l'a révélé. 

La plus-value qualitative du partenariat sur les opérations de communication de  

l'Union  européenne  réside  dans  la  qualité  des  prestations  réalisées  par  les  partenaires 
retenus  comme  le  montre  par  exemple  le  très  bon  impact  sur  la  perception  de  l'Union 
européenne du film diffusé pour l'élection au Parlement européen de 2009. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
france2008-2010_en.pdf 

EN 153 

Error! Unknown document property name.  EN 

 
 
 
Evaluation of the management partnership's actions within the scope of the 
2009-2012 communication plans in Poland 

1) ABB activities: 16 03  

2) Timing: December 2012 

3) Budget (annual): EUR 600,000    

4) Background, scope and focus:  

The  evaluation  of  Management  Partnerships  (MP)  was  envisaged  in  the  Commission 
Communication COM (2004)196 and Commission Decision C (2004)4841. Guidelines for 
the evaluation of MP set the evaluation goal to assess: 

-  its  effectiveness  in  terms  of  increased  impact  of  the  Commission's  communication 
priorities  at  Member  State  level  and  achieved  goals  in  reaching  the  intended  public  and 
yielding the desired results; 

-  its  added  value  and  its  efficiency  in  bringing  about  synergies  and  an  increased 
coordination between the Member State and the Commission as well as in its mechanisms 
and procedures functioning.   

5) Summary of findings and conclusions of the evaluation:  

The  evaluation  report  emphasised  that  the  projects  selected  for  financing  and  orders 
fulfilled were in line with the Communication Priorities and at the same time contributed 
to their popularisation. 

The adopted formula of execution of tasks broken down into tender projects and subsidy 
projects has been deemed correct. Each of them aimed at achieving different goals - tender 
projects were to reach the society or information disseminators, whereas subsidy projects 
were to go local.  

The  evaluators  found  difficult  to  assess  effectiveness  of  projects  undertaken  due  to  high 
diversity  of  actions  undertaken  on  the  one  hand,  and  on  the  other,  due  to  insufficient 
manner of measurement of results of projects (in particular in the case of competitions for 
subsidies). 

The analysis of cost effectiveness proved that it was worth supporting projects which used 
the internet as information carrier - that pertains in particular to publications/information 
brochures. 

Considering the Management partnership the evaluation report, based on the fact that all 
representatives  of  the  institutions  voiced  the  need  to  increase  a  partner  approach  to  the 
execution  of  assumptions  of  the  Delegation  Agreement  signed  by  et  between  the 
Commission and the then Office of the Committee for European Integration (later Ministry 
of Foreign Affairs), stressed to increase the role of the Coordination Team.  

6) Availability of the report on Europa: 

EN 154 

Error! Unknown document property name.  EN 

 
 
 
 
http://ec.europa.eu/dgs/communication/about/evaluation/documents/2012-partnership-ec-
poland_en.pdf 

EN 155 

Error! Unknown document property name.  EN 

 
 
 
ABB 17 – Health and Consumer Protection   

Assessment of the added value of the EU alcohol strategy and its implementation to 
enhancing action, cooperation and coordination to reduce alcohol related harm 

6)  ABB activities concerned: 17 03 

7)  Timing: December 2012, (Period covered by the report: 2006-2011) 

8)  Budget: 

No  spending programme  is  directly  linked  to  the  EU  alcohol  strategy,  however  since  2007,  the 
EU Health Programme has supported alcohol related projects with approximately EUR 9 million, 
and  the  EU  Research  Framework  Programmes  provided  approximately  EUR  49  million  for 
studies on alcohol and health. 

9)  Background, scope and focus 

The  evaluation  was  carried  out  in  the  context  of  the  EU  strategy  to  support  Member  States  in 
reducing alcohol-related harm that was adopted in October 2006. 

The purpose of the assignment was to assess the added value of the EU alcohol strategy and its 
implementation  at  European  and  national  level.  Specific  attention  was  given  to  the  main 
implementing instruments of the strategy: the Committee on National Alcohol Policy and Action 
(CNAPA),  which  main  objective  is  the  coordination  and  support  to  Member  States,  and  the 
European  Alcohol  and  Health  Forum  (EAHF),  with  special  attention  to  stakeholders' 
commitments to action. 

27  Member  States  and  all  representatives  of  EAHF  which  comprises  65  member  organisations 
were consulted for this exercise.  

10)  Summary of findings and conclusions of the evaluation or evaluation related study: 

EU value added, effectiveness and efficiency: 

The  evaluation  concluded  that  the  EU  alcohol  strategy's  outputs  support  Member  States  policy 
development  and  that  the  strategy  has  provided  an  EU-wide  foundation  for  action  on  alcohol-
related  harm.  It  considers  that  without  it,  a  common  approach  across  the  EU  would  not  have 
developed  and  EU  work  on  a  common  knowledge  base  would  likely  have  been  significantly 
reduced. National efforts to address cross-policy aspects would have been less strong without an 
EU-wide exchange of information. Dialogue and cooperation across a broad range of stakeholders 
at EU level would have been unlikely to take place to a comparable extent in the absence of an 
EU strategy. 

The information gathered in the report indicates that costs of the EU alcohol strategy, connected 
mainly with organisation of the CNAPA and EAHF meetings, appear reasonable compared with 
its value added as a foundation and catalyst for EU-wide action on alcohol-related harm. 

Proposed modification in the management of the programmes to improve their final impacts: 

The evaluation report concluded that the CNAPA work can be enhanced through greater political 

EN 156 

Error! Unknown document property name.  EN 

 
 
 
visibility  by  continuing  with  setting  up  high-level  meetings  and  establishing  links  to  the  EU 
Presidency  agendas.  Also  enhancing  current  work  on  cross-policy  issues  through  greater 
interaction  with  other  policy  areas,  including  both  Commission  services  and  national 
governments would be beneficial.  

Moreover, the evaluation suggested that the consistency and continuity of CNAPA’s work could 
be  strengthened  by  (1)  adoption  a  multi-annual  work  plan  and  monitoring  its  implementation 
through brief annual reports; (2) drawing on synergies with work on other risk factors for chronic 
non-communicable diseases. 

As  to  the  EAHF  the  evaluation  suggested  that  further  effort  to  bring  in  members  in  under-
represented  areas,  such  as  alcohol  retailers  and  health  and  social  insurers,  could  strengthen  the 
EAHF's  effectiveness.  Also  the  report  advised  strengthening  membership  in  sectors  where  the 
EAHF has provided a major stimulus for action, such as the media sector, including digital media 
and  exploring  ways  to  involve  further  sectors,  such  as  law  enforcement  and  local  and  regional 
government.  Moreover  the  need  to  increase  participation  of  stakeholders  from  new  Member 
States was underlined. 

The evaluation also proposed strengthening synergies between CNAPA and EAHF by providing 
CNAPA members with a yearly overview at national level of EAHF commitments to action and 
encouraging Member States’ active participation in EAHF meetings.  

Limitations of the report:  

The  overall  conclusions  on  the  EU  Alcohol  Strategy  are  positive  and  well  reflect  the  general 
stakeholders'  appreciation  of  the  process,  but  it  needs  to  be  underlined  here  that  the  evaluation 
report does not completely fulfil the required Commission evaluation standards. Main limitations 
of the evaluation report include lack of rigour in data collection, low response rate and misleading 
reporting of findings. What is more, there is insufficient level of analysis and assessment and in 
this respect some of the offered conclusions do hardly reach beyond the summary of results. 

6) Availability of the report on Europa:  

The report is not yet available on the Europa website.  

External evaluation of the Consumer Protection Cooperation Regulation 

11)  ABB activities concerned: 17 02 

12)  Timing: 17 December 2012, (Period covered by the report: 2007 – 2011)  

13)  Budget: not applicable 

14)  Background, scope and focus:  

At  the  end  of  2010  the  Commission  initiated  an  assessment  of  the  enforcement  coordination 
mechanisms established by the CPC Regulation. This process was inserted as a new review clause in 
the  CPC  Regulation  in  the  framework  of  an  amendment  to  the  legal  text  (Regulation  (EU)  No. 
954/2011).  The evaluation was part of this process. The main focus was put on the CPC Regulation, 

EN 157 

Error! Unknown document property name.  EN 

 
 
 
 
its  implementation  in  the  Member  States  and  the  infrastructural  and  budgetary  support  for  it.  The 
evaluation  addressed  both  (1)  the  cooperation  arrangements  in  the  framework  of  the  CPC  Network 
established  by  the  CPC  Regulation  in  the  context  of  the  enforcement  of  the  legislation  listed  in  the 
Annex to the Regulation and (2) the broader forms of cooperation established by the CPC Regulation 
under its Chapter IV. 

15)  Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related  study: 
Relevance of the CPC Regulation:  

The  report  concluded  that  the  rationale  for  intervention  is  as  valid  and  appropriate  today  as  it  was 
when  the  CPC  Regulation  was  introduced  in  2004.  It  may  even  be  said  that  the  CPC  Regulation 
framework  is  even  more  needed  than  before  given  developments  in  technology  and  consumption 
patterns of households (i.e. in digital and online purchasing) and the complexity of trader activity in 
aspects such as advertising, bundling of products and trading internationally have reinforced the need 
to provide tools to deter rogue traders and to enforce the law. Even though, the current objectives of 
the  CPC  Regulation  are  appropriate  and  relevant,  none  of  the  specific  objectives  of  the  CPC 
Regulation have been fully achieved. Therefore it would be appropriate to improve the clarity of the 
text within the CPC Regulation in order to ensure more uniform understanding among national actors. 

Scope of the CPC Regulation: The analysis concluded that the current scope of the CPC Regulation 
annex is broadly correct in terms of the sensitivity of the legislation to public enforcement dimension, 
collective  consumer  interest,  cross-border  relevance  and  consistency.  To  maintain  consistency  in 
consumer  protection  the  evaluation  report  suggested  the  possible  inclusion  of  the  Rail  Passengers' 
Rights  Regulation,  the  Roaming  Regulation  and  the  Mortgage  Credit  Directive  in  the  scope  of  the 
CPC Regulation. Also it was recommended to examine further the possible inclusion of some specific 
provisions within certain legislative acts, such as the Air Services Regulation. 

       Legal barriers to cooperation under the CPC Regulation  

Applicable substantive law 

The  CPC  Regulation  does  not  regulate  which  applicable  substantive  law  applies  in  mutual 
assistance  requests.  This  issue  has  been  raised  as  a  source  of  difficulties  hindering  the 
effectiveness  of  cross-border  enforcement  cooperation.  The  evaluation  offered  following 
recommendations to alleviate constraints arising due to differences in substantive law: 

•  Provide guidance on applicable law in relation to the handling of mutual assistance requests 
under  the  CPC  Regulation  in  order  to  promote  better  awareness  and  compliance  with  EU 
consumer legislation; 

•  Foster a common understanding in relation to the application of EU consumer law, inter alia 
through  guidance,  on-line  legal  resources,  and  interactive  forums  to  also  facilitate  greater 
knowledge exchange; 

•  Monitor  the  handling  of  CPC  mutual  assistance  requests  to  determine  the  nature  of  legal 
issues; 

•  Consider  formalising  an  approach  through  ad  hoc  rules  reflecting  the  logic  of  the  CPC 
Regulation and catering for the specific situations of real conflict between applicable rules in case 
the instances of such situations increase; 

•  Consideration should be given to the establishment of an ‘Outsourced legal helpdesk’. 

Procedural  laws  and  powers:  Another  type  of  legal  barrier  that  stands  in  the  way  of  effective 

EN 158 

Error! Unknown document property name.  EN 

 
cross-border  enforcement  arises  from  differences  in  national  procedural  rules  used  to  stop  the 
infringement.  In  most  cases  they  are  not  designed  or  applied  in  practice  to  take  account  of  the 
needs  for  cross-border  cooperation.  Recommendations  to  alleviate  constraints  arising  due  to 
differences in procedural law and enforcement powers: 

•  Provide guidance and foster best-practice to improve the quality of mutual assistance requests 
under the CPC Regulation. 

•  Investigate possible gaps in the implementation of minimum powers and assess the need for 
action in this respect. 

•  Examine options to revise the CPC Regulation to include minimum procedural standards. 

•  Examine  options  to  revise  the  CPC  Regulation  to  expand  the  minimum  investigative  (e.g. 
broader  spot  check  powers  on  companies'  premises)  and  enforcement  powers  (e.g.  including 
redress  powers)  of  national  competent  authorities  in  order  to  overcome  certain  difficulties 
reported for the efficiency of cooperation and to address a potential deterrent gap stemming from 
national fragmentation.  

The  four  cooperation  mechanisms  of  the  CPC  Network:  The  CPC  Regulation  establishes 
following  cooperation  mechanisms  and  common  activities  among  Member  State  authorities  and 
with  the  Commission:  mutual  assistance  requests;  coordinated  surveillance  activities;  common 
activities  to  foster  a  common  approach  to  enforcement;  general  reporting  and  Committee 
meetings. The use of the mutual assistance requests varies considerably across the Member States. 
A  strong  downward  trend  in  the  number  of  alerts  is  observed.  Regarding  the  common  activities 
each  year  one  or  two  projects  have  been  financed  involving  sometimes  a  large  number  of 
countries, but there has only been a limited take up of co-funding for exchanges of officials. The 
biennial assessments of the CPC Regulation pointed further to possible barriers to cooperation due 
to resources constraints at the level of Member State authorities, divergence in participation to the 
CPC  network  and  a  possible  need  to  review  the  Commission's  role  in  the  CPC  framework. 
Recommendations in relation to functioning of the CPC Network and common actions: 

•  The Commission should continue developing guidance clarifying the functioning of the tools 
of the CPC Regulation and the obligations placed on Member States. 

•  The  text  of  the  CPC  Regulation  could  be  revised  in  order  to  increase  the  clarity  of  the 
objectives, the obligations and the understanding of current provisions. 

•  Consideration  should  be  given  to  whether  the  Commission  could  make  greater  use  of  its 
powers to introduce infringement proceedings against the Member States that do not fulfil their 
obligations under the CPC Regulation. 

•  Where several Member States seek enforcement against the same trader, guidance should be 
provided to help coordinate enforcement actions.  

•  Consideration should be given to providing a legal base for a European body: to arbitrate in 
Member  State  conflicts  hindering  enforcement,  to  facilitate  case  handling  coordination  and  to 
enforce EU law in a trans-national context. 

•  Consider  and  develop  possibilities  to  cooperate  and  learn  from  other  systems  (such  as  IMI, 
RAPEX, and RASFF). 

•  A  common  national  reporting  protocol  should  be  adopted  in  terms  of  its  format  and  the 
information  to  be  reported  so  that  market  monitoring  intelligence  gathered  can  be  used 

EN 159 

Error! Unknown document property name.  EN 

 
systematically  and  ‘Joint’  EU  level  reports  that  both  reflect  on  the  achievements  of  CPC  and 
anticipate future challenges can be produced. 

•  Consideration  should  be  given  to  the  establishment  of  an  ‘Observatory’  that  would  use  the 
experience  and  information  collected  under  the  CPC  Regulation  (and  information  from  other 
sources) on the extent of infringements and challenges and successes of enforcement to generate 
material that would increase awareness amongst national authorities and consumer representative 
organisations, and ultimately EU citizens.  

•  Given  the  wide  variations  in  resources  committed  to  CPC  activities  at  the  national  level  the 
Commission  should  encourage  Member  States  committing  relatively  low  and  relatively  high 
levels  of  resources  to  compare  and  to  ‘peer  review’  their  practices  with  Member  States 
committing  average  (pro  rata)  levels  of  resources  and  acting  effectively  under  the  CPC 
Regulation. 

6) Availability of the report on Europa: 
http://ec.europa.eu/consumers/enforcement/docs/cpc_regulation_inception_report_revised290212_en.
pdf 

Evaluation of the EU rapid response network, crisis management and communication 
capacity regarding certain transmissible animal diseases 

6) 

ABB activities concerned: 17 04 

7)  Timing:    1  August  2012  (duration  of  the  evaluation  09.2012-07.2012,  Period  covered  by  the 
report: 1998 – 2009) 

8)  Budget: At the EU level the activity has no directly allocated budget. The relevant activities of 
the  Member  States  (MS)  are  covered  by  national  arrangements.  Following  outbreaks  of  certain 
diseases on the territory of MS, specific costs directly related to the emergency measures taken by 
them (e.g. compensating to owners for the slaughter, destruction of animals, costs incurred for the 
cleaning, disinfection of holdings and equipment, cost of supply of the vaccine etc.) are met by 
financial  contributions  met  by  the  EU,  adopted  as  Commission  Decisions  in  application  of 
Council Decision 2009/470/EC. 

9)  Background, scope and focus:   The  evaluation  was  carried  out  in  the  context  of  the  Animal 
Health  Strategy  for  the  European  Union  "Prevention  is  better  than  cure"  (Commission 
Communication  from  September  2007  COM(2007)539  final)  and  was  required  by  the  afferent  
Action  Plan  (Commission  Communication  from  September  2008-COM  (2008)  545  final).  The 
focus was put on assessing the current legislative and non-legislative environment as regards the 
state  of  preparedness  and  the  capacity  of  the  EU  rapid  response  network,  where  the  latter 
encompasses  the  Commission  Services,  MS  (especially  the  veterinary  Competent  Authorities) 
and  stakeholders,  such  as  private  sector  veterinarians  and  the  economic  operators  concerned. 
During its work, the contractor involved all Member States and major stakeholders. 

10) 

Summary  of  findings  and  conclusions  of  the  evaluation  or  evaluation  related

EN 160 

Error! Unknown document property name.  EN 

 
 
 
 
     study: Effectiveness, efficiency and added value of the EU rapid response network:  

The EU rapid response system is considered generally to work effectively and efficiently and to 
have improved very significantly over the last decade.  The availability of well developed, tested 
and  up  to  date  Contingency  Plans  (CPs)  as  an  indicator  of  preparedness  can  help  prevent  an 
emergency  from  becoming  a  crisis.  The  effectiveness  of  the  response  also  relies  on  good 
cooperation  and  coordination  within  the  overall  rapid  response  network,  including  between  the 
Commission and MS, regular and timely exchange of information (including scientific knowledge 
and  advice)  between  laboratories  and  with  stakeholders,  and  the  building  and  maintenance  of 
confidence  and  trust  between  all  parties.  The  evolution  of  the  EU  animal  health  co-financing 
indicates  a  decrease  in  the  amount  of  EU  co-funding  for  emergency  veterinary  measures  from 
some EUR65 million in 2000 to EUR30 million in 2011. Over the last five years EU co-financing 
has  averaged  EUR37  million,  well  below  the  average  over  the  whole  period  (EUR91  million, 
2000-2011 in terms of outturn payments, i.e. the sum of credits generated by a MS in a specific 
year). This points to the more efficient use of funds to achieve longer term objectives such as the 
reinstating  of  disease  free  status  for  major  diseases  in  the  EU.  The  comprehensive  set  of 
legislation now in place (including Contingency Plans and the EU emergency network in all its 
components) can be considered as a valuable shield against traditional contagious animal diseases 
and appears to be quite effective in terms of triggering the relevant steps and control measures to 
fight against emerging diseases or new "profiles" of known diseases. As a result of this, over the 
evaluation period, out of a significant number of outbreaks, relatively few have developed into a 
crisis. On the basis of the criteria of financial cost and economic impact, the following crises were 
identified: Classical Swine Fever (1997 DE); Avian Influenza-AI (1999/2000 IT); AI (2003 NL); 
H5N1 Influenza (2005-06); Foot and Mouth Disease (2001, UK); Bovine Tuberculosis (2007/08, 
DE/FR/NL/BE).  In  the  last  4  years  the  EU  has  not  experienced  an  animal  health  crisis,  and  in 
particular the potential of an African Swine Fever crisis due to the risk of re-introduction of this 
disease  from  the  Caucasus  region  was  reduced  to  nil.  The  extent  of  the  economic  and  social 
impacts  for 
the  wider  economy,  of  major  animal  health 
emergencies/crises that have occurred in the EU27 during the last two decades is however very 
significant.  On  the  basis  of  existing  studies,  impacts  can  extend  from  several  million  EUR  in 
direct losses, to hundreds of millions EUR or even several billion EUR if the indirect losses to the 
affected  sector  and  the  wider  economy  are  also  included.  In  recent  years,  due  to  improved 
preparedness,  effective  use  of  the  lessons  learnt  from  the  management  of  outbreaks  and 
development of networks of the actors involved in the EU rapid response system the EU 27 has 
no longer suffered from such extensive levels of losses.  

the  affected  sectors  and 

Recommendations as to the modification in the management to improve the final impacts:  

The  most  important  in  terms  of  economic  and  social  impacts  is  to  prevent  an  animal  health 
emergency from becoming a crisis and for this MS' implementation of Contingency Plans(CP) is 
crucial.  Room  for  improvement  has  been  identified,  such  as  enhanced  collaboration  between 
neighbouring MS, clearer minimum requirements for a "generic" (not disease specific) CPs and 
sufficient  flexibility  for  the  disease  specific  parts,  guidance  documents  to  be  provided  by  the 
Commission  for  the  drafting  of  the  plans  as  well  as  increased  stakeholder  involvement  in  the 
drafting process. In addition contingency planning should better address possibly future emerging 
diseases. While the current Commission's approval of the plans via comitology procedure is not 
deemed the most useful, the role of the Food and Veterinary Office in auditing the plans is felt 
very  effective  and  expandable.  The  adoption  of  containment  measures  in  relation  to  disease 
outbreaks  by  the  Commission  in  the  framework  of  the  Standing  Committee  on  the  Food  Chain 

EN 161 

Error! Unknown document property name.  EN 

 
and Animal Health is considered a very powerful tool in emergency response. Those Committee 
meetings also serve as an excellent platform for information exchange. Only minor suggestions 
are  made  on  making  better  use  of  modern 
tools 
(telephone/videoconferencing,  a  CIRCA  platform  for  uploading  of  meeting  documents)  also  in 
view of cost savings. 

technicalities  and  communication 

6) Availability of the report on Europa: 
http://ec.europa.eu/food/animal/diseases/strategy/pillars/docs/23_final_report_eu_rapid_response.pdf 

EN 162 

Error! Unknown document property name.  EN 

 
 
ABB 18 – Home affairs 

Evaluation on the European Union Crime Prevention Network 

1) ABB activities: 15 05  

2) Timing: 30.11.2012 (Period covered mid-2009-2012) 

3) Budget (2010+2011): The EUCPN benefits from financial support from the ISEC 

Programme through a grant of EUR 845,000 covering the period mid 2011-mid 2014. 

4) Background, scope and focus:  

According  article  9  of  the  Council  Decision  2009/902/JHA,  the  Commission  had  to 
present a report on the activities of the EU Crime Prevention Network with a special focus 
on the efficiency of the Network and its secretariat, taking due account of the interaction 
between the Network and other relevant stakeholders. Furthermore, the European Council 
invited  the  Commission  to  submit  a  proposal  to  setting  up  an  ‘Observatory  for  the 
Prevention  of  Crime’  (OPC),  "the  tasks  of  which  will  be  to  collect,  analyse  and 
disseminate  knowledge  on  crime,  including  organised  crime  (including  statistics)  and 
crime  prevention,  to  support  and  promote  Member  States  and  EU  institutions  when  they 
take  preventive  measures  and  to  exchange  best  practice.  The  evaluation  done  provided 
recommendations  as  regards 
including  considering  the  feasibility  of 
the  future, 
establishing a Crime Prevention Observatory. 

5) Summary of findings and conclusions of the evaluation: 

The  evaluation,  done  on  the  basis  of  an  independent  external  study,  addressed  several 
aspects  of  the  EUCPN,  including  its  performance,  organisation  and  governance.  It 
demonstrated  that  overall  the  network  is  functioning  relatively  well  and  has  made  good 
progress in relation to the objectives set out in the 2009 Council Decision and the 2010-
2015  Multiannual  Strategy  (be  a  point  of  reference  for  the  EUCPN  target  groups; 
disseminate  knowledge  on  crime  prevention;  support  crime  prevention  at  national  and 
local levels; develop EU policy and strategies with regard to crime prevention). Compared 
to  the  situation  when  the  previous  evaluation  was  conducted  (2008-2009),  many 
shortcomings have been addressed allowing for the Network activities to be more targeted 
and the quality and quantity of its outputs improved. Together research projects (18 since 
2009),  initiatives  such  as  producing  thematic  papers,  setting  up  a  stakeholder  database, 
redesigning the newsletter or launching a stakeholder feedback survey have all contributed 
to better cooperation, contact exchanges of information and experience between actors in 
the field of crime prevention 

However,  there  are  still  a  number  of  weaknesses  which  would  need  to  be  addressed  to 
increase  EUCPN's  performance,  impact  and  added  value  such  as  a  more  systematic 
alignment of the Network with agreed EU priorities, a strengthened role in making inputs 
to  EU  and  Member  State  policymaking,  a  more  strategic  approach  to  determining 
activities,  supported  by  the  development,  in  the  longer  term,  of  an  observatory-type 
monitoring function, further collaboration with Eurostat, CEPOL and EUROPOL. Even if 
overall the evaluation confirmed that the EUCPN Secretariat was performing well despite 

EN 163 

Error! Unknown document property name.  EN 

 
 
 
its  limited  capacity,  some  recommendations were  made  on  the  EUCPN  organisation  and 
governance; the effectiveness and capacity of the rotating EUCPN Presidencies should be 
improved as well as the role of the National Representatives in the Board. The role of the 
Board observers should also be clarified, and the role of the EUCPN's contact points better 
defined. 

On  the  basis  of  this  evaluation,  the  Commission  considered  that  the  EUCPN  is  an 
increasingly  useful  instrument  to  support  policymakers  at  EU  and  national  levels,  with 
considerable potential to create added value at local level. The positive development of the 
Network  over  the  past  two  years  makes  the  establishment  of  a  Crime  Prevention 
Observatory  neither  corresponding  to  a  pressing  need  nor  politically  or  financially 
desirable in the short term. The Commission rather recommended enhancing the EUCPN, 
notably by a better resourced Secretariat, to concentrate on consolidating progress made so 
far  and  further  improving  the  functioning  of  the  Network  on  the  basis  of  the 
recommendations made during the evaluation. 

6) Availability of the report on Europa: 

http://ec.europa.eu/dgs/home-affairs/e-library/documents/policies/organized-crime-and-
human-trafficking/crime-prevention/docs/20121130_eucpn_report_en.pdf 

EN 164 

Error! Unknown document property name.  EN 

 
 
 
ABB 19 - External Relations 

Evaluation of the EU cooperation with Colombia 

1) ABB activities: 19 09:  

2) Timing: October 2012, (period covered by the report: 2002-2011) 

3) Budget :Disbursements over the evaluation period: EUR196.61 million 

4) Background, scope and focus: 

The evaluation provides an overall independent assessment of the Commission’s past and 
current cooperation with Colombia over the period 2002-2011 and identifies key lessons in 
order  to  improve  the  current  and  future  strategies  and  programmes  of  the  Commission, 
particularly considering the new development policy set in the “Agenda for Change”.  

The evaluation focuses on the following areas of investigation: Peace, Stability and Socio-
economic  Development;  Rule  of  Law  and  Justice;  Human  Rights  and  Victims  of  the 
Armed  Conflict;  Good  Governance;  Productivity  and  Competitiveness;  Environmental 
Sustainability  and  the  Linkage  between  Relief,  Rehabilitation  and  Development.  It  also 
analyses the instruments and aid modalities made available by the Commission, as well as 
the  coherence,  coordination  and  complementarity  ("3Cs")  of  EU  cooperation  and  the 
European added value. 

5) Summary of findings and conclusions of the evaluation  

The  EU  cooperation  strategy  with  Colombia  from  2002  to  2010  is  characterized  by  the 
core objective of peace building. In general terms, the EU strategy was coherent with the 
Colombian policies in relation to peace-building by contributing to address the root causes 
for conflict. 

Key conclusions: 

•  Although  the  EU  strategy  has  supported  transition  from  relief  to  rehabilitation  in  a 
highly  sensitive  conflict  situation,  it  did  not  sufficiently  succeed  to  link  the  long  term 
objective of development to these medium term objectives. Determining factors were the 
lack  of  a  comprehensive  and  structured  strategic  framework  to  mainstream  good 
governance principles as well as a non-adaptive capacity development scheme, notably as 
regards  the  territorial  development.  Weaknesses  have  also  been  identified  as  regards 
coordination  and  complementarity  with  other  donors.  The  distinctive  policy  strategies  of 
active EU Member States hampered visibility of EU added value. 

• 
Conflict prevention and resolution: The EU contributed in strengthening civil society, 
creating  social  networks  and  promoting  productive  alternatives  and  reactivation  in  zones 
most affected by conflict. However, sustainability remains a major concern. 

• 
Rule of law, justice and human rights: By supporting the new Penal Oral Accusatory 
System, the EU has helped to improve the application of the Justice and Peace Law. It has 
provided  political  and  technical  support  for  the  protection  of  victims'  rights  and  the 
promotion  of  human  rights.  However,  impunity  in  relation  to  human  rights  violations 

EN 165 

Error! Unknown document property name.  EN 

 
 
 
remains  a  key  problem  and  specific  improvements  can  not  conclusively  be  attributed  to 
EU actions. 

• 
Trade  and  competitiveness:  By  supporting  the  Local  Economic  Development 
approach, the EU has contributed to create the basis for development and competitiveness 
notably  in  rural  areas.  The  next  step  needs  to  facilitate  participation  in  national  and 
international markets. 

Environmental sustainability: Albeit not being a priority in its strategy and not being 
• 
coordinated  with  the  responsible  ministry,  the  EU  succeeded  in  improving  the 
environmental sustainability. 

Key recommendations: 

• 
Conduct  impact  evaluations  in  order  to  inform  future  decision  making  policy  and 
strategy which should take into account the political dimension and reinforce the political 
dialogue at all levels. Fine tune and consolidate the LRRD strategy. Review the Backbone 
Strategy  and  its  EuropeAid  toolkit  for  improving  adaptability  of  capacity  development, 
notably for territorialized interventions. 

•  Maintain support to and ensure sustainability of civil organizations and the platforms 
for dialogue in the zones most affected by conflict. 

• 
Create a new cooperation strategy in this area based on inclusive political and policy 
dialogue with all stakeholders concerned. Use the dialogue as catalyst for developing and 
implementing an effective human rights policy. 

•  Use  the  recently  signed  EU-Colombia  Trade  Agreement  to  effectively  continue 
promoting  local  economic  development  working  towards  a  multiplication  effect  in  the 
national economy. 

• 
Formulate  the  environmental  strategy  with  the  Ministry  of  Environment  and 
Sustainable  Development and implement it in coordination with the national policy. The 
conclusions  and  recommendations  of  this  country  level  evaluation  will  be  taken  into 
account in the current programming exercise to be finalised in 2014. In this regards, some 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013. 

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2013/1315_docs_en.htm 

Evaluation of the EU cooperation with Ecuador 

1) ABB activities: 19 09  

2) Timing: September 2012; (Period covered by the report : 2003-2010) 

3) Budget: Disbursements over the evaluation period: EUR 204.8M. 

EN 166 

Error! Unknown document property name.  EN 

 
 
 
 
 
4) Background, scope and focus : 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 
cooperation  strategy  and  support  in  the  country  and  to  draw  out  key  lessons  to  help 
improve current and future EC strategies and programmes. 

The evaluation covers the EC’s cooperation programme with Ecuador over the 1996-2010 
period. The thematic focus covered (i) social services (education and health), (ii) trade and 
economic  development,  (iii)  natural  resources  management;  (iv)  sustainable  rural 
development, and (v) governance and democracy. 

5) Summary of findings and conclusions of the evaluation:  

EU  cooperation  was  well  aligned  to  the  Government's  priorities  and  to  the  population 
needs,  particularly  in  the Health  and  Education  sectors,  allowing  for  positive  results  and 
ensuring sustainability. The non-financial input in the context of Budget Support improved 
alignment to national policies and established dialogue with the Government as well as the 
adoption of national systems for programme design, managing, monitoring and reporting. 
Coherence was good at a strategic level, but limited in implementation. 

Key conclusions:  

• 
Strong  alignment  to  Government  of  Ecuador  priorities  and  responsiveness  to 
population needs. The two main interventions in the social sectors (Health and Education) 
contributed  to  the  implementation  of  significant  reforms.  In  the  environment  sector,  the 
Commission  has  succeeded  in  strengthening  decentralised  management  and  a  strong 
ownership at local level shows support responded well to population needs. In the areas of 
trade  and  economic  development,  Commission's  activities  aimed  at  enhancing  efficiency 
and competitiveness of the MSME, which absorbs the majority of the country’s workforce. 
Commission's support to rural development responded to the needs of poor regions while 
giving rise to sector policies both at central and local level.  

•  Main  EU  added  value  was  in  the  use  of  BS  modality,  which  allowed  enhancing 
alignment to national policies and establishing a privileged dialogue with the Government 
of  Ecuador  and  the  adoption  of  national  systems  for  programme  design,  managing, 
monitoring and reporting.  

•  Good  coherence  at  strategic  level  but  limited  at  implementation  level,  despite 
converging objectives few synergies have been found; particularly between thematic and 
bilateral cooperation and between national, sub-regional and regional interventions.  

•  Adequate  choice  of  aid  modalities  in  the  light  of  country  context.  The  Commission 
has  implemented  successfully  a  project  approach,  with  influence  on  policies  and  good 
sustainability; the subsequent shift towards budget support has further increased ownership 
and alignment. As regards Budget support operations it is recognised the relevance of non-
financial  inputs;  furthermore,  the  role  played  by  CSOs  in  reporting  has  impacted  and 
raised their profile.  

Key recommendations:  

The uncertainty on the continuity of bilateral co-operation with Ecuador did not allow for 
very specific and tailor-made recommendations. 

EN 167 

Error! Unknown document property name.  EN 

 
 
Ensure  continuity  of  support  to  the  environment  sector  through  sub-regional  co-

• 
operation and thematic budget lines;  

Ensure continuity of support to trade and economic development through regional co-

• 
operation in support of both the private and the public sectors;  

Ensure continuity of support to adult education in order to strengthen the Government 

• 
capacity;  

Strategies  should  be  elaborated  in  order  to  ensure  smooth  exit  from  sectors  of 

• 
intervention, were relevant;  

Improve  quality  of  investments  and  quality  of  indicators  in  the  on-going  SBS 

• 
programmes.  

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013. 

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1312_docs_en.htm 

Evaluation of the EU cooperation with Honduras 

1) ABB activities: 19 09 

2) Timing: April 2012, (period covered by the report: 2002-2009) 

3) Budget: Disbursements over the evaluation period: EUR 234 million. 

4) Background, scope and focus: 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 
cooperation  strategy  and  support  in  the  country  and  to  draw  out  key  lessons  to  help 
improve current and future EC strategies and programmes. 

During  the  evaluation  period  (2002-2009),  cooperation  with  Honduras  focused  on 
combating  poverty  and  raising  standards  of  living  of  the  poorest  groups  in  the  country. 
This  included  support  for  food  security,  local  development  and  management  of  natural 
resources,  education  and  health  sectors  along  with  the  support  to  the  decentralisation 
reform. More recently the EU has engaged in supporting the Legal and Judicial sector in 
the country; however it is still too early to assess its results. 

5) Summary of findings and conclusions of the evaluation: 

The  EU  strategy  had  positive  results,  though  implementation  was  difficult.  The  EU 
programme  was  aligned  with  the  Government’s  policies,  yet  these  changed  over  time, 
undermining  implementation  and  the  sustainability  of  some  actions.  Co-operation  was 
suspended  following  the  political  crisis  of  2009.  In  order  to  ensure  results  and  to  offset 
weaknesses in capacity, continued support through technical assistance, and strengthening 

EN 168 

Error! Unknown document property name.  EN 

 
 
 
 
the involvement of civil society is necessary. 

Key conclusions:  

 •  The  EU  concentrated  its  cooperation  in  sectors  addressing  well  the  needs  of  the 
population. Support was aligned with Government's priorities, though these have changed 
over  time,  which  impacted  implementation  and  results.  Three  sectors,  natural  resources 
management,  food  security  and  decentralisation,  were  strongly  intertwined  but  synergies 
were not fully exploited. 

• 
Positive  results  were  obtained  in  reconstruction,  rehabilitation,  food  security  and 
vocational training subsectors, where the EC proved to have an added value. Furthermore 
the EC has been very active and effective in promoting donor co-ordination and division of 
labour.  

• 
The aid modalities proved to be adequate with the exception of sector budget support 
towards decentralisation, mainly because of the discontinuity of the country's commitment 
to public sector reforms.  

• 
The EC has provided international technical assistance and has involved Civil society 
in order to ensure cooperation results and to compensate for weak capacity in government 
institutions, affected by high turnover within its staff.  

The  linkages  between  the  relief  operation  (following  hurricane  Mitch)  and 

• 
rehabilitation were not very effective and delays occurred.  

• 
The  overall  approach  recently  designed  to  strengthen  public  security  is  positively 
assessed,  except  for  the  insufficient  involvement  of  Civil  society  organisations  in  the 
process, to promote reforms and to monitor implementation.  

Key recommendations: 

• 
The difficult context requires the EU to do a thorough analysis in order to identify the 
most  appropriate  sectors  of  intervention  and  aid  delivery  modalities.  Analysis  should 
include  political  consensus  and  commitment  with  relation  to  specific  sectoral  policies, 
national capacity and risk analysis.  

•  Despite difficulties in mapping the complexity (and conflicting interests) among Civil 
society organisations it is important to involve them in monitoring the reform  processes, 
while ensuring this does not hamper the dialogue with the Government.  

Continuity  and  sustainability  of  achievements,  particularly  in  food  security  and 

• 
vocational training sectors, should be improved.  

The EC has been successful in a dialogue on natural resources management at micro 

• 
level but needs to address the topic at macro level, on a government-wide basis.  

Support  provided  to  food  security  and  natural  resources  management  impact  the 

• 
decentralised level of government: the EC should fully exploit these synergies.  

• 
International  technical  assistance  should  be  maintained  if  BS  operations  are  to  be 
continued.  It  is  also  suggested  to  embed  and  maintain  a  system  of  incentives  within  a 
conditionality framework.  

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 

EN 169 

Error! Unknown document property name.  EN 

 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013.      

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1304_docs_en.htm 

Evaluation of the EU cooperation with Nepal 

1) ABB activities: 19 10  

2) Timing: March 2012; (Period covered by the report : 2002-2010) 

3) Budget: Disbursements over the evaluation period : EUR 58 million    

4) Background, scope and focus: 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 
cooperation strategy and support in the country over the 2002-2010 period and to draw out 
key lessons to help improve current and future EC strategies and programmes. 

Political instability has been a major issue throughout the whole period. Important delays 
in  the  formulation  and  implementation  of  EC-funded  interventions  reduced  significantly 
their  effectiveness.  The  complex  set-up  with  two  EU  Delegations  involved  in  the 
formulation and implementation of the EC strategy in Nepal increased transaction costs. 

Thematic  focus:  (i)  Education,  (ii)  Rural  Development,  (iii)  Renewable  Energy  and 
Environment,  (iv)  Peace  Building  and  Consolidation  of  Democracy,  and  (v)  Trade 
facilitation and integration in the international economy.   

5) Summary of findings and conclusions of the evaluation  

The  Commission  helped  to  keep  the  poverty  reduction  strategy  on  track  but  the  direct 
contribution to increased production, employment and incomes remained limited. The mix 
of  instruments  and  aid  modalities  was  good.  Synergies  with  other  Development  Partners 
and  donor  co-ordination  were  rather  good  overall  but  limited  in  others.  The  role  of  the 
Commission  in  sector  policy  dialogue,  notably  in  the  areas  of  trade,  rule  of  law  and 
democracy was modest, in line with the size of its support. 

Key conclusions:  

By  staying  engaged  with  the  Government  of  Nepal  in  fighting  poverty,  the  EC  has 
• 
helped  to  keep  the  poverty  reduction  strategy  on  track  in  achieving  the  Millennium 
Development  Goals.  The  EC  interventions  improved  conditions  for  further  poverty 
reduction.  However,  the  direct  contribution  to  increased  production,  employment  and 
incomes remained rather limited as few and only small interventions were implemented in 
productive sectors. 

The EC has actively participated in the policy dialogue with the Government of Nepal 
• 
in line with the size of its support (the EC is a relatively small Development Partner), but 
more could have been done in several sectors of cooperation (e.g. trade, the rule of law and 

EN 170 

Error! Unknown document property name.  EN 

 
 
 
 
democracy).  

• 
The EC portfolio is characterised by a suitable mix of instruments and aid modalities. 
Despite the relatively weak institutional environment and the unstable political context, the 
move towards budget support was justified and has led to some tangible results. However, 
exclusively relying on budget support in Nepal is insufficient to improve the quality of the 
overall  environment  in  which  development  takes  place,  as  well  as  the  quality  of  sector 
management.  

•  Although  important  improvements  were  made  in  recent  years  in  the  process  of 
establishing  a  fully-fledged  Delegation  in  Nepal,  human  resources  in  the  two  EU 
Delegations involved were constrained during most of the period. In addition, during most 
of  the  evaluation  period,  the  ownership  and  accountability  of  the  Government  of  Nepal 
was low due to the unstable and rapidly changing political conditions.  

Key recommendations:  

• 
The  EC  should  focus  on  key  areas  where  it  is  recognised  as  having  extensive 
experience  and  where  it  can  provide  added  value.  Potential  key  areas  are:  trade, 
environment  and  disaster  preparedness.  Moreover,  the  EC  needs  to  leverage  its  assets, 
which consist in large part of the relationships of trust that have been built up with national 
partners (including Government of Nepal) and other Development Partners. It also needs 
to build on and strengthen the professional expertise which it has in-country in the areas of 
democracy  (elections),  human  rights,  public  financial  management,  peace  and  stability, 
and education.  

The EC should increase the support to productive sectors, particularly in rural areas 

• 
so as to strengthen the economic impact of the EC interventions.  

The  EC  needs  to  engage  more  confidently  in  policy  dialogue  with  national 

• 
stakeholders, including both the Government of Nepal and Non State Actors.  

Synergies between the various EC aid modalities and financing instruments as well as 
• 
the  role  of  Non  State  Actors  in  the  implementation  of  the  strategy  should  be  further 
strengthened.  

The EC, and particularly the EUD, should align its own capacity with the objectives 
• 
set  in  its  strategy,  and  continue  to  consolidate  them.  As  soon  as  the  political  conditions 
improve and Nepal benefits from a stable and fully endorsed and accountable government, 
the  EC  should  also  give  a  greater  role  to  national  stakeholders  in  the  design  and 
implementation of its co-operation strategy. The conclusions and recommendations of this 
country level evaluation will be taken into account in the current programming exercise to 
be  finalised  in  2014.  In  this  regards,  some  information  on  the  contribution  of  the 
evaluation to the design of the next country programme may be available in 2013.  

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1302_docs_en.htm 

EN 171 

Error! Unknown document property name.  EN 

 
 
Thematic evaluation of EC support to Decentralisation processes 

1) ABB activities: 19 11 

2) Timing: February 2012; (Period covered by the report: 2000-2009) 

3) Budget: Disbursements over the evaluation period : EUR 640 million (only direct 
activities)     

4)  Background,  scope  and  focus:  European  Commission  support  to  decentralisation 
processes  over  the  period  2000-2009,  worldwide,  including  both  direct  and  indirect 
activities (indirect e.g. support to sectors at a decentralised level). If including both direct 
and  indirect  support,  during  this  period  budget  allocations  rose  from  nil  to  EUR  100 
million  per  year,  74%  being  implemented  in  Africa.  The  EU  has  developed  a  Policy 
framework  over  this  period,  responding  to  a  demand  from  partner  countries  and  the 
emergency of local authorities as actors in development.   

5)  Summary  of  findings  and  conclusions  of  the  evaluation:  EU  support  has  been  most 
effective in contexts at relative early stages of reform, in strengthening local governments' 
management  and  administrative  capacities,  in  promoting  greater  transparency  in  funds' 
allocation from central to local level and improving access to service delivery. Substantial 
challenges  remain,  such  as  addressing  central  government  through  effective  policy 
dialogue,  raising  participation  of  civil  society,  and  improving  the  quality  of  service 
delivery. Continued support is important in poor and fragile contexts. 

Key conclusions: 

• 
The  EU  has  developed  a  comprehensive  policy  framework  for  direct  and  indirect 
support  to  decentralisation  processes,  particularly  supportive  of  the  role  of  local 
governments  as  stakeholders  for  i)  improving  governance  and  ii)  service  delivery, 
responding to a demand from partners' countries reform agenda and the emergency of local 
authorities as actors in development.   

Substantial  challenges  remain  and  continued  support  is  of  utmost  relevance, 

• 
particularly in poor and fragile contexts.  

The  EC  has  focused  its  assistance  in  strengthening  local  governments'  management 
• 
and  administrative  capacities,  in  promoting  greater  transparency  in  governments  systems 
for allocation of funds from central to local level, in improving access to service delivery 
and, to a certain extent, in development of decentralisation policies.  

EC support has been most effective in countries with relative early stages of reform 

• 
and less in more mature decentralised contexts. 

The  EC  was  less  successful  in  addressing  the  central  governmental  level,  lacking 
• 
capacity  for  establishing  an  effective  policy  dialogue  for  comprehensive  institutional 
arrangements, for supporting legal reforms and human resources management.  

The EC has had little impact in upraising the participative profile of civil society and 

• 
the quality aspects of service delivery.  

• 

The  main  constraint  towards  stronger  EU  engagement  at  policy  level  has  been  the 

EN 172 

Error! Unknown document property name.  EN 

 
 
 
lack  of  dedicated  human  resources  both  in  HQ  and  delegations,  along  with  insufficient 
dissemination of documents and training events.  

• 
EC programming has been increasingly aligned with partners countries' contexts and 
demand,  namely  with  the  nationally  owned  development  strategies,  poverty  reduction 
plans and decentralisation reforms.  

• 
In terms of impact and effectiveness the use of different aid modalities has not proved 
to be a decisive factor; but efficiency has improved through increasing EC engagement in 
working  with  other  donors  through  joint  aid  modalities  and  the  introduction  of  new  aid 
modalities. The EC has been proactive and leader in organising donors' co-ordination. 

To  ensure  sustainability  of  interventions,  it  is  necessary  to  align  with  partners' 
• 
countries  owned  priorities.  All  interventions  have  to  be  anchored  to  the  political 
framework and to sectoral policies; the latter would also drastically increase coherence. 

Key recommendations:  

• 
The EU should exploit its potential added value and acknowledge decentralisation as 
part of Public sector reform. Addressing public reforms shall require the strengthening of 
an  informed  policy  dialogue  with  a  wide  range  of  stakeholder  (central  and  local),  while 
recognising the need to embed support into the broader political context and of developing 
capacity  to  provide  realistic  assessment  of  the  nationally  owned  strategies  and  context 
variables. 

EU  should  develop  clear  intervention  strategies  to  adapt  to  different  contexts  and 

• 
maturity of reforms. 

EU should give priority to strengthening partner countries' monitoring and evaluation 

• 
of the reforms;  

EU  should  build  upon  its  know-how  on  selected  aspect  of  local  fiscal  reforms  and 

• 
exploit it further. 

• 
EU  shall  develop  more  sectoral  capacity  in  terms  of  dedicated  Human  resources, 
develop operational guidelines, disseminate information and organise appropriate training. 
internal  knowledge 
Furthermore,  EU  should  strengthen 
management to ensure systemic internal sharing of results. 

internal  capacity  of 

its 

Uptake of the evaluation recommendations: 

Recommendations were on the overall agreed by EC services. Some concrete actions are 
already  underway, namely: 

•  A “Communication on Local authorities in development” is being prepared in order 
to  improve  the  policy  framework  of  local  governments’  role  towards  improved 
governance; 

• 
Efforts  are  consecrated  to  respond  to  the  recommendations  calling  for  improved 
sectoral  capacity  and  selected  aspects  of  reform;  specific  guidance  is  being  elaborated, 
particularly:  a)  Guidance  on  project  modality  to  support  decentralisation  and  Local 
governance; b) “Practical guidance on fiscal decentralisation”.  

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1300_docs_en.htm 

EN 173 

Error! Unknown document property name.  EN 

 
Thematic evaluation of the European Commission support to respect of Human 
Rights and Fundamental Freedoms (including solidarity with victims of 
repression) 

1) ABB activities: 19 04 

2) Timing: December 2011, (Period covered by the report : 2000-2010) 

3) Budget (annual): Disbursements over the evaluation period : EUR 4.036 million 

4) Background, scope and focus:  

The  evaluation  provides  an  independent  assessment  of  the  Commission’s  human  rights 
work in non-member countries. The evaluation concerns both the funds contracted by the 
European Commission over the period covered and the so-called ‘non-financial activities”, 
notably  the  political  and  policy  dialogues  that  are  central  to  the  EU  approach  to  human 
right promotion in third countries.  

This  study  assessed  how  the  EC  advanced  the  human  rights  agenda  in  different  political 
and institutional environments. It examined to what extent and how the EC managed to: (i) 
use  its  political  clout  to  leverage  change;  (ii)  strategically  combine  various  instruments; 
(iii) mobilise the various actors (states, civil society, regional organisations, UN); (iv) pro-
actively promote the mainstreaming of human rights; (v) foster the application of the 3Cs’ 
in  the  field  of  human  rights  and  (vi)  achieve  results  and  impact.  In  the  process,  it  took 
stock  of  the  dilemmas  encountered,  the  innovative  practices  employed  and  the  lessons 
learnt.   

5) Summary of findings and conclusions of the evaluation:  

In many countries, the EC has made relevant contributions to promoting the Human Rights 
agenda at various levels through the use of funding and non-funding instruments. Evidence 
of  results  and  positive  impacts  has  been  identified  in  relation  to  both  the  promotion  and 
protection of human rights. However, a deficit in the EC/EU political commitment towards 
implementing  an  effective  and  coherent  human  rights  policy  and  a  lack  of  related 
knowledge,  capacities  and  incentives  have  been  highlighted.  As  a  consequence,  the 
political status of human rights in the EC/EU external action should be upgraded so as to 
ensure coherent action and increase impact. 

Key conclusions: 

•  At a macro level, the sheer presence of the EU as a global player promoting a human 
rights  agenda  has  helped  to  protect  and  eventually  enlarge  the  space  to  address  human 
rights issues; 

In several settings (including highly restrictive environments) the EC has been able to 
• 
intelligently  mobilise  the  different  instruments  at  its  disposal  with  a  view  to  pushing  for 
legal changes or effective application of ratified conventions; 

•  EU  political  demarches  have  helped  to  prevent  a  deterioration  of  human  rights 
situation  (e.g.  when  contributing  to  halt  legislative  reforms  that  would  re-introduce  the 
death penalty); 

EN 174 

Error! Unknown document property name.  EN 

 
 
 
•  The  EC  support  to  human  rights  defenders  and  civil  society  organisations  has 
repeatedly been described as a ‘lifeline’ for the actors involved; 

•  Several  EC-supported  programmes  have  contributed  to  promoting  joint  action 
between state and non-state actors on human rights; 

•  EC support to justice sector reforms and the fight against impunity have contributed 
to improving the overall environment for the protection of human rights; 

•  Though  poorly  documented,  there  is  evidence  of  impact  achieved  with  capacity 
building initiatives (which consume a large share of EC aid for human rights). 

Key recommendations: 

The political status of human rights in the EC/EU external action should be upgraded so 
as  to  ensure  coherent  action  and  increase  impact.  Bold  decisions  are  needed  to  ensure 
that human rights can leave the ‘ghetto’ in which they have all too often been relegated. 
The  EC/EU  needs  to  clarify  ‘upstream’  how  much  weight  it  wants  to  give  to  human 
rights  and  how  it  can  better  reconcile  values  and  interests  in  this  critical  area  of  its 
external  action.  It  needs  to  build  stronger  bridges  between  human  rights  and  other 
domains of EU external action. 

•  Clarify the political agenda of the EU with regard to human rights and translate this in 
common implementation strategies; 

•  Develop a comprehensive strategy to localise human rights; 

•  Revitalize  the  political  dialogue  on  human  rights  by  clarifying  its  objectives  while 
ensuring an inclusive, iterative and result-oriented approach; 

•  Overcome the divide between human rights and development through smart forms of 
mainstreaming and direct support to human rights; 

•  Better use the added value of the EC to support systemic reforms that help realising 
rights; 

•  Deepen  the  strategic  engagement  with  citizens,  civil  society  political  actors  and 
regional organisations; 

•  Create  an  enabling  institutional  environment  for  effective  delivery  of  a  coherent 
EC/EU action on human rights. 

Uptake of the evaluation recommendations: 

The  Commission  has  accepted  all  7 

(one  overall  policy 
recommendation and 6 specific recommendations) formulated by the evaluators and there 
has been already a good follow up on them.      

recommendations 

Actions already taken in accordance with the recommendations: 

In accordance with the overall policy recommendation of the evaluation to upgrade the 
political  status  of  human  rights  in  the  EC/EU  external  action  so  as  to  ensure  coherent 
action  and  increased  impact  and  with  the  recommendation  to  create  an  enabling 
institutional  environment  for  effective  delivery  of  a  coherent  EC/EU  action  on  human 

EN 175 

Error! Unknown document property name.  EN 

 
 
 
rights an EU Strategic Framework for Human Rights and Democracy and a related Action 
Plan were endorsed by the European Council on 25th June 2012. Moreover, an EU Special 
Representative (EUSR) on Human Rights has been appointed on 26th July 2012 in order 
to  enhance  the  effectiveness  and  visibility  of  the  EU’s  HR  policy  and  ensure 
implementation of the Action Plan.  

In response to the recommendation to clarify the political agenda of the EU with regard 
to  human  rights  and  translate  it  in  common  implementation  strategies,  the  two  strategic 
documents provide benchmarks and objectives of a clear political agenda for human rights 
and democracy involving the Commission, EEAS and Member States. This is the first time 
that the European Union has a unified Strategic Framework for human rights with a wide-
ranging plan of action for its implementation. 

In  response  to  the  recommendation  to  overcome  the  divide  between  human  rights  and 
development  through  smart  forms  of  mainstreaming  and  direct  support  to  human  rights, 
for the next programming period the human rights dimension is integrated in EU supported 
programmes  related  to  economic  governance,  domestic  accountability  and  other  key 
sectors such as health and education.  

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2011/1298_docs_en.htm 

Evaluation of Visibility of EU external action 

1) ABB activities: 19 11  

2) Timing: June 2012, (Period covered by the report : 2005-2010) 

3) Budget (annual): Not applicable 

4) Background, scope and focus:  

This  evaluation  assesses  the  visibility  of  the  EU's  external  action  over  the  period  2005-
2010. The basis of the evaluation is the work carried out by the Commission largely prior 
to the reorganisation of the EU's external services, but the recommendations are set in the 
current Post-Lisbon context.  

The  evaluation  was  structured  in  different  phases  and  makes  good  use  of  evaluation 
questions,  judgment  criteria  and  indicators.    In  addition  to  documentary  evidence  and 
reports  consulted,  the  tools  used  are:  6  in-country  thematic  studies,  media  coverage 
analysis (over 1000 media reports were reviewed), web-based attitudinal surveys (around 
220 people responded), interviews (some 260) with three categories of actors in Brussels 
(NGOs  &  Think  Tanks,  EU  officials,  journalists),  analysis  of  Eurobarometer  reports,  3 
cases studies of specific events and partnerships. The criteria for assessment were the five 
DAC  criteria  (relevance,  effectiveness,  efficiency,  impact  and  sustainability)  along  with 
EU specific criteria of coherence and added value. 

Thematic focus: Visibility, Communication, Outreach   

5) Summary of findings and conclusions of the evaluation: 

EN 176 

Error! Unknown document property name.  EN 

 
 
 
Key conclusions:  

• 
The  image  of  EU  external  action  is  in  line  with  pre-Lisbon  official  priorities. 
Stakeholders are however often critical of the quality of the EU’s image and feel there is 
often a gap between rhetoric and reality. 

• 
Communication on EU external action lacks overall direction and leadership, notably  
since  the  post-Lisbon  Treaty  reorganisation.  While  there  are  some  efforts  to  re-establish 
internal coordination (ERIC) and on communication strategy there is still no strong sense 
of  central  leadership  or  of  a  new  overall  EU  external  action  communication  strategy 
emerging. 

•  Working  in  partnership  with  others  is  essential  but  there  is  a  trade-off  in  lower  EU 
visibility.  This  remains  a  source  of  tension  and  too  much  insistence  being  put  on  EU 
visibility  can  undermine  the  effectiveness  of  the  cooperation  and  the  sense  of  ownership 
felt by partners. 

• 

The image of EU external action varies both geographically and by constituency. 

While certain high profile features of the EU (e.g. the EUR) are known around the world, the 
image of the EU’s external action varies from place to place and among stakeholders. Some 
of the EU’s closest and best-informed observers are the most critical. 

• 
The  nature  of  the  EU  imposes  constraints  that  impact  on  its  visibility.  For  instance 
problems  of  internal  competition  for  visibility  between  EU  actors  continue,  a  lack  of 
cooperation  and  coordination  between  the  EU  and  the  Member  States  in  external  action 
remains  a  key  problem  and  policy  coherence  is  a  more  of  an  issue  for  informed  external 
observers than officials often seem to think. 

• 

The resources for promoting the visibility of EU external action have been adequate. 

On  the  other  hand  the  distribution  of  resources,  particularly  at  the  country  level,  does  raise 
some concerns, with EU Delegations seemingly have too few resources in some cases and not 
all implementing agencies using the funds they are given well. 

Key recommendations:  

Reaffirm,  renew  and  strengthen  the  established  visibility  strategy  of  EU  external 
• 
action  by  improving  its  quality  and,  particularly  by  communicating  more  on  results  and  by 
avoiding raising unrealistic expectations. 

Provide  stronger  central  direction  and  leadership  for  communication  work  on  EU 
• 
external  action.  A  single  overall  communication  strategy  for  EU  external  action  with 
complementary  sectoral  strategies  is  needed.  Ensure  regular  close  coordination  on 
communication  between  services,  establish  an  urgent  action  cooperation  mechanism  and 
coordinate EU external action communication messages with Member States. 

• 
Agree  that  ‘working  in partnership’  is  an  intrinsic  part  of  the  image  of  EU  external 
action  and  a  key  message  for  communication.  One  of  the  key  features  of  the  way  the  EU 
works in international affairs is its continuing effort to work in partnership with other actors. 
Build  the  external  action  communication  strategy  around  this  principle.  Accept  that  some 
reduction  of  visibility  is  a  by-product  of  such  partnerships  that  can  even  be  positive.  EU 
senior management should provide clear leadership on the balance to take. 

• 

The EU needs to manage sensitively the geographic and constituency variations in the 

EN 177 

Error! Unknown document property name.  EN 

 
visibility of EU external action. The EU’s world-wide network of Delegations is a major asset 
in  this  respect.  In  Europe,  communication  with  informed  and  critical  audiences  closest  to 
home needs to be tackled with particular care and attention. 

Pay special attention to the impact on visibility of the EU’s specific nature. All EU 
• 
external  action  staff  needs  to be  very  aware  of  the  importance  of  projecting  a single  EU 
image.  Promoting  a  well-coordinated  EU  image  with  Member  States  should  be  a  key 
objective  for  all  and  particularly  those  in  leadership  positions.  Close  attention  should  be 
paid  to  policy  coherence  both  in  practical  terms  and  in  terms  of  external  images  and 
messages. 

Review  the  distribution  of  resources  for  communication  work  particularly  at  the 
• 
country  level.  EU  Delegations  may  consider  examining  their  need  to  perhaps  have  more 
resources with adequate expertise.  The funding of communication and information work 
in  EUDs  should  also  be  reviewed  in  the  light  of  their  new  political  functions  under  the 
Lisbon Treaty. 

Uptake of the evaluation recommendations: 

The Commission has accepted all 6 key recommendations formulated by the evaluators. 

For instance, as to the recommendation on a strengthened visibility strategy, DG DEVCO 
has already produced an ambitious communication strategy in 2011. DG DEVCO, EEAS 
and FPI collaborate with the view to create synergy and to have a consistent approach in 
all aspects of cooperation. This lead to the dissemination of a "Communication Handbook 
for  EU  Delegations".  Collaboration  with  MS  is  strongly  encouraged  at  the  Delegation's 
level  and  Headquarters  foresee  increasing  collaboration  with  the  MS  development 
agencies communicators from 2013 onwards. 

As  to  resources,  for  EU  Delegations  with  relevant  management  capabilities  pooling 
projects'  budgets  could  be  one  way  to  improve  substantially  the  visibility  of  the  EU 
Development cooperation work. 

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1307_docs_en.htm 

EN 178 

Error! Unknown document property name.  EN 

 
 
 
ABB 20 – Trade  

Evaluation of the economic impact of the trade pillar  

of the EU-Chile association agreement 

1) ABB activities: 20 02 01 

2) Timing: March 2012, (period covered 2003- 2009) 

3) Budget (2010+2011): EUR DG Trade’s overall operational budget for the seven years of 
the present MFF is EUR 96.3 million at the level of commitments; or roughly EUR 13.8 
million per annum. 

4) Background, scope and focus: 

An  agreement  to  establish  an  association  between  the  European  Community  and  its 
member states and the Republic of Chile was signed in November 2002.  The trade pillar 
was implemented from the beginning of 2003, and the association agreement came fully 
into force in March 2005. 

The objectives of the study were to: 

1.  To identify and evaluate the actual effects and overall economic impact of the EU–
Chile FTA more than six years after its entry into force, distinguishing carefully between 
immediate  measurable  trade  effects,  and  to  identify,  at  least  at  a  preliminary  stage, 
consequential economic, social and environmental impacts of the FTA. 

2.  To  pave  the  way  for  future  systematic  ex  post  evaluation  of  the  EU's  FTAs  and  to 
identify  useful  lessons  that  may  be  learnt  with  respect  to  future  (ex  ante  and  ex  post) 
impact assessments of similar agreements. 

5) Summary of findings and conclusions of the evaluation:  

T  (1)  Trade  in  goods  The  EU’s  market  share  in  Chilean  imports  tended  to  decline 
following  the  agreement  –  but  this  was  a  period  in  which  EU  exports  were  seriously 
outpaced  by  exports  from  economically  more  dynamic  regions,  and  several  other 
agreements  were  being  phased  in  by  Chile.    Overall  the  agreement  is  likely  to  have 
prevented the EU’s market shares from falling substantially further. 

Econometric simulations suggest that the EU’s tariff cuts had a significant impact on the 
level of EU imports from Chile, compared to other suppliers, raising them by as much as a 
quarter.    Wines  and  fruits  are  the  sectors  that  benefited  most.    These  simulations  also 
suggest that the EU’s exports to Chile are at least two-thirds higher than they would be in a 
counterfactual scenario that does not include the EU-Chile agreement.  

Computable  general  equilibrium  (CGE)  modelling  enables  the  most  heavily  impacted 
sectors to be identified.  On the Chilean side, the main winners include fruit growing, wine 
making, fisheries and fish processing.  On the EU side, the main winners are machinery, 
transport equipment and the chemicals industries.  At an aggregate level, the CGE model 
suggests that Chile’s exports to the EU are up by about 20%, and the EU’s exports to Chile 
by  more  than  60%,  when  compared  to  a  counterfactual  that  excludes  the  EU-Chile 
agreement. 

EN 179 

Error! Unknown document property name.  EN 

 
 
The  results  suggest  that  the  EU-Chile  FTA  has  protected  the  EU’s  exporters  from  the 
significant  diversions  effects  that  might  have  resulting  from  Chile’s  competing  trade 
agreements.  In other words, the enforcement of several of Chile’s other FTAs – including 
with the USA – could have significantly crowded out European exporters from the Chilean 
market, and to a lesser extent diverted Chilean exporters from the EU market. 

(2)  Trade  in  services  and  foreign  direct  investment  (FDI)  Analysis  must  begin  with  the 
EU’s high level of commitments under GATS, and a relatively low level for Chile.  The 
additional commitments made under the FTA result in a very high level of commitments 
by the EU (outside health-related, cultural, recreational, and transport services).  For Chile, 
commitments remain limited in several sectors (construction, educational, environmental, 
and health-related), and intermediate in communications and financial services.  However, 
there are significant new commitments in distribution, recreational, tourism, business, and 
transport services. 

Analysing  the  extent  to  which  liberalisation  is  correlated  to  trade  performance  across 
service sectors shows that the EU’s services exports tended to increase more, following the 
FTA,  in  those  sectors  where  commitments  brought  a  higher  level  of  liberalisation.  
However, the link is not necessarily causal. Taking into account trends that are not specific 
to the EU-Chile bilateral relationship suggests that the correlation may also reflect the fact 
that priority was given in the FTA to commitments in service sectors that were expected to 
have the highest trade potential. 

An  analysis  of  Chile’s  services  exports  to  the  EU  fails  to  find  any  relationship  with  the 
EU’s commitments in the FTA.  However, taking into account exogenous trends suggests 
that  those  commitments  may  nevertheless  have  actually  spurred  Chilean  exports.  
Certainly, Chile’s services exports to the EU performed relatively well in several sectors 
where the EU’s FTA commitments significantly improved on those made in the GATS. 

For  both  EU  FDI  in  Chile  and  Chilean  FDI  in  Europe,  investment  decisions  seem  to  be 
strictly  related  to  business  opportunities;  and  recent  changes  probably  have  more  to  do 
with  the  economic  dynamism  of  investing  countries  than  with  the  FTA.    The  agreement 
did  not  significantly  change  the  legal  framework  or  the  guarantees  offered  to  European 
investors.  Its benefit lies in the additional security derived from having consolidated the 
conditions  for  investment  prevailing  before  the  agreement.    Such  benefits  are  not 
negligible, especially in the long run. 

(3)  Institutional  and  regulatory  aspects  The  agreement  provisions  regarding  sanitary  and 
phyto-sanitary  (SPS)  measures,  technical  standards,  and  wines  and  spirits  required 
substantial  adjustment  on  the  Chilean  side,  given  the  stringency  of  EU  requirements  in 
these  areas.    The  institutional  structure  set  up  by  the  agreement  was  implemented 
effectively, apparently to the satisfaction of both parties.  Many technical issues have been 
raised, but all have been solved through dialogue. 

The practical consequences are far-reaching. The improvement of SPS standards in Chile’s 
agriculture  is  widely  recognized,  and  is  at  least  partly  attributable  to  the  FTA.    The 
requirements – initially viewed in Chile as constraints – are now seen as having spurred an 
upgrade in production practices, and has having eased access to a wider range of foreign 
markets.    The  same  is  true  for  the  disciplines  imposed  by  the  FTA  on  the  use  of 
geographical designations.  

(4)  Environmental  impact  Focus  on  particular  sectors  using  input/output  analysis  shows 

EN 180 

Error! Unknown document property name.  EN 

 
that the growth in economic activity stimulated by the FTA, in particular through exports 
to  the  EU,  has  had  only  a  very  marginal  impact  on  Chilean  energy  consumption  and 
greenhouse  gas  emissions.  The  impact  on  water  quality  is  harder  to  quantify,  given  the 
very uneven distribution of the resource in Chile. 

The FTA has had a limited impact on the growth of the sectors that pollute most. The EU’s 
imports from Chile that generate the most pollution are from the ores and minerals sectors.  
Most  of  these  already  entered  the  EU  duty  free,  and  the  agreement  liberalised  trade  in 
rather marginal products. 

Chilean  exports  to  the  EU  of  wood  products,  salmon  products,  fish,  wine,  fruits  and 
vegetables  grew  significantly,  at  least  partly  as  a  result  of  the  agreement.    Given  their 
production  externalities,  these  are  the  sectors  where  the  FTA  is  likely  to  have  had  some 
negative environmental impact. 

The agreement has played a significant role in the growth of Chile’s agricultural exports.  
Input  decomposition  analysis  shows  that  the  FTA  has  contributed  to  a  limited  by 
noticeable increase in the use of fertilizers.  Larger exports of fruit and wine may have also 
led to an increase in pesticide use; but the EU’s strict standards – and the fact that the shift 
towards agricultural exports tends to reduce the intensity of pesticide use – suggest that the 
FTA has had little overall impact in this area. 

Statistics  show  a  significant  increase  in  trade  in  the  environmental  goods  that  were 
liberalised as part of the FTA.  This  supports the idea –  widespread among stakeholders 
consulted – that the FTA has contributed to the adoption of greener technology and stricter 
domestic environmental standards.  However, the agreement has also led to an increase in 
goods transported by air – especially seafood products and to a more limited extent, fruit. 

(5) Social impacts The adjustment brought about by the agreement is mainly characterised, 
for  medium-  and  lower-skilled  workers,  by  an  increased  demand  in  several  agricultural 
sectors and in fisheries; together with reduced demand in some industrial sectors, notably 
machinery.  Nevertheless, the adjustments are small when compared to the rapid structural 
change that the Chilean economy has undergone since entry into force of the FTA. 

In  agriculture,  the  export  increase  spurred  by  the  agreement  is  substantial,  and  it  is  far 
from being neutral in terms of the size and type of farms involved.  Small and subsistence 
farmers  are  less  likely  to  have  benefited  from  the  new  opportunities,  while  large, 
consolidated farms will have been able to reap the full benefit.   

This may be reflected in increasing inequality within the agricultural sector, but also by an 
increase in average agricultural incomes, resulting in lower inequality between agriculture 
and  other  sectors.  The  analysis  suggests  that  although  small  farmers  may  have  lost  out 
relative to larger, more commercial farm businesses, they will still have benefited overall 
from the changes brought about by the FTA, when the consequences for both incomes and 
consumption prices are taken into account. 

In  the  fruit,  wine,  aquaculture  and  mollusc  sectors,  the  FTA  has  led  to  higher  incomes, 
with visible positive consequences (canning, salmon industry, wine).  The agreement has 
also contributed to a significant increase in the employment of women in agriculture, even 
though the jobs are often seasonal. 

6) Availability of the report on Europa: 

EN 181 

Error! Unknown document property name.  EN 

 
http://trade.ec.europa.eu/doclib/docs/2012/august/tradoc_149881.pdf 

Evaluation of the EU’s trade defence instruments (TDIs) 

1) ABB activities: 20 02 01 

2) Timing: February 2012, (period covered 2005-2010) 

3) Budget (annual): DG Trade’s overall operational budget for the seven years of the present 
MFF is EUR 96.3 million at the level of commitments; or roughly EUR 13.8 million per 
annum. 

4) Background, scope and focus: 

The  rules-based  international  trading  system  provides  remedies  against  unfair  trade 
practices.  Anti-dumping  (AD)  measures  may  be  imposed  in  case  of  goods  imported  and 
sold at less than fair market value; and anti-subsidy (AS) measures in the case of imported 
goods that have benefited from subsidies provided by foreign governments; provided that 
the dumped or subsidised imports cause or threaten to cause injury to domestic industry in 
the importing country.  

The evaluation examined the EU’s procedures for, and use of, these two TDIs.  The EU’s 
third TDI – safeguard measures – was not included as part of the project.  The objectives 
of the study were to provide: 

1)  A concise description of the EU’s TDIs and of its current practice in this area; 

2)  A balanced economic analysis of the fundamental arguments for and against the use 
of  TDI,  and  of  their  use  in  the  context  of  the  current  international  legal  framework  and 
economic realities; 

3)  An  evaluation  of  the  performance,  methods,  utilisation  and  effectiveness  of  the 
present TDI scheme in achieving its trade policy objectives; 

4)  An evaluation of the effectiveness of the existing and potential policy decisions of the 
EU  in  comparison  with  selected  peer  countries:  Australia,  Canada,  China,  India,  New 
Zealand, South Africa, United States; 

5)  An  examination  of  the  basic  AD  and  AS  regulations  in  light  of  the  administrative 
practice  of 
the 
institutions, 
recommendations of the WTO Dispute Settlement Body (DSB). 

judgments  of  European  courts  and 

the  EU 

the 

5) Summary of findings and conclusions of the evaluation:  

The study’s main conclusions are:  

(1)  The  economic  analysis  confirmed  that  the  stated  rationale  for  EU  TDI  –  countering 
unfair trading practices and market-distorting subsidies – cannot be sustained on the basis 
of the actual pattern of use. Nonetheless, the analysis identified a number of considerations 
that  greatly  mitigate  the  perceived  negative  economic  effects  of  TDI.  In  fact,  given  the 
main  de  facto  purpose  that  TDI  serves,  the  study  argues  that  its  use  has  been  welfare 
improving for the EU.  

EN 182 

Error! Unknown document property name.  EN 

 
 
 
(2)  Moreover,  the  study  makes  clear  that  the  actual  (international)  construction  of  trade 
defence  (TD)  law  is  inappropriate  for  its  de  facto  role,  resulting  in  lack  of  clarity  for 
trading  firms  and  opening  the  system  up  to  the  possibility  of  protectionist  abuse.  The 
economic  analysis  further  confirmed  that  the  construction  of  trade  defence  law  is 
increasingly  out  of  step  with  the  modern  trading  environment.  The  report  therefore 
recommends using the flexibility within the system to apply it in ways that minimise the 
risks  of  adverse  outcomes  for  the  EU,  while  working  at  the  WTO  towards  a  system  of 
trade remedies constructed in a way better suited to the actual tasks they perform. 

(3)  The  analysis  of  EU  court  cases  and  WTO  disputes  showed  that  the  number  of 
litigations  related  to  the  EU’s  implementation  of  TDI  is  low.  It  also  confirmed  a  high 
degree of compliance, as evidenced by a high share of claims against the EU institutions 
rejected  by  the  EU  courts.  EU  TDIs  are  also  rarely  challenged  before  the  WTO  DSB. 
However, the EU’s success rate at the WTO DSB is lower, with about half of the claims 
being  granted.  Nevertheless,  the  number  of  amendments  to  the  two  basic  Regulations 
required in response to either EU court or WTO DSB decisions is limited. 

(4) The international comparison highlighted that EU TD practice stands out in a number 
of ways. Notably, regular application of the public interest test and frequent reduction of 
duties through application of the lesser duty rule distinguish EU practice from that in most 
other  countries.  This  leaves  the  EU  better  placed  than  the  other  countries  reviewed  in 
dealing with the evolution of globalised production systems and the heterogeneity of firms 
in international trade.  

(5)  The  study  analyses  the  options  for  improving  accessibility  to  AD/AS  instruments  by 
amending  initiation  policies  –  such  as  the  right  for  workers  to  file  complaints  or  greater 
use  of  ex  officio  initiation  of  investigations  –  and  ensuring  cooperation.  Furthermore,  it 
highlights two areas where EU TD practice may benefit from drawing on peer countries’ 
experience:  namely  the  transparency,  and  the  duration,  of  investigations.  Finally,  the 
international  comparison  showed  that  the  EU  TDI  system  is  not  more  prone  to 
politicisation than most other countries’ systems. 

(6) The evaluation of EU TD practice validated most of the methodologies and procedures. 
The overall finding therefore is that EU TD policies and practice are sound. A number of 
specific  issues  were  identified.  With  regard  to  substantive  issues,  these  relate  to  certain 
aspects  in  the  dumping  and  subsidy  analysis,  injury  and  causation  analysis,  the  Union 
interest test and the calculation of the non-injurious price. Concerning procedural issues, 
the study found that, in general, the Commission’s practice with regard to the participation 
of interested parties in proceedings is more inclusive than what would be required by the 
two  basic  regulations.  Transparency  of  proceedings  has  improved,  but  could  still  be 
improved  further.  Also,  the  relatively  long  period  required  from  injury  to  measures  was 
noted. Last but not least, the Hearing Officer’s role was evaluated positively. 

(7) The evaluation team developed recommendations in respect of the following aspects of 
EU TDI policy: Mission statement and intervention logic; initiation of investigations and 
treatment  of  non-cooperation;  shortening  the  process  for  provisional  determinations; 
changes  in  the  Union  interest  test;  duration  of  measures  and  dynamic  impacts  of  TDI; 
consolidated  statement  of  administrative  practice;  access  to  confidential  information.  In 
addition,  the  evaluation  team  recommended  a  number  of  changes  to  the  two  basic 
regulations,  as  well  as  suggesting  changes  to  a  number  of  specific  issues  related  to  the 

EN 183 

Error! Unknown document property name.  EN 

 
EU’s implementation of TDI. 

Limitations  in  the  design  or  execution  of  the  evaluation  in  meeting  the  study  aims  and 
objectives. 

The  lack  of  officially  adopted  intervention  logic  for  TDI  posed  a  challenge  for  the 
formulation  of  the  evaluation questions.  Furthermore,  confidentiality  issues  prevented  an 
in-depth analysis of many methodologies and tools which the Commission applies in AD 
and AS investigations. Also, for two of the peer countries, China and India, only limited 
primary  information  could  be  obtained.  Therefore,  for  these  two  countries  secondary 
sources have been used extensively. Finally, due to time and resource constraints, a firm 
level analysis of the effects of TDI could not be undertaken. 

6) Availability of the report on Europa: 

http://trade.ec.europa.eu/doclib/docs/2012/march/tradoc_149236.pdf 

Evaluation of the Market Access Partnership (MAP) 

1) ABB activities: 20 02 01 

2) Timing:  October 2012, (period covered 2007-2012) 

3) Budget (annual) : DG Trade’s overall operational budget for the seven years of the 
present MFF is EUR 96.3 million at the level of commitments; or roughly EUR 13.8 
million per annum. 

4) Background, scope and focus:  

The EU's Market Access Strategy (MAS) was launched in 1996 to enforce multilateral and 
bilateral trade deals and open third country markets to EU exports. The strategy was based 
on two pillars: 1) providing EU businesses with information on market access conditions 
and  2)  creating  a  framework  within  which  the  barriers  to  trade  in  goods  and  services, 
intellectual property and investment are tackled. 

The  first  pillar  took  the  form  of  the  Market  Access  Database  (MADB),  which  provides 
information on tariff and non-tariff issues per country for companies wishing to export to 
third countries. It was the subject of a separate evaluation in 2011. 

The  second  pillar  emerged  as  the  Market  Access  Partnership  (MAP),  which  was 
introduced in 2007 to remove barriers for European businesses exporting and investing in 
third country markets. It consists of a partnership between the European Commission, the 
member states and EU businesses.  

The  purpose  of  this  evaluation  was  to  assess  the  relevance,  impact,  effectiveness  and 
efficiency of the MAP, as well as to suggest improvements in its structure and scope. 

5) Summary of findings and conclusions of the evaluation  

The MAP was found to be relevant because it focuses on market access for EU businesses 
in accordance with the objectives of Trade, Growth and World Affairs. The MAP has been 
working  on  key  barriers  in  a  systematic  way  while  also  devoting  time  to  address  many 

EN 184 

Error! Unknown document property name.  EN 

 
 
 
other barriers in third countries. 

The  MAP  was  considered  to  be  generally  efficient  when  looking  to  its  input-output 
relation.  However,  it  could  be  made  more  efficient  by  organising  the  Market  Access 
Advisory  Committee  (MAAC)  meetings  in  a  different  way.  It  also  needs  better  internal 
communication,  so  as  to  be  able  to  confront  systematic  industry  lobbying  to  tackle  and 
address barriers that have been previously analysed, and found to offer very little prospect 
of success. 

Regarding  the  effectiveness  of  the  MAP,  many  potential  barriers  could  be  addressed 
before  they  become  entrenched  in  law.  On  the  other  hand,  most  of  the  existing  market 
access  barriers  are  unsolved.  The  effectiveness  of  the  MAP  seems  to  stem  not  just  from 
joint action, but maybe more importantly from a common voice of the three MAP partners. 

Having achieved limited results, the quantifiable impact of the MAP is low as well, although 
some barriers have been avoided via “early warning” follow-up actions. Given the long time 
it  requires  to  tackle  complex  market  access  barriers,  it  may  also  be  too  early  to  assess  the 
overall impact of the MAP. 

Positive aspects of the MAP include: 

• 

• 

• 

It creates a clear window for market access issues; 

It works on actual problems and finding solutions; 

It creates more leverage for smaller member states; and 

• 
It  allows  companies  to  take  a  European  route  rather  than  a  bilateral  one,  which  is 
often  a  preferred  option  because  this  creates  more  anonymity  (the  bilateral  route  means  a 
complaint can be more easily traced back to a specific company). 

Problems for the MAP include:  

The  larger  member  states  sometimes  prefer  to  act  unilaterally,  which  limits  the 

• 
possibilities for joint and coordinated action; 

The  MAP  may  become  a  victim  of  its  own  success  because  the  number  of  cases  is 

• 
increasing but resources within the EC and the MSs are limited; 

• 
It has a limited set of instruments if the third countries are not willing to co-operate to 
solve the problem: not all market issues can be challenged in the WTO, and dispute settlement 
can only be used in a limited number of cases.  

Recommendations 

Relevance: 

a. 
Stakeholders  do  not  support  Commission  proposal  to  prioritise  fewer  barriers  or 
fewer  countries.  In  a  period  of  economic  crisis,  the  work  of  the  MAP  is  increasingly 
important to them. 

b. 

Clarify the difference in the treatment of key and non-key barriers in the MAP. 

Raise  awareness  on  the  function  of  the  MAP  with  the  local  market  access  teams 

c. 
(MATs). 

EN 185 

Error! Unknown document property name.  EN 

 
 
Efficiency: 

Improve  the  distribution  of  analytical  work  by  delegating  more  of  that  work  to  the 

d. 
member states. 

If business brings up a barrier, it should also provide relevant supporting evidence on 

e. 
what action can be taken. 

f.  Ensure good preparation of information sent to participants in advance of the MAAC 
meetings. 

g.  More  direct  communication  lines  at  a  technical  level  between  the  market  access 
working groups (MAWGs) in Brussels and the MAT working groups in key markets. 

Effectiveness: 

h.  Work together with like-minded non-EU actors for the MAP. 

i.  Consider  technical  assistance  in  combination  with  gentle  persuasion  to  overcome 
barriers. 

j.  Make  processes  and  procedures  clearer  and  more  transparent  without  losing 
flexibility. 

k.  Create better institutional memory by documenting more of what has been done. 

Share experiences from different MATs and MAWGs on the approach, process and 

l. 
procedures for dealing with market access barriers. 

6) Availability of the report on Europa 

http://trade.ec.europa.eu/doclib/docs/2013/april/tradoc_150847.pdf 

Trade sustainability impact assessment (Trade SIA) in support of 

EU negotiations for deep and comprehensive free trade agreements (DCFTAs) 

with Georgia and the Republic of Moldova 

1) ABB activities: 20 02 01 

2) Timing: October 2012 

3) Budget (annual): DG Trade’s overall operational budget for the seven years of the present 
MFF is EUR 96.3 million at the level of commitments; or roughly EUR 13.8 million per 
annum. 

4) Background, scope and focus:  

is  negotiating  association  agreements  (AAs)  –  comprising  deep  and 
The  EU 
comprehensive  free  trade  agreements  (DCFTAs)  with  both  Georgia  and  the  Republic  of 
Moldova. 

Trade  SIAs  support  those  negotiations  by  analysing  how  the  trade  and  trade-related 
provisions  in  the  proposed  DCFTAs  will  affect  economic,  social,  and  environmental 
developments in the EU and (respectively) Georgia and Moldova. 

EN 186 

Error! Unknown document property name.  EN 

 
 
 
A trade SIA combines quantitative and qualitative research with an extensive programme 
of stakeholder consultation. 

5) Summary of findings and conclusions of the evaluation:  

(1)  Economic  effects  for  the  EU  of  an  agreement  with  either  Georgia  or  Moldova  are 
found  to  be  negligible.    For  Georgia,  the  study  forecasts  a  sizeable  long  run  increase  in 
national income (4.3% growth in GDP) as a result of a deal with the EU.  For Moldova, a 
deal with the EU might be worth as much as 5.4% in terms of long run increase in GDP. 

Georgian exports are forecast to increase by 12% following an agreement, and imports by 
7.5%  In the case of Moldova, exports might increase by 16% and imports by 8%. 

Average  wages  in  Georgia  are  projected  to  increase  by  3.6%  in  the  long  run,  while  the 
consumer price index (CPI) might fall by about 0.6%.  For Moldova, the equivalent figures 
are 4.8% for average wages, and a fall of about 1.3% for the CPI. 

The  effects  on  third  countries  are  very  limited.    In  the  case  of  an  EU-Georgia  DCFTA, 
Russia and Azerbaijan are expected to benefit to a small extent.  An EU-Moldova DCFTA 
might deliver some modest benefits for Russia and Ukraine.  

Following  an  EU-Georgia  agreement,  the  study  suggests  that  chemicals,  rubber  and 
plastic, other machinery and equipment, and primary metals will be the sectors that enjoy 
the  largest  increases  in  output.    The  largest  falls  in  output  are  expected  to  arise  in  the 
livestock  and  meat  products,  other  processed  food,  electronics  and  computers,  and  other 
manufacturing sectors.  The largest gains in output following an EU-Moldova agreement 
are  expected  in  the  sugar,  textiles  and  clothing,  and  air  transport  sectors.  Livestock  and 
meat products, motor vehicles, electronics and computing, and other manufacturing are the 
sectors where the largest falls in output may occur. 

(2)  Social  impacts:  for  both  Georgia  and  Moldova,  the  main  drivers  of  overall  social 
impact are expect to derive from effects related to employment and wages.   

The  study  suggests  that  following  an  agreement  with  the  EU,  about  4%  of  the  Georgian 
work  force  might  need  to  change  employment  sector,  and  about  5%  in  the  case  of 
Moldova.    The  extent  of  this  reallocation  would  be  slightly  higher  for  lower-skilled 
workers than for the higher-skilled.  

As  stated,  wages  are  expected  to  increase  and  consumer  prices  to  fall.  Nevertheless  the 
poorer strata in the population appear to benefit less from a DCFTA than those on average 
or  above-average  incomes.  This  is  because  the  poor  are  affected  disproportionately  by 
changes in food prices; and food prices are expected to rise slightly even within an overall 
fall in the CPI. 

There is thus a risk of a rise in certain poverty indicators, especially for those at the bottom 
of the income distribution. A DCFTA may also exacerbate inequality (and slightly worsen 
relevant indicators such as relative poverty or the Gini coefficient). 

The overall effect of a DCFTA (for either partner country) on labour rights is expected to 
be positive. 

A DCFTA with the EU might indirectly have a positive effect on equality to the extent that 
increasing  living  standards  begin  to  support  gradual  changes  in  societal  preferences  on 

EN 187 

Error! Unknown document property name.  EN 

 
 
equality  issues.    More  immediately  however,  the  sectoral  reallocations  that  will  occur 
following  a  DCFTA  may  disproportionately  affect  the  weakest  groups  in  the  workforce:  
those  with  low  educational  and  skill  levels.    The  extent  of  inequality  currently  observed 
might therefore be exacerbated. 

(3) Environmental effects expected from a DCFTA – for either Georgia or Moldova – are 
mixed. This is against a backdrop (baseline situation) for Georgia which includes problems 
related to air and water pollution, improper waste management and use of land, as well as 
marine and coastal contamination.  Moldova faces several environmental problems related 
to  air,  water,  and  soil  pollution.  Inadequate  urban  waste  management,  unsustainable 
agricultural  practices,  and  improper  forest  management  lead  to  significant  land 
degradation  and  damage  to  biodiversity.    The  country’s  worsening  social  situation  and 
high incidence of poverty contribute to environmentally damaging activities such as illegal 
logging, use of obsolete technologies, poor energy efficiency and underinvestment in basic 
infrastructure such as water, sanitation, roads, and energy. 

The  estimated  effects  in  Georgia  of  a  DCFTA,  in  terms  of  CO2  emissions  and  land  use 
intensity, are expected to be very small.  Air emissions of other pollutants are expected to 
rise by about 3% in the long run.  The equivalent estimates for Moldova are very similar. 

For  either  country,  a  DCFTA  is  expected  to  have  a  weak  but  positive  effect  on  the 
environment by encouraging more effective implementation of international environmental 
agreements.    This  should  gradually  contribute  to  solving  some  of  the  outstanding 
environmental  challenges.    This  mechanism  may  prove  important  in  greening  economic 
growth  in  the  two  countries,  and  in  limiting  the  environmental  burden  from  the  boost  to 
economic development provided by a DCFTA. 

6) Availability of the report on Europa: 

http://trade.ec.europa.eu/doclib/docs/2012/november/tradoc_150105.pdf 

EN 188 

Error! Unknown document property name.  EN 

 
 
ABB 21 – Development and Relations with African, Caribbean and Pacific 
(ACP) States  

Evaluation of the Commission’ cooperation with the Council of Europe (focused 
on the joint programmes) 

1) ABB activities: 21 07  

2) Timing: September 2012, (Period covered by the report : 2000 – 2010) 

3) Budget (annual): EUR 90 million disbursed 

4) Background, scope and focus:  

This  evaluation  covers  the  cooperation  of  the  European  Union  (EU)  with  the  Council  of 
Europe  and  mainly  the  implementation  of  the  joint-programmes  over  the  period  2000-
2010. The objectives of this study are to provide an independent assessment and to identify 
key lessons learned in order to improve future cooperation.  

5) Summary of findings and conclusions of the evaluation:  

Political  and  strategic  relations  between  the  EU  and  CoE  experienced  a  marked 
improvement over the evaluation period. 

The CoE comparative advantage is in its expertise particularly in the sectors of rule of law, 
human rights and democracy, its legal and moral authority and its unique combination of 
roles whereas its comparative disadvantage is its weakness as an implementing agency.  

It  is  to  be  noted  that  the  recent  large-scale  institutional  reform  of  the  CoE  has  been 
addressing many of the implementation weaknesses identified notably in this evaluation. 

Key conclusions: 

Political  and  strategic  relations  between  the  EU  and  CoE  experienced  a  market 

• 
improvement over the evaluation period. 

Inside  the  EU,  DG  DEVCO  and  the  EEAS  (European  External  Action  Service) 

• 
appear to have differing perspectives on the role of the CoE as a partner. 

The  CoE’s  comparative  disadvantage  is  widely  recognised  to  be  its  weakness  as  an 

• 
implementation agency. 

• 

Impacts of EC-CoE cooperation at country level are mixed. 

• 
There  is  need  to  provide  a  stronger  foundation  for  the  evident  presupposition  that 
training, capacity building, and awareness rising are the keys to institutional development 
and change. 

Key recommendations: 

Encourage the CoE to adopt institution-wide approach to delivering assistance in line 

• 
with international best practices. 

• 

Strengthen strategic joint priority setting at country level. 

Insist  on  and  support  stronger  project-cycle  management  in  EU  co-financed 

• 
programme. 

EN 189 

Error! Unknown document property name.  EN 

 
 
Ensure  stability,  predictability  and  reasonable  flexibility  of  the  funding  for  the  EC-

• 
CoE joint programme. 

Strengthen the foundation for capacity building activities and establish their links to 

• 
results. 

Uptake of the evaluation recommendations: 

Out  of  9  key  recommendations  formulated  by  the  evaluators,  the  Commission  and  the 
EEAS have fully accepted 7 recommendations, the other 2 being only partially accepted.     

Example: 

Encourage the CoE to adopt an institution-wide approach to the delivery of assistance in 
line with international best practices. 

Answer:  The  Evaluation  Unit  in  DG  DEVCO  has  already  commenced  a  fruitful 
cooperation  with  the  evaluation  group  at  the  CoE  HQs.  If  the  newly  created  Audit  and 
Oversight  Directorate  in  the  CoE  coordinates  with  DEVCO  on  the  development  of  a 
proper evaluation methodology, the newly created Directorate for Programming should be 
the  counterpart  to  identify  appropriate  guidelines  for  project  management  to  then  be 
applied by all the CoE services. 

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1311_docs_en.htm 

Evaluation of the EU cooperation with the Caribbean Region 

1) ABB activities: 21 06:  

2) Timing: August 2012 (Period covered by the report: 2003-2010) 

3) Budget : Disbursements over the evaluation period : EUR108.9 million 

4) Background, scope and focus: 

The evaluation was carried out to provide meaningful feedback to decision makers and to 
the general public on the results achieved by the Commission’s co-operation strategies and 
their implementation for the period 2003-2010 at regional level of the Caribbean Region. It 
covered the Regional Strategy Papers for the European Development Fund (EDF9; 2003-
2007  and  EDF10  2008-2013),  but  also  took  into  consideration  activities  during  the 
evaluation  period  that  had  been  financed  with  resources  from  EDF8.  Thematically,  the 
focus  was  on  regional  economic  and  political  integration  but  also  private  sector 
development,  human  resources  development,  natural  disaster  prevention  and  mitigation, 
and reduction of drug related crime. 

5) Summary of findings and conclusions of the evaluation:  

The EU strategy was relevant and made good use of the increased resources channelled to 
the region, critical for the EPA negotiations and a model for donor coordination. However 
results in other sectors could be improved by more suitable aid modalities, better selection 

EN 190 

Error! Unknown document property name.  EN 

 
 
 
 
of implementing partners, more effective complementarity between regional and national 
programmes,  and  strengthening  of  the  EU  Delegation  in  charge  of  the  regional 
programme. 

Key conclusions :  

• 
The  regional  strategies  of  the  EDF9  and  EDF10  were  highly  responsive  to  the 
priorities  of  CARIFORUM,  thanks  to  an  increased  political  dialogue.  Nevertheless, 
regarding  regional  integration,  the  effectiveness  of  the  international  efforts  to  enhance  it 
has been significantly reduced by the lack of a donor coordination framework. 

• 
Important efficiency issues are the limited capacity at the national level to implement 
policies  decided  at  the  regional  level  and  the  weakness  of  the  CARICOM  Secretariat 
(CCS). 

• 
Effectiveness  and  impact  of  the  regional  EU  support  is  limited  by  the  fact  that 
regional integration has lost some political momentum notably in the CARICOM member 
states. 

• 
In most sectors reviewed during the evaluation, the results of EU interventions were 
modest, with the positive exception of the support to the EPA negotiations and the growth 
of some exports like rum for the entire region and bananas for DR. 

Key recommendations :  

The  EU  should  continue  to  support  the  regional  strategies  by  the  development  and 
• 
adoption of a comprehensive and efficient donor coordination system that was due to be 
presented  by  CCS  already  in  June  2011.  This  will  help  design  an  EDF  11  regional 
programme more focused and therefore easier to implement. 

• 
In application of new programming guidelines, complementarity should be enhanced 
by drafting the Regional strategy before the National Strategies and by making it a frame 
of reference for the bilateral Cooperation of EU Member States with the region. 

• 
Continue  support  to  the  completion  of  both  the  Caribbean  Single  Market  and  the 
Caribbean  Single  Economy,  while  also  addressing  the  main  obstacles  to  competitiveness 
and investment in each country. 

• 
The EDF should allocate more funds to the competitiveness issues of the Caribbean 
region  notably  through  support  to  productive  development  policies,  business  climate 
reforms,  clustering  initiatives  and  small  and  medium  size  enterprise  development. 
Regarding EPA implementation, the EDF 10 programme should include indicators. 

The  conclusions  and  recommendations  of  this  regional  evaluation  will  be  taken  into 
account in the programming process which will start in 2013 and be finalised in 2014.  

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1309_docs_en.htm 

Evaluation of the EU cooperation with the Republic of Congo 

1) ABB activities: 21 06  

EN 191 

Error! Unknown document property name.  EN 

 
 
 
2) Timing: March 2012, (period covered by the report : 2000 – 2009) 

3) Budget : Disbursements over the evaluation period: EUR160 million 

4) Background, scope and focus  

Le rapport présente les résultats de l'évaluation de la stratégie de l'Union européenne  avec 
la  République  du  Congo  pour  la  période  2000-2009.  Les  principaux  objectifs  poursuivis 
sont de fournir aux services de l’UE et à un public plus large une évaluation indépendante 
et globale des relations passées et présentes de la coopération avec la RC et d’identifier les 
leçons clefs afin d’améliorer les stratégies et les programmes actuels et futurs de l’UE dans 
ce  pays.  Thematic  focus:  transport  infrastructure,  institutional  development,  macro-
economic sector; thematic budget lines: natural resource management, post-conflict; water 
and energy facilities; business climate. 

5) Summary of findings and conclusions of the evaluation  

La stratégie de coopération de l'UE avec la République du Congo au cours de la période 
d’évaluation  est  jugée  pertinente.  Elle  a  montré  une  forte  adaptation  à  l’évolution  du 
contexte  de  la  RC,  notamment  le  passage  d’une  situation  de  post-conflit  à  une 
problématique  de  croissance  durable  et  moins  dépendante  du  seul  secteur  pétrolier. 
L’impact  des  interventions  financées  a  néanmoins  été  fortement  hypothéqué  par  les 
faiblesses  structurelles  et  le  dysfonctionnent  qui  caractérisent  la  conduite  de  l’action 
publique en RC. Ces difficultés ont été accentuées du fait du faible poids que représente 
l’aide  dans  les  agrégats  macroéconomiques  de  la  RC.  En  ce  qui  concerne  la  mise  en 
œuvre,  la  mobilisation  d’une  gamme  diversifiée  d’instruments  et  l’intervention  dans  un 
large  éventail  de  domaines  ont  favorisé  l’adaptation  de  la  stratégie  de  coopération  à 
l’évolution du contexte du pays. 

Principales conclusions: 

•  La  pertinence  de  la  stratégie  de  coopération  UE-RC  est  élevée  au  cours  de  la  période 
d’évaluation.  Elle  présente  une  forte  adaptation  à  l’évolution  du  contexte  de  la  RC, 
notamment  le  passage  d’une  situation  de  post-conflit  à  une  problématique  de  croissance 
durable  et  moins  dépendante  du  seul  secteur  pétrolier.  Néanmoins,  les  impacts  des 
interventions  financées  par  l’UE  ont  été  fortement  hypothéqués  par  les  faiblesses 
structurelles qui caractérisent la conduite de l’action publique en RC.  

•  La priorité donnée à la gouvernance de l’action publique est très pertinente, mais n’a pas 
encore produit tous les effets escomptés, en particulier en matière de finances publiques et 
dans les secteurs des transports, de la gestion des ressources naturelles et de la santé. Dans 
ces  domaines,  les  appuis  de  l’UE  ont  abouti  à  des  réformes  importantes  (adoption  de 
textes, création d’institutions). En revanche, dans chacun de ces domaines, il apparaît des 
carences  importantes  dans  l’application  concrète  de  ces  réformes,  ce  qui  hypothèque 
lourdement la viabilité de celles-ci.  

•  Au cours de la période d’évaluation, la coopération UE – RC a été marquée par une 
utilisation  judicieuse  des  instruments  disponibles,  avec  en  particulier  une  bonne 
complémentarité  entre  les  actions  financées  par  le  FED  et  celles  relevant  des  lignes 
budgétaires.  

Principales recommandations: 

• 

Il  est  suggéré  de  structurer  la  future  stratégie  autour  des  2  axes  suivants  :  La 

EN 192 

Error! Unknown document property name.  EN 

 
Gouvernance  de  l’action  publique  dans  le  domaine  économique  et  la    Consolidation  de 
l’Etat de Droit. En plus, dans le hors concentration : poursuivre  la reconstruction sociale 
dans les zones qui ont été les plus touchées par les conflits civils; poursuivre des actions de 
renforcement des acteurs  non étatiques; appuyer de façon ciblée  le secteur de  la santé  et 
soutenir les démarches de capitalisation des actions entreprises. 
• 

Pour ce qui est des 4 principaux secteurs de coopération: 

Finances publiques: mettre l’accent sur la transparence des recettes et l’efficacité de 

- 
la chaîne des dépenses. 

Transports:  considérer  l’amélioration  de  la  gouvernance  comme  un  préalable  au 

- 
redéploiement d’appuis financiers plus conséquents. 

-  Gestion des ressources naturelles: préserver les acquis et poursuivre les appuis à une 
meilleure gouvernance des ressources forestières et de la biodiversité. 

-  Acteurs  non  étatiques:  poursuivre  les  actions  de  renforcement  des  capacités  en  les 
inscrivant dans un cadre stratégique. 

Sélectionner  les  procédures  et  les  processus  afin  d’améliorer  l’efficacité  de  l’aide 

• 
(cofinancements, études sectorielles préalables, sélection des assistants techniques, …). 

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013. 

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1303_docs_en.htm 

Evaluation of the EU cooperation with Djibouti 

1) ABB activities: 21 06  

2) Timing: June 2012, (Period covered by the report : 1996-2012) 

3) Budget: Disbursements over the evaluation period: EUR 60.7 million. 

4) Background, scope and focus: 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 
cooperation  strategy  and  support  in  the  country  and  to  draw  out  key  lessons  to  help 
improve current and future EC strategies and programmes. 

The  EC’s  cooperation  programme  with  Djibouti  was  problematic  throughout  the  1996-
2010 period. The strategic objectives established between the government and the EC for 
the  cooperation  programme  have  not  been  achieved  to  the  level  at  which  they  were 
initially planned. The thematic focus covered road and rail infrastructure, macroeconomic 
support, access to water and sanitation, objectives related to decentralization and the NSAs 

EN 193 

Error! Unknown document property name.  EN 

 
 
 
 
 
and development of the transportation industry cluster in the country. 

5) Summary of findings and conclusions of the evaluation:  

Due  to  a  lack  of  prioritisation  in  the  government's  policies  focussing  on  a  sectorial 
approach for national or regional development was impossible. Cooperation was limited to 
a  series  of  interventions  not  embedded  in  a  strategic  vision.  The  EC  did  not  address 
governance or the strengthening of civil society, nor establish itself as a strategic partner of 
government  and  civil  society.  The  recent  establishment  of  an  EU  Delegation  in  Djibouti 
may improve cooperation. 

Key conclusions:  

Bien  que  les  interventions  de  la  CE  aient  généralement  répondu  aux  besoins 
• 
économiques et sociaux du pays, le gouvernement de Djibouti n’a pas établi des priorités 
pouvant  être  directement  appuyées.  Dès  lors,  le  niveau  d’alignement  aux  stratégies 
nationales est un concept non pertinent dans le cas de cette coopération. 

• 
Les concepts et stratégies qui ont été explicités dans les Documents de Stratégie- Pays 
(DSP) ont été réduits à une série d’interventions simples et linéaires, sans référence à une 
approche sectorielle de développement national ou régional. 

• 
La  CE  n’a  pas  réussi  à  se  faire  valoir  comme  partenaire  stratégique  vis-à-vis  du 
Gouvernement de Djibouti ou de la Société Civile. Le dialogue politique a été relativement 
inefficace  et  les  programmes  d’aide  n’ont  pas  été  développés  pour  renforcer  ni  la 
gouvernance ni la société civile. 

• 
Bien que la stabilité post-conflit ait été un des piliers de la stratégie de coopération de 
la CE avec Djibouti, la CE n’a pas eu d’impact perceptible sur la stabilité post-conflit du 
pays. 

Key recommandations:  

•  Dans le but de réduire la pauvreté et de créer des emplois permanents, collaborer avec 
le  Gouvernement  de  Djibouti,  le  secteur  privé,  la  société  civile  et  les  autres  bailleurs  de 
fonds  pour  développer  une  stratégie  de  développement  économique,  industriel  et 
commercial pour le pays. 

• 
S’assurer  que  toute  la  planification  et  la  programmation  de  la  coopération  de  la  CE 
avec Djibouti soient faites selon une approche sectorielle ou programmatique, en intégrant 
le 
tous 
développement d’un secteur ciblé. 

l’atteinte  des  objectifs  stratégiques  pour 

les  éléments  nécessaires  pour 

• 
Etant  donné  que  le  développement  futur  de  Djibouti  est  directement  lié  à  son  rôle 
géostratégique  dans  la  «  corne  de  l’Afrique  »  et  que  ses  stratégies  de  développement 
dépendent  des  relations  politiques  entre  plusieurs  autres  pays  de  la  région,  la CE  devrait 
s’assurer que ses stratégies reflètent cette vision « régionale ». 

•  Appuyer les instances djiboutiennes afin de créer une stratégie de développement des 
Acteurs  Non  Etatiques  qui  permettrait  à  la  société  civile  et  aux  diverses  associations  de 
devenir des partenaires proactifs dans le développement du pays. 

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 

EN 194 

Error! Unknown document property name.  EN 

 
 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013. 

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1306_docs_en.htm 

Evaluation of the EU cooperation with Ethiopia 

1) ABB activities: 21 06  

2) Timing: January 2012, (period covered by the report : 2004-2008) 

3) Budget: Disbursements over the evaluation period: EUR 1.259 million 

4) Background, scope and focus : 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 
cooperation  strategy  and  support  in  the  country  and  to  draw  out  key  lessons  to  help 
improve current and future EC strategies and programmes. 

The  evaluation  covered  the  period  of  economic  recovery  which  followed  the  war  with 
Eritrea and it also included the 2005 post-election violence and the subsequent cooling of 
Ethiopia-donor  relations.  The  evaluation  focuses  on  the  EU‘s  assistance  during 
implementation  of  the  CSP  and  9th  EDF  NIP  (2002-2007)  (including  a  variety  of 
instruments), and on the choices made for planning aid disbursements under the 10th EDF 
(2008-2013). 

The  thematic  focus  covered  i)  education,  health  and  the  related  social  accountability 
component, ii) transport infrastructure, iii) food security and agriculture, iv) Civil Society 
empowerment and v) conflict prevention.   

5) Summary of findings and conclusions of the evaluation : 

EU support was relevant to the country's needs. The alignment with Government's policies 
and  programmes  through  budget  support  was  particularly  effective  in  fighting  poverty, 
strengthening ownership and ensuring sustainability. However cooperation had less impact 
on democratic governance and related capacity building. Support to food security was too 
dominant  in  comparison  to  agricultural  development.  Programmes  were  too  ambitious, 
ignoring  the  limitations  of  specific  instruments  and  particularly  staff  capacity  of  the 
Delegation. 

Main conclusions:  

• 
In general, by aligning most of its support on government policies and programmes 
through  various  types  of  budget  support  (global,  sectoral,  targeted),  the  EU  was 
particularly effective in fighting poverty. This approach contributed also to reinforcing an 
already strong degree of ownership, thus ensuring sustainability.  

• 
But those gains were not equally matched in areas such as democratic governance and 
decentralisation  which  are  fundamental  to  the  long-term  development  prospects  of 
Ethiopia.  

• 

The  performance  evaluation  framework  agreed  with  the  government  for  the  bulk  of 

EN 195 

Error! Unknown document property name.  EN 

 
 
 
EU budget support was too aligned on five-year sectoral programming documents to be a 
true incentive for policy reform or even significant operational restructuring.  

• 
Local  government  capacity  shortcomings  remained  out  of  EU  reach,  either  directly 
(PSCAP)  or  indirectly  (PBS,  Road  SPSP,  PSNP).  They  impeded  the  effectiveness  and 
impact of EU-supported programmes, particularly in emerging regions.  

The effectiveness and inclusiveness of policy dialogue have been limited even in EU 

• 
focal sectors.  

The  balance  between  EU  support  to  food  security  and  agricultural  development, 

• 
although improved, still predominantly leans towards food security.  

EU-sponsored  innovative  approaches  in  PBS  and  PSNP  implementation  (essentially 
• 
social  accountability  and  CSOs  involvement)  were  adopted  by  the  government,  even 
though it provided only a limited financial contribution.  

Main recommendations: 

Prescribe  the  policy  dialogue  organisation  best  suited  to  EU  objectives  and 

• 
instruments.  

•  Modify  the  allocation  structure  within  sectors  to  strengthen  EU  advocacy  capacity 
and involvement in regional integration and sectoral institutional reforms.  

•  Adjust the strategic framework and the content of policy dialogue to overcome limits 
currently faced in promoting institutional reform and democratic governance.  

• 
Reconsider  the  overall  balance  in  EU  financial  support  between  food  security  and 
agricultural development by focusing progressively on the graduation process and on the 
subsequent phases of households’ accumulation of agricultural assets.  

• 
Prepare  for  the  resumption  of  GBS  while  reducing  its  lack  of  predictability  by 
defining  a  fixed  tranche  that  is  both  more  robust  in  the  face  of  politically-induced 
uncertainties and also more closely linked to development outcomes. The conclusions and 
recommendations of this country level evaluation will be taken into account in the current 
programming  exercise  to  be  finalised  in  2014.  In  this  regards,  some  information  on  the 
contribution  of  the  evaluation  to  the  design  of  the  next  country  programme  may  be 
available in 2013.  

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1301_docs_en.htm 

Evaluation of the EU cooperation with Jamaica 

1) ABB activities: 21 06 

2) Timing: September 2012, (period covered by the report: 2002-2009) 

3) Budget: Disbursements over the evaluation period: EUR 467million 

4) Background, scope and focus: 

The  evaluation  aims  at  providing  an  overall  independent  assessment  of  the  entire  EC 

EN 196 

Error! Unknown document property name.  EN 

 
 
 
cooperation  strategy  and  support  in  the  country  and  to  draw  out  key  lessons  to  help 
improve current and future EC strategies and programmes. 

The  evaluation  covers  co-operation  strategies  and  implementation  including  a  mix  of  all 
activities  and  modalities  during  the  period  2002-2009,  although  it  has  also  taken  into 
account of subsequent developments. It covers two EDF periods, the 9th EDF (2002-2007) 
and  the  10th  EDF  (2008-2013).  In  addition,  it  covers  support  to  the  sugar  and  banana 
sectors, mainly provided through instruments outside the EDF. 

Thematic  focus:  Governance,  Mix  of  instruments,  Macro-economic  support,  Support  to 
civil  society  organisations,  Transport,  Security  and  Justice,  Trade,  Private  sector 
development.   

5) Summary of findings and conclusions of the evaluation:  

The EU strategy in Jamaica was positive, notably with the more recent interventions. EU 
support  has  proved  more  effective  focussed  on  high  priority  needs  and  using  Budget 
Support.  Support  for  the  Security  and  Justice  sector  was  appropriate  and  the  evaluation 
recommends drawing on additional sources of expertise and instruments, e.g. those used in 
EU  Neighbourhood  countries.  The  use  of  process  indicators  for  budget  support  worked 
well,  and  helped  to  catalyse  actions  by  Government  stakeholders.  The  Delegation  used 
Technical Assistance effectively to support budget support operations. 

Key conclusions:  

• 
The  effectiveness  of  EU  support  has  improved  over  the  evaluation  period  as  a 
consequence of focusing on high priority needs. The mid-term evaluation of the 9th EDF 
flagged  up  that  security  and  justice  were  the  highest  priority  of  the  population,  with 
support  to  the  transport  and  private  sector  development  focal  sectors  being  replaced  by 
support to the security and justice sector.  

• 
The efficiency and effectiveness of support provided by the EU has increased due to 
the  use  of  budget  support  as  the  preferred  aid  modality.    Where  project  modalities  have 
been used they were typically subject to significant delays, and these often related to the 
recruitment  of  technical  assistance  and  oversight  mechanisms.  Early  budget  support 
operations suffered from delays but these have subsequently been reduced due to improved 
programme  design  and  focus,  the  use  of  clearly  and  broadly  owned  indicators,  together 
with provision of technical assistance to assist with preparation of budget support payment 
dossiers. However performance varies: sector budget support for transport did not fulfil its 
objectives.  

•  Macro-economic  and  PMF  risks  remain  considerable  in  fragile  circumstances.  As  a 
small, open economy with a high debt overhang and no access to HIPC-type debt relief, 
Jamaica  remains  vulnerable  to  adverse  international  and  domestic  events.  The  EU  and 
other  3  development  partners  mitigate  against  risks  through  effective  monitoring  with  a 
matrix with key milestones.  

The  impact  and  longer  term  sustainability  of  EU  support  is  difficult  to  quantify.  At 
• 
the macroeconomic level growth rates remain low at less than 1%. EU support has helped 
to  promote  macro-economic  stability  and  adjustment  to  new  circumstances  during  an 
extremely challenging period.  

Key recommendations: 

EN 197 

Error! Unknown document property name.  EN 

 
The broad approach in terms of focal sectors and aid instruments is appropriate and 
• 
should be continued. They are consistent with the Paris Declaration objectives, and the use 
of budget support is helping to create fiscal space for the Government. The mix of General 
Budget  Support  and  Sector  Budget  Support  works  well,  and  helps  to  achieve  a  positive 
policy dialogue in key areas.  

• 
The  EU  should  refine  its  instruments  to  facilitate  working  more  effectively  in 
complex, process-orientated areas such as 1) macro-economic management and 2) security 
and justice. With their formal three year structure and onerous process of riders to modify 
agreements, the EC's budget support instruments are not flexible enough. This has been a 
problem  in  Jamaica  in  terms  of  GBS,  where  the  decision  to  move  to  accrual  accounting 
was  reversed,  and  SBS,  where  security  and  justice  sector  needs  have  evolved  during 
programme implementation. Considering the examples of the WP and the IDB, a solution 
could  be  to  maintain  the  existing  setup  with  multiannual  policy  matrices  but  consider 
greater  flexibility  in  adapting  them  when  changes  in  context  and/or  IDP  coordination  so 
require.  

• 
Support to the security and justice sectors through the SSRP should continue and be 
consolidated. Future planning should strengthen synergies between JSIF, MNS, CSJP and 
SDC  by  detailing  expected  responsibilities,  expectations  and  expected  outputs  added  by 
each  organisation  in  the  design  of  the  programme.  Identify  realistic  specific  targets  and 
revise  conditions  that  are  no  longer  priorities  for  the  national  strategy  and/or  cannot  be 
fulfilled during the implementation period of the financing agreement for SRRP. Maintain 
the  focus  of  the  intervention  on  the  key  reform  areas  while  applying  some  degree  of 
flexibility  on  the  specific  wording  of  the  conditions.  The  European  Instrument  for 
Democracy and Human Rights (EIDHR) instrument has considerable value in Jamaica and 
could  be  expanded.  The  scope  could  include  governance  issues  such  as  for  example  the 
monitoring  of  social  spending,  anticorruption  and  transparency,  etc.  These  interventions 
would have a high degree of complementarity with budget support programmes and other 
projects.  

• 
Strengthen  EUD  capability  in  specialist  areas,  in  particular  related  to  security  and 
justice.  Consideration  should  be  given  to  using  existing  EU  tools  such  as  Twinning, 
Sigma, Taiex, which are used in the ENP countries where the EU has a longer cooperation 
history in the field of security and justice. 

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013. 

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2013/1314_docs_en.htm 

EN 198 

Error! Unknown document property name.  EN 

 
 
Evaluation of the EU Co-operation with Zambia 

1) ABB activities: 21 06 

2) Timing: September 2012, (Period covered by the report: 2001 – 2010) 

3) Budget: Disbursements over the evaluation period : EUR 475 million     

4) Background, scope and focus: 

This  evaluation  assesses  the  European  Commission's  support  to  the  Zambia  during  the 
period 2001-2010.   

The  EU  interventions  were  focused  on  economic  transition  through  infrastructure  (road 
sector)  development,  regional  integration,  Public  Finance  Management  and  support  for 
private sector development. 

The  purpose  of  the  evaluation  was  to  provide  an  overall  independent  assessment  of  the 
EU's  past  and  current  cooperation  relations  with  Zambia  and  to  identify  key  lessons  in 
order to improve the current and future strategies. 

5) Summary of findings and conclusions of the evaluation:  

The EU support to Zambia from 2001 to 2010 has been found well-focused on the needs 
of  the  population,  especially  in  the  first  part  half  of  the  evaluation  period.  The  EDF10 
response strategy underestimated the speed of evolution of the macro-economic situation 
in Zambia. The EU contribution in dialogue on public financial management issues was a 
driver of reforms in this area. However,  the administrative set-up for PFM demonstrated 
only limited regulatory autonomy.   

Key conclusions:  

• 
The  EU  response  strategy  has  been  in  line  with  the  policy  priorities  of  the 
Government  of  Zambia.  It  was  relatively  well-focused  on  the  needs  of  the  population  in 
the first half of the period under review (mostly covered by EDF9). The EDF10 response 
strategy,  though  comprehensive  and  well-articulated,  underestimated  the  speed  of 
evolution  of  the  emerging  macro-economic  situation  (Zambia  is  now  a  lower  middle 
income country).  

• 
The EU contribution in dialogue on public financial management issues was a driver 
of  reforms  in  this  area.  However,  the  whole  new  administrative  set-up,  both  PFM  and 
indeed  sector-wise,  remained  extremely  susceptible  to  external  political  pressure  and 
demonstrated only limited regulatory autonomy, if indeed any at all, as promoted by the 
EU. 

•  While  some  progress  has  been  made  in  the  strengthening  and  expansion  of  the 
Auditor General Office, there remains only weak domestic accountability by government 
for  the  budgetary  process,  all  the  more  so  given  the  absence  of  concrete  involvement  of 
Parliament and CSOs during the first three years of MDG-Contract implementation. 

Key recommendations:  

Engage in accelerated adjustment of EU strategy in response to the steady decrease in the 

EN 199 

Error! Unknown document property name.  EN 

 
 
 
leverage capacity of EU financial inputs. 

• 
Increase further the focus of EU cooperation and free up more leverage capacity by 
gradually moving away from the infrastructure sector; the EU exit strategy from the road 
sector  will  be  balanced  by  the  multiplicity  of  alternative  funding  opportunities  with  far 
lower transaction costs and policy reform requirements. 

• 
Reinforce  EU  support  for  drivers  of  sustainable  economic  growth  (Vision  2030) 
typical of lower-middle and middle income countries, such as trade liberalisation (through 
Regional Economic Communities followed by harmonization at national level) and SME 
development,  with  progressive  widening  to  include  social  security  system  development, 
higher education and research.  

• 
Pursue  a  comprehensive  and  integrated  capacity  strengthening  programme  with 
demand-driven,  customised  services  for  the  different  main  types  of  CSO,  appropriately 
clustered. 

Engage in supporting economic regulation, that is to say separating the government's 

• 
policymaking and regulatory roles by establishing independent regulatory mechanisms.  

The conclusions and recommendations of this country level evaluation will be taken into 
account in the current programming exercise to be finalised in 2014. In this regards, some 
information  on  the  contribution  of  the  evaluation  to  the  design  of  the  next  country 
programme may be available in 2013.    

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2013/1316_docs_en.htm 

Thematic global evaluation of the EC support to Agricultural Commodities in 
ACP Countries 

1) ABB activities: 21 08  

2) Timing: April 2012; (Period covered by the report: 2000 – 2009) 

3)  Budget  (annual):  Contracted  amount:  EUR  2.1billion  (Stabex,  EDF,  budget  lines  food
    security/sugar/bananas, etc.)   

4)  Background,  scope  and  focus:  This  evaluation  covers  EC  support  to  agricultural 
commodities in all ACP countries over the period 2000-2009 in the context of the policies 
and approaches laid down in two communications: Communication (2004)89: Agricultural 
Commodity  Chains,  Dependency  and  Poverty  –  A  proposal  for  an  EU  Action  Plan,  and 
Communication  (2004)87:  proposal  for  an  EU-Africa  Partnership  on  support  of  cotton 
sector  development  accompanied  by  the  Action  Plan  for  Cotton.  The  objective  is  to 
provide an overall independent assessment of the EC past and current cooperation support 
to agricultural commodities at a general level and to identify key lessons learnt in order to 
improve the current and future EC strategies and programmes.   

5) Summary of findings and conclusions of the evaluation: The communications were 
implemented  at  international  and  all  ACP  levels  and  through  sugar  and  bananas  budget 
lines  but  the  operations  conducted  locally  through  EDF  and  Stabex  resources  were 

EN 200 

Error! Unknown document property name.  EN 

 
 
 
unaffected by the communications. The most notable value added and the showcase of the 
communications is the achievement in establishing a functional EU-Africa Partnership on 
Cotton.  The  flexibility  of  the  Stabex  instrument  allowed  offering  a  flexible  response  to 
the  effects  on 
unforeseen  demands  for  support.  When  support  was 
competitiveness were significant. When punctual interventions were financed, it had rarely 
an impact on competitiveness but it improved the beneficiaries’ situation. 

important, 

Key conclusions: 

• 
The  Communications  were  implemented  at  international  and  all  ACP  levels  and 
through  the  sugar  and  banana  budget  lines.  However,  the  design  and  implementation  of 
support  to  agricultural  commodities  funded  by  the  EDF  and  Stabex  resources  remained 
unaffected by the Communications. 

The  most  notable  value  added  and  the  showcase  of  the  Communications  is  the 

• 
achievement in establishing a functional EU-Africa Partnership on Cotton.  

•  Whilst  the  rigidity  of  procedures  often  represented  a  constraint  for  projects,  the 
flexibility  of  the  Stabex  instrument  allowed  Delegations  to  offer  a  flexible  response  to 
unforeseen demands for support.  

• 
The  Commission’s  sector-wide  interventions  have  had  significant  effects  on 
competitiveness  when  support  was  important.  On  the  other  hand,  support  to  agricultural 
commodities 
improvement  of 
beneficiaries’ situation but this was often temporary, on a small scale and not capitalized 
upon; sector competitiveness was rarely improved.  

interventions  contributed 

through  punctual 

to  an 

• 
The  Communications  and  the  Commission’s  support  to  the  competitiveness  of 
agricultural  commodity  sectors  considered  exit  strategies  from  the  sector  only  for  the 
support to the banana and sugar producing countries.  

•  At  headquarter  level  efforts  are  made  to  ensure  coherence  between  different  EU 
policies  including  in  the  way  they  affect  particular  sectors.  However  timeliness  and 
administrative issues have at times hampered their effectiveness at field level. 

Key recommendations: 

Based on analyses of competitiveness and livelihood possibilities, decide whether to 

• 
support competitiveness or to exit from a sector and provide means to do so. 

Place the value chain approach in a comprehensive framework in line with an overall 

• 
agricultural support policy.  

Implement  the  current  Communication  by  focusing  available  resources  on  specific 

• 
sectors that have long term prospects.  

•  Deepen  focus  and  research  on  possibilities  to  prevent  the  impact  of  agricultural 
commodity price volatility on farmers, including the poorest.  

•  Allocate resources for the implementation of policy level Communications.  

•  Adapt  implementation  mechanisms  and  financing  modalities  to  capacities  and 
systems in place.   

Uptake of the evaluation recommendations: 

Out of 10 key recommendations formulated by the evaluators, the Commission has fully 

EN 201 

Error! Unknown document property name.  EN 

 
accepted  5  recommendations,  the  other 5  being  only partially  accepted.  Example  :  Build 
complementarities  and  synergies  within  EU  support  between  projects,  programmes  and 
non-financial support 

Answer:  The  mechanics  of  designing,  implementing  and  reporting  on  commodity 
programmes  should  be  those  mentioned  in  the  national  programmes,  aid  effectiveness 
principles,  internal  instructions  etc.  Improved  coherence  between  EU  programmes  in  a 
given country will be sought also through closer interaction with aid instruments such as 
budget support or join programming. 

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1305_docs_en.htm 

Final evaluation of the EU Food Facility 

1) ABB activities: 21 02 

2)  Timing:  Date  of  the  report:  December  2012;  (Period  covered  by  the  report:  2008
   - 2011) 

3)  Budget:  The  total  allocation  for  the  EU  Food  Facility  is  EUR  1000  million;  The  entire 

amount has been disbursed by 2011    

4)  Background,  scope  and  focus:  The  volatility  of  food  prices  and  agricultural  inputs  in 
2007  and  2008  put  numerous  developing  countries  and  their  populations  in  a  dramatic 
situation. The rising prices have resulted in riots, unrest and instability in several countries, 
jeopardising  the  achievements  of  years  of  political  development  and  peacekeeping 
investments. On 16 December 2008, the European Parliament and the Council adopted the 
Regulation, establishing a “facility for rapid response to soaring food prices in developing 
countries”  (this  became  the  European  Union  Food  Facility).The  evaluation  assesses  the 
European  Union  Food  Facility  as  an  instrument  and  the  European  Commission’s 
cooperation  activities  under  this  instrument  over  the  period  2008  to  2011.  The 
geographical  scope  includes  all  49  countries  where  Food  Facility  funded  activities  were 
undertaken.   

5) Summary of findings and conclusions of the evaluation: The EU, through its decision to 
support the coordinated international response to the Food Price Crisis, has brought food 
security and rural development at the forefront of its own development cooperation agenda 
and  of  the  international  development  agenda.  In  addition,  the  EU  has  gained  significant 
visibility through the implementation of the EU FF, thus strengthening its leading role in 
the international response. 

Key conclusions:  

The  Commission’s  political  decision  to  create  a  new  financing  instrument  as 
• 
articulated in the Regulation 1337/2008 was relevant, although the implementation period 
foreseen for the instrument was very short; 

EU Food Facility (EU FF) interventions were effective in mitigating the effects of the 
• 
Food Price Crisis, but activities were implemented at least one year after the peak in global 

EN 202 

Error! Unknown document property name.  EN 

 
 
 
food  prices  and  only  reached  a  limited  share  of  the  vulnerable  population  in  target 
countries; 

• 
The  EU  FF  instrument  had  a  high  flexibility  for  intervention  and  was  intended  to 
benefit a high number of partner countries. However, the dispersion of interventions in 49 
countries and 232 interventions reduced the impact at country level; 

• 
There  was  little  evidence  of  the  major  effects  of  the  EU  FF  on  food  prices  or  food 
security  beyond  direct  beneficiaries.  Positive  effects  have  been  limited  to  direct 
beneficiaries. 

Key recommendations:  

The  EU  should  consider  converting  the  EU  FF  into  a  permanent  “Stand-by” 
• 
instrument,  in  order  to  respond  rapidly  to  upcoming  and  sudden  Food  Price  Crisis,  and 
mitigate  impacts  on  food  insecurity  situations.  In  the  case  of  permanent,  recurrent,  or 
cyclical food insecurity situations, the instrument could mainly be used for mitigating the 
effects that have “deepened” existing food insecurity situations; 

• 
The  design  of  future  specific  instruments  should  be  more  focused,  so  that  expected 
results can be achieved in the initially foreseen time-frame and resources. In this respect 
and as advocated in the “Agenda for Change”, the EU should concentrate its support to the 
most affected countries and strategic beneficiaries to ensure resources are allocated where 
maximum impact can be achieved; 

• 
Prior  to  the  creation  of  any  instrument,  it  is  recommended  to  carry  out  a  problem 
analysis and adopt a single primary objective tackling specific issues and clearly defined 
beneficiaries; 

• 
The EU (under its programmed cooperation) should continue to play an active role in 
policy dialogue at the country level and help the governments of partner countries realise 
the importance and multi-sector dimension of food security.  

6) Availability of the report on Europa: 

http://ec.europa.eu/europeaid/what/development-policies/intervention-
areas/ruraldev/documents/euff-final_report-en.pdf 

Evaluation of the EU Support to the Health sector 

1) ABB activities: 21 05 

2) Timing: August 2012, (Period covered by the report : 2002-2010) 

3)  Budget:  During  the  evaluation  period,  the  EU  supported  the  health  sector  with  direct 
support  amounting  to  EUR  4.1  billion.  This  represents  6%  of  total  EU  support  to  all 
sectors during the period 

4) Background, scope and focus: The evaluation provides an independent assessment of the 
European  Commission’s  past  and  current  support  to  the  health  sector  by  looking  at  the 
relevance, efficiency, effectiveness, impact and sustainability of the EU support provided. 
It also assesses the coherence of EU health support with other European Union and donor 

EN 203 

Error! Unknown document property name.  EN 

 
 
 
policies and activities, as well as the specific EU added value within the health sector. The 
evaluation covers EU aid delivery over the period 2002 to 2010, including all geographical 
programmes (EDF, DCI, ENPI and predecessors) and thematic budget lines. It comprises 
all countries under the mandate of DG  DEVCO  and assesses every aid modality used in 
the health sector.   

5) Summary of findings and conclusions of the evaluation: The EU support to the health 
sector was found coherent with the EU Development policy and well-focused on poverty 
reduction.  However,  the  Commission  still  lacks  a  clearly  articulated  and  implemented 
global  strategy  for  health  co-operation.  Health  strategies  have  tended  to  focus  on  the 
present, not on the longer term. The interventions in the health sector have been generally 
effective. However, in several areas, such as health finance reform and human resources, 
results  have  been  found  to  be  small  compared  to  the  scope  of  the  challenges.  Key 
conclusions:  

EC co-operation in health was relevant to needs and coherent with EU development 
• 
policy. In general, the poverty focus of health cooperation was  well  maintained over the 
evaluation  period.  However,  the  Commission  still  lacks  a  clearly  articulated  and 
implemented global strategy for health co-operation with developing countries. 

•  Health  strategies  have  tended  to  focus  on  the  present,  not  on  the  longer  term  over 
which  health  sector  development  takes  place  (such  as  urbanisation  and  the  demographic 
and epidemiological transitions). 

• 
EU co-operation in the health sector has been generally effective. However, in a few 
areas, such as health finance reform and human resources, results have been found to be 
small compared to the scope of the challenges. A factor limiting effectiveness throughout 
has been the chronic shortage of technical expertise in EUDs. 

• 
EU  support,  especially  SBS,  has  led  to  improved  health  sector  policy  making 
capacities  and  improved  management  practices.  While  capacity  for  better  PFM  has  been 
increased,  the  ultimate  impact  on  health  sector  PFM  has  often  not  been  seen.  Not  only 
PFM  capacity,  but  also  increased  national  resource  allocations  to  the  health  sector  are 
needed. 

Through  support  to  research,  infectious  disease  control,  the  EU  significantly 

• 
contributed to the production of global and regional public goods for health. 

Key recommendations:  

• 
Consolidate  various  global  policy  statements  and  approaches  into  a  comprehensive 
health  co-operation  strategy  that  can  be  effectively  operationalized  at  the  field  level  in 
conformity with national sector development plans; 

In defining focus of support, take more carefully into account the shifting burden of 

• 
disease and structural shifts such as urbanisation; 

•  While  continuing  to  support  global  initiatives  such  as  GFATM  and  GAVI,  the 
Commission  should  use  its  influence  to  encourage  further  moves  towards  the  health 
systems strengthening components of such vertical programmes and in particular address 
the human resource consequences; 

• 
The  EU  needs  to  strengthen  the  availability  of  technical  health  capacity  in 
Delegations and, in countries where this is impossible, should consider reducing its direct 

EN 204 

Error! Unknown document property name.  EN 

 
participation in the health sector, delegating to others by participation in pooled funding, 
or drawing on expertise in EU MS embassies. 

Uptake  of  the  evaluation  recommendations:  Out  of  12  key  recommendations  formulated 
by  the  evaluators,  the  Commission  has  fully  accepted  8  recommendations,  the  other  4 
being only partially accepted. Examples that will be taken by the Commission: Following 
the recommendation of the evaluation to consolidate various global policy statements and 
approaches  into  a  comprehensive  health  co-operation  strategy,  the  Commission  (DG 
DEVCO)  is  now  preparing  an  internal  programme  for  action  that  will  integrate  existing 
policies and commitments for improved implementation of development programmes. In 
response to the recommendation to strengthen the availability of technical health capacity 
in  the  EU  Delegations,  the  Commission  is  committed  to  strengthen  its  own  sector  and 
thematic  expertise,  among  others  through  the  creation  of  DEVCO  regional  hubs.  The 
Commission will also concentrate the number of sectors of support in each partner country 
and will ensure that appropriate capacity will be available in the EU delegations to those 
15-20  countries  that  choose  health  as  a  focal  support  sector.  Regular  training  including 
regional health seminars will also be provided with strong HQ support. 

6) Availability of the report on Europa 

http://ec.europa.eu/europeaid/how/evaluation/evaluation_reports/2012/1308_docs_en.htm 

EN 205 

Error! Unknown document property name.  EN 

 
 
 
ABB 22 – Enlargement  

Evaluation of governance, rule of law, judiciary reform and fight against 
corruption and organised crime in the Western Balkans (Lot 2) 

1) ABB activities: 22 02 

2) Timing: 31 May 2012 

Period covered by the report: The evaluation covered the pre-accession assistance 
deployed under the following programmes: Phare 2002-2006; CARDS 2001-2006 and IPA 
2007- 2011 programmes under Component I. 

3) Budget  : Not applicable 

4) Background, scope and focus: 

Western  Balkan  countries  are  moving  closer  to  the  EU,  albeit  with  considerable 
differences  from  country  to  country,  by  meeting  established  criteria  and  conditions. 
Irrespective of the differences in the accession process, in much of the Western Balkans, 
better  governance,  rule  of  law,  judicial  reform  and  the  fight  against  corruption  and 
organised crime remain a major issue. The overall objective and scope of this evaluation 
were to support EU's efforts for strengthening Governance, Rule of Law, Judiciary Reform 
and  Fight  against  Corruption  and  Organised  Crime  in  the  Western  Balkans,  namely  in 
Albania,  Bosnia  and  Herzegovina,  Croatia,  Former  Yugoslav  Republic  of  Macedonia, 
Kosovo, Montenegro and Serbia in the context of EC enlargement policy. This evaluation 
was  focused  at  assisting  the  EC  in  further  developing  and  strengthening  its  assessment 
tools  in  the  area  of  Good  Governance  and  Rule  of  Law  and  providing  recommendations 
for  improving  the  assessment  process  and  tools  in  the  above  areas,  including 
recommendations  on  possible  SMART  objectives  and  indicators  of  measurement  of 
performance of financial assistance and reforms. 

5) Summary of findings and conclusions of the evaluation : 

The  evaluation  contributed  to  elaboration  of  a  sample  set  of  SMART  (Specific, 
Measurable, Achievable, Realistic and Time-bound) objectives in the areas of Governance, 
Rule  of  Law,  Judiciary  Reform  and  Fight  against  Corruption  and  Organised  Crime.  For 
each  of  the  objectives,  the  evaluation  team  proposed  a  balanced  basket  of  indicators.  In 
addition,  the  evaluation  team  assessed  the  availability  of  each  proposed  indicator  and 
recommended  in  which  sources  the  indicators  are  to  be  found.  The  combination  of 
indicators  contained  progress  or  law  indicators  and  performance  or  achievement 
indicators. Further, the evaluation explored and recommended existing verification sources 
for all indicators. The conclusions and recommendations shall be reflected in programming 
and operational aspects for the upcoming EU pre-accession assistance IPA II 2014-2020. 
Some of the proposed indicators are currently used to develop an indicator framework for 
the programming exercise for IPA H assistance  

6) Availability of the report on Europa: 

http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/2012_final_report_l
ot_2.pdf 

EN 206 

Error! Unknown document property name.  EN 

 
 
Thematic Evaluation of Rule of Law, Judicial Reform and Fight against 
Corruption and Organised Crime in the Western Balkans - Lot 3 

1) ABB activities: 22 02 

2) Timing: 20 February 2013, (Period covered by the report: 2007-2011) 

3) Budget : 

Indication of the overall multi-annual budget of the evaluated IPA program for a period 
2007- 2011 is EUR 3.6 billion. As this is a thematic evaluation, the amount is notional to 
indicate its part in relation to the overall Programme budget. 

4) Background, scope and focus 

The  overall  objective  of  the  evaluation  was  to  provide  findings  and  recommendations  to 
assist  the  General  Directorate  Enlargement  of  the  European  Commission  in  the 
programming of pre-accession assistance to candidate and potential candidate countries in 
the  Western  Balkans  and  to  improve  the  EC  instruments  to  support  the  reform  process, 
with particular regard to the areas of  Governance, Rule of Law, Judiciary Reform, Fight 
against Corruption and Organised Crime. The specific scope and  focus of the evaluation 
were  to  identify  measures,  reforms  and  actions  having  had  an  impact  in  the  areas  of 
Judiciary  Reform  and  Fight  against  Corruption  and  Organised  Crime  in  transition 
countries, including which were the key factors for success/failure and providing guidance 
on how measures and reforms should be prioritised and, where relevant, sequenced. 

5) Summary of findings and conclusions of the evaluation  

The  evaluations  found  out  that  in  all  seven  Western  Balkan  countries,  there  have  been 
major improvements to the fundamental institutional frameworks and reforms in the rule 
of law sector. Nevertheless, the challenges ahead are stronger political will, ownership by 
the  beneficiary  and  local  administrative  capacity  which  remain  crucial  conditions  for  a 
sustainable  judicial  reform.  The  evaluation  recommended  that  on  the  beneficiary  side, 
judicial  reform  programmes  should  have  clear  priorities  reflected  in  national  policy  and 
budget  propositions,  ensuring  stable  planning  parameters  and  financing.  Support  to 
strengthening  of  public  sector  accountability  actors  and  non-state  actors  is  further 
welcomed for the sustainable judicial reform. On the EU side, the evaluator recommended 
that the EU shall elaborate overall and programme objectives with regard to Governance 
and  Rule  of  Law  and  shall  re-assess/increase  the  IPA  funding  for  strengthening  Rule  of 
Law following the evidence based programming. Priority programmes shall follow multi-
annual plan and have a four-to-seven year horizon with clear 'stoppage points' for review 
and  adjustment.  The  simplification  of  IPA  II  programming  to  fewer  instruments,  more 
sector  programming  with 
the 
implementation  of  IPA  assistance,  it  was  recommended  further  flexibility  in  the 
deployment  of  the  assistance  to  be  introduced  and  significant  reduction  of  time  between 
prioritization  and  actual  activity  design.  Taking  into  account  the  results  of  the  current 

is  strongly  supported. 

time-horizons 

longer 

In 

EN 207 

Error! Unknown document property name.  EN 

 
 
 
 
 
evaluation,  a  notion  for  introduction  of  multi-annual  planning  in  IPA  II  assistance, 
covering  the  duration  of  the  next  multi-annual  financial  framework,  with  a  mid-term 
review, was conceived. 

6) Availability of the report on Europa: 

http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/2013_final_main_r
eport_lot_3.pdf 

Thematic evaluation on judiciary and fundamental rights in Turkey 

1) ABB activities : 22 02  

2) Timing: 31 October 2012, (Period covered by the report: 2004- 2009) 

3) Budget:  

Indication of the overall multi-annual budget of the evaluated subject to total IPA budget 
2007-2009 and pre-IPA assistance to the Turkey 2004-2006 is about EUR 112.6 million. 
As this is a thematic evaluation, the amount is notional to indicate its part in relation to the 
overall Programme budget. 

4) Background, scope and focus: 

This was a thematic evaluation of EU pre-accession assistance in the areas of judiciary and 
fundamental  rights  in  Turkey  (chapter  23  of  the  acquis  communataire).  The  overall 
objective  of  the  evaluation  was  to  provide  findings  and  recommendations  to  assist  DG 
Enlargement  in  improving  the  programming  and  implementation  of  EU  pre-accession 
assistance  in  the  area  of  the  political  criteria  and  judiciary,  and  fundamental  rights  in 
Turkey. The specific scope and focus of the evaluation were to provide a judgement on the 
performance  of  assistance  in  the  field  of  political  criteria  and  judiciary;  to  provide  a 
judgement  on  the  performance  of  assistance  in  the  field  of  fundamental  rights  and  to 
provide  operational  recommendations  for  programming  future  EU  assistance  in  the  field 
and for taking corrective measures to improve the implementation and monitoring of on-
going actions. 

5) Summary of findings and conclusions of the evaluation  

EU assistance in judiciary and fundamental rights in Turkey was considered as supportive, 
adding  value  in  enhancing  human  rights  reforms  to  meet  the  political  criteria  for  EU 
accession,  strengthening  the  effectiveness  of  the  court  system  to  take  into  account 
European  human  rights  standards  and  demonstrating  an  adequate  level  of  delivery  of 
results  in  view  of  the  available  resources.  However,  the  evaluation  found  out  several 
weaknesses  in  the  programming  and  implementation  of  assistance,  which  deprive  the 
assistance of some of its added value. In a context of continued distrust between national 
institutions and independent human rights actors, as well as of a lack of will on some of 
the  authorities  to  implement  certain  human  rights  reforms,  the  effectiveness  of  EU 
assistance  in  relation  to  human  rights  needs  further  improvements.  The  lengthy 
programming process was the most significant challenge to efficiency since it affected the 
overall  performance  of  each  project.  Sustainability  was  more  challenging  in  relation  to 

EN 208 

Error! Unknown document property name.  EN 

 
 
 
 
 
  
in 

the  broader  security  sector.  Following 

civil  society  projects  since  NGOs  often  lack  the  capacity  to  maintain  activities  without 
project funding. The highest levels of sustainability were achieved by those projects that 
were  most  closely  aligned  with  domestic  policy  priorities.  Obstacles  to  achievement  of 
impact  included  weak  sharing  of  information  among  institutions  and  limited  cooperation 
among  ministries 
the  evaluation's 
recommendations  to  streamline  the  programming  process  (while  strengthening  -the 
ownership  by-relevant  'institutions'  and  enhance  its  transparency  to  beneficiaries),  some 
amendments  in  programming  IPA  II  were  planned.  Elements  of  flexibility,  improved 
governance and growing ownership by the beneficiary country will be introduced in new 
IPA  II  programming  to  cater  for  emerging  needs  and  give  incentives  to  improve 
performance.  The  evaluation  also  contains  a  number  of  recommendations  focus  on  the 
programming process and implementation of sector-based approach, which were taken on 
board when proposals for new IPA II programme were drafted. 

6) Availability of the report on Europa: 

http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/2012_turkey_jhr_fi
nal_report.pdf 

Thematic Evaluation of EU's Support to Civil Society in the Western Balkans  
and Turkey 

1) ABB activities: 22 02 07 02   

2) Timing: 20 June 2012, (Period covered by the report: IPA 2007-2009) 

3) Budget: 

Indication of the overall multi-annual budget of the evaluated subject to total IPA budget 
2007-2009 is about EUR 1.4 billion. As this is a thematic evaluation, the amount is 
notional to indicate its part in relation to the overall Programme budget. 

4) Background, scope and focus:  

The EU's support to the sustainable development of civil society in partner countries aims 
at both improving the legal and financial conditions for civil society and at supporting the 
development  of  civil  society  organisations'  capacities.  EU  assistance  for  civil  society 
development  and  dialogue  in  the  Western  Balkans  and  Turkey  has  been  given  special 
attention  under  IPA.  A  broad  range  of  organisations  have  received  support,  including  in 
the areas of inter-ethnic relations, protection of minority rights, including Roma, poverty 
reduction,  environmental  protection  and  social  development.  This  was  a  thematic 
evaluation of the EU's support to Civil Society in the Western Balkans (namely Albania, 
Bosnia  and  Herzegovina,  Croatia,  Former  Yugoslav  Republic  of  Macedonia,  Kosovo, 
Montenegro  and  Serbia)  and  Turkey.  The  primary  objectives  of  the  evaluation  were  to 
provide findings and recommendations to assist DG Enlargement in the programming and 
implementation of EU pre-accession assistance to Civil Society in candidate and potential 
candidate countries and to assess the performance of financial assistance in achieving its 

EN 209 

Error! Unknown document property name.  EN 

 
 
 
 
 
  
objectives in relation to its relevance, efficiency, effectiveness, impact and sustainability. 

5) Summary of findings and conclusions of the evaluation:  

The evaluation concluded that the objective of supporting the development and capacity of 
Western  Balkans  and  Turkey  Civil  Society  Organizations  (CSOs)  has  largely  been 
achieved. The planned administrative and organisational structures have generally been set 
up,  but  have  highly  variable  effectiveness.  The  introduction  of  the  Technical  Assistance 
for  Civil  Society  and  its  regional  and  national  activities  has  so  far  been  favourably 
perceived. The outreach of EU's support has not yet been balanced in terms of supporting 
not only large, but also smaller CSOs. The impact and sustainability of the EU's support to 
Civil Society Organizations have been hampered by institutional constraints and could be 
improved  by  reinforcement  of  the  multi-beneficiary  support,  diversification  of  thematic 
programmes and promotion of wider use of geographical / sectoral or thematic small grant 
schemes. The evaluation recommended strengthening the external and internal monitoring 
of EU support to civil society. In addition the evaluation proposed better synergy between 
EU and non-EU interventions, in the programming and implementation of IPA assistance. 
The evaluation contributes to proposing some modifications in the management of the new 
1PA  program  in  terms  of  increasing  cooperation  with  other  donors  and  International 
financial  institutions  at  strategic  level  and  enhancing  stakeholder's  participation  in 
programming 
through  working  groups.  Sectoral  Monitoring  Sub-Committee  and 
Transition Assistance and Institution Building meetings. 

6) Availability of the report on Europa: 

http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/2012_eval_cs_final
_report_2.pdf 

Strategic/interim evaluation of SIGMA programme 

1) ABB activities: 22 02 

2) Timing: 31 October 2012, (period covered by the report: 2009-2011) 

3) Budget : 

Indication of the overall multi-annual budget of the evaluated subject to total IPA budget 
2009-2011 is about EUR 10 million. As this is a thematic evaluation, the amount is 
notional to indicate its part in relation to the overall Programme budget. 

4) Background, scope and focus:  

Sigma assistance started in 1992 when the EC and the OECD launched the first SIGMA 
programme to help the Central and Eastern European Countries to modernise their public 
administrations.  SlGMA's  main  objective  is  to  assist  partner  countries  to  develop  public 
governance systems which are appropriate to a democracy operating under the rule of law 
and  supporting  a  market  economy.  SlGMA's  previous  evaluation  was  done  in  2007  and 
covered  financing  to  SIGMA  from  PHARE  Multi-Beneficiary  programme.  The  primary 
to  provide  relevant  findings,  conclusions  and 
objective  of 
recommendations to the Commission about the performance of SlGMA's activities, based 

this  evaluation  was 

EN 210 

Error! Unknown document property name.  EN 

 
 
 
on  both  the  evaluator's  assessment  and  as  perceived  by  the  beneficiaries,  in  the  area  of 
Public  governance  reform/  Public  Administration  Reform.  It  assessed  the  SlGMA's 
performance under the Instrument for Pre- Accession (IPA) 2009-2011. 

5) Summary of findings and conclusions of the evaluation : 

intervention 

time  overview  of  SlGMA's 

the  evaluation  proposed  establishing  a  SlGMA's 

The evaluation reported that SlGMA's input is considered indispensable and effective, but 
it  also  acknowledged  that  the  process  is  driven  by  mutual  demand  as  institutional  and 
legislative  developments  are  decisive  factors  in  agreeing  on  collaboration.  A  structured 
and traceable hierarchy of general and specific objectives and integrated (political) risks is 
currently  lacking,  which  hamper  positive  impact  of  SlGMA's  input.  To  overcome  the 
logic, 
bottleneck, 
representing the programme as a whole. Furthermore, the evaluators advised that both EC 
services and SIGMA shall search for a new agreement, which could be useful to provide a 
real 
implementation  context.  The  prioritisation  of 
peer/stakeholder dialogue, at a national or regional level, is considered a crucial modality 
for achieving impact and awareness. Integrated dialogue modalities in SIGMA support and 
advice  are  often  praised  by  stakeholders  as  a  significant  capacitating  influence  which 
facilitates engagement, ownership and awareness. It was recommended that both SIGMA 
and the Commission services would benefited from greater political leverage to maximise 
the  potential  effect  of  SIGMA  outcomes.  In  this  respect  a  proposal  for  participation  of 
relevant  EUD  staff  in  programming  and  (general)  implementation  of  SlGMA's  missions, 
where and when feasible, was welcomed. These findings contributed to the Commission's 
future approach on planning and programming SIGMA with the view to responding better 
to the strategic goals of enlargement policy in the area of Public Administration Reform. 
In line with the evaluation recommendations, the purpose of the SIGMA assessment will 
be broadened. Starting in 2013, in three countries, the assessment will become the basis for 
a more effective policy dialogue, a tool for informing IPA national and regional funding, 
and a tool for SIGMA to develop country reform plans. 

6) Availability of the report on Europa: 

Not yet available. 

EN 211 

Error! Unknown document property name.  EN 

 
 
ABB 23 – Humanitarian Aid 

Evaluation and review of humanitarian access strategies in DG ECHO funded 
interventions 

1) ABB activities: 23 02 

2) Timing: June 2012 

3) Budget: not applicable 

4) Background, scope and focus 

Lack  of  access  means  that  humanitarian  partners  are  not  able  to  conduct  adequate  needs 
assessments of populations in need, nor can they subsequently implement and monitor their 
humanitarian  assistance  safely  and  effectively.  The  selective  restriction  of  access  to  certain 
groups,  whether  by  armed  groups  or  by  governments,  may  also  compromise  the 
independence, neutrality and impartiality of humanitarian aid. 

The  main  objective  is  to  have  an  independent  structured  evaluation  and  review  of 
humanitarian access strategies in line with Regulation (EC) 1257/96 concerning humanitarian 
aid to provide DG ECHO with an assessment of its own practices and those of its partners, 
considering  these  also  in  relation  to  those  of  other  donors  and  implementing  partners,  and 
with a policy framework and practical guidelines for functioning effectively in situations of 
restricted access. 

5) Summary of findings and conclusions of the evaluation  

Humanitarian  access  is  both  ability  of  humanitarian  organizations  to  reach  populations 
affected  by  crises  and  the  ability  of  the  humanitarian  organizations  to  reach  populations 
affected by the crises and the ability of affected populations to access humanitarian services. 

The  evaluation  analyses  the  most  important  constraints  to  access,  the  strategies  applied  by 
humanitarian actors to overcome them and the compromises these strategies involve.  

Access  constraints  identified  are  those  related  to  security  restrictions,  those  imposed  by  the 
Governments  wanting  to  limit  access  through  immigration  policies  or  imposing  travel 
restrictions.  The  indirect  constraints  such  as  internal  security  rules  of  organisations, 
legislation preventing organisations from engaging with armed actors considered as terrorists, 
or "politicization of aid" might also restrict access to people in need.  

The evaluation has identified activities that can be undertaken to expand or preserve access: 
first to be tacked at the source of constraints by trying to persuade those in control to allow 
more access; second to mitigate and manage the security risks to continue the assistance, the 
third  to  operate  through  remote  management  and  finally  getting  non-Western  powers 
influencing governments on board.  

Also, it should be decided what compromises are acceptable to continue serving those in need 
and/or  eventually  when  to  disengage.  DG  ECHO's  standard  instruments  permit  deciding 
whether or not to disengage, however potential negative effects of humanitarian activities are 
not sufficiently considered. 

When attempting to maintain or increase access, the balance between humanitarian principles 
and minimum requirements for providing assistance should be duly analysed in order to not 

EN 212 

Error! Unknown document property name.  EN 

 
 
compromise the credibility in the long term and running the risk of unintended harm. 

Main recommendations: 

1. 
More  active  and  strategic  role  in  advocacy  should  be  adopted,  as  well  positions  on 
anti-terrorism legislation and UN integration. Efforts should be increased to define coherent 
positions  on  humanitarian  issues  within  the  Commission,  the  European  Parliament  and  EU 
Member States. Encourage OCHA to focus more on liaising with governments. 

Increase efforts to engage non-Western actors, particularly influential for increase of 

2. 
access.  

Support staff members and ensure consistent decisions; further develop and practice 

3. 
with teaching cases to achieve coherent decision-making on moral and practical dilemmas.  

Adopt  a  common  definition  of  remote  management,  develop  operational  guidance 

4. 
and improve DG ECHO's ability to monitor projects directly. 

5. 
Go  into  "hibernation"  when  compromises  become  excessive;  reduce  the  overall 
budget and restrict funding to strategic partners and enable partners to maintain a networking 
capacity on the ground and support small-scale projects.  

6) Availability of the report on Europa 

http://ec.europa.eu/echo/files/evaluation/2012/GPPi_Access-Report_July-2012.pdf 

The evaluation of dg echo's disaster preparedness and disaster risk reduction 
actions in southern Africa & the Indian ocean 

1) ABB activities: 23 02 

2) Timing: December  2011 

3) Budget: EUR 16,568,343 

4) Background, scope and focus: 

The  south-east  African  and  south-west  Indian  Ocean  region  is  one  of  the  most  disaster-
prone regions in the world, experiencing multiple types of natural disasters. 

A  first  Action  Plan  for  Disaster  Preparedness  was  launched  under  DG  ECHO's  Disaster 
Preparedness  ECHO  (DIPECHO)  programme  (implementation  timeframe  from  October 
2008  –  March  2010),  and  was  followed  up  by  a  second  Action  Plan  for  Disaster 
Preparedness  from  June  2010  to  December  2011).    The  Action  Plans  aim  to  increase 
resilience  and  decrease  the  vulnerability  of  local  communities  and  institutions  by 
supporting  strategies  that  enhance  their  capacities  and  enable  them  to  better  prepare  for, 
mitigate and respond adequately to natural disasters. 

Due to their pilot nature and to the DG ECHO aim for development donors and national 
governments  to scale-up successful approaches tested through its DRR interventions, the 
evaluation should focus on the effectiveness of the different sets of activities implemented 
and  their  "fitness  for  purpose"  in  order  to  allow  DG  ECHO  to  select  those  interventions 

EN 213 

Error! Unknown document property name.  EN 

 
 
 
 
which should be prioritized for consolidation in the third Action Plan (2012 – 2013) and 
thus  implement  a  viable  exit  strategy  of  the  current  geographical  areas  of  intervention 
within the parameters of DG ECHO. 

5) Summary of findings and conclusions of the evaluation:  

The countries, the regions and the communities selected are among the most exposed to the 
selected priority hazards. The main impact of DIPECHO was in the strengthening (or set 
up)  of  local  DRR  committees  at  community  level,  a  highly  praised  achievement.  The 
national  Disaster  Risk  Management  organization  expressed  above  all  a  need  for  direct 
funding  for  their  activities  and  contribution  to  the  DIPECHO  DRR  projects:  an  option 
currently  not  available.  DG  ECHO  dedicated  considerable  efforts  to  promote  inter-
governmental  coordination.  The  effectiveness  of  its  actions  was  in  terms  of  prospective 
(but hypothetical) benefits in case of disaster but above all in immediate and more tangible 
results (better crop, access to safe water, etc.). In terms of location (access and number of 
communities),  hazard  (drought,  earthquake),  thematic  priority  (urban  or  rural),  there  is  a 
need  for  improving  and  explaining  the  coherence  and  consistency  of  the  DG  ECHO 
strategy.  The  selection  of  partners  and  projects  appears  to  be  participative  within  DG 
ECHO,  but  does  not  meaningfully  involve  outside  stakeholders  essential  for  ensuring 
sustainability:  national  authorities,  EU  delegation  and  some  agencies  with  critically 
relevant mandate such as UNDP and Red Cross. The concept of an exit strategy is poorly 
understood and not taken seriously by Partners. It is time for consolidation to ensure full 
ownership and moving on to new innovative untested ideas, expanding the hazards or the 
context and including new unchartered countries. 

The recommendations are grouped in three themes: 1) improving the strategy, 2) ensuring 
sustainability and 3) specific suggestions for DIPECHO III. 

1) Adoption of operational criteria for selection of targets specific to increase the chance of 
success  and  visibility  of  pilot  projects.  Inclusion  of  food/livelihood  security  DRR 
activities.  Extension  of  duration  of  financing  to  24  months.  Continuing  the  focus  on 
communities,  increasing  impact  at  national  level  and  curtailing  intergovernmental 
coordination  initiatives.  2)  Identifying  among  the  “best  practices”  those  that  are  actually 
scalable  up  and  focusing  on  their  marketing  before  other  donors.  Establishing  true 
partnership  with  EU  Delegation  and  other  development  instruments.  In  partnership  with 
EuropeAid,  organize  a  meeting  of  global  donors  to  promote  a  few  specific  interventions 
most  suitable  for  integration  into  projected  DRR  related  programs.3)  Favouring  a 
consortium  approach  (one  joint  project)  in  each  country.  Limiting  isolated  projects  to 
highly  innovative  interventions  too  specific  or  risky  for  adoption  by  all  partners  of  the 
consortium. Sharing decision making on selection of projects with the EU delegation and 
involving national authorities in the process. Identifying a mechanism for funding of the 
involvement  of  the  national  DRR  Authority  (Civil  protection,  UNDP  or  other). 
Establishing a  roster  and  mobilization  mechanism  for  rapid  and systematic  evaluation  of 
DRR  effectiveness  (impact)  in  the  aftermath  of  a  disaster  in  the  target  communities. 
Limiting  regional  projects  mostly  to  the  provision  of  technical  assistance  and  support. 
Extending  DIPECHO  to  Namibia  in  soliciting  a  joint  (consortium)  project  from  UN  and 
Red Cross and preparing the ground for further expansion in Southern Africa. 

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/DIPECHO_south_africa_indian_ocean.pdf 

EN 214 

Error! Unknown document property name.  EN 

 
Evaluation of the European Commission in Ethiopia 

1) ABB activities: 23 02 

2) Timing: July 2012 

3) Budget: EUR 192.76 million since 2007 

4) Background, scope and focus: 

Since  2007,  DG  ECHO  has  adopted  18  Financial  Decisions  related  to  Ethiopia.  In  these 
Decisions a total of over EUR 192.76 million has been committed in order to respond to 
humanitarian emergencies, targeting millions of people each year. 

The  main  purposes  of  the  evaluation  are  lesson  learning  and  accountability  in  order  to 
improve future performance.  

5) Summary of findings and conclusions of the evaluation:  

The  strengths  of  the  DG  ECHO  contribution  to  preventing  catastrophic  crises  are  in  its 
focus  on  communities  and  its  potential  for  innovations.  These  strengths  have  not  been 
fully  channelled  within  the  framework  of  a  clear  strategy.  The  resources  spread 
increasingly thinly over a large number of projects, places and beneficiaries. Little impact 
should be expected from interventions providing mono-sectoral services to many dispersed 
targets. In addition to scattered over too many projects, the action is also lacking integrated 
multi-sectoral  approach.  Livelihood,  nutrition,  WASH  or  health  activities  are  usually 
isolated and disconnected from each other. 

Linking with EU development instruments has been priority. This collaboration with DG 
DEVCO  and  promotion  towards  combined  effort  are  ultimately  on  the  point  to  produce 
concrete  results:  i.e.  the  EU  SHARE  initiative  should  provide  the  EU  delegation  with  a 
tool  to  build  on  the  work  of  the  humanitarian  partners,  with  DG  ECHO  as  a  source  of 
technical  expertise  and  advice.  Regarding  draught  risk  reduction,  partners'  projects  are 
reasonably  inserted  in  LRRD  perspective.  However,  the  LRRD  problem  seems  to  be  the 
relief  part  of  the  concept,  as  some  of  the  worthwhile  projects  have  limited  link  with 
emergency relief.  

The Rapid Response to crises is DG ECHO's public trademark. However approval process 
is  perceived  by  most  partners  as  comparatively  slower  and  less  flexible.  Once  activated, 
the assistance covered the scope of needs across sectors. 

Malnutrition  rates  are  affected  by  multiple  factors  and  influenced  by  much  larger 
programmes than those of DG ECHO. A change of acute malnutrition or crude mortality 
rates is not necessarily reflecting the effectiveness or lack of emergency projects. 

Main recommendations: 

1.  DG  ECHO  should  adopt  distinct  strategies  for  protracted  food  crises,  acute  food 
emergencies and sudden onset crises and conflicts. 

2.  The  number  of  projects  directly  funded  and  monitored  by  its  staff  should  be 
considerably reduced.  

EN 215 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
  
3.  A cost-effective multi-sectoral package to fewer beneficiaries in a limited number of 
woredas  in  protracted  crises  should  be  foreseen.  In  particular,  in-depth  review  could  be 
launched specifically to define the supporting role of health sector in tackling food security 
and nutritional problems. 

In  situations  requiring  a  rapid  response,  DG  ECHO  should  explore  a  mechanism  to 

4. 
pre-assign funding to select partners.  

DG  ECHO  and  its  partners  should  pay  more  attention  to  the  selection  of  appropriate 
indicators  to  measure  the  impact  of  their  interventions.  This  is  especially  critical  for 
changes in rates of acute malnutrition in protracted food crises 

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/ethiopia_final_report.pdf 

Evaluation of the European Commission's Humanitarian Activities in 
Bangladesh 

1) ABB activities: 23 02 

2) Timing: October 2012 

3) Budget: Humanitarian aid to Bangladesh has totalled over EUR 107 million since 2000. 

4) Background, scope and focus:  

The  European  Commission’s  Directorate  General  for  Humanitarian  Aid  and  Civil 
Protection (DG ECHO) has been working in Bangladesh for many years, both in disaster 
preparedness and in response activities. Humanitarian aid to Bangladesh has totalled over 
EUR 107 million since 2000. 

DG  ECHO  has  also  contributed  more  than  EUR  53  million  in  humanitarian  aid  for  the 
victims of cyclones Sidr and Aila since 2007 and over EUR 29 million for flood victims 
since  2004.  The  funding  was  used  to  help  the  most  vulnerable  people,  some  of  them 
displaced from their homes, with basic humanitarian life saving support (food assistance, 
drinking  water;  sanitation  and  hygiene  facilities;  emergency  shelter  materials)  and  to 
regain self-sufficiency through income generating activities. 

The scope is to assess the appropriateness and effectiveness of the Commission’s actions, 
in accordance with DG ECHO’s mandate, in order to establish whether they have achieved 
their  objectives  and  to  produce  recommendations  for  strengthening  the  impact  of  future 
operations in Bangladesh. The results of the evaluation will be used in the preparation of 
the 2013 strategy for Bangladesh. 

5) Summary of findings and conclusions of the evaluation:  

In  no  other  country  DG  ECHO's  expertise  is  more  needed  than  in  Bangladesh;  given  its 
unique vulnerability to disasters. Due to the widespread network of implementing partners 
DG ECHO's emergency response programmes effectively assisted the affected population, 
recovered livelihoods and reinstalled coping mechanisms. Response times are considered 
as very quick and the speed of funding is very pleasing for most local NGOs. 

EN 216 

Error! Unknown document property name.  EN 

 
 
 
The  'bottom  up'  approach  of  DIPECHO  programmes  has  proved  that  affected  local 
communities  were  better  prepared  for  disasters,  the  coordination  on  local  level  was 
enhanced,  and  the  most  vulnerable  people  were  addressed.  The  evaluation  of  the 
Consortium  System  has  also  shown  convincing  results  in  managing  several  agencies 
engaged in a single project; even though group agreements might take up more time and 
potentially slow down the overall response process.  

The relationship and coordination between DG ECHO and the Government of Bangladesh 
and  other  humanitarian  donors  leaves  potential  for  improvement.  So  far,  DG  ECHO's 
assistance  to  unregistered  refugees  provides  additional  value  to  UNHCR  activities  in  the 
country.  However,  in  order  to  make  cooperation  on  local  level  more  effective,  a  more 
targeted and prioritised approach would be beneficial.  

Due to the lack of a clearly articulated, national LRRD strategy DG ECHO's  emergency 
response  programmes  rather  do  not  have  long-term  effects.  A  national  strategy  would 
allow  replication  of  successful  projects,  facilitate  responses  for  chronic  emergencies, 
improve  the  issues  of  malnutrition  and  food  security,  and  contribute  to  the  overall 
development process of the country. 

Main recommendations: 

1.)  Especially  in  the  case  of  bigger  disasters,  DG  ECHO's  activities  need  to  be  more 
targeted and prioritised to increase coordination with other donors and thus effectiveness.  

2.)  DIPECHO programmes should be complementary to governmental work (CDMP), in 
order to achieve more long-term stability and foster the multiplier effect. 

3.)  DG  ECHO  needs  to  continue  emphasising  the  abuse  of  human  rights  of  Rohingya 
refugees in the region. 

4.)  The  current  strategy  towards  malnutrition  should  be  continued  and  yet  extended  by 
rigorous prioritisation and by decisions on entry and exit strategies. 

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/Bangladesh.pdf 

Evaluation of the European Commission's Humanitarian Activities in Colombia 

1) ABB activities: 23 02 

2) Timing: November 2012 

3) Budget: The budget allocated for humanitarian emergency assistance to IDPs is around 
EUR 200 million per year 

4) Background, scope and focus : 

The  main  humanitarian  actor  in  Colombia  is  the  Government  of  Colombia  (GoC).  The 
budget allocated for humanitarian emergency assistance to IDPs is estimated at some EUR 
200  million  per  year.  Performance  of  the  social  agencies  of  the  GoC  to  deliver 
humanitarian  aid  has  increased  during  the  past  years,  particularly  in  capitals  of 
departments.  Nonetheless,  official  assistance  to  rural  zones  affected  by  the  conflict  is 

EN 217 

Error! Unknown document property name.  EN 

 
 
 
scarce;  civil  servants  are  afraid  of  working  in  those  zones  due  to  prevailing  insecurity. 
ECHO  has  progressively concentrated  more  humanitarian  aid  in those  rural  zones  where 
this gap exists. 

The purpose of the evaluation is to assess the appropriateness of DG ECHO’s actions, in 
accordance with DG ECHO’s mandate, in order to establish whether they have achieved 
their objectives and to produce recommendations for improving the effectiveness of future 
operations in Colombia. 

5) Summary of findings and conclusions of the evaluation:  

Due  to  its  expertise  DG  ECHO  took  on  a  leading  role  in  terms  of  coordinating 
humanitarian  donors  in  Colombia.  As  a  result  the  Directorate  assisted  UN  OCHA  in 
setting  up  a  common  humanitarian  framework  and  ensured  that  LRRD  was  effectively 
integrated into a broader strategy. 

The  evaluation  revealed  that  DG  ECHO's  emergency  response  to  natural  disasters  was 
conducted in a timely manner, addressed the right needs due to effective prioritisation, and 
close monitoring of on-going projects allowed for quick adaptation if unforeseen changes 
occurred.  

However, in regards to DG ECHO's focus on supporting registered asylum seekers, poor 
selection  of  eligibility  criteria  by  local  partners  resulted  in  the  exclusion  of  many 
unregistered IDPs and PNIPs. Legal and institutional changes in Colombia, but also in the 
neighbouring Ecuador, additionally reduced the number of people with refugee status. For 
this reason, there is disagreement on the official number of IDPs and PNIPs, and therefore 
the coverage of DG ECHO's support can only be estimated. 

Although successfully implemented, DG ECHO's LRRD efforts were severely hindered by 
the armed conflict in the country. A long-term approach to improve the livelihood of the 
population  depends  on  the  commitment  of  the  local  government,  which  is  especially 
difficult when the government itself is a party in the conflict. 

Main recommendations: 

1.  Change  eligibility  criteria  and  increase  coverage  of  registered  and  non-registered 
IDPs and PNIPs. 

2.  Review  data  collection  procedures  in  order  to  improve  monitoring  DG  ECHO's 
efficiency. 

3.  LRRD coordination with government institutions and other humanitarian actors needs 
to be improved. 

4.  The  Directorate  should  continue  its  community/neighbourhood  approach  and 
encourage other implementing partners to support the basic needs of vulnerable groups.  

5.  DG ECHO should extend its activities to urban settings 

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/Colombia.pdf 

EN 218 

Error! Unknown document property name.  EN 

 
 
Evaluation and Review of the DG ECHO financed Livelihood Interventions in 
Humanitarian Crises 

1) ABB activities: 23 02 

2) Timing: March 2012 

3) Budget: Overall budget humanitarian aid EUR 6.1 billion over period 2007-13.  

4) Background, scope and focus:  

Livelihoods  comprise  the  capabilities,  assets  (including  natural,  material  and  social 
resources)  and  activities  used  by  a  household  for  survival  and  future  well-being. 
Livelihood  strategies  are  the  practical  means  or  activities  through  which  people  access 
income,  while  coping  strategies  are  temporary  responses  forced  by  food  insecurity.  A 
household’s  livelihood  is  secure  when  it  can  cope  with  and  recover  from  shocks,  and 
maintain or enhance its capabilities and productive asset base. 

The  main  objective  of  the  evaluation  exercise  is  to  provide  DG  ECHO  with  a  multi-
regional  evaluation  of  its  activities  in  the  livelihoods  support  sector.  The  evaluation  will 
cover  the  entire  range  of  livelihoods  support  activities  financed  across  the  spectrum  of 
humanitarian interventions. This will involve analysing a selection of DG ECHO funded 
livelihood interventions or interventions with livelihoods support components undertaken 
since the creation of the food aid budget line in 2007 (although the evaluation will not be 
exclusively limited to livelihoods support actions targeting food insecurity).   

5) Summary of findings and conclusions of the evaluation:  

DG  ECHO  has  made  considerable  progress  in  making  the  livelihood  programmes  more 
defined, coherent and innovative, even though there have been missed opportunities for the 
exchange  of  good  practices  across  countries  and  regions.  Standardized  programming 
options have not clearly demonstrated to be effective.  

The  Directorate's  emergency  livelihood  activities  are  characterized  by  well  coordination 
and open dialogue with local partners. Although it was not always possible to evaluate DG 
ECHO's work with much precision – because of the lack of thorough documentation, the 
team's  response  to  the  chronic  food  insecurity  in  the  Sahel  or  disaster  risk  reduction 
programmes in the Horn of Africa, were considered good examples for LRRD.  

However, LRRD implementation in the area of food security bears a challenge. The risk 
exists that a central focus on livelihood approaches may lead to an omission of vital relief 
activities. A key issue here is the rather short timeframe of livelihood programmes of 12 
months  which  threatens  the  effectiveness  of  project  implementation.  Certain  livelihood 
recovery and protection projects can just not be tied to annual programme cycles. 

Because of limited technical capacities of its staff response analysis and impact evaluation 
are notable weaknesses of DG ECHO and its partners in the field. For the same reasons, it 
is  hard  to  establish  purposeful  mechanisms  that  allow  for  learning  and  knowledge 
exchange.  Structural  changes  would  facilitate  the  usage  of  evaluation  findings  and 
contribute to the effectiveness of individual projects. 

EN 219 

Error! Unknown document property name.  EN 

 
 
 
Main recommendations: 

1)  Clear documentation, adequate monitoring of activities and collection of quality data 
will  increase  the  meaningfulness  of  analysis  and  evaluations  and  in  turn  have  a  positive 
effect on single projects. 

2)  DG  ECHO  needs  to  continue  its  drive  for  innovation  and  not  follow  tendencies  to 
standardized default responses. 

3)  DG  ECHO  needs  to  extend  timeframes  for  particular  livelihood projects  in  order  to 
increase the project's effectiveness. 

4)  Regular knowledge/information exchange and the creation of associated mechanisms 
will lead to greater efficiency and better practices across countries and regions.  

5)  The Directorate needs to take care that LRRD approaches combine relief assistance, 
long-term development strategies and regional integration. 

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/Livelihood_interventions.pdf 

The evaluation of echo's intervention in the occupied Palestinian territory and 
Lebanon 

1) ABB activities: 23 02 

2) Timing: March 2012 

3) Budget: EUR 600 million since 2000. 

4) Background, scope and focus : 

The European Commission has provided almost EUR 600 million in humanitarian aid to 
help to meet the basic needs of Palestinians in the Occupied Palestinian Territory (oPt) as 
well  as  the  Palestinian  refugees  living in  Lebanon, Jordan,  and Syria  since  2000.  ECHO 
has  been  active  in  the  oPt  since  2000,  (beginning  of  the  Second  Intifada).  2006  noted  a 
peak in terms of funding with a total allocation of EUR 84M to address the humanitarian 
needs  of  Palestinians  in  the  oPt  following  the  Hamas  take-over  of  Gaza  and  the  ensuing 
events that resulted directly or indirectly from this result -including multiple incursions by 
the Israeli Army in the Gaza strip, the freeze by the Israeli government of Palestinian VAT 
revenue  transfers,  the  continuation  of  the  construction  of  the  Wall,  and  the  reduction  by 
main donors of their foreign assistance. 

The purpose of the evaluation is to assess the appropriateness of ECHO’s actions vis à vis 
its  mandate, in order to establish whether objectives have been achieved; to evaluate the 
effectiveness  and  timeliness  of  ECHO's  strategic  approach  to  the  Palestinian  refugees  in 
the oPt and Lebanon; and finally to produce recommendations for defining ECHO´s multi-
sectoral strategy and for improving the effectiveness of future operations in addressing the 
needs of the Palestinian Refugees in the oPt and Lebanon. 

5) Summary of findings and conclusions of the evaluation: 

EN 220 

Error! Unknown document property name.  EN 

 
 
 
Over  the  reporting  period  in  West  Bank  and  Gaza  (WBG),  DG  ECHO  has  supported 
different humanitarian and protection activities in order to assist the suffering population, 
such  as  food  assistance,  livelihoods,  shelter,  healthcare,  monitoring  violations  of 
International Humanitarian Law and International Human Rights Law. However 'victims’ 
humanitarian  needs  in  OPT  and  Gaza  are  not  prima  facie  as  severe  as  those  normally 
addressed by DG ECHO in other theatres and, this is not therefore a humanitarian crisis 
but a rights crisis with humanitarian consequences. The causes of the humanitarian needs 
are clear but addressing symptoms only, and not the threats or violations, does not address 
the  vulnerabilities  in  a  sustainable  manner.  While  protection  has  been  identified  as  the 
priority sector, intervention has had little impact on stopping brutal violations and denials 
of  human  rights  such  that  associated  interventions  constitute  little  more  than  short-term 
alleviations of the symptoms. Within its mandate, ECHO has the opportunity to do both, 
providing  humanitarian  assistance  and  advocating  for  addressing  the  root  causes  of  the 
vulnerabilities, thus increasing its impact in alleviating humanitarian suffering.  

Main Recommendations: 

While DG ECHO is limited by its mandate to politically address the causal factors of the 
consequent  urgent  humanitarian  needs,  it  can  still  take  a  stronger  principled  stand  with 
regard  to  victims’  vulnerabilities  and  translate  this  into  more  effective  concrete  actions 
(both in the courts and through international and Israel-based advocacy) while continuing 
to  address  those  needs.  ECHO  needs  to  respond,  not  only  to  ex-post  symptoms  of 
violations,  but  also  to  victims’  vulnerabilities  to  violations,  particularly  with  regard  to 
demolitions,  settler  violence  and  forced  displacements  and  ensure  it  is  done  in  a 
comprehensive way through encouraging coordination between relevant actors. DG ECHO 
has not well been able to monitor partners’ efforts and how successful each has or has not 
been. It should provide better guidance on its definitions and implementation approach, as 
well  as  sharing  good  practices.  It  should  develop  a  tool  that  would  enable  it  to  monitor 
how  partners  are  implementing  this  in  the  field  and  where  their  successes  lie.  The 
Protection  Cluster’s  emergency  response  mechanism  (established  mostly  for  the  West 
Bank)  which  comprises  a  multi-partner  humanitarian  assistance,  protection  and  litigation 
response is excellent, and needs to be expanded and supported.      

6) Availability of the report on Europa: 

Not available on Europa.  

Evaluation of the European Commission supported humanitarian aid in urban 
settings 

1) ABB activities: 23 02 

2) Timing: June 2012 

3) Budget: Overall budget humanitarian aid EUR 6.1 billion over period 2007-13.  

4) Background, scope and focus : 

Urban  areas  are  complex  settings  to  implement  humanitarian  assistance  and  vary  from 
rural  areas  in  terms  of  needs  and  vulnerabilities.  Institutions,  partners,  and  intervention 

EN 221 

Error! Unknown document property name.  EN 

 
 
 
methods also vary considerably among small, medium and large cities and among regions. 
Humanitarian assistance in urban settings is dependent upon a range of local, national and 
international  interlocutors.  The  capacities,  human  and  financial  resources,  tools  and 
procedures and level of preparedness for disasters can vary among cities to a large degree. 

There are two main objectives for this evaluation. Firstly, the evaluation should provide an 
independent structured evaluation of the results of DG ECHO’s strategic and operational 
capacity  and  ability  to  fund  needs  based  humanitarian  operations  in  urban  areas  in  line 
with DG ECHO legal basis 1257/96, with the overall purpose of improving performance 
through  lesson  learning  and  accountability.  Secondly,  the  evaluation  should  assess  the 
scope, lessons learned, outcomes, limits and comparative advantage of DG ECHO funded 
actions  in  urban  contexts  to-date,  and  to  provide  advice  to  DG  ECHO,  particularly  in 
relation to its institutional practices and tool kit. 

5) Summary of findings and conclusions of the evaluation:  

DG  ECHO  has  provided  resources  to  a  broad  range  of  stakeholders  within  the 
humanitarian  operations  supporting  urban  settings.  However  while  many  of  them  had 
some  experience  in  urban  settings,  none  of  them  had  a  clear  strategy  for  conducting 
operations. Humanitarian response in urban settings requires a specific approach in view 
of the complexity of the urban environment. Delivering humanitarian aid in urban settings 
is often dangerous, logistically difficult and requires significant resources in view of urban 
disasters.  

Engaging humanitarian aid in urban settings requires both an appropriate analyses of the 
evolution  of  the  on-going  crises  and  understanding  of  the  urban  settings.  There  are 
challenges  to  be  addressed  that  depend  on  specific  contexts  such  as  slums  or  organized 
urban violence. Risks associated with getting involved have to be assessed carefully; exit 
strategies need to be identified.  

Urban  disasters  need  to  be  addressed  very  quickly:  introduction  of  "primary  emergency 
decision"  and  "small–scale  emergency  decision"  are  appropriate  and  important  DG 
ECHO's  funding  tools,  however  lack  of  coherence  or  coordination  and  competition 
between donors and agencies are harmful to address urban disasters. 

LRRD (Linking Relief, Rehabilitation and Development) in urban context proved difficult 
within  European  aid  framework.  Urban  issues  are  seldom  included  in  development 
programmes, and if so, in a superficial way. DG ECHO is involved in Post-Disaster Needs 
Assessment  with  other  donors,  however  when  the  Commission's  humanitarian  and 
development  directorates  are  engaged,  completed  with  experts  from  Member  States,  the 
Post-Disaster Need Assessment works even better. 

Main recommendations: 

1.  The  Commission  should  ensure  that  operational  concepts  and  methods  used  make 
optimal  use  of  all  opportunities  and  options  available  in  urban  settings.  City-to-city 
cooperation  and  agreement  between  humanitarian  agencies  and  municipal  technical 
departments should be further strengthening. 

2.  The  entry  criteria  and  exist  strategies  for  engaging  in  urban  settings  should  be 
clarified. The exit strategies should be developed at the early stages of needs and capacity 
assessment and the design of intervention logic. 

EN 222 

Error! Unknown document property name.  EN 

 
3.  DG ECHO should continue its efforts to be a driving force in terms of coordination 
between donors. It should also ensure that community participation and coordination with 
local civil society is the rule rather than exception. 

To  address  issues  of  LRRD  the  DG  ECHO  should  strengthen  its  dialogue  with  other 
donors 

6) Availability of the report on Europa 

Not available.  

Evaluation of the DG ECHO Legal Framework for Funding of Humanitarian 
Actions (FPA 2008) 

1) ABB activities: 23 02 

2) Timing: June 2012 

3) Budget: not applicable 

4) Background, scope and focus : 

The  partnership  has  been  established  since  1993  through  the  signing  of  a  Framework 
Partnership  Agreements  which  operates  as  a  long-term  cooperation  mechanism  between 
the  Parties  designed  to  ensure  that  ECHO-  funded  humanitarian  partners  and  operations 
meet the highest standards of performance and quality. 

The scope of the evaluation will cover the evolution of the different FPAs with a special 
focus on the implementations of the FPA rules and procedures that entered into force on 
January 2008 and which are still applied today.   

5) Summary of findings and conclusions of the evaluation:  

Although  working  relationships  between  ECHO  and  the  partners  are  generally  good,  the 
principles  of  trust,  mutual  respect  and  ownership  in  the  asymmetric  partnership  between 
donors and recipient agencies are not always applied in a consistent manner.  

The  effectiveness  of  a  small  group  of  "quality  partners"  would  benefit  from  greater 
flexibility and some type of non-earmarked funding. Categories of partners should be more 
accurately  defined  with  operational  and  not  only  financial  criteria.  It  should  be  also 
possible for eligible weaker partners to become stronger.  

There  is  need  for  greater  flexibility  of  funding,  in  particular  to  ensure  rapid  response  to 
new or worsening emergencies. Regular delays in contracting and funding of emergency 
projects  may  be  detrimental  to  the  delivery  of  aid  to  the  beneficiaries.  However  funding 
modalities  and  timeframes  are  imposed  by  the  Humanitarian  Aid  Regulation  and  the 
Financial  Regulation,  or  are  not  always  interpreted  in  a  flexible  enough  or  adequate 
manner. 

The  presence  of  ECHO  experts  in  the  field,  the  focus  on  forgotten  crises  and  non-
partisan/political approach, the promotion of humanitarian principles , the focus on results, 

EN 223 

Error! Unknown document property name.  EN 

 
 
 
 
the diversity of partners and the scale of funding are key perceived advantages of ECHO 
compared to other donors. However there is no guidance on how implement a result-based 
approach. The efficiency has been hampered by a number of key problems at the proposal 
and liquidation stages.  

Recommendations 

1. 
Improved quality criteria should be used to define categories of partners according to 
their  operational  and  not  only  on  financial  capacities  in  particular  a  category  of  "quality 
partners".  Selected  "quality  partners  should  benefit  from  more  flexible  fast-truck 
procedures.  DG  ECHO  should  continue  working  with  the  various  for  a  used  for 
consultation process with partners. 

2. 
In order for the FPA to be flexible "fit for purpose" instruments, procedures should be 
better adapted to the different categories of partners and situations. The new FPA should 
include the possibility of providing flexible funding to partners' disaster response tools and 
assets. 

3.  Efficiency  could  be  improved  if,  for  instance,  at  the  project  proposal  selection  of 
proposals  should  be  more  transparent,  clarified  terminology  and  timeline  of  expected 
results; regarding implementation more efficient e-tools should be proposed and used;  at 
the  liquidation  stage  more  inter-service  coherence  and  coordination  between  ECHO 
operational, Finance, Audit units should considered. 

4.  ECHO should advocate in the revision of the Financial regulation for more flexibility 
regarding indirect (transaction) costs.  

6) Availability of the report on Europa: 

http://ec.europa.eu/echo/files/evaluation/2012/fpa2008_en.pdf 

EN 224 

Error! Unknown document property name.  EN 

 
 
ABB 26 – Commission's administration 

Evaluation of the language training organised in Brussels by DG HR 

1) ABB activities: 26 01 

2) Timing: 24/01/2013, (period covered: 2012) 

3) Budget: EUR 84,990  

4) Background, scope and focus: 

The evaluation covers the language trainings courses organised for EU Institutions staff in 
Brussels by DG HR. 

The objective of this evaluation is to support DG HR in (1) defining the learning needs and 
the new language training priorities of the Commission; (2) assessing the effectiveness of 
the current language training within the Commission; and (3) analysing the most effective 
way  to  introduce  new  alternative  methods  of  language  training,  such  as  e-Learning  or 
blended learning. 

5) Summary of findings and conclusions of the evaluation:  

Main results of the evaluation 

Language learning needs are better addressed and language learning is more efficient 

• 
for units where language learning is strongly related to their performance.  

• 
There is a strong requirement by services and learners for proficiency; however, since 
most of the learners are beginners, the actual language needs of the services and staff will 
be met over a longer period.   

•  A more systematic evaluation of the courses' outcomes should be introduced, as well 
as evaluation of the degree to which the new skills are effectively used to respond to the 
actual needs and objectives of learners and services. 

Staff  generally  considers  that  the  current  language  training  offer  responds  to  their 

• 
needs, but the extent to which this is achieved is medium. 

Course  attendance  should  be  enhanced  by  several  incentives  e.g.  promotion  of 

• 
officials. More formal consequences should be introduced if courses are abandoned. 

Blended and virtual learning are likely to increase in popularity as delivery methods 
• 
of workplace learning in the future. These methods will increase learning options and may 
improve learning efficiency.  

•  Alternative methods facilitate differentiation in language learning content and allow 
addressing different target groups and preferred learning style of learners. They also offer 
a means to reduce absenteeism. 

Overall conclusion  

The  current  language  training  provision  is  the  most  used  means  through  which 
Commission staff acquires new language skills, which highlights its role and importance. 

EN 225 

Error! Unknown document property name.  EN 

 
 
 
However, attaining the highest effectiveness possible would require that the language offer 
and content be customised and become more flexible. 

Recommendations 

The evaluation report proposes a set of recommendations to improve various aspects of the 
policy of language training:  

• 
Redefining language learning priorities, by putting the accent on professional training 
in the context of policies related to multilingualism in the Commission and staff mobility.  

•  Actively involving units in conducting an in-depth training needs analysis. This will 
lead  to  the development  of  language  training policies  and  criteria  for  assessing  language 
training needs and authorising participation in the language courses. 

• 
Introducing  feedback  mechanisms  to  measure  the  effectiveness  of  immediate 
language course outputs (how courses enhance new skills) and longer term outcomes (how 
new skills are used to serve the learning objectives of services and staff). 

• 
Introducing  measures  to  increase  motivation  and  commitment  of  staff  to  follow 
language  courses  until  they  reach  their  learning  objectives.  These  measures  will  also 
decrease the  important dropout rates currently observed and focus attention on advanced 
knowledge of languages which are mostly needed in the Commission. 

• 
Increasing  the  flexibility  of  the  current  training  provision  through  customisation  of 
the learning content and the possibility offered to motivated learners to catch up on missed 
time through personal effort. 

•  Developing alternative learning methods, as a means to support course customisation 
and to respond better to the different learning styles of participants. 

6) Availability of the report on Europa: 

 For internal use of the Commission services.  

Interim evaluation of the ISA programme 

1) ABB activities concerned: 26 03 

2) Timing: January 2013, (period covered: end 2010 - mid 2012) 

3) Budget: 

The overall multi-annual budget of the ISA programme is EUR 164.1million. This amount 
will  have  to  be  reviewed  at  the  light  of  the  MFF  currently  under  discussion,  which  will 
influence the expenditure for the years 2014-2015. 

4) Background, scope and focus 

By decision of the European Parliament and the Council (the ISA Decision)1 the six-year 

1  Decision No 922/2009/EC of the European Parliament and of the Council of 16 September 2009 
on  interoperability  solutions  for  European  public  administrations  (ISA),  OJ  L  260,  3.10.2009, 
p. 20. 

EN 226 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
                                                            
programme  on  interoperability  solutions  for  European  public  administrations  (the  ISA 
programme)  was  launched  on  1  January  2010  as  a  follow-up  programme  to  the  IDABC 
programme1.  The  objective  of  the  ISA  programme  is  to  support  cooperation  between 
European public administrations by facilitating the efficient and effective electronic cross-
border  and  cross-sectoral  interaction  between  such  administrations,  including  bodies 
performing  public  functions  on  their  behalf,  with  a  view  to  enabling  the  delivery  of 
electronic public services supporting the implementation of EU policies and activities2. 

The need to carry out an interim evaluation of the Programme stems from Article 13(3) of 
the  ISA  Decision.  The  evaluation  was  performed  by  the  Commission  using  a  team  of 
independent  experts  from  a  consultancy  company  (the  evaluation  team).  Representatives 
of  Commission  services  have  overseen  the  evaluation  through  a  Commission  steering 
group3. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

The  evaluation  was  largely  positive,  describing  the  ISA  programme  as  aligned  with  the 
policy  priorities  of  the  European  Commission  and  the  needs  of  Member  States  and 
implemented  efficiently  and  coherently,  delivering  results  that  are  reused  by  both 
Commission  services  and  Member  States.  Nevertheless  the  evaluation  report  also 
highlights some shortcomings and makes recommendations with regard to: 

Communication and raising awareness 

The ISA programme made important efforts last year to increase participation in national 
events  and  improve  communication  and  information  dissemination.  Further  to  these 
efforts, the Commission is revising the communication strategy and will complement the 
overall strategy by pursuing dedicated communication activities in specific areas. 

The Commission will analyse the cost-effectiveness of the evaluation recommendations at 
the light of financial perspectives for the years 2014-2015 and will increase collaboration 
with  other  stakeholders,  i.e.  other  European  institutions,  academia  and  the  ICT  industry 

1  Decision  2004/387/EC  of  the  European  Parliament  and  of  the  Council  of  21  April  2004  on 
interoperable  delivery  of  pan-European  eGovernment  services  to  public  administrations, 
businesses and citizens (IDABC), OJ L 144, 30.4.2004 (see OJ L 181, 18.5.2004, p. 25). 

2  Article 1(2). 

3   DG CONNECT, DIGIT, MARKT, SG, TAXUD. 

4   Commission  IT  Governance  bodies  are:  Comité  Technique  Informatique  (CTI),  High  Level 
Commitee  on  Information  Technologies  (HLCIT),  Activity  Based  Management  Steering 
Committee  covering  IT  (ABM+IT)  and  the  Information  Systems  Project  Management  Board 
(ISPMB). 

5   Also  addressing  one  recommendation  in  the  IDABC  final  evaluation:  a  common  ‘promotion’ 
document, focusing on the policy alignments and the synergies between the different eGovernment 
programmes, should be produced. 

6   The  "Annex  I  to  the  Commission  communication  on  interoperability  -  European  Interoperability 

Strategy (EIS)" is available at http://ec.europa.eu/isa/documents/isa_annex_i_eis_en.pdf 

7   These 

in 
http://ec.europa.eu/isa/library/index_en.htm under the "ISA Work Programme" heading. 

ISA  Work 

actions 

recent 

found 

can 

the 

be 

Programme 

at: 

EN 227 

Error! Unknown document property name.  EN 

 
                                                                                                                                                                          
accordingly. 

Engagement of stakeholders and project management continuity 

The  Commission  proposes  to  support  the  Member  States  in  implementing  this 
recommendation in various ways. 

In  addition,  the  ISA  programme  will  give  priority  to  activities  to  assess  the  ICT 
implications of EU legislation, which Member States consider to be an important issue that 
has not yet been addressed. 

The  Commission  is  closely  following  up  the  procedure  for  framework  contracts,  which 
will assure the timely launch of the required call for tenders. 

Avoiding  overlaps  and  duplications,    of  work,  increasing  reusability  and  ensuring 
sustainability 

The Commission takes a holistic approach to addressing the reusability and sustainability 
of interoperability solutions by acting at different levels: 

• 
At  governance  level,  by  improving  the  links  between  the  Commission’s  IT 
governance9, the ISA programme and through the ISA committee with the Member States. 
The  ISA  is  currently  presented  annually  to  the  CTI  and    individual  ISA  actions  led  by 
Commission services must be approved by the ISPMB; 

• 
At the strategic level, by putting more emphasis on reusability and sustainability in 
future  ISA  work  plans  and  if  necessary  by  a  possible  review  of  the  European 
Interoperability Strategy10 (EIS) 11; 

• 
At operational level, by taking action to ensure better reusability of interoperability 
solutions  and 
to  develop  sustainability  enablers  (e.g.  European  Interoperability 
Architecture  (EIA),  European  Federated  Interoperability  Repository  (EFIR),  Assessment 
of trans-European networks supporting EU policies, Sharing and reuse strategy)  12 and by 
considering possible means of financial sustainability including the proposed CEF. 

Finally,  in  line  with  the  evaluator's  recommendation,  the  ISA  unit  will  identify  actions 
producing  concrete  results  by  reviewing  ISA  solutions  every  two  years  as  specified  in 
Article 13(2) of the ISA Decision. 

No conclusions can be drawn at this at this stage in what concerns the effectiveness, utility 
and sustainability criteria since the programme is still at its early stages. 

6) Availability of the report on Europa:  

http://ec.europa.eu/isa/documents/interim_evaluation_of_the_isa_programme.pdf 

EN 228 

Error! Unknown document property name.  EN 

 
ABB 29 – Statistics  

Rolling Review for Regional Statistics 

1) ABB activities: 29 02  

2) Timing: 27 October /2012 

3) Budget: Included in the overall budget of the CSP 2008-2012 

4) Background, scope and focus: A Rolling Review is a formative evaluation of the statistical 
data, the processes involved in compiling these data, working structures and data quality. The 
Review  for  Regional  Statistics  focuses  on  a  number  of  thematic  domains  from  a  spatial 
perspective and consists of three main parts:  

• 

• 

• 

Review of user satisfaction. 

Review of partner satisfaction. 

Review of special production processes.  

Findings  are  used  to  formulate  recommendations  for  improvement  and  identify  ways  to 
implement them. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation:  The  outcomes  of  the  Review 
reveal  a  high  demand  for  regional  statistics.  The  satisfaction  with  statistical  information 
services offered by Eurostat in general is very high. The data quality aspects of accessibility, 
accuracy and comparability are considered as very good. Similarly, aspects of data coherence 
and  clarity  are  considered  as  rather  positive.  The  opinion  about  the  internal  coordination 
within  Eurostat  as  regards  requests  for  regional  data  from  the  NSIs,  harmonisation  of 
regional statistics as well  as compilations and publications in the field of regional statistics 
are very well assessed. However, the results also point out areas for future improvement. In 
particular, further enhancements are needed in provision of better access to data, familiarity 
of  partners  with  regional  statistical  programmes,  closer  cooperation  with  NSIs,  better  data 
timeliness,  completeness  and  clarity.  Notices  on  the  need  for  more  statistics  and  more 
efficient  documentation  have  also  been  registered.    With  regards  to  regional  statistics,  10 
recommendations  for  improvement  were  made.  4  of  these  relate  to  improvement  of 
dissemination  products,  3  to  improvement  of  data  qualities  and  3  in  other  fields.  As  for 
recommendations  on  special  production  processes,  some  8  suggestions  were  made  with 
majority  concerning  regional  labour  market  statistics  and  indicators  on  metro-regions  and 
urban-rural typology. 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20Regional%20Statis
tics%20Executive%20Summary%20FINAL%20to%20be%20pub.pdf 

EN 229 

Error! Unknown document property name.  EN 

 
 
Rolling Review for GISCO 

1) ABB activities: 29 02  

2) Timing: October 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus: 

GISCO  is  a  Eurostat  service  which  promotes  and  stimulates  the  use  of  Geographical 
Information  Systems  (GIS)  within  the  European  Statistical  System  and  the  Directorates-
General  of  the  Commission.  Rolling  Review  for  GISCO  is  a  part  of  the  overall  Quality 
Assurance Framework. This review was implemented between May 2011 and August 2012, 
and conducted in four parts: 

• 

Two user satisfaction surveys (for data and maps). 

Two checklists, completed by the GIS staff, assessing major steps within the provision 

• 
of geographical reference data and production of maps.  

The  results  of  these  assessments  are  used  to  formulate  recommendations  for  improvements 
and to identify ways for implementing these improvements. 

5) Summary of findings and conclusions of the evaluation:  

An increasing relevance of geographically oriented analysis for administrative and political 
decisions  was  recognised  in  the  last  years.  A  great  majority  of  respondents  stressed  the 
importance and even essentiality of georeference data. Characteristics such as  accessibility, 
quality,  clarity  and  comparability  were  generally  assessed  as  good.  Positive  results  of  user 
satisfaction are also found with quality of Eurostat maps. 

What emerged from the review is the normal tension between the needs of the users of the 
georeference  data  and  those  who  buy  and  supply  the  data.  Users  want  a  wider  range  of 
geographic data, while GISCO is more concerned with spending the limited resources to buy 
data which are required to support Eurostat’s publications (thematic maps) and geographical 
oriented applications of the Commission (projects). 

A  total  of  18  recommendations  were  provided  with  highest  priorities  concentrating  on  the 
fields  of  future  trend  for  GISCO  responsibilities  and  strengthening  geographical  data 
production and dissemination according to user demands. 

6) Availability of the report on Europa 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20GISCO%20Execut
ive%20Summary%20FINAL.pdf 

EN 230 

Error! Unknown document property name.  EN 

 
 
 
 
Rolling Review for Harmonised Indices of Consumer Prices and related price 
statistics 

1) ABB activities: 29 02  

2) Timing: April 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus: As part of Eurostat’s quality assurance framework, a rolling 
review  of  Harmonised  Indices  of  Consumer  Prices  (HICPs)  and  related  price  statistics  was 
undertaken  starting  in  the  late  Summer  of  2011  to  assess  the  quality  of  HICP  data  and  the 
statistical processes used to produce them. The survey was conducted in three parts: 

•  User survey. 

• 

• 

Partner survey. 

Self-assessment performed by the staff of Eurostat Unit C5. 

A number of recommendations for improvement were made based on the views, comments 
and concerns of users, partners and HICP staff. 

5)  Summary  of  findings  and  conclusions  of  the  evaluation:  The  HICP  and  related  price 
statistics  domain  is  a  well-established  and  well  regarded  statistical  domain  by  users, 
partners and Eurostat alike. Data quality, assessed under the criteria of accuracy, timeliness, 
accessibility,  completeness,  clarity,  comparability  and  coherence  is  rated  as  very  good  or 
good  by  users,  partners  and  the  staff,  with  all  criteria  being  of  importance.    Partners  are 
very  positive  about  Eurostat’s  role  in  the  organisation  and  functioning  of  the  HICP 
programme  and  its  direction  of  change.  Data  treatment,  validation  and  dissemination, 
publications, data confidentiality and information exchange tools are all very highly rated. 
User  responses  also  point  out  a  good  support  service  provided  by  Eurostat.  This  is 
consistent with staff evaluation of systems and processes for data production which are well 
established  and  work  well.  While  the  rolling  review  assessments  positively  endorse  the 
areas above, 14 sets of recommendation are provided with highest priorities assigned to: 

•  Compiling of detailed inventories / technical manuals of national HICP implementation 
practices and procedures. •  Reviewing compliance monitoring. 

•  Adopting a formal risk management approach for the HICP production processes at Unit 
level, coherent with Eurostat’s overall risk management strategy. 

• 

• 

Improving to the on-going development of the HICP. 

Improving bilateral communications with key stakeholders 

6) Availability of the report on Europa:  

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20HICP%20Executiv
e%20Summary.pdf 

EN 231 

Error! Unknown document property name.  EN 

 
 
Rolling Review for Postal Statistics 

1) ABB activities: 29 02  

2) Timing: August 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus : 

Rolling  Review  for  Postal  Statistics  is  part  of  the  Quality  Assurance  Framework  related  to 
streamlining  the  existing  quality  activities  in  Eurostat  and  to  position  them  in  the  wider 
framework of the European Statistics Code of Practice and Total Quality Management. The 
focus  of  this  review  is  assessment  of  postal  statistics  which  supports  the  purpose  of 
Community  policy  in  the  postal  sector.  The  importance  of  postal  services  both  for  the 
economic prosperity and social well-being and cohesion of the EU make this a priority area 
for  Community  action  and  emphasises  the  importance  of  postal  statistics.  The  review  was 
implemented during September 2011 to May 2012 and conducted in three parts: 

•  User satisfaction survey. 

• 

Partner satisfaction survey. 

•  Assessment of the production process by the professional staff within Eurostat. 

Results  are  used  to  formulate  the  actions  to  improve  the  performance  of  the  European 
Statistical System in the field. 

5) Summary of findings and conclusions of the evaluation:  

The results of user survey indicate there is a major demand for postal statistics.  

Users, in general, perceive the quality of postal data as good or adequate. However, concerns 
were raised with regards to timeliness and confidentiality. 

Partners had a favourable opinion of Eurostat’s work and were also very positive about the 
direction of change in Eurostat’s performance over the past 3 years. Concerning data quality, 
partners had a positive appreciation of the Eurostat postal statistics, aside from timeliness. 

A total of 14 recommendations were provided as the outcome of the survey. Importance was 
highlighted  to  the  possible  collection  of  European  postal  statistics  by  DG  MARKT  and 
processes related to the transfer as well as improvement of data collection 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20Postal%20Statistic
s%20Executive%20Summary%20FINAL%20VERSION_0.pdf 

EN 232 

Error! Unknown document property name.  EN 

 
 
 
 
Rolling Review for Innovation and R&D Statistics 

1) ABB activities: 29 02  

2) Timing: March 2012 

3) Budget (annual): Included in the overall budget of the CSP 2008-2012. 

4) Background, scope and focus: Rolling Review for Innovation and R&D Statistics is a part 
of  the  Quality  Assurance  Framework  aimed  at  streamlining  the  existing  quality  control 
activities  in  Eurostat.  A  review  was  undertaken  on  Innovation  statistics,  covering  both 
innovation surveys and R&D statistics. 

The  main  components  of  the  review,  which  were  implemented  between  February  and 
November 2011, were: 

•  User satisfaction survey. 

• 

• 

Partner satisfaction survey. 

Self-assessment checklist implemented by the staff of Eurostat Unit G6. 

The survey aimed to identify and give recommendations for improvement which have been 
made available for general publication. 

5) Summary of findings and conclusions of the evaluation : 

The results of user survey indicate that both R&D and Innovation statistics are essential tools 
for  majority  of  respondents.  The  data  are  mainly  used  for  research,  policy  preparation  or 
trend  analysis.  The  quality  was  considered  to  be  quite  good  with  respect  to  accuracy, 
coherence  and  relevance;  however  criteria,  such  as  accessibility,  completeness  and 
comparability  needed  further  enhancements.  Self-assessment  responses  indicate  that 
information seems to be rather complete concerning both the data and metadata. The R&D 
figures  published  by  Eurostat  are  regarded  as  being  accurate  and  the  timeliness  of 
dissemination considered satisfactory. There seems to be general partner satisfaction with the 
way  the  system  of  R&D  and  Innovation  statistics  works  within  the  European  Statistical 
System.  Two  thirds  of  countries  think  that  data  quality  has  increased  as  a  result  of  the 
introduction of the 

Regulation for R&D statistics. However, several suggestions were made for simplification of 
the statistics. 

A total of 8 recommendations were given, with improvement of Eurostat internal processes 
given the highest priority. 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/RR%20Innovation%20and
%20RD%20Statistics%20Executive%20Summary_0.pdf 

EN 233 

Error! Unknown document property name.  EN 

 
 
 
Programme for the Modernisation of European Enterprise and Trade Statistics 
(MEETS) 

1) ABB activities: 29 02  

2) Timing: November 2012 

3) Budget (annual): In total, the EU budget for the 2012 work programme is set at nearly EUR 
9 million, and it covers 17 annual actions, plus the necessary technical and administrative 
support to implement the programme.  

4) Background, scope and focus:  

The report provides an overview of the progress made by the Member States and the 

Commission (Eurostat) in implementing the activities under the MEETS Programme during 
2012. 

4 main objectives have been identified which resulted in a number of MEETS actions being 
integrated: 

•  Objective 1: To review priorities and develop target sets of indicators for new areas. 

•  Objective 2: To achieve a streamlined framework of business-related statistics. 

•  Objective  3:  To  support  the  implementation  of  a  more  efficient  method  of  producing 
enterprise and trade statistics. 

•  Objective 4: To modernise Intrastat. 

5) Summary of findings and conclusions of the evaluation:  

The activities started in 2009 and early 2010, have done sterling work and main pillars of the 
MEETS  programme  (consistency  of  legal  acts,  data  warehouse  and  data  linking  initiatives) 
are  now  well  on  track.  Whereas  in  2011  they  were  still  dealing  with  preparatory  work,  in 
2012  they  have  already  started  to  produce  practical  results.  In  addition,  financing  through 
individual  grants  has  continued,  mainly  to  accompany  progress  in  methodology  and  test 
recommendations,  but  also  to  develop  the  tools  to  make  extraction,  transmission  and 
processing of data more efficient. A number of results are already available, but most actions 
are still on-going. 

6) Availability of the report on Europa: 

http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/MEETS_0.pdf 

EN 234 

Error! Unknown document property name.  EN 

 
 
 
 
 
 
ABB 32 – Energy 

Evaluation of 5 TEN-E completed projects of European interest 

1) ABB activities concerned: 32 03  

2) Timing: April 2012, (Period covered by the report: 2007-2012) 

3) Budget: The TEN-E budget for the period 2007-2013 is EUR 150 million. 

4) Background, scope and focus  

The evaluation was conducted in the context Article 8.5 of the Decision No 1364/2006/EC of 
the EP and Council laying down guidelines for TEN-E. It stipulates that evaluation should be 
carried  out  five  years  after  the  completion  of  projects  of  European  Interest.  The  evaluation 
was conducted for 5 projects. 

The aim the evaluation was to assess the socio-economic impact, the impact on environment, 
the impact on trade between Member States and the impact on territorial cohesion. 

5) Summary of findings and conclusions of the evaluation or evaluation related study 

− 

key issues of EU value added, effectiveness and efficiency; 

The  5  TEN-E  completed  projects  provided  2  electricity  interconnections  (Finland-Estonia, 
France-Belgium and reinforcement), reinforcement of the electricity internal grid in Denmark 
and  Italy  and  one  submarine  gas  pipeline  (Greenstream)  between  Libya  and  Italy.  Projects 
were completed during the 2003-2006. 

The implementation of these projects was successful as it increased energy capacities between 
regions  in  Europe,  contributed  to  integrate  energy  markets  within  the  EU  and  allowed  for 
transportation  of  energy  from  locations  with  low  cost  production  to  the  locations  of  the 
consumers. Such integration activities generally contribute to more competitive wholesale and 
retail pricing of energy, which has the potential of decreasing overall energy costs. 

Regarding  effectiveness,  the  evaluation  showed  that  overall  the  5  projects  have  been 
successful,  although  there  are  some  variations  between  the  projects.  The  success  is  mainly 
related  to  the  increased  potential  of  exchanging  energy  between  regions  in  Europe,  with 
planned  addition  to  capacity  being  attained  in  all  projects.  Exchange  of  energy  is  a 
cornerstone of a single European energy market, as it allows for transportation of energy from 
locations  with  low  cost  production  (gas/power)  to  the  locations  of  the  consumers  (load 
centres).  This  creates  more  competitive  wholesale  and  retail  pricing  of  energy,  which 
encourages effective production of energy and lower prices for consumers. 

Significant problems or obstacles in the construction phase have only been encountered in the 
building of overhead power lines, due to their land use interference and visibility, and to the 
relative sensitivity of the topic of electromagnetic fields. This often affects the authorisation 
processes considerably where some starting dates were significantly behind schedule. 

Regarding  efficiency,  documentation  prove  that  the  interconnector  between  Finland  and 
Estonia, Estlink 1 and the Libya-Italy submarine gas pipeline, Greenstream have created very 
high  benefits  from  day  one  of  implementation.  It  had  high  capacity  factors,  i.e.  the  full 
physical capacity of the lines were used more than 65% of the time. While the utilisation of 

EN 235 

Error! Unknown document property name.  EN 

 
the  Avelin-Avelgem,  Vester  Hassing-Trige  and  S.  Fiorano  –  Nave  –  Gorlago  has  so  far 
remained  more  modest,  the  utilisation  is  expected  to  increase  in  the  near  future  as  higher 
shares of intermittent (more volatile  renewable electricity (RES-e) will lead to more demand 
for  capacity  also  for  these  lines.  Another  benefit  associated  with  the  projects  is  that  the 
strategic  security  of  supply  has  indisputably  increased  within  the  Baltic  area,  as  electricity 
now can be sourced from other regions than the region east to Baltic states (through Estlink1), 
and since the supply of gas to the European region (Italy) now stems from a new source of 
production (through Greenstream). 

information  contributing  to  modifications  in  the  management  of  the  programmes 

− 
which could contribute to improve their final impacts;  

− 
The evaluation recognised that reporting on the implementation of projects was rather 
limited, given that every two years the Commission in close collaboration with the Member 
States  has  to  present  a  report.  The  Commission  proposed  a  new  legislation  on  Energy 
Infrastructure to be formally adopted by April 2013 (entry into force foreseen by June 2013) 
where the present TEN-E guidelines will be completely revised. This will widely address the 
reporting  activities  on  projects  in  much  more  developed  way  than  the  current  TEN-E 
Guidelines. 

examples of evaluation results having contributed to the design of the successor 

− 
measures or influenced policy making; 
Permit  granting  has  been  assessed  as  one  of  the  major  element  to  delay  development  of 
infrastructure  projects  and  this  is  fully  addressed  in  the  new  legislation  on  Energy 
Infrastructure. 

− 
any limitations, constraints and difficulties (e.g. early timing of the evaluation, lack of 
data, reliance on information from MS) that prevent drawing conclusions on the issues above. 
The assessment of the five TEN-E completed projects demonstrated that access to information 
from  project  promoters  (mainly  private  companies)  five  years  after  their  completion  was  a 
challenging  exercise.  A  complete  picture  of  the  project  from  the  study  to  implementation 
phases would require the availability of data across 15 years. 

6) Availability of the report on Europa 

http://ec.europa.eu/energy/evaluations/doc/2012_ten_e_projects_final_report.pdf 

EN 236 

Error! Unknown document property name.  EN 

 
SECTION 3: REFERENCES TO REPORTING ON PERFORMANCE IN THE ANNUAL ACTIVITY 
REPORTS 2012  

- The AAR 2012 (PART 1): http://ec.europa.eu/atwork/synthesis/aar/index_en.htm  

EN 237 

Error! Unknown document property name.  EN