Special Report No 16 // 2016#EU education objectives: programmes aligned but shortcomings in performance measurement #(pursuant to Article 287(4), second subparagraph, TFEU)
Date: 2016-05-11
Institutions: European Court of Auditors
Subjects: European Social Fund, economic and social cohesion, education, education policy, level of education, operational programme, project evaluation

EN 

2016 

NO 16

Special Report  EU education objectives: 

programmes aligned  
but shortcomings in 
performance measurement

EUROPEAN
COURT
OF AUDITORS

 
EUROPEAN COURT OF AUDITORS 
12, rue Alcide De Gasperi 
1615 Luxembourg 
LUXEMBOURG

Tel. +352 4398-1
Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx
Website: eca.europa.eu
Twitter: @EUAuditorsECA

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2016

Print 
PDF 
EPUB 

ISBN 978-92-872-5187-9 
ISBN 978-92-872-5169-5 
ISBN 978-92-872-5162-6 

ISSN 1831-0834 
ISSN 1977-5679 
ISSN 1977-5679 

doi:10.2865/573605 
doi:10.2865/940216 
doi:10.2865/995337 

QJ-AB-16-016-EN-C
QJ-AB-16-016-EN-N
QJ-AB-16-016-EN-E

© European Union, 2016
Reproduction is authorised provided the source is acknowledged.

EN 

2016 

NO 16

Special Report

EU education objectives: 
programmes aligned 
but shortcomings in 
performance measurement

(pursuant to Article 287(4), second subparagraph, TFEU)

Audit team

02

The ECA’s special reports set out the results of its performance and compliance audits of specific budgetary areas or 
management topics. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks 
to performance or compliance, the level of income or spending involved, forthcoming developments and political and 
public interest.

This performance audit was produced by Audit Chamber II, headed by ECA Member Henri Grethen, which specialises in 
structural policies, transport and energy spending areas. The audit was led by the Reporting Member Iliana Ivanova, sup-
ported by Tony Murphy, Head of Private Office and Mihail Stefanov, Attaché; Alain Vansilliette, Principal Manager; Paolo 
Pesce, Head of Task; Zsuzsanna Csák, Sandra Diering, Xavier Gilquin, Zuzana Gullova, Attila Horvay-Kovács, Jean-François 
Hynderick, Thierry Lavigne and Ana Popescu, auditors.

From left to right: Z. Gullova, M. Stefanov, T. Murphy, A. Vansilliette, I. Ivanova, A. Horvay-Kovács,  
P. Pesce and Z. Csák.

Contents

03

Paragraph

Glossary

I - IX 

Executive summary

1 - 18 

Introduction

1 

The role of education

2 - 14 

Education and the EU

15 - 18 

Shared management arrangements

19 - 21  Audit scope and approach

22 - 81  Observations

22 - 52  Assessment of 2007-2013 operational programmes

24 - 38 

EU education objectives are addressed in the 2007-2013 ESF operational programmes examined

39 - 44 

Insufficient link between financial reprogramming of operational programmes and targets set

45 - 52 

Performance of examined projects could not be systematically demonstrated

53 - 81  Assessment of the establishment of operational programmes for the 2014-2020 programme 

period

55 - 61 

Commission supported Member States in establishing 2014-2020 operational programmes

62 - 68 

Examined operational programmes consistent with EU education objectives

69 - 73 

Performance monitoring arrangements improved, but with some limitations

74 - 81  Difficulties in linking education measures to employment prospects

82 - 93  Conclusions and recommendations

 
Contents 

04

Annex I  —  Evolution towards reaching the Europe 2020 education targets

Annex II  —  Overview of programming, implementation, monitoring and reporting

Annex III  —  List of the 2007-2013 ESF operational programmes examined

Annex IV  — 

 List of the 2014-2020 partnership agreements and operational programmes 
examined

Annex V  — 

 Audit methodology for assessing the objectives addressed by the operational 
programmes

Annex VI  — 

 Overview of the modifications in the 2007-2013 ESF operational programmes 
examined

Reply of the Commission

 
 
 
 
 
 
 
Glossary

05

Baseline: Baselines are intended to establish a reference value against which targets are subsequently set and 
assessed.

Early childhood education: ‘Early childhood education’ refers to children between 4 years old and the age for 
starting compulsory primary education.

Early school leavers: ‘Early school leavers’ are defined as the share of the population aged 18-24 with only lower 
secondary education or less and no longer in education or training.

Education and Training 2010 (ET 2010): Framework for European cooperation in the field of education and 
training adopted by the Council in 2003 and based on common objectives. It includes a series of indicators and 
reference levels of European average performance.

Education and Training 2020 (ET 2020): Updated strategic framework for European cooperation in education and 
training adopted by the Council in 2009. It builds on the progress made under Education and Training 2010 and 
provides common strategic objectives for the European Union and its Member States for the period up to 2020.

European Social Fund (ESF): The European Social Fund aims to strengthen economic and social cohesion within 
the EU by improving employment and job opportunities (mainly through training measures), encouraging a high 
level of employment and creating more and better jobs.

European Structural and Investment Funds (ESIF): The ESIF cover five separate funds that aim to reduce regional 
imbalances across the Union, with policy frameworks set for the 7-year multiannual financial framework. The funds 
include: the European Regional Development Fund (ERDF), the European Social Fund (ESF), the Cohesion Fund (CF), 
the European Agricultural Fund for Rural Development (EAFRD) and the European Maritime and Fisheries Fund 
(EMFF).

Europe 2020 strategy: The EU’s 10-year jobs and growth strategy. It was launched in 2010 to create the conditions 
for smart, sustainable and inclusive growth.

Ex ante conditionalities: Ex ante conditionalities are conditions, based on pre-defined criteria established in 
partnership agreements, which are regarded as necessary prerequisites for the effective and efficient use of the 
Union support covered by those agreements. When preparing ERDF, CF and ESF operational programmes under the 
2014-2020 programme period, Member States have to assess whether these conditions are fulfilled. If they have not 
been fulfilled, action plans need to be prepared to ensure fulfilment by 31 December 2016.

Impact: Longer-term socioeconomic consequences that can be observed after a certain period after the completion 
of an intervention, which may affect direct beneficiaries of the intervention or other indirect beneficiaries (e.g. 
decrease in unemployment levels, improvements in water quality).

Glossary 

06

International Standard Classification of Education (ISCED): The International Standard Classification of 
Education is used to define the levels and fields of education. The ISCED classification was initially developed 
by Unesco in the mid-1970s and was first revised in 1997. Due to subsequent changes in education and learning 
systems at the start of the 21st century, a further review of ISCED was undertaken between 2009 and 2011).

The current classification (ISCED 2011) comprises the following levels.

01

02

1

2

3

4

5

6

7

8

Early childhood educational development

Pre-primary education

Primary education

Lower secondary education

Upper secondary education

Post-secondary non-tertiary education

Short-cycle tertiary education

Bachelor’s or equivalent level

Master’s or equivalent level

Doctoral or equivalent level

n
o
i
t
a
c
u
d
e

y
r
a
i
t
r
e
T

Lifelong learning: For the purpose of this report, ‘lifelong learning’ refers to the percentage of population aged 
25-64 participating in education and training in the 4 weeks prior to the survey.

Low achievers in basic skills: ‘Low achievers in basic skills’ refers to the share of low-achieving 15-year-olds in 
reading, mathematics and science.

Managing authority: A managing authority is a national, regional or local public authority (or any other public 
or private body) which has been designated by a Member State to manage an operational programme. Its 
tasks include selecting projects to be funded, monitoring how projects are implemented and reporting to the 
Commission on financial aspects and results achieved.

Mathematics, science and technology: ‘Mathematics, science and technology’ refers to the increase in the total 
number of tertiary graduates from the mathematics, science and technology fields.

Milestone: Intermediate targets set for indicators for which a target value has been set.

Operational programme (OP): An operational programme sets out a Member State’s priorities and specific 
objectives, and how the funding (EU and national public and private co-financing) will be used during a given 
period (generally 7 years) to finance projects. These projects must contribute towards achieving a certain number 
of objectives specified within the OP’s priority axis. OPs exist for each of the funds in the cohesion area (i.e. ERDF, CF 
and ESF). An OP is prepared by the Member State and has to be approved by the Commission before any payments 
from the EU budget can be made. OPs can only be modified during the period covered if both parties agree.

 
Glossary 

07

Outcome: A change that arises from an intervention, normally relating to its objectives (e.g. trainees who have 
found employment, decrease in pollutants in treated waste water, decrease in travel time). Outcomes may be 
expected or unexpected, positive or negative.

Output: Something that is produced or accomplished with the resources allocated to an intervention (e.g. training 
courses delivered to unemployed young people, number of sewage plants or km of roads built).

Partnership agreement: Agreements entered into between the European Commission and each Member State 
for the 2014-2020 programme period. They set out the national authorities’ plans on how to use funding from the 
European Structural and Investment Funds and outline each country’s strategic goals and investment priorities, 
linking them to the overall aims of the Europe 2020 strategy for smart, sustainable and inclusive growth. They also 
include, among other things, details of any ex ante conditionalities and performance management frameworks. 
They are prepared by the Member State in consultation with the Commission and must be adopted by the 
Commission.

Programme period: The multiannual framework within which Structural Funds and Cohesion Fund expenditure is 
planned and implemented.

Tertiary education attainment: ‘Tertiary education attainment’ refers to the percentage of those aged 30-34 who 
have successfully completed tertiary level education.

Upper secondary education: For the purpose of this report, ‘upper secondary education’ refers to the percentage 
of those aged 22 who have successfully completed at least upper secondary education.

Executive  
summary

08

I
Investment in education is crucial for improving labour productivity, professional development and ultimately eco-
nomic growth in the European Union. Other benefits of education may be more active citizenship, reduced crime 
and improved average health conditions. Moreover, the level of education attained has a direct link both to employ-
ability and to the quality of employment obtained. EU unemployment in recent years has been three times lower for 
those with tertiary education compared to those with lower secondary or less.

II
Within the EU, education policy is a full and exclusive competence of Member States. EU strategies are adopted in 
the form of Council conclusions, which are not legally binding for EU Member States. However, the Union has the 
competence to support, coordinate or supplement the actions of the Member States in the area of education.

III
Eurostat estimates that while 3 200 billion euro was spent on education in the EU Member States between 
2007 and 2011, the overall level of expenditure on education slightly decreased between 2009 and 2013 in most of 
them.

IV
While the amount of EU funding directly allocated to education is limited in comparison to national investment in 
this field, the ESF nevertheless contributed 33.7 billion euro to education measures over the 2007-2013 programme 
period. For the 2014-2020 programme period, planned ESF allocations to education are significantly lower (with an 
estimated budget of up to 27.1 billion euro).

V
The EU’s education objectives are outlined in the Education and Training 2010, Education and Training 2020 and 
Europe 2020 strategic frameworks. Our audit assessed whether the objectives had been adequately considered in 
the operational programmes (OPs) and related projects for 2007-2013 and in the design of OPs for the 2014-2020 
programme period.

VI
During the audit we reviewed 37 ESF OPs from the 2007-2013 programme period, which represent about 28 bil-
lion euro or 83 % of the total ESF allocation to education. In addition, we visited five Member States’ authorities 
(Germany, France, Italy, Portugal and Romania) and examined a sample of projects. For the 2014-2020 programme 
period, we reviewed five partnership agreements and eight OPs, which represent about 7 billion euro of the ESF 
allocation to education.

Executive summary  

09

VII
Overall, we concluded that EU education objectives have been adequately considered in the 2007-2013 OPs. How-
ever, for some OPs, there were elements of the intervention logic which were not described, and there were short-
comings in the monitoring tools framework. In addition, the performance of audited projects could not be system-
atically demonstrated due to a lack of quantified objectives and performance indicators. For the majority of the OPs 
reviewed which were subject to a change in the financial allocation, clear explanations about the implications of the 
change of the financial allocations for the related target values were missing.

VIII
Regarding the design of 2014-2020 OPs, we found that EU education objectives have been adequately considered, 
with improvements in the description of the intervention logic. However, there are still some weaknesses in the 
framework which may have an impact on performance monitoring and reporting at OP and project level. In addi-
tion, there is not always a clear link between education measures and their impact on employability.

IX
We recommend that:

(a)  During the preparation of the next programme period, the Commission should consider specifying the result 
indicator(s) on outcomes for which baseline and target values should be set for each investment priority.

(b)  For the 2014-2020 programme period:

(i)  Where modifications to OPs are requested by Member States, the Commission should:

 ο

 ο

encourage the establishment of a clear link between OPs’ investment priorities and appropriate quanti-
fied and measurable result indicators;

ensure that Member States provide explanations for the reprogramming of financial allocations, in-
cluding both qualitative and quantitative information on the expected change in output and results 
indicators.

(ii)  Member States should ensure that:

 ο

 ο

there is a clear link between the selected projects and the achievement of EU education objectives em-
bedded in the OP;

appropriate result indicators are put in place to demonstrate the actual effects of the project on the 
final participants in a systematic manner and to monitor progress towards achieving the OPs’ education 
objectives.

(iii)  Where relevant, the Commission and the Member States should better target OPs’ funding on measures 

which reinforce the link between education and employment, and should ensure that outcomes are moni-
tored appropriately.

Introduction

10

The role of education

01 
Investment in education is crucial for improving labour productivity, professional 
development and, ultimately, economic growth in the European Union. Other 
benefits of education may be more active citizenship, reduced crime and im-
proved average health conditions1. A person’s level of education has a direct link 
both to employability and to the quality of job obtained. It is related to individual 
earnings and even more closely to unemployment levels. As shown in Figure 1, 
EU unemployment in recent years has been three times lower for those with ter-
tiary education compared to those with lower secondary or less.

1 

European Expert Network on 
Economics of Education 
(EENEE), ‘The economic case 
for education’, December 2014 
and OECD, ‘Education at 
a glance 2015 — OECD 
indicators’, 2015.

1 Unemployment rate by educational attainment in the EU
e
r
u
g
i
F

25 %

l

)
d
o
s
r
a
e
y
4
6
-
5
1
(
e
t
a
r

t
n
e
m
y
o
p
m
e
n
U

l

20 %

15 %

10 %

5 %

0 %

2002

2004

2006

2008

2010

2012

2014

Lower secondary education or less
Average
Upper secondary education
Tertiary education 

Source: Eurostat.

 
 
 
 
 
11

2  Preamble of the Treaty on the 
Functioning of the European 
Union (TFEU).

3 

4 

5 

TFEU, Article 6.

TFEU, Article 165.

European Council of 
23-24 March 2000.

6  Council conclusions of 

5 May 2003 on reference levels 
of European average 
performance in education and 
training (Benchmarks) (OJ 
C 134, 7.6.2003, p. 3), further to 
COM(2002) 629 final of 
20 November 2002 ‘European 
benchmarks in education and 
training: follow-up of the 
Lisbon European Council’.

Introduction 

Education and the EU

02 
One of the founding principles of the EU is promoting knowledge by giving EU 
citizens ‘wide access to education and through its continuous updating’2. Within 
the EU, education policies are a full and exclusive competence of Member States. 
EU strategies are adopted in the form of Council conclusions, which are not 
legally binding for EU Member States (‘soft law’). These conclusions are political 
statements by the Council that lay the foundations for cooperation between two 
or more Member States in a defined area.

03 
The EU has competencies to support, coordinate or supplement actions by the 
Member States in the area of education3. In addition, ‘The Union shall contribute 
to the development of quality education by encouraging cooperation between 
Member States and, if necessary, by supporting and supplementing their action, 
while fully respecting the responsibility of the Member States for the content 
of teaching and the organisation of education systems and their cultural and 
linguistic diversity’4.

04 
In 2000, the Heads of State or Government approved the Lisbon strategy5, 
which had the aim of making the EU ‘the most competitive and dynamic knowl-
edge-based economy in the world capable of sustainable economic growth with 
more and better jobs and greater social cohesion’ by 2010.

05 
Following the Lisbon strategy, the Council had agreed6 to establish a framework 
for European cooperation in the field of education and training, based on com-
mon objectives (known as ‘Education and Training 2010’ – ET 2010). This included 
a series of indicators and reference levels of European average performance in 
education to be used as one of the tools for monitoring the achievement of ob-
jectives for education and training systems in Europe.

12

7  Council Conclusions of 

12 May 2009 on a strategic 
framework for European 
cooperation in education and 
training (‘ET 2020’) (OJ C 119, 
28.5.2009, p. 2).

8 

European Council of 
17 June 2010, Conclusions 
further to COM(2010) 2020 
final of 3 March 2010 ‘EUROPE 
2020 A strategy for smart, 
sustainable and inclusive 
growth’.

Introduction 

06 
In 2009, the Council adopted7 an updated strategic framework for European 
cooperation in education and training known as ‘Education and Training 2020’ – 
ET 2020, building on the progress made under the ‘ET 2010’.

07 
In 2010, the Europe 2020 strategy8 for jobs and smart, sustainable and inclusive 
growth was adopted by the Council. It identifies five measurable EU headline 
targets for 2020 that were translated into national targets and trajectories. These 
targets encompass education objectives.

08 
The EU education objectives have remained relatively stable and prominent on 
the EU agenda. Based on the above-mentioned strategic documents, the EU edu-
cation objectives considered for the purpose of this report are:

 ο

reduction of the number of early school leavers;

 ο

increase of tertiary education attainment;

 ο

increase of the participation in lifelong learning;

 ο

reduction of the number of low achievers in basic skills;

 ο

increase of the participation in early childhood education;

 ο

increase of the participation in higher education in the field of mathematics, 
science and technology; and

 ο

increase of upper secondary education attainment.

Table 1 presents further details and the targets set at European level.

09 
While it is difficult to establish a direct correlation with the impact of EU support, 
significant progress has been made in recent years towards achieving the Europe 
2020 education objectives. However, efforts are still needed to reach the targets 
set. Despite the fact that the EU average is close to the Europe 2020 education 
targets, some Member States are still lagging behind in achieving their national 
targets.

Introduction 

13

1
e
l
b
a
T

EU objectives in the field of education

Area covered

Education and Training 2010
May 2003

Education and Training 2020
May 2009

Europe 2020
June 2010

Objectives to be reached by 2010 at EU level

Objectives to be reached by 2020 at EU level

Early school leavers

The share of early leavers from education and 
training should be less than 10 %

The share of early leavers from education and training should be less than 
10 %

Tertiary education 
attainment
(ISCED 5-8)

Lifelong learning

Not mentioned

The share of 30-34 year old with tertiary educational attainment should be at 
least 40 %

At least 12.5 % of adults (25-64 years old) should 
participate to lifelong learning

At least 15 % of adults (25-64 years old) should 
participate to lifelong learning

Low achievers in basic 
skills

The percentage of low-achieving 15 year olds 
in reading literacy should have decreased by at 
least 20 % compared to 2000

The share of low-achieving 15-year olds in 
reading, mathematics and science should be less 
than 15 %

Early childhood 
education (ISCED 0)

Not mentioned

Mathematics, science 
and technology

Upper secondary 
education attainment
(ISCED 3)

The total number of graduates in mathemat-
ics, science and technology should increase by 
at least 15 % (from 2003 to 2010), while at the 
same time, the level of gender imbalance should 
decrease

At least 85 % of 22 year olds should have com-
pleted upper secondary education

At least 95 % of children between 4 years old 
and the age for starting compulsory primary 
education should participate in early childhood 
education

No new targets set

No new targets set

 
Introduction 

14

10 
Figures 2 and 3, for the visited Member States9, show the trend in the reduction 
of the number of early school leavers and in the increase in tertiary education at-
tainment. In Portugal, a 2009 reform made it compulsory to remain in education 
until the age of 18, the age at which students normally attain an upper second-
ary education qualification. This has contributed significantly to the reduction in 
early school leaving.

9  Germany, France, Italy, 
Portugal and Romania.

2
e
r
u
g
i
F

3
e
r
u
g
i
F

Early school leavers in the EU1

45 %

40 %

35 %

30 %

25 %

20 %

15 %

10 %

 5 %

0 %

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

Tertiary education attainment in the EU1

50 %
45 %
40 %
35 %
30 %
25 %
20 %
15 %
10 %
5 %
0 %

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

1  EU target is not necessarily the same as the specific national targets set.
Source: Eurostat.

EU (28 countries)
Germany
France
Italy
Portugal
Romania

EU target: below 10 %

EU target: above 40 %

EU (28 countries)
Germany
France
Italy
Portugal
Romania

 
 
Introduction 

15

11 
For information purposes, the progress made during the period 2006-2015 by 
all 28 Member States towards reaching these Europe 2020 targets is shown in 
Annex I.

10  European Commission, 
‘Education and Training 
Monitor 2014’, p. 14 and 
‘Education and Training 
Monitor 2015’, p. 25.

12 
Education is one of the biggest expenditure items in the Member States’ national 
budgets. Overall, Eurostat, the Commission’s statistical service, estimates that 
3 200 billion euro were spent on education by the EU Member States between 
2007 and 2011.

13 
Despite various calls from the Commission for Member States to protect invest-
ment in growth-friendly policies, such as education, the level of expenditure on 
education, as a share of gross domestic product, decreased slightly between 
2009 and 201310 in most Member States (see Table 2).

14 
While the amount of EU funding directly allocated to education is limited in com-
parison to national investments in this field, the ESF nevertheless contributed 
33.7 billion euro to measures in the area of education throughout the 2007-2013 
programme period. For the 2014-2020 programme period, planned ESF alloca-
tions to education are significantly lower (with an estimated budget of up to 
27.1 billion euro).

Introduction 

16

EU-28 Member States — Education expenditures as a share of gross domestic 
product

2
e
l
b
a
T

2009

2010

2011

2012

2013

EU (28 countries)

Belgium

Bulgaria

Czech Republic

Denmark

Germany

Estonia

Ireland

Greece

Spain

France

Croatia

Italy

Cyprus

Latvia

Lithuania

Luxembourg

Hungary

Malta

Netherlands

Austria

Poland

Portugal

Romania

Slovenia

Slovakia

Finland

Sweden

United Kingdom

Source: Eurostat.

5.5

6.2

4.3

4.8

8.0

4.4

7.1

5.4

4.3

5.1

6.2

-

4.6

7.2

6.8

6.8

5.3

5.3

5.4

5.9

5.7

5.6

6.8

4.1

6.5

4.3

6.6

7.2

6.9

5.3

6.1

3.7

5.1

7.2

4.4

6.6

4.9

4.0

4.5

5.6

5.0

4.4

6.8

6.2

6.4

5.2

5.6

5.6

5.7

5.1

5.6

7.7

3.3

6.6

4.9

6.6

6.5

6.6

5.1

6.3

3.5

5.1

6.9

4.3

6.3

4.7

4.4

4.4

5.5

5.0

4.1

6.6

5.9

6.1

5.1

5.1

5.7

5.6

5.0

5.5

7.3

4.1

6.6

4.6

6.5

6.5

6.0

5.0

6.3

3.4

5.0

7.1

4.3

6.3

4.4

4.4

4.1

5.5

5.0

4.1

6.1

5.7

5.8

5.4

4.7

5.8

5.6

5.0

5.4

6.5

3.0

6.4

4.4

6.4

6.5

5.8

5.0

6.4

3.8

5.2

7.0

4.3

6.0

4.1

4.5

4.0

5.5

5.3

4.1

6.5

5.7

5.6

5.6

4.7

5.9

5.5

5.0

5.3

6.8

2.8

6.5

5.0

6.5

6.6

5.5

 
17

11  The set-up is completed by 

a certifying authority, whose 
function is to certify to the 
Commission that the 
expenditure being declared 
for reimbursement is accurate, 
results from a reliable 
accounting system and is 
compliant with applicable EU 
and national rules. An audit 
authority audits the 
effectiveness of the control 
systems and is responsible for 
the closure declaration and 
the report to the Commission.

12  Regulation (EU) No 1303/2013 
of the European Parliament 
and of the Council of 
17 December 2013 laying 
down common provisions on 
the European Regional 
Development Fund, the 
European Social Fund, the 
Cohesion Fund, the European 
Agricultural Fund for Rural 
Development and the 
European Maritime and 
Fisheries Fund and laying 
down general provisions on 
the European Regional 
Development Fund, the 
European Social Fund, the 
Cohesion Fund and the 
European Maritime and 
Fisheries Fund and repealing 
Council Regulation (EC)  No 
1083/2006 (OJ L 347, 
20.12.2013, p. 320).

Introduction 

Shared management arrangements

15 
The ESF is implemented by the Commission and the Member States through 
shared management. Member States prepare multiannual OPs that are negoti-
ated with the Commission and adopted in the form of Commission decisions. An 
OP is a strategic document that analyses the problems and needs of the geo-
graphical area covered, identifies the groups that need assistance, sets the priori-
ties of the intervention and the related budget and sets the goals to be achieved.

16 
To manage and implement the OPs, the Member States designate managing 
authorities to ensure that their implementation is effective, legal and regular, 
particularly as regards the selection of projects to be funded, the monitoring of 
their implementation and the reporting to the Commission on financial aspects 
and results achieved. The managing authorities can decide to delegate part of 
the implementation work to one or more intermediate bodies. The projects are 
carried out by beneficiaries. Beneficiaries have to provide monitoring informa-
tion on project progress to managing authorities, which have to send aggregate 
information to the Commission in the form of annual implementation reports11.

17 
For the 2014-2020 programme period, changes have been introduced for estab-
lishing the OPs12, and in particular the partnership agreements and the ex ante 
conditionalities. Partnership agreements are contracts between the Commission 
and the Member States, and replace the national strategic reference frameworks 
that were in place during the 2007-2013 programme period. The partnership 
agreements present an overarching strategic plan with priorities for investment 
for each of the Member States. They are intended to ensure that the use of the 
European Structural and Investment Funds in the Member States is aligned with 
the Europe 2020 strategy, and to put in place arrangements to ensure that funds 
are used effectively. Partnership agreements are then broken down into OPs. Ex 
ante conditionalities are conditions which are regarded as necessary prerequi-
sites for the effective and efficient use of the Union support. Where these condi-
tions are not fulfilled, actions plans need to be prepared by Member States to 
ensure they are fulfilled by 31 December 2016.

18 
Annex II sets out, in simplified terms, the main aspects of the programming, 
implementation and reporting processes for ESF actions.

Audit scope and approach

18

19 
In this audit, we assessed whether EU education objectives had been adequately 
considered in the OPs and related projects for 2007-2013 and in the design of 
OPs for the 2014-2020 programme period. It was not possible to select specific 
projects for review for the period 2014-2020 because it was still too early in the 
programme period.

13  Directorate-General for 

Employment, Social Affairs 
and Inclusion and 
Directorate-General for 
Education and Culture.

20 
In particular, we assessed whether:

(a)  EU education objectives had been adequately addressed in the 2007-2013 ESF 

OPs and projects;

(b)  the Commission had supported Member States during the establishment 

of 2014-2020 OPs with a view to improving them compared to the previous 
programme period.

21 
The audit was based on:

 ο

interviews with Commission officials of directorates-general13;

 ο

an analytical review of relevant EU and national documentation;

 ο

a desk review of 37 ESF OPs from the 2007-2013 programme period covering 
15 Member States (see Annex III), and of five partnership agreements and 
eight OPs (adopted between October 2014 and February 2015) for the 2014-
2020 programme period covering five Member States: Germany, France, Italy, 
Portugal and Romania (see Annex IV). The detailed audit methodology used 
for the analysis is described in Annex V;

 ο

visits to the abovementioned five Member States’ authorities;

 ο

visits to beneficiaries of 15 projects from the 2007-2013 programme period.

Observations

19

Assessment of 2007-2013 operational programmes

22 
The EU education objectives are essential elements in helping the EU become the 
most competitive and dynamic knowledge-based economy in the world. Each 
Member State has its own specific needs and focuses its investment priorities ac-
cordingly in the relevant OPs submitted to the Commission for approval.

14  Although the objective of 
improving vocational 
education and training was 
not one of the EU education 
objectives defined by ET 2010, 
it had already been included 
by the Member States in 10 of 
the 37 2007-2013 ESF OPs 
examined and was therefore 
included in the analysis.

23 
We assessed whether:

 ο

 ο

 ο

the 2007-2013 ESF OPs had adequately addressed the EU education 
objectives14;

any modifications had been made to the OPs during the programme period 
following the adoption of the ET 2020 and Europe 2020 strategies in 2009 
and in 2010 respectively;

it could be demonstrated that the sampled projects had contributed to 
achieving education objectives.

EU education objectives are addressed in the 2007-2013 ESF 
operational programmes examined

24 
Where OPs include specific education objectives, we consider that the related 
intervention logic and the tools for monitoring performance should be outlined 
and described.

25 
We carried out an analysis of 37 ESF OPs for the 2007-2013 programme period in 
order to assess the extent to which these criteria had been outlined.

Observations 

20

26 
The initial budget of the OPs reviewed totalled 87.38 billion euro, of which 
59.41 billion euro were allocated from the ESF. The ESF allocation to education is 
28.4 billion euro, i.e. 48 % of the total ESF support. The examined OPs represent 
about 83 % of the total ESF allocation to education for the 2007-2013 programme 
period, which totalled to 33.7 billion euro (see paragraph 14).

15  Belgian Flanders OP did not 
specifically address any EU 
education objective, but 
nonetheless received ESF 
funds allocations under the 
education priority codes.

27 
Our analysis of the OPs examined showed that they generally included educa-
tion objectives15. However, for some OPs, there were elements of the intervention 
logic which were not described and/or there were other shortcomings in the 
monitoring tools framework which are further detailed in the following two sec-
tions. An overview of the shortcomings is presented in Table 3.

Assessment of 2007-2013 ESF OPs: EU education objectives, intervention logic and 
tools for monitoring

3
e
l
b
a
T

Objectives

OPs that 
included the 
EU objectives 
(number and 
percentage of 
OPs examined)

Intervention logic

Tools for monitoring

Without any 
description of 
context
(%)

Without any 
needs analysis
(%)

Without any ex-
ample of actions 
to achieve the 
objective
(%)

Without any 
quantification of 
the objective
(%)

Without any indica-
tor for measuring 
the achievement of 
the objective
(%)

Early school leavers

28 (76 %)

Tertiary education 
attainment

Lifelong learning

Low achievers in 
basic skills

Early childhood 
education

Mathematics, 
 sciences and 
technology

Upper secondary 
education attainment

Vocational education 
and training

20 (54 %)

35 (95 %)

11 (30 %)

4

5

11

27

29

20

17

27

4 (11 %)

none

none

18 (49 %)

25 (68 %)

10 (27 %)

151 cases1

33

4

10

12

44

28

10

22

none

10

none

36

25

17

8

10

13

11

40

none

36

50

22

20

none

22

14

25

none

45

50

22

12

none

21

1  The total number of cases is the sum of all the identified education objectives in all 37 OPs examined.

 
Observations 

21

Clear intervention logic was not always presented for the chosen 
objectives

16  Article 96.2.b(iii) of Regulation 

(EU) No 1303/2013.

28 
OPs are strategic documents which support EU intervention. They cannot be 
expected to contain exhaustive information. However, as the Commission has to 
assess and approve the OPs, we expect them to provide appropriate information, 
or at least references to other national documents, concerning the logic behind 
the EU intervention: description of the context, identification of the needs and of 
the actions necessary to meet the needs and remedy the initial weaknesses. This 
is the basis for allocating resources in an effective and efficient way.

29 
Each time an EU education objective had been included in an examined OP, we 
assessed whether a clear intervention logic had been described. This logic should 
include a sound description of the context, followed by a detailed needs analysis, 
identifying the appropriate actions required. While the applicable regulation for 
the 2007-2013 programme period did not require Member States to list their in-
tended actions, this is considered good practice and was introduced as a require-
ment for the 2014-2020 programme period16. Hence, we considered this good 
practice as a criterion for assessing the 2007-2013 ESF OPs. Table 3 summarises 
the cases in which the assessment showed that neither the OPs nor the related 
documents had presented the necessary information.

30 
Most of the OPs examined had presented a description of the context with 
qualitative and quantitative data or at least some minimum level of information. 
However, in 12 % of the cases, the OPs did not provide any information concern-
ing the context, i.e. the situation before the programme’s implementation. The 
absence of any description of the context implies that the needs can be identi-
fied only in general terms and therefore prevents the development of a clear 
strategy. Moreover, we identified differences in the quality of the information 
provided by the OPs and related documents within a Member State (see Box 1).

1
x
o
B

Example of differences in the information provided by the OPs and related 
documents within a Member State

In Italy, four ESF OPs included the objective of reducing the number of early school leavers. Among them, 
three OPs provided qualitative and quantitative information concerning the situation before the programmes’ 
implementation, while one OP did not present an adequate level of information in this regard.

 
Observations 

31 
Table 3 indicates that a needs analysis was found to be lacking in 22 % of the 
cases. Our analysis also found that while a needs analysis with qualitative or 
quantitative data was present in about a fifth of the cases, almost half of these 
referred only to the need to achieve the EU target without a detailed underlying 
analysis.

32 
Finally, we found that most of the OPs examined provided examples of actions 
planned to achieve the objectives. However, in 13 % of the cases, the OPs did 
not provide any example of actions that could be taken to achieve the related 
objective.

Inadequate tools for monitoring performance

Absence of outcome indicators and baselines, milestones and targets 
hampers effective monitoring and evaluation of the overall ESF 
performance

33 
We have previously highlighted several times in other reports17 that the Commis-
sion was in a difficult position to adequately monitor overall ESF performance 
during the 2007-2013 programme period. The 2007-2013 regulatory framework 
required Member States to define specific output and result indicators on 
outcomes as well as quantitative targets at the level of priority axes for each OP. 
However, in relation to common indicators it was only mandatory to report the 
ones in relation to participants and/or participations18. During the 2007-2013 
period, there were no common result indicators on outcomes required, which 
creates problems for aggregating the monitoring information on results. Weak-
nesses in monitoring also have an impact on the capacity of the Commission and 
Member States to carry out evaluations.

22

17  Special Report No 3/2000 on 

the European Social Fund and 
the European Agricultural 
Guidance and Guarantee Fund 
(Guidance section) — 
Measures to assist the 
employment of young 
persons (OJ C 100, 7.4.2000); 
Special Report No 12/2001 
concerning certain structural 
measures to improve the 
employment situation: the 
impact of ERDF aid on 
employment and ESF 
measures to combat 
long-term unemployment (OJ 
C 334, 28.11.2001); Special 
Report No 1/2006 on the 
contribution of the European 
Social Fund in combating early 
school leaving (OJ C 99, 
26.4.2006); Special Report 
No 17/2009 ‘Vocational 
training actions for women 
co-financed by the European 
Social Fund’ and Special 
Report No 25/2012 ‘Are tools in 
place to monitor the 
effectiveness of European 
Social Fund spending on older 
workers?’ (http.//eca.europa.
eu).

18  Annex XXIII to Commission 

Regulation (EC) No 1828/2006 
of 8 December 2006 setting 
out rules for the 
implementation of Council 
Regulation (EC) No 1083/2006 
laying down general 
provisions on the European 
Regional Development Fund, 
the European Social Fund and 
the Cohesion Fund and of 
Regulation (EC) No 1080/2006 
of the European Parliament 
and of the Council on the 
European Regional 
Development Fund (OJ L 371, 
27.12.2006, p. 1).

Observations 

23

34 
In its 2013 strategic report, the Commission states that while the 2007-2013 OPs 
had strong mechanisms for tracking the absorption of funds, the mechanisms 
for setting, monitoring and evaluating objectives were weaker19. In addition, the 
final synthesis report by external Expert Evaluation Network on the main ESF 
achievements of the 2007-2013 programme period highlighted that ‘although 
comprehensive data are available on outputs for each of the Member States, the 
position for results is much less favourable. Even for three key results — job entry, 
qualifications achieved and self-employment — it was not possible to obtain the 
absolute numbers on a consistent basis for a number of Member States. In the ab-
sence of absolute numbers it is not possible to generate aggregate figures across 
Member States for the results associated with ESF supported interventions’20.

19  COM(2013) 210 final of 

18 April 2013 ‘Report from the 
Commission to the European 
Parliament, the Council, the 
European Economic and Social 
Committee and the 
Committee of the Regions —
Cohesion policy: Strategic 
report 2013 on programme 
implementation 2007-2013’, 
p. 12.

20  ESF Expert Evaluation 

Network — Final synthesis 
report: Main ESF 
achievements, 2007-2013, 
26.3.2014, p. 6.

35 
Our analysis of the examined OPs considered two aspects relating to monitoring 
tools: whether there was any quantification of the objective and whether any 
indicator had been set for measuring the achievement of the objective. Table 3 
shows that 22 % of the cases did not have any quantification of the objective and 
21 % did not include any indicator for measuring the achievement of the objec-
tive. Our analysis also found that around half of the OPs presented both output 
and outcome indicators, with baselines and target values, while in 31 % of cases, 
one of the two types of indicators was missing.

36 
Baselines, milestones and targets are essential to efficiently assess and moni-
tor the performance of the OPs. They provide the management with the tools 
required to properly assess the extent to which the objectives are being met, and 
make it possible to initiate corrective measures including reprogramming fin-
ancial resources in the event of inadequate performance. The fact that goals are 
not set in measurable terms hampers the evaluation and reporting of the ESF’s 
performance.

Data on participants in ESF measures are likely to be overstated since 
reporting is based on the number of participations

37 
Among the output indicators, the most commonly available one is that related 
to number of participants. This output indicator allows the managing authorities 
to monitor the number of participants/participations in a measure, but does not 
provide information concerning the results achieved and/or the impact on the 
participants’ situations.

Observations 

24

38 
Moreover, this indicator does not provide a clear distinction between individual 
participants and participation. The 2014 annual activity report of the Director-
ate-General for Employment, Social Affairs and Inclusion mentions an average of 
11.2 million participants per year for the 2007-2013 programme period. However, 
it also specifies that these figures are at risk of being overstated because ‘Mem-
ber States report on the number of participations to ESF projects and therefore 
one participant could take part in several actions’21.

21 

 Directorate-General for 
Employment, Social Affairs 
and Inclusion, 2014 Annual 
activity report 
— Ares(2015)1425867.

22  OP No 2007SK05UPO001 — 

Operačný program 
Vzdelávanie.

Insufficient link between financial reprogramming of 
operational programmes and targets set

39 
Member States should consider whether there is a need to review the initially es-
tablished OP objectives in the light of an updated EU education strategic policy. 
Where this is the case, there should be a reassessment of the targets previously 
set and the required financial allocations.

40 
An updated strategic framework on education was adopted by the Council dur-
ing the 2007-2013 programme period (see paragraphs 6 and 7), and although it 
was not mandatory to modify OPs in that regard, we checked whether the 37 OPs 
reviewed needed to be updated accordingly.

41 
While 35 out of the 37 OPs examined had been modified throughout the pro-
gramme period, only two included modifications concerning education ob-
jectives to ensure that the OP was aligned with the Europe 2020 strategy (see 
Annex VI). However, our assessment showed that, in general, the education ob-
jectives included in the OPs examined had already been in line with the updated 
EU education objectives.

42 
Indicators, or their target values (and in some cases the baselines), related to the 
education objectives, had been modified in 27 OPs and the majority of these 
had their financial allocation to education modified. For some OPs, information 
concerning the reprogramming including resetting of targets was available. For 
example, the Slovak OP22 provided a detailed analysis, including the formula for 
calculating the target values of indicators. However, for the majority of the modi-
fied OPs, clear explanations about the implications of the change in the financial 
allocations for the related target values were missing.

Observations 

25

43 
For two OPs in Italy, the financial allocations to education were reduced by 9 mil-
lion euro and 30 million euro (respectively 8 % and 10 % of the initial budget), 
without either modifying the indicators and their target values or providing 
an adequate justification23. For the second OP for which the initial budget was 
reduced by 10 % (Campania), almost all of the initial targets had already been 
overachieved at the time of the modification of the OP, having spent only around 
half of the initial budget. This illustrates that limited attention was given to per-
formance aspects.

44 
Where required, not updating the target values when modifying the financial 
allocations or updating them inconsistently weakens the validity of the indica-
tors and their usefulness. Target values, if not carefully established, impede the 
assessment of the effectiveness of a programme.

23  OP No 2007IT051PO001 — 
Programma operativo 
obiettivo convergenza fondo 
sociale europeo 2007-2013 
regione Campania and 
OP No 2007IT052PO004 — 
Programma operativo del 
fondo sociale europeo 
obiettivo competitività 
regionale e occupazione 
regione Lazio 2007-2013.

24  Article 60 of Council 

Regulation (EC) No 1083/2006 
of 11 July 2006 laying down 
general provisions on the 
European Regional 
Development Fund, the 
European Social Fund and the 
Cohesion Fund and repealing 
Regulation (EC) No 1260/1999 
(OJ L 210, 31.7.2006, p. 25).

Performance of examined projects could not be 
systematically demonstrated

45 
Projects supported through ESF OPs are expected to deliver results which fit into 
the OP strategy designed to reach one of the EU education objectives. Respon-
sibility for the sound financial management of co-financed activities rests with 
the managing authority of each OP24. The managing authorities should first make 
sure that the grant applications include relevant quantified objectives and indica-
tors and assess the adherence of the project to the OP strategy and the adequacy 
of the likely results.

46 
For the purpose of the audit, we examined 15 projects, i.e. three projects in each 
of the five Member States visited (see paragraph 21), that, according to the man-
aging authorities, had contributed positively to achieving at least one EU educa-
tion objective. For each project, we examined to what extent it could be demon-
strated that the results achieved had contributed to the education objectives set 
out in the OP. Figure 4 provides additional information on the location, scope, 
cost and participation in the projects examined.

Observations 

26

4
e
r
u
g
i
F

Overview of ESF projects examined

City: Paris
Education objective: Training for 
adults with poor basic skills
Total cost: 1.3 million euro
No of participants: 1 014

City: Paris
Education objective: Training of 
low qualified workers
Total cost: 7.8 million euro
No of participants: 448

City: Créteil 
Education objective: Prevention 
of early school leaving
Total cost: 1.4 million euro
No of participants: 2 294

City: Lousado
Education objective: Training for
workers
Total cost: 0.6 million euro
No of participants: 1 240

City: Vila Nova de Famalicão
Education objective: Prevention of 
early school leaving through 
development of personal and 
professional competences
Total cost: 1.7 million euro
No of participants: 315

City: Norte region
Education objective: Scholarships 
for higher education students
Total cost: 38.9 million euro
No of participants: 20 002

City: Dresden
Education objective: Development
and implementation of vocational
training modules
Total cost: 0.3 million euro
No of participants: 26

City: Lengenfeld
Education objective: Enabling 
participants to obtain a formal
qualification in their current area
of employment
Total cost: 0.3 million euro
No of participants: 32

City: Dresden
Education objective: Prevention
of early school leaving
Total cost: 3.4 million euro
No of participants: 673

City: Bucharest
Education objective: Prevention
of early school leaving
Total cost: 4.0 million euro
No of participants: 20 861

City: Bucharest
Education objective: Non-formal
education programmes to increase
skills in key European competences
on active citizenship
Total cost: 2.4 million euro
No of participants: 24 929

City: Bucharest
Education objective: Internship 
programme for students enrolled in 
higher medical veterinary education
Total cost: 2.6 million euro
No of participants: 3 005

City: Palermo
Education objective: Prevention
of early school leaving
Total cost: 0.2 million euro
No of participants: 156

City: Palermo
Education objective: Training for
adults with poor basic skills
Total cost: 0.1 million euro
No of participants: 39

City: Bagheria (PA)
Education objective: Training to 
improve students basic skills
Total cost: 0.1 million euro
No of participants: 247

 
Observations 

27

47 
Table 4 presents, for each EU education objective, the number of projects exam-
ined and the financial amounts involved. Although some projects contributed to 
more than one EU objective in the field of education, for reporting purposes they 
have been listed under the most relevant one based on our assessment.

4 EU education objective addressed by the projects examined
e
l
b
a
T

Objectives

Number and location of the projects examined

Early school leavers

Lifelong learning

Low achievers in basic skills

Tertiary education 
attainment

Upper secondary education 
attainment

Total

5 projects 
(Germany, France, Italy, Portugal and Romania)

5 projects 
(Germany, 2 in France, Italy and Portugal)

2 projects 
(Italy and Romania)

2 projects 
(Portugal and Romania)

1 project in Germany

15 projects

Number of 
participants

Total allocation 
(million euro)

Total ESF allocation 
(million euro)

24 299

2 773

25 176

23 007

26

75 281

10.6

10.0

2.5

41.5

0.3

64.9

7.1

4.5

1.8

35.3

0.2

48.9

 
Observations 

28

Insufficient focus on achievement of projects’ objectives and use 
of relevant performance indicators

48 
For most of the projects examined, only financial aspects and outputs had been 
monitored, not their contribution to the selected objectives as such. Failing to 
assess the skills acquired by the participants may lead to training that does not 
meet the participants’ learning needs and does not improve the targeting of 
the learning activities if needed (see Box 2). In almost half of the projects, as the 
objectives had not been quantified, the monitoring procedures in place did not 
provide information on the extent to which the objectives had been achieved.

49 
In two cases, while comprehensive monitoring information was available at the 
level of the beneficiary, it had not been requested and therefore not transmitted 
to the managing authority.

2
x
o
B

Unsatisfactory monitoring procedure

The objective of a project in Palermo (Italy) was to strengthen lifelong learning, i.e. to support the access of 
adults, whether employed or not, to education pathways through which to recover initial educational deficits 
and/or obtain qualifications.

The selection criteria in the call for projects provided for a specific criterion related to enhanced and clear 
ongoing and ex post evaluation methodologies both for outputs and outcomes. The only monitoring put in 
place was a self-evaluation in which the 39 participants expressed a judgment on their knowledge before and 
after the training.

We consider that this method should have been accompanied by other monitoring procedures such as assess-
ment by the training provider to better assess the effectiveness of the project.

 
Observations 

29

50 
Almost all the projects had been carried out within the budget and without 
delays. Where budget overruns had occurred, it was because of a significant 
increase in the number of the initially planned participants, and this had been ap-
proved by the relevant managing authority.

51 
Overall, the activities related to the 15 projects examined corresponded to their 
respective EU education objectives set out in the OPs. However, because almost 
half of the examined projects had not set indicators other than the number of 
participants/participations, the possibility to ascertain the extent to which a pro-
ject had actually produced any change in the situation of the participants was 
limited.

52 
During the audit, we identified good practice in the case of one managing 
 authority, where payments were linked to project performance (see Box 3).

3
x
o
B

Example of result-oriented management: linking payments to performance

In Romania, the projects examined included both output and outcome indicators with target values.

The procedures in place gave the managing authority/intermediate body the possibility to decrease propor-
tionally the amount of the grant where the monitoring showed that the output/outcome indicators’ target 
values had not been attained at the end of the project25.

25  See for example Romanian managing authority instruction No 71/2013 applicable to the 2007-2013 projects that were not yet completed at the 

time when the instruction was issued.

 
Observations 

30

Assessment of the establishment of operational 
programmes for the 2014-2020 programme period

53 
Our approach to assessing the inclusion of EU education objectives in the 2014-
2020 OPs was the same as that for the 2007-2013 OPs. In addition, the audit 
considered the changes in the applicable regulations26 such as partnership agree-
ments and the ex ante conditionalities.

54 
We assessed whether:

 ο

the Commission had supported Member States in establishing the 2014-2020 
OPs;

 ο

2014-2020 OPs adequately addressed the EU education objectives27;

 ο

the performance monitoring framework has been improved;

 ο

a link had been established between EU education objectives and 
employment.

26  Mainly Regulation (EU) 

27 

No 1303/2013 and Regulation 
(EU) No 1304/2013 of the 
European Parliament and of 
the Council of 
17 December 2013 on the 
European Social Fund and 
repealing Council Regulation 
(EC) No 1081/2006 (OJ L 347, 
20.12.2013, p. 470).

Improving vocational 
education and training was 
not included in the Europe 
2020 objectives nor covered 
by any specific benchmark of 
ET 2020. However, this 
objective has been included in 
our assessment because the 
legal framework includes this 
objective under thematic 
objective 10 and introduced 
the thematic ex ante 
conditionality 10.4 requiring 
the existence of a national or 
regional strategic policy 
framework for increasing the 
quality and efficiency of 
vocational education and 
training systems.

Commission supported Member States in establishing 2014-
2020 operational programmes

55 
The Commission should provide support to Member States to ensure that the 
OPs submitted by them for its approval include those EU policy objectives which 
the Member States wish to benefit from ESF funding.

56 
We consulted with the Commission services involved in the process of establish-
ing the 2014-2020 OPs and reviewed the related documentation, including the 
partnership agreement for each of the five Member States visited (see paragraph 
21) focusing on education objectives.

31

28  For an analysis of the 
country-specific 
recommendations see also 
Special Report No 3/2015 ‘EU 
Youth Guarantee: first steps 
taken but implementation 
risks ahead’, paragraphs 73 to 
78 (http://eca.europa.eu).

Observations 

57 
The process of negotiating and approving partnership agreements and OPs for 
the 2014-2020 programme period started in 2012, with the preparation of the 
 position papers by the Commission. These position papers set out the Commis-
sion’s preliminary analysis of the situation in the different Member States and 
regions and formed the framework for the dialogue between the Commission 
and the Member States on preparing the partnership agreements and OPs. 
They set out the key country-specific challenges and presented the Commission 
services’ preliminary views on the main funding priorities. The starting point 
of the Commission’s considerations was its assessment of the Member States’ 
progress towards their Europe 2020 targets, underpinned by the country-specific 
recommendations addressed by the Council and country-specific development 
challenges28.

58 
The Commission provided support to Member States in setting up these strategic 
documents. In particular, the Commission issued guidance on different aspects 
of the new programme period, such as a template and guidelines on the content 
of the partnership agreement and OPs. In addition, it issued guidance fiches on 
intervention logic and on how to comply with the four education-related ex ante 
conditionalities: ‘Early school leaving’, ‘Higher education’, ‘Lifelong learning’ and 
‘Vocational education and training’.

59 
The Commission also developed appraisal procedures for assessing the draft 
partnership agreements and OPs in order to ensure that they properly address EU 
education objectives set in ET 2020 and Europe 2020.

60 
The comprehensive assessment carried out by the Commission for the sampled 
partnership agreements and OPs successfully identified several shortcomings 
(such as unclear specific objectives, lacking or poor quality indicators, poor per-
formance framework information and a failure to comply with the ex ante condi-
tionalities). The observations were submitted to the Member States, which, where 
necessary, revised and resubmitted the partnership agreements and the OPs.

Observations 

32

61 
Our assessment of the process for the establishment of 2014-2020 OPs identified 
some shortcomings. For example, in the Norte OP (Portugal), despite financial 
support having been allocated to ‘Vocational education and training’, the cor-
responding ex ante conditionality had not been considered applicable or fulfilled.

Examined operational programmes consistent with EU 
education objectives

62 
Where OPs include specific education objectives, the related intervention logic 
and the tools for monitoring performance should be outlined and described.

63 
We carried out an analysis of eight OPs which included an ESF allocation for the 
2014-2020 programme period, in order to assess the extent to which these cri-
teria had been outlined.

64 
The total initial budget of the reviewed OPs is 23.33 billion euro, of which 
13.76 billion euro is from the ESF. The ESF allocation to education is 7.14 bil -
lion euro, i.e. 52 % of the total ESF support. Annex IV provides the budgetary 
information on the OPs examined.

65 
Our analysis of the OPs examined showed that they generally included educa-
tion objectives. There was a significant improvement in the description of the 
intervention logic compared to the 2007-2013 programme period. We also noted 
improvements in the monitoring tools framework. An overview of the remaining 
shortcomings is presented in Table 5.

Observations 

33

Assessment of 2014-2020 OPs: EU education objectives, intervention logic and tools 
for monitoring

5
e
l
b
a
T

Objectives

OPs that included the 
EU objectives (num-
ber and percentage 
of OPs examined)

Intervention logic

Tools for monitoring

Without any 
description of 
context 
(%)

Without any 
needs analysis 
(%)

Without any ex-
ample of actions 
to achieve the 
objective 
(%)

Without any 
quantification of 
the objective 
(%)

Without any indica-
tors for measuring 
the achievement of 
the objective 
(%)

Early school 
leavers 

Tertiary educa-
tion attainment

Lifelong learning

Low achievers in 
basic skills

Early childhood 
education

Vocational 
education and 
training

7 (88 %)

5 (63 %)

6 (75 %)

3 (38 %)

2 (25 %)

7 (88 %)

30 cases1

none

none

none

none

none

29

5

none

none

none

33

none

none

6

none

none

none

none

none

none

0

none

none

17

33

50

none

17

1  The total number of cases is the sum of all the identified education objectives in all 8 OPs examined.

none

none

none

33

50

none

14

66 
The assessment showed that, overall, the OPs presented a satisfactory descrip-
tion of the situation existing before the implementation of the OP and coherently 
identified the needs to be fulfilled.

67 
The OPs also presented indicative actions for achieving the objectives and 
identified the guiding principles for selecting operations and, where appropri-
ate, identifying the main target groups, specific territories targeted and types of 
beneficiaries. While the process for establishing the examined OPs shows some 
improvements compared to the 2007-2013 programme period, the crucial next 
stage, which is in the hands of Member States, is to select and implement appro-
priate projects to support the achievement of education objectives outlined in 
the OPs.

 
34

29  Annex I to Regulation (EU) 

No 1304/2013.

30  Programming Period 

2014-2020 — Monitoring and 
Evaluation of European 
Cohesion Policy — European 
Social Fund — Guidance 
Document — June 2015, p. 15.

Observations 

68 
We found shortcomings in some OPs, mainly regarding the quantification of the 
objectives and the lack of baselines and target values for indicators. As detailed 
in Table 5, these concern ‘Lifelong learning’, ‘Low achievers in basic skills’ and 
‘Early childhood education’ objectives.

Performance monitoring arrangements improved, but with 
some limitations

69 
The monitoring and reporting framework should provide sufficient information 
for the Commission to be able to adequately monitor and report on the perfor-
mance achievements of ESF funded measures.

70 
To assess this aspect, we reviewed the 2014-2020 legal framework and the related 
guidance issued by the Commission in relation to the eight OPs examined.

71 
To overcome the weaknesses identified for the 2007-2013 programme period (see 
paragraphs 33 to 38), the 2014-2020 legal framework introduced common result 
indicators29. Article 5 of the ESF regulation provides that ‘all common output and 
result indicators shall be reported for all investment priorities’. This reporting will 
take place in the annual implementation report and should deliver consistent 
and comparable information about the achievement of the OP objectives.

72 
According to the guidance document on monitoring and evaluation of the  
EU cohesion policy30, ‘only the indicators that represent the main outputs expect-
ed from the specific objective should be linked to a target’. In addition, this guid-
ance specifies that ‘baselines shall be fixed for all common and programme-spe-
cific result indicators for which a cumulative quantified target value for 2023 has 
been set’.

Observations 

35

73 
In its guidance, however, the Commission did not detail the exact requirements 
in terms of the result indicator(s) for outcomes to be chosen for each investment 
priority to ensure consistency between the OPs. For the objective of reducing the 
number of early school leavers, in particular, we consider that this led to some 
cases where baseline and target values for additional indicators would have been 
helpful to better assess the effectiveness of the measure (see Box 4).

4 Example where targets for additional outcome indicators would have been helpful
x
o
B

For the specific objective ‘Reducing and preventing early school leaving’, the 2014-2020 regional OP for Italy 
(Sicily) has selected as an outcome indicator ‘participants who are in employment, including self-employment, 
6 months after leaving’ (together with a baseline and a target).

However, given that many of the recipients will be students still enrolled in school, this indicator alone is not 
sufficient for assessing the intervention’s effectiveness.

Since this measure also aims at keeping participants in school, we consider that it would also have been nec-
essary to set a target for an additional indicator such as the common result indicator ‘participants in educa-
tion/training upon leaving’ or the indicator employed by the national Italian ESF OP measuring the situation 
of the participants 1 year after the end of an intervention.

Difficulties in linking education measures to employment 
prospects

74 
Monitoring and reporting systems for the 2014-2020 programme period should 
provide information on the impact of relevant education measures on partici-
pants’ employability.

75 
To address this criterion, we reviewed the Council employment guidelines and 
other relevant EU documentation.

 
Observations 

76 
The Council employment guidelines that were valid at the time the 2007-2013 ESF 
OPs31 were adopted already emphasised the importance of expanding, improving 
and adapting the investment in human capital (i.e. education) to foster employ-
ability. This principle was confirmed both in the 2010 and in the 2013 employ-
ment guidelines32, especially with reference to vocational education and training, 
lifelong learning and tertiary education attainment.

77 
The Council conclusions of 12 May 2009 on a strategic framework for European 
cooperation in education and training (ET 2020), in recognising the importance of 
education and training to enhance employability, mandated the Commission to 
propose a possible European benchmark in this area.

78 
In 2011, the Commission proposed33 a new benchmark concerning the employ-
ment rate for recent graduates34. With the aim of highlighting what education 
and training policies can contribute to boosting employment success and to 
increasing the employability of graduates, the Council endorsed this benchmark 
in May 201235.

79 
According to the benchmark, the employment rate for recent graduates should 
reach at least 82 % by 2020. The actual rate, available at Eurostat, the Com-
mission’s statistical service, shows that the current figures are well below the 
planned target. In 2014, the employment rate for recent graduates in the EU as 
a whole was 76 %. Out of the five visited Member States, only Germany already 
achieved the target. The others have shown a steady decline since 2006 (see 
Figure 5).

36

31  Council Decision of 

12 July 2005 on guidelines for 
the employment policies of 
the Member States (OJ L 205, 
6.8.2005, p. 21).

32  Council Decision 

(2013/208/EU) of 22 April 2013 
on guidelines for the 
employment policies of the 
Member States (OJ L 118, 
30.4.2013, p. 21), which 
maintains the employment 
guidelines annex to Decision 
2010/707/EU.

33  SEC(2011) 670 final of 

24 May 2011 ‘Commission Staff 
Working Paper on the 
development of benchmarks 
on education and training for 
employability and on learning 
mobility’.

34  The indicator ‘employment 
rates for recent graduates’ 
presents the employment 
rates for persons aged 20 to 34 
fulfilling the following 
conditions: first, being 
employed according to the 
ILO definition, second, having 
attained at least upper 
secondary education (ISCED 3) 
as the highest level of 
education, third, not having 
received any education or 
training in the four weeks 
preceding the survey and four, 
having successfully completed 
their highest educational 
attainment 1, 2 or 3 years 
before the survey. The 
indicator is calculated based 
on data from the EU Labour 
Force Survey.

35  Council Conclusion of 
11 May 2012 on the 
employability of graduates 
from education and training 
(OJ C 169, 15.6.2012, p. 11).

Observations 

37

5
e
r
u
g
i
F

Employment rate of recent graduates in visited Member States

100 %

90 %

80 %

70 %

60 %

50 %

40 %

30 %

20 %

10 %

0 %

Target: 82 %

Germany

France

Italy

Portugal

Romania

2006

2007

2008

2009

2010

2011

2012

2013

2014

Source: Eurostat.

80 
The common result indicators introduced by the 2014-2020 ESF regulation in-
clude both immediate and longer-term result indicators that capture the impact 
on employment of the measures implemented. However, for the three educa-
tion objectives most closely linked with employment — vocational education 
and training, lifelong learning and tertiary education attainment — baseline 
and target values had been set in only a third of the cases of the 2014-2020 OPs 
examined.

81 
Even when baseline and targets were set for employment-related indicators, the 
Italian OP for Sicily is the only case where, for two of the three education objec-
tives most closely linked with employment36, there is a clear link between the 
number of intended participants and the expected employment outcomes of the 
planned education interventions.

36  Tertiary education attainment 

and lifelong learning.

 
Conclusions and 
recommendations

38

82 
We concluded that while EU education objectives have been adequately consid-
ered in the 2007-2013 OPs, the performance of the audited projects could not 
be systematically demonstrated due to the insufficient use of quantified objec-
tives and performance indicators. Regarding the design of the 2014-2020 OPs, we 
found that the Commission provided support to Member States in drawing up 
their partnership agreements and OPs and subsequently assessed them, identify-
ing several shortcomings. We also concluded that EU education objectives had 
been adequately considered, with improvements in the description of the inter-
vention logic. However, there are still some weaknesses in the framework which 
may impact performance monitoring and reporting at OP and project level. In 
addition, there is not always a clear link between education measures and their 
impact on participants’ employability.

EU education objectives addressed in the 2007-2013 ESF OPs 
examined, but shortcomings relating to OP modifications 
and monitoring

83 
Our analysis of the examined OPs showed that, generally, EU education objec-
tives had been considered and included. However, for some OPs, elements of 
the intervention logic were lacking, such as the description of the context or the 
needs analysis (see paragraphs 27 to 32). In addition, there were shortcomings 
in the monitoring tools framework, as objectives had not been quantified and 
indicators for measuring them were lacking (see paragraphs 33 to 38).

84 
In general, the ET 2020 and Europe 2020, which were adopted during the 2007-
2013 programme period, had little impact on the OPs examined as they were 
already in line with the updated objectives therein. For the majority of those OPs 
reviewed which were subject to a change in the financial allocation, clear expla-
nations about the implications of the change of the financial allocations for the 
related target values were missing (see paragraphs 39 to 44).

Conclusions and recommendations 

39

Performance of examined projects could not be 
systematically demonstrated

85 
We examined a sample of 15 projects that, according to the managing author-
ities, had contributed positively to achieving at least one EU education objective. 
On this basis, we would have expected the impact of these projects on partici-
pants and on the overall achievement of the specific education objective to have 
been clearly demonstrated.

86 
The performance of the audited projects could not be systematically demon-
strated. This was mainly due to the fact that the contribution to the selected edu-
cation objectives had not been monitored, and indeed objectives had not been 
quantified for almost half of the projects audited. In addition, almost half of the 
projects had not set indicators other than number of participants/participations, 
creating difficulties in drawing conclusions on the real impact of the projects on 
participants (see paragraphs 48 to 52).

2014-2020 OPs consistent with EU education objectives

87 
The Commission provided support to Member States in drawing up their partner-
ship agreements and OPs and subsequently assessed them, identifying several 
shortcomings. Our assessment of the process for establishing 2014-2020 OPs 
identified some shortcomings (see paragraphs 55 to 61).

88 
Our analysis of the examined OPs showed that they generally included educa-
tion objectives. There was a significant improvement in the description of the 
intervention logic compared to the 2007-2013 programme period. We also noted 
improvements in the monitoring framework for the 2014-2020 period (see para-
graphs 65 to 68).

Conclusions and recommendations 

40

89 
While the process for establishing the examined OPs shows some improvements 
compared to the 2007-2013 programme period, the crucial next stage, which is in 
the hands of Member States, is to select and implement appropriate projects to 
support the achievement of education objectives outlined in the OPs.

Improved performance monitoring arrangements, but some 
limitations remain

90 
To overcome the weaknesses identified for the 2007-2013 programme period, the 
2014-2020 legal framework introduced common result indicators. In its guidance, 
however, the Commission did not detail the exact requirements in terms of result 
indicator(s) on outcomes to be chosen for each investment priority to ensure 
consistency between the OPs (see paragraphs 71 to 73).

91 
Education has a clear impact on participation in the labour market and earnings, 
and brings many social advantages. To implement the Europe 2020 strategy for 
jobs and smart, sustainable and inclusive growth, education interventions should 
have a clear focus on employment outcomes, especially when targeting adults.

92 
The common result indicators introduced by the ESF regulation include both 
immediate and longer-term result indicators that measure the effect on employ-
ment of the measures implemented. However, for the three education objec-
tives most closely linked with employment — vocational education and training, 
lifelong learning and tertiary education attainment — baseline and target values 
had been set in only a third of the cases of the 2014-2020 OPs examined (see 
paragraphs 76 to 81).

Conclusions and recommendations 

41

93 
We recommend that:

Recommendation 1

The Commission should consider specifying the result indicator(s) on outcomes 
for which baseline and target values should be set for each investment priority.

Target implementation date: during the preparation of the next programme 
period.

Recommendation 2

Where modifications to OPs are requested by Member States, the Commission 
should:

—  encourage the establishment of a clear link between OPs’ investment prior-

ities and appropriate quantified and measurable result indicators;

—  ensure that Member States provide explanations for the reprogramming of 

financial allocations, including both qualitative and quantitative information 
on the expected change in output and results indicators.

Target implementation date: immediately.

Recommendation 3

Member States should ensure that:

—  there is a clear link between the selected projects and the achievement of EU 

education objectives embedded in the OP;

—  appropriate result indicators are put in place to demonstrate the actual ef-
fects of the project on the final participants in a systematic manner and to 
monitor progress towards achieving the OPs’ education objectives.

Target implementation date: immediately.

Conclusions and recommendations 

42

Recommendation 4

Where relevant, the Commission and the Member States should better target 
OPs’ funding on measures which reinforce the link between education and em-
ployment, and should ensure that outcomes are monitored appropriately.

Target implementation date: immediately.

This Report was adopted by Chamber II, headed by Mr Henri GRETHEN, Member 
of the Court of Auditors, in Luxembourg at its meeting of 11 May 2016.

For the Court of Auditors

Vítor Manuel da SILVA CALDEIRA
President

 
 
 
Annexes

43

I
x
e
n
n
A

Evolution towards reaching the Europe 2020 education targets

I. Percentage of early school leavers from education and training

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

TARGET

15.3

12.6

17.3

5.1

9.1

13.7

13.4

12.2

15.1

30.3

12.7

4.7

20.4

14.9

15.6

8.8

14.0

12.5

32.2

12.6

10.0

5.4

38.5

17.9

5.6

6.6

9.7

8.6

11.3

14.9

12.1

14.9

5.2

12.9

12.5

14.4

11.8

14.3

30.8

12.8

4.5

19.5

12.5

15.6

7.8

12.5

11.4

30.2

11.7

10.8

5.0

36.5

17.3

4.1

6.5

9.1

8.0

14.7

12.0

14.8

5.6

12.5

11.8

14.0

11.4

14.4

31.7

11.8

4.4

19.6

13.7

15.5

7.5

13.4

11.7

27.2

11.4

10.2

5.0

34.9

15.9

5.1

6.0

9.8

7.9

14.2

11.1

14.7

5.4

11.3

11.1

13.5

11.7

14.2

30.9

12.4

5.2

19.1

11.7

14.3

8.7

7.7

11.5

25.7

10.9

8.8

5.3

30.9

16.6

5.3

4.9

9.9

7.0

16.6

16.9

15.7

13.9

11.9

13.9

4.9

11.0

11.9

11.0

11.5

13.5

28.2

12.7

5.2

18.6

12.7

12.9

7.9

7.1

10.8

23.8

10.0

8.3

5.4

28.3

19.3

5.0

4.7

10.3

6.5

14.8

13.4

12.3

11.8

4.9

9.6

11.6

10.6

10.8

12.9

26.3

12.3

5.0

17.8

11.3

11.6

7.4

6.2

11.4

22.7

9.2

8.5

5.6

23.0

18.1

4.2

5.1

9.8

6.6

12.7

12.0

12.5

5.5

9.1

10.5

10.3

9.7

11.3

24.7

11.8

5.1

17.3

11.4

10.6

6.5

8.1

11.8

21.1

8.9

7.8

5.7

20.5

17.8

4.4

5.3

8.9

7.5

11.9

11.0

12.5

5.4

8.0

9.8

9.7

8.4

10.1

23.6

9.7

4.5

16.8

9.1

9.8

6.3

6.1

11.9

20.5

9.3

7.5

5.6

18.9

17.3

3.9

6.4

9.3

7.1

11.2

9.8

12.9

5.5

7.8

9.5

11.4

6.9

9.0

21.9

9.0

2.7

15.0

6.8

8.5

5.9

6.1

11.4

20.3

8.7

7.0

5.4

17.4

18.1

4.4

6.7

9.5

6.7

10.9

10.0

13.1

5.8

7.6

9.6

11.8

6.9

8.3

20.3

9.2

3.0

14.6

5.4

9.5

5.5

8.6

11.6

20.1

8.3

7.3

5.3

14.4

18.9

5.4

7.1

9.3

6.6

10.0

9.5

11.0

5.5

10.0

10.0

9.5

8.0

9.7

15.0

9.5

4.0

16.0

10.0

13.4

9.0

10.0

10.0

10.0

8.0

9.5

4.5

10.0

11.3

5.0

6.0

8.0

10.0

14.9

13.4

12.3

11.8

11.0

Not set

EU (28 Member 
States)

Belgium

Bulgaria

Czech Republic

Denmark

Germany

Estonia

Ireland

Greece

Spain

France

Croatia

Italy

Cyprus

Latvia

Lithuania

Luxembourg

Hungary

Malta

Netherlands

Austria

Poland

Portugal

Romania

Slovenia

Slovakia

Finland

Sweden

United Kingdom

Source: Eurostat.

 
Annexes 

44

I
x
e
n
n
A

II. Share of 30-34-year-olds with tertiary educational attainment

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

TARGET

29.0

41.4

25.3

13.1

43.0

25.8

32.5

41.3

26.9

39.4

39.7

16.7

17.6

46.1

19.3

39.4

35.5

19.4

20.7

35.8

21.1

24.7

18.3

12.4

28.1

14.4

46.2

39.5

36.4

30.1

41.5

26.0

13.3

38.1

26.5

33.5

43.3

26.3

40.9

41.4

16.8

18.6

46.2

25.7

36.4

35.3

20.6

20.8

36.4

20.9

27.0

19.5

13.9

31.0

14.8

47.3

41.0

38.3

31.1

42.9

27.1

15.4

39.2

27.7

34.4

46.3

25.7

41.3

41.0

18.5

19.2

47.1

26.3

39.9

39.8

22.8

21.0

40.2

21.9

29.7

21.6

16.0

30.9

15.8

45.7

42.0

39.5

32.3

42.0

27.9

17.5

40.7

29.4

36.3

48.9

26.6

40.7

43.0

21.3

19.0

45.0

30.5

40.4

46.6

24.0

21.9

40.5

23.4

32.8

21.3

16.8

31.6

17.6

45.9

43.9

41.4

33.8

44.4

27.7

20.4

41.2

29.8

40.2

50.1

28.6

42.0

43.2

24.5

19.9

45.3

32.6

43.8

46.1

26.1

22.1

41.4

23.4

34.8

24.0

18.3

34.8

22.1

45.7

45.3

43.1

34.8

42.6

27.3

23.7

41.2

30.6

40.2

49.7

29.1

41.9

43.1

23.9

20.4

46.2

35.9

45.7

48.2

28.2

23.4

41.2

23.6

36.5

26.7

20.3

37.9

23.2

46.0

46.8

45.5

36.0

43.9

26.9

25.6

43.0

31.8

39.5

51.1

31.2

41.5

43.3

23.1

21.9

49.9

37.2

48.6

49.6

29.8

24.9

42.2

26.1

39.1

27.8

21.7

39.2

23.7

45.8

47.9

46.9

37.1

42.7

29.4

26.7

43.4

32.9

42.5

52.6

34.9

42.3

44.0

25.6

22.5

47.8

40.7

51.3

52.5

32.3

26.0

43.2

27.1

40.5

30.0

22.9

40.1

26.9

45.1

48.3

47.4

37.9

43.8

30.9

28.2

44.9

31.4

43.2

52.2

37.2

42.3

43.7

32.2

23.9

52.5

39.9

53.3

52.7

34.1

26.5

44.8

40.0

42.1

31.3

25.0

41.0

26.9

45.3

49.9

47.7

38.5

43.1

32.0

29.5

46.7

31.8

45.2

52.3

39.4

41.1

44.9

31.7

24.9

54.2

41.0

56.4

50.5

34.9

27.0

46.4

39.1

43.2

31.3

25.5

42.6

27.9

45.3

50.0

47.7

40.0

47.0

36.0

32.0

40.0

42.0

40.0

60.0

32.0

44.0

50.0

35.0

26.0

46.0

34.0

48.7

66.0

30.3

33.0

40.0

38.0

45.0

40.0

26.7

40.0

40.0

42.0

40.0

Not set

EU (28 Member 
States)

Belgium

Bulgaria

Czech Republic

Denmark

Germany

Estonia

Ireland

Greece

Spain

France

Croatia

Italy

Cyprus

Latvia

Lithuania

Luxembourg

Hungary

Malta

Netherlands

Austria

Poland

Portugal

Romania

Slovenia

Slovakia

Finland

Sweden

United Kingdom

Source: Eurostat.

 
Annexes 

45

I
I
x
e
n
n
A

Overview of programming, implementation, monitoring and reporting

European Commission

Member State

Preparation of the 
partnership agreement

Preparation of the 
operational agreement

Selection of the projects
or project providers

Implementation of 
the projects

Provision of monitoring
information on projects

Preparation of annual
implementation reports

Analysis and approval of
the partnership agreement

Analysis and approval of
the operational programme

Analysis and appraisal
of the annual implementation
reports

Source: European Court of Auditors.

 
Annexes 

46

I
I
I
x
e
n
n
A

List of the 2007-2013 ESF operational programmes examined

Member 
State

Operational programmes

Total allocation 
(million euro)

Total ESF 
contribution 
(million euro)

Initial ESF 
allocation for 
education1 
(million euro)

% of total 
ESF

PT

PL

DE

2007PT05UPO001 — Programa Operacional 
Temático Potencial Humano 2007-2013

2007PT051PO001 — Programa Operacional do 
Fundo Social Europeu Para a Região Autónoma dos 
Açores 2007-2013 Pro-Emprego

2007PT052PO001 — Programa Operacional de 
Valorização do Potencial Humano e Coesão Social 
da Região Autónoma Da Madeira

2007PL051PO001 — Program Operacyjny Kapitał 
Ludzki Narodowe Strategiczne Ramy Odniesienia 
2007-2013

2007DE05UPO001 — Operationelles Programm 
des Bundes für den Europäischen Sozialfonds 
Förderperiode 2007-2013

2007DE051PO004 — Operationelles Programm 
des Freistaates Sachsen für den Europäischen 
Sozialfonds (ESF) in der Förderperiode 
2007 bis 2013

2007DE052PO008 — Operationelles Programm 
zur Umsetzung des ESF in NRW in der Förderphase 
2007-2013

2007DE051PO006 — Operationelles Programm 
für den Einsatz des Europäischen Sozialfonds im 
Freistaat Thüringen in den Jahren 2007 bis 2013

2007DE051PO002 — Europäischer Sozialfonds 
(ESF) Operationelles Programm des Landes 
Mecklenburg-Vorpommern im Ziel Konvergenz
Förderperiode 2007 bis 2013

2007DE051PO001 — Operationelles Programm 
des Landes Brandenburg für den Europäischen 
Sozialfonds (ESF) in der Förderperiode 
2007 bis 2013 Ziel Konvergenz Brandenburg 
Nordost und Brandenburg Südwest

8 736

6 117

5 056

224

156

11 420

6 031

1 157

1 368

836

557

826

190

125

9 707

3 488

872

684

629

417

620

104

82

4 330

914

571

432

299

275

260

83

55

66

45

26

65

63

48

66

42

 
Annexes 

47

I
I
I
x
e
n
n
A

Member 
State

Operational programmes

Total allocation 
(million euro)

Total ESF 
contribution 
(million euro)

Initial ESF 
allocation for 
education1 
(million euro)

% of total 
ESF

2007IT051PO007 —Programma Operativo 
Nazionale Competenze per lo sviluppo

2007IT051PO003 — Programma Operativo 
Regionale Sicilia per il Fondo Sociale Europeo 
2007-2013

2007IT051PO005 — Regione Puglia
Programma Operativo Regionale 2007-2013  
Fondo Sociale Europeo

2007IT051PO001 — Programma Operativo 
Obiettivo Convergenza Fondo Sociale Europeo 
2007-2013 Regione Campania

2007IT052PO006 — Programma Operativo 
Regionale della Lombardia Ob. 2 FSE 2007-2013

2007IT052PO004 — Programma Operativo del 
Fondo Sociale Europeo Obiettivo Competitività 
Regionale e Occupazione Regione Lazio 2007-2013

2007HU05UPO001 — Társadalmi Megújulás 
Operatív Program

2007CZ05UPO002 — Operační program  
Vzdělávání pro konkurenceschopnost

2007CZ05UPO001 — Operační program Lidské 
zdroje a zaměstnanost 2007-2013

2007RO051PO001 — Programul Operaţional 
Sectorial Dezvoltarea Resurselor Umane  
2007-2013

2007GR05UPO002 — Επιχειρησιακο Προγραμμα 
Εκπαιδευση Και Δια Βιου Μαθηση

2007GR05UPO001 — Επιχειρησιακο Προγραμμα 
Αναπτυξη Ανθρωπινου Δυναμικου

2007GR05UPO003 — Επιχειρησιακο Προγραμμα 
Διοικητικη Μεταρρυθμιση

IT

HU

CZ

RO

EL

1 487

2 099

1 279

1 118

798

736

4 097

2 151

2 157

4 089

2 215

2 825

675

743

1 050

640

559

338

368

3 483

1 829

1 837

3 476

1 440

2 260

505

698

302

247

209

163

103

2 358

1 617

268

1 862

1 418

369

173

94

29

39

37

48

28

68

88

15

54

99

16

34

 
Annexes 

48

I
I
I
x
e
n
n
A

Member 
State

Operational programmes

2007ES051PO005 — Programa Operativo  
Fondo Social Europeo 
2007-2013 Andalucía

2007ES05UPO001 — Programa Operativo 
Plurirregional Adaptabilidad y Empleo  
Fondo Social Europeo 2007-2013

ES

2007ES051PO004 — Programa Operativo Fondo 
Social Europeo de Galicia 2007-2013

2007ES051PO003 — Programa Operativo Fondo 
Social Europeo 2007-2013 Extremadura

2007ES052PO008 — Programa Operativo Fondo 
Social Europeo 2007-2013 Madrid

2007ES051PO002 — Programa Operativo Fondo 
Social Europeo 2007-2013 Castilla la Mancha

2007UK05UPO001 — ESF England 2007-13 
Operational Programme

2007UK051PO002 — West Wales and the Valleys 
Convergence Programme Operational Programme 
for the European Social Fund 2007-2013

2007FR052PO001 —Compétitivité Régionale  
et Emploi 2007-2013  
Programme Opérationnel National  
Fonds Social Européen

2007FR051PO004 — Ile de La Réunion 
Programme Opérationnel FSE 2007-2013

2007SK05UPO001 — Operačný program 
Vzdelávanie

2007BG051PO001 — Оперативна програма 
“Развитие на човешките ресурси” 2007-2013 г.

2007BE052PO005 — Operationeel Programma 
ESF Doelstelling 2 Vlaanderen 2007-2013

2007AT052PO001 — Operationelles Programm 
Beschäftigung Österreich 2007-2013

UK

FR

SK

BG

BE

AT

Total allocation 
(million euro)

Total ESF 
contribution 
(million euro)

Initial ESF 
allocation for 
education1 
(million euro)

% of total 
ESF

1 445

5 925

448

333

514

225

6 049

1 500

8 991

742

727

1 214

1 111

1 115

1 156

4 301

359

250

257

180

3 090

834

4 495

517

618

1 032

469

472

477

1 313

153

128

102

45

1 047

431

953

290

490

490

167

190

41

31

43

51

40

25

34

52

21

56

79

47

36

40

48

Total

87 377

59 406

28 385

1 

In the 2007-2013 programme period, EU priority codes 62, 72, 73 and 74 are the ones identified as corresponding to allocations to education.

 
Annexes 

49

V

I
x
e
n
n
A

List of the 2014-2020 partnership agreements and operational programmes 
examined

Member 
State

DE

FR

IT

PT

RO

Partnership agreement number

2014DE16M8PA001 — Partnerschaftsvereinbarung zwischen Deutschland und der europäischen Kommission für die Umsetzung der 
ESI-Fonds unter dem gemeinsamen strategischen Rahmen in der Förderperiode 2014 bis 2020

2014FR16M8PA001 — Accord de partenariat 2014-2020 France

2014IT16M8PA001 — Accordo di partenariato 2014-2020 Italia

2014PT16M8PA001 — Portugal 2020 — acordo de parceria 2014-2020

2014RO16M8PA001 — Acordul de parteneriat România

Member 
State

OP number

Total allocation 
(million euro)

Total ESF 
contribution 
(million euro)

Initial ESF 
allocation for 
education1 
(million euro)

% of total 
ESF

2014DE05SFOP002 — Operationelles Pro-
gramm ESF Bund Deutschland 2014-2020

2014DE05SFOP012 — Operationelles Pro-
gramm ESF Sachsen 2014-2020

2014FR05M0OP001 — Programme Opération-
nel au titre de l’objectif «Investissement pour la 
croissance et l’emploi»

2014IT05M2OP001 — Per la Scuola — Compe-
tenze e Ambienti per l’apprendimento

2014IT05SFOP014 — POR Sicilia FSE

2014PT05SFOP001 — PO Capital Humano

2014PT16M2OP001 — Programa Operacional 
Regional do Norte 2014-2020

2014RO05M9OP001 — Program Operational 
Capital Uman

DE

FR

IT

PT

RO

4 830

828

968

3 019

820

3 642

4 166

5 059

2 689

663

280

1 615

615

3 096

583

4 221

Total

23 332

13 762

883

247

213

1 056

193

3 030

263

1 257

7 142

33

37

76

65

31

98

45

30

52

1 

In the 2014-2020 programme period, EU priority codes 115, 116, 117 and 118 are the ones identified as corresponding to allocations to education.

 
Annexes 

50

V
x
e
n
n
A

Audit methodology for assessing the objectives addressed by the Operational 
programmes

Was the objective selected in the OP?

Yes

No

No further
assessment

Assessment

Why it is relevant

Outcome

Was there a 
description of the 
context?

The absence of a description of the context implies that the
needs cannot be identiﬁed or can be identiﬁed only partially 
or in general terms. Analysing the starting situation is 
necessary to develop a clear strategy.

Was there a needs 
analysis?

The identiﬁcation of the needs and of the link between these 
needs and the actions designed to address them is necessary
in order to ensure an eﬀective allocation of resources for
achieving EU education objectives.

Are indicative actions 
presented?

Deﬁning the possible categories of projects, beneﬁciaries, 
population targeted and the timeframe for the actions helps
to implement them eﬀectively.

Was the objective 
quantiﬁed?

To assess whether the objective has been achieved, it is 
necessary to quantify what the intervention should remedy 
in the initial situation. The objectives should be Speciﬁc, 
Measurable, Attainable, Relevant and Time-bound (SMART). 

Are relevant indicators 
deﬁned?

Indicators are necessary to monitor the outreach and results 
of the implemented actions. Indicators should be Relevant, 
Accepted, Credible, Easy and Robust (RACER).

Qualitative evaluation
ranging from  ’No 
description‘ to 
’Comprehensive 
description‘

Qualitative evaluation 
ranging from ’No 
analysis‘ to 
’Comprehensive
 analysis‘

Qualitative evaluation 
ranging from ’No 
planned actions‘ to 
’Comprehensive action
plan‘

Yes / No

Yes / No

Source: European Court of Auditors.

 
Annexes 

51

I

V
x
e
n
n
A

Overview of the modifications of the 2007-2013 ESF operational 
programmes examined

Member 
State

OP number

Was the OP 
modified during 
the programme 
period?

Were the 
education 
objectives 
modified?

Were allocations 
to education1 
modified?

Were education 
related 
indicators or 
their targets 
modified?

PT

PL

DE

2007PT05UPO001 — Programa Operacional 
Temático Potencial Humano 2007-2013

2007PT051PO001 — Programa Operacional do 
Fundo Social Europeu Para a Região Autónoma dos 
Açores 2007-2013 Pro-Emprego

2007PT052PO001 — Programa Operacional de 
Valorização do Potencial Humano e Coesão Social 
da Região Autónoma Da Madeira

2007PL051PO001 — Program Operacyjny Kapitał 
Ludzki Narodowe Strategiczne Ramy Odniesienia 
2007-2013

2007DE05UPO001 — Operationelles Programm 
des Bundes für den Europäischen Sozialfonds 
Förderperiode 2007-2013

2007DE051PO004 — Operationelles Programm 
des Freistaates Sachsen für den Europäischen 
Sozialfonds (ESF) in der Förderperiode 
2007 bis 2013

2007DE052PO008 — Operationelles Programm 
zur Umsetzung des ESF in NRW in der Förderphase 
2007-2013

2007DE051PO006 — Operationelles Programm 
für den Einsatz des Europäischen Sozialfonds im 
Freistaat Thüringen in den Jahren 2007 bis 2013

2007DE051PO002 — Europäischer Sozialfonds 
(ESF) Operationelles Programm des Landes 
Mecklenburg-Vorpommern im Ziel Konvergenz
Förderperiode 2007 bis 2013

2007DE051PO001 — Operationelles Programm 
des Landes Brandenburg für den Europäischen 
Sozialfonds (ESF) in der Förderperiode 
2007 bis 2013 Ziel Konvergenz Brandenburg 
Nordost und Brandenburg Südwest

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

No

No

No

Yes

No

No

No

No

No

No

Yes

No

No2

Yes

Yes

Yes

No

Yes

No

Yes

Yes

No

No

Yes

Yes

Yes

Yes

Yes

No

Yes

 
Annexes 

52

I

V
x
e
n
n
A

Member 
State

OP number

Was the OP 
modified during 
the programme 
period?

Were the 
education 
objectives 
modified?

Were allocations 
to education1 
modified?

Were education 
related 
indicators or 
their targets 
modified?

2007IT051PO007 — Programma Operativo 
Nazionale Competenze per lo sviluppo

2007IT051PO003 — Programma Operativo 
Regionale Sicilia per il Fondo Sociale Europeo 
2007-2013

2007IT051PO005 — Regione Puglia 
Programma Operativo Regionale 2007-2013  
Fondo Sociale Europeo

2007IT051PO001 — Programma Operativo 
Obiettivo Convergenza 
Fondo Sociale Europeo 2007-2013  
Regione Campania

2007IT052PO006 — Programma Operativo  
Regionale della Lombardia Ob. 2 FSE 2007-2013

2007IT052PO004 — Programma Operativo del 
Fondo Sociale Europeo Obiettivo Competitività 
Regionale e Occupazione Regione Lazio 2007-2013

2007HU05UPO001 — Társadalmi Megújulás 
Operatív Program

2007CZ05UPO002 — Operační program 
Vzdělávání pro konkurenceschopnost

2007CZ05UPO001 — Operační program Lidské 
zdroje a zaměstnanost 2007-2013

2007RO051PO001 — Programul Operaţional 
Sectorial Dezvoltarea Resurselor Umane
2007-2013

2007GR05UPO002 — Επιχειρησιακο Προγραμμα 
Εκπαιδευση Και Δια Βιου Μαθηση

2007GR05UPO001 — Επιχειρησιακο Προγραμμα 
Αναπτυξη Ανθρωπινου Δυναμικου

2007GR05UPO003 — Επιχειρησιακο Προγραμμα 
Διοικητικη Μεταρρυθμιση

IT

HU

CZ

RO

EL

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

No

No

No

No

No

No

No

No

No

No

No

Yes

No

No

Yes

Yes

Yes

No

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

Yes

Yes

Yes

No

No

No

Yes

Yes

Yes

No

Yes

Yes

Yes

 
Annexes 

I

V
x
e
n
n
A

Member 
State

OP number

2007ES051PO005 — Programa Operativo Fondo 
Social Europeo 
2007-2013 Andalucía

2007ES05UPO001 — Programa Operativo 
Plurirregional Adaptabilidad y Empleo Fondo Social 
Europeo 2007-2013

ES

2007ES051PO004 — Programa Operativo Fondo 
Social Europeo de Galicia 2007-2013

2007ES051PO003 — Programa Operativo Fondo 
Social Europeo 2007-2013 Extremadura

2007ES052PO008 —Programa Operativo Fondo 
Social Europeo 2007-2013 Madrid

2007ES051PO002 — Programa Operativo Fondo 
Social Europeo 2007-2013 Castilla la Mancha

2007UK05UPO001 — ESF England 2007-13 
Operational Programme

2007UK051PO002 — West Wales and the Valleys 
Convergence Programme Operational Programme 
for the European Social Fund 2007-2013

2007FR052PO001 — Compétitivité Régionale et 
Emploi 2007-2013 
Programme Opérationnel National Fonds Social 
Européen

2007FR051PO004 — Ile de La Réunion
Programme Opérationnel FSE 2007-2013

2007SK05UPO001 — Operačný program 
Vzdelávanie

2007BG051PO001 — Оперативна програма 
“Развитие на човешките ресурси” 2007-2013 г.

2007BE052PO005 — Operationeel Programma 
ESF Doelstelling 2 Vlaanderen 2007-2013

2007AT052PO001 — Operationelles Programm 
Beschäftigung Österreich 2007-2013

UK

FR

SK

BG

BE

AT

53

Was the OP 
modified during 
the programme 
period?

Were the 
education 
objectives 
modified?

Were allocations 
to education1 
modified?

Were education 
related 
indicators or 
their targets 
modified?

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

No

No

No

No

No

No

No

No

No

No

No

No

No

Yes

Yes

Yes

No

Yes

Yes

No

Yes

Yes

Yes

Yes

No

No

Yes

Yes

Yes

Yes

No

Yes

Yes

No

Yes3

Yes

Yes

Yes

Yes4

No

Yes

1 
2 
3 
4 

EU priority codes 62, 72, 73 and 74 are the ones identified as corresponding to allocations to education.
The allocation to priority code 74 was increased by 100 000 euro, which represents about 0.12 % of the allocations to education.
Target values were modified for programme-level indicators.
The target value of the context indicator concerning early school leavers was modified the light of the Europe 2020 targets.

 
Reply of  
the Commission

Executive summary

54

IV
Concerning the decrease of the funding allocated to education, the Commission notes that new thematic concen-
tration requirements have been introduced for the 2014-2020 programme period in order to ensure that support 
focusses primarily on the needs and challenges identified to achieve the Europe 2020 targets and objectives (see 
Article 18 of Regulation (EU) No 1303/2013).

VII
The Commission notes that while in the absence of a legal requirement to set common result indicators, aggrega-
tion at EU level has been a challenge, the work of the ESF Expert Evaluation Network bringing together Member 
States' monitoring and evaluation evidence have helped to partially overcome the obstacles. 

Moreover, the 2007-2013 ex-post evaluation and in particular the thematic studies launched have included a rigor-
ous exercise of converting specific result indicators to the common result categories (i.e.: gaining a job, a qualifica-
tion or another positive result) and to estimate success ratios based on the priority axes where consistent output 
and results were available. This included the conversion of result indicators expressed in percentages to absolute 
figures wherever possible. 

The Commission notes that the 2007-2013 regulatory framework contained no obligation to set targets and output/
result indicators at project level.

The Commission also notes that an increase or a decrease in the financial allocation does not imply automatically 
the need to change the target value. Once implementation has started and the economic context changes, Member 
States can realize that more or less or different actions need to be taken to achieve the initial target and this might 
require additional or less funding. Moreover, it is also important to recall that many of the 2007-2013 Operational 
programmes' amendments occurred in the midst of the economic crisis. Some measures to bring people back to the 
labour market might have then required more intensive investments to achieve the results initially planned before 
the economic crisis.

VIII
The Commission underlines the significant improvements in the monitoring and evaluation system of the 2014-2020 
programme period, particularly related to the obligation of reporting on common result indicators and to carry out 
impact evaluations at the level of priority axes. It also reminds of the improvements to the monitoring at operations 
level (see Commission replies to paragraphs 68 and 90).

As regards the measurement of effects on employment, the Commission underlines that common immediate and 
longer-term result indicators are to be collected and reported for all investment priorities.

Reply of the Commission 

55

IX (a)
The Commission accepts the recommendation and will consider for the post-2020 programme period specifying the 
result indicator(s) for which baseline and target values should be set for each investment priority.

IX (b) (i) First indent
The Commission accepts the recommendation and will implement it when considering Member States' requests for 
OP modifications.

IX (b) (i) Second indent
The Commission accepts this recommendation and is already implementing it in line with the applicable legal 
framework. Article 30 of Regulation (EU) No 1303/2013 requires that requests for amendment of programmes 
submitted by a Member State are duly justified and in particular set out the expected impact of the changes to the 
programme on achieving the Union strategy for smart, sustainable and inclusive growth and the specific objectives 
defined in the programme, taking account of this Regulation and the Fund-specific rules, the horizontal principles 
referred to in Articles 5, 7 and 8, as well as of the Partnership Agreement.

IX (b) (ii)
The Commission notes that this recommendation is addressed to the Member States.

IX (b) (iii)
The Commission accepts the recommendation.

When the need to reinforce the link between education and employment was identified, i.e. when a country-spe-
cific recommendation was issued by the Council, this was reflected in the ESF OPs at the time of programming. Arti-
cle 23 of Regulation (EU) No 1303/2013 moreover allows the Commission to request a Member State to review and 
propose amendments to its Partnership Agreement and relevant programmes to support the implementation of 
relevant country-specific recommendations. The Commission therefore already has the possibility to request, where 
relevant - i.e. when new Country Specific Recommendations identify weak links between education and employ-
ment - that funding is allocated to reinforcing the link between education and employment.

In addition, the current monitoring system allows the monitoring of all common output and result indicators as 
Member States are obliged to report on all common output and result indicators (Article 5 of the ESF Regulation 
for 2014-2020). First data will be submitted to the Commission in the first annual implementation reports due by 
31 May 2016. It will therefore be possible throughout the programme period to monitor the employment results 
under the education thematic objective even when the Member States have not set targets for these results indica-
tors in their OPs. The Commission will make sure that these results are made visible through the summary of the 
annual implementation reports.

Reply of the Commission 

56

Introduction

14
Concerning the decrease of the funding allocated to education, the Commission notes that new thematic concen-
tration requirements have been introduced for the 2014-2020 programme period in order to ensure that support 
focusses primarily on the needs and challenges identified to achieve the Europe 2020 targets and objectives (see 
Article 18 of Regulation (EU) No 1303/2013). 

26
See Commission reply to paragraph 14.

Common Commission reply to paragraphs 33 and 34  
While in the absence of a legal requirement to set common result indicators, aggregation at EU level has been 
a challenge, the work of the ESF Expert Evaluation Network bringing together Member States' monitoring and 
evaluation evidence have helped to partially overcome the obstacles. 

Moreover, the 2007-2013 ex-post evaluation and in particular the thematic studies launched have included a rigor-
ous exercise of converting specific result indicators to the common result categories (i.e.: gaining a job, a qualifica-
tion or another positive result) and to estimate success ratios based on the priority axes where consistent output 
and results were available. This included the conversion of result indicators expressed in percentages to absolute 
figures wherever possible. 

42
The Commission notes that an increase or a decrease in the financial allocation does not imply automatically the 
need to change the target value. Once implementation has started and the economic context changes, Member 
States can realize that more or less or different actions need to be taken to achieve the initial target and this might 
require additional or less funding. Moreover, it is also important to recall that many of the 2007-2013 OP amend-
ments occurred in the midst of the economic crisis. Some measures to bring people back to the labour market 
might have then required more intensive investments to achieve the results initially planned before the economic 
crisis. 

43
The Commission notes that for two OPs, Campania and Lazio, the financial allocation to education did not change 
radically in relative terms. In relation to Campania, the Commission assured itself, on the basis of data on outputs 
and results already achieved at the time of the modification request, that the targets initially set remained valid 
despite the reduction of the financial allocation.

Furthermore, the Commission would like to underline that it encourages Member States to be as ambitious as pos-
sible in terms of results planned and cost/participant ratio. If the Member States consider they can achieve the same 
level of results at a lower cost, the Commission will not question this. On the contrary, a substantial increase of the 
financial allocation without a change in the target for participants reached would have prompted a further analysis 
by the Commission during the OP modification procedure.

Reply of the Commission 

57

44
See Commission reply to paragraph 42.

48
The Commission notes that the 2007-2013 regulatory framework contained no obligation to set targets and output/
result indicators at project level. 

The ESF expert evaluation network and the ex-post evaluation have shown the need to improve the monitoring 
arrangements so that comprehensive, reliable and comparable result indicators are available at project level, espe-
cially after the completion of the intervention, to allow for an assessment of the performance and sustainability of 
results. 

This lesson is already reflected in the 2014-2020 framework. Each operation supporting participants has to col-
lect data on individual participants based on common indicators for participants and relevant specific indicators, 
and performance data should be recorded and stored by operation in a computerised form in order to allow them 
to be aggregated where necessary for the purpose of monitoring, evaluation, financial management, verification 
and audit (Article 24.2 of Commission Delegated Regulation 480/2014). Targets have to be linked to the output and 
result indicators stored where relevant.

51
See Commission reply to paragraph 48.

61
The Commission acknowledges a limited number of inconsistencies in the text of the 2014-2020 OPs regarding cri-
teria and actions on ex-ante conditionalities. The Commission and the Member States have been working to identify 
any such inconsistencies and to address them in compliance with Article 30 of Regulation (EU) No 1303/2013.

65
See Commission reply to paragraph 68.

67
The selection criteria are indeed drawn up and applied by the Managing Authority. These criteria need to ensure 
that the operations selected contribute to the achievement of the specific objectives and results of the relevant 
priority (Article 125(3)(a)(i) of Regulation (EU) No 1303/2013). These criteria need to be approved by the Monitoring 
Committee (Article 110(2)(a) of Regulation (EU) No 1303/2013) in which the Commission participates in an advisory 
capacity.

The Commission has drawn Member States’ attention to the importance of the selection criteria for achieving the 
specific objectives (in particular to deliver on relevant country-specific recommendations) in its discussions in the 
ESF Technical Working Group. 

Reply of the Commission 

58

68
The Commission notes that in case the Member State was not able to set a quantified target for a result indicator 
due to the lack of reliable baselines, general ex-ante conditionality on statistical systems and result indicators has 
been considered as unfulfilled and an action plan was set up to remedy the shortcoming. 

73
The Commission notes that result indicators are to be designed in line with the intervention logic of the programme 
so as to reflect the specific objectives pursued. Their quantified targets enable to measure the progress towards 
meeting the specific objective. The latter reflect the specific challenges, in particular the country specific recom-
mendations of the Member State in line with thematic concentration and increased contribution of the ESF support 
to the Europe 2020 strategy. 

The ensuing diversity of specific objectives that can contribute to a specific investment priority makes it difficult to 
standardize suitable indicators for target-setting in advance.

At the same time, although not always linked to quantified targets, all common output and result indicators will be 
monitored and reported for all investment priorities, allowing to calculate success ratios.

Box 4 - Example where targets for additional outcome indicator would have been 
helpful
The Commission notes that: 

—  the description of the specific objective also includes the promotion of activation and employability of young 

people;

—  the indicator is coherent with the recipients targeted in the specific objective, which include students and fami-

lies but also inactive and unemployed young people and corresponds to some of the activities foreseen (train-
eeships, guidance and school-work experiences);

—  the focus on the employment effects of interventions is also coherent with the fact that the OP will intervene in 
co-ordination with (and not duplicating) the interventions of the National OP ‘Per la Scuola’, which focuses on 
the education side of early school leaving.

80
As regards the measurement of effects on employment, the common immediate and longer-term result indicators 
will be collected and reported for all investment priorities. As regards target setting to pre-defined result indicators, 
see Commission reply to paragraph 73.

Reply of the Commission 

59

82
The Commission notes that the 2007-2013 regulatory framework contained no obligation to set targets and output/
result indicators at project level. 

83
The Commission notes that according to Article 37 of Regulation (EC) No 1083/2006 an OP needs to include an 
analysis of the situation of the eligible area or sector in terms of strengths and weaknesses and the strategy chosen; 
a justification of the priorities chosen as well as information on the priority axes and their specific targets. How-
ever, when a Member State has a Country Specific Recommendation on education, the needs analysis can be fairly 
limited.

Monitoring and evaluation of performance at Operational Programme level was possible. While in the absence of 
a legal requirement to set common result indicators, aggregation at EU level has been a challenge, the work of the 
ESF Expert Evaluation Network bringing together Member States' monitoring and evaluation evidence have helped 
to partially overcome the obstacles. 

Moreover, the 2007-2013 ex-post evaluation and in particular the thematic studies launched have included a rigor-
ous exercise of converting specific result indicators to the common result categories (i.e.: gaining a job, a qualifica-
tion or another positive result) and to estimate success ratios based on the priority axes where consistent output 
and results were available. This included the conversion of result indicators expressed in percentages to absolute 
figures wherever possible. 

84
The Commission notes that an increase or a decrease in the financial allocation does not imply automatically the 
need to change the target value. Once implementation has started and the economic context changes, Member 
States can realize that more or different and more costly actions need to be taken to achieve the initial target and 
this might require additional funding. Moreover, it is also important to recall that many of the 2007-2013 OP amend-
ments occurred in the midst of the economic crisis. Some measures to bring people back to the labour market 
might have then required more intensive investments to achieve the results initially planned before the economic 
crisis.

86
The Commission notes that the 2007-2013 regulatory framework contained no obligation to set targets and output/
result indicators at project level.

87
The Commission acknowledges a limited number of inconsistencies in the text of the 2014-2020 OPs regarding cri-
teria and actions on ex-ante conditionalities. The Commission and the Member States have been working to identify 
any such inconsistencies and to address them in compliance with Article 30 of Regulation (EU) No 1303/2013.

Reply of the Commission 

60

89
The selection criteria are indeed drawn up and applied by the Managing Authority. These criteria need to ensure 
that the operations selected contribute to the achievement of the specific objectives and results of the relevant 
priority (Article 125(3)(a)(i) of Regulation (EU) No 1303/2013). These criteria need to be approved by the Monitoring 
Committee (Article 110(2)(a) of Regulation (EU) No 1303/2013) in which the Commission participates in an advisory 
capacity.

The Commission has drawn Member States’ attention to the importance of the selection criteria for achieving the 
specific objectives (in particular to deliver on relevant country-specific recommendations) in its discussions in the 
ESF Technical Working Group.

90
The Commission notes that result indicators are to be designed in line with the intervention logic of the programme 
so as to reflect the specific objectives pursued. Their quantified targets enable to measure the progress towards 
meeting the specific objective. The latter reflect the specific challenges, in particular the country specific recom-
mendations of the Member State in line with thematic concentration and increased contribution of the ESF support 
to the Europe 2020 strategy. 

The ensuing diversity of specific objectives that can contribute to a specific investment priority makes it difficult to 
standardize suitable indicators for target-setting in advance.

At the same time, although not always linked to quantified targets, all common output and result indicators will be 
monitored and reported for all investment priorities, allowing to calculate success ratios.

Recommendation 1
The Commission accepts the recommendation and will consider for the post-2020 programme period specifying the 
result indicator(s) for which baseline and target values should be set for each investment priority.

Recommendation 2 First indent
The Commission accepts the recommendation and will implement it when considering Member States' requests for 
OP modifications. 

Recommendation 2 Second indent
The Commission accepts this recommendation and is already implementing it in line with the applicable legal 
framework. Article 30 of Regulation (EU) No 1303/2013 requires that requests for amendment of programmes 
submitted by a Member State are duly justified and in particular set out the expected impact of the changes to the 
programme on achieving the Union strategy for smart, sustainable and inclusive growth and the specific objectives 
defined in the programme, taking account of this Regulation and the Fund-specific rules, the horizontal principles 
referred to in Articles 5, 7 and 8, as well as of the Partnership Agreement.

Reply of the Commission 

61

Recommendation 3
The Commission notes that this recommendation is addressed to the Member States.

Recommendation 4
The Commission accepts the recommendation.

When the need to reinforce the link between education and employment was identified, i.e. when a country-spe-
cific recommendation was issued by the Council, this was reflected in the ESF OPs at the time of programming. Arti-
cle 23 of Regulation (EU) No 1303/2013 moreover allows the Commission to request a Member State to review and 
propose amendments to its Partnership Agreement and relevant programmes to support the implementation of 
relevant country-specific recommendations. The Commission therefore already has the possibility to request, where 
relevant - i.e. when new Country Specific Recommendations identify weak links between education and employ-
ment - that funding is allocated to reinforcing the link between education and employment.

In addition, the current monitoring system allows the monitoring of all common output and result indicators as 
Member States are obliged to report on all common output and result indicators (Article 5 of the ESF Regulation 
for 2014-2020). First data will be submitted to the Commission in the first annual implementation reports due by 
31 May 2016. It will therefore be possible throughout the programme period to monitor the employment results 
under the education thematic objective even when the Member States have not set targets for these results indica-
tors in their OPs. The Commission will make sure that these results are made visible through the summary of the 
annual implementation reports.

HOW TO OBTAIN EU PUBLICATIONS

Free publications:

(cid:127) 

one copy: 
via EU Bookshop (http://bookshop.europa.eu);

(cid:127)   more than one copy or posters/maps: 

from the European Union’s representations (http://ec.europa.eu/represent_en.htm);  
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm);  
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or  
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*).

(*) 

The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:

(cid:127) 

via EU Bookshop (http://bookshop.europa.eu).

Event

Adoption of Audit Planning Memorandum / Start of audit

Official sending of draft report to Commission (or other auditee)

Adoption of the final report after the adversarial procedure

Commission’s (or other auditee’s) official replies received in all languages

Date

12.11.2014

31.3.2016

11.5.2016

17.6.2016

 
 
Investment in education is crucial for improving labour 
productivity, professional development and economic 
growth. The level of education achieved has a direct link 
to both employability and the quality of employment 
obtained. The European Social Fund contributed nearly 
34 billion euros to supporting education measures between 
2007 and 2013 and plans to allocate a further 27 billion 
between 2014 and 2020. We assessed whether the EU’s 
education objectives had been adequately considered in 
the programmes and related projects undertaken before 
2013 and in their design for the future. Overall, we 
concluded that they have been adequately considered. 
However, we found some shortcomings in monitoring and 
project performance could not always be demonstrated 
because of a lack of quantified objectives and indicators.

EUROPEAN
COURT
OF AUDITORS