Selection and monitoring for ERDF and ESF projects in the 2014–2020 period are still mainly outputs-oriented. Special report No 21, 2018.
Date: 2018-10-31
Institutions: European Court of Auditors
Subjects: Cohesion Fund, audit, budget appropriation, economic and social cohesion, financial aid, operating result, project evaluation, report

EN 

2018 

NO 21

Special Report  Selection and monitoring for 
ERDF and ESF projects in the 
2014–2020 period are still 
mainly outputs-oriented

(pursuant to Article 287(4), second subparagraph, TFEU) 

 
Audit team 

The ECA’s special reports set out the results of its audits of EU policies and programmes, or of 
management-related topics from specific budgetary areas. The ECA selects and designs these audit 
tasks to be of maximum impact by considering the risks to performance or compliance, the level of 
income or spending involved, forthcoming developments and political and public interest. 

This performance audit was carried out by Audit Chamber II Investment for cohesion, growth and 
inclusion spending areas, headed by ECA Member Iliana Ivanova. The audit was led by ECA Member 
Ladislav Balko, supported by Branislav Urbanič, Head of Private Office and Zuzana Frankova, Private 
Office Attaché; Myriam Cazzaniga, Principal Manager; Pekka Ulander Head of Task; Michaela Binder, 
Seconded National Expert.  

 
 
 
CONTENTS 

Glossary 

List of abbreviations 

Executive summary 

Introduction 

Cohesion policy is the EU’s main investment policy 

Performance orientation is a key priority for the Commission and the 
Member States during the 2014-2020 period 

The importance of project selection and monitoring to achieve 
performance 

Main participants and their role in project selection and monitoring 

Audit scope and approach 

Observations 

Project selection is insufficiently focused on results 

The selection procedures support the selection of projects relevant 
to OPs’ objectives 

The focus on results when assessing project applications was weak, 
and seldom included quantified results 

Beneficiaries were encouraged to apply for funding 

Selection procedures did not usually involve any comparison 
between project applications, resulting in a risk that the best projects 
will not be selected 

2 

Paragraph 

I - V 

1 - 9 

1 - 2 

3 - 4 

5 - 6 

7 - 9 

10 - 14 

15 - 73 

15 - 34 

18 - 22 

23 - 27 

28 - 31 

32 - 34 

The quality of monitoring data is at risk, and it is mostly outputs-oriented 

35 - 73 

Assurance on the quality of monitoring data may not be provided in 
time for the 2019 performance review 

The information generated on the basis of the data collected is still 
mostly outputs-oriented 

Conclusions and recommendations 

38 - 62 

63 - 73 

74 - 83 

 
 
3 

Annex I - 

List of projects selected for this audit 

Annex II - 

Thematic objectives 

Annex III - 

List of selection procedures examined in this audit 

Annex IV - 

List of OPs examined in this audit 

Annex V - 

Priority axis, investment priorities and specific objectives to which the selected 
projects in Annex I correspond 

Annex VI -  OP’s output and result indicators for the priority axis described under Annex V 

funded with ERDF 

Annex VII -  Output and result indicators for the priority axis described under Annex V 

funded with ESF 

Annex VIII -  Outputs and results in the selection procedures and applications examined 

The Commission’s replies 

 
 
 
 
4 

GLOSSARY 

Europe 2020 Strategy: The EU’s growth strategy for the 2010–2020 period, aimed at 

allowing it to recover from the crisis. The strategy is split into five headline targets covering 

employment; research and development; climate and energy; education; and social inclusion 

and poverty reduction. 

European Code of Conduct on Partnership: A set of principles set out in Commission 

Delegated Regulation (EU) No 240/2014 to support Member States in organising 

partnerships for drawing up and implementing partnership agreements and operational 

programmes. 

European Structural and Investment Funds (ESIF or ESI funds): The ESI funds are five 

separate funds that aim to reduce regional imbalances across the Union, with policy 

frameworks set for the seven-year MFF budgetary period. The funds include the European 

Regional Development Fund (ERDF), the European Social Fund (ESF), the Cohesion Fund (CF), 

the European Agricultural Fund for Rural Development (EAFRD) and the European Maritime 

and Fisheries Fund (EMFF). 

Ex-ante conditionalities: Conditions which Member States must meet before they can 

receive any funding under the European Structural and Investment Funds. When preparing 

operational programmes in the 2014-2020 programme period, Member States were 

required to assess whether these conditions have been fulfilled. If they have not, action 

plans were required to be drawn up to ensure fulfilment by 31 December 2016. 

Financial indicators: Values used to monitor progress in terms of the (annual) commitment 

and payment of the funds available for any project, measure or programme in relation to its 

eligible cost. 

Indicator: A measurable proxy for an objective, providing information to support an 

assessment of the degree to which an objective has been met. 

Intervention logic: The link between the assessed needs, objectives, inputs (planned and 

allocated), outputs (targeted and achieved) and results (intended and actual). 

 
 
5 

Investment priorities: For each priority axis, Member States must set out in their 

operational programmes the investment priorities and corresponding specific objectives. 

They are listed in the specific Regulations governing the ERDF, ESF and CF. 

Managing authority (MA): A managing authority is a public or private body which has been 

designated by a Member State to manage an operational programme. Its tasks include 

selecting projects to be funded, monitoring how projects are implemented, and reporting to 

the Commission on financial aspects and achieved results. 

Monitoring: The regular examination of expenditure, outputs and results, which provides 

up-to-date information on whether projects/programmes are progressing as intended. 

Monitoring committee: A committee which monitors the implementation of an OP. It 

consists of “representatives of the relevant Member State authorities” (e.g. representatives 

of the managing, certifying and audit authorities, implementing bodies, employer or 

employee organisations, and civil society). The Commission also has an observer role. 

Operational Programme (OP): A statement of a Member State’s priorities and specific 

objectives which describes how funding (EU and national public and private co-financing) will 

be used over a given period (currently seven years) to finance projects. The projects within 

an OP must contribute to a certain number of objectives specified at the level of the OP’s 

priority axes. OP funding may come from the ERDF, CF and/or ESF. The OP is drawn up by 

the Member State and has to be approved by the Commission before any payments can be 

made from the EU budget. OPs can only be modified during the programme period if both 

parties agree. 

Outputs: Something that is produced or accomplished with the resources allocated to an 

intervention (e.g. training courses delivered to unemployed young people, number of 

sewage plants, or a length of road built). 

Output indicators: Values used to measure the outputs of the operations supported or the 

outputs at operational programme level. 

 
 
6 

Performance framework: A set of milestones and targets defined for each priority axis in an 

Operational Programme, forming an important pillar of the performance-orientation 

approach. 

Performance reserve: Resources constituting 6 % of the resources allocated to the ERDF, ESF 

and CF or to the EAFRD and EMFF, to be made available after the 2019 performance review 

if certain requirements are met or exceeded. 

Priority axis: One or more investment priorities associated with one thematic objective. The 

funding in an operational programme is organised by priority axis. 

Results: Immediate changes that arise for beneficiaries at the end of their participation in an 

intervention (e.g. improved accessibility to an area due to the construction of a road, 

trainees who have found a job). 

Result indicators: Values used to measure the results generated by supported projects, or 

the results achieved at operational programme level. 

Selection/award criteria: Predefined criteria used in selection procedures to assess 

applicants’ ability to complete the proposed action or work programme. 

Specific objective: The envisaged result to which EU action aims to contribute. 

 
 
 
 
 
7 

LIST OF ABBREVIATIONS 

AA 
AIR 
CA 
CF 
CPR 
CSF 
DMA 
EC 
ERDF 
ESF 
ESI funds or ESIF 
IB 
IP 
MA 
MS 
OP 
PA 
SMEs 
SO 
TO 

Audit Authority 
Annual Implementation Report 
Certifying authority 
Cohesion Fund 
Common Provision Regulation 
Common Strategic Framework 
Delegated Managing Authority 
European Commission 
European Regional Development Fund 
European Social Fund 
European Structural and Investment Funds 
Intermediate body 
Investment priority 
Managing Authority 
Member State 
Operational Programme 
Priority axis 
Small and medium-sized enterprises 
Specific objective 
Thematic objective 

 
 
 
 
 
8 

EXECUTIVE SUMMARY 

I. 

For the 2014-2020 programming period, the EU has allocated almost €349.4 billion 

towards achieving its objectives in terms of cohesion policy. This policy aims to support job 

creation, business competitiveness, economic growth, sustainable development and 

improvement in quality of life. To achieve these objectives, funding is allocated to projects 

delivered by beneficiaries in Member States. Authorities in Member States select projects to 

fund and monitor their implementation. It is critical that this funding is allocated effectively, 

namely through the delivery of expected results. 

II. 

In order to support this goal, the Commission implemented various measures aimed at 

increasing focus on results in the 2014-2020 period. In previous reports, we have welcomed 

the improvements that these have brought. But we have also highlighted serious 

weaknesses in the effectiveness of these measures, and the quality of the monitoring 

information relating to spending on Cohesion Policy. This report consolidates and develops 

these previous findings. 

III.  We examined how well the focus on results is embedded within the way projects are 

selected, and how well the Commission and Member States can demonstrate the EU budget 

is well spent. As such, we have examined the selection and monitoring arrangements of 

projects using Cohesion funding. 

IV.  We conclude that, despite longstanding intentions, the design of selection procedures 

and the processes themselves continue to emphasise outputs and absorption rather than 

results. Moreover, shortcomings in the monitoring arrangements made it difficult to assess 

the extent to which EU funding has contributed to the achievement of the EU’s and Member 

States’ objectives. Specifically, for the OPs we visited, we made the following findings: 

•  On project selection, potential beneficiaries were well informed and provided with 

appropriate support to access to EU funding. The procedures we examined were 

designed to support the selection of projects relevant to the OPs’ objectives, but 

selection criteria seldom required applicants to define quantified result indicators at 

project level. Furthermore, most project selection was done on a first-come first-served 

 
 
9 

basis. Applications were only scored and ranked against each other in one selection 

procedure. 

•  On monitoring, we found that monitoring systems had only become functional at a late 

stage, mainly because of delays in adopting the legislative framework. There remain 

weaknesses in some of the IT systems used for collecting and aggregating monitoring 

data. Furthermore, slow progress in the audits of the respective monitoring systems 

introduced the risk that there now may not be enough time to take any necessary 

corrective action before the performance review in 2019. 

•  Monitoring information remains mostly outputs-oriented. The Commission’s main 

report to measure achievements presents progress for the main output indicators 

alongside the implementation of funding. However, there is only limited information on 

the achievement of results. 

V.  We therefore recommend the following: 

• 

In order to ensure a consistent and genuine results-oriented approach to the selection 

of projects, Member States should ensure that a comparison between project 

applications takes place, require beneficiaries to define at least one genuine result 

indicator for each project and carry out an assessment of the expected results and 

indicators in the assessment report for the applications. 

• 

In order to ensure a results-oriented monitoring, Member States should include one or 

more genuine and quantified result indicators in the grant agreement, which contribute 

to the result indicators set at OP level, and the Commission should define common 

result indicators for the ERDF based on a common definition of “results”. 

• 

The Commission should improve the reporting on performance and ensure that a 

meaningful performance review takes place in 2019. 

 
 
 
 
10 

INTRODUCTION 

Cohesion policy is the EU’s main investment policy 

1.  Cohesion policy is the EU’s main investment policy. It is aimed at supporting job 

creation, business competitiveness, economic growth, sustainable development and 

improvements in quality of life. Around one-third of the EU’s budget is allocated to cohesion 

policy. At current prices, this amounted to around €230 billion in the 2000-2006 programme 

period, €346.5 billion in the 2007-2013 period and €349.4 billion in the 2014-2020 period. 

2.  Cohesion policy is delivered through the European Regional Development Fund (ERDF), 

the European Social Fund (ESF) and the Cohesion Fund (CF). These funds, together with the 

European Agricultural Fund for Rural Development (EAFRD) and the European Maritime and 

Fisheries Fund (EMFF), are part of the European Structural and Investment Funds (ESIF). 

They are implemented through operational programmes (OPs) which set out investment 

priorities and specific objectives1. These also describe how funding will be used during the 

programme period to finance projects. The OPs are implemented by the Member States and 

their regions. This means that the Member States and their regions select, monitor and 

evaluate projects. 

Performance orientation is a key priority for the Commission and the Member States 

during the 2014-2020 period 

3.  The Europe 2020 Strategy is the EU’s ten-year jobs and growth strategy. It was launched 

in 2010 to create conditions conducive to smart, sustainable and inclusive growth. With a 

view to meeting the objectives and targets set in the Europe 2020 Strategy, the Commission 

emphasised in 2010 the need to improve the effectiveness of the cohesion policy by focusing 

on results2. 

1 

2 

Investment priorities and specific objectives are established based on the Member State’s 
partnership agreement, which is an agreement between the Member State and the Commission 
outlining the country’s strategic goals and investment priorities. 

COM(2010) 700 final of 19 October 2010 “The EU Budget Review”. 

 
 
                                                      
11 

4.  The legislative package for cohesion policy in the 2014-2020 programme period 

therefore introduced significant changes intended to increase focus on performance3. The 

main measures are the following: 

• 

The introduction of ex-ante conditionalities (EACs)4 which require a Member State to 

fulfil certain conditions for the effective use of ESI funds (e.g. existence of 

policy/strategic frameworks) by the end of 2016. One of them, general ex-ante 

conditionality No 75, requires “the existence of a system of result indicators necessary 

to select actions which most effectively contribute to desired results, to monitor 

progress towards results”. 

• 

The introduction of a performance reserve, setting aside part of the EU funding 

allocated to the OPs, to be released only if a subset of indicators (mainly 

spending/output indicators) reach pre-defined milestones. The release of this reserve in 

2019 will be based on a performance review carried out in 20196. 

3 

4 

5 

6 

All these aspects were dealt with in our Special Report No 2/2017 “The Commission’s 
negotiation of 2014-2020 Partnership Agreements and programmes in Cohesion: spending more 
targeted on Europe 2020 priorities, but increasingly complex arrangements to measure 
performance” and Special Report No 15/2017 “Ex ante conditionalities and performance 
reserve in Cohesion: innovative but not yet effective instruments”. 

Article 19 of Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 
17 December 2013 laying down common provisions on the European Regional Development 
Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural 
Development and the European Maritime and Fisheries Fund and laying down general 
provisions on the European Regional Development Fund, the European Social Fund, the 
Cohesion Fund and the European Maritime and Fisheries Fund and repealing Council Regulation 
(EC) No 1083/2006 (OJ L 347, 20.12.2013, p. 320). 

General Ex ante conditionality 7, Annex XI Part II of Regulation (EU) No 1303/2013. 

Articles 20, 22 and 96 and Annex II of Regulation (EU) No 1303/2013. 

 
 
                                                      
12 

•  A reinforced intervention logic during the programming exercise7 requiring Member 

States to assess their needs and commit to a set of results before considering where 

and how to spend cohesion funds. 

•  A more consistent use of performance indicators to measure progress towards 

achieving results8. 

The importance of project selection and monitoring to achieve performance 

5. 

In 2017, we concluded9 that the Commission and Member States had been successful in 

developing OPs with a more robust intervention logic. This means that the OPs set out in a 

consistent way the aims of the interventions (specific objectives/results) and how these will 

be achieved (required funding, actions to be undertaken and expected outputs). 

6.  However, to achieve policy effectiveness, it is not only important to set clear goals and 

consider results during the programming exercise. It is essential that a results-oriented 

approach is followed through during the implementation phase of the OP, as it is mainly the 

quality of projects funded and their added value in terms of tangible results which will 

determine the effectiveness of a policy. Project selection, monitoring and reporting is thus 

essential for achieving performance. An overview of the implementation cycle of an OP and 

the bodies concerned is provided in Figure 1. The processes relevant to our audit are 

highlighted in yellow. 

7 

8 

9 

Articles 9 and 96 of Regulation (EU) No 1303/2013. 

Annex I of Regulations (EU) No 1300/2013, No 1301/2013 and No 1304/2013, Article 5 of 
Regulation (EU) No 1301/2013, article 3 of Regulation (EU) No 1304/2013 and Article 96(2)(b)(ii) 
of Regulation (EU) No 1303/2013. 

Special Report No 2/2017. 

 
 
                                                      
13 

Figure 1 - Overview of the implementation cycle of an OP 

Programming  and implementation

Reporting

Member State

Preparation of the operational 
programme

European Commission

Analysis and approval of the 
operational programme

Analysis and appraisal of the 
annual implementation reports

Managing authority (or intermediate body)

Selection of the projects or 
project providers

Preparation of annual 
implementation  reports including 
aggregated monitoring 
information and evaluation 
results

Project providers

Implementation  of the projects

Provision of monitoring 
information on projects

Source: ECA. 

Main participants and their role in project selection and monitoring 

7.  Projects are selected by the national and regional authorities responsible for managing 

the OPs. These managing authorities (MAs), or intermediate bodies (IBs) to which they 

delegate powers, lay down selection criteria, organise selection committees and decide 

 
 
 
14 

which projects will receive European funding. The Commission is not involved in the 

selection of projects as opposed to its crucial role in the negotiation of OPs. However, the 

Commission is involved through its advisory role in the OPs monitoring committees10 where 

the methodology and criteria used for selecting projects are approved. It also provides 

guidance, based on best practices and shared experience with Member States. 

8.  The responsibility for monitoring projects lies with the Member States. Managing 

authorities monitor the implementation of OPs and projects, aggregate monitoring 

information and submit Annual Implementation Reports (AIRs) to the Commission. The 

monitoring committee also reviews the implementation of the OPs and approves AIRs. Here 

also, the Commission’s role is limited to issuing guidance, examining the performance of OPs 

together with the MAs, and making observations on AIRs. 

9. 

In its reporting role nevertheless, the Commission submits its annual synthesis of 

Member States’ AIRs and strategic reports (in 2017 and 2019) to the discharge authority 

(European Parliament), and to the European Economic and Social Committee and the 

Committee of the Regions11. 

AUDIT SCOPE AND APPROACH 

10.  This audit sought to answer the question of whether the selection of projects and 

monitoring systems for ERDF and ESF projects in the 2014-2020 programme period were 

results-oriented. Given the level of advancement of the OPs’ implementation, and building 

on the audits already carried out, this audit allowed us for the first time to follow the 

“results-oriented approach” through the whole implementation phase as the first results are 

becoming available. It is also of high relevance to the performance review scheduled for 

2019. 

11.  In the first part, we report on our analysis of selection procedures in place for the 

audited OPs. We assess the design of these selection procedures, especially the 

10 

11 

Article 48(3) of Regulation (EU) No 1303/2013. 

Article 53(2) of Regulation (EU) No 1303/2013. 

 
 
                                                      
15 

determination of the selection criteria and their result orientation, as well as the phase of 

evaluation of the applications. In the second part, we examine the management and control 

systems in place and the monitoring that has been carried out so far by the MAs/IBs of the 

visited OPs to measure operational performance at programme and national level. We 

review the reliability of data, its availability and capacity to produce relevant information for 

management and for reporting to the Commission. 

12.  In total, we examined 34 projects (see list of projects in Annex I). We focused on the 

ERDF and ESF, and more specifically on three thematic objectives (TOs)12 (see a description 

of the 11 thematic objectives in Annex II): 

• 

• 

• 

TO3, enhancing the competitiveness of SMEs (16 projects), 

TO8, promoting sustainable and quality employment and supporting labour mobility (12 

projects), and 

TO9, promoting social inclusion and combating poverty and discrimination (6 projects). 

13.  We looked at projects selected directly by the MA (or intermediary body) through 20 

selection procedures (see list in Annex III), from seven OPs located in four Member States: 

the Czech Republic, France, Italy and Finland (see list in Annex IV). These OPs and Member 

States were selected according to the materiality of funding allocated and the stage of the 

projects’ implementation. We selected the projects based on the amount of grant received 

and their level of advancement at the time of the audit. The priority axis, investment 

priorities and specific objectives to which the selected projects correspond are listed in 

Annex V. The related output and result indicators are listed in Annex VI for the ERDF and 

Annex VII for the ESF. 

14.  We also reviewed the actions taken by the Commission in relation to project selection 

and monitoring of operational programmes for the ESF and the ERDF in general. 

12 

Article 9 of Regulation (EU) No 1303/2013. 

 
 
                                                      
16 

OBSERVATIONS 

Project selection is insufficiently focused on results 

15.  The selection procedures put in place by the managing authorities should prioritise the 

selection of those projects that are the most relevant to the objectives pursued and which 

have the greatest likelihood of delivering results, as well as ensuring that available funding is 

spent in line with the rules. We checked the extent to which results had been reflected in 

the design of selection procedures (e.g. the selection criteria used for assessing an 

application, the method used for assessing projects and the documentation available to 

potential bidders) as well as in the implementation of the selection procedures and thus in 

the projects selected. 

16.  Previous ECA audit work13 showed that the Commission and Member States had been 

successful in developing OPs with a more robust intervention logic i.e. with a clear link 

between development needs, specific objectives and result indicators. This intervention logic 

should be reflected in project selection procedures. We consider that there are a number of 

key criteria that should be taken into account by MAs when designing procedures to ensure 

that they are results-oriented: 

• 

• 

• 

• 

the selection criteria applied should ensure that projects are aligned with OP objectives; 

the selection criteria should support the selection of projects that have not only 

quantified output indicators but also quantified result indicators to measure projects 

achievements; 

beneficiaries should be encouraged to submit applications to ensure that a high number 

of projects are available for selection; 

finally, the project selection should be based on a direct comparison of applications to 

identify and prioritise the projects best suited for funding. 

13 

See ECA’s SR No 2/2017. 

 
 
                                                      
17 

17.  The extent to which MAs applied these variables while processing the applications 

received and how they influenced the selection of results-oriented projects is described in 

the next paragraphs (see paragraphs 18 to 38). 

The selection procedures support the selection of projects relevant to OPs’ objectives 

18.  Two measures were introduced for the 2014-2020 programme period legislative 

framework which are designed to trigger the selection of projects relevant to OPs’ 

objectives: 

•  OPs must set out “guiding principles” for the selection of projects for each priority 

axis14; 

•  MAs must design selection procedures and criteria “that ensure the contribution of 

operations to the achievement of the specific objectives and results of the relevant 

priority”15. 

19.  In addition to these, the following requirements were also introduced for the 

2014-2020 period to trigger the selection of more results-oriented projects: 

• 

under general ex-ante conditionality 7 (see also paragraph 4), “a system of result 

indicators necessary to select actions which most effectively contribute to desired 

results” must be in place; 

•  MAs must also draw up selection procedures and criteria that “ensure the contribution 

of operations to the achievement of the […] results of the relevant priority”16. 

20.  The “guiding principles” act as guidance for project selection, detailing the main 

selection criteria to be used. Coupled with the description of the projects to be allocated 

funding, providing examples of such projects, and beneficiaries, they give a good overview of 

the type of projects to be funded under each specific objective. These principles are used as 

14 

15 

16 

Article 96(2)(b) of Regulation (EU) No 1303/2013. 

Article 125(3)(a)(i) of Regulation (EU) No 1303/2013. 

Article 125(3)(a) of Regulation (EU) No 1303/2013. 

 
 
                                                      
18 

a basis for preparing the selection criteria, and may be referred to directly within selection 

criteria. 

21.  We found that, in some cases, the principles were of a general nature; however, they all 

referenced the need for selected projects to be in line with the specific objectives of the OP, 

as described under each priority axis. The selection criteria, for the selection procedures we 

examined, were consistent with the guiding principles. Each procedure entailed at least one 

criterion about projects’ relevance and their contribution to the specific objective. 

22.  Overall, the selected applications we examined were assessed against the criteria 

defined within the selection procedure. We found that the 34 projects we examined for this 

audit were relevant to the OP specific objectives. 

The focus on results when assessing project applications was weak, and seldom included 

quantified results 

23.  We examined how MAs carried out their selection of projects in practice, in particular 

the existence in the project proposal of clear expected results and objectives accompanied 

by clearly defined and quantified output and result indicators (linked to the specific 

objectives). We also assessed whether the MAs had assessed each applicant’s ability to 

achieve expected results. 

24.  We found that 18 out of 20 of the selection procedures we examined included criteria 

requiring beneficiaries to describe their project’s expected results (see Annex VIII). However, 

only four out of 20 procedures included criteria requiring the quantification of result 

indicators at project level (selection procedures 17, 18, 19 and 20). And in three cases, these 

result indicators did not correspond directly to those defined at OP level (see example in 

Box 1). 

 
 
19 

Box 1 - Example of result indicators at project level different from result indicators at OP level: 

project No 32, selected through selection procedure No 17 

Result indicators at project level 

•  amount of additional turnover generated 

(see Annex I, project No 32) 

•  amount of additional direct exports 

•  number of new jobs created 

Result indicators at OP level 

•  evolution of the business dynamic index 

(see also Annex VI, PA 1, SO 3d) 

•  number of growing companies 

25.  Our review of the 34 applications of selected projects corroborates these findings. All 

the applications provided quantified information for output indicators that corresponded to 

those at OP level, or such data could be obtained directly from the IT system (for instance for 

indicators such as number of enterprises receiving support). However, only 14 applications 

provided quantitative information on expected result indicators. In six of these, the result 

indicators did not correspond directly to the OP indicators. The other applications mainly 

provided a narrative description (see Figure 2 and see Annex VIII). 

Figure 2 - Information provided in the applications upon expected outputs and results 

Projects examined (results) 34

14

0

10

12

20

8

30

40

YES, applications providing quantified result indicators similar or not to the OP result indicators

NO, but applications outlining expected results and their contribution to the OP objectives

NO, applications not providing quantified results

26.  Regarding the assessment of applications, we found that the MAs had assessed the 

overall feasibility and likelihood of the projects to succeed. However, while there was an 

assessment of the achievability of the expected results for all 18 ERDF projects, this 

assessment was only performed for 2 of the 16 ESF projects we examined. We also found 

that the assessment of achievability remained vague in the assessment reports, except in 

 
 
 
 
20 

seven cases17 which included a more detailed description about the relevance of the 

indicators and the likelihood of the targets being met. 

27.  In terms of scoring the applications, we found that the provision of expected results was 

part of the scoring process in only 11 cases. This indicates that, as long as the envisaged 

projects were aligned with the type of actions set out in the selection procedure, the MAs 

assumed that it would contribute to the OP’s objectives. However, they did not focus on the 

magnitude of this contribution. 

Beneficiaries were encouraged to apply for funding 

28.  To ensure timely absorption throughout the programming period and at the same time 

ensure the focus on results, there is a need for a sufficient number of bidders able to provide 

good applications. The promotion of a call and related guidance to potential applicants by 

MAs is an important variable determining the overall effectiveness of selection procedures. 

A broader applicant population is likely to ensure that better projects are selected. 

29.  The measures aiming at promoting calls to potential bidders were prepared by the MAs 

on the basis of the detailed requirements set in the EU regulation. They were submitted to 

the Monitoring Committees for approval as part of the OP’s communication strategy that is 

required by the Regulation18. 

30.  We found that MAs had encouraged beneficiaries to participate in the selection 

procedures. First, MAs used several sources to promote a call to potential bidders. These 

included the involvement of Chambers of Commerce, local business associations, and online 

tools. MAs also relied on partners to circulate information on upcoming calls within their 

networks, based on the knowledge they acquired in the monitoring committees. 

31.  Furthermore, the guidance on funding provided by MAs to applicants was publicly 

available and, except for one MA at the time of the audit, it set out clear conditions for 

17 

18 

Projects Nos 27, 28, 30, 31, 32, 33 and 34. 

Article 116 and Annex XII(3) of Regulation (EU) No 1303/2013. 

 
 
                                                      
21 

receiving support, provided detailed information on the procedure for application and 

financing arrangements, referring explicitly to the selection criteria to be applied. 

Selection procedures did not usually involve any comparison between project applications, 

resulting in a risk that the best projects will not be selected 

32.  In our review of procedures, we checked that projects submitted for funding were 

assessed against each other. This is another important measure to ensure that the most 

results-oriented projects are selected. 

33.  In the framework of shared management, selection procedures were defined by the 

MAs and approved by the monitoring committees, based on the requirements in the EU 

Regulations19 and the Commission’s guidance20. Neither the ESI Funds Regulations nor any 

Commission guidance specifies the type of selection procedure to be used; this is left to the 

MAs’ discretion. 

34.  Out of the 20 procedures we examined, 10 were temporary calls for proposals, 6 were 

permanent calls, 3 were direct awarding procedures, and one a reiterated call (see also 

details in Annex III). We found that in all selection procedures except one (IT, OP Piemonte, 

No 13) no comparison between project applications had taken place. As a result, funding 

may not have been allocated to the best projects. This was mitigated to some extent in some 

of the OPs visited, where an informal (French and Finnish OPs) or a formal (Italy Puglia) 

pre-selection had taken place, allowing the MAs to limit the submission of applications to 

the most suitable projects. We found the following: 

• 

Fifteen out of twenty procedures were based on the first-come first served principle, 

where an application was selected provided that it met the criteria and that sufficient 

funding was available. Of these, 13 had been subject to a formal or informal 

pre-selection. We noted that some had been required to reach a minimum score for 

19 

20 

Article 132 of Financial Regulation (EU, Euratom) No 966/2012, Articles 34 and 125 of 
Regulation (EU) No 1303/2013. 

COM guidelines: http://ec.europa.eu/regional_policy/en/information/legislation/guidance/. 

 
 
                                                      
22 

selection (see selection procedures Nos 4 and 5 for the Czech OPEIC and 16, 17, 18, 19 

and 20 for the Finnish OP). 

• 

In the three direct awards, there was, by definition, no competition (see selection 

procedures No 1, 3 and 12). 

• 

In one case (IT, OP Piemonte, selection procedure No 8)21, funding had been awarded to 

projects already selected during the previous 2007-2013 programming period simply by 

extending their duration. This allowed the IB to avoid interruptions to the provision of 

training to young people, and allowed it to start absorbing the funds. But it also 

prevented potential new applicants from participating. 

The quality of monitoring data is at risk, and it is mostly outputs-oriented 

35.  We checked that the monitoring systems allowed the systematic collection of data on 

specified indicators, thus enabling the measurement of the progress made at project, 

programme and EU levels. We also examined whether this had been reported on in a way 

that allowed the success of the financed projects, and thus the effective and efficient use of 

the EU budget, to be demonstrated. 

36.  The following section contains our assessment of the necessary conditions to enable 

monitoring and reporting on the achievement of objectives. We first checked whether the 

MAs had set up functional monitoring systems before starting to implement the funds. We 

also looked at whether the performance information reported by the beneficiaries had been 

reliable, complete and timely. We also examined the role of the MAs and the Commission. 

37.  Finally, we consider whether the systems in place produced results-oriented 

performance information; specifically, whether they produced meaningful aggregated 

information, as this is a key element to allow reporting on the effective use of funds at 

Member State and EU level via annual monitoring and reporting. 

21  OP Piemonte, Italy, Call “Obbligo d’istruzione” 2015/2016. 

 
 
                                                      
23 

Assurance on the quality of monitoring data may not be provided in time for the 2019 

performance review 

38.  The monitoring of an OP’s progress towards its objectives is done on the basis of the 

data collected from various sources. This data is aggregated at OP and Member State level, 

and forms the basis for the reports sent to the OP’s monitoring committee and to the 

Commission. In order to present a correct view of the situation at OP or Member State level, 

it is essential that the data collected is reliable, complete and timely. To achieve this, the 

MAs have to set up a monitoring system including the collection of data and checks on the 

source of the data. In order to gain assurance on the quality of the data, the Commission 

relies on the audit work carried out by the audit authorities at Member State level as well as 

its own audit work. 

39.  In the specific context of the performance review, it is also important that the data, and 

the systems used to produce them, be audited as early as possible, to allow Member State 

authorities to address any detected weaknesses regarding the data quality well before the 

end of June 2019. It is then that the 2018 Annual Implementation Reports (AIRs) – the basis 

for the performance review and the allocation of the performance reserve – will be sent to 

the Commission. 

Extensive EU rules lay down how Member States should set up their monitoring systems for 

the 2014-2020 period 

40.  With a view to achieving the systematic collection and monitoring of data produced at 

OP level, the regulatory framework for the 2014-2020 programme period introduced several 

changes concerning data collection and monitoring arrangements: 

•  MAs must put in place a management and control system (MCS) covering adequate 

systems for reporting and monitoring22. 

22 

Articles 72, 73 and 74 of Regulation (EU) No 1303/2013. 

 
 
                                                      
24 

•  MAs are responsible for establishing a computerised system to record all the data 

related to the OPs indicators23 for monitoring and reporting purposes. 

•  Under EAC 7, MAs must put in place “a system of result indicators necessary to select 

actions which most effectively contribute to desired results, to monitor progress 

towards [those] results”24. OPs were required to demonstrate fulfilment of all the 

ex-ante conditionalities (EAC) by 31 December 201625 (see paragraph 4). 

• 

By 31 December 201526, Member States were required to ensure that all exchanges of 

information between beneficiaries and a MA, Certifying Authorities (CA), AAs and IBs 

could be carried out electronically27. 

41.  The Commission has also established specific guidance28, for its staff and the Member 

States’ responsible authorities, on a common methodology for assessing management and 

control systems in the MS, including the existence of adequate systems for monitoring and 

reporting. 

Performance information is not always reliable, complete and timely, and this has an impact 

on the implementation reports issued by the Member States 

Data reported by the beneficiaries 

42.  For the ERDF, in many cases, the data for result indicators is collected from national 

statistics bodies (see examples in Annex VI). The remainder comes from beneficiaries. For 

ESF, most of the data is reported via data collected from beneficiaries (see Annex VII). 

23 

24 

25 

26 

27 

28 

Article 125(2)(d) of Regulation (EU) No 1303/2013, this requirement is also part of the 
Management and control system, Article 72(d) of Regulation (EU) No 1303/2013. 

Ex ante conditionality 7, Annex XI Part II of Regulation (EU) No 1303/2013. 

Article 19 of Regulation (EU) No 1303/2013. 

Article 122 of Regulation (EU) No 1303/2013. 

Article 122(3) of Regulation (EU) No 1303/2013, their actual use by the beneficiaries is not 
obligatory. 

EGESIF_14-0010-final 18/12/2014, Guidance for the Commission and Member States on a 
common methodology for the assessment of management and control systems in the Member 
States. 

 
 
                                                      
25 

Managing authorities check that the data collected from beneficiaries is correct at project 

completion, when receiving their final report. During project implementation, checks can 

also be done on-the-spot, on a sample basis. 

43.  A number of plausibility and consistency checks are carried out automatically by the 

systems in place on the data provided by the beneficiaries, especially for the ESF. MAs also 

have tools to monitor the completion of the data provided on participants which allow them 

to remind beneficiaries who have submitted incomplete data to provide further information. 

44.  Our audit revealed issues regarding the quality of the data collected. We identified a 

general issue regarding the completeness and correctness of the data concerning 

participants to ESF co-financed actions. This data is classified as sensitive in national 

legislation29, so it is up to participants to decide whether to declare it. It is also difficult for 

the Member States and the Commission to check its accuracy. An additional weakness 

regarding the collection of data is presented in Box 2. 

Box 2 - Example 

Issues regarding timeliness of the data 

For the French national ESF OP, larger beneficiaries submitted their data for the 2014-2015 period 

late. This was corrected in 2016, and resulted in significant differences between the values reported 

in the 2015 AIR and those reported in the 2016 AIR  for the year 2015. 

45.  In its audit of performance data reliability (2014-2020), the Commission reported on 

other issues regarding the reliability of data, for instance: 

• 

data not collected which resulted in achievements not being recorded and thus not 

being taken into account in order to determine the progress of an OP, leading to an 

incorrect understanding of the situation; 

29 

For instance information on the participant’s background: migrants, foreign background, 
minorities including marginalised minorities such as the Roma, participants with disabilities, 
other disadvantaged. 

 
 
                                                      
26 

• 

a misunderstanding on the MAs’ part regarding the definition of common indicators (for 

instance, Common Output indicator 26, 'Enterprises cooperating with research 

organisations') resulting in an overstatement of the number of companies reported 

under this indicator; 

• 

double counting/failure to cap the value for indicator 'population served' at the number 

of area residents, resulting in inconsistent values. 

Availability and reliability of performance information 

46.  The existence of functional monitoring systems at the time MAs start implementing 

programmes is a key element ensuring that performance data is generated in time, which 

ensures that monitoring can take place. 

47.  The late approval, in December 2013, of the legislative framework for the period 

2014-2020 at EU level, delayed the development of the IT systems used for monitoring. For 

the ESF indicators, added complexity concerning the sensitivity of the data and the definition 

of the indicators, required additional discussions well into 2014. This further delayed the 

finalisation of the IT systems. In two cases, other factors led to the delays: reorganisation of 

the regions (France, OP Lorraine et Vosges), and the design of a new single IT system for all 

2014-2020 ESI funds in the Czech Republic. 

48.  The effects of the delay are reflected in the level of implementation of EAC 7 on the 

existence of a system of results indicators. At the end of 2016, the deadline to complete 

action plans, 50 plans remained incomplete, i.e. around 11 % of all OPs still did not fulfil this 

requirement. However, by the end of February 2018, all the action plans related to EAC 7 

had been completed. 

49.  In the first half of 2017, the fourth year after the start of the programme period, we 

found that some IT systems were still not fully functional; this poses risks to the 

completeness and accuracy of the data collected (see examples in Box 3). In its audit of 

 
 
performance data reliability (2014-2020)30, the Commission drew attention to the absence 

of electronic data exchange systems with beneficiaries, resulting in the manual encoding of 

the data on indicators by the MA into the system, a potential source of clerical errors. 

27 

Box 3 - Examples of IT systems still not fully functional and related risks 

Czech Republic - OPE 

• 

Problems with transferring data between the Labour Office and the OP. Data only started to be 

reported electronically to the OP from 31.3.2017. Until then, data was shared via Microsoft Excel 

files. This jeopardises the transfer of data between systems, especially given the large amount of 

data involved. 

• 

The AIR 2016 was the first AIR containing relevant data. 

The MA discovered inconsistencies in the data reported by the system for the AIR. This required to 

recalculate and fill them in manually in the AIR. 

Italy - OP Puglia 

The tools used for exchanging data with beneficiaries electronically were not working properly at the 

time of the audit31, and the system for collecting and storing the data was empty. Data was merely 

stored on staff computers. As a result, the aggregation of data with a view to producing the 2016 AIR 

(to be submitted to the Commission by 30 June 2017) was done manually, in separate files. 

Implementation reports issued by the Member States 

50.  The MAs must submit to the Commission an Annual Implementation Report (AIR) for 

each OP, which is approved by the Monitoring Committee. The AIR delivers key information 

on the implementation of the OP, mostly by providing quantitative and qualitative 

30  On the basis of 20 Commission’s audits on the reliability of performance data carried out mostly 

in 2017. 

31 

The MA managed to settle this later during 2017. 

 
 
                                                      
28 

information on common and programme-specific indicators32. The 2016 AIRs, submitted in 

June 2017, were the first reports sent to the Commission on the “progress made towards 

achieving the objectives of the programme”33. The Commission carries out a quality review 

of the AIRs (consistency of number of participants with population of the areas covered, for 

instance). 

51.  At the end of 2017, 26 AIRs had still not been accepted by the Commission due to 

missing data for the indicators used in the performance framework, inconsistencies in the 

data reported with the information provided on the implementation of the OP, or a lack of 

explanation for some of the data reported. Around 40 % of ERDF OPs had not reported any 

values for their performance framework indicators in 2017, on which there was no 

Commission analysis at the time of the audit. 

52.  The shortcomings identified in the AIRs raise questions about the reliability of the 

information provided as well as the level of progress reached until now as presented in the 

documents issued by the Commission based on the AIRs34: the annual synthesis report to 

the Parliament and the Council, European Economic and Social Committee and the 

Committee of the Regions and, in 2017 and 2019 only, the strategic progress report, which 

summarises progress made. 

32 

33 

34 

Article 50(2) of Regulation (EU) No 1303/2013. 

Article 50(4) of Regulation (EU) No 1303/2013. 

As well as on the Member States progress reports for 2017. Article 52 of Regulation (EU) 
No 1303/2013, to be submitted by 31 August 2017 and 31 August 2019 on the implementation 
of the Partnership Agreements as at 31 December 2016 and 31 December 2018. Progress 
reports present, namely, the “progress made towards the achievement of the Union strategy for 
smart, sustainable and inclusive growth, as well as of the Fund-specific missions referred to in 
Article 4(1), through the contribution of the ESI Funds to the thematic objectives selected, and 
in particular with regard to the milestones set out in the performance framework for each 
programme, and to the support used for climate change objectives”. 

 
 
                                                      
29 

Audits on management and control systems and the reliability of collected data started late, 

limiting the degree of assurance on the monitoring systems and reducing the time available 

to address detected weaknesses 

53.  Appropriate arrangements for the verification and correction of the performance 

information are necessary to ensure its quality and hence, its use in the context of 

monitoring and reporting. Good practices requires audits on the systems producing the data 

- and associated corrective actions - to be carried out and finalised before the data is 

included in AIRs and strategic reports. 

54.  Under the legislative framework, both the Member States’ audit authorities and the 

Commission35 audit the functioning of the monitoring systems. These audits should, in 

particular, ensure the existence of reliable systems for collecting, recording and storing 

performance data, and assess whether this performance data is reliable. 

Audit authorities’ assessment of the functioning of the management and control systems 

55.  An assessment of the systems in place at MA level was carried out during the early 

stages of the implementation of the OPs as part of the designation procedure36. In this 

procedure, an independent audit body (often the AA) was required to produce a report and 

an opinion on whether the MAs had fulfilled the criteria relating to the internal control 

environment, risk management, management and control activities and monitoring37. The 

35 

36 

37 

See Articles 75 and 127(4) of Regulation (EU) No 1303/2013 and Annex VII of Commission 
Implementing Regulation (EU) 2015/207 of 20 January 2015 laying down detailed rules 
implementing Regulation (EU) No 1303/2013 of the European Parliament and of the Council as 
regards the models for the progress report, submission of the information on a major project, 
the joint action plan, the implementation reports for the Investment for growth and jobs goal, 
the management declaration, the audit strategy, the audit opinion and the annual control 
report and the methodology for carrying out the cost-benefit analysis and pursuant to 
Regulation (EU) No 1299/2013 of the European Parliament and of the Council as regards the 
model for the implementation reports for the European territorial cooperation goal (OJ L 38, 
13.2.2015, p. 1) and EGESIF_14-0010-final 18.12.2014, Guidance for the Commission and 
Member States on a common methodology for the assessment of management and control 
systems in the Member States - key requirement 6 and 15. 

Articles 123 and 124 of Regulation (EU) No 1303/2013. 

Article 124(2) and Annex XIII of Regulation (EU) No 1303/2013. 

 
 
                                                      
30 

designation was required to take place before the first submission of an interim payment for 

the OPs. Thus, without this designation, OPs cannot be reimbursed. At this stage, the audit 

covers the existence of systems and procedures, but it does not cover good implementation, 

which is checked in the context of the audit authorities’ system audits on management and 

control systems. 

56.  Due to the late approval of the OPs and late finalisation of the IT systems (see 

paragraph 47), the designation process was late. This was also affected by national factors 

(reorganisation of the regions in France in 2015, complex management structures - for 

instance, the French PON FSE has 13 delegated managing authorities and 120 intermediary 

bodies). For the OPs examined in this audit, the MAs were designated between 

September 2016 and February 2018, more than two and four years into the programme 

period respectively (see Annex IV). 

57.  Due to the late implementation of OPs and the resulting limited amount of data 

collected, systems audits were only beginning at the time of our audit visits i.e. during the 

first half of 2017. Any problems regarding the quality of data thus did not become apparent 

until the second half of 2017. This was almost four years after the start of the seven-year 

programme period and one-and-a-half years before the 2018 AIRs, which form the basis for 

the allocation of the performance reserve, are forwarded to the Commission. 

The Commission’s audits 

58.  In addition to relying on the audit work and systems in place in the Member States, the 

Commission also carries out its own audits. There are two types: “early preventive systems 

audits”, aimed at obtaining reasonable assurance at an early stage of the implementation 

that the management and control systems are functioning well, and “performance reliability 

data audits”. But given the aforementioned delays, and thus the late finalisation of the 

monitoring systems as well as the minimal reporting of relevant data, these audits were 

conducted later than planned. Especially for the ERDF, audits could only start after the 2016 

AIRs were received. In total, by the end of 2017, the Commission managed to carry out 23 

audits in 12 Member States: 12 performance reliability data audits and 11 early preventive 

systems audits. 

 
 
31 

59.  For most of these audits, reporting was ongoing at the beginning of 2018. According to 

the Commission, the preliminary results for the audited systems were rather positive, even 

though some shortcomings were highlighted. At the same time, however, around 40 % of 

ERDF OPs did not report any values for their performance framework indicators38 in the 

2016 AIRs submitted in 2017. For these OPs, in February 2018, the Commission had not 

carried out audits to determine whether the absence of reporting was due to delays in 

implementation or a lack of a functioning data-reporting system. 

60.  The late undertaking of the systems audits means that the overview of the monitoring 

systems’s status is not complete. Shortcomings are particularly susceptible to being 

identified late and there may not be sufficient time to apply the necessary adjustments. This 

jeopardises the implementation of the performance review in 2019, which will be based on 

data reported by Member States in the 2018 AIRs by 30 June 2019. 

The performance review may not be based on realistic milestones 

61.  In the OPs we visited, on the basis of the 2015 AIRs or in the monitoring information 

available at the time of the audit, we noted that there were already cases where OPs had 

achieved the targets set for 2023, as well as risks of underachievement. Requests for 

amendments in order to adjust the targets to a more realistic value, had not been submitted 

for the OPs visited. Only one (Czech Republic, OPEIC) had already asked for, and obtained, 

amendments, mainly in relation to the modification of allocations within priority axes. On 

the other hand, according to the Commission, in February 2018, out of the 17 480 indicators 

used in the existing 430 OPs, the corresponding target value for 2023 had been modified 

upwards in 1445 instances and downwards in 941 instances. 

38 

Performance indicators are a set of indicators defined for each priority axis in an OP, which 
values at the end of 2018 will be compared by the Commission with that of set milestones in 
order to decide upon the release of the performance reserve in 2019. 

 
 
                                                      
32 

62.  This indicates that a number of milestones39 of the performance framework are no 

longer realistic. In case milestones need to be revised, this requires that amendments be 

made to the OP, which can only be done at the Member States initiative40. Given the 

duration of the amendment procedure (set in the Regulation at three months), there is a risk 

that the values reached at the end of 2018 by the performance-framework indicators are not 

compared with realistic milestones. 

The information generated on the basis of the data collected is still mostly outputs-

oriented 

63.  Performance information is collected so that the Member States and the Commission 

can use it for reporting purposes, in order to be accountable for what has been achieved 

with funding. 

64.  Result indicators and targets laid down in the grant agreements signed between 

beneficiaries and MAs are essential for measuring the achievement of results. However, 

several key issues in the set-up of the monitoring systems are affecting the measurement 

and monitoring of the contribution of projects to the achievement of the OP objectives. 

A set-up which relies mostly on output indicators, especially in the early years of 

implementation for the ERDF, and a lack of quantified result indicators defined at project 

level 

65.  The current set-up allows for a more systematic collection of data for a number of well-

defined indicators, though the number of indicators is very high. The existence of common 

indicators also allows for the aggregation of this data at OP, Member State and EU levels. 

However, in the set-up of the logical framework for the ERDF, many result indicators at OP 

level are not directly related to the funded interventions. These indicators are often national 

indicators which take into account the influence of external factors, and their sources are 

39  Milestones are the values set for the indicators of the performance framework to be reached by 

the end of 2018. 

40 

See Article 30(1) CPR, CPR Annex II and Article 5(6) of Regulation (EC) No 215/2014. 

 
 
                                                      
33 

mainly national statistics offices (see examples in Box 4 and see also Annex VI). These result 

indicators do not isolate the impact attributable to the ERDF interventions, and cannot 

therefore be considered immediate results of the OPs. 

Box 4 - Examples of result indicators for which the data is collected from published statistics 

rather than funded projects 

France, OP Grand Est: 

•  Priority axis 2; TO 3, investment priority A: 

development of SMEs: 

Source: Banque de France. 

Specific result indicator: evolution of the region’s 
industrial SME investment 

Czech Republic, OPEIC: 

•  Priority axis 2; TO 3, investment priority 3b: 

increase in exports of SMEs: 

Source: Ministry of Industry and Trade. 

Specific result indicator: percentage of exports in the 
total turnover of companies 

Finland, OP Sustainable growth and work: 

•  Priority axis 1, TO 3, investment priority 3d: 

Source: National statistics office of Finland. 

Specific result indicator: Business dynamics index 

66.  To mitigate the fact that results generated by ERDF projects often materialise (well) 

after project completion, at a time when there is no monitoring of the projects as such, MAs 

must carry out evaluations at priority axis level41. At least once during the programming 

period, these evaluations assess how support from the ESI Funds has contributed to the 

objectives for each priority. But given the late start of the implementation, the number of 

evaluations already carried out is still limited. As a consequence, for the time being, the 

progress of the OP is mainly measured through the output indicators. 

41 

Article 56 of Regulation (EU) No 1303/2013. 

 
 
                                                      
34 

67.  At project level, we found that result indicators were not systematically mentioned in 

the grant agreements (see paragraph 25). Only six42 out of the 18 ERDF projects examined in 

this audit contained results indicators in the applications and thus in the grant agreement. 

However, these result indicators were not the same as those used at OP level. This means 

that they cannot be used for reporting purposes or to measure the progress made in 

achieving OP objectives. In four other cases43, the results indicators were not quantified (i.e. 

no target was given) thus preventing an assessment of their contribution to the OP objective 

from being made and limiting the measurement of the project’s achievement to its outputs. 

68.  For the ESF projects, result indicators were mentioned in 7 out of 16 projects. For the 

remaining projects, the results described were either not related to the OP indicators, or 

they were not quantified. In any given project, this makes it impossible to compare the value 

reached with an estimated target. However, in the case of the ESF, even if results and result 

indicators are not defined at project level or quantified, the measurement of the values for 

the common result indicators is still taking place, as project beneficiaries have to report 

those data in the MA’s IT systems. 

69.  We have previously mentioned the lack of result indicators defined at project level44 in 

our annual reports. This makes it difficult to measure and to monitor the extent to which 

projects contribute to the achievement of the OP objectives. 

The release of the performance reserve is based on indicators that are mostly input- and 

outputs-oriented 

70.  Member States can demonstrate progress in programme implementation through 

milestones (to be reached by 2018) and target values (to be reached by 2023) for three types 

42 

43 

Projects 27, 28, 31, 32, 33 and 34. 

Projects 6, 7, 8 and 9. 

44   E.g. AR 2015 - Chapter 6 paragraph 6.86 “Result indicators were not defined for 38 % of the 

projects”. Or AR 2016 - Chapter 6 paragraph 6.56 “For 42 % of the projects it was not possible to 
identify and measure a specific contribution to the overall programme objectives, since no 
result indicators or targets were defined at project level”. 

 
 
                                                      
35 

of indicators: financial, output and result. They can be complemented by key 

implementation steps (KIS) to measure projects that are on-going or scheduled to be started, 

but for which it is unlikely that any outputs can be achieved by December 2018. Overall, the 

indicators used in the performance framework must be representative of the actions under 

the priority axis. In order to trigger the release of the performance reserve, the values 

reached at the end of 2018 for these indicators must be between 75 % and 85 % of the 

milestone value. 

71.  As shown in a previous ECA report45, the vast majority of the indicators used in the 

performance framework under the cohesion policy related to output (57 %), key 

implementation steps (9 %) and financial indicators (33 %), with the use of result indicators 

remaining marginal. Thus, the release of the reserve will to a large extent remain input- and 

outputs-oriented, and not focused on results (see also paragraph 62). 

Most reporting at EU level concerns output indicators 

72.  The main objective of the Commission’s strategic report 201746, which encompasses the 

2017 annual synthesis report drawn up based on the 2016 AIRs, is to report on the 

achievements of the ESI Funds towards attaining the EU2020 objectives. However, the 

report mainly presents the level of implementation as well as the values reached for the 

main output indicators by the end of 2016. It presents no information on the achievement of 

results, except for the following ESF result indicators: the number of participants that found 

a job immediately after the training, and the number of participants that gained a 

qualification immediately after training. For the ERDF, this is also due to the fact that there 

are no common result indicators; those used at Member State level cannot, therefore, be 

aggregated. 

45 

46 

SR No 15/2017. 

COM(2017) 755 final of 13 December 2017 “Strategic report 2017 on the implementation of the 
European Structural and Investment Funds”. 

 
 
                                                      
36 

73.  As we have reported recently, there is a fundamental problem affecting result 

indicators in particular. In fact, ERDF and ESF result indicators measure different things in 

different ways47. Furthermore, the concept of a “result” is interpreted differently in the 

fund-specific regulations. The ERDF result indicators cannot be used to measure the 

immediate results of the OPs. In addition, the high number of different performance 

indicators pose a real challenge to the collection and reporting of performance 

information48. 

CONCLUSIONS AND RECOMMENDATIONS 

74.  Although the set-up of OPs is now more results-oriented, with a stronger intervention 

logic and an extensive set of indicators, overall we conclude that the selection of projects 

under the ERDF and ESF is not fully results-oriented and that the monitoring in place is still 

more outputs-oriented. 

Selection procedures were not results-oriented in important aspects 

75.  For the OPs we visited, selection procedures were used to support the selection of 

projects relevant to the OPs’ objectives. However, they do not systematically include 

selection criteria requiring the definition of quantified result indicators at project level 

corresponding to those at OP level. Consequently, result indicators were rarely included in 

applications; even when this was the case, they did not necessarily correspond to the OP 

indicators, or had not been quantified. Furthermore, only one-third of the assessment 

reports included a specific assessment of the expected results in relation to OP objectives 

(see paragraphs 18 to 27). 

76.  We found that MAs had provided comprehensive information on the OPs, on access to 

EU funding, and on the selection procedures through several channels. Beneficiaries were 

47 

48 

See SR No 2/2017, paragraph 150. 

See SR No 2/2017, paragraph 131. 

 
 
                                                      
37 

generally well informed, or could rely on support from specialised organisations (see 

paragraphs 28 to 31). 

77.  However, only 1 of the 20 procedures examined involved a scoring and a ranking of 

applications against each other. Projects were generally approved on a first-come 

first-served nature. As a result, funding may not have been allocated to the best projects 

(see paragraphs 32 to 34). 

There are risks to the quality of monitoring data, which remains more outputs-oriented 

rather than results-oriented 

78.  Following the late adoption of the legislative framework, we found that some of the 

OPs’ monitoring systems were not operational at the start of the programme nor fully 

operational at the time of the audit. Consequently, data was input manually and at a later 

stage. Data collection also occurred outside the IT systems, creating errors, which, in some 

cases, impacted upon the data presented in the AIRs (see paragraphs 35 to 52). 

79.  The late adoption of the regulatory framework led, in most OPs, to a late start of the 

implementation. Until the end of 2016, only a limited amount of data was collected in IT 

systems. Consequently, the audits necessary to provide assurance on the monitoring 

systems and the reliability and availability of the data mostly started in 2017. The results of 

these audits only became available at the end of 2017, four years into the seven-year 

programme period. This means that assurance obtained upon the monitoring data is, up to 

now, only partial. The Commission thus does not have an overview of the functioning of the 

monitoring systems, including IT systems. The performance review will take place in 2019 

and, in the remaining time available, there is a risk that any corrective measures cannot be 

completed in time and that the performance review will not be based on correct information 

(see paragraphs 53 to 60). 

80.  We found that a number of milestones of the performance framework are no longer 

realistic. In case milestones need to be revised, there is no certainty that the necessary OP 

amendments can be made in time for the performance review (see paragraphs 61 and 62). 

 
 
38 

81.  Regarding result indicators, ERDF and ESF indicators measure different things in 

different ways. In fact, the concept of a “result” is interpreted differently in the fund-specific 

regulations. For both funds, we identified the risk that a meaningful aggregation of 

performance data (in particular regarding results) may not be feasible at EU level49. 

Furthermore, we recommended50 that the Commission should define a common definition 

of “outputs” and “results” (see paragraph 73). 

82.  For the ERDF, many result indicators often use other sources, such as data from national 

statistics offices. Evaluations should measure achievements against the OPs’ objectives. 

However, given the late start of implementation, few evaluations were available at the time 

of our audit. Consequently, for the ERDF, it is difficult to monitor to what extent the projects 

contribute to the achievement of the OP objectives (see paragraphs 63 to 69). This problem 

also applies to the ESF, as many grant agreements do not include quantified result 

indicators. 

83.  Similarly, output and financial indicators predominantly form the basis of the 

performance framework. This means that the release of the performance reserve will be 

mainly based on these types of indicators rather than on the results achieved and the actual 

progress made towards the OP objectives. Also, the information published in the 

Commission’s reporting is mostly outputs-related, except for the ESF where the aggregated 

value for some result indicators is provided (see paragraphs 70 to 72). 

49 

50 

See SR No 2/17, paragraph 150. 

See SR No 2/17, recommendation 3. 

 
 
 
                                                      
39 

Recommendation 1 – Results-oriented selection 

In order to ensure a consistent and truly results-oriented approach to the selection of 

projects, Member States should: 

(a)  have selection criteria requiring the beneficiaries to define at least one genuine result 

indicator based on a common definition of what constitutes a “result” for their project, 

including a baseline and a target. This result indicator should contribute to the result 

indicators set at OP level; 

(b)  include an assessment of these expected results and indicators in the assessment report 

for the applications; 

(c)  when deciding on the selection procedures to be used, ensure that a comparison 

between project applications takes place. 

Target implementation date: starting in 2019 for upcoming calls for proposals. 

Recommendation 2 - Results-oriented monitoring 

(a)  In order to allow the monitoring of projects’ contributions to OP objectives, Member 

States should include quantified result indicators in the grant agreement, contributing to 

the results indicators set at OP level. 

Target implementation date: starting in 2019 for upcoming calls for proposals. 

(b)  In order to make the ERDF monitoring framework more results-oriented, and in 

particular to allow the aggregation of performance information, the Commission should 

define common result indicators for this fund based on a common definition of 

“results”. 

Target implementation date: preparation of the next MFF. 

 
 
 
 
40 

Recommendation 3 - Reporting on performance and preparation of the performance 

review in 2019 

To improve the Commission’s reporting on performance and to carry out a meaningful 

performance review with reliable data and realistic milestones, the Commission should: 

(a)  have an overview of the main weaknesses in and uncertainties about the OPs’ 

monitoring systems based on audits by both the Commission and the AAs (the necessary 

systems audits/performance-reliability audits); 

(b)  ensure that OP amendments requested by Member States relating to justified revisions 

of milestones of the performance framework are processed in time for the performance 

review. 

Target implementation date: before the finalisation of the performance review. 

This Report was adopted by Chamber II, headed by Mrs Iliana IVANOVA, Member of the 

Court of Auditors, in Luxembourg at its meeting of 27 June 2018. 

For the Court of Auditors 

Klaus-Heiner LEHNE 

President 

 
 
 
 
 
 
 
 
 
 
List of projects selected for this audit 

41 

ANNEX I 

(amounts in million euros) 

No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

Czech Republic - OP Employment (OPE) 2014CZ05M9OP001 (amounts in million CZK) 

Yes 

Yes 

4 011 

3 304.6  ESF 

8 

1.1 

1.1.1 

1.4.2015 
ongoing 

Participants in employment: 
14 304 

4.9 

4.2  ESF 

8 

1.2 

1.2.1 

1.1.2016 - 
31.12.2017 

Places in kindergarten: 24 

Selection 
procedure 
No (see 
Annex III) 

1 

2 

1 

2 

3 

4 

5 

”New working opportunities” (Call 
Instruments of Active labour 
market policy) 

Child care centre for preschool 
children #1 (Call Support of 
creation and operation of day 
care services for children of pre-
school age for companies and 
public outside the City of Prague) 

Child care centre for preschool 
children #2(a) (Call Support of 
creation and operation of day 
care services for children of pre-
school age for companies and 
public outside the City of Prague) 

Child care centre for preschool 
children #2(b) (Call Support of 
creation and operation of day 
care services for children of pre-
school age for companies and 
public outside the City of Prague) 

“Support of selected social 
services (asylum houses, daily 
centres, etc) in one region” 

Yes 

4.1 

3.4  ESF 

8 

1.2 

1.2.1 

1.1.2016 - 
31.12.2017 

Places in kindergarten: 20 

2 

Yes 

4.9 

4.2  ESF 

8 

1.2 

1.2.1 

1.1.2016 - 
ongoing 

Places in kindergarten: 20 
Number of persons using a 
childcare facilities: 30 

Yes 

411.8 

350  ESF 

9 

2.1 

2.1.1 

1.1.2016 - 
ongoing 

Actual realisation: NA yet, 
project in start up phase 

2 

3 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

42 

Selection 
procedure 
No (see 
Annex III) 

Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) 2014CZ16RFOP001 (amounts in million CZK) 

6 

Support to exports for the 
companies 

Yes 

1.2 

0.6  ERDF 

3 

2.1 

3b 
SO 2.2 

3.7.2015 - 
10.5.2016 

7 

Participations in trade fairs 
abroad in 2016 

Yes 

3.3 

1.6  ERDF 

3 

2.1 

3 b 
SO 2.2 

9.9.2015 - 
21.12.2016 

Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) 2014CZ16RFOP001 (amounts in million CZK) 

8 

Building of a production hall for 
woodworking division 

Yes 

8.6 

3.8  ERDF 

3 

2.1 

3 c 
SO 2.2 

1.11.2015 - 
25.1.2016 

9 

Reconstruction of a Building 

Yes 

21.2 

7.4  ERDF 

3 

2.1 

3 c 
SO 2.2 

7.9.2015 - 
6.9.2016 

4 

4 

5 

5 

Number of participations in 
exhibitions and fairs abroad 3 
(but the same company): one 
company supported 
Grants previously received: 
12 projects OPPI 265.2 million 
CZK, 
4 Fair grants 53 million CZK 

Number of participations in 
exhibitions and fairs abroad 3 
(but the same company): one 
company supported 
Grants previously received:  
8 projects from OPPI 
117.3 million CZK 
1 OPLZZ 5.3 million CZK 

Revitalisation of the company’s 
site (844 mq) 
Number of companies using the 
revitalised infrastructure: 1 
Grants previously received: 
7.2 million CZK 

Revitalisation of the company’s 
site (2721 mq) 
Number of companies using the 
revitalised infrastructure: 1 
Grants previously received: 
OPPI 12 projects 123.7 million 
CZK, 
OPLZZ project 2.2 million CZK 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

France - OP ERDF/ESF Lorraine et Vosges - 2014FR16M0OP015 

10 

11 

12 

Take over a printing activity (due 
to retirement of the previous 
owner) 

Acquisition of equipment for 
production (e.g.: presse 140 tons 
with bi-injection system) 

Investments for the development 
of a ski resort 

13 

Acquisition of equipment for 
production (hydraulic tools) 

No 

No 

No 

No 

2 

0.04  ERDF 

3 

2 

2.3A 

0.6 

0.05  ERDF 

3 

2 

2.3A 

6.7 

1.1  ERDF 

3 

9 

9.3A 

1.6 

0.02  ERDF 

3 

2 

2.3A 

France - National OP ESF (PON FSE) - 2014FR05SFOP001 

14 

15 

Support to people benefitting 
from the minimum income in the 
department 

Accompany/ guiding people in an 
insertion process, including 
learning French language 

Yes 

Yes 

France - National OP ESF (PON FSE) - 2014FR05SFOP001 

10.7 

5.3  ESF 

3 

9 

1.1 

2 

1  ESF 

3 

9 

1.1 

18.9.2014 - 
12.10.2015 

1.1.2014 - 
31.12.2016 

1.1.2015 - 
31.12.2016 

2.7.2014 - 
31.2.2016 

Number of enterprises 
receiving support: 1 
Jobs: + 2 

Number of enterprises 
receiving support: 1 
Jobs: + 2 

Number of enterprises 
receiving support: 1 
Number of enterprises 
receiving subsidy: 1 

Number of enterprises 
receiving support: 1 
Number of enterprises 
receiving subsidy: 1 
Jobs: + 6 

1.1.2014 - 
31.12.2016 

1.1.2014 - 
31.12.2016 

Number of participants 
expected: 6360 / year or 19 080 
in total 

Number of participants to the 
action (2014-2016): 657 
Positive outcome within 3 
months after the project 
participation: 40 % 

16 

Professional training of 
employees from the construction 
sector (Paris and neighbouring 
region) 

17 

Professionalization of networks 
about the creation of activities 

Yes 

9.5 

4.8  ESF 

2 

8 

5.3 

1.1.2014 - 
31.12.2015 

Number of participants to the 
action (2014-2016): 2 464 

Yes 

5.2 

2.6  ESF 

1 

8 

3.2 

1.1.2014 - 
31.12.2016 

No participants 

43 

Selection 
procedure 
No (see 
Annex III) 

6 

6 

7 

6 

8 

9 

10 

11 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

18 

Training activities for unemployed 
people 

Yes 

80 

30.7  ESF 

2 

8 

5.4 

1.1.2014 - 
31.12.2016 

Number of participants to the 
training: 21 860 (targeted: 
22 734) 

44 

Selection 
procedure 
No (see 
Annex III) 

12 

Italy - OP ESF Piemonte - 2014IT05SFOP013 

19 

20 

21 

22 

Training courses to help insertion 
in the labour market (Technical 
e-commerce) 

Training courses to help insertion 
in the labour market (hairdresser) 

Vocational training to back up 
compulsory education 
(Carpenter) 

Vocational training to back up 
compulsory education (Agro-food 
processor) 

Yes 

Yes 

Yes 

Yes 

Italy - OP ERDF/ESF Puglia - 2014IT16M2OP002 

23 

24 

Acquisitions of 10 new buses for 
passenger transport 

Acquisition of equipment for 
production of gluten-free pasta 

Yes 

Yes 

Italy - OP ERDF/ESF Puglia - 2014IT16M2OP002 

25 

26 

Restructuring and adaptation 
(building) of an existing school 
into a crèche 

Restructuring and adaptation 
(building) of an existing school 
into a crèche 

Yes 

Yes 

0.69 

0.69  ESF 

8 

1 

8.x 

1.10.2015 - 
15.6.2016 

Technical e-commerce course 
(600 hours) 

13 

0.69 

0.69  ESF 

0.99 

0.99  ESF 

8 

8 

1 

2 

8.x 

8.x 

0.99 

0.99  ESF 

8 

2 

8.x 

4.12 

1.41  ERDF 

2.7 

0.8  ERDF 

3 

3 

3 

3 

3a 

3a 

0.5 

0.44  ERDF 

9 

9 

9a 

0.35 

0.31  ERDF 

9 

9 

9a 

1.10.2015 - 
15.06.2016 

1.10.2015 - 
15.06.2016 

1.10.2015 - 
15.6.2016 

5.5.2015 - 
31.12.2015 

16.6.2015 - 
31.12.2015 

1.1.2016 - 
31.12.2016 

1.1.2016 - 
31.12.2016 

Hairdresser course (600 hours) 

13 

Carpenter course (1 050 hours) 

14 

Agro-food course (1 050 hours) 

14 

none 

- Turnover (2017 o/s) 
- Increase 4 work units 

15 

15 

57 new places created in crèche  16 

42 new places created in crèche  16 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001 

45 

Selection 
procedure 
No (see 
Annex III) 

Yes 

0.85 

0.32  ERDF 

3 

1 

3d  2.1 

16.2.2015 - 
30.6.2016 

Yes 

2.45 

0.619  ERDF 

3 

1 

3d  2.1 

2.3.2015 - 
30.9.2016 

17 

17 

Turnover: + €0.3 million, + 10 % 
of current one 
Direct export: + €0.07 million, 
+ 100 % from the current level 
Jobs: + 2 (of which 0 are 
women’s jobs) 
Other: Significant energy 
savings per produced unit, at 
least 10 % of energy use 

Right after completion: 
Turnover: + €0.41 million 
Jobs: + 1 
2 years after completion (not 
known yet): 
Turnover: + €2.5 million, + 20 % 
of current turnover 
Jobs: + 7 
Project contributes to low 
carbon economy 

Yes 

0.24 

0.17  ESF 

9 

5 

8i 

2.1.2015 -
31.12.2016 

Beneficiaries: 39 persons 
(target: 80 persons) 

18 

27 

28 

29 

Investment in a powder painting 
line, acoustic wall element line 
and steel bending machine. 
Production reorganized according 
to LEAN 

Investment for undertaking 
concrete production line 
development, internal logistics re-
organisation as well as develop a 
new range of products 

To develop and confirm the 
service model used in one city’s 
Labour Service Center for young 
adults that have a background of 
substance abuse to encourage 
them to normalize their life so 
that they may envisage to start 
looking for a job 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001 

Yes 

0.31 

0.22  ESF 

8 

3 

9i 

1.1.2015 - 
31.12.2016 

Beneficiaries: 82 persons 
(target 100 persons) 

30 

31 

To offer training for people 
employed intermittently to 
increase their chances of finding 
jobs and their choice of jobs 
(social economy) 

Development of a user interface 
software for medical ultrasound 
scanner 

Yes 

0.34 

0.17  ERDF 

3 

1 

3a  1.1 

30.1.2015 - 
30.4.2016 

32 

Development of new wooden 
cladding products coating line 

Yes 

1.4 

0.56  ERDF 

3 

1 

3d  2.1 

27.1.2015 - 
27.1.2017 

46 

Selection 
procedure 
No (see 
Annex III) 

18 

17 

17 

At project completion / 
estimate 2 years after: 
Turnover:+ €0.25 million 
+ 100 % / + €10 million + 100 % 
Direct exports: + € 0.25 million 
+ 100 % / + €9.8 million + 100 % 
Jobs: + 3 / 15 (of which women 
0 / 5) (target 35) Jobs in R&D: 
+ 3 / 10 (of which women 0 / 3) 
Other intangible rights 0 / 3 
(target 4) 

At project completion / 
estimated 2 years after 
completion: 
Turnover: + €1 million +12 % / 
+ €2 million + 24 % (target 
€3 million) 
Direct exports: NA / 
+ €0.3 million + 33 % (target 
€1 million) 
Jobs: + 3 / + 6 of which female: 
+ 1 / + 2 (target 2) 

 
 
No 

Project Description 

Grants 
received 
previously 

Total 
amount of 
investment 

Amount 
of EU 
Grant 

Fund 

TO 
(a) 

PA 
(b) 

IP/SO 
(c) 

Start & 
end date 

Outputs and results achieved 

Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001 

33 

Extension of a factory for the 
development of production and 
logistics operations 

Yes 

1.17 

0.35  ERDF 

3 

1 

3d  2.1 

1.6.2014 - 
31.12.2014 

34 

Extension of the machinery park 
for modern technology for piping 
and steel structure production 
and re-organise the lay-out of the 
production site 

Yes 

1.43 

0.5  ERDF 

3 

1 

3d  2.1 

13.2.2015 
– 
31.12.2016 

Notes: (a) TO: thematic objective. (b) PA: priority axis. (c) IP: investment priority and SO: specific objective. 

At project completion / 
estimated 2 years after 
completion: 
Turnover: + €10 million + 150 % 
(target €11 million) / 
+ €35 million + 350 % 
Direct exports: + €9 million 
+ 643 % (target € 0.9 million)/ 
+ €26 million + 999 %) 
Jobs: + 10 / + 50 of which 
women + 3 / + 5 (target 50) 
R&D jobs: + 2 /+ 2 of which 
women + 0/ + 1 (target 4) 
Other intangible rights 1 / 3 
Project contributes to low 
carbon economy 

At project completion / 
estimated 2 years after 
completion: 
Turnover: 0 / + €1.4 million 
+ 34 % 
Direct exports: 0 / 
+ €0.46 million + 136 % (target 
€0.75 million) 
Jobs: + 5 / + 8  of which female 
0 / 0 
Other : energy savings NA / + 1 
MWh/a 

47 

Selection 
procedure 
No (see 
Annex III) 

17 

17 

 
 
 
48 

ANNEX II 

Thematic objectives 

Description 

Strengthening research, technological development and innovation 
Enhancing access to, and use of quality of, information and communication 
technologies 
Enhancing the competitiveness of SMEs 
Supporting the shift towards a low-carbon economy in all sectors 
Promoting climate change adaptation, risk prevention and management 
Preserving and protecting the environment and promoting resource 
efficiency 
Promoting sustainable transport and removing bottlenecks in key network 
infrastructures 
Promoting sustainable and quality employment and supporting labour 
mobility 
Promoting social inclusion, combating poverty and any discrimination 
Investing in education, training and vocational training for skills and lifelong 
learning 
Enhancing institutional capacity of public authorities and stakeholders and 
an efficient public administration 

Number 
1 

2 

3 
4 
5 

6 

7 

8 

9 

10 

11 

Source: Article 9 and Annex XI Part I of Regulation (EU) No 1303/2013. 

 
 
 
 
No 

Name / 
identification 

Czech Republic OPE 

1 

2 

3 

Instruments of 
Active labour 
market policy 
(No 03_15_001) 

Childcare services 
outside the City of 
Prague 
(No 03_15_035) 

“Social services” 
(No 03_15_005) 

List of selection procedures examined in this audit 

Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

49 

ANNEX III 

Projects 
audited (see 
Annex II) 

ESF 

8 

1.1.1 

ESF 

8 

1.2.1 

ESF 

9 

2.1.1 
2.1.2 

Selection of actions to increase employment of 
supported persons, especially older, low-
qualified and disadvantaged ones 
The beneficiary (CZ’s Labour office) set in the 
Partnership Agreement 

To decrease the differences in the situation of 
women and men 
Support of creation and operation of day care 
services for children of pre-school age for 
companies and public outside the City of 
Prague (“Childcare services”) 

2.1.1 - To increase employability of persons in 
danger of social exclusion or socially excluded 
persons in the labour market and 
2.1.2 - Development of social economy sector 
Support of selected social services linked to the 
regional mid-term plans of development of 
social services (“Social services”) 
The beneficiaries (all CZ’s regions) as well as the 
envelope per region, were set in the 
Partnership Agreement 

Direct award 
29.7.2015 - 
30.4.2016 

Temporary call for 
projects: 
11.11.2015 - 
8.1.2016 

Direct award 
launched: 
26.6.2015 
Applications 
accepted between: 
20.7.2015 - 
13.12.2019 

NA 

1 

YES 

2, 3, 4 

NA 

5 

 
 
 
 
 
Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

ERDF 

3 

PA2, SO 2.2 

Temporary call  
2.6.2015 - 
30.11.2015 

YES provided 
that the 
applications 
score a 
minimum of 
60 points 
out of 100 

50 

Projects 
audited (see 
Annex II) 

6, 7 

No 

Name / 
identification 

Czech Republic OPEIC 

4 

Marketing I 

5 

Real Estate I 

Objective SO 2.2: To increase 
internationalization of SMEs 
- Services for SMEs focused on international 
competitiveness enabling entry to foreign 
markets; 
- Sophisticated consultancy of experts in 
international markets and consultancy for 
strategic management and innovation 
management, incl. mentoring and coaching; 
- Services to support SME networking in 
international research cooperation (Horizon 
2020, COSME). 

Objective SO 2.3: To increase the use of 
entrepreneurial infrastructure 
- Modernization of production facilities and 
reconstruction of existing obsolete 
infrastructure, and  
- Reconstruction of brownfields (excluding 
costs for the removal of ecological burdens) 
and their transformation into modern business 
premises and newly reconstructed areas. 

ERDF 

3 

PA2, SO 2.3 

Temporary call in 
two phases: 
1.6. till 31.8.2015 
Eligible applications 
only: 
1.12.2015 till 
31.1.2016 

8, 9 

YES provided 
that the 
applications 
score a 
minimum of 
60 points 
out of 100 

France OP Lorraine et Vosges 

6 

Entreprenariat et 
entreprises 

Objective: To select projects with a view to 
support the investment in SMEs, at all stages of 
their development, to generate sustainable 
jobs and added-value 

ERDF 

3 

PA 2 - SO 2.3.A 

Permanent call 
1 year revolving 

10, 11, 13 

YES provided 
that the 
criteria are 
met 

 
 
 
 
 
Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

51 

Projects 
audited (see 
Annex II) 

Objective: to increase the number of tourists in 
the Vosges area 

ERDF 

3 

PA 9 - SO 9.3.A 

Permanent call, 
1 year revolving 

No 

7 

Name / 
identification 

Développement 
économique et 
touristique du 
massif 

France PON FSE 

8 

Call for project 

Call launched by the Intermediate Body 

ESF 

9 

PA3 - 3.9.1.1 

9 

Call for projects - 
AAP interne 2014 - 
2017 

Call launched by the Intermediate Body 

ESF 

9 

PA3 - 3.9.1.1 

10 

Call for project - 
2014IDF-AXE2-01 

Call launched by the Delegated Managing 
Authority 

ESF  

8 

PA2 - 2.8.5.3 

11 

Call for project 

Call launched by the Intermediate Body 

ESF 

8 

PA1 - 1.8.3.2 

12 

Call for projects - 
CSP 2014/15 

Call launched by the Managing Authority 

ESF 

8 

PA2 - 2.8.5.4 

Temporary call 
16.12.2014 - 
31.3.2015 

Temporary call  
9.6.2015 - 
30.7.2015 

Temporary call  
11.8.2014 - 
17.11.2014 

Permanent call 
revolving 
1.1.2014 - 
31.12.2016 

Direct award 
revolving 
1.1.2014 - 
31.12.2017 

12 

14 

15 

16 

17 

18 

YES provided 
that the 
criteria are 
met 

YES provided 
that the 
criteria are 
met 

YES provided 
that the 
criteria are 
met 

YES provided 
that the 
criteria are 
met 

YES provided 
that the 
criteria are 
met 

YES provided 
that the 
criteria are 
met 

 
 
 
 
 
 
 
 
Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

ESF 

8 

PA1 
SO: 1.8.i.1.3 

Temporary call 
10.7.2015 - 
29.7.2015 

YES scoring 
and ranking 

52 

Projects 
audited (see 
Annex II) 

19, 20 

No 

Name / 
identification 

Italy OP Piemonte 

13 

“Mercato del 
Lavoro” in the 
region Piedmont 
(except Turin 
province) 

14 

“Obbligo 
d’Istruzione” CMT 
2015/2016 in Turin 
province 

Italy OP Puglia 

15 

D.D. No 2487 of 
22/12/2014 

16 

DD 367 of 6.8.2015 

To select trainings to shorten the access to the 
labour market of mainly unemployed youths 
and adults (19-25 years sometimes older) with 
low levels of education and exposed to a range 
of factors that put them at a greater risk of 
long-term unemployment (Formazione 
professionale finalizzata alla lotta contro la 
disoccupazione) 
Call managed by the Managing Authority 

To select courses to allow young people that 
have difficulties to integrate and have been 
excluded from school mainstream education, or 
even dropped out of school /compulsory 
education/instruction to fulfil their right/duty 
to 10 years education and training 

Objective: Facilitate the creation of new 
production units; the extension of existing 
production units; the diversification of the 
production to new, additional products; 
fundamental change in the overall production 
process of an existing unit 

To select projects aiming at improving and 
upgrading the network of social welfare, 
educational and healthcare facilities across 
Puglia region. Its purpose is to fill in the gaps in 
the supply of social-care services to people, 
families and communities in the Region, by 
co-financing social investment plans or 
structural measures in the social, healthcare or 

ESF 

8 

PA1 
SO: 1.8ii.2.4 

NA This call was not a selection procedure as such 
but consisted in the prolongation of the contracts 
for beneficiaries selected previously via a similar 
call 2011/2012. For simplicity and time saving, the 
MA decided to reiterate the same training activity 
for the period 2015-2018. 

21, 22 

ERDF 

3 

PA3 - 3.6 

Permanent call 

ERDF 

9 

9.10 and 9.11 

Permanent call 

23, 24 

YES provided 
that the 
criteria are 
met. 

25, 26 

YES as long 
as 
applications 
score a 
minimum of 
70 points 
out of 100 

 
 
 
 
 
 
No 

Name / 
identification 

Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

53 

Projects 
audited (see 
Annex II) 

educational fields. Eligible beneficiaries are e.g. 
public institutions, mainly municipalities , and 
private bodies offering social care services, 
which were formerly public welfare and charity 
institutions or other public utilities (e.g. local 
health services, chamber of commerce) 

Finland - OP Structural Funds 

17 

Enterprise 
development and 
support schema 

Complementary ERDF support schema 
executed in four Centres for Economic 
Development, Transport and the Environment 

ERDF 

3 

3 / all, 
discretional 
focussing on 
regional 
targeted actions 

Permanent, valid 
for the whole MFF 
2014-20. Annual 
national budgetary 
appropriation 

18 

19 

Centre for Economic 
Development, 
Transport and the 
Environment South 
Savo Finland, 
Mikkeli 

Centre for Economic 
Development, 
Transport and the 
Environment, North 
Ostrobothnia, Oulu 

ERDF: focus on Intelligent specialization 
ESF: focus on Support for youth actions and 
Integration of actors and enhanced cooperation 

ESF 
ERDF 

3 
8 
9 

ERDF - PA2 
ESF - PA3 
ESF - PA4 
ESF - PA5 

Temporary call: 
8.12.2014-
16.2.2015 

all 

All PA/IP/SO 

ESF 
ERDF 

Temporary call: 
5.5.2014-29.8.2014 

Yes, as long 
as 
applications 
score a 
minimum 
threshold 

Yes, as long 
as 
applications 
score a 
minimum 
threshold 

Yes, as long 
as 
applications 
score a 
minimum 
threshold 

27, 28, 31, 
32, 33, 34 

29, 30 

N/A  
The projects 
selected 
were not 
advanced 
enough, 
thus not 
audited 

 
 
 
 
 
 
No 

Name / 
identification 

Finland - OP Structural Funds 

Remarks 

Fund 

TO 
(a) 

PA/IP/SO (b) 

Type of selection 
procedure (c) 

Competitive 

1st in 1st 
serve 

54 

Projects 
audited (see 
Annex II) 

20 

Centre for Economic 
Development, 
Transport and the 
Environment, North 
Ostrobothnia, Oulu 

Under ESF: special focus on the integration of 
the immigrants into society 
Under ERDF: focus on environmental and 
natural resources in particular bio-recycling 
economy as well as energy and material 
efficiency and  
on operations/projects linked to research-and 
development 

Notes: 

(a) TO: thematic objective.

(b) PA: priority axis, IP: investment priority, SO: specific objective.

ESF 
ERDF 

3 
8 
9 

ERDF - PA1 
ERDF - PA2 
ESF - PA3 
ESF - PA4 
ESF - PA5 

Temporary call: 
ESF: 12.12.2015 -
5.2.2016 ERDF: 
12.12.2015 -
12.2.2016 

Yes, as long 
as 
applications 
score a 
minimum 
threshold 

(c) Type of selection procedures:
Temporary Calls: Calls that have a duration inferior of 12 months, usually few weeks to a few months.
Permanent Calls: Calls open with a duration superior of 12 months, sometimes for the whole duration of the programme period.
Direct awards: Involve the earmarking of funding to an organisation, usually a public body, who then redistributes the money to external beneficiaries. 

Source: Documentation of the selection procedures. 

55 

ANNEX IV 

List of OPs examined in this audit 

ERDF 

ESF 

Total EU 
funds 

Other 
funds 

Total EU + 
other funds 

Amounts in million euros except for CZ 

TOs 
covered 

Partnership 
Agreement 
approval 
date 

EC approval 
date of the 
OP 

Latest 
version and 
approval 
date 

Designation of 
National 
Authorities by EC/ 
MS 

Managing authority (MA) 
If applicable: Intermediate Body (IB) or 
Delegated Managing Authority (DMA) 

Czech Republic - OP Employment (OPE) - 2014CZ05M9OP001 (amounts in million CZK) 

NA 

2 119 

2 119 

450 

2 569 

8,9,11 

13.4.2014 

6.5.2015 

21.7.2017 

EC: 16.9.2016 
MS: 13.9.2016 

MA: Ministry of Labour and Social Affairs 
NO IB 

Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) - 2014CZ16RFOP001 (amounts in million CZK) 

4 331 

NA 

4 331 

3 611 

7 942 

1,2,3,4,7 

13.4.2014 

29.4.2015 

1.9.2017 

EC: 13.3.2017 
MS: 13.3.2017 

MA: Ministry of Industry and Trade 
IB: Agency for Entrepreneurship and Innovation 

France - OP ERDF/ESF Lorraine et Vosges - 2014FR16M0OP015 

337 

72 

409 

279 

688 

1,2,3,4,5,6,
9,10 

8.8.2014 

11.12.2014 

NA 

EC: 13.1.2017 
MS: 21.12.2016 

MA (since 1 January 2016): L’autorité régionale 
pour la nouvelle région Grand Est 

France - National OP ESF for employment and social inclusion (PON FSE) - 2014FR05SFOP001 

NA 

2 894 

2 894 

2 577 

5 471 

8,9,10,11 

8.8.2014 

10.10.2014 

NA 

EC : 12.1.2017 
MS :21.10.2016 

MA: Ministère du travail, de l’emploi, de la 
formation professionnelle et du dialogue social 
13 DMA 
Several IB: of which Conseil départemental de la 
Seine-Saint-Denis, Conseil départemental du 
Val-de-Marne, Association France active, Fonds 
paritaire de la sécurisation des parcours 
professionels 

 
 
56 

ERDF 

ESF 

Total EU 
funds 

Other 
funds 

Total EU + 
other funds 

Amounts in million euros except for CZ 

Italy - OP ESF Piemonte - 2014IT05SFOP013 

TOs 
covered 

Partnership 
Agreement 
approval 
date 

EC approval 
date of the 
OP 

Latest 
version and 
approval 
date 

Designation of 
National 
Authorities by EC/ 
MS 

Managing authority (MA) 
If applicable: Intermediate Body (IB) or 
Delegated Managing Authority (DMA) 

NA 

436 

436 

436 

872 

8,9,10,11  29.10.2014 

11.7.2017 

12.12.2014 

EC:8.9.2017 
MS: 24.10.2016 

MA: Region Piemonte (unit “Social Cohesion”) 
IB: Citta Metropolitana Torino 

Italy - OP Puglia - 2014IT16M2OP002 

1 394 

386 

1 780 

1 780 

3 560 

1,2,3,4,5,6,
7,8,9,10, 
11,12 

29.10.2014 

17.7.2014 

13.8.2015 

EC: 21.2.2018 
MS:18.7.2017 

MA: Region Puglia (unit “Servizio Attuazione del 
programma”) 
IB: Puglia Sviluppo 

Finland - OP Sustainable Growth and work - 2014FI16M2OP001 

766 

513 

1 279 

1 279 

2 558 

1,3,4,8,9, 
10 

7.10.2014 

11.12.2014 

24.5.2016 

EC: 20.12.2016 
MS: 24.11.2016 

MA: The Department of Enterprise and Regional 
Development at the Ministry of Economic 
Affairs and Employment 
IB: ELY-Centers in Mikkeli and Oulu 

 
 
 
 
Priority axis, investment priorities and specific objectives to which the selected projects in 

Annex I correspond 

57 

ANNEX V 

Investment priority 

Specific Objective 

Priority 
Axis 

Thematic 
Objective 

Czech Republic - OPE 
8 
PA1 

PA2 

9 

8i - Access to employment for 
job-seekers and inactive people, 
including the long-term 
unemployed and people far from 
the labour market, also through 
local employment initiatives and 
support for labour mobility of 
workers 
8iv - Equality of women and men in 
all areas, including access to 
employment and career 
promotion, harmonisation of 
working and private life and 
promotion of equal pay the same 
work 
9i - Active inclusion, including with 
a view to promoting equal 
opportunities and active 
participation, and improving 
employability 

Czech Republic - OPEIC 
3 
PA2 

3 (b) - Developing and 
implementing new business 
models for SMEs, in particular with 
regard to internationalisation 
3 (c) - Supporting the creation and 
the extension of advanced 
capacities for product and service 
development 

France - OP Lorraine et Vosges 
PA2 

3 

A - Promotion of entrepreneurship 
spirit 
A - Promotion of entrepreneurship 
spirit 

PA9 

3 

France - PON FSE 
PA1 

8 

1.1 - Increase employment of 
supported persons, especially older, 
low-qualified and disadvantaged 
ones 

1.2 - Reduce the differences in the 
situation of women and men at the 
labour market 

2.1 - Increase the assertion of 
persons in danger of social exclusion 
and socially excluded persons in the 
society and at the labour market 

2.2 - Increase the internalisation of 
small and medium sized enterprises 

2.3 - Increase business infrastructure 
utilization 

To support investment in SMEs at all 
stages of their development 
To increase tourism in the Vosges 
mountains 

8iii - To support self- employment, 
entrepreneurship and creation of 

2 -0 1. Reinforcement and sharing of 
the service offer within networks 

 
 
 
 
 
 
Priority 
Axis 

Thematic 
Objective 

Investment priority 

Specific Objective 

58 

businesses, including innovating 
micro- and SMEs 

8v - Employees’, businesses’ and 
entrepreneurs’ adaptation to 
change 
9i - Active inclusion, including the 
promotion of equal rights, the 
active participation and a better 
ability to work 

8i - Access to employment for job-
seekers and inactive people, 
including the long-term 
unemployed and people far from 
the labour market, also through 
local employment initiatives and 
support for labour mobility 
8.ii - Sustainable integration into 
the labour market of young 
people, in particular those not in 
employment, education or 
training, including young people at 
risk of social exclusion and young 
people from marginalised 
communities, including through 
the implementation of the Youth 
Guarantee 

3.a - Promoting entrepreneurship, 
in particular facilitating the 
economic exploitation of new 
ideas and fostering the creation of 
new businesses 
9.a - Investing in health and social 
infrastructure, which contributes 
towards development, reduced 

PA2 

PA3 

8 

9 

Italy - OP Piemonte 
8 
PA1 

Italy - OP Puglia 
PA3 

3 

PA9 

9 

and or between the different actors 
supporting the creation or take over 
of activities and the strengthening of 
activities 
1 - The professionalisation of 
employees and volunteers from the 
support networks to the creation of 
businesses and supporting 
organisations for strengthening the 
activities 
3 - Actions allowing to meet the 
conditions and prerequisites for an 
effective access to training 
1 - a) Implementation of 
personalised paths, strengthening 
employability, with regard to the 
different obstacles to overcome, in a 
global approach 

ER 8.5 - Promote the entry onto the 
labour market and the employment 
of the long-term unemployed and 
those with the greatest difficulties in 
finding work, and support for people 
at risk of long-term unemployment 

ER 8.1 - Increase youth employment 

ER 3.5 - Promoting the set-up and 
strengthening of micro-enterprises 
and SMEs 

ER 9.3 - Increasing/ strengthening/ 
improving the quality of 
socio-educational services and 

 
 
 
 
 
 
 
Priority 
Axis 

Thematic 
Objective 

Investment priority 

Specific Objective 

59 

health inequalities and better 
service accessibility 

infrastructure for children and 
welfare services and infrastructure 
for people of restricted autonomy, 
and upgrading the infrastructure 
network and local healthcare and 
welfare services 

1.1 - Creating new business activities 

2.1 - Promoting the growth and 
expansion abroad of SMEs 

6.1 - Promoting the employment of 
young people and those in a weak 
position in the job market 

10.1 - Improving the ability of those 
outside employment to work and 
function 

Finland - OP Sustainable growth and work 
PA1 

3 

PA1 

3 

PA3 

8 

PA5 

9 

3a - Promoting entrepreneurship, 
in particular by facilitating the 
economic exploitation of new 
ideas and fostering the creation of 
new firms, including through 
business incubators 
3d - Supporting SMEs so that they 
can be part of the growth in the 
local, national and international 
markets and innovative processes 
8i - Access to employment for job-
seekers and inactive people, 
including the long-term 
unemployed and people far from 
the labour market, also through 
local employment initiatives and 
support for labour mobility of 
workers 
9i - Active inclusion, including with 
a view to promoting equal 
opportunities and active 
participation, and improving 
employability 

 
 
 
OP’s output and result indicators for the priority axis described under Annex V funded with ERDF 

PA/IP
/SO 

Output indicators 

Less 
developed 

Transition 

Developed 

Total 

Result indicators and 
source 

Transition 

Developed 

Baseline 

Target 2023  Baseline 

Target 

Call 

Project 

Regions 

60 

ANNEX VI 

CZECH REPUBLIC - OP Entreprise and Innovation for competitiveness (OPEIC) 2014CZ16RFOP001 
PA 2 
3b.2.2 

CO1: Number of entreprises receiving support 
CO2: Number of enterprises receiving grants 
CO4: Number of entreprises receiving a non 
financial support 
CO6: Private investment matching public 
support to enterprises (grants) (in million CZK) 

4 600 
2 100 
3 000 

PA 2 

3c.2.2 

CO1: Number of entreprises receiving support  

CO2: Number of enterprises receiving grants 
CO6: Private investment matching public 
support to enterprises (grants) (in million CZK) 
CO22: Total area of regenerated territory in 
hectares 

France - OP GRAND EST FR16M0OP015 
PA 2 
3.A 

CO1: Number of entreprises receiving support 
CO2: Number of enterprises receiving grants 
CO3: Number of entreprises receiving a 
support other than grants 
CO4: Number of entreprises receiving a non 
financial support 
CO8: Employment increase in supported 
enterprises 

PA 9 
3.A 

CO1: Number of entreprises receiving support  
CO2: Number of enterprises receiving grants 
CO4: Number of entreprises receiving a non 
financial support 
CO9: Increase of number of expected visits 

84 

695 

695 
335 

150 

1 100 
805 
530 

520 

960 

46 
34 
12 

520 

960 

79 
58 
21 

33 
24 
9 

4 600 
2 100 
3 000 

SR: percentage of exports 
in the total turnover of 
companies (source: MPO) 

2011: 
21.3 % 

22.3 % - 
23.8 % 

84 

695 

695 
335 

150 

SR: total area of 
regenerated territory in ha 
(source: national data base 
on regenerated 
borwnfields) 

2011: 
25 875 

25 900 – 
26 050  

4 

6, 7 

5 

8, 9 

1 100 
805 
530 

SR4: evolution of the 
investment of the region's 
industrial SMEs (source: 
national statistical office) 

2012: 
- 8.0 % 

3.0 % 

6 

10 ,11 
and 13 

11 760 

8 240 

20 000 

2011: 
1 692 

1 861 

2011: 
1 468 

1 614 

7 

12 

SR22: Number of expected 
visits and of skiers days 
in millions (sources: 
Observatoires régionaux du 
Tourisme and Domaine 
skiable de France) 

 
 
 
 
 
 
 
 
 
 
 
  
  
 
  
  
  
  
  
  
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
  
  
 
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
PA/IP
/SO 

Output indicators 

Less 
developed 

Transition 

Developed 

Total 

Result indicators and 
source 

Transition 

Developed 

Baseline 

Target 2023  Baseline 

Target 

Call 

Project 

Regions 

61 

ITALY - OP PUGLIA  2014IT16M2OP002 
PA 3 
3.a 

CO1: Number of entreprises receiving support  

CO5: Number of start ups benefiting from 
support 
CO8: Employment increase in supported 
enterprises 

FINLAND - OP Sustainable growth and work (2014FI16M2OP001) 
CO1: Number of enterprises receiving support 
PA 1 
CO2: Number of enterprises receiving grants 
 3a 
CO3: Number of enterprises receiving financial 
support other than grants 
CO5: Number of new enterprises supported 
CO6: Private investment matching public 
support to enterprises (grants in million euros) 

CO7: Private investment matching public 
support to enterprises (non-grants in million 
euros) 
CO8: Employment increase in supported 
enterprises 
6: Number of companies that start a new 
business operation after receiving funding 

7: Number of companies with a significant 
increase in turnover or staff 
8: Number of companies promoting low-
carbon solutions 
10: Number of SMEs with improved 
accessibility as a result of the project 
(Outermost or northern sparsely populated 
regions) 
9: Other investments promoting 
entrepreneurship as a result of the project 
(Outermost or northern sparsely populated 
regions - in million euros) 

SR3002: company net 
turnover rate (source: 
ISTAT) 

2012: 
- 0.6 % 

0.4 % 

2 000 

500 

50 

15 & 
16 

23, 24, 
25 and 
26 

1 

2 

2 465 
2 455 
10 

2 465  Business dynamics index 
2 455 
10 

(source: Statistics Finland) 

SME locations 

2010: 1 

1.1 

17 

31 

2010: 
16 761 

18 437  

1 200 
51.697 

1 200 
51.697 

10 

10 

7 800 

7 800 

850 

850 

1 010 

1 010 

615 

220 

615 

220 

13.7 

13.7 

PA 1 
3d  

CO2: Number of enterprises receiving grants 
CO3: Number of enterprises receiving financial 
support other than grants 

3 175 
20 

3 175 
20 

Business dynamics index 
(source: Statistics Finland) 

1 

1.10 

17  

27, 28, 
32, 33, 
34 

 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   
   
   
   
62 

Transition 

Developed 

Baseline 

Target 2023  Baseline 
4 326 

Target 

4 759 

Call 

Project 

PA/IP
/SO 
 PA 1 
3d  

Output indicators 

Less 
developed 

Transition 

Developed 

Total 

Result indicators and 
source 

Regions 

CO6: Private investment matching public 
support to enterprises (grants) in million euros 
CO7: Private investment matching public 
support to enterprises (non-grants) in million 
euros 
CO8: Employment increase in supported 
enterprises  
Specific indicators: 

Number of companies in projects where the 
main objective is to promote growth and 
international business operations 
Number of companies that are starting to 
export or that expand their exports to a new 
market area 
Energy saved (MWh) 
Companies with a significant increase in 
turnover or staff 
Companies promoting low-carbon solutions 

337.15 

337.15  Growing companies 

(employs min. 3 people and 
average growth of staff 
exceeds 10 %) (source: 
Statistics Finland)  

10 

10 

4 900 

4 900 

5 170 

5 170 

1 330 

1 330 

460 435 
1 340 

460 435 
1 340 

650 

650 

Legend: CO: common output indicator; CR: common result indicator, SR: specific result indicator (under the ERDF, all result indicators are specific to the Member 
State / OP). 

 
 
 
 
 
 
 
 
 
 
 
  
  
  
  
  
 
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
63 

ANNEX VII 

Output and result indicators for the priority axis described under Annex V funded with ESF 

PA/IP 

Output indicators 

M 

F 

Total 

Result indicators and source 

Baseline 

Target (2023) 

M 

F 

Total 

M 

F 

Total 

Call 

Project 

CZECH REPUBLIC - OP Employment 2014CZ05M9OP001 
PA1 
1.1 

CO06: Participants below 25 years of age 

CO07: Participants above 54 years of age  

CO20: Number of projects fully or 
partially implemented by social partners 
or NGOS 

Specific indicators: 
CESF0: Total number of participants 

Unemployed participants, including the 
long-term unemployed persons 

Participants with completed primary 
ISCED) or lower-secondary (ISCED 2) 
education 

38 571  CR01: Inactive participants who started 
seeking a job again after termination of 
their participation (IS ESF 2014+) 

51 429  CR03: Participants gaining a qualification 

after termination of their participation  

100  CR04: Participants employed, including 
the self-employed, after termination of 
their participation 
CR05: Disadvantaged participants seeking 
a job after termination of their 
participation, involved in education/ 
training, improving their qualification or 
already employed, including the self-
employed  
CR06: Participants employed 6 months 
after termination of their participation, 
including the self-employed  

178 000 

172 493  CR007: Participants whose position on the 

labour market has improved 6 months 
after termination of their participation 

8 900  CR008: Participants at the age of 54+ 

employed 6 months, including self-
employed, after termination of their 
participation  

2013: 
1 210 

13 268 

2 256 

4 089 

1 805 

903 

1 292 

3 500 

1 

1 

67 000 

58 740 

42 000 

82 000 

1 700 

12 300 

Inactive participants 

5 687  CR09: Disadvantaged participants 

268 

41 000 

employed 6 months after termination of 
their participation, including the self-
employed 

 
 
 
 
  
 
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
 
  
 
 
  
 
  
  
 
  
 
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Baseline 

Target (2023) 

M 
134 

F 
 1 220 

Total 
 1 354 

M 
100 

F 
 1 400 

Total 

1 500 

2  

2, 3, 4  

64 

Call 

Project 

PA/IP 

PA1 
1.2  

Output indicators 

M 

F 

Total 

Result indicators and source 

90  CR01: Inactive participants who started 
seeking a job again after termination of 
their participation (IS ESF 2014+) 

CO20: Number of projects fully or 
partially implemented by social partners 
or NGOS 

CO21: Number of projects targeting 
sustainable employment of women and 
sustainable career progression of women 
CO22: Number of projects targeting 
public authorities or public services at 
national, regional and local levels 
CO23: Number of micro, small and 
medium sized enterprises supported 
(incl. cooperatives and social businesses) 

CO35: Capacity of supported child-care 
facilities or educational facilities  

Specific output indicators: 
CESF0: Total number of participants 

940 

9 460 

80500: The number of analytical and 
strategic documents written and 
published (incl. evaluation ones) 

50100: Number of supported child-care 
facilities or educational facilities  

50105: Number of employers supporting 
flexible forms of work  

410  CR03: Participants gaining a qualification 

49 

440 

489 

50 

450 

500 

after termination of their participation 

60  CR04: Participants employed, including 
the self-employed, after termination of 
their participation 

26 

245 

271 

48 

432 

480 

130  CR05: Disadvantaged participants seeking 

69 

622 

691 

131 

1 184 

1 315 

a job after termination of their 
participation, involved in 
education/training, improving their 
qualification or already employed, 
including the self-employed  
4 000  CR06: Participants employed 6 months 
after termination of their participation, 
including the self-employed 
CR007: Participants whose position on the 
labour market has improved 6 months 
after termination of their participation 
CR008: Participants at the age of 54+ 
employed 6 months, including self-
employed, after termination of their 
participation 

10 400 

35  CR09: Disadvantaged participants 

employed 6 months after termination of 
their participation, including the self-
employed 
Specific indicators: 

333  50110: Number of persons using a 

childcare facility for pre-school children 

70  50130: Number of persons using a flexible 

form of work 

48 

432 

480 

55 

495 

550 

50 

243 

105 

105 

495 

600 

4 

34 

38 

5 

45 

50 

336 

400 

50 

150 

6 000 

500 

 
 
 
 
  
 
  
  
 
  
  
 
  
  
 
  
  
 
  
 
  
 
  
 
  
 
  
 
 
 
 
 
 
 
 
 
 
 
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
PA/IP 

PA2 
2.1  

CO20: Number of projects fully or 
partially implemented by social partners 
or NGOS 
CO22: Number of projects targeting 
public authorities or public services at 
national, regional and local levels 
CO23: Number of micro, small and 
medium sized enterprises supported 
(incl. cooperatives and social businesses) 

Output indicators 

M 

F 

Total 

Result indicators and source 

Baseline 

Target (2023) 

M 

F 

Total 

415  CR01: Inactive participants who started 
seeking a job again after termination of 
their participation (IS ESF 2014+) 
14  CR04: Participants employed, including 
the self-employed, after termination of 
their participation 

231  CR05: Disadvantaged participants seeking 

a job after termination of their 
participation, involved in 
education/training, improving their 
qualification or already employed, 
including the self-employed  
CR06: Participants employed 6 months 
after termination of their participation, 
including the self-employed  
CR007: Participants whose position on the 
labour market has improved 6 months 
after termination of their participation 

M 

F 

Total 

2 527 

1 010 

934 

665 

3 326 

143 

455 

65 

Call 

Project 

3 326 

3 

5 

1 386 

5 636 

860 

16 724 

138 

832 

Specific indicators: 
CESF0: Total number of participants 

167 244 

67001: Capacity of supported services  

32 063  CR008: Participants at the age of 54+ 

67101: Number of supporting institutions 
supported 

80500: The number of analytical and 
strategic documents written and 
published (incl. evaluation ones) 
67401: New or innovated social services 
related to housing 

10210: Number of social enterprises 
established thanks to the support 

67300: Number of participants provided 
with consulting on social 
entrepreneurship 

employed 6 months, including self-
employed, after termination of their 
participation 

5  CR09: Disadvantaged participants 

employed 6 months after termination of 
their participation, including the self-
employed 
Specific indicators: 
67010: Use of supported services 

18 

247  67110: Number of ancillary institutions 

which operate even after termination of 
the support  

138  62910: Participants employed including 
self-employed 12 months upon leaving 

80  67310: Former participants of projects in 
which intervention by means of social 
work served its purpose 
including the self-employed  

16 000 

74 000 

2 

333 

5 

430 

27 914 

62 558 

 
 
 
   
 
   
 
   
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
 
  
 
 
  
 
  
  
 
  
 
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
 
  
 
  
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
  
 
  
 
  
PA/IP 

Output indicators 

M 

F 

Total 

Result indicators and source 

   10211: Number of social enterprises 

established thanks to support, operating 
even after termination of support 

1 400 000  CR02: Participants following a 

675 000 

qualification after termination of their 
participation  

   CR03: Participants gaining a qualification 
after termination of their participation  

CR04: Participants employed, including 
the self-employed, after termination of 
their participation 

1 000 000 

230 000 

Specific indicators : 
R91.4: Number of actions for coordinating 
and facilitating implemented 

   R91.6: Number of organisations with 
social utility and of employers that 
received support 

340 400 

Specific indicators (no common indicators 
for this specific objective) the values for 
different categories of regions are 
cumulated: 
R83.1: Number of businesses created 
129 200  R83.2: Number of businesses created 

priority districts from the City policy 

34 000  R83.3: Number of sharing activities 

achieved 

   R83.4: Number of businesses created by 

women 

66 

Baseline 

Target (2023) 

M 

F 

Total 

100 

M 

F 

Total 

92 

Call 

Project 

91 322 

25 961 

281 063 

180 000 

14 & 15 

8 & 
9  

52 500 

575 000 

11 

17 

90 000 

9 000 

36 000 

9 900  10 600 

20 500  CR06: Disadvantaged participants in 

34 

36 

35 

45 

45 

45 

13 

19 & 20 

employment, including self-employment, 
6 months after leaving training 

2 700 

FRANCE - NATIONAL OP ESF 2014FR05SFOP001 
PA3 
9i 

CO01: Unemployed, including long-term 
unemployed 
CO03: Participants inactive 

Specific indicators: 
9i3: Number of women 

9i4: Number of participants from priority 
districts from the City policy 

9i5: Number of projects aiming at 
coordinating and facilitating the offer 
related to social insertion 
9i7: Number of projects aiming at 
mobilising employers from the 
commercial and non-commercial sectors 

PA 1 
8iii 

Specific indicators : 
8iii1: Number of entrepreneurs or buyers 
that received support  

8iii2: Number of female entrepreneurs 
that received support 

8iii3: Number of entrepreneurs from 
priority districts from the City policy that 
received support 

ITALY - OP PIEMONTE 2014IT05SFOP013 
PA 1 
1.8i 

CO01: Unemployed, including long-term 
unemployed 
CO23: Number of supported micro and 
SMEs 

 
 
  
  
  
  
  
  
  
  
  
  
   
  
  
 
 
  
  
  
 
  
  
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   
  
 
  
  
 
 
 
  
  
  
  
  
  
  
  
  
 
 
  
  
  
  
  
  
  
  
  
 
 
  
 
  
  
  
  
  
  
  
 
 
 
  
  
  
  
  
  
  
  
 
 
Output indicators 

M 

F 

Total 

Result indicators and source 

Baseline 

Target (2023) 

M 

F 

Total 

M 

F 

Total 

Call 

Project 

67 

CO01: Unemployed, including long-term 
unemployed 
CO23: Of micro, small and medium sized 
enterprises supported (incl. cooperatives 
and social businesses) 

CO01: Unemployed, including long-term 
unemployed 
CO03: Participants inactive 
CO06: Participants below 25 years of age 

8 500 

8 500 

17 000  CR06: Disadvantaged participants in 

30.5 

30.5 

30.5 

40.5 

40.5 

40.5  14  

21 & 22 

13 300 
7 700 
18 150  12 550 

21 000 
30 700 

employment, including self-employment, 
6 months after leaving training 

PA/IP 

PA 1 
1.8i 

PA 1 
1.8ii 

FINLAND – OP Sustainable growth and work (2014FI16M2OP001) 
PA 3 
8i 

Specific indicators: 
Number of participants under 30 years of 
age who are unemployed or outside the 
labour force 
Number of participants over 54 years of 
age who are unemployed or outside the 
labour force 

PA5  
9i 

CO02: The long-term unemployed 

CO04: Those outside employment and 
not in training 
CO20: Number of projects realised or 
partly realised by social partners or civic 
organisations 

26 000 

Specific indicator: 
Participants under 30 in employment 
when leaving the measure 
(source: statistics Finland) 

8 600  Participants over 54 in employment when 

leaving the measure 

11 250 

4 100 

Specific indicator: 
Improved work and functional capacity of 
the participants (Separate review) 

55 

Legend: CO: common output indicator; CR: common result indicator; M: male; F: female. 

2013: 
23 

17 

2012: 
5.5 

18 

29  

30 

23 

5.9 

18 

30  

 
 
  
  
  
 
 
 
 
 
 
 
 
  
  
  
  
 
 
 
 
 
 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
  
 
  
  
 
 
  
 
  
 
  
  
 
  
 
  
 
  
  
 
 
 
  
 
  
  
 
  
 
  
 
 
68 

ANNEX VIII 

Outputs and results in the selection procedures and applications examined 

(a) 

(b) 

Is there a selection criteria 
about result indicators / 
expected results? 

Output indicators 
corresponding to those at OP 
level and quantified 

Result indicators 
corresponding to those at OP 
level and quantified 

Czech Republic OPE 2014CZ05M9OP001 

Output 

Results 

Are there quantified indicators in the applications? 

Is there an assessment of the indicators and the values 
provided? 

1 

2 

3 

1 

2 

3 

4 

5 

Expected results to be 
outlined, result indicators 
mentioned and corresponds 
to the OP indicators but no 
target / quantification 
required 

Czech Republic OPEIC 2014CZ16RFOP001 

4 

5 

6 

7 

8 
9 

Expected results to be 
outlined, result indicators 
mentioned and correspond to 
the OP indicators but no 
target / quantification 
required 

France OP Lorraine et Vosges 2014FR16M0OP015 

6 

10 

11 

13 

Expected results to be 
outlined (project have to 
prove how they contribute to 

NA, no information on results 

Use of “unit costs” (c) 

NA, no information on results 

Applications outlining expected 
results and their contribution to 
the OP objectives, no 
quantification of the result 
indicator related to the OP 

They are binding and are 
mentioned in the grant 
agreement 

Assessment used for the 
scoring 

Assessment used for the 
scoring 

Applications outlining expected 
results and their contribution to 
the OP objectives 

Limited assessment of the 
expected results in the 
assessment reports 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
  
Are there quantified indicators in the applications? 

Output indicators 
corresponding to those at OP 
level and quantified 

Result indicators 
corresponding to those at OP 
level and quantified 

Is there an assessment of the indicators and the values 
provided? 

Output 

Results 

69 

Limited assessment of the 
expected results in the 
assessment reports (result 
indicator not assessed) 

(a) 

7 

(b) 

12 

Is there a selection criteria 
about result indicators / 
expected results? 
the achievement of the 
indicators) 

France PON FSE 2014FR05SFOP001 
14 

8 

9 

15 

10 

11 

12 

16 

17 

18 

Not explicitely requested but 
this is required in the 
standard format application 

Output indicators not related to 
OP indicators and not 
quantified 

But result indicators different 
from the OP result indicators 

(Indicators different from the 
OP’s were also provided) 

But result indicators different 
from the OP result indicators 

Only qualitative information on 
the expected results 

Only the relevance of the 
indicators used was 
assessed 

Only relevance of the indicators 
used was assessed 

Italy OP Piemonte 2014IT05SFOP013 

13 
14 

19&20 

21&22 

Italy OP Puglia 2014IT16M2OP002 

15 

16 

23&24 

25&26 

Request for expected results 
only through the business 
plan 

(Indicators different from the 
OP’s were also provided) 

Consistency with amount 
of grant 

Use of unit costs (c) 

NA, no info on results 

Applications outlining expected 
results and their contribution to 
the OP objectives 

Assessment of the described 
expected results but with 
weaknesses 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
(a) 

(b) 

Is there a selection criteria 
about result indicators / 
expected results? 

Output indicators 
corresponding to those at OP 
level and quantified 

Result indicators 
corresponding to those at OP 
level and quantified 

Output 

Results 

Are there quantified indicators in the applications? 

Is there an assessment of the indicators and the values 
provided? 

70 

Finland OP Sustainable growth and work 2014FI16M2OP001 
17 ERDF 

27, 28 
31, 32 
33, 34 

But they do not correspond to 
the OP indicators 

No information on output 
indicators but data may be 
obtained directly from the IT 
system 

29 

30 

18 ESF 

19 ESF/ERDF 

20 ESF/ERDF 

But for the ERDF they do not 
correspond to the OP 
indicator 

YES 

Partially 

NO 

NA: not applicable 

NA, no info on outputs 

But indicators do not 
correspond to the OP indicator 

Assessment used for the 
scoring 

NA, no info on results 

NA, no projects examined under these selection procedures 

Note: (a) See Annex III list of selection procedures. (b) See Annex I list of projects examined. (c) Unit costs is a methodology to determine the amount of eligible costs where 
this amount is equal to the output to be reached (for instance the number of places to be created in a crèche, here number of places created) multiplied by the unit cost per 
place. 

Source: (1) Documentation for the selection procedures. (2) Applications submitted for the projects examined during this audit. (3) Assessment reports for these applications. 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
REPLIES OF THE COMMISSION TO THE SPECIAL REPORT OF THE EUROPEAN 
COURT OF AUDITORS 

"SELECTION AND MONITORING FOR ERDF AND ESF PROJECTS IN THE 2014-2020 
PERIOD ARE STILL MAINLY OUTPUTS-ORIENTED" 

EXECUTIVE SUMMARY 

IV.  The  selection  of  projects  is  the  exclusive  responsibility  of  the  Member  States  (MS).  The 
monitoring  arrangements,  as  well  as  the  achievements  reported  in  the  Annual  Implementation 
Reports (AIRs) and progress reports, allow to assess progress towards the objectives quantified by 
the output and result indicator targets as programmed in the operational programmes (OP) (which 
include  baselines  and  targets).  Evaluations  make  it  possible  to  assess  the  contribution  to  policy 
objectives. 

First bullet: The Commission considers that there is a difference between direct results of projects 
financed and the results to be achieved at the level of the OP (reflected by the result indicators).  

As  regards  the  basis  of  selecting  projects,  the  Commission  considers  that  in  many  types  of 
assistance,  it  is  more  efficient  to  set  quality  thresholds  and  accept  all  projects  that  meet  this 
threshold than to apply a method of direct comparison of applications. The latter approach may lead 
to  a  high  number  of  initial  project  applications  which  in  itself  do  not  necessarily  result  in  better 
overall  project  quality  and  would  entail  unnecessary  administrative  burden  for  the  managing 
authorities and for applicants. 

Second bullet: Audits of the monitoring systems can only be carried out in a meaningful way once 
sufficient  data  on  the  projects  implemented  has  been  collected,  aggregated  and  reported  to  the 
Commission. In the cases of the European Regional Development Fund (ERDF) and the Cohesion 
Fund (CF), due to longer project duration, this explains a later start of the reporting and, therefore, 
of  the  audits  on  reliability  of  the  reported  values.  The  Commission  will  continue  its  own  audits 
during  2018  and  undertake  further  efforts  to  raise  the  awareness  of  audit  authorities  about  the 
importance of providing assurance on the reliability of performance data monitoring systems ahead 
of the performance review. 

The  monitoring  systems  fall  under  the  responsibility  of  Member  States  who  should  have  put 
everything  in  place  according  to  the  regulatory  requirements.  The  Commission  has  continuously 
made  every  possible  effort  to  provide  guidance  and  practical  support  (e.g.  monitoring  guidance 
document,  evaluation  partnership  meetings,  the  ESF  Data  Support  Centre)  to  Member  States  in 
setting up robust monitoring systems. 

Third  bullet:  The  Commission  considers  that  the  established  monitoring  framework  is  now  more 
results-oriented.  

The  2017  Strategic  report  is  based  on  MS  reports  as  of  end-2016.  Given  the  slow  start  of 
programme implementation, it was too early to report on results. Until the next strategic report, the 
achievement  values  for  the  European  Structural  and  Investment  Funds  (ESI  Funds)  common 
indicators are available on the Open Data platform. 

V. 

Second bullet: The definition of result indicators, applicable to all Funds, is included in Article 2 of 
the  Commission’s  proposal  for  a  Common  Provisions  Regulation  (CPR)  for  the  post-2020 
programming period. 

The  Commission’s  legislative  proposal  for  the  post-2020  programming  period  includes  a  list  of 
common  result  indicators  for  ERDF  and  the  Cohesion  Fund  in  line  with  the  Better  Regulation 
Guidelines  and  with  the  concept  of  ‘results’  used  under  the  European  Social  Fund  (ESF).  These 
result  indicators  would measure  the  effects  on  the  beneficiaries,  including  the  short-term  ones,  of 
the interventions supported. 

1 

 
The  Better  Regulation  guidelines  and  toolbox  provide  definitions  of  output,  result  and  impact 
indicators, as well as their links with objectives. 

Third  bullet:  The  Commission  will  continue  working  with  the  Member  States  to  improve  the 
reliability  of  data.  The  Commission  has  invited  the  Member  States  to  submit  relevant  OP 
amendments  necessary  to  revise  targets  and  milestones  by  30  June  2018  in  order  to  ensure  their 
timely processing. 

OBSERVATIONS 

16. 

Fourth  bullet:  As  regards  the  basis  of  selecting  projects,  the  Commission  considers  that  in  many 
types of assistance, it is more efficient to set quality thresholds and accept all projects that meet this 
threshold than to apply a method of direct comparison of applications. The latter approach may lead 
to  a  high  number  of  initial  project  applications  which  in  itself  do  not  necessarily  result  in  better 
overall  project  quality  and  would  entail  unnecessary  administrative  burden  for  the  managing 
authorities and for applicants. 

25. The Commission considers that there is a difference between direct results of projects financed 
and  the  results  to  be  achieved  at  the  level  of  the  OP  (reflected  by  the  result  indicators).  Whereas 
direct  results  are  influenced  only  by  the  action  co-financed,  results  indicators  are  affected  by 
different  external  factors  outside  the  control  of  the  managing  authorities.  A  well-  designed 
intervention logic shall ensure that the outputs and direct results of the selected projects contribute 
to achieving the expected results of the OP. 

26. The selection of projects is the exclusive responsibility of the managing authorities that define 
the selection criteria and apply them when examining the applications for funding. 

27.  Selection  criteria  do  not  necessarily  have  to  include  scoring  of  expected  results.  The 
intervention logic of the programme sets out how the outputs of the individual projects contribute to 
achieving the expected results of the OP.  

See also the Commission reply to paragraph 25. 

28. A broad population of applicants is not always necessary, depending on the policy area. 

32. The Commission considers that in many types of assistance, it is more efficient to set a quality 
threshold  and  accept  all  projects  that  meet  them  than  to  apply  a  method  of  direct  comparison  of 
applications. In other types of assistance competitive selection procedure may not contribute to the 
specific  objectives  of  the  OP  (e.g.  transport  infrastructure  projects  or  projects  of  municipalities 
where competition between public authorities/municipalities is not necessary). 

34.  The  Commission  notes  that  in  the  case  of  Finland  the  procedure  included  the  allocation  of  a 
minimum  score  to  ensure  that  only  quality  projects  are  selected.  Comparison  between  projects 
would not bring more quality except if there was insufficient funding to select all the projects.  

The  Commission  considers  that  the  quality  of  projects  selected  depends  more  on  the  selection 
criteria applied than on the type of selection procedure.  

See also Commission reply to paragraph 16. 

44. The classification is in accordance with the applicable EU rules on data protection. Indeed, in 
case  of  the  special  category  of  data  (sensitive  data),  the  participant  can  refuse  to  declare  it. 
Nevertheless,  the  managing  authority  (MA)  is  required  by  the  ESF  monitoring  guidance  to 
document the attempt of collection. Moreover, the Directive lays down that subject to the provision 
of suitable safeguards, MS may, for reasons of substantial public interest, lay down exemptions to 
the participant's consent. 

2 

 
Having regard to the participants' right to refuse consent, the participation records are considered as 
complete if all non-sensitive data are recorded.  

The minimum requirement for participation records complete with all non-sensitive data has been 
clarified in the ESF monitoring guidance. 

Box 2 - Examples 

The French MA has taken measures targeting delegated MAs, IB and beneficiaries in order to foster 
a timely collection and submission of data, as advised by the Commission and national evaluations. 
Correction of AIR data allowed to report about more than 925 000 participants at the end of 2016. 

Box 3 - Examples of IT systems still not fully functional and related risks 

First  bullet:  The  Commission  constantly  monitors  the  performance  of  the  Czech  single  IT 
monitoring system for all OPs. It has been a regular point of the Partnership Agreement follow-up 
meetings with the National Coordinating Authority which is in charge of the functioning of the ESI 
Funds IT monitoring system. 

Second  bullet:  The  issue  raised  by  the  ECA  is  in  line  with  the  results  of  the  Commission's 
performance data reliability audit of this programme. 

52.  The  shortcomings  in  the  AIRs  do  not  necessarily  put  into  question  the  reliability  of  the  data 
reported. When the Commission has doubts about the indicator values reported, the issue is raised 
in the observation letters on the AIRs. Any relevant audit findings are followed up as well. 

53. Performance data should be verified by the managing authority before it is included in the AIR. 
On the other hand, system audits (by audit authority or the EC) may be carried out more usefully 
after submission of the AIR, to allow backtesting of the data submitted to the Commission, in order 
to provide assurance on the reliability of the entire system for collecting, aggregating and reporting 
the data including the last step of submitting the data to the EC through SFC2014. 

The legal base, however, does not give enough time to audit AIR data before the AIRs and strategic 
reports are issued. Most recent, corrected and updated data can always be found on the Open Data 
Platform. 

60. To mitigate the risk the ECA refers to in this paragraph, the Commission will continue its own 
audits during 2018 and undertake further efforts to raise the awareness of audit authorities about the 
importance of providing assurance on the reliability of performance data monitoring systems ahead 
of the performance review.  

61. A meaningful assessment concerning the underachievement of targets set for 2023 can only be 
performed once the timeline for meeting the targets is reached. 

Linking,  coupling  or  modifying  allocations  across  priority  axes,  as  well  as  the  upward  and 
downward  revisions  of  target  values  for  2023  can  be  misleading  as,  whilst  such  modifications  of 
allocations between priority axes indeed "naturally" lead to changes in target values, there are other 
reasons that can justify the change of 2023 target values. 

62.  The  Member  States  may  propose  revisions  of  milestones  and  targets  when  changes  in 
allocations  for  a  given  priority  occur  or  in  duly  justified  cases  provided  for  in  the  Regulations 
(Article 30(1) CPR, CPR Annex II, Article 5(6) of Regulation (EC) No 215/2014). 

In order to make sure that such OP amendments are submitted and processed in due time before end 
2018 and to avoid changes in the performance framework designed to simply adjust values to actual 
implementation  on  the  ground,  Member  States  have  been  invited  to  officially  submit  any  such 
request  by  30  June  2018.  The  assessment  of  these  OP  amendment  requests  follows  a  standard 
procedure in order to ensure consistent treatment. 

3 

 
65.  The  logical  framework  for  the  ERDF  in  2014-2020  period  takes  account  of  the  fact  that 
achievement of the objectives set for the programmes is due to the actions co-financed by the public 
intervention,  for  example  by  the  Funds,  as  well  as  other  factors.  Therefore,  while  the  result 
indicators  allow  to  make  a  judgement  on  whether  or  not  the  indicators  move  in  the  desired 
direction, to disentangle the effects of the EU intervention from the contribution of other factors and 
to understand the functioning of a programme it is necessary to carry out an impact evaluation. 

69. See Commission reply to paragraph 25.  

71. The performance framework relies on various indicators to ensure that programmes are on track 
to deliver what was agreed. Result indicators may not be appropriate (for ERDF/CF), because the 
timing  when  results  can  be  achieved  and  captured  by  the  system  is  not  compatible  with  the 
compulsory timeframe for a useful performance reserve allocation. 

72. All ESI Funds common indicator achievements as reported by the MAs are publicly available in 
the Open Data Platform. 

The  2017  Strategic  report  is  based  on  MS  reports  as  of  end-2016.  Given  the  slow  start  of 
programme implementation, it was too early to report on results. 

73.  The  definition  of  result  indicators,  applicable  to  all  Funds,  is  included  in  Article  2  of  the 
Commission’s  proposal  for  a  Common  Provisions  Regulation  for  the  post-2020  programming 
period. 

The  Commission’s  legislative  proposal  for  the  post-2020  programming  period  includes  a  list  of 
common  result  indicators  for  ERDF  and  the  Cohesion  Fund  in  line  with  the  Better  Regulation 
Guidelines  and  with  the  concept  of  ‘results’  used  under  the  ESF.  These  result  indicators  will 
measure  the  effects  on  the  beneficiaries,  including  the  short-term  ones,  of  the  interventions 
supported. 

The ESF and ERDF context are fundamentally different. However, the Commission has set up an 
inter-service  group  to  address  the  differences  of  indicators  across  all  ESI  Funds.  In  this  way,  the 
highest  possible  consistency  as  regards  the  concepts  will  be  reached,  taking  into  account  the 
different nature of the operations co-funded by the ESI Funds.  

The  Better  Regulation  guidelines  and  toolbox  provide  definitions  of  output,  result  and  impact 
indicators, as well as their links with objectives. 

CONCLUSIONS AND RECOMMENDATIONS 

74.  The  Commission  considers  that  the  established  monitoring  framework  is  now  more  results-
oriented.  

Delegated regulation 480/2014 requires the recording and storing of monitoring data, in particular 
the  target  value  of  the  output  and  result  indicator  (data  fields  33  and  38),  and  the  achievement 
values at operation level.   

In the context of ESI Funds, outputs and results are equally measured, monitored and reported on 
regularly  through  for  instance  the  Annual  Implementation  Reports.  Member  States  have  the 
possibility  to  report  either  on  partially  or  fully  completed  operations  according  to  the  regulatory 
provisions.  Also,  due  to  the  duration  of  the  participations,  result  indicators  are  reported  with  a 
certain time-lag compared to output indicators. The achievement values of all ESI Funds common 
indicators are available in the Open Data Platform. 

75. The Commission considers that there is a difference between direct results of projects financed 
and  the  results  to  be  achieved  at  the  level  of  the  OP  (reflected  by  the  result  indicators).  Whereas 
direct  results  are  influenced  only  by  the  action  co-financed,  results  indicators  are  affected  by 
different  external  factors  outside  the  control  of  the  managing  authorities.  A  well-  designed 

4 

 
intervention logic shall ensure that the outputs and direct results of the selected projects contribute 
to achieving the expected results of the OP. 

The selection of projects is the exclusive responsibility of the managing authorities that define the 
selection criteria and apply them when examining the applications for funding. 

77. The Commission considers that in many types of assistance, it is more efficient to set a quality 
threshold  and  accept  all  projects  that  meet  them  than  to  apply  a  method  of  direct  comparison  of 
applications. In other types of assistance competitive selection procedure may not contribute to the 
specific  objectives  of  the  OP  (e.g.  transport  infrastructure  projects  or  projects  of  municipalities 
where competition between public authorities/municipalities is not necessary). 

The  Commission  considers  that  the  quality  of  projects  selected  depends  more  on  the  selection 
criteria applied than on the type of selection procedure.  

78. The monitoring systems fall under the responsibility of Member States who were required to put 
these into place according to the regulatory requirements.  

The Commission has followed a proactive approach in relation to monitoring by providing guidance 
to Member States and its desk officers,  by carrying out plausibility tests of the data submitted by 
Member States, and by introducing additional validation rules in the IT system ‘SFC’ minimizing 
the possibility to report erroneous data. Updates of data are normal and have been encouraged by 
the  Commission.  Data  validation  takes  time,  especially  when  setting  up  the  MIS.  Most  recent, 
corrected and updated data can always be found on the Open Data Platform. 

79.  The  Commission  will  carry  out  additional  audits  of  monitoring  systems  under  its  specific 
enquiry during 2018 and continue collaboration with the audit authorities to increase assurance on 
the reliability of performance data monitoring systems ahead of the performance review. 

80.  The  Member  States  may  propose  revisions  of  milestones  and  targets  when  changes  in 
allocations  for  a  given  priority  occur  or  in  duly  justified  cases  provided  for  in  the  Regulations 
(Article 30(1) CPR, CPR Annex II, Article 5(6) of Regulation (EC) No 215/2014). 

In  order  to  make  sure  that  such  OP  amendments  are  submitted  and  processed  in  due  time  before 
end- 2018 and to avoid changes in the performance framework designed to simply adjust values to 
actual  implementation  on  the  ground,  Member  States  have  been  invited  to  officially  submit  any 
such request by 30 June 2018. 

REGIO  has  made  an  internal  analysis  of  upcoming  OP  amendments  relevant  to  performance 
framework changes, prioritised the upcoming performance review in its management plan and has 
planned resources accordingly. 

81. The ESF and ERDF context are fundamentally different. However, the Commission has set up 
an inter-service group to address the differences in concepts across all ESI Funds. In this way, the 
highest  possible  consistency  will  be  reached  taking  into  account  the  different  nature  of  the 
operations co-funded by the ESI Funds. 

The  definition  of  result  indicators,  applicable  to  all  Funds,  is  included  in  Article  2  of  the 
Commission’s  proposal  for  a  Common  Provisions  Regulation  for  the  post-2020  programming 
period. 

The  Commission’s  legislative  proposal  for  the  post-2020  programming  period  includes  a  list  of 
common  result  indicators  for  ERDF  and  the  Cohesion  Fund  in  line  with  the  Better  Regulation 
Guidelines  and  with  the  concept  of  ‘results’  used  under  the  ESF.  These  result  indicators  would 
measure  the  effects  on  the  beneficiaries,  including  the  short-term  ones,  of  the  interventions 
supported. 

5 

 
82. Individual projects deliver outputs and direct results, which are not necessarily the same as the
specific  objectives  of  the  programmes.  It  is  the  totality  of  the  projects  financed  under  a  specific
objective that shall contribute to achieving the expected results.

The intervention logic of the programmes describes how the outputs of the individual projects shall 
contribute  to  achieving the  objectives  of  the  programme.  It  is  the  task of  evaluation  to  assess ex-
post whether the intervention logic was effective to reach the objectives. 

Concerning  the  ESF,  the  grant  agreement  does  not  require  quantified  targets  according  to  the 
delegated act (480/2014, see above).  

83. In  the  ESF  context,  outputs  and  results  are  equally  measured,  monitored  and  reported  on
regularly  for  instance  through  the  Annual  Implementation  Reports.  It  is  to  be  noted  that  Member
States have the possibility to report either on partially or fully completed projects according to the
regulatory provisions.

The performance framework relies on various indicators to ensure that programmes are on track to 
deliver  what  was  agreed.  Result  indicators  may  not  be  appropriate  (for  ERDF/CF),  because  the 
timing  of  when  results  can  be  achieved  and  captured  by  the  system  is  not  compatible  with  the 
compulsory timeframe for a useful performance reserve allocation. 

All ESI Funds common indicator achievements as reported by the MAs are publicly available in the 
Open Data Platform. 

Recommendation 1 – Results-oriented selection 

The Commission notes that this recommendation is addressed to the Member States. 

Recommendation 2 – Results-oriented monitoring 

(a) The Commission notes that this sub-recommendation is addressed to the Member States.

(b) The Commission accepts the recommendation.

The  definition  of  result  indicators,  applicable  to  all  Funds,  is  included  in  Article  2  of  the 
Commission’s  proposal  for  a  Common  Provisions  Regulation  for  the  post-2020  programming 
period. 

The  Commission’s  legislative  proposal  for  the  post-2020  programming  period  includes  a  list  of 
common  result  indicators  for  ERDF  and  the  Cohesion  Fund  in  line  with  the  Better  Regulation 
Guidelines  and  harmonised  with  ESF.  These  result  indicators  would  measure  the  effects  on  the 
beneficiaries, including the short-term ones, of the interventions supported. 

Recommendation 3 – Reporting on performance and preparation of the performance review 
in 2019 

(a) The Commission accepts this recommendation and considers it as being implemented.

The  audit  units  in  DG  EMPL  have  already  carried  out  a  significant  number  of  audits  on  the 
reliability of performance data and continue to include this aspect in their Early Preventive Audit 
missions.  Furthermore,  a  specific  performance  audit,  covering  also  the  monitoring  systems,  was 
carried  out  by  an  external  auditor  –  under  the  responsibility  of  DG  EMPL  –  and  any  issues  of 
concern have been raised and addressed by the Member States.  

The audit plan of DG REGIO for 2017-2018 includes 20 audit missions under the specific enquiry 
on  performance  data  reliability.  On  28  May  2018,  the  management  of  DG  REGIO  discussed  an 
overview of the results from 13 of such audits of reliability of performance data monitoring systems 
carried out so far in nine Member States. Further audits are planned for the remainder of 2018, and 
collaboration with the Audit Authorities (AAs) will continue in order to increase assurance on the 
reliability of the monitoring systems ahead of the 2019 performance review. 

6 

(b) The Commission accepts this recommendation and considers it as being implemented.

Where Commission or AA audits identify weaknesses, the necessary corrective actions are set out 
in the respective audit reports with a corresponding deadline. 

(c) The Commission partially accepts this recommendation.

OP amendments are submitted at the initiative of the Member States but there is no legal deadline to 
submit them. The Commission took the necessary steps to organise itself that the OP amendments 
related to  the  performance  framework  are  processed  in  good  time.  Furthermore,  in  order  to  make 
sure  that  such  OP  amendments  are  submitted  and  processed  in  time,  Member  States  have  been 
invited  to  submit  officially  any  such  request  by  30  June  2018.  The  assessment  of  these  OP 
amendment  requests  follows  a  standard  procedure  in  order  to  ensure  consistent  treatment.  The 
Commission will deal with these OP amendments in the regulatory deadline provided by the CPR.  

7 

Event 

Date 

Adoption of Audit Planning Memorandum (APM) / Start of audit 

18.1.2017 

Official sending of draft report to Commission (or other auditee) 

3.5.2018 

Adoption of the final report after the adversarial procedure 

27.6.2018 

Commission’s (or other auditee’s) official replies received in all 
languages 

23.7.2018 

PDF 

ISBN 978-92-847-0548-1 

doi:10.2865/390387  QJ-AB-18-018-EN-N

HTML 

ISBN 978-92-847-0523-8 

doi:10.2865/439457  QJ-AB-18-018-EN-Q

For the Cohesion policy funds during the 2014–2020 
programme period, we examined how well Member States 
focus on results in selecting projects and how well the 
Commission and Member States can demonstrate, through 
their monitoring, that the EU budget is well spent. We 
found that although the Commission has taken various 
measures to increase results-orientation, selection 
procedures continue to emphasize outputs and absorption 
rather than results. Moreover, shortcomings in the 
monitoring arrangements made it difficult to assess the 
extent to which EU funding has contributed to the 
achievement of the EU’s and Member States’ objectives. 
We recommend that, during the selection process, the 
Member States ensure a comparison of project applications, 
require beneficiaries to identify at least one real result 
indicator for each project to be included in the grant 
agreement and contributing to the OP indicators, and 
assess expected results and indicators in the evaluation 
report of the applications. The Commission should define 
common result indicators for the ERDF, improve reporting 
on performance, and ensure a meaningful performance 
review in 2019.

EUROPEAN COURT OF AUDITORS 
12, rue Alcide De Gasperi 
1615 Luxembourg 
LUXEMBOURG

Tel. +352 4398-1 

Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx 
Website: eca.europa.eu
Twitter: @EUAuditors

© European Union, 2018

For any use or reproduction of photos or other material that is not under the European Union copyright, permission 
must be sought directly from the copyright holders.