Special Report No 12/2013 ‘Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?’
Date: 2013-11-26
Institutions: European Court of Auditors
Subjects: EU budget, evaluation method, expenditure, rural development

EUROPEAN
COURT OF AUDITORS

4
3
8
0
-
1
3
8
1
N
S
S

I

3
1
0
2

2
1
o
N

t
r
o
p
e
R

l

a
i
c
e
p
S

CAN THE COMMISSION AND MEMBER STATES 
SHOW THAT THE EU BUDGET ALLOCATED TO THE 

RURAL DEVELOPMENT POLICY IS WELL SPENT? 

EN

 
 
 
 
Special Report No 12 

 2013

CAN THE COMMISSION AND MEMBER 
STATES SHOW THAT THE  
EU BUDGET ALLOCATED TO THE 
RURAL DEVELOPMENT POLICY IS 
WELL SPENT?

(pursuant to Article 287(4), second subparagraph, TFEU)

EUROPEAN COURT OF AUDITORS

EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
1615 Luxembourg
LUXEMBOURG

Tel. +352 4398-1
Fax +352 4398-46410
E-mail: eca-info@eca.europa.eu
Internet: http://eca.europa.eu

Special Report No 12 

 2013

More information on the European Union is available on the Internet (http://europa.eu).

Cataloguing data can be found at the end of this publication.
Luxembourg: Publications Office of the European Union, 2013

ISBN 978-92-9241-428-3
doi:10.2865/42819

© European Union, 2013
Reproduction is authorised provided the source is acknowledged.

Printed in Luxembourg

3

CONTENTS

Paragraph

GLOSSARY

I–VII 

EXECUTIVE SUMMARY

1–14 

INTRODUCTION

7–14 

 THE RURAL DEVELOPMENT POLICY AND THE COMMON MONITORING 
AND EVALUATION FRAMEWORK (CMEF)

15–18 

AUDIT SCOPE AND APPROACH

19–75 

OBSERVATIONS

19–34 

22–25 

 ARE THERE CLEAR STATEMENTS ON WHAT THE RURAL DEVELOPMENT EXPENDITURE IS INTENDED TO 
ACHIEVE?

 THE EU’S RURAL DEVELOPMENT POLICY DOES NOT QUANTIFY OBJECTIVES OR SET TARGETS: THIS SHOULD BE 
DONE BY THE MEMBER STATES IN THE RDPS …

26–34 

… BUT THE RDPS DID NOT SUFFICIENTLY SET SPECIFIC OBJECTIVES WITH TARGETS FOR THE RESULTS

35–57 

 HAS RELIABLE INFORMATION ON RESULTS BEEN PUBLISHED SHOWING WHAT THE EXPENDITURE HAS 
ACHIEVED AND HOW EFFICIENTLY IT HAS DONE SO?

37–42 

COMMON MONITORING INDICATOR DATA ON RESULTS IS NOT SUFFICIENTLY RELIABLE

43–45 

46–50 

51–57 

 THE MID-TERM EVALUATIONS DID NOT GIVE SUFFICIENTLY EVIDENCE-BASED CONCLUSIONS ON THE EFFECTS 
OF THE RURAL DEVELOPMENT MEASURES

 THE COMMISSION HAS ENCOURAGED MEMBER STATES TO IMPROVE MONITORING AND EVALUATION AND 
THERE IS USEFUL INFORMATION ON WHAT THE EU’S BUDGET IS SPENT ON …

 … BUT THERE HAS BEEN INSUFFICIENT REPORTING OF THE RESULTS ACHIEVED TO DEMONSTRATE THAT THE 
EU’S BUDGET HAS BEEN SPENT WELL

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

 
4
4

58–75 

 HAS MONITORING AND EVALUATION INFORMATION BEEN USED TO IMPROVE THE EFFECTIVENESS AND 
EFFICIENCY OF THE EXPENDITURE?

61–62 

63–64 

65–75 

 THE COMMISSION DID NOT TAKE ACCOUNT OF THE EFFICIENCY OF THE MEASURES WHEN INITIALLY APPROVING 
RDPS

 MEMBER STATES HAVE USED MONITORING AND EVALUATION INFORMATION TO HELP IMPROVE TAKE-UP OF THE 
MEASURES …

 … BUT THERE IS LITTLE EVIDENCE OF THE MEMBER STATES OR THE COMMISSION USING INFORMATION ON 
RESULTS TO IMPROVE THE EFFECTIVENESS OR EFFICIENCY OF THE MEASURES

76–87 

CONCLUSIONS AND RECOMMENDATIONS

ANNEX 

— 

STATUS OF RESULT INDICATOR TARGETS SET IN RDPS AND DATA PROVIDED 
BY THE MEMBER STATES IN THEIR ANNUAL PROGRESS REPORTS FOR 2011

REPLY OF THE COMMISSION

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

 
GLOSSARY

5

Axis: Rural development in the 2007–13 programming period is implemented under three thematic axes, 
which represent coherent groups of rural development measures, and one horizontal axis dedicated to the 
Leader approach (Leader axis).

CMEF: Common Monitoring and Evaluation Framework.

Community Strategic Guidelines: A Council Decision that identifies the European Union’s priorities for rural 
development.

Deadweight: A situation where a subsidised operation would have been wholly or partly undertaken without 
the grant aid.

Diversification measures: Three RDP measures aiming at diversification of the rural economy: ‘diversification 
into non-agricultural activities’; ‘support for the creation and development of microenterprises’; and ‘encour -
agement of tourism activities’.

Effectiveness: Achieving the objectives set.

Efficiency: Achieving the best relationship between effectiveness and costs.

Evaluation: A periodic collection and analysis of evidence to form conclusions on the effectiveness and effi -
ciency of ‘interventions’. This may include consideration of the relevance, coherence and European added-value 
of the policy.

Impacts: Longer term socioeconomic and environmental effects.

Indicator: A measurable proxy for an objective that will give useful information to assess the degree to which 
an objective is met.

(Public) Intervention: An action, such as an ‘operation’, ‘measure’, programme or project carried out by or 
funded by a public authority.

Managing Authority: A national or regional body designated by the Member State to manage a Rural Devel-
opment Programme (RDP).

Measure: An aid scheme for implementing a policy. A measure defines the rules for the ‘operations’ that can 
be financed.

Monitoring: Regular examination of the resources, outputs and results of ‘interventions’.

Objective: A statement of the outputs, results and impacts that an ‘intervention’ intends to achieve.

Operation: A project, contract or other action financed under a  ‘measure’.

Outputs: Something that is produced with the resources allocated to an ‘intervention’.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

6
6

RDP: Rural Development Programme, a programming document prepared by a Member State to plan and 
implement the EU’s rural development policy. An RDP may be prepared at regional or national level and is 
approved by the Commission.

Results: The direct effects or changes that arise due to the ‘intervention’.

Shared Management: A method of implementing the EU budget where the Commission delegates imple -
mentation tasks to the Member States, while retaining final responsibility.

Sound financial management: Managing the EU budget in accordance with the principles of economy, 
efficiency and effectiveness.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

7

EXECUTIVE SUMMARY

I.
For  the  2007–13  programming  period,  the  EU  has 
allocated almost 100  billion euros to achieve its rural 
development objectives. This audit examines the ques-
tion of whether the Commission and Member States 
can show that the EU budget allocated to the rural 
development policy is well spent. A key element for 
this is a system of monitoring and evaluation.

II.
The  system  of  monitoring  and  evaluation  should 
provide information on where spending is, or is not, 
effective and efficient. Such information is needed for 
accountability: to inform the European Parliament, the 
Council and the public of what has been achieved 
with the EU budget and to demonstrate that it has 
been spent efficiently and effectively in accordance 
with the principles of sound financial management. 
Monitoring and evaluation also allows weaknesses to 
be identified and changes to be made both within 
the 7-year programming period and when designing 
future policies, legislation and spending programmes.

III.
In its previous reports, the Court has repeatedly high-
lighted serious weaknesses in the availability and qual-
ity of the monitoring and evaluation information relat-
ing to the spending on rural development. This report 
consolidates and develops these previous findings.

IV.
The Court concludes that the Commission and Mem-
ber States have not sufficiently shown what has been 
achieved in relation to the rural development policy 
objectives and there is a lack of assurance that the EU’s 
budget has been spent well.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

8

EXECUTIVE SUMMARY

V.
Specifically the Court found that:

VII.
The Court therefore recommends:

(a)  The objectives set for rural development expend- 

iture were not sufficiently clear.

(b)  There was insufficient information on and report-
ing  of  the  results  achieved  to  demonstrate  the 
extent to which the objectives set have been met 
and that the EU’s budget has been spent effec-
tively and efficiently.

(c)  The monitoring and evaluation information that 
is available has not sufficiently been used to im-
prove the efficiency and effectiveness of the rural 
development expenditure.

VI.
Improvements to monitoring and evaluation can and 
should be made for the remainder of the current pro-
gramming period, for which the expenditure will be 
concluded in 2015. Moreover, the forthcoming 2014–
20 programming period presents an opportunity for 
the Commission and Member States to make the effec-
tiveness and efficiency of the expenditure much more 
central in measure design and programming decisions 
and subsequently when selecting the operations and 
projects to be financed.

(a)  The Member States and the Commission should 
make use of monitoring and evaluation to increase 
their focus on achieving results for the remainder 
of the current programming period.

(b)  Member States and the Commission should take 
action now to ensure that good quality, compar- 
able information can be obtained from the ex post 
evaluations in 2015 of the 2007–13 programming 
period.

(c)  For the 2014–20 programming period, the Com-
mission should improve the CMEF design to pro-
duce more timely, relevant, reliable and compar- 
able information, in particular on the effective -
ness and efficiency of the measures in achieving 
results.

(d)  Member States and the Commission should ensure 
that the 2014–20 RDPs provide a better foundation 
for  sound  financial  management,  with  specific, 
measurable objectives and plans for monitoring 
and evaluation of the programmes’ efficiency and 
effectiveness.

(e)  Member States should design effectiveness and 
efficiency into the 2014–20 RDP implementation 
systems, with the information gathered for the ap-
plication, selection and payment processes reused 
for monitoring and evaluation.

(f )  Member States and the Commission should im-
prove  their  repor ting  on  the  results  achieved 
with  the  EU  budget  to  enhance  accountability. 
This should include more systematic follow-up of 
evaluation findings.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

INTRODUCTION

9

1. 

2. 

3. 

4. 

The EU has allocated almost 100 billion euros to achieve rural develop-
ment objectives in the 2007–13 programming period. Member States 
have also committed 58  billion  euros of their own resources to co-fi -
nance the rural development programmes.

The European Council has stressed that every euro of the EU budget must 
not only be spent correctly but also spent well. However, the Court’s 
audits have consistently shown that the design and the implementation 
of the key rural development measures do not sufficiently ensure that 
the measures are targeted towards achieving policy objectives in the 
most efficient way. This means that the systems for implementing the 
budget cannot be relied on to ensure that the expenditure is effective 
and efficient. Therefore, it is all the more important that the Member 
States and Commission can show what results have been achieved in 
relation to the rural development policy objectives.

A key element in ensuring that the EU budget is well spent is monitoring 
and evaluation. Monitoring is the regular examination of expenditure, 
outputs and results, which provides up-to -date information on whether 
programmes are progressing as intended. Evaluation is a periodic collec -
tion and analysis of evidence — including the information produced for 
monitoring  — to form conclusions on the effectiveness and efficiency 
of a programme or policy. With information from monitoring and evalu -
ation on where spending is not effective  — or not efficient  — changes 
can be made within the 7-year programming period and when designing 
legislation and programmes for the future.

Such information is also needed for accountability: to inform the Euro-
pean Parliament, Council and the public of what has been achieved with 
the EU budget and to demonstrate that it has been spent well.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

10

5. 

Yet the Court has repeatedly found serious weaknesses in the availability 
and quality of the rural development monitoring and evaluation inform- 
ation. Table 1 provides a summary of the Court’s recent findings related 
to these weaknesses.

TABLE 1

FINDINGS ON MONITORING AND EVALUATION OF THE RURAL 
DEVELOPMENT POLICY IN THE COURT’S SPECIAL AND ANNUAL 
REPORTS

Special Report No 5/2010
Implementation of the 
Leader approach for rural 
development

The EU budget contributed over 2 billion euros to Leader+ and has 
programmed over 5 billion euros for the 2007–13 period. The Com‑
mission has not yet demonstrated the effectiveness or efficiency of 
the expenditure …

Special Report No 7/2011
Is agri‑environment support 
well designed and managed?

Special Report No 8/2012
Targeting of aid for the 
modernisation of agricultural 
holdings

Special Report No 1/2013
Has the EU support to the 
food processing industry 
been effective and efficient in 
adding value to agricultural 
products?

The objectives were too vague to be useful for assessing the extent 
to which they have been achieved. The common monitoring and 
evaluation framework represents progress but produces little 
information on the environmental benefits achieved (22 billion  
euros programmed).

The common monitoring and evaluation framework … does not 
generate relevant data that can be used for monitoring the results 
obtained with the funds spent. The indicators defined do not allow 
progress in the achievement of the EU’s priorities to be monitored, 
and the data were found to be of low reliability and do not allow 
comparisons between Member States (and/or regions) to be drawn 
(11 billion euros programmed).

… the Commission and Member States have an insufficient basis 
to assess the effectiveness of the 9 billion euros allocated to the 
measure when programming the aid for the 2014–20 period. 

Special Report No 6/2013
Have the Member States and 
the Commission achieved 
value for money with the 
measures for diversifying the 
rural economy?

There was a lack of effective monitoring and evaluation of the 
measures so that there is a real risk that the results of the funding 
allocated to the projects will not be known. The EU’s strategic 
priority of job creation was not well targeted and there was a lack 
of active management when it was apparent that the targets set 
would not be met (5 billion euros programmed).

Annual Report 2011, 
Chapter 10

… [Commission Directorates‑General] did not set or report on 
objectives for operational activities relating to economy (the cost of 
inputs) or efficiency … the description of the policy achievements 
provide limited information on results and impacts; they tend to 
focus more on inputs and outputs … reporting of performance 
achieved [was] based on incomplete data, …

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

11

1  Council Decision 
2006/144/EC of 
20 February 2006 on 
Community strategic 
guidelines for rural 
development (programming 
period 2007 to 2013) (OJ L 55, 
25.2.2006, p. 20).

2  Article 80 of Council 
Regulation (EC) No 1698/2005 
of 20 September 2005 
on support for rural 
development by the 
European Agricultural Fund 
for Rural Development 
(EAFRD) (OJ L 277, 21.10.2005, 
p. 1).

6. 

7. 

8. 

9. 

The introduction of the new programming period presents an opportun- 
ity for the Commission and Member States to address these weaknesses 
and put systems in place to provide reliable and timely information on 
the results of the 2014–20 expenditure. With this in mind, in this audit, 
the Court consolidates and develops its previous findings with the aim of 
forming practical recommendations that can contribute to improving the 
sound financial management of the 2014–20 rural development policy.

THE RURAL DEVELOPMENT POLICY AND THE 
COMMON MONITORING AND EVALUATION 
FRAMEWORK (CMEF)

The EU’s rural development policy for 2007 to 2013 is organised in three 
‘thematic axes’. Each axis consists of a number of measures designed 
to achieve the policy objectives (which are shown later in this report 
in Table 2 ). A fourth axis, known as Leader, aims to achieve the same 
objectives through community-led local development strategies.

The EU set the ‘priorities for rural development’ in the Community Stra- 
tegic Guidelines — Council Decision 2006/144/EC1, which put the policy 
in the context of wider EU objectives, notably the Göteborg sustainabil-
ity goals and the Lisbon strategy for growth and jobs. According to this 
decision, the ‘overarching priority’ for the ‘Axis 3’ resources, for example, 
is to create employment opportunities and the conditions for growth.

Member States develop Rural Development Programmes (RDPs) at na-
tional or regional levels. Once approved by the Commission, these pro-
grammes are implemented by the Member States under ‘shared manage-
ment’ arrangements.

10. 

The EU’s rural development regulation established a ‘Common Monitor-
ing and Evaluation Framework’ (CMEF) for the 2007–13 period2. This was 
a new system, introduced in 2007, with common indicators intended to 
capture the progress towards rural development objectives in a way that 
allows each of the RDPs to be compared and the data to be aggregated. 
The CMEF also requires for each RDP an ex ante evaluation, a mid-term 
evaluation in 2010 and an ex post evaluation in 2015, to be carried out 
by independent evaluators.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

12

3  Commission Regulation 
(EC) No 1974/2006 of 
15 December 2006 laying 
down detailed rules 
for the application of 
Council Regulation (EC) 
No 1698/2005 on support 
for rural development by 
the European Agricultural 
Fund for Rural Development 
(EAFRD). The complete list 
of common indicators is 
given in Annex VIII (OJ L 368, 
23.12.2006, p. 15).

4  Handbook on 
Common Monitoring and 
Evaluation Framework. 
Guidance document, 
Directorate-General 
for Agriculture and 
Rural Development, 
September 2006.

5  An assessment of the 
technical assistance costs was 
not within the scope of this 
audit.

11. 

12. 

13. 

14. 

A Commission regulation3 sets more specific requirements and lists com-
mon baseline, output, result and impact indicators. Definitions of the 
indicators, common evaluation questions and methodological guidance 
are given in a non-binding ‘CMEF handbook’ issued by the Commission 4.

Member States were responsible for setting ‘specific, verifiable objec-
tives’ in their RDPs, with targets for the common indicators. As common 
indicators cannot fully capture all the effects of individual RDPs, Member 
States were also required to set a limited number of additional indicators. 
Further responsibilities of Member States are to establish data collec-
tion systems and organise evaluations. They prepare an Annual Progress  
Report  based  on  the  monitoring  and  evaluation  information,  which 
should show the progress of the programme in relation to the objectives 
set. The Member States’ Managing Authorities and Monitoring Commit-
tees are required to use the monitoring and evaluation information to 
help improve the quality of the programmes and their implementation.

The Commission coordinates the CMEF. Commission staff review the An-
nual Progress Reports and aggregate the monitoring data. They may par-
ticipate as observers in the Member States’ Monitoring Committees and 
hold an annual meeting with the Managing Authorities to examine the 
main results of the previous year for each RDP. The Commission contracts 
evaluators to make syntheses of the mid-term and  ex post evaluations, 
and publishes these reports. The Commission is also required to produce 
several reports that concern the implementation and achievements of 
the rural development policy (see paragraph 51).

Monitoring  and  evaluation  has  a cost.  In  the  Member  States,  the  EU 
co-finances ‘technical assistance’ which can include the costs incurred in 
collecting and processing monitoring data (staff, computer systems, etc.) 
and expenditure on studies and evaluations. Just under 2 % of the RDP 
budgets (1,9  billion  euros) have been allocated to technical assistance 
for the 2007–13 period, which also includes other technical activities 
such as checking the correctness of the expenditure. Additional costs 
directly incurred by the Commission include syntheses of the Member 
States’ ex post and mid-term RDP evaluations and the running costs of 
an Evaluation Helpdesk and Evaluation Expert Committee5.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

AUDIT SCOPE AND APPROACH

13

15. 

The Court examined whether the Commission and Member States can 
show that the EU budget allocated to the rural development policy is 
well spent. Three sub-questions structured the audit work:

(a)  Are there clear statements on what the rural development expend- 

iture is intended to achieve?

(b)  Has reliable information been published showing what the expend- 

iture has achieved and how efficiently it has done so?

(c)  Has monitoring and evaluation information been used to improve 

the effectiveness and efficiency of the expenditure?

16. 

The audit primarily concerns the setting of objectives and indicator tar-
gets for the 2007–13 RDPs and the subsequent production, reporting 
and use of monitoring and evaluation information. The audit focused 
on the rural development measures audited by the Court since 2010. 
Together, these represent 53 % of the total rural development expendi-
ture programmed for 2007–13. Table 2  provides a brief description of the 
measures audited by the Court since 2010 with the related objectives 
and result indicators.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

14

TABLE 2

SELECTED RDP MEASURES AND RELATED RESULT INDICATORS

Objectives of the Axes

Selected measures

Description of the measures 

Common result indicators 

AXIS 1: to improve the 
competitiveness of 
agriculture and 
forestry by supporting 
restructuring, develop‑
ment and innovation

Modernisation of agricultural 
holdings
Article 20(b)(i) of Regulation 
(EC) No 1698/2005

Grants for investments in farm machinery 
and equipment, such as tractors, harvest‑
ers, farm buildings, manure storage, 
irrigation facilities, etc. 

Adding value to agricultural 
and forestry products
Article 20(b)(iii) of Regula‑
tion (EC) No 1698/2005

Grants for investments in food processing 
equipment such as for producing olive oil, 
dairies, fruit and vegetable packaging, 
meat processing, etc. 

AXIS 2: to improve the 
environment and the 
countryside by support‑
ing land management

Agri‑ environment payments
Article 36(a)(iv) of Regulation 
(EC) No 1698/2005

Payments to farmers or other land man‑ 
agers who make a voluntary agri‑environ‑
mental commitment for a period between 
5 and 7 years. The payments per hectare 
or per livestock unit cover additional costs 
or income foregone due to the specific 
commitment taken (e.g. organic farming, 
protection of water sources, encouraging 
wildlife, etc.). 

Diversification into non‑
agricultural activities
Article 52(a)(i) of Regulation 
(EC) No 1698/2005

Grants for investments by members of 
a farm household who diversify into non‑
agricultural activities, such as farm shops, 
agri‑tourism or other businesses.

AXIS 3: to improve the 
quality of life in rural 
areas and 
encourage diversifica‑
tion of economic activity

Support for business creation 
and development
Article 52(a)(ii) of Regulation 
(EC) No 1698/2005

Grants for investments by rural micro 
enterprises (fewer than 10 workers and 
less than 2 million euros turnover) or for 
setting up new micro‑enterprises.

Encouragement of tourism 
activities
Article 52(a)(iii) of Regulation 
(EC) No 1698/2005

Grants for investments in tourist accom‑
modation and services, small‑scale tourism 
infrastructure (such as hiking trails and 
signposting of tourist sites) and marketing 
of rural tourism.

AXIS 4: LEADER: 
implementing local 
development strategies 
with a view to achieving 
the objectives of one or 
more of the three other 
axes

Article 63(a) of Regulation 
(EC) No 1698/2005

Support for rural communities to 
implement integrated and innovative 
strategies that contribute to the long‑term 
sustainable development of local areas. 
Leader follows a bottom‑up approach with 
decision‑making power for local action 
groups concerning the elaboration and 
implementation of local development 
strategies. 

‑   Number of holdings introdu‑ 
cing new products and/or 
techniques;

‑   Increase in gross value added in 

supported holdings.

‑   Number of enterprises intro‑
ducing new products and/or 
techniques;

‑   Increase in gross value added in 

supported enterprises.

Areas under successful land 
management contributing to:
•  improvement of biodiversity,
•  improvement of water quality, 
•  mitigating climate change, 
•  improvement of soil quality, 
•   avoidance of marginalisation 

and land abandonment.

‑   Increase in non‑agricultural 

gross value added in supported 
businesses.

‑   Gross number of jobs created 
(division according to gender, 
age and on‑farm/off‑farm jobs).

‑   Gross number of jobs created 
(division according to gender, 
age and on‑farm/off‑farm jobs). 

‑   Increase in non‑agricultural 

gross value added in supported 
businesses.

‑   Additional number of tourist 

visits (division according to the 
number of overnight stays and 
the number of day visitors). 
‑   Gross number of jobs created 
(division according to gender, 
age and on‑farm/off‑farm jobs).

‑   Gross number of jobs created 
(division according to gender, 
age and on‑farm/off‑farm jobs).

‑   Number of participants that 
successfully ended a training 
activity.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

15

17. 

18. 

The work done for this audit, undertaken between January and April 
2013, is concentrated on 12 of the RDPs that were included in the Court’s 
previous audits 6. These RDPs together represent just over 50  % of the  
total rural development expenditure programmed for 2007–13. The Fig-
ure shows the RDPs covered by this audit and their share of the total 
budget allocated to rural development for the 2007–13 programming 
period.

6  The Court excluded other 
RDPs from the scope of this 
audit where it had audited 
only one RDP measure 
(Ireland, Greece, Lithuania 
and Portugal) or where the 
Court had audited several 
RDPs in the same Member 
State.

The Member States concerned and the Commission provided inform- 
ation on the action taken following the Court’s previous audits. In ad-
dition, the Court examined the most recent monitoring and evaluation 
information issued in respect of the selected measures. In most cases, 
this was the 2011 Annual Progress Report finalised in the second half of 
2012. The Court reviewed the Commission’s management of the CMEF, 
its reporting of policy implementation and achievements, its supervisory 
activities in relation to the selected measures and the selected RDPs and 
the use of monitoring and evaluation information when approving RDP 
amendments.

FIGURE

SELECTED RURAL DEVELOPMENT PROGRAMMES: ALLOCATED 
BUDGET AS A PERCENTAGE OF THE TOTAL (2007–13)

Germany - 
Baden - Württemberg  

Spain - Andalucía 

Czech Republic

Other Rural Development
Programmes (not selected) 

France - Mainland
(Hexagone) 

Italy - Campania

Luxembourg

Hungary 

Austria 

Poland

Romania

Sweden

United Kingdom - England 

Source: Directorate-General for Agriculture and Rural Development.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

7  Part I, Title II, Chapter 7 
of Council Regulation (EC, 
Euratom) No 1605/2002 of 
25 June 2002 on the Financial 
Regulation applicable to 
the general budget of the 
European Communities 
(OJ L 248, 16.9.2002 p. 1).

OBSERVATIONS

16

19. 

20. 

21. 

22. 

ARE THERE CLEAR STATEMENTS ON WHAT THE 
RURAL DEVELOPMENT EXPENDITURE IS INTENDED 
TO ACHIEVE?

Objectives communicate what the policy and programmes intend to 
achieve, allowing choices to be made  about allocating  budgets  and 
allowing measures to be designed and implemented with an appro-
priate balance between the expected results and the costs. Objectives 
also provide a reference point against which the results obtained can 
be compared. This allows underachievement to be identified, enabling 
corrective action to be taken and effectiveness and efficiency improved. 
Subsequently, they provide a basis for assessing performance  — for ac-
countability: for holding the managers of EU funds responsible for the 
effective and efficient use of the EU’s budget. For these purposes, objec-
tives should be SMART (Specific, Measurable, Achievable, Relevant and 
Timed), as defined by the EU’s financial regulation7.

In particular, the objectives should describe a  future situation  — the 
change or outcome that the policy or programme intends to achieve. 
They must be expressed in terms that make it possible to check if they 
have been achieved. And they must be quantified, or have relevant indi-
cators with targets, to provide a basis for ensuring that the programmes 
represent good value for money and for managing the measures (se-
lecting operations for funding) in a way that achieves the objectives 
efficiently.

In all of its performance audits of rural development, the Court has as-
sessed whether the objectives set in the legislation and the RDPs were 
specific and measurable. The Court has reviewed the Commission and 
Member States’ responses to the previous findings and has examined 
the amendments subsequently made to the 12 selected RDPs.

THE EU’S RURAL DEVELOPMENT POLICY DOES NOT QUANTIFY 
OBJECTIVES OR SET TARGETS: THIS SHOULD BE DONE BY THE MEMBER 
STATES IN THE RDPS …

The EU’s priorities for rural development are set out in the Community 
Strategic Guidelines, and the rural development regulation sets three 
objectives for the policy (see paragraphs 7 to 8). The regulation describes 
a number of measures that Member States may implement in order to 
achieve these objectives but does not give more specific objectives per 
measure.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

17

8  This also applies to the 
Commission’s proposals for 
the 2014–20 regulations. In its 
Opinion No 1/2012 on certain 
proposals for regulations 
relating to the common 
agricultural policy for the 
period 2014–20, the Court 
found that the proposals 
presented ‘a disparate and 
very wide ranging set of 
objectives … It is not clear … 
what these objectives intend 
to achieve at EU level. The 
Court considers that the rural 
development policy would 
benefit from a less complex 
and a more precise set of 
objectives including clear 
targets (expected results and 
impacts as well as indicators). 
This would help to focus the 
policy on delivering results on 
key EU priorities’.

23. 

24. 

25. 

26. 

27. 

The  objectives  for  rural  development  are  not  quantified  at  EU  level. 
The  objectives are phrased in terms of improving competitiveness and 
encouraging diversification. How much improvement or how much di-
versification the EU intends to be achieved with the EU budget is not 
stated8.

The legislation requires Member States to set ‘specific verifiable objec-
tives and indicators’ in their RDPs that ‘allow the programme’s progress, 
efficiency and effectiveness to be measured’. Before approving RDPs, 
the Commission is required to ensure that they are ‘consistent’ with the 
Community Strategic Guidelines (the EU’s priorities) and the regulation.

Several measures contribute to each EU objective: 16 different measures 
aim at improving the competitiveness of agriculture and forestry, the 
objective of Axis 1. A measure may also contribute to more than one 
objective: equipment financed under the measure ‘modernising agricul-
tural holdings’ (Axis 1), for example, may also reduce the environmental 
impact of farming (objective of Axis 2). A clear plan for the results that 
each measure is intended to achieve is therefore needed if RDPs are to 
provide a reasoned basis for allocating resources in the first place, for 
managing the measure to ensure that it is achieving what is intended, 
and finally for reporting on how effective and efficient the measure has 
been.

… BUT THE RDPS DID NOT SUFFICIENTLY SET SPECIFIC OBJECTIVES 
WITH TARGETS FOR THE RESULTS

As described above, the EU regulation requires Member States to set 
specific objectives according to their different needs established in the 
programming phase. However, with few exceptions, the RDP objectives 
are  phrased  in  general,  open- ended  terms  (to support, to improve, to 
contribute to, to promote, etc.). These describe what the programmes aim 
to do, but leave it uncertain as to what future situation they intend to 
achieve. With such non-specific objectives, it is not possible to ascertain 
whether they have been sufficiently met or not.

This lack of specific objectives in RDPs not only makes monitoring and 
evaluation more difficult, it also contributes to an insufficient focus on 
effectiveness and efficiency by the Member States when implementing 
the measures, as the Court found in its previous audits (see Table 1 ).

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

18

28. 

The targets for the common result and impact indicators required by the 
CMEF should represent a certain quantification and clarification of the 
Member States’ RDP objectives. However, as illustrated in the following 
paragraphs:

•	

•	

•	

the RDPs did not all set targets for the indicators;

the targets set were estimates of expected results rather than quan-
tified objectives;

the indicators (and hence the targets) were not always relevant to 
the RDP objectives.

29. 

30. 

31. 

Romania was the only one of the 12 Member States that set targets in 
their initial RDP for all result indicators for the selected measures accord-
ing to the CMEF requirements. The Commission approved the other RDPs 
despite the missing targets. The initial RDP for Austria, for example, had 
no output or result indicator targets for the agri- environment measure, 
although this represented 45 % of the total programmed expenditure.

Member States calculated the results targets required for the CMEF by 
estimating what they expected the measure to produce with the budget 
allocated. A more logical approach would have been to first identify the 
results required to fulfil a need or achieve an objective, and subsequently 
to design the measure and identify the  extent to  which the need  or 
objective could be met with the available funding.

Some of the RDP targets appear to be over-optimistic estimates of the 
expected results, such as the target in the Romanian RDP of 7,65 million 
additional tourist visits to be achieved by the small-scale investments in 
tourism activities, which was 10 times more than any of the other RDPs 
reviewed. This target was subsequently cut by 5 million.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

19

32. 

33. 

34. 

The common result indicators are not always relevant to the Member 
States’ RDP objectives, so they do not have the effect of quantifying what 
the programmes intend to achieve. For example, in Baden-Württemberg, 
the indicator ‘number of holdings introducing new  products or new 
techniques’ is of only limited relevance to the RDP objectives as it has 
a target of just seven of the 3  000 holdings supported by the measure 
for modernising agricultural holdings. In this case, the RDP established 
additional indicators with targets for the measure that were relevant 
to its objectives, such as a target for the number of farms introducing 
enhanced-welfare pig stalls. However, many RDPs did not use additional 
indicators in this way. This is illustrated by the Czech Republic RDP, in 
which the indicator for ‘new products or techniques’ is not considered 
relevant for the measures ‘modernising agricultural holdings’ and ‘add-
ing value to agricultural products’. No alternative indicators have been 
established for these measures.

The weaknesses in the indicator targets, combined with the lack of spe-
cific objectives described above, have the consequence that the RDPs 
do not clearly set out what they intend to achieve.

With common indicator targets for outputs and results, the Commission 
should be able to compare the planned efficiency of different RDPs (see 
paragraph 61 for more on this) and should be able to aggregate the tar -
gets at EU level to show the intended results of the rural development 
policy. However, some Member States presented targets in a different 
way, such as ‘an increase of X %’ or with a different definition to that in 
the CMEF guidance, which prevents their comparison or aggregation. 
Similarly, where RDP targets are missing, implausible or not comparable, 
the value of the consolidated targets at EU level is limited. The  Annex 
shows the targets that were missing or did not correspond to the CMEF 
requirements.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

20

9  Article 13 of Regulation 
(EC) No 1698/2005.

35. 

36. 

37. 

HAS RELIABLE INFORMATION ON RESULTS BEEN 
PUBLISHED SHOWING WHAT THE EXPENDITURE HAS 
ACHIEVED AND HOW EFFICIENTLY IT HAS DONE SO?

The key monitoring and evaluation information for demonstrating ef-
fectiveness and efficiency is that related to results, as it shows the direct 
effects of the expenditure. It must be accurate, consistent to allow ag-
gregation, and should show the effects of the expenditure in relation 
to the EU and RDP objectives. The information should be reported to 
the budgetary authorities and public in a way that allows the success or 
otherwise of the measures financed to be demonstrated and accounts 
for the effective and efficient use of the EU budget.

The Court has examined the quality of monitoring and evaluation in-
formation in all its recent audits of rural development. Where relevant, 
findings have been updated for the 12 selected RDPs. The Court reviewed 
the Member States’ Annual Progress Reports and ‘Summary reports’9 and 
the Commission’s work in checking the quality of the information pro-
vided. The Court also reviewed the reports issued by the Commission 
(see paragraph 51).

COMMON MONITORING INDICATOR DATA ON RESULTS IS NOT 
SUFFICIENTLY RELIABLE

The CMEF is a valuable tool-set, which is necessary to obtain consistent, 
comparable information at EU level. However, it soon became apparent 
that the CMEF — newly introduced in 2007 — had not been sufficiently 
well prepared. Member States had not all planned in advance how to 
collect data for the common indicators. Many made ad hoc arrangements 
in parallel with their administrative systems (the systems for assessing, 
approving, checking and paying aid applications), leading to additional 
work for the national and regional authorities as well as the beneficiaries. 
This also led to a situation where monitoring information was compiled 
separately, with risks for data integrity.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

21

38. 

39. 

40. 

41. 

Most commonly, Member States took results data from the beneficiaries’ 
forecasts of the expected results, which formed part of their aid applica-
tions, or in some cases, the payment claim. In England and Sweden, the 
authorities checked the plausibility of this data and adjusted it where 
necessary. This allowed them to assess causality: that the job created, 
the increase in ‘gross value added’, the new product or the increase in 
tourism really was an effect of the aid and not just coincidental. Few 
other Member States had such arrangements in place to check the data 
quality.

In Poland, the authorities had accurate information in their administrative 
systems on jobs created, as beneficiaries had to provide evidence of the 
new jobs as a condition for payment of the grant. But this information 
was not used for the CMEF as the authorities used a survey questionnaire 
to the beneficiaries — an inherently less reliable method, for which no 
proof was required or checking of reliability done. In the Czech Republic 
and Hungary, indicator data was also obtained by survey with no checks 
on data accuracy.

The Member States audited tended to see the CMEF data collection as 
an additional task that was necessary to comply with the regulation, but 
which was of little value for their own management of the measures, and 
consequently paid little attention to its quality. Reflecting this attitude, 
several Member States responded to the Court’s findings of weaknesses 
in their monitoring and evaluation by explaining that they had done 
what was necessary to comply with the CMEF requirements. For exam-
ple, following the Court’s finding that they had no plans for monitoring 
or evaluating Leader local development strategies, one Member State 
replied that it had put in place the CMEF as defined at EU level and if 
this did not cover Leader, it was not the fault of the Member State.

As a consequence of the weaknesses described above, the Court has 
found in its recent audits that the CMEF results data frequently does 
not correspond to the real situation. Discrepancies stem from misunder-
standings of what information is required and the difficulty of predict-
ing at the time of the aid application results such as ‘gross value added’, 
which depend on many factors, including the state of the economy. In 
some cases, the design of the measure created incentives for the ap-
plicants to exaggerate the likely results, such as the number of jobs to 
be created, to increase their chances of obtaining a grant.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

22

42. 

43. 

The common result indicators, in particular, left too much room for in-
terpretation, data was costly or difficult to collect and some indicators 
were of limited value, as they did not show the results of the measures, 
as illustrated in Box 1 . On top of this, as shown in paragraph 32, the com-
mon indicators were not always relevant enough or sufficient to show 
whether the RDP objectives were met.

THE MID‑TERM EVALUATIONS DID NOT GIVE SUFFICIENTLY 
EVIDENCE‑BASED CONCLUSIONS ON THE EFFECTS OF THE RURAL 
DEVELOPMENT MEASURES

The Member States’ weaknesses in monitoring results contributed to 
the generally unsatisfactory quality of the mid-term evaluations. The 
regulation requires mid-term evaluations to examine not only the de -
gree of utilisation of resources but also the effectiveness and efficiency 
of the programming of the EU budget, the socioeconomic impact and 
the impact on the Community priorities. For the mid-term evaluations, 
which were undertaken in 2010, many evaluators simply did not have in-
formation available on which to base their evaluation, whether because 
measures had not been running long enough to generate results, or 
because the Member States had not collected relevant and reliable data. 
The mid-term evaluations focused on financial and output data, with 
many producing useful analyses of implementation and of weaknesses 
in the monitoring data. However, they rarely provided useful information 
on the achievement of objectives or efficiency.

BOX 1

EXAMPLES OF DIFFICULTIES EXPERIENCED WITH RESULT INDICATORS

The Commission did not issue guidance on what should be considered as a ‘new product or new technique’ 
in relation to the measure for modernising farms. The Romanian data indicates that 98 % of the investments 
resulted in a new product or technique, compared with 13 % in Poland and 35 % in Hungary. Such a large differ -
ence cannot be explained by differences in the measures as they covered a similar range of investments. Almost 
any investment in new farm buildings, machinery or equipment met the Romanian definition for the indicator.

The result indicator ‘area under successful land management’ is in practice identical to the output indicator ‘area 
under agri- environment support’. This is because Member States consider all areas supported to be ‘success-
ful’ without any assessment of whether an improvement of biodiversity, water quality, etc. was achieved. The 
Court identified cases where the agri- environment contract was not successful, but the area was still included 
in the results data.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

23

10  See paragraphs 75 to 77 
of Special Report No 1/2013 
Has the EU support to the 
food-processing industry 
been effective and efficient in 
adding value to agricultural 
products? 
(http://eca.europa.eu).

11  In its audit of the 
measure for adding value 
to agricultural products, the 
Court found that eight of 
the 23 mid-term evaluation 
reports reviewed gave no 
conclusions on the evaluation 
question ‘To what extent 
have supported investments 
contributed to improving 
the competitiveness of the 
agricultural and forestry 
sector?’ despite this being the 
EU objective for which the aid 
is given.

44. 

In many cases, the mid-term evaluations relied on inadequate meth -
odologies, such as making assumptions or relying on surveys without 
undertaking other procedures or evidence collection to support the 
conclusions10. While case studies can be valuable to illustrate findings or 
help explain data, some evaluators used just a handful of cases to draw 
conclusions for a whole measure or programme. Some mid-term evalu -
ations made no attempt to answer the common evaluation questions 
established in the Commission CMEF guidance 11. In France, all Axis 3 
and Axis 4 measures were excluded from the scope of the mid-term 
evaluation.

45. 

The mid- term evaluations consequently failed to give sufficiently evi-
dence-based answers to the evaluation questions that concerned effi-
ciency and the achievement of objectives, and they were too disparate 
to give meaningful information at EU level.

46. 

47. 

THE COMMISSION HAS ENCOURAGED MEMBER STATES TO IMPROVE 
MONITORING AND EVALUATION AND THERE IS USEFUL INFORMATION 
ON WHAT THE EU’S BUDGET IS SPENT ON …

The Commission recognised the problems with the indicators and the 
evaluations from an early stage, worked with the Member States to im-
prove the guidance and offered support, notably through an ‘Evaluation 
Helpdesk’. Following the mid-term evaluations, the Commission analysed 
the weaknesses and has used the experience gained when planning the 
CMEF for the 2014–20 period.

The  Commission  checks  the  Member  States’  monitoring  data  (which 
is submitted with the Annual Progress Reports) and requires Member 
States to correct obvious errors before it will accept the Annual Pro-
gress Reports as complying with the regulation. The consistency and 
completeness of the data have much improved over the past 5 years, 
but at the time of the audit in April 2013, the Commission had still not 
accepted the Annual Progress Reports for 2011 for France, Hungary and 
England, so the latest information is for 2010. However, there are limits to 
what the Commission can do if Member States do not provide the data. 
Hungary has not provided indicator data for 2011 with the explanation 
that it is still revising its indicators. Important data is also still missing 
for other Member States, including Romania, which has not reported 
any result indicator data for the six measures reviewed. Of the 12 RDPs 
reviewed, only Austria and Sweden have provided the full set of common 
result indicator data for 2011. The Annex shows the data provided by 
the Member States in their Annual Progress Reports for 2011. The lack 
of up-to -date data means it is of little use for its intended purpose of 
monitoring the outputs and results.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

24

48. 

49. 

50. 

51. 

Even where results data is available, the Commission has little assurance 
that it is reliable. Member States do not have to describe the definitions 
used or data collection methods in the Annual Progress Reports and the 
Commission has not checked whether Member States have systems to 
ensure the quality of the data, how each interprets terms like ‘new prod-
ucts’, or which Member States follow the methodological guidance. The 
Court found that without explicitly informing the Commission, Hungary 
changed the description of the ‘jobs created’ indicator to include ‘jobs 
maintained’, which would greatly inflate the results.

For all the problems encountered, the CMEF nevertheless remains an 
essential tool, without which the possibilities for monitoring and evalu- 
ating the EU’s budget would be poorer. While the monitoring information 
in the early years and many of the mid-term evaluations were not of an 
acceptable quality, the Commission has noted a development of exper -
tise and of an evaluation culture. Valuable lessons have been learned, 
which should lead to a more robust CMEF in the future. The Court has 
noted examples of good practice in monitoring and evaluation, for ex-
ample in Austria, Sweden and England for the agri-environment measure 
(see paragraph 65), which show what can be achieved.

For the output indicators, the CMEF data is relatively consistent and 
complete and can provide useful information on what the EU’s budget is 
spent on. Regarding the results achieved, however, the monitoring and 
evaluation has not yet delivered sufficiently relevant, accurate, consist-
ent and comparable information.

… BUT THERE HAS BEEN INSUFFICIENT REPORTING OF THE  RESULTS 
ACHIEVED TO DEMONSTRATE THAT THE EU’S BUDGET HAS BEEN 
SPENT WELL

The regulations require the Member States and Commission to report 
periodically on the implementation of the budget and the achievement 
of rural development objectives. Table 3  provides an overview of the 
reports produced by the Member States and Commission that concern 
the efficiency and effectiveness of the rural development expenditure.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

25

TABLE 3

REPORTING ON THE EFFICIENCY AND EFFECTIVENESS OF THE RURAL DEVELOPMENT 
EXPENDITURE

Report

Member States

Annual Progress Report for 
each RDP
(Article 82 of 
Regulation(EC)1698/2005)

Purpose of the report

Comments

To show the financial implementation of the 
programme; the progress of the programme in 
relation to the objectives set; the steps taken 
to ensure the quality and effectiveness of 
programme implementation

The reports present the common indicators, describe the 
physical and financial implementation of the programme and 
give some information on evaluation activities. 

Strategic Monitoring – Sum‑
mary report
(Article 13 of 
Regulation (EC) 1698/2005)

To show the progress made in implementing 
the national strategy plan and objectives and 
the contribution to the achievement of the 
Community strategic guidelines

The reports have similar content to the Annual Progress 
Reports.

Commission

Report on the evaluation of 
the Union’s finances
COM(2012) 40 final
COM(2012) 675 final

Summary report on the im‑
plementation of the national 
strategies
(COM(2011) 450 final
At the time of the audit 
(April 2013) the Summary 
Report due at the start of 
2013 had not been published 
by the Commission 

Annual Activity Report, 
Directorate‑General for 
Agriculture and Rural 
Development

To provide the European Parliament and Council 
with evidence of the results achieved
(NB: not only for the rural development policy)

A summary of the main current developments, 
trends and challenges relating to the imple‑
mentation of the national strategy plans and 
the Community strategic guidelines, including 
the measures taken to follow up the report’s 
conclusions

To show results of DG Agriculture and Rural 
Development ’s operations in relation to the 
objectives set

Activity statements of oper‑ 
ational expenditure: Working 
Document accompanying the 
Draft General Budget

To justify the appropriations requested by the 
Commission in the Draft Budget

The reports present the evaluations that were undertaken in 
the year. Rural development was not covered as there were 
no EU‑level evaluations published. The Court has commented 
on these reports in Court Opinion 04/2012 and in a presiden‑
tial letter available at: http://eca.europa.eu/portal/pls/portal/
docs/1/19934783.PDF

The report presents data on financial and physical implemen‑
tation aggregated from the Member States’ Annual Progress 
Reports with just a few of the results indicators included. 
There is no analysis of whether the achievement of results is 
satisfactory or of the costs involved.

The reports present impact indicators to show achievement 
of the Commission’s objectives (which differ in some respects 
from those of the rural development regulation). For 5 of the 
6 indicators the data for 2012 is unchanged from 2011, which 
the Court found to be unreliable (see Chapter 10 of the An‑
nual Report 2011). The report’s annexes include selected find‑
ings from external evaluations and result indicators compiled 
from the Member States’ Annual Progress Reports.

The statement for the 2013 budget describes ‘major achieve‑
ments’. These relate to the Commission’s activities. No link 
is made between the activities and the information shown 
on results, which is an aggregation of the common indicator 
data.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

26

12  In Chapter 10 of its 
2011 Annual Report, the 
Court found that the 
2011 Directorate-General 
for Agriculture and Rural 
Development Annual Activity 
Report presentation of the 
impact of the programme 
was based on insufficient 
data. For example, the 
increase in renewable energy 
was calculated from data 
for just 9 of the 88 RDPs 
(OJ C 344, 12.11.2012).

13  The Commission publishes 
the external evaluations, 
including the ex post 
evaluation and mid-term 
evaluation synthesis, with 
a caveat to the effect that the 
conclusions are those of the 
evaluator and not necessarily 
shared by the Commission.

52. 

53. 

54. 

These reports present the available indicator data, and in general give 
a sufficient description of the implementation of the programmes in 
terms of the amount spent and the outputs. However, the reports display 
common weaknesses, described in the following paragraphs.

The reports give insufficient consideration to the reliability of the data 
sources or methods when presenting the information. For example, some 
Member States extrapolated data from just a handful of cases, as did the 
Commission in its Annual Activity Report12. The poor quality of many of 
the mid-term evaluations also limits the reliance that can be placed on 
the evaluation findings but the Member States and Commission did not 
sufficiently draw attention to this.

The Member State and Commission reports show indicator values in 
relation to targets that the Member States have changed to match the 
expected results. Box 2  explains why this should not be done. In a re -
cent example the Commission approved an amendment to the Czech 
Republic RDP that reduces the target for job creation by over 90 % (from 
29 000 to 2 700). If the programme eventually creates 2 700 jobs it will 
meet just 9 % of the target set in the approved RDP  — a poor result. 
Yet the Annual Progress Report will compare it with the revised target 
and consider it a 100  % success. The Managing Authority will not have 
to explain the failure to meet the original target. This practice devalues 
the reports to the point that they are of little use for accountability: for 
showing the extent to which the programme managers have achieved 
the rural development objectives set. The Commission reports include 
a  footnote to say that the targets are ‘subject to revision’ but the Mem-
ber States’ Annual Progress Reports reviewed by the Court do not even 
mention when targets have been reduced. The Annex shows the result 
indicator targets that have been changed since the initial RDP.

55. 

The evaluation findings presented in the Commission and Member State 
reports are mostly the work of independent evaluators. However, the 
reports do not inform the reader of the conclusions the Commission or 
Member State themselves draw from the evaluations or if the Commis-
sion or Member State accepts the evaluation findings as valid13.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

56. 

57. 

Moreover, Member States and the Commission reported only selected 
evaluation findings  — in some cases only the positive findings. For ex -
ample, several mid-term evaluations showed deadweight for the invest -
ment measures of around 50  %14 — a conclusion that is important for 
the effectiveness and efficiency of the programmes  — but this was not 
reported in the Member States’ Annual Progress Reports. The reports also 
do not systematically show the action taken in response to the evalu- 
ation findings.

Finally, the Member State and Commission reports do little more than 
present the available information and give some analysis of implementa-
tion. They do not include any analysis of the efficiency of the expenditure 
or give conclusions on the extent to which the RDP and EU objectives 
have been met. As such, the reports give little assurance as to how well 
the Member States and Commission have spent the EU budget15.

27

14  See paragraph 61 of 
Special Report No 1/2013 
‘Has the EU support to the 
food-processing industry 
been effective and efficient in 
adding value to agricultural 
products?’

15  Directorate-General 
for Agriculture and 
Rural Development’s 
Annual Activity Report 
has a ‘Conclusion on the 
assurance to be taken 
with regard to shared 
management of EAGF and 
EAFRD expenditure’. This 
concerns only the legality and 
regularity of the payments. 
There is no equivalent 
conclusion regarding the 
results achieved. A section 
on Sound Financial 
Management considers only 
implementation issues such 
as delays in payments.

BOX 2

THE USE OF INDICATOR TARGETS

The Member States should actively manage the measures to ensure that result targets (and hence the EU’s 
objectives) are met, for example by adjusting eligibility and selection criteria, thus improving the ‘targeting’ 
of the measures concerned. Where the targets still cannot be met, despite the Member States’ best efforts, 
the Member States should explain in the Annual Progress Reports why they were not met and the action that 
they have taken to try to achieve them. Targets should not simply be reduced to reflect the latest projection or 
a ‘revised estimate’ of the results, as this would give the false impression that the measure had been successful 
when this was not actually the case.

Reducing results targets represents a reduction in the planned efficiency and effectiveness of the EU budget, as 
less is being achieved for the same amount of money. For this reason, targets should only be reduced exception-
ally and in clearly justified cases, with the Commission’s approval. Where, exceptionally, targets are changed, 
this should be clearly explained whenever the indicator data is reported in relation to the target.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

28

58. 

59. 

60. 

HAS MONITORING AND EVALUATION INFORMATION 
BEEN USED TO IMPROVE THE EFFECTIVENESS AND 
EFFICIENCY OF THE EXPENDITURE?

The previous section examined whether the monitoring and evaluation 
information met the requirements for accountability: to demonstrate 
what has been achieved. But the purpose of monitoring and evaluation 
is not only to provide information for reporting: it should also be used 
to ensure that the rural development budget is well spent.

Monitoring and evaluation should allow the Commission to learn lessons 
from the current and previous programming periods and apply this when 
preparing proposals for new regulations. It should also give the Member 
States a sound evidence base on which to develop their RDPs, ensuring 
that the most effective and efficient measures are programmed. Subse-
quently, the information should alert decision-makers when objectives 
are not being reached or when the efficiency of the measures is not 
satisfactory, allowing them to improve the design or the implementation 
of the measures during the programming period.

Member States are required to report in the Annual Progress Report on 
the steps taken to ensure the quality and effectiveness of programme 
implementation, and to review progress annually with the Commission. 
For the 12 selected RDPs, the Court examined Annual Progress Reports 
and the annual meeting documents for evidence of action taken to im-
prove effectiveness and efficiency. The Court also reviewed documenta -
tion of amendments to the 12 programmes to determine whether they 
were based on monitoring and evaluation information stemming from 
the CMEF. Finally, the Court reviewed the information presented by the 
Commission in support of its proposals for the 2014–20 programming 
period.

THE COMMISSION DID NOT TAKE ACCOUNT OF THE EFFICIENCY OF 
THE MEASURES WHEN INITIALLY APPROVING RDPS

61. 

Where Member States give targets in their RDPs, these can be analysed 
to indicate whether the programmes represent good value for money 
in relation to the EU’s priorities and objectives. Variations between RDPs 
are to be expected, but the Commission should examine any large dif-
ferences to ensure that they are justified. Examples of large differences 
are shown in Box 3.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

29

For example, while there may be specific national priorities justifying the 
wide variations, the diversification measures in RDPs such as Campania 
and the Czech Republic appear at first sight to represent a poor use 
of the EU budget in relation to the EU’s priorities. Such cases deserve 
analysis and explanation but the Commission’s assessment of RDP pro-
posals did not include such analysis of the efficiency of the measures. 
Consequently, cases such as these are not given appropriate attention 
to ensure that they are effective and efficient means of achieving the 
EU’s objectives.

MEMBER STATES HAVE USED MONITORING AND EVALUATION 
INFORMATION TO HELP IMPROVE TAKE‑UP OF THE MEASURES …

Clearly, if the EU budget is not spent, it cannot have any effect. Informa-
tion on take-up is therefore important to alert programme managers that 
they may need to revise publicity arrangements, eligibility conditions, 
aid rates or selection criteria to make the measures effective. The Court 
found that the Member States were attentive to this, using data from 
administrative sources to monitor take-up closely in relation to their 
budgets. Member States also used evaluation information, particularly 
the mid-term evaluation, to identify reasons for low take -up.

62. 

63. 

BOX 3

EXAMPLES OF VARIATIONS BETWEEN RDPS HIGHLIGHTED BY ANALYSIS OF INDICATOR 
TARGETS

The result indicator for the measure for diversifying agricultural activities is the ‘increase in non-agricultural 
gross value added’ in the farms supported, which can also indicate progress in relation to the EU’s priority of 
creating the conditions for growth. The Czech Republic RDP target shows that for every one euro of additional 
annual ‘gross value added’ to be created, public support of 65 euros is needed. This compares with 3,2 euros of 
support in Hungary and 0,4 euros in Austria.

In relation to the EU’s priority of creating jobs, the result indicator targets for all measures combined show a cost 
in public support per job created ranging from around 10 000  euros (Andalucía, Czech Republic, England, Ro-
mania and Sweden) up to around 90 000  euros (Hungary and Luxembourg). The value for one RDP (Campania: 
203 000 euros) lies far outside this range.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

30

The regulation and the CMEF provide a good basis for the Commission’s 
supervision of the implementation of the Member States’ programmes 
in financial and output terms. For most of the RDPs reviewed, the Com-
mission has approved one or more amendments that aimed to improve 
take-up of the measures.

… BUT THERE IS LITTLE EVIDENCE OF THE MEMBER STATES OR THE 
COMMISSION USING INFORMATION ON RESULTS TO IMPROVE THE 
EFFECTIVENESS OR EFFICIENCY OF THE MEASURES

Improving take-up does not guarantee that the RDP measures will be 
effective: that the intended results will be achieved. Timely, reliable and 
relevant information on results achieved is therefore also necessary. The 
examples in Box 4 illustrate how strong initial targeting combined with 
monitoring and evaluation of results can be used to enhance the effec-
tiveness and efficiency of the measures.

64. 

65. 

BOX 4

USE OF MONITORING AND EVALUATION TO ENHANCE THE EFFECTIVENESS AND 
EFFICIENCY OF THE MEASURES (RDP FOR ENGLAND)

For the agri-environment measure, the authorities analysed which activities were most cost-effective in achieving 
environmental benefits in different geographical areas and prioritised applications on that basis. The agreements 
with the farmers included verifiable objectives (‘indicators of success’) such as to sow plots with seed mixes 
that provide sustained supply for wild birds throughout the winter until 15 February. The authorities monitored 
the achievement of these objectives by visiting a sample of farms each year. Evaluation of the results led the 
authorities to change the prioritisation to improve the targeting and to identify the most cost-effective activ- 
ities to support in the future.

The managing authority set a target for the ‘cost of public support per job created’ and used this as one of 
the criteria for assessing grant applications for the diversification measures and for monitoring progress. The 
mid-term evaluation then analysed the actual results to identify which measures and types of projects were 
most efficient. For example, projects under the microenterprises measure were considerably more efficient at 
creating jobs than farm diversification projects. Following a review, support for the farm diversification measure 
has been reduced.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

31

66. 

67. 

68. 

MEMBER STATES DID NOT JUSTIFY RDP AMENDMENTS BY REFERENCE TO RESULTS DATA

All  12  of  the  selected  RDPs  had  been  amended  at  least  once  and  in 
some cases nine or ten times. The Court searched for examples where 
the Member State justified the changes by reference to the CMEF results 
data. Thus, the Court searched for justifications such as ‘farmers have not 
created enough new products to meet the RDP objective’ or ‘changes 
to the RDP are needed to achieve the jobs target’. No such cases were 
identified. The justifications given were primarily budgetary: the amend-
ments were made because expenditure was higher or lower than forecast 
rather than because the results achieved differed from those intended.

If the RDP expenditure is lower than the Member States planned, after 
2 years the EU allocation to the programme is reduced accordingly in 
a procedure known as ‘decommitment’. This tends to be seen by Member 
States as ‘losing’ the EU funds. To avoid this, Member States amended 
RDPs to reallocate funds from measures with low take-up towards meas-
ures with higher demand. For example, the justification put forward for 
changes to the Andalucía RDP included ‘the need to take corrective ac-
tions at this moment in order to avoid possible decommitments in the 
future’.

In many cases, the approved changes had the effect of reducing rather 
than increasing the planned efficiency of the measures. Andalucía in-
creased  the  budget  for  the ‘microenterprises’  measure  by  194 %  but 
decreased the targets for the outputs (number of microenterprises sup-
ported) by 82 % and job creation results by 96 %. Campania amended 
the RDP to increase the aid rate to 100 % for two of the diversification 
measures in a bid to increase take -up: applicants were reimbursed the 
full costs of their investment projects without any additional obligations 
to achieve results and without having to contribute. The job creation 
target for the farm diversification measure was reduced from 250 in the 
initial RDP to 95.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

32

69. 

70. 

71. 

THE COMMISSION FOCUSED ON OUTPUTS AND FINANCIAL IMPLEMENTATION RATHER 
THAN ON ACHIEVING RESULTS

Even where results indicator data showed that targets would not be 
met, the Commission’s comments on proposed RDP amendments and 
the annual meetings with the Member States focused on outputs and 
spending the budget rather than on how to achieve the intended results. 
The annual meetings did not achieve their purpose according to the 
regulation, which is to examine the main results of the previous year.

16  COM(2011) 450 final of 
20 July 2011.

This same focus on spending the budget and disregard for information 
on the results achieved can be seen in the Commission reports. In the 
Summary Report to the European Parliament and Council on the imple-
mentation of the national strategies16, the Commission commented on 
the measure for modernising agricultural holdings that in some Mem-
ber States ‘the implementation rate seems to be faster than expected, 
requiring the need to increase the estimated budget’. The Commission 
did not explain why ‘faster implementation’ should justify an increase in 
the budget. Results data presented in the same report — such as the 
results for the same measure showing only 10  % of the target for new 
products/techniques being achieved — was not analysed or commented 
on in the report by the Commission.

In DG Agriculture and Rural Development’s Annual Activity Report for 
2011, the cumulative results data presented in an annex shows that four 
years into the programming period only 4 % of the target for creating 
jobs had been achieved. There is no mention in the document or in the 
following DG’s Management Plan of any planned activity by the Com-
mission to improve this situation, despite the importance given in the 
EU’s strategy to the priority of creating jobs.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

33

17  Proposal for a Regulation of 
the European Parliament and 
of the Council on support 
for rural development by 
the European Agricultural 
Fund for Rural Development 
(EAFRD), COM(2011) 627 
final/2 of 19 October 2011.

18  Synthesis of Ex ante 
Evaluations of Rural 
Development Programmes 
2007–13, Final Report, DG 
Agriculture and Rural 
Development, 11.12.2008.

72. 

73. 

74. 

THE  COMMISSION  DID  NOT  USE  CMEF  RESULTS  INFORMATION TO  SUPPORT  ITS 
PROPOSALS FOR THE 2014–20 PROGRAMMING PERIOD

In October 2011, the Commission issued the proposals for the 2014–20 
programming period17. The explanatory memorandum presents the re-
sults of the impact assessment, which is intended to give the European 
Parliament and Council information relevant to their decisions. It de-
scribes the needs that the rural development policy should address, but 
does not show that the proposed measures are effective and efficient 
means of meeting those needs.

Annex 4 of the impact assessment gives an ‘assessment of the current 
policy framework’ in which the Commission presents results targets for 
just seven of the 40 RDP measures. No data is shown on what has been 
achieved in relation to the targets. In Annex  4a, the Commission gives 
a brief account of the strengths and weaknesses of the measures for 
modernising agricultural holdings, agri-environment, forestry measures, 
microenterprises,  and  Leader  but  gives  no  information  on  the  other 
measures proposed for the 2014–20 period, including financially im-
portant measures, such as that for ‘adding value to agricultural products’ 
(5,6 billion euros EU allocation in 2007–13). This annex presents financial 
and output data for all the measures but no information on the results 
achieved. Annex 11 has an overview of evaluation findings, which shows 
that the impact assessment is based largely on the ex ante evaluations 
of the 2007–13 RDPs, of which the synthesis was published in 200818.

This illustrates the weaknesses in the timing and availability of moni-
toring and evaluation information. The Commission made its proposals 
before the mid-term and  ex post evaluation syntheses were available, 
so the proposals were made not only without information on the effec-
tiveness and efficiency of the measures in the 2007–13 period, but also 
without the ex post evaluation of the 2000–06 period. Although these 
evaluations have weaknesses, they nevertheless should be taken into 
account when deciding on the 2014–20 policy. For example, the ex post 
evaluation concludes that the effects of the measure ‘adding value to 
agricultural products’ on the competitiveness of agriculture seemed to 
be ‘generally rather marginal’. The mid-term evaluations reported a high 
incidence of deadweight (see paragraph  56). Yet the Commission pro-
posed this measure for the 2014–20 period without providing inform- 
ation to the European Parliament and Council on the deadweight issue 
or what conditions should be established for the measure to ensure that 
deadweight can be minimised.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

34

75. 

When making choices on future policy at EU level, and for program-
ming at national and regional level, information should be available to 
answer basic but important questions such as ‘has this measure proved 
effective at achieving the objectives in the past?’; ‘is this measure the 
most efficient means of achieving the objective?’ and ‘what lessons can 
be drawn from the past implementation of this measure to improve its 
effectiveness in the future?’. Yet the results data that should show the 
effectiveness and efficiency of the measures was not available in time for 
the impact assessment; information subsequently published, in annexes 
to DG Agriculture and Rural Development’s Annual Activity Report, for 
example, is not sufficiently relevant, reliable or complete. Ultimately, 
the Commission has failed to provide the information needed to inform 
decisions on the most effective and efficient measures for the 2014–20 
period.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

CONCLUSIONS AND RECOMMENDATIONS

35

76. 

77. 

78. 

79. 

The Treaty and the EU’s Financial Regulation require Member States to 
cooperate with the Commission so that EU funds are used in accordance 
with the principles of Sound Financial Management, which are effec-
tiveness, efficiency and economy. To do this in the context of the rural 
development policy, the Commission and Member States need timely 
information on whether the rural development measures are achieving 
the policy objectives in an efficient way. They should account for their 
performance by reporting this information to the European Parliament, 
Council and to the EU public.

The Court concludes that the Commission and Member States have not 
sufficiently shown what has been achieved in relation to the rural devel-
opment policy objectives and there is a lack of assurance that the EU’s 
budget for rural development has been spent well.

The Common Monitoring and Evaluation Framework (CMEF) is the main 
instrument put in place for providing the necessary information and as-
surance. Its weaknesses contributed to the lack of assurance described 
above. The CMEF is a set of tools, whose design can (and should) be 
improved. But better tools will not solve the problem unless they are 
properly used. The Court found that the present lack of accountability is 
attributable to a more fundamental weakness: programmes that have fo -
cused too much on spending the budget and not enough on efficiently 
achieving results. This has led to a lack of attention to the monitoring 
and evaluation of results and impacts and consequently a lack of con -
sideration for the quality of the information.

The Court found that with general, open-ended objectives and insuf-
ficient quantification of the intended results in relation to those objec-
tives, the RDPs drawn up by the Member States did not provide a sound 
basis for allocating the EU’s budget in an effective and efficient way. Nor 
did they target the spending to where it would have the most effect, 
or provide a standard against which achievements could be measured 
(paragraphs 19 to 34).

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

36

80. 

81. 

82. 

Secondly, there is an information deficit: despite support from the Com-
mission, Member States have not produced monitoring and evaluation 
information that is reliable, consistent and relevant enough to show the 
results achieved in relation to the objectives set. The Member States and 
the Commission reported the common indicator data that was avail-
able but did not interpret the data in relation to the RDP or the EU’s 
objectives. Evaluation conclusions were selectively quoted and not sys-
tematically followed up. The Commission and Member States did not 
present their own conclusions on the efficiency or effectiveness of the 
expenditure (paragraphs 35 to 57).

Thirdly, imperfect though the available monitoring and evaluation infor-
mation may be, the Commission and Member States have not made suf-
ficient use of it to improve the efficiency and effectiveness of the meas -
ures. In the day-to -day management of the programmes, the Member 
States have focused on spending the budget more than on achieving 
the intended results. The Commission supported its proposals for the 
2014–20 programming period with information from the ex ante evalu-
ations of the RDPs for the current period, without presenting any infor-
mation on the results achieved (paragraphs 58 to 75).

The Commission’s proposals for 2014–20 promise an increased focus on 
results. This increased focus does not require new legislation; it can and 
should start now to ensure that the EU’s budget is spent well, particularly 
as the end of the 2007–13 programming period approaches and the risk 
increases that the remaining budget is spent hastily to meet the 2015 
cut-off date for payments. In view of this, the Court recommends:

RECOMMENDATION 1

The Member States and the Commission should make use of monitor-
ing and evaluation to increase their focus on achieving results for the 
remainder of the current programming period.

Member States should justify all proposed changes to RDP measures, 
and explain their intended effects, in terms of their effectiveness and 
efficiency in achieving results in relation to the RDP and the EU’s ob-
jectives. The Commission should assess the proposed changes in the 
same terms. The annual examination of programmes should focus on 
results, as required by the regulation.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

37

83. 

The 2007–13 mid-term evaluations suffered from a lack of reliable data, 
poor evaluation methods and a lack of consistency that seriously limited 
their usefulness and hindered the possibilities to draw valid conclusions 
at EU level. The latest available result monitoring data is still inconsistent 
and of unsatisfactory quality. As a consequence, there is a risk that the 
ex post evaluations in 2015 may not be of sufficient quality.

RECOMMENDATION 2

Member States and the Commission should take action now to ensure 
that good quality, comparable information can be obtained from the 
ex post evaluations in 2015 of the 2007–13 programming period.

(i) 

Drawing on the lessons of the mid-term evaluations, the Commis-
sion should issue specific guidance aimed at ensuring the quality 
and consistency of the ex post evaluations.

(ii)  Member  States  should  plan  how  to  provide  the  information 
needed for ex post evaluation, including the wealth of informa-
tion gathered for administrative purposes. Member States should 
ensure that the evaluation contracts include quality criteria re-
quiring evaluators to follow the CMEF guidance and to answer the 
common evaluation questions based on sound data and evalu- 
ation methods.

(iii)  The Commission should be proactive by checking whether Mem-
ber States are following the guidance for the common indicators 
and the ex post evaluations. Where this is not the case, the Com-
mission should work with the Member States involved to identify 
solutions that allow good quality, comparable information to be 
obtained.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

38

84. 

Weaknesses in the design of the 2007–13 CMEF contributed to the cur-
rent lack of timely, reliable and consistent information, for results data, 
in particular. The Commission has already made some improvements 
to indicator definitions and guidance but this does not go far enough. 
Targets for results, with regular monitoring of achievements, are a key 
element in sound financial management. However, the Commission’s 
current plans for the 2014–20 CMEF are that targets would not be re-
quired for results indicators.

RECOMMENDATION 3

For the 2014–20 programming period, the Commission should improve 
the CMEF design to produce more timely, relevant, reliable and com-
parable information, in particular on the effectiveness and efficiency 
of each measure in achieving results.

(i) 

The purpose of each of the common indicators and its intended 
use at EU level should be clear. Result indicators should show the 
direct effects of the expenditure  — what has been achieved  — 
and should be set for each of the EU’s priorities and objectives. 
Data should be collected for each measure that contributes to 
those priorities and objectives such that the effectiveness and 
efficiency of specific measures can be assessed.

(ii) 

The Member States and Commission should check the quality of 
the common indicator data and provide assurance as to its reli-
ability.

(iii)  Regarding the objectives specific to RDPs, the Member States 
should be obliged to account in a timely manner for the outputs, 
results  and  impacts achieved (see  Recommendation 6) rather 
than to produce specific indicator data or evaluations.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

39

85. 

The Court found that the 2007–13 RDPs did not set specific, measurable 
objectives, which led to poor targeting of the measures as well as mak-
ing monitoring and evaluation difficult. The 2014–20 RDPs present an 
opportunity to put the spending of the EU budget on a sounder footing.

RECOMMENDATION 4

Member States and the Commission should ensure that the 2014–20 
RDPs provide a better foundation for sound financial management, with 
specific, measurable objectives and plans for monitoring and evalu- 
ation of the programmes’ efficiency and effectiveness.

(i) 

(ii) 

In the 2014-20 RDPs, Member States should set specific objectives 
that are measurable, with targets for the results. The chosen RDP 
measures and the intended results should be justified by refer-
ence to the effectiveness and efficiency of the related measures 
in 2007–13.

The RDPs should specify for each of the measures what inform- 
ation will be required to show the output, results and impacts. 
Where the common indicators are not sufficient, additional indi-
cators or evaluation work should be specified. For each element 
of information needed, the RDPs should show the source, how it 
will be collected (with due regard to the administrative burden), 
and how the reliability will be assured.

(iii)  The Commission should check RDPs not just for compliance with 
the regulation but also for the planned effectiveness and effi-
ciency: the Commission should ensure that RDPs provide a suf -
ficient basis for sound financial management by assessing the 
intended results in relation to the budgeted costs.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

40

86. 

The Court has found in its rural development audits that many Member 
States did not sufficiently target the RDP measures through eligibility 
conditions and selection criteria, so there was little ex ante assurance 
that the measures would be efficient and effective. This increases the 
need to obtain ex post assurance from monitoring and evaluation, with 
the associated costs and administrative burden of data collection.

RECOMMENDATION 5

Member  States  should  design  ef fectiveness  and  ef ficiency  into  the 
2014–20 RDP implementation systems, with the information gathered 
for the application, selection and payment processes reused for moni-
toring and evaluation.

Member States should make an ex ante appraisal of effectiveness and 
efficiency not only at the time of programming, but also when select-
ing operations  for funding. This approach reduces the risk of wast-
ing money on inef fective projects that could have been eliminated 
in project selection. The information collected for project selection 
or payment (job creation, environmental benefits targeted, etc.) can 
reduce the need for subsequent data collection and thereby reduce 
the administrative burden.

87. 

While the Member States and Commission published the reports by the 
independent evaluators, in their own annual and summary reports they 
did not sufficiently account for how well they have spent the EU budget. 
In these reports, the Member States and Commission did not use the 
financial and output indicator data to explain the results achieved and 
did not present information on efficiency. It was left to the readers of 
the Member States’ and Commission’s reports to make their own judge -
ments on how much the evaluations could be relied on, or to find out 
if some evaluation conclusions had been omitted and why.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

41

RECOMMENDATION 6

Member States and the Commission should improve their reporting 
on the results achieved with the EU budget to enhance accountability. 
This should include a more systematic follow-up of evaluation findings.

(i) 

From the mid-point of the programming cycle onwards, Mem-
ber States and the Commission should present conclusions in 
their  annual  reports  on  what  the  programmes  have  achieved 
in relation to each of the RDP and EU objectives. These reports 
should have a primary focus on the results achieved and should 
explicitly consider the efficiency of each measure, with the re-
sults compared to the costs. Where the common indicators do 
not sufficiently allow achievement of the RDP objectives to be 
shown, they should be complemented by administrative data 
and evaluation findings, taking due account of the limits of their 
scope and reliability.

(ii) 

In their reporting of evaluation findings, Member States and the 
Commission should make clear what action they are taking or 
explain the reason if no action is planned. All evaluation conclu -
sions should be presented and followed up, not just a selection.

(iii)  To ensure that the full value is obtained from the ex post evalu- 
ations of the 2007–13 programmes, the Commission and Member 
States should include in their annual reports for the year 2015 
an analysis of the implications of the evaluation findings for the 
2014–20 period.

This Report was adopted by Chamber I, headed by Mr Ioannis SARMAS, Member of 
the Court of Auditors, in Luxembourg at its meeting of 9 October 2013.

For the Court of Auditors

Vítor Manuel da SILVA CALDEIRA
President

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

42

ANNEX 

STATUS OF RESULT INDICATOR TARGETS SET IN RDPS AND DATA PROVIDED BY THE 
MEMBER STATES IN THEIR ANNUAL PROGRESS REPORTS FOR 2011

Measure

Modernisation of agricultural holdings

Adding value to agricultural and forestry 
products

Agri-environment

Increase in gross value 
added in supported 
holdings/enterprises

Number of holdings/
enterprises introducing 
new products and/or 
new techniques

Increase in gross value 
added in supported 
holdings/enterprises

Number of holdings/
enterprises introducing 
new products and/or 
new techniques

Area under successful 
land management1

Diversification into non-agricultural activities

Business creation and development

Encouragement of tourism activities

Increase in gross value 

added in supported 

holdings/enterprises

Gross number of jobs 

created

Increase in gross value 

added in supported 

holdings/enterprises

Gross number of jobs 

Gross number of jobs 

created

created

Additional number of 

tourist visits (number of 

day visitors)

Targets 
defined

Target 
increased

RDP

Austria

Czech Republic

UK — England

France — Mainland 
(Hexagone)

Germany — 
Baden‑Württemberg

Hungary

Italy — Campania

Luxembourg

Poland

Romania

Spain — Andalucía

Sweden

Target 
increased

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Target 
increased

Target 
increased

Target 
increased

Target 
increased

Target 
reduced

Target 
increased

Target 
increased

Target 
increased

Target 
increased

Target 
increased

Target 
reduced

Target 
increased

Target 
reduced

Target 
reduced

Targets 
increased/
reduced

Targets 
reduced

Targets 
increased

Targets 
increased/
reduced

Targets 
increased

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Target 

increased

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Target 

increased

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Target 

reduced

Targets defined:

Targets defined in the initial and latest RDP (with indication if target was modified between initial and latest RDP)

Data provided:

Data provided and accepted by Commission (Annual Progress Report 2011)

Targets defined only in the initial or only in the latest RDP; targets not according to CMEF Handbook so not comparable

Data not provided per measure; data not up to date; Annual Progress Report not accepted by Commission

Targets not defined in the initial and latest RDP

Measure not programmed

Data not provided

Measure not implemented

1  The result indicator for this measure has five components (areas contributing to improvement of biodiversity, improvement of water quality, mitigat-
ing climate change, improvement of soil quality, avoidance of marginalisation and land abandonment). This table shows if a target was defined and 
data provided for at least one of these components. In some cases, targets were increased for one component and decreased for another. This is shown 
as ‘target increased/reduced’.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

43

Measure

Modernisation of agricultural holdings

Adding value to agricultural and forestry 

products

Agri-environment

Increase in gross value 

added in supported 

holdings/enterprises

Number of holdings/

enterprises introducing 

new products and/or 

new techniques

Increase in gross value 

added in supported 

holdings/enterprises

Number of holdings/

enterprises introducing 

new products and/or 

new techniques

Area under successful 

land management1

Diversification into non-agricultural activities

Business creation and development

Encouragement of tourism activities

Increase in gross value 
added in supported 
holdings/enterprises

Gross number of jobs 
created

Increase in gross value 
added in supported 
holdings/enterprises

Gross number of jobs 
created

Gross number of jobs 
created

Additional number of 
tourist visits (number of 
day visitors)

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 

defined

Data 

provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Targets 
defined

Data 
provided

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Target 
increased

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Target 
increased

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Target 
reduced

Targets defined:

Targets defined in the initial and latest RDP (with indication if target was modified between initial and latest RDP)

Data provided:

Data provided and accepted by Commission (Annual Progress Report 2011)

Targets defined only in the initial or only in the latest RDP; targets not according to CMEF Handbook so not comparable

Data not provided per measure; data not up to date; Annual Progress Report not accepted by Commission

Targets not defined in the initial and latest RDP

Measure not programmed

Data not provided

Measure not implemented

1  The result indicator for this measure has five components (areas contributing to improvement of biodiversity, improvement of water quality, mitigat-

ing climate change, improvement of soil quality, avoidance of marginalisation and land abandonment). This table shows if a target was defined and 

data provided for at least one of these components. In some cases, targets were increased for one component and decreased for another. This is shown 

as ‘target increased/reduced’.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

Targets 

defined

Target 

increased

RDP

Austria

Czech Republic

UK — England

France — Mainland 

(Hexagone)

Germany — 

Baden‑Württemberg

Hungary

Italy — Campania

Luxembourg

Poland

Romania

Spain — Andalucía

Sweden

Target 

increased

Target 

increased

Target 

increased

Target 

increased

Target 

increased

Target 

reduced

Target 

increased

Target 

increased

Target 

increased

Target 

increased

Target 

increased

Target 

reduced

Target 

increased

Target 

reduced

Target 

reduced

Targets 

increased/

reduced

Targets 

reduced

Targets 

increased

Targets 

increased/

reduced

Targets 

increased

44

REPLY OF THE 
COMMISSION

EXECUTIVE SUMMARY

I.
The  Commission  notes  that  this  audit  report  essentially 
focuses  on  the  monitoring,  evaluation  and  reporting  of 
the effectiveness and efficiency and the use made of that 
monitoring and evaluation information regarding the rural 
development expenditure.

IV.
Regarding  how  well  the  EU  budget  allocated  to  Rural 
Development has been spent, certain results and impacts 
can only be properly assessed well into the programming 
period and afterwards through evaluations.

V. (a)
The  Commission  considers  that  the  three  overarching 
objectives  for  rural  development  are  sufficiently  clear 
and spelled out in Article 4 of R1698/2005. They are fur-
ther specified and substantiated in the recitals and in the 
Community strategic guidelines. These objectives are to be 
used by the Member States (MS) in their programming

When approving the Rural Development Plans (RDPs), the 
Commission  carries  out  an  analysis  to  assess  that  pro -
grammes and measures are consistent with the Commu-
nity strategic guidelines, the national strategy plans and 
comply with the relevant legal provisions.

V. (b)
For multi-year programmes such as the Rural Development 
Programmes, certain results and impacts can only be prop-
erly assessed well into the programming period, and after-
wards through evaluations. 

Furthermore, useful information is provided by the Annual 
Progress Reports on implementation as regards financial 
execution and outputs.

Moreover, the Commission publishes in their entirety the 
evaluations carried out on its behalf, together with their 
quality assessments on the Europa site, and transmits them 
to the European Parliament, Council and Court of Auditors.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

V. (c)
The Commission considers that, where available, the Com-
mission - for the design - and the Member States - for the 
implementation - have used the available monitoring and 
evaluation input. 

Monitoring  information  is  used  to  support  the  case  for 
Rural Development Programme modification. Similarly, the 
Mid-Term Evaluations served a steering function, as a  num-
ber of Rural Development Programme modifications were 
linked to Mid-Term Evaluations findings.

VI.
The  information  available  through  the  monitoring  and 
evaluation  system,  while  incomplete  and  possibly  not 
always fully reliable, was the best available at this stage, 
tak ing  into  account  financial  and  practical  limitations 
faced by the Commission and Member States. 

The Commission together with Member States undertook 
an exercise in 2008 and 2009 to fill in gaps in Rural Devel-
opment  Programmes.  For  each  Rural  Development  Pro -
gramme, the Commission services have pointed out the 
missing targets and targets not adequately quantified and 
invited Managing Authorities to improve their set of tar-
gets. This exercise led to improve significantly the set of 
targets via Rural Development Programme modifications. 

The Commission has proposed solutions for a number of 
problems  related  to  the  Monitoring  and  Evaluation  sys-
tem for the next programming period, however, there will 
always be the need to find a balance between the benefits 
of monitoring and evaluation and the costs and adminis-
trative burden related to it.

The Commission considers that it has to be prudent when 
setting quantified targets in relation to objectives, which 
are affected by many external factors (e.g. the economic 
climate).

45

VII. (a)
The justification for the proposed Rural Development Pro-
grammes modifications is already a legal requirement. The 
Commission  will  continue  to  monitor  the  effectiveness 
and efficiency of programmes and towards the end of the 
period will increasingly focus on results. 

The  Commission  and  Member  States  use  the  available 
information in the framework of monitoring and evalua-
tion. For instance, the results of the Mid-Term Evaluations 
were taken into account in programme amendments.

VII. (b)
The Commission accepts the recommendation. 

A large number of Mid-Term Evaluations findings related 
to preparation for the ex-post evaluation, and actions have 
already been undertaken at Member States level.

VII. (c)
The proposals for the 2014-2020 Monitoring and Evalua-
tion system incorporate a number of changes compared 
to the CMEF. 

Instead of a  Mid-Term Evaluation, there will be enhanced 
Annual Implementation Reports (AIR) in 2017  and  2019, 
which will incorporate evaluation findings. Member States 
will be required to carry out evaluation activities through-
out the programming period in accordance with the Eval-
uation  Plan  included  as  part  of  the  Rural  Development 
Programmes.

Thus, assessment and reporting on the results and impacts 
of the policy will be linked more appropriately to the stage 
of implementation of the Rural Development Programmes.

VII. (d)
For  the  2014-2020  period,  the  Commission  and  Mem-
ber  States  are  elaborating  a monitoring  and  evaluation 
framework for the Common Agricultural Policy as a whole, 
including the monitoring and evaluation framework for the 
Rural Development Programmes. In this context, a number 
of compulsory common indicators have been defined, with 
a requirement for additional indicators where needed to 
address programme specificities.

The  common  indicators  will  also  provide  the  basis  to 
link the Rural Development policy to the broader 20/20 
Strategy.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSIONFor the 2014-2020 period, quantified target indicators are 
foreseen for each of the Focus Areas. Annual reporting in 
the Annual Implementation Reports is required for each of 
these. For some Focus Areas, the target indicator is set at 
output level. However, where the relevant result indicator 
is more complex and not suitable for use as a target moni -
tored annually, its value will be assessed through evalua-
tion, and reported in the 2017, 2019 and ex post reports.

VII. (e) 
The Monitoring and Evaluation system will use as much 
as possible existing information; although as evaluation 
requires also the assessment of impacts of a policy, this 
cannot immediately be derived from the programme itself.

VII. (f )
For the future, instead of Mid-Term Evaluations, the Com-
mission has proposed enhanced Annual Implementation 
Reports in 2017 and 2019. 

Those Annual Implementation Reports will include a sum -
mary  of  evaluation  findings,  and  it  is  intended  that  this 
should include relevant findings from the 2007-2013 ex-
post evaluations.

Moreover, the Commission publishes the complete reports 
of evaluations carried out on its behalf, together with their 
quality assessments, so all conclusions are presented.

46

INTRODUCTION

2.
The CMEF is not the only means to supervise the imple -
mentation of the budget.

For instance, the Commission carries out audit missions 
to check the correct implementation of the rural develop-
ment policy. Follow-up of these audits might include finan-
cial corrections.

3.
Monitoring and evaluation not only provide information on 
the degree of effectiveness or efficiency of a policy, but 
also on the reasons for this. 

This allows for an adaptation of the policy implementation 
or a redesign of the policy if needed.

5.
For the 2007 programming period, the CMEF was a new 
feature. It is evident that for Member-States, some of them 
relatively new to the EU or with a very limited evaluation 
culture, there is a learning curve, both regarding the meth-
odology of data collection, processing and the use of the 
evaluation results.

The  CMEF  constitutes  a significant  improvement  com -
pared to the monitoring and evaluation of previous pro-
gramming  periods.  Both  methods  and  definitions  have 
improved since 2010, although in particular (annual) moni-
toring of result indicators has remained a difficult task for 
Managing Authorities. 

S e e   a l s o   t h e   Co m m i s s i o n’s   re p l i e s   to   Ta b l e   1   u n d e r 
Table 1 bis in the Commission’s reply section. Please note 
that those replies were published together with the Court’s 
reports  referred  to  in Table  1.  As  a consequence,  those 
replies reflect the situation at the moment of the audit and 
have not been updated.

6.
For the 2014-2020 period, the Commission and the Mem-
ber States are putting in place a monitoring and evalua -
tion system for the Common Agricultural Policy as a whole. 
This system takes into account lessons learned from the 
past. However, inevitably every monitoring and evaluation 
system  has  a number  of  constraints,  such  as  data  avail -
ability, the cost of measurement, time needed for result/
impact of a policy measure to become visible or conflict -
ing demands, such as the wish for administrative burden 
reduction. For this reason, to the maximum possible, the 
new monitoring and evaluation system is based on exist-
ing data sources as well as monitoring data generated dur-
ing policy implementation. 

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION47

OBSERVATIONS

Common reply to paragraphs 19 and 20
The Financial Regulation imposes the use of SMART objec-
tives at the level of activity. Activity Based Budgeting 04 
(i.e.  rural  development  expenditure)  is  implemented  at 
the level of the rural development programmes which are 
composed of a  set of measures, which can contribute to 
several objectives. Thus, the overall objectives are set out 
at the programme level and their achievements are meas-
ured  at  that  level  (see  also  Commission  replies  to  para-
graphs 22 to 25).

SMART objectives should allow a judgement of whether 
they  have  been  reached,  not  necessarily  a quantified 
target.

22.
In  the  recitals  of  the  regulation  the  rationale  for  each 
measure  is  given.  At  the  level  of  an  individual  measure 
the outputs of the measure should be specific and verifi-
able. To achieve the objectives at programme level usually 
a combination of measures will be needed, organised in an 
axis structure for the 2007-2013 programming period.

23.
The  objectives  at  EU  level  are  formulated  in  qualitative 
terms  and  specify  the  direction  in  which  policy  should 
shift competitiveness and diversification. The Commission 
considers that it has to be prudent when setting quanti-
fied targets in relation to objectives which are affected by 
many external factors (e.g. the economic climate).

The Commission considers that its proposals for the 2014-
2020 Regulations include a clear hierarchy of objectives 
and sets targets at EU level (EU 2020) and Member States 
should set quantified targets consistent with this at RDP 
level for each focus area.

25.
The architecture established by Regulation 1698/2005 is 
based on a coherent toolkit designed to address the three 
Rural Development objectives. This toolkit took duly into 
account the available information on the effectiveness and 
efficiency of the individual measures. 

Common reply to paragraphs 26 and 27
The CMEF was introduced for the first time for 2007-2013 
programming period. Its purpose is to back up the pro-
gramme objectives by setting targets for outputs, results 
and impacts. 

Establishing quantified targets was a new exercise, and was 
a learning experience for both Managing Authorities and 
the Commission. Where Rural Development Programmes 
were adopted with an incomplete set of targets there was 
a commitment from the Member States side to complete 
them. 

The Commission, together with Member States undertook 
an exercise in 2008 and 2009 to fill in gaps in Rural Devel-
opment  Programmes.  For  each  Rural  Development  Pro -
gramme, the Commission services have pointed out the 
missing targets and the targets not adequately quantified 
and invited Managing Authorities to improve their set of 
targets. This exercise led to improve significantly the set of 
targets via Rural Development Programmes modifications. 

It is the task of the evaluation, which uses among other 
sources  the  monitoring  data,  to  ascer tain  if  a cer tain 
programme  has  contributed  to,  or  improved,  promoted 
a given goal and hence if the objectives have been met.

Evaluating complex policy instruments is challenging, and 
methodology and its application is continually improving 
to allow better demonstration of policy achievements.

27.
S e e   a l s o   t h e   Co m m i s s i o n’s   re p l i e s   to   Ta b l e   1   u n d e r 
Table 1 bis in the Commission’s reply section. Please note 
that those replies were published together with the Court’s 
reports  referred  to  in Table  1.  As  a consequence,  those 
replies reflect the situation at the moment of the audit and 
have not been updated

28.
For the 2014-2020 period, the Commission and the Mem-
ber  States  are  elaborating  a monitoring  and  evaluation 
framework for the Common Agricultural Policy as a whole, 
including the monitoring and evaluation framework for the 
Rural Development Programmes. In this context, a number 
of compulsory common indicators have been defined, with 
a requirement for additional indicators where needed to 
address programme specificities.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION48

The  common  indicators  will  also  provide  the  basis  to 
link the Rural Development policy to the broader 20/20 
Strategy.

28. First bullet
This was especially true in the beginning of the program-
ming  period  but  in  many  cases,  the  target  values  were 
complemented later on.

28. Second bullet
In the Commission’s view, a target can be both: a quanti -
fied objective and an estimate of expected results.

28. Third bullet
A number of common indicators will never be sufficient to 
capture all situations. The legal framework offers scope for 
Managing Authorities to define programme-specific indi-
cators where necessary.

29.
See replies to paragraphs 26 and 27.

The  Rural  Development  Programmes  of  Austria  subse -
quently provided output and result indicator targets for 
the agri-environment measures such as: number of hold-
ings  suppor ted;  suppor ted  area;  number  of  contracts; 
cultivated area contributing to biodiversity, water quality, 
climate change, soil quality, prevention of marginalisation. 

30.
Based on the strategy, the allocation of the budget reflects 
the priorities among the needs identified by the Member 
States.

31.
The CMEF was introduced for the first time for 2007-2013 
programming  period. The  legislation  requires  that  only 
indicative (non-binding) targets are set beforehand. Estab-
lishing  quantified  targets  was  a new  exercise,  and  was 
a learning experience for both Managing Authorities and 
the Commission. In this context, it is understandable that 
some initial targets may have been unrealistic and there-
fore it was appropriate to revise them down.

32.
See replies to paragraphs 26 and 27.

The  inclusion  of  additional  indicators  is  foreseen  under 
Regulation  1698/2005  (Ar ticle  81(2)).  As  stated  in  this 
article, the intention is not to have an exhaustive set of 
additional indicators, but rather to define some additional 
indicators to allow monitoring and evaluation of the spe-
cificities of the Rural Development Programmes.

Under the shared management and subsidiarity principle, 
it is up to the Member States to define which indicators 
are relevant. The indicator ‘number of holdings introduc-
ing new products or new technologies’ was not considered 
by Czech Authorities relevant for the measures ‘modernis-
ing agricultural holdings’ and ‘adding value to agricultural 
products’, since innovation-focused projects are financed 
under M124. This explanation was provided following DG 
AGRI’s  letter  on  the  Annual  Progress  Repor ts  2010  and 
accepted because it reflects the CZ measures’ design.

33.
See replies to paragraphs 26-27 and 31.

34.
See reply to paragraph 26.

The Commission is well aware about the discrepancies in 
indicator definition, and in the new monitoring and evalu-
ation framework it has paid a lot of attention to the proper 
definition of the indicators.

The CMEF was introduced for the first time for 2007-2013 
programming period. Establishing quantified targets for 
Rural Development was a new exercise, and was a learn -
ing  experience  for  both  Managing  Authorities  and  the 
Commission. 

35.
Monitoring and evaluation systems are designed to pro-
vide information at 3 levels: output, result and impact. All 
three levels of performance indicators are needed to be 
able to assess the policy on both short and long term. The 
information on results is an integral part of the system. 

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSIONCommon reply to paragraphs 37, 38 and 39
In the 2007 programming period, the CMEF was a new fea -
ture. It is evident that for Member-States, some of them 
relatively new to the EU or with a very limited evaluation 
culture, there is a learning curve, both regarding the meth-
odology of data collection, processing and the use of the 
evaluation results.

The CMEF Handbook provides guidance on how to imple-
ment the CMEF. In addition, the Commission services have 
and  will  continue  (by  working  documents,  Questions  & 
Answers…)  to  provide  further  guidance  to  the  Member 
States through different fora available (Rural Development 
Committee, Evaluation Expert Network and European Net-
work for Rural Development).

However, according to the subsidiarity principle, some flex-
ibility is left to the Managing Authorities in establishing 
their system.

For  the  new  programming  period,  from  the  onset  the 
data  needs  for  monitoring  and  evaluation  were  taken 
into account, balancing data needs and the reduction of 
administrative burden, preferring a system that where pos-
sible builds on existing data sources. 

40.
Due to its bottom up nature, developing CMEF indicators 
for the Leader axis was a particular challenge. But Leader is 
part of the CMEF.

The Commission provided guidance to Member States, also 
for evaluating Leader. In the context of the European Net-
work for Rural Development, work has been carried out on 
self-assessment of Local Development Strategies.

Common reply to 41 and 42
See reply to paragraph 37.

49

The CMEF constitutes a significant improvement compared 
to  the  monitoring  and  evaluation  of  previous  program-
ming periods. Nevertheless, the early years of its imple -
mentation  have  been  a  ‘learning  by  doing’  exercise  for 
both the Commission and the Member States. Both meth-
ods and definitions have improved since 2010, although 
in particular (annual) monitoring of result indicators has 
remained a difficult task for Managing Authorities. 

Box 1 
While the differences pointed at by the Court might indeed 
not be fully explained by differences in measures, they may 
also be explained by the differences in the starting point 
for each Member State. It is common use in Rural Develop-
ment policy to consider that a ‘new product or technique’ 
meant one that was new to the region. The Commission 
works closely with the Member States to improve consist-
ency as far as possible.

It is acknowledged that there were some difficulties in cor-
rectly recording data for the output related to ‘area under 
agri-environment support’and result indicator regarding 
‘area under successful land management’. The output indi-
cator is set at measure level, whereas the result indicator is 
at Axis level and covers the range of relevant measures and 
schemes under Axis 2. In some cases, depending on the 
measures and schemes implemented, the correct figures 
may be the same, but in most cases the data will differ.

43.
The mid-term evaluation did provide a good overview of 
the implementation of Rural Development Programmes. 
However, for multi-annual programmes such as the Rural 
Development Programmes, certain results, such as Gross 
Value Added and impacts can only be properly assessed 
well into the programming period, and afterwards. Such 
effects can only be measured after sufficient time since 
implementation has passed; for most interventions, this 
was not yet the case at the moment of the mid-term evalu-
ation. For example, measures under Axis 3 and Axis 4 take 
more time to implement and, as a consequence, for results 
and impacts to be visible.

Consequently,  cer tain  results  and  impacts  of  the  pro -
grammes could not yet be measured, unlike the expendi-
ture or outputs. 

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION50

48.
In a system of shared management, every partner has to 
assume their own responsibilities.

Anomalies in the monitoring data and in Annual Progress 
Reports have been systematically discussed with Member 
States.

The Commission has requested the HU authorities to rein-
troduce the initial wording. 

For the new programming period, the Commission services 
and Member States have collaborated to define ex-ante 
the data definition and data sources.

50.
The CMEF constitutes a significant improvement compared 
to  the  monitoring  and  evaluation  of  previous  program-
ming periods. Nevertheless, the early years of its imple -
mentation  have  been  a  ‘learning  by  doing’  exercise  for 
both the Commission and the Member States. 

Both methods and definitions have improved since 2010, 
although in particular (annual) monitoring of result indica-
tors has remained a difficult task for Managing Authorities.

51.
As a complement to the information provided in Table 3, 
the  Commission  also  publishes  the  evaluations  carried 
out on its behalf with a quality assessment by the steering 
group.

44.
The use of information from a limited number of cases to 
support conclusions for the whole measure or programme 
can  be  methodologically  sound  if  correctly  applied,  it 
depends on the representativeness of the sample.

See also reply to paragraph 56.

45.
See reply to paragraph 43. 

For many programmes, the Mid-Term evaluations came too 
early to assess efficiency. This can only be done once the 
results/impacts of the programmes are visible. The Com-
mission continues to encourage Member States to improve 
the quality and consistency of evaluations and provides 
methodological support.

Moreover, since most evaluators at this stage in the pro-
gramme lifecycle focused on (implementation of ) individ-
ual measures, it was too early to draw lessons on a  more 
general level regarding the attainment of overall objec-
tives at EU level.

47.
As stated, the CMEF was introduced for the first time for 
2007-2013 programming period and the first years were 
a ‘learning by doing’ experience. 

The  3  reports  referred  to  and  submitted  in  July  2012  in 
accordance with the Regulation, had not been accepted by 
April 2013 because they required further corrections. Nev-
ertheless 89 of the 92 reports submitted for the year 2011 
had been accepted. In these Annual Progress Reports 2011, 
the set of output indicators was close to be complete.

For some of the indicators shown in Annex, although ini-
tially it was foreseen their values would be submitted every 
year as part of the monitoring data included in the Annual 
Progress Reports (APRs), it has been recognised that they 
require evaluators’ input in order to be calculated and/or 
a certain time to elapse before they can be measured. It is 
therefore not efficient for them to be assessed every year 
and guidance to this effect has been issued.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION51

53.
The  Qualit y  judgement  of  the  synthesis  of  the  M id-
Term  Evaluations,  published  by  the  Commission  on  the 
Europa  website,  states  clearly ‘As  the  Mid-Term  Evalua-
tions  repor ts  used  as  source  data  are  often  of  variable 
quality  and  detail  as  regards  their  information  on  par-
ticular topics, this fact is necessarily also reflected in the 
synthesis of the findings and can be a limiting factor to 
the  overall  qualit y  of  the  findings’  (http://ec.europa.
eu/agriculture/evaluation/rural- development-repor ts/
synthesis-mte-2007-2013_en.htm).

The Commission considers that it did not extrapolate the 
data, but recognises the need to improve the presentation 
of the Annual Activity Report table in question. The infor-
mation provided in the document just reported on a state 
of play of a given situation versus estimated targets.

54.
T h e   l e g i s l a t i o n   ( A r t i c l e   6 2 ( 2 )   o f   R e g u l a t i o n   ( E C ) 
No 1974/2006)  refers  to  indicative  (non-binding)  target 
values. Furthermore, the initial targets were set before the 
crisis affecting the world economy.

The Commission considers that changing economic cir-
cumstances can be a good justification for changing the 
targets. To stick to a target that has become outdated is 
not  useful,  neither  for  management  purposes,  nor  for 
accountability purposes. Targets can also be modified to 
reflect  changes  in  the  Rural  Development  Programmes 
strategy.

It  is  correct  that  the  Czech  authorities  have  set  initially 
rather  ambitious  targets  related  to  job  creation.  At  the 
same time, the economic/financial crisis has significantly 
hindered creation of employment. 

Therefore, following the mid-term evaluation, the Annual 
Examination  Meeting  2011  and  following  the  Cour t ’s 
visit,  a revision  of  these  targets  has  been  proposed  on 
27/09/2012 under the 8th Rural Development Programmes 
modification, accepted by the Commission on 13/03/2013.

Box 2 
While targets should not be revised simply to reflect the 
latest projection of results, there is no point either to keep 
sticking to targets that are –despite best efforts- not realis-
tic anymore, e.g. due to changing economic circumstances 
in which the measures are applied. 

The  Commission  considers  that  reducing  result  targets 
does not, per definition, represent a reduction in planned 
efficiency and/or effectiveness. It can also be an acknowl-
edgement  of  the  changing  economic  contex t,  which 
makes it harder to achieve a number of targets, e.g. job 
creation.

As  targeting  tak es  many  for ms,  targeting  tak es  into 
account identified structural and territorial needs and may 
result in directing the support to those in greatest needs 
while the support could have resulted in higher values for 
indicators in other areas.

Targets should not been looked at in isolation but in the 
context of the strategy of the programme as a whole.

55.
The Commission publishes in due time the findings of the 
evaluations carried out on its behalf, but also the quality 
assessment by the steering group, which includes the use-
fulness of the report for the Commission. 

56.
The  Commission  publishes  the  complete  reports  of  the 
independent evaluators on the Europa website.

The  evaluations  of  the  Rural  Development  Programmes 
revealed certain difficulties in demonstrating the effective-
ness, efficiency, achievements and impacts of investment 
measures in a comprehensive and consistent manner. For 
this reason, the Commission has launched an evaluation 
intended to improve capacity to identify and understand 
the impact of measures to support physical investments: 
http://ec.europa.eu/agriculture/calls-for-tender/tender-
documents/2013/275093/specs_en.pdf ;  this  tackles  i.a. 
the  issue  of  the  assessment  of  deadweight.  I t  should 
also improve the demonstration of policy achievements 
and  also  contribute  to  future  policy  development  and 
implementation. 

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION52

57.
The reporting of results in the annual progress reports is 
intended  to  give  a picture  of  progress  in  achieving  the 
objectives of the Rural Development Programmes (as well 
as in the Annual Activity Report of DG AGRI) not necessar-
ily its evaluation. Furthermore, useful information is pro-
vided by the Annual Progress Reports on implementation 
as regards financial execution and outputs.

See also Commission reply to paragraph 43.

Regarding  how  well  the  EU  budget  allocated  to  Rural 
Development has been spent, certain results and impacts 
can only be properly assessed well into the programming 
period and afterwards through evaluations.

58.
The Commission considers that improving the implemen-
tation  (as  far as  monitoring is  concerned)  or  the  design 
(evaluation) and accountability are the prime purposes of 
the monitoring and evaluation system. 

61.
As already underlined, the target setting was a learning 
by doing process which necessarily extended beyond the 
approval stage of the programmes.

Box 3 – First paragraph
The Commission would also like to point out that ‘creat-
ing new jobs’ is only one of the concrete objectives linked 
to the implementation of Axis III where the overarching 
objective  is  to  improve  the  quality  of  life  in  rural  areas 
and to encourage diversification of economic activity. The 
Commission considers that this can partly explain the dif-
ferences in public expenditure per ‘job created’. The RDP of 
Campania is an example of this.

62.
See Commission reply to paragraph 61.

The explicit introduction of commonly defined target indi-
cators underpinned by an indicator plan for the new pro-
gramming period as proposed by the Commission should 
make  it  easier  to  assess  planned  outputs  and  financial 
resources against the targets to be achieved.

66.
It takes time for the results and impacts of a programme 
to become visible, therefore it can be expected that the 
modifications to programmes particularly during the early 
part of the programming period are more related to input- 
or output related factors (e.g. budgetary constraints or lack 
of uptake). 

The  Commission  will  examine  the  ex-ante  evaluations 
and the plausibility of the targets proposed for the new 
period. However, the Commission cannot unduly delay the 
approval of the programmes. 

However, the conclusions derived from the Mid Term Eval-
uations, which were among others using available moni-
toring data, were taken into consideration in programme 
amendments.

67.
Amendments to the programmes to avoid ‘decommitment’ 
need to be assessed in the context of the overall strategy.

Box 3  
As  stated,  the  CMEF  was  introduced  for  the  2007-2013 
period. These early years have been a ‘learning by doing’ 
experience for the Commission and Member States. In the 
first versions of Rural Development Programmes, lack of 
benchmarks and reference information could have led to 
unrealistic targets being set.

The difference in cost can also mirror the economic situ-
ation and salary level in the different areas, hence a varia -
tion can indeed be expected.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION53

68.
The CMEF was introduced for the first time for 2007-2013 
programming period. Establishing quantified targets for 
rural development measures was a  new exercise, and was 
a learning experience for both Managing Authorities and 
the Commission. The legislation requires that only indica-
tive (non-binding) targets are set beforehand. 

Where  Rural  Development  Programmes  were  adopted 
with  an  incomplete  set  of  targets  there  was  a commit -
ment to complete them. Certain Member States had obvi-
ously overestimated the results that could achieve with 
the funds allocated and have afterwards corrected these 
estimates.

Furthermore,  the  economic  context  has  been  changed 
drastically and required certain adaptation of the expected 
results.

69.
In  the  annual  meetings,  all  available  data  and  informa-
tion, e.g. financial, monitoring data, evaluation findings, 
are taken into account to assess progress in implementa-
tion and any need for adjustments in strategy and financial 
allocations are discussed.

70.
Since the report quoted by the Court is an implementation 
report on strategy plans/strategic guidelines, it cannot go 
in the detailed reasoning behind the budget for each indi-
vidual Rural Development measure. 

Moreover,  it  refers  to  results,  but  lets  the  reader  under-
stand  that  is  too  early  for  them  to  become  visible.  (see 
point 4: ‘outlook’ in the report).

71.
Poor results for one indicator are not a sufficient basis to 
change a policy with a 7 to 9 year implementation period.

72.
Although evaluation results for the current programming 
period  (2007-13)  were  not  yet  available,  the  Commis -
sion’s Common Agricultural Policy reform proposals were 
strongly based on the lessons learnt so far with the CMEF, 
the on-going experience with implementation of the Rural 
Development Programmes and on the evaluations of pre-
vious  programming  periods  and  of  rural  development 
measures. This allowed the Commission to develop and 
underpin a more results oriented policy with a reinforced 
monitoring and evaluation system, overcoming some of 
the shortcomings identified for the 2007-13 programming 
period.

Common reply to 73 and 74
The Commission was bound to present a Common Agri -
cultural Policy reform proposal for the period 2014-2020 
early enough in order that it could enter into force in 2014. 
Taking into account the necessary consultation process 
and the legislative procedures, the exercise needed to be 
launched  in  2010. This  led  to  a need  to  use  the  evalua -
tion data available at that time for the impact assessment. 
The Commission could not wait for more recent evaluation 
results, i.e. the synthesis of the ex-post evaluation reports 
of 2000-2006 programming period or the mid-term evalu-
ation reports of the programming period 2007-2013, with-
out substantially delaying the reform. 

However,  the  Commission  publishes  in  due  time  all  the 
evaluations performed on its behalf together with their 
quality  assessments,  so  the  information  does  become 
available to Parliament and Council. 

The seven measures presented in Annex 4a of the Impact 
Assessment represent important rural development meas-
ures and provide an overview of different types of support 
measures.

See also reply to paragraph 56.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION75.
See replies to paragraphs 73 and 74.

Policy choices are based on a wide range of information, 
including  monitoring  and  evaluation,  and  on  different 
political visions on prioritisation of problems and possible 
solutions. 

The Commission considers that the questions referred to 
by the Court come in only once the objectives have been 
decided  and  focus  on  the  choice  of  measures  to  reach 
these objectives. 

The  information  available  through  the  monitoring  and 
evaluation  system,  while  incomplete  and  possibly  not 
always fully reliable, was the best available at this stage, 
taken into account financial and practical limitations faced 
by the Commission and Member States. The Commission 
has proposed solutions to a number of problems related 
to the Monitoring and Evaluation system for the next pro-
gramming period, however, it should be noted that there 
will  always  be  the  need  to  find  a balance  between  the 
benefits of monitoring and evaluation and the costs and 
administrative burden related to it. Moreover, the results 
and impacts of a measure will also be impacted by exter -
nal factors, hence there will never be 100  % certainty in 
advance  about  what  is  the  most  effective  and  efficient 
measure to reach a given target.

54

CONCLUSIONS AND RECOMMENDATIONS

76.
The Commission considers that, where available, the Com-
mission - for the design - and the Member States - for the 
implementation - have used the available monitoring and 
evaluation input. 

The Commission publishes the entire reports of evalua-
tions carried out on its behalf together with their quality 
assessments on the Europa site, and transmits them to the 
European Parliament, Council and Court of Auditors. 

77.
The mid-term evaluation did provide a good overview of 
the implementation of Rural Development Programmes. 
However, for multi-annual programmes such as the Rural 
Development Programmes, certain results, such as Gross 
Value Added and impacts can only be properly assessed 
well into the programming period, and afterwards. 

78.
The Commission considers that the CMEF is a valid instru -
ment to monitor and evaluate progress towards achiev-
ing  the  EU’s  rural  development  policy  objectives  and  it 
has many strengths, although some difficulties have been 
encountered. 

The  information  available  through  the  monitoring  and 
evaluation  system,  while  incomplete  and  possibly  not 
always fully reliable, was the best available at this stage, 
tak ing  into  account  financial  and  practical  limitations 
faced by the Commission and Member States. The Com-
mission, Member States and various stakeholders devote 
much effort to continuously improving the system. This 
experience  is  being  used  for  the  development  of  an 
improved monitoring and evaluation system for the next 
programming period. 

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION55

The Commission has proposed solutions for a number of 
problems related to the monitoring and evaluation system 
for the next programming period, however, it should be 
noted that there will always be the need to find a balance 
between the benefits of monitoring and evaluation and 
the costs and administrative burden related to it. Moreover, 
the results and impacts of a measure will also be impacted 
by external factors, hence there will never be 100  % cer-
tainty in advance about what is the most effective and effi-
cient measure to reach a given target.

81.
The Commission considers that, where available, the Com-
mission - for the design - and the Member States - for the 
implementation - have used the available monitoring and 
evaluation input.

Monitoring  information  is  used  to  support  the  case  for 
Rural Development Programmes modification. Similarly the 
Mid-Term Evaluations served a steering function, as a  num-
ber of Rural Development Programmes modifications were 
linked to Mid-Term Evaluations findings.

79.
The Commission considers that the objectives expressed 
in Regulation 1698/2005 and in the Community Strategic 
Guidelines provide sufficient basis to identify the objec-
tives of the policy. Rural Development Programmes pro-
posals  have  been  assessed  by  the  Commission  against 
these objectives and against the needs identified in each 
programming region (or country) and the money has been 
allocated on that basis.

The Commission was bound to present a Common Agricul -
tural Policy reform proposal for the period 2014-2020 early 
enough in order that it would enter into force in 2014. This 
led to a need to use the evaluation data available at that 
time. The Commission could not wait for more recent eval-
uation results, i.e. the synthesis of the ex-post evaluation 
reports of 2000-2006 programming period or the mid-term 
evaluation reports of the programming period 2007-2013, 
without substantially delaying the reform. 

As  targeting  tak es  many  for ms,  targeting  tak es  into 
account identified structural and territorial needs and may 
result in directing the support to those in greatest needs 
while the support could have resulted in higher values for 
indicators in other areas.

80.
For multi-year programmes such as the Rural Development 
Programme, certain results and impacts can only be prop-
erly assessed well into the programming period, and after-
wards through evaluations.

Furthermore, useful information is provided by the Annual 
Progress Reports on implementation as regards financial 
execution and outputs.

82.
The  information  available  through  the  monitoring  and 
evaluation  system,  while  incomplete  and  possibly  not 
always fully reliable, was the best available at this stage, 
taken into account financial and practical limitations faced 
by the Commission and Member States. The Commission 
has proposed solutions for a number of problems related 
to  the  Monitoring  &Evaluation  system  for  the  next  pro-
gramming period, however, there will always be the need 
to  find  a balance  between  the  benefits  of  monitoring 
and evaluation and the costs and administrative burden 
related to it. Moreover, the results and impacts of a meas -
ure will also be impacted by external factors, hence there 
will  never  be  100 %  certainty  in  advance  about  what  is 
the most effective and efficient measure to reach a given 
target.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION56

Recommendation 1
The Commission is already addressing this recommenda-
tion within the constraints of the legislative framework for 
the current programming period.

The justification for the proposed Rural Development Pro-
grammes modifications is already a legal requirement. The 
Commission  will  continue  to  monitor  the  effectiveness 
and efficiency of programmes and towards the end of the 
period will increasingly focus on results. 

The  Commission  and  Member  States  use  the  available 
information in the framework of monitoring and evalua-
tion. For instance, the results of the Mid-Term Evaluations 
were taken into account in programme amendments.

83.
The problems related to the mid-term evaluations were 
mainly due to the fact that they were carried out too early 
for the impacts to become visible. They did however pro-
vide  useful  information  on  the  implementation  of  the 
programmes. 

As far as the ex-post evaluation in 2015 is concerned, the 
evaluator can –and should- use data from different sources 
(‘triangulation’). For the new programming period, from 
the onset the data needs for monitoring and evaluation 
were  taken  into  account,  balancing  data  needs  and  the 
reduction of administrative burden, preferring a system 
that where possible builds on existing data sources. 

Recommendation 2
The Commission accepts the recommendation. 

A large number of Mid-Term Evaluations findings related 
to preparation for the ex-post evaluation, and actions have 
already been undertaken at Member States level.

Recommendation 2 (i)
A Thematic Working Group of the Evaluation Network will 
be launched by the Commission in autumn 2013 to pre -
pare  guidelines  for  the  ex-post  evaluation  of  2007-2013 
Rural Development Programmes.

Recommendation 2 (iii)
The Commission together with Member States undertook 
an exercise in 2008 and 2009 to fill in gaps in Rural Devel-
opment  Programmes.  For  each  Rural  Development  Pro -
gramme, the Commission services have pointed out the 
missing targets and the target not adequately quantified 
and invited Managing Authorities to improve their set of 
targets. This exercise led to improve significantly the set of 
targets via Rural Development Programmes modifications. 
Afterwards, on an on-going basis, efforts are made to solve 
the fewer remaining issues. 

The CMEF Handbook provides guidance on how to imple-
ment the CMEF. In addition, the Commission services have 
and  will  continue  (by  working  documents,  Questions  & 
Answers…)  to  provide  further  guidance  to  the  Member 
States through different fora available (Rural Development 
Committee, Evaluation Expert Network and European Net-
work for Rural Development).

84.
See reply to recommendation 3.

Recommendation 3
The Commission has already addressed the recommenda-
tion by proposing a new design for the 2014-2020 Moni -
toring and Evaluation system. However, in the proposed 
design  of  the  new  monitoring  and  evaluation  system, 
results will not be assessed at individual measure level but 
at the more appropriate focus area level. It incorporates 
a number of changes compared to the CMEF. Instead of 
a Mid-Term  Evaluations,  there  will  be  enhanced  Annual 
Implementation  Repor ts  in  2017  and  2019,  which  will 
incorporate evaluation findings. The first will concentrate 
on elements related to programme steering, and the sec-
ond on providing an initial assessment of the impact of 
the Rural Development Programmes. Member States are 
required  to  carr y  out  evaluation  activities  throughout 
the programming period in accordance with the Evalua-
tion Plan included as part of the Rural Development Pro-
grammes. Thus, assessment and reporting on the results 
and impacts of the policy will be linked more appropriately 
to the stage of implementation of the Rural Development 
Programmes.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION57

Quantified target indicators are foreseen for each of the 
Focus Areas. Annual reporting in the Annual Implementa-
tion Reports is required for each of these, and one of the 
lessons of implementing the CMEF is that this is not always 
possible or appropriate for result. For some Focus Areas, 
the target indicator is set at output level. However, where 
the relevant result indicator is more complex and not suit-
able for use as a target monitored annually, its value will 
be assessed through evaluation, and reported in the 2017, 
2019 and ex post reports.

Recommendation 3 (i)
While indicators can contribute to the evaluation of a pol -
icy, an indicator value alone is of little value and may be 
meaningless. It is rarely a  single measure that contributes 
to the achievement of an objective, more often multiple 
measures contribute together, and equally, an individual 
measure  may  contribute  to  multiple  objectives.  Output 
data is collected at measure level. The Commission pro-
poses  for  the  new  period  that  result  indicators  will  be 
assessed at the level of the Focus Area, which may be the 
result of a number of measures acting in complementary 
fashion. 

The  strategic  approach  to  the  2014-2020  policy,  based 
on priorities and focus areas, with measures contributing 
flexibility to these, as shown in the associated interven-
tion logic and indicator hierarchy demonstrates the links 
between different types of indicator and the different lev-
els of objectives.

Recommendation 3 (ii)
Checks  of  the  quality  of  the  data  are  an  integral  par t 
of  a monitoring  system,  in  the  respect  of  the  shared 
management. 

85.
The  Commission  considers  that  the  three  overarching 
objectives for rural development are sufficiently clear and 
spelled out in Article 4 of Regulation (EC) No 1698/2005. 
They are further specified and substantiated in the recit-
als  and  in  the  Community  strategic  guidelines.  These 
objectives were to be used by the Member States in their 
programming.

When approving the Rural Development Programmes, the 
Commission  carried  out  an  analysis  to  assess  that  pro-
grammes and measures are consistent with the Commu-
nity strategic guidelines, the national strategy plans and 
comply with the relevant legal provisions.

For  the  period  2014-2020,  the  Commission  will  develop 
and underpin a more results oriented policy with a rein -
forced  monitoring  and  evaluation  system,  overcoming 
some of the shortcomings identified for the 2007-13 pro-
gramming period.

Recommendation 4
For the 2014-2020 period, the Commission and the Mem-
ber  States  are  elaborating  a monitoring  and  evaluation 
framework for the Common Agricultural Policy as a whole, 
including the monitoring and evaluation framework for the 
Rural Development Programmes. In this context, a number 
of compulsory common indicators have been defined, with 
a requirement for additional indicators where needed to 
address programme specificities.

The common indicators also provide the basis to link the 
Rural Development policy to the broader 20/20 Strategy.

Recommendation 4 (i)
Quantified target indicators are foreseen for each of the 
Focus Areas. Annual reporting in the Annual Implementa-
tion Reports is required for each of these. For some Focus 
Areas, the target indicator is set at output level. However, 
where the relevant result indicator is more complex and 
not  suitable  for  use  as  a target  monitored  annually,  its 
value will be assessed through evaluation, and reported in 
the 2017, 2019 and ex post reports.

Recommendation 4 (ii)
The Commission does not accept this part of the recom-
mendation. In the proposed design of the new monitor-
ing and evaluation system, results and impacts will not be 
assessed at measure level. 

The  Commission  and  M ember  States  are  elaborating 
a monitoring and evaluation framework for the Common 
Agricultural Policy as a whole, including the monitoring 
and evaluation framework for the Rural Development Pro-
grammes. In this context, a  number of compulsory com-
mon indicators have been defined, with a requirement for 
additional indicators where needed to address programme 
specificities.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION58

Measure ‘fiches’ for common indicators will include infor-
mation such as source, provider, frequency of collection, 
unit of measurement etc. to ensure a harmonised and con -
sistent approach.

This framework takes account of the information needs for 
policy  implementation  and  development,  however  also 
tries to limit the administrative burden and costs related 
to the data collection.

Recommendation 4 (iii)
Each Rural Development Programme is subject to ex-ante 
evaluation, which will appraise, among others, the internal 
coherence of the proposed programme, the consistency 
of the allocation of budgetary resources with the objec-
tives of the Rural Development Programme and whether 
the quantified targets are realistic.

Results of ex-ante evaluations and its recommendations 
are considered in the approval process of the Rural Devel-
opment Programmes. 

86.
Targeting  can  be  achieved  in  many  ways  in  addition  to 
selection criteria and eligibility criteria; using sectorial or 
regional differentiation, differentiation of aid intensities by 
type of beneficiary or by type of investment or by setting 
aid ceilings.

Regardless the process of selection of the Rural Develop-
ment projects, there is a need for measuring the output, 
result and impact of the programmes in order to draw les-
sons for the future. This implies that in any case data col-
lection and analysis is needed, with the associated costs.

87.
See Commission replies to paragraph 57 and 77.

The Commission publishes the entire reports of the evalu-
ations carried out on its behalf together with their quality 
assessments on the Europa site, and transmits them to the 
European Parliament, Council and Court of Auditors. 

Recommendation 6
The Commission considers that there was sufficient follow-
up of the Mid-Term Evaluations findings. Letters were sent 
to the Managing Authorities enquiring what actions were 
to be taken in relation to Mid-Term Evaluations recommen-
dations, and the Annual Review meetings also addressed 
this issue. In addition, the 2011 Focus Group meetings con-
ducted in each Member State by the Evaluation Helpdesk 
focussed on the Mid-Term Evaluations findings.

Recommendation 6 (i)
Instead  of  a Mid-Term  Evaluation,  the  Commission  has 
proposed  for  the  2014-2020  period  enhanced  Annual 
Implementation  Repor ts  in  2017  and  2019,  which  will 
incorporate evaluation findings. The first will concentrate 
on elements related to programme steering, and the sec-
ond on providing an initial assessment of the impact of 
the Rural Development Programmes. Member States are 
required  to  carr y  out  evaluation  activities  throughout 
the programming period in accordance with the Evalua-
tion Plan included as part of the Rural Development Pro-
grammes. Thus, assessment and reporting on the results 
and impacts of the policy will be linked more appropriately 
to the stage of implementation of the Rural Development 
Programmes.

Recommendation 6 (ii)
The Commission publishes the entire reports of the evalu-
ations carried out on its behalf together with their quality 
assessments on the Europa site, and transmits them to the 
European Parliament, Council and Court of Auditors. The 
Commission considers that the objectives of the evalua-
tions are to improve decision-making (by explaining what 
works and why) and to demonstrate accountability on the 
basis of work of an independent contractor. 

Recommendation 6 (iii)
While lessons can be drawn from the ex-post evaluations, 
the 2015 report is not necessarily the best place to ana-
lyse  the  implications  for  any  future  programmes.  The 
Annual Implementation Reports for 2014-2020 will include 
a summary of evaluation findings, and it is intended that 
this should include relevant findings from the 2007-2013 
ex-post evaluations.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION59

TABLE 1 – COMMISSION’S REPLIES TO THE FINDINGS 
ON MONITORING AND EVALUATION OF THE RURAL 
DEVELOPMENT POLICY IN THE COURT’S SPECIAL 
AND ANNUAL REPORTS

Special Report No 5/2010   –  Implementation of 
the Leader approach for rural development
The  ex-post  evaluation  of  Leader+…  is  considering]  the 
issues  of  effectiveness  and  efficiency  of  expenditure.  It 
is normal that the effectiveness and efficiency of policy 
instruments are assessed on an ex-post basis. This evalu-
ation forms part of the standard policy evaluation cycle.

Special Report No 1/2013  –  Has the EU support to 
the food-processing industry been effective and 
efficient in adding value to agricultural products?
The forthcoming synthesis of MTEs is expected to give an 
overview of the available information on measure 123 with 
respect to the implementation of the measure and on the 
achievement of the result indicators .

The relevant information contained in the Leader+ ex-post 
evaluation will provide the basis for the Commission’s fur-
ther work on the Leader approach.

In  the  ex-post  evaluation,  Member  States  should  again 
report  on  the  achievement  of  the  result  indicators,  this 
time covering the full programming period.

Special Report No 6/2013  –  Have the Member 
States and the Commission achieved value for 
money with the measures for diversifying the 
rural economy?
For the next programming period the Commission is devel-
oping a new version of the CMEF, together with the Mem -
ber States, which will allow the assessment, for each RDP, 
of progress in implementation against commonly defined 
target indicators for the priorities and focus areas selected 
for the programme. At the basis is an indicator plan which 
for each focus area sets the target and the planned out-
puts and expenditure for the measures that will be used 
to achieve the targets and objectives of the programme. 
The indicator plan represents more accurately the quanti-
fied intervention logic for each individual programme than 
does the current rigid axis structure.

Special Report No 7/2011  –  Is agri-environment 
support well designed and managed?
The Commission has verified that the objectives are spe-
cific, meas¬urable, achievable, realistic and timely to the 
extent possible during the procedure of the programmes’ 
approval. The Commission will further underline the need 
for  Member  States  to  ensure  consist¬ency  of  the  sub-
measures’ objectives with the general agri-environmental 
objectives. The actual environmental benefits of sub-meas-
ures can only be determined after a certain period of time 
of their implemen¬tation.

Special Report No 8/2012  –  Targeting of aid for 
the modernisation of agricultural holdings
The current ‘learning by doing’ experience shows that the 
definition of result indicators for improving competitive -
ness is challenging and the Commission, Member States 
and various stakeholders have devoted much effort to con-
tinuously improve methodology, reliability and compara-
bility. The monitoring and evaluation system is subject to 
review by the Commission and Member States in order to 
improve its efficiency and effectiveness, especially in terms 
of reliability and comparability, and foster ownership by 
sharing good practices and building capacity.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSION60

Annual Report 2011, Chapter 10
The Commission services complied with the obligations 
set up in the Financial Regulations and in the instructions 
for the drafting of the Annual Activity Reports (AARs). The 
Commission strengthened the standing instructions for the 
2011 AARs, in terms of economy, efficiency and effective-
ness, and requested the directorates-general to add indica-
tors on voluntary basis on sound financial management in 
Part 3. The Commission is confident that AARs as a whole 
provide sufficient information on results/outcomes.

Most of the mid-term evaluations did not provide quanti-
fied impact stating that the mid-term evaluations were car-
ried out too early in the programming period in order to 
be able to capture measurable effect and impact. Impacts 
are dealing with long-term processes.

Special Report No 12/2013 – Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

REPLY OF THE  COMMISSIONEuropean Court of Auditors

Special Report No 12/2013
Can the Commission and Member States show that the EU budget allocated to the rural development policy is well spent?

Luxembourg: Publications Office of the European Union

2013 — 60 pp. — 21 × 29.7 cm

ISBN 978-92-9241-428-3
doi:10.2865/42819

HOW TO OBTAIN EU PUBLICATIONS

Free publications:

•	

•	

	one	copy: 
via EU Bookshop (http://bookshop.europa.eu);

	more	than	one	copy	or	posters/maps: 
from the European Union’s representations (http://ec.europa.eu/represent_en.htm);  
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm);  
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or 
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*).

(*) 

The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:

•	

via	EU	Bookshop	(http://bookshop.europa.eu).

Priced subscriptions:

•	

	via	one	of	the	sales	agents	of	the	Publications	Office	of	the	European	Union 
(http://publications.europa.eu/others/agents/index_en.htm).

Q
J
-
A
B
-
1
3
-
0
1
2
-
E
N

-
C

FOR THE 2007–13 PROGRAMMING PERIOD, THE EU HAS ALLOCATED ALMOST 

100 BILLION EURO TO ACHIEVE ITS RURAL DEVELOPMENT OBJECTIVES. IN 

THIS  REPORT, THE  COURT  FINDS THAT THERE  IS  A LACK  OF  ASSURANCE 

THAT THE MONEY HAS BEEN WELL SPENT: THE COMMISSION AND MEMBER 

STATES HAVE NOT SUFFICIENTLY ACCOUNTED FOR THEIR PERFORMANCE IN 

ACHIEVING VALUE FOR MONEY WITH THE EU’S BUDGET. DESPITE SUPPORT 

FROM THE COMMISSION, MEMBER STATES’ MONITORING AND EVALUATION 

HAS NOT BEEN RELIABLE, CONSISTENT AND RELEVANT ENOUGH TO SHOW 

WHAT HAS BEEN ACHIEVED IN RELATION TO THE POLICY OBJECTIVES. THE 

AVAILABLE INFORMATION HAS NOT BEEN USED TO MAKE THE RURAL DE-

VELOPMENT MEASURES MORE EFFECTIVE AND EFFICIENT, WITH THE FOCUS 

BEING ON SPENDING THE BUDGET RATHER THAN EFFICIENTLY ACHIEVING 

RESULTS. ULTIMATELY, THE CURRENT FRAMEWORK HAS FAILED TO PROVIDE 

THE INFORMATION NEEDED IN TIME TO INFORM DECISIONS ON THE MOST 

EFFEC TIVE AND EFFICIENT MEASURES FOR THE 2014–20 PROGRAMMING 

PERIOD.

EUROPEAN COURT OF AUDITORS