# C_2007124EN.01000101.xml

| 5.6.2007 | EN | Official Journal of the European Union | C 124/1 |
| --- | --- | --- | --- |

5.6.2007

EN

Official Journal of the European Union

C 124/1

---

SPECIAL REPORT No 1/2007

SPECIAL REPORT N

concerning the implementation of the mid-term processes on the Structural Funds 2000-2006 together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, EC)

(2007/C 124/01)

TABLE OF CONTENTS

| I-V | SUMMARY |
| --- | --- |

I-V

| 1-12 | INTRODUCTION |
| --- | --- |

1-12

| 1-2 | The Structural Funds |
| --- | --- |

1-2

| 3-6 | The Structural Funds 2000-2006 |
| --- | --- |

3-6

| 7-10 | Evaluation and appraisal of the Structural Funds |
| --- | --- |

7-10

| 11-12 | Mid-term evaluation and review and the allocation of the performance reserve |
| --- | --- |

11-12

| 13-15 | SCOPE OF THE AUDIT |
| --- | --- |

13-15

| 16-17 | TIMETABLE |
| --- | --- |

16-17

| 18-27 | ASSESSMENT OF MID-TERM EVALUATION |
| --- | --- |

18-27

| 20 | The results of the evaluations |
| --- | --- |

20

| 21-22 | Commission assessment of evaluations |
| --- | --- |

21-22

| 23-27 | Assessment by the Court |
| --- | --- |

23-27

| 28-36 | ALLOCATION OF PERFORMANCE RESERVE |
| --- | --- |

28-36

| 30-36 | Assessments by Member States and allocation of the performance reserve |
| --- | --- |

30-36

| 37-44 | IMPACT OF MID-TERM REVISION |
| --- | --- |

37-44

| 45-46 | ARRANGEMENTS FOR 2007-2013 |
| --- | --- |

45-46

| 47-52 | CONCLUSIONS |
| --- | --- |

47-52

| 53 | RECOMMENDATIONS |
| --- | --- |

53

The Commission's replies

The Commission's replies

SUMMARY

| I. | The Structural Funds Regulation provided for a series of steps roughly half way through the 2000-2006 programme period:






  — | a mid-term evaluation of programmes; |
| --- | --- | --- |

I.

The Structural Funds Regulation provided for a series of steps roughly half way through the 2000-2006 programme period:

—

a mid-term evaluation of programmes;

| — | allocation of the performance reserve; and |
| --- | --- |

—

allocation of the performance reserve; and

| — | mid-term revision of expenditure plans. |
| --- | --- |

—

mid-term revision of expenditure plans.

| II. | The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds: in particular, whether






  — | mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure; |
| --- | --- | --- |

II.

The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds: in particular, whether

—

mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure;

mid-term evaluations

| — | the performance reserve was allocated so as to favour effective and well-managed spending programmes; |
| --- | --- |

—

the performance reserve was allocated so as to favour effective and well-managed spending programmes;

performance reserve

| — | the mid-term revision updated priorities and improved the targeting of spending programmes. |
| --- | --- |

—

the mid-term revision updated priorities and improved the targeting of spending programmes.

mid-term revision

| III. | The Court concluded as follows:






  — | Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes. |
| --- | --- | --- |

III.

The Court concluded as follows:

—

Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes.

a fortiori

| — | The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption. |
| --- | --- |

—

The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption.

| — | In a number of cases the mid term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant impact of mid-term revision was to give priority to activities where money could be spent. |
| --- | --- |

—

In a number of cases the mid term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant impact of mid-term revision was to give priority to activities where money could be spent.

| IV. | The Court considers that it remains appropriate to include, in expenditure programmes such as the Structural Funds, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds, which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision:






  — | Expenditure should be efficient, effective and economic. On the other hand, the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as limits on spending. Absorption is an implicit objective for EU spending on cohesion. |
| --- | --- | --- |

IV.

The Court considers that it remains appropriate to include, in expenditure programmes such as the Structural Funds, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds, which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision:

—

Expenditure should be efficient, effective and economic. On the other hand, the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as limits on spending. Absorption is an implicit objective for EU spending on cohesion.

| — | The N + 2 rule provides a discipline to ensure that spending is well planned, but also creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease. |
| --- | --- |

—

The N + 2 rule provides a discipline to ensure that spending is well planned, but also creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease.

| — | Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure. |
| --- | --- |

—

Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure.

| V. | The Court makes a number of recommendations designed to increase the usefulness of mid-term review and revision in the context of the more flexible arrangements for the Structural Funds which will apply for the 2007-2013 programme period:






  — | speedy and efficient closure of the 2000-2006 programmes; |
| --- | --- | --- |

V.

The Court makes a number of recommendations designed to increase the usefulness of mid-term review and revision in the context of the more flexible arrangements for the Structural Funds which will apply for the 2007-2013 programme period:

—

speedy and efficient closure of the 2000-2006 programmes;

| — | better performance indicators; |
| --- | --- |

—

better performance indicators;

| — | careful choice of timing for evaluation of expenditure (taking advantage of the increased flexibility of the reformed system); |
| --- | --- |

—

careful choice of timing for evaluation of expenditure (taking advantage of the increased flexibility of the reformed system);

| — | shorter and quicker mid-term revision; |
| --- | --- |

—

shorter and quicker mid-term revision;

| — | timely and relevant guidance from the Commission; |
| --- | --- |

—

timely and relevant guidance from the Commission;

| — | special attention to the programmes and management arrangements for expenditure for EU policy priorities. |
| --- | --- |

—

special attention to the programmes and management arrangements for expenditure for EU policy priorities.

INTRODUCTION

The Structural Funds

| 1. | The Structural Funds are the Union's principal policy instrument in support of the Treaty objectives of economic and social cohesion. |
| --- | --- |

1.

The Structural Funds are the Union's principal policy instrument in support of the Treaty objectives of economic and social cohesion.

| 2. | There are four Structural Funds: the European Regional Development Fund (ERDF), the European Social Fund (ESF), the European Agricultural Guidance and Guarantee Fund, ‘Guidance section’ (EAGGF-Guidance), and the Financial Instrument for Fisheries Guidance (FIFG). Expenditure is planned and carried out on a multiannual basis in ‘programme periods’. It supports investment in physical and human capital within Member States. Member States pay part of the cost of (co-finance) the investment projects concerned. |
| --- | --- |

2.

There are four Structural Funds: the European Regional Development Fund (ERDF), the European Social Fund (ESF), the European Agricultural Guidance and Guarantee Fund, ‘Guidance section’ (EAGGF-Guidance), and the Financial Instrument for Fisheries Guidance (FIFG). Expenditure is planned and carried out on a multiannual basis in ‘programme periods’. It supports investment in physical and human capital within Member States. Member States pay part of the cost of (co-finance) the investment projects concerned.

The Structural Funds 2000-2006

| 3. | The European Council agreed at its meeting in Berlin (March 1999) that 195 billion euro (219 billion euro in 2005 prices) should be made available from the Structural Funds for the period 2000-2006. Over 90 % of spending from the funds was to be used in pursuit of three objectives:






  — | Objective 1: promoting the development and structural adjustment of regions whose development is lagging behind (regions whose per capita GDP is less than 75 % of the Community average). About 70 % of total spending was to be devoted to Objective 1; |
| --- | --- | --- |

3.

The European Council agreed at its meeting in Berlin (March 1999) that 195 billion euro (219 billion euro in 2005 prices) should be made available from the Structural Funds for the period 2000-2006. Over 90 % of spending from the funds was to be used in pursuit of three objectives:

—

Objective 1: promoting the development and structural adjustment of regions whose development is lagging behind (regions whose per capita GDP is less than 75 % of the Community average). About 70 % of total spending was to be devoted to Objective 1;

| — | Objective 2: supporting the economic and social conversion of areas (outside Objective 1 regions) facing structural difficulties (industrial, rural, urban areas and areas dependent on fisheries); |
| --- | --- |

—

Objective 2: supporting the economic and social conversion of areas (outside Objective 1 regions) facing structural difficulties (industrial, rural, urban areas and areas dependent on fisheries);

| — | Objective 3: supporting (outside Objective 1 regions) the adaptation and modernisation of policies and systems of education, training and employment. |
| --- | --- |

—

Objective 3: supporting (outside Objective 1 regions) the adaptation and modernisation of policies and systems of education, training and employment.

| 4. | The planned expenditure totals agreed at Berlin were allocated to some 400 national and regional programmes (1) through a complex 'programming' process, whose main elements were as follows (2):






  — | The Commission established (3), for the initial programming exercise and subsequently for the mid-term review, indicative programming guidelines in relation to the objectives of the Funds; |
| --- | --- | --- |

4.

The planned expenditure totals agreed at Berlin were allocated to some 400 national and regional programmes (1) through a complex 'programming' process, whose main elements were as follows (2):

1

2

—

The Commission established (3), for the initial programming exercise and subsequently for the mid-term review, indicative programming guidelines in relation to the objectives of the Funds;

3

| — | Member States negotiated with the Commission ‘Community Support Frameworks’ (CSF), which set out the strategy and priorities for action of the Funds and the Member State; |
| --- | --- |

—

Member States negotiated with the Commission ‘Community Support Frameworks’ (CSF), which set out the strategy and priorities for action of the Funds and the Member State;

| — | Each CSF was implemented by means of ‘operational programmes’ (OP) (4) approved by the Commission; |
| --- | --- |

—

Each CSF was implemented by means of ‘operational programmes’ (OP) (4) approved by the Commission;

4

| — | Regions or Member States then approved individual projects to implement operational programmes; |
| --- | --- |

—

Regions or Member States then approved individual projects to implement operational programmes;

| — | There was provision for programmes to be revised midway through the programme period (mid-term revision) (5). |
| --- | --- |

—

There was provision for programmes to be revised midway through the programme period (mid-term revision) (5).

5

| 5. | Expenditure was to be supervised by ‘Monitoring Committees’ set up by the Member State concerned and including Commission representatives (6) (in an advisory capacity). |
| --- | --- |

5.

Expenditure was to be supervised by ‘Monitoring Committees’ set up by the Member State concerned and including Commission representatives (6) (in an advisory capacity).

6

| 6. | Two innovations of relevance to this report were introduced into the management of the Structural Funds for the 2000-2006 programme period: a performance reserve and the ‘N + 2’ rule (7). See boxes 1 and 2.

  Box 1The performance reserveThe Structural Fund Regulations for the 2000-2006 programme period required the Commission, when allocating Structural Fund resources, to withhold 4 % from each operational programme. In order to encourage the efficient and effective use of Community funds, this reserve was to be allocated among and within each Member State's operational programmes in 2004, on the basis of the comparative success of these programmes.The performance reserve was a limited measure. Funds could not be switched among Member States nor between progammes coming under different objectives. Nor could any part of the reserve be withheld further. |
| --- | --- |

6.

Two innovations of relevance to this report were introduced into the management of the Structural Funds for the 2000-2006 programme period: a performance reserve and the ‘N + 2’ rule (7). See boxes 1 and 2.

7

Box 1The performance reserveThe Structural Fund Regulations for the 2000-2006 programme period required the Commission, when allocating Structural Fund resources, to withhold 4 % from each operational programme. In order to encourage the efficient and effective use of Community funds, this reserve was to be allocated among and within each Member State's operational programmes in 2004, on the basis of the comparative success of these programmes.The performance reserve was a limited measure. Funds could not be switched among Member States nor between progammes coming under different objectives. Nor could any part of the reserve be withheld further.

Box 1The performance reserveThe Structural Fund Regulations for the 2000-2006 programme period required the Commission, when allocating Structural Fund resources, to withhold 4 % from each operational programme. In order to encourage the efficient and effective use of Community funds, this reserve was to be allocated among and within each Member State's operational programmes in 2004, on the basis of the comparative success of these programmes.The performance reserve was a limited measure. Funds could not be switched among Member States nor between progammes coming under different objectives. Nor could any part of the reserve be withheld further.

Box 2The N + 2 ruleThe N + 2 rule is designed to prevent an excessive build up of unused commitments for an operational programme, which can complicate closure and interfere with successor programmes, by providing an incentive to ‘spend the money’ in line with targets. It provides that if a commitment has not led to payments within two years of being entered in the budget (8), the commitment is cancelled (decommitted) and not replaced, i.e. spending falls short of the totals agreed at the Berlin European Council.

Box 2The N + 2 ruleThe N + 2 rule is designed to prevent an excessive build up of unused commitments for an operational programme, which can complicate closure and interfere with successor programmes, by providing an incentive to ‘spend the money’ in line with targets. It provides that if a commitment has not led to payments within two years of being entered in the budget (8

Evaluation and appraisal of the Structural Funds

| 7. | The Financial Regulation, which sets out rules for the management of the Community budget, provides that EU spending must be economical, efficient and effective. Objectives must be set for all sectors of activity covered by the budget, and their achievement must be monitored by performance indicators. Expenditure must also be subject to ex ante and ex post evaluations (9). |
| --- | --- |

7.

The Financial Regulation, which sets out rules for the management of the Community budget, provides that EU spending must be economical, efficient and effective. Objectives must be set for all sectors of activity covered by the budget, and their achievement must be monitored by performance indicators. Expenditure must also be subject to ex ante and ex post evaluations (9).

ex ante

ex post

9

| 8. | The Structural Funds Regulations for the 2000-2006 programme period set out how these principles were to be applied to the Structural Funds. In particular there were to be:






  — | an ex ante evaluation, for each CSF or OP, carried out within Member States, to provide a basis for preparing the development plans (10); |
| --- | --- | --- |

8.

The Structural Funds Regulations for the 2000-2006 programme period set out how these principles were to be applied to the Structural Funds. In particular there were to be:

—

an ex ante evaluation, for each CSF or OP, carried out within Member States, to provide a basis for preparing the development plans (10);

ex ante

10

| — | a mid-term evaluation, carried out by an independent assessor under the responsibility of the authority managing the expenditure, and in cooperation with the Commission and the Member State; its purpose was to examine, in the light of the ex ante evaluation, initial results, their relevance, the achievement of targets, the use of financial resources, monitoring and implementation (11); |
| --- | --- |

—

a mid-term evaluation, carried out by an independent assessor under the responsibility of the authority managing the expenditure, and in cooperation with the Commission and the Member State; its purpose was to examine, in the light of the ex ante evaluation, initial results, their relevance, the achievement of targets, the use of financial resources, monitoring and implementation (11);

ex ante

11

| — | an ex post evaluation, covering the use of resources, and the effectiveness and efficiency of intervention, carried out by the Commission in collaboration with the Member State and the authority managing the expenditure concerned (12). |
| --- | --- |

—

an ex post evaluation, covering the use of resources, and the effectiveness and efficiency of intervention, carried out by the Commission in collaboration with the Member State and the authority managing the expenditure concerned (12).

ex post

12

| 9. | The Court audited the arrangements for ex ante evaluation and programming of the 2000-2006 generation of Structural Funds in its Special Report No 7/2003 (13). The Court noted, inter alia, that:






  — | budgetary allocations were determined less by a well-established development strategy and the effectiveness of the Structural Funds than by maximising likely take-up of funding, |
| --- | --- | --- |

9.

The Court audited the arrangements for ex ante evaluation and programming of the 2000-2006 generation of Structural Funds in its Special Report No 7/2003 (13). The Court noted, inter alia, that:

ex ante

13

inter alia

—

budgetary allocations were determined less by a well-established development strategy and the effectiveness of the Structural Funds than by maximising likely take-up of funding,

| — | ex ante evaluation had little impact on the programming process. Evaluation was confined to justifying the programming as proposed by Member States and only rarely considered its validity and the underlying strategy. There was no critical appraisal or analysis of alternative strategies. |
| --- | --- |

—

ex ante evaluation had little impact on the programming process. Evaluation was confined to justifying the programming as proposed by Member States and only rarely considered its validity and the underlying strategy. There was no critical appraisal or analysis of alternative strategies.

ex ante

| 10. | The Court has also audited the Commission's ex post evaluations of most Structural Fund expenditure for the previous programme period (1994-1999) (14). The Court found a number of significant weaknesses in the evaluations carried out and in the Commission's oversight of them; and recommended ensuring that available data and allocated resources were in line with set objectives, which should be realistic, and some changes in scope and approach for future evaluation exercises. |
| --- | --- |

10.

The Court has also audited the Commission's ex post evaluations of most Structural Fund expenditure for the previous programme period (1994-1999) (14). The Court found a number of significant weaknesses in the evaluations carried out and in the Commission's oversight of them; and recommended ensuring that available data and allocated resources were in line with set objectives, which should be realistic, and some changes in scope and approach for future evaluation exercises.

ex post

14

Mid-term evaluation and review and the allocation of the performance reserve

| 11. | The Structural Funds Regulation thus provided for a series of procedures roughly half way through the 2000-2006 programme period:






  — | Member States were required to carry out a mid-term evaluation of programmes and to submit the results to the Commission by the end of 2003 (15). |
| --- | --- | --- |

11.

The Structural Funds Regulation thus provided for a series of procedures roughly half way through the 2000-2006 programme period:

—

Member States were required to carry out a mid-term evaluation of programmes and to submit the results to the Commission by the end of 2003 (15).

15

| — | By the same date, under the arrangements for the performance reserve, they were required to assess the performance of operational programmes (16). |
| --- | --- |

—

By the same date, under the arrangements for the performance reserve, they were required to assess the performance of operational programmes (16).

16

| — | By 31 March 2004 the Commission was to allocate the performance reserve on the basis of a proposal from the Member State concerned (17). |
| --- | --- |

—

By 31 March 2004 the Commission was to allocate the performance reserve on the basis of a proposal from the Member State concerned (17).

17

| — | Spending plans could, if necessary, be subject to a subsequent mid-term revision (18). |
| --- | --- |

—

Spending plans could, if necessary, be subject to a subsequent mid-term revision (18).

18

| 12. | The Structural Funds Regulation provided that mid-term evaluations were intended as an input into both the allocation of the performance reserve and any mid-term revision of programmes: the Commission was therefore under an obligation to examine whether the MTEs were suitable for those purposes (19). |
| --- | --- |

12.

The Structural Funds Regulation provided that mid-term evaluations were intended as an input into both the allocation of the performance reserve and any mid-term revision of programmes: the Commission was therefore under an obligation to examine whether the MTEs were suitable for those purposes (19).

19

SCOPE OF THE AUDIT

| 13. | The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds. |
| --- | --- |

13.

The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds.

| 14. | In particular, the audit aimed to establish whether






  — | mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure; |
| --- | --- | --- |

14.

In particular, the audit aimed to establish whether

—

mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure;

mid-term evaluations

| — | the performance reserve was allocated so as to favour effective and well-managed spending programmes; |
| --- | --- |

—

the performance reserve was allocated so as to favour effective and well-managed spending programmes;

performance reserve

| — | the mid-term revision updated priorities and improved the targeting of spending programmes. |
| --- | --- |

—

the mid-term revision updated priorities and improved the targeting of spending programmes.

mid-term revision

| 15. | The audit concentrated on the ERDF and ESF, as the main components of the Structural Funds. In the course of the audit, the Court:






  — | reviewed Commission guidelines for the three processes, |
| --- | --- | --- |

15.

The audit concentrated on the ERDF and ESF, as the main components of the Structural Funds. In the course of the audit, the Court:

—

reviewed Commission guidelines for the three processes,

| — | examined evaluation reports and proposals for use of the performance reserve and for mid-term revision for 5 CSFs and 20 operational programmes/single programming documents (20), |
| --- | --- |

—

examined evaluation reports and proposals for use of the performance reserve and for mid-term revision for 5 CSFs and 20 operational programmes/single programming documents (20),

20

| — | considered the Commission's own assessment of the evaluation exercise, |
| --- | --- |

—

considered the Commission's own assessment of the evaluation exercise,

| — | analysed the allocation of the performance reserve and the impact of mid-term revision. |
| --- | --- |

—

analysed the allocation of the performance reserve and the impact of mid-term revision.

TIMETABLE

| 16. | These 'mid-term' processes occurred at an early stage in the 2000-2006 programme period in terms of expenditure undertaken.






  — | Community budgets since 2000 have been consistent with the Berlin European Council agreement that the Union should enter into commitments during 2000-2006 to spend 195 billion euro from the Structural funds. As is normal for multiannual programmes, however, payments — spending the money — have lagged behind commitments. |
| --- | --- | --- |

16.

These 'mid-term' processes occurred at an early stage in the 2000-2006 programme period in terms of expenditure undertaken.

—

Community budgets since 2000 have been consistent with the Berlin European Council agreement that the Union should enter into commitments during 2000-2006 to spend 195 billion euro from the Structural funds. As is normal for multiannual programmes, however, payments — spending the money — have lagged behind commitments.

| — | Operational programmes were adopted more slowly than planned (see Special Report No 7/2003, paragraphs 18-24): most of them in 2001, some only in 2002. |
| --- | --- |

—

Operational programmes were adopted more slowly than planned (see Special Report No 7/2003, paragraphs 18-24): most of them in 2001, some only in 2002.

| — | In 2000 and 2001 the primary concern of Member States and regions was to complete the implementation of the 1994-1999 generation of Structural Fund programmes, which slowed down the implementation of new programmes. |
| --- | --- |

—

In 2000 and 2001 the primary concern of Member States and regions was to complete the implementation of the 1994-1999 generation of Structural Fund programmes, which slowed down the implementation of new programmes.

| — | Since Member States had to submit the mid-term evaluation by the end of 2003, evaluators were obliged to base their assessment on information and data available at the end of 2002 or, at the latest, autumn 2003. |
| --- | --- |

—

Since Member States had to submit the mid-term evaluation by the end of 2003, evaluators were obliged to base their assessment on information and data available at the end of 2002 or, at the latest, autumn 2003.

| 17. | At the end of 2002, almost the entire Berlin totals had been allocated to programmes, but only about 20 billion euro (about 1/10 of the total) had been spent. Even at the end of 2003, only about 60 billion euro (about one third) had been spent. |
| --- | --- |

17.

At the end of 2002, almost the entire Berlin totals had been allocated to programmes, but only about 20 billion euro (about 1/10 of the total) had been spent. Even at the end of 2003, only about 60 billion euro (about one third) had been spent.

ASSESSMENT OF MID-TERM EVALUATION

| 18. | As explained in paragraph 8, the Structural Funds Regulation itself set out limited aims for the mid-term evaluation. A Commission working paper (Working Paper No 8: The Mid-Term Evaluation of Structural Fund Intervention, 5.12.2000) provided more extensive guidance and more ambitious goals to Member States for the mid-term evaluation. The core of the evaluation was to be the assessment of how well the form of assistance was performing in reality, judged on the basis of its continued relevance and the effectiveness and efficiency of its implementation. The working paper set out the evaluation questions and the content of an evaluation report (See box 3).

  Box 3Role of the MTE according to Working Paper No 8





  — | To assess whether the overall form of assistance remains the appropriate means to address the issues confronting the region or sector. |
| --- | --- | --- |

18.

As explained in paragraph 8, the Structural Funds Regulation itself set out limited aims for the mid-term evaluation. A Commission working paper (Working Paper No 8: The Mid-Term Evaluation of Structural Fund Intervention, 5.12.2000) provided more extensive guidance and more ambitious goals to Member States for the mid-term evaluation. The core of the evaluation was to be the assessment of how well the form of assistance was performing in reality, judged on the basis of its continued relevance and the effectiveness and efficiency of its implementation. The working paper set out the evaluation questions and the content of an evaluation report (See box 3).

Box 3Role of the MTE according to Working Paper No 8





  —

To assess whether the overall form of assistance remains the appropriate means to address the issues confronting the region or sector.

| — | To review whether the strategic axes, priorities and objectives are coherent and still relevant, how far progress has been made towards the achievement of these objectives and the extent to which they can actually be achieved. |
| --- | --- |

—

To review whether the strategic axes, priorities and objectives are coherent and still relevant, how far progress has been made towards the achievement of these objectives and the extent to which they can actually be achieved.

| — | To assess the quantification of objectives, specifically the extent to which they have facilitated monitoring and evaluation. |
| --- | --- |

—

To assess the quantification of objectives, specifically the extent to which they have facilitated monitoring and evaluation.

| — | To assess the extent to which horizontal priorities — equal opportunities and the environment in particular — have been integrated into the forms of assistance. |
| --- | --- |

—

To assess the extent to which horizontal priorities — equal opportunities and the environment in particular — have been integrated into the forms of assistance.

| — | To analyse the adequacy of the implementation and monitoring arrangements. |
| --- | --- |

—

To analyse the adequacy of the implementation and monitoring arrangements.

| — | To present the results against the indicators agreed for the performance reserve. |
| --- | --- |

—

To present the results against the indicators agreed for the performance reserve.

| 19. | The same Commission Working Paper indicated that the scale of the evaluation (and therefore its cost) should be proportional to the scale of the programme — giving the example that macro-economic modelling would be appropriate for large Objective 1 CSFs, but not for small Objectives 2 or 3 programmes. The fees for the evaluation were financed partly from EU funds. The final cost of evaluation fees amounted to 40 million euro, and thus remained below 0,1 % of annual expenditure on the Structural Funds. (In addition the Commission incurred administrative costs in dealing with evaluations: these are not readily quantifiable). |
| --- | --- |

19.

The same Commission Working Paper indicated that the scale of the evaluation (and therefore its cost) should be proportional to the scale of the programme — giving the example that macro-economic modelling would be appropriate for large Objective 1 CSFs, but not for small Objectives 2 or 3 programmes. The fees for the evaluation were financed partly from EU funds. The final cost of evaluation fees amounted to 40 million euro, and thus remained below 0,1 % of annual expenditure on the Structural Funds. (In addition the Commission incurred administrative costs in dealing with evaluations: these are not readily quantifiable).

The results of the evaluations

| 20. | Almost all evaluations concluded that the initial development strategy remained relevant but that it was too early to assess effectiveness. The Commission summary used the findings of the evaluations in its report ‘The Mid-Term Evaluation in Objective 1 and 2 Regions: Growing Evaluation Capacity’ (November 2004). The Commission's summary is set out in Box 4.

  Box 4Evaluators' response to the evaluation questions





  — | In nearly all cases the evaluators concluded that the strategies adopted in the programmes were still appropriate. |
| --- | --- | --- |

20.

Almost all evaluations concluded that the initial development strategy remained relevant but that it was too early to assess effectiveness. The Commission summary used the findings of the evaluations in its report ‘The Mid-Term Evaluation in Objective 1 and 2 Regions: Growing Evaluation Capacity’ (November 2004). The Commission's summary is set out in Box 4.

Box 4Evaluators' response to the evaluation questions





  —

In nearly all cases the evaluators concluded that the strategies adopted in the programmes were still appropriate.

were still appropriate

| — | It was too early to assess effectiveness in many cases, given the late or slow start-up of programmes. A difficulty was presented by the monitoring system which often required revision of the definition of indicators and their targets. |
| --- | --- |

—

It was too early to assess effectiveness in many cases, given the late or slow start-up of programmes. A difficulty was presented by the monitoring system which often required revision of the definition of indicators and their targets.

too early to assess effectiveness

| — | It was likewise too early to measure impact, although evaluations did in some cases recommend the revision of targets based on the experience of the early years of implementation. Macro economic modelling undertaken for large CSFs suggests that programmes are on course to achieve their global objectives. |
| --- | --- |

—

It was likewise too early to measure impact, although evaluations did in some cases recommend the revision of targets based on the experience of the early years of implementation. Macro economic modelling undertaken for large CSFs suggests that programmes are on course to achieve their global objectives.

too early to measure impact

| — | Efficiency was more often defined as financial absorption rather than the cost per output or result achieved. Financial absorption is clearly improved (i.e. money was spent more quickly than in the past), driven by the requirements of the N + 2 rule. |
| --- | --- |

—

Efficiency was more often defined as financial absorption rather than the cost per output or result achieved. Financial absorption is clearly improved (i.e. money was spent more quickly than in the past), driven by the requirements of the N + 2 rule.

Financial absorption is clearly improved (i.e. money was spent more quickly than in the past)

| — | Evaluations found an urgent need to quantify objectives better. |
| --- | --- |

—

Evaluations found an urgent need to quantify objectives better.

need to quantify objectives better

| — | Implementation systems were found to fulfil the regulatory requirements and to be significantly improved on the past. |
| --- | --- |

—

Implementation systems were found to fulfil the regulatory requirements and to be significantly improved on the past.

Implementation systems

significantly improved

Source:

Commission assessment of evaluations

| 21. | Member States, monitoring committees and the Commission all played a role in evaluating the quality of evaluations. The services of the Commission examined the evaluation reports at draft and final stage. The Commission had to operate within the ambitious deadlines set by the Regulation and assess the evaluation reports on about 400 programmes simultaneously from January to March 2004. The Commission hired outside evaluators in addition to its own staff in order to finish the task on time. |
| --- | --- |

21.

Member States, monitoring committees and the Commission all played a role in evaluating the quality of evaluations. The services of the Commission examined the evaluation reports at draft and final stage. The Commission had to operate within the ambitious deadlines set by the Regulation and assess the evaluation reports on about 400 programmes simultaneously from January to March 2004. The Commission hired outside evaluators in addition to its own staff in order to finish the task on time.

| 22. | The Commission review of the findings of evaluators (21), is set out in ‘Growing Evaluation Capacity’. The Commission found that:






  — | evaluations were in general based upon appropriate methodology; |
| --- | --- | --- |

22.

The Commission review of the findings of evaluators (21), is set out in ‘Growing Evaluation Capacity’. The Commission found that:

21

—

evaluations were in general based upon appropriate methodology;

| — | poor monitoring systems had hampered many evaluations; |
| --- | --- |

—

poor monitoring systems had hampered many evaluations;

| — | primary research into the reality of programmes required improvement; |
| --- | --- |

—

primary research into the reality of programmes required improvement;

| — | some evaluations were weak on analysis, or evaluations often did not produce sufficiently operational conclusions and recommendations; |
| --- | --- |

—

some evaluations were weak on analysis, or evaluations often did not produce sufficiently operational conclusions and recommendations;

| — | results of economic modelling, generally limited to larger programmes, could not be compared because of great variations in the approaches applied; |
| --- | --- |

—

results of economic modelling, generally limited to larger programmes, could not be compared because of great variations in the approaches applied;

| — | some evaluators overemphasised financial analysis, with a limited focus on what was being achieved by the programmes because of the unavailability of monitoring data, but also because of the strong focus on spending due to the demands of N + 2. This led to recommendations which were purely finance-driven, rather than reviewing the strategic implications of such reallocations (22). |
| --- | --- |

—

some evaluators overemphasised financial analysis, with a limited focus on what was being achieved by the programmes because of the unavailability of monitoring data, but also because of the strong focus on spending due to the demands of N + 2. This led to recommendations which were purely finance-driven, rather than reviewing the strategic implications of such reallocations (22).

22

Assessment by the Court

| 23. | In its audit, the Court identified a number of problems faced by, and difficulties identified by, evaluators in giving a clear answer as to whether the overall form of assistance remained appropriate to address the issues confronting the region or sector, and whether objectives could be achieved.






  — | All evaluators had to cope with the problem that only limited spending had been made at the time of evaluation. |
| --- | --- | --- |

23.

In its audit, the Court identified a number of problems faced by, and difficulties identified by, evaluators in giving a clear answer as to whether the overall form of assistance remained appropriate to address the issues confronting the region or sector, and whether objectives could be achieved.

—

All evaluators had to cope with the problem that only limited spending had been made at the time of evaluation.

| — | Almost half of the evaluators of the programmes which the Court examined identified important issues of concern linked to concrete development needs and proposed stronger emphasis on certain policy fields, especially linked to the Lisbon and Gothenburg strategies. |
| --- | --- |

—

Almost half of the evaluators of the programmes which the Court examined identified important issues of concern linked to concrete development needs and proposed stronger emphasis on certain policy fields, especially linked to the Lisbon and Gothenburg strategies.

| — | Other evaluators drew attention to the intrinsic difficulty of evaluation, such as the difficulty of determining the impact of additional investment in road construction or of establishing the impact of ESF intervention on net employment levels. For example commenting on investment in roads, the German evaluator suggested that further investment in economies with good existing infrastructure provision could even lead to slower growth than without it (23). The evaluators for the UK and Gibraltar programmes commented that, when considering the employment impact of the ESF, only gross job creation could be monitored. |
| --- | --- |

—

Other evaluators drew attention to the intrinsic difficulty of evaluation, such as the difficulty of determining the impact of additional investment in road construction or of establishing the impact of ESF intervention on net employment levels. For example commenting on investment in roads, the German evaluator suggested that further investment in economies with good existing infrastructure provision could even lead to slower growth than without it (23). The evaluators for the UK and Gibraltar programmes commented that, when considering the employment impact of the ESF, only gross job creation could be monitored.

23

| 24. | The Commission's summary report ‘Growing Evaluation Capacity’ gave less attention to these problems and difficulties, than to more positive evaluation outcomes. |
| --- | --- |

24.

The Commission's summary report ‘Growing Evaluation Capacity’ gave less attention to these problems and difficulties, than to more positive evaluation outcomes.

| 25. | Some evaluators were also able to identify significant problems in the way programmes were operating. For example:






  — | the evaluator of the Italian OP PONSIL (financial aid for enterprises) estimated a deadweight (24) effect of 50 %; |
| --- | --- | --- |

25.

Some evaluators were also able to identify significant problems in the way programmes were operating. For example:

—

the evaluator of the Italian OP PONSIL (financial aid for enterprises) estimated a deadweight (24) effect of 50 %;

24

| — | in the UK, evaluators commented that the programme had had only very modest success in securing sizeable net movements of individuals into the labour market who were previously outside it, and noted that ESF support tended to reinforce existing patterns of training within companies, in which well-qualified individuals generally received more training; However the report also stated that the programme had been ‘disproportionately’ successful in helping the most disadvantaged beneficiaries find work; |
| --- | --- |

—

in the UK, evaluators commented that the programme had had only very modest success in securing sizeable net movements of individuals into the labour market who were previously outside it, and noted that ESF support tended to reinforce existing patterns of training within companies, in which well-qualified individuals generally received more training; However the report also stated that the programme had been ‘disproportionately’ successful in helping the most disadvantaged beneficiaries find work;

| — | the evaluators of the Austrian Burgenland operational programme observed that the N + 2 rule encouraged higher spending rather than effective allocation; |
| --- | --- |

—

the evaluators of the Austrian Burgenland operational programme observed that the N + 2 rule encouraged higher spending rather than effective allocation;

| — | where evaluators used a macroeconomic model (Germany, Spain, Portugal and Italy) estimates of the impact of the funds (in particular on GDP growth) were often significantly reduced as compared with the ex ante appraisal, suggesting that programmes were unlikely to meet their objectives; |
| --- | --- |

—

where evaluators used a macroeconomic model (Germany, Spain, Portugal and Italy) estimates of the impact of the funds (in particular on GDP growth) were often significantly reduced as compared with the ex ante appraisal, suggesting that programmes were unlikely to meet their objectives;

ex ante

| — | out of the 25 selected CSFs, OPs and SPDs more than half of the MTEs noted weaknesses with regard to project selection and the integration of horizontal principles such as the environment, equal opportunities and the information society. In many cases the evaluations however only described weaknesses without proposing operational recommendations. |
| --- | --- |

—

out of the 25 selected CSFs, OPs and SPDs more than half of the MTEs noted weaknesses with regard to project selection and the integration of horizontal principles such as the environment, equal opportunities and the information society. In many cases the evaluations however only described weaknesses without proposing operational recommendations.

| 26. | In some other respects, however, the Court's audit confirmed the Commission's analysis. Evaluators responded to the range of questions set out in Working Paper No 8, and attempted to assess the continuing coherence and validity of the strategies underlying CSFs and OPs (in most cases re-examining social and economic factors and preparing a revised analysis of programmes' strengths, weaknesses, opportunities and threats). The Court's audit showed that implementation was more robustly assessed than effectiveness, efficiency and impact:






  — | the evaluations contained clear assessments of the physical and financial progress of OPs and CSFs; |
| --- | --- | --- |

26.

In some other respects, however, the Court's audit confirmed the Commission's analysis. Evaluators responded to the range of questions set out in Working Paper No 8, and attempted to assess the continuing coherence and validity of the strategies underlying CSFs and OPs (in most cases re-examining social and economic factors and preparing a revised analysis of programmes' strengths, weaknesses, opportunities and threats). The Court's audit showed that implementation was more robustly assessed than effectiveness, efficiency and impact:

—

the evaluations contained clear assessments of the physical and financial progress of OPs and CSFs;

| — | estimates of effectiveness and impact, on the other hand, were not always carried out, or were carried out in a partial manner (e.g. gross jobs created were in almost all cases presented as equivalent to the employment effects, concerning both ERDF and ESF measures); |
| --- | --- |

—

estimates of effectiveness and impact, on the other hand, were not always carried out, or were carried out in a partial manner (e.g. gross jobs created were in almost all cases presented as equivalent to the employment effects, concerning both ERDF and ESF measures);

| — | deadweight and substitution effects were seldom considered; very often evaluations did not attempt directly to assess these effects; |
| --- | --- |

—

deadweight and substitution effects were seldom considered; very often evaluations did not attempt directly to assess these effects;

| — | efficiency was not always analysed, and where it was, no conclusions or benchmarks were identified, nor recommendations made on policy; |
| --- | --- |

—

efficiency was not always analysed, and where it was, no conclusions or benchmarks were identified, nor recommendations made on policy;

| — | in some cases the validity of the underlying strategy was not properly addressed. |
| --- | --- |

—

in some cases the validity of the underlying strategy was not properly addressed.

| 27. | The Court's overall assessment is that the evaluation process provided a check on the progress being made with programmes and confirmed that implementation systems were improved from the previous generation of Structural Funds. On the other hand, there were significant weaknesses in monitoring, and it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes. |
| --- | --- |

27.

The Court's overall assessment is that the evaluation process provided a check on the progress being made with programmes and confirmed that implementation systems were improved from the previous generation of Structural Funds. On the other hand, there were significant weaknesses in monitoring, and it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes.

a fortiori

ALLOCATION OF PERFORMANCE RESERVE

| 28. | The Regulation from the outset provided a limited number of quantified indicators. The Commission provided guidance to Member States in a working paper issued in 1999 (25) on how to assess programmes against the criteria for allocation of the performance reserve: effectiveness, the quality of management and financial implementation. Performance assessment was to be based on monitoring indicators measuring the mid-term results in relation to their specific initial targets. |
| --- | --- |

28.

The Regulation from the outset provided a limited number of quantified indicators. The Commission provided guidance to Member States in a working paper issued in 1999 (25) on how to assess programmes against the criteria for allocation of the performance reserve: effectiveness, the quality of management and financial implementation. Performance assessment was to be based on monitoring indicators measuring the mid-term results in relation to their specific initial targets.

25

| 29. | The Commission's working paper included examples of indicators (26) to be taken into account in carrying out the review of performance.
  For effectiveness:






  — | a basket of some outputs covering at least half the value of the programme; |
| --- | --- | --- |

29.

The Commission's working paper included examples of indicators (26) to be taken into account in carrying out the review of performance.

26

For effectiveness:

—

a basket of some outputs covering at least half the value of the programme;

| — | comparison of actual and planned results for employment (temporary/permanent jobs created or maintained) or employability of target groups. |
| --- | --- |

—

comparison of actual and planned results for employment (temporary/permanent jobs created or maintained) or employability of target groups.

For management:

| — | four types of indicators, measuring monitoring, financial control, project selection and evaluation systems. |
| --- | --- |

—

four types of indicators, measuring monitoring, financial control, project selection and evaluation systems.

For financial performance:

| — | absorption and rate of private funding. |
| --- | --- |

—

absorption and rate of private funding.

Assessments by Member States and allocation of the performance reserve

| 30. | Member States were required to evaluate performance for a period when on average only 20 % of expenditure had been undertaken. Some evaluators drew attention to this problem. For example, the Italian evaluator pointed out that it was too early to carry out a general analysis of effectiveness. There were however significant differences in the speed with which programme priorities could be implemented. Overall, the programmes examined by the Court showed faster spending at mid-term for physical infrastructure than for some priorities and measures related to the Lisbon and Gothenburg objectives. |
| --- | --- |

30.

Member States were required to evaluate performance for a period when on average only 20 % of expenditure had been undertaken. Some evaluators drew attention to this problem. For example, the Italian evaluator pointed out that it was too early to carry out a general analysis of effectiveness. There were however significant differences in the speed with which programme priorities could be implemented. Overall, the programmes examined by the Court showed faster spending at mid-term for physical infrastructure than for some priorities and measures related to the Lisbon and Gothenburg objectives.

| 31. | Member States tried to assess performance against indicators, though the process was handicapped by the limitations of some of the indicators developed in the ex ante assessment of 2000-2006 programmes. For example, for the German and Italian programmes, effectiveness was only monitored through output indicators (27). |
| --- | --- |

31.

Member States tried to assess performance against indicators, though the process was handicapped by the limitations of some of the indicators developed in the ex ante assessment of 2000-2006 programmes. For example, for the German and Italian programmes, effectiveness was only monitored through output indicators (27).

ex ante

27

| 32. | Member States' proposals for allocation of the performance reserve were based on the criteria listed in paragraph 29 above. In the case of the Spanish, Portuguese and Irish Community Support Frameworks, Member States went further and assessed performance by reference to a wider set of indicators than recommended by the Commission. |
| --- | --- |

32.

Member States' proposals for allocation of the performance reserve were based on the criteria listed in paragraph 29 above. In the case of the Spanish, Portuguese and Irish Community Support Frameworks, Member States went further and assessed performance by reference to a wider set of indicators than recommended by the Commission.

| 33. | Nevertheless, Member States' proposals for the allocation of the performance reserve were significantly constrained by a perceived need to maximise absorption of EU funds. The Commission itself emphasised the importance of absorption. For example, in a Commission note of 16 October 2003 to the members of the CDCR ‘preparation for the allocation of the performance reserve’, communicated also to the Member States, it was stressed that ‘the Commission will pay particular attention to the absorption of funds in the case of significant losses under the N + 2 mechanism and the quality of the mid-term evaluation (28).’ Examples include the following:






  — | the German transport OP aimed to spend approximately equal amounts on road and rail. But on absorption grounds it was proposed to allocate all of the performance reserve to road construction. When submitting the proposal to the Commission, the managing authority stated that the performance reserve was not employed in line with environmental or transport policy objectives, but according to progress in spending; |
| --- | --- | --- |

33.

Nevertheless, Member States' proposals for the allocation of the performance reserve were significantly constrained by a perceived need to maximise absorption of EU funds. The Commission itself emphasised the importance of absorption. For example, in a Commission note of 16 October 2003 to the members of the CDCR ‘preparation for the allocation of the performance reserve’, communicated also to the Member States, it was stressed that ‘the Commission will pay particular attention to the absorption of funds in the case of significant losses under the N + 2 mechanism and the quality of the mid-term evaluation (28).’ Examples include the following:

28

—

the German transport OP aimed to spend approximately equal amounts on road and rail. But on absorption grounds it was proposed to allocate all of the performance reserve to road construction. When submitting the proposal to the Commission, the managing authority stated that the performance reserve was not employed in line with environmental or transport policy objectives, but according to progress in spending;

| — | SPD Réunion: Some environmental and equal opportunities measures did not receive any allocation of the performance reserve. The managing authority underlined that it was in some cases difficult to reconcile certain Community objectives with the need to spend funds; |
| --- | --- |

—

SPD Réunion: Some environmental and equal opportunities measures did not receive any allocation of the performance reserve. The managing authority underlined that it was in some cases difficult to reconcile certain Community objectives with the need to spend funds;

| — | the proposal of Brandenburg, Germany, made mainly in the light of the analysis of absorption, excluded environmental investment measures from the allocation. Transport infrastructure received the highest amount. The evaluators of the programme also concluded that the Commission's indicators were rather based on absorption than on effectiveness. |
| --- | --- |

—

the proposal of Brandenburg, Germany, made mainly in the light of the analysis of absorption, excluded environmental investment measures from the allocation. Transport infrastructure received the highest amount. The evaluators of the programme also concluded that the Commission's indicators were rather based on absorption than on effectiveness.

| 34. | In addition there were several allocations of the performance reserve exclusively to the ERDF, mainly on the grounds of absorption, for example in Germany (OP Saxony), Ireland (the performance reserve of all Irish OPs was entirely allocated to one ERDF financed programme), Italy (OP Sviluppo Imprenditoriale Locale) and some Objective 2 Austrian, German, French, and Spanish and UK OPs. |
| --- | --- |

34.

In addition there were several allocations of the performance reserve exclusively to the ERDF, mainly on the grounds of absorption, for example in Germany (OP Saxony), Ireland (the performance reserve of all Irish OPs was entirely allocated to one ERDF financed programme), Italy (OP Sviluppo Imprenditoriale Locale) and some Objective 2 Austrian, German, French, and Spanish and UK OPs.

| 35. | The time-table for allocation of the performance reserve was tight. The proposals were received in January 2004 and agreement in principle had to be concluded by early February. |
| --- | --- |

35.

The time-table for allocation of the performance reserve was tight. The proposals were received in January 2004 and agreement in principle had to be concluded by early February.

| 36. | Some use was made of the performance reserve to allocate money to programmes on the basis of effectiveness and quality of management. Overall, however, the performance reserve was primarily used to maximise spending rather than to concentrate spending on activities which were assessed as particularly effective. |
| --- | --- |

36.

Some use was made of the performance reserve to allocate money to programmes on the basis of effectiveness and quality of management. Overall, however, the performance reserve was primarily used to maximise spending rather than to concentrate spending on activities which were assessed as particularly effective.

IMPACT OF MID-TERM REVISION

| 37. | The 2000-2006 reform of the Structural Funds modified and strengthened the Funds' objectives. The intention was to promote growth and employment in a way which took account of and combined interrelated economic, social and environmental policy objectives (29). In particular, expenditure from the funds was to take account of new technologies and innovation, research, the information society, clean and efficient use of energy (including renewable energy), better protection of the environment, social inclusion and life-long learning. |
| --- | --- |

37.

The 2000-2006 reform of the Structural Funds modified and strengthened the Funds' objectives. The intention was to promote growth and employment in a way which took account of and combined interrelated economic, social and environmental policy objectives (29). In particular, expenditure from the funds was to take account of new technologies and innovation, research, the information society, clean and efficient use of energy (including renewable energy), better protection of the environment, social inclusion and life-long learning.

29

| 38. | Allocation of the performance reserve and shifts of funds following the mid-term revision had to take account of the programming principles set out in the Commission's programming guidelines (30). The Commission however issued its revised programming guidelines only in August 2003. This was too late for most Member States and regions to take the revised guidelines into account in the MTR. On the other hand the revised guidelines involved only a number of qualitative shifts in the initial guidelines, for instance in the consolidation of the Lisbon strategy, in the fields of transport (emphasis on intermodality, shift from road to rail and waterways) and energy. |
| --- | --- |

38.

Allocation of the performance reserve and shifts of funds following the mid-term revision had to take account of the programming principles set out in the Commission's programming guidelines (30). The Commission however issued its revised programming guidelines only in August 2003. This was too late for most Member States and regions to take the revised guidelines into account in the MTR. On the other hand the revised guidelines involved only a number of qualitative shifts in the initial guidelines, for instance in the consolidation of the Lisbon strategy, in the fields of transport (emphasis on intermodality, shift from road to rail and waterways) and energy.

30

| 39. | All operational programmes and Community support frameworks had to be revised as a result of the allocation of the performance reserve. This involved the adaptation of each programme through a similar process as applied during the initial programming, including consultation of relevant Commission Directorates General and formal adoption by the Commission. |
| --- | --- |

39.

All operational programmes and Community support frameworks had to be revised as a result of the allocation of the performance reserve. This involved the adaptation of each programme through a similar process as applied during the initial programming, including consultation of relevant Commission Directorates General and formal adoption by the Commission.

| 40. | There was no obligation on Member States to revise OPs/CSFs for reasons other than allocation of the performance reserve. Of the 25 OPs and CSFs examined by the Court, revisions for other purposes were made in 15 cases. |
| --- | --- |

40.

There was no obligation on Member States to revise OPs/CSFs for reasons other than allocation of the performance reserve. Of the 25 OPs and CSFs examined by the Court, revisions for other purposes were made in 15 cases.

| 41. | In many cases, the mid-term revision improved the way in which OPs took account of policy priorities or changes in EU policies. In some cases performance indicators were improved and changes made to management structures. Changes of these types were for example made in Ireland, Portugal and Italy. |
| --- | --- |

41.

In many cases, the mid-term revision improved the way in which OPs took account of policy priorities or changes in EU policies. In some cases performance indicators were improved and changes made to management structures. Changes of these types were for example made in Ireland, Portugal and Italy.

| 42. | As already noted in respect of the performance reserve, however, priority was given to activities where money could be spent, for instance:






  — | SPD Burgenland, Austria. Main changes: the reduction of planned spending on research, technology and development and landscape conservation because of absorption difficulties, |
| --- | --- | --- |

42.

As already noted in respect of the performance reserve, however, priority was given to activities where money could be spent, for instance:

—

SPD Burgenland, Austria. Main changes: the reduction of planned spending on research, technology and development and landscape conservation because of absorption difficulties,

| — | SPD Hainaut, Belgium: absorption played an important role, for instance planned spending on the ‘management of the environment’ was substantially reduced on the grounds of low absorption and transferred to aid for redevelopment of industrial sites, |
| --- | --- |

—

SPD Hainaut, Belgium: absorption played an important role, for instance planned spending on the ‘management of the environment’ was substantially reduced on the grounds of low absorption and transferred to aid for redevelopment of industrial sites,

| — | OP Brandenburg, Germany: planned spending on environmental measures was reduced substantially, the measure ‘Renewable Energy’ was abandoned, |
| --- | --- |

—

OP Brandenburg, Germany: planned spending on environmental measures was reduced substantially, the measure ‘Renewable Energy’ was abandoned,

| — | OP Transport, Germany: the allocation of the performance reserve only to road projects substantially changed the ratio of spending on roads to spending on rail, |
| --- | --- |

—

OP Transport, Germany: the allocation of the performance reserve only to road projects substantially changed the ratio of spending on roads to spending on rail,

| — | OP Calabria: A large part of the revised programme concerns ‘compatible’ projects. These are public projects, which were already financed with national funds prior to the current programme period. They were not foreseen in the initial programme. This approach was mainly applied in order to avoid decommitments, |
| --- | --- |

—

OP Calabria: A large part of the revised programme concerns ‘compatible’ projects. These are public projects, which were already financed with national funds prior to the current programme period. They were not foreseen in the initial programme. This approach was mainly applied in order to avoid decommitments,

| — | CSF Italy, S&E OP Ireland: The financial plans were substantially changed because private funding initially foreseen was reduced (CSF Italy by 95 %, S&E OP by 70 %) and partly replaced by national public funds. The result was to increase nationally funded spending and thus avoid decommitments. |
| --- | --- |

—

CSF Italy, S&E OP Ireland: The financial plans were substantially changed because private funding initially foreseen was reduced (CSF Italy by 95 %, S&E OP by 70 %) and partly replaced by national public funds. The result was to increase nationally funded spending and thus avoid decommitments.

| 43. | The Court's examination found that transfers of funds for environmental measures were made mainly in favour of aid for enterprises and transport infrastructure (especially roads). This was the case, for example, in Germany, Spain, Italy and Belgium, amounting to around 800 million euros (about 1 % of Structural Fund allocations for these Member States). |
| --- | --- |

43.

The Court's examination found that transfers of funds for environmental measures were made mainly in favour of aid for enterprises and transport infrastructure (especially roads). This was the case, for example, in Germany, Spain, Italy and Belgium, amounting to around 800 million euros (about 1 % of Structural Fund allocations for these Member States).

| 44. | Overall the Court found that, in respect of a number of Structural Fund programmes, mid-term revision led to improvements (for example taking account of policy priorities or changes in EU policies or improving performance indicators). A significant impact in practice however was to shift budgets away from underspending measures. |
| --- | --- |

44.

Overall the Court found that, in respect of a number of Structural Fund programmes, mid-term revision led to improvements (for example taking account of policy priorities or changes in EU policies or improving performance indicators). A significant impact in practice however was to shift budgets away from underspending measures.

ARRANGEMENTS FOR 2007-2013

| 45. | Revised arrangements for evaluation, review and revision of programmes have been agreed for the next Structural Fund programme period 2007-2013. They differ in a number of respects from the arrangements which applied in 2000-2006. In general, where changes have been made, they provide for a greater degree of flexibility than under the 2000-2006 arrangements and should assist in ‘fine tuning’ mid-term evaluation and the revision of spending plans to the particular circumstances of individual programmes.






  — | There is still a requirement for evaluation before, during and after the programme period. Evaluations must, in particular, take account of the objective of sustainable development and of the relevant Community legislation concerning environmental impact and strategic environmental assessment (31). |
| --- | --- | --- |

45.

Revised arrangements for evaluation, review and revision of programmes have been agreed for the next Structural Fund programme period 2007-2013. They differ in a number of respects from the arrangements which applied in 2000-2006. In general, where changes have been made, they provide for a greater degree of flexibility than under the 2000-2006 arrangements and should assist in ‘fine tuning’ mid-term evaluation and the revision of spending plans to the particular circumstances of individual programmes.

—

There is still a requirement for evaluation before, during and after the programme period. Evaluations must, in particular, take account of the objective of sustainable development and of the relevant Community legislation concerning environmental impact and strategic environmental assessment (31).

31

| — | On the other hand there is no obligation to carry out, for all programmes, a mid-term review at a specific point in the programme period. |
| --- | --- |

—

On the other hand there is no obligation to carry out, for all programmes, a mid-term review at a specific point in the programme period.

| — | The principle of ‘proportional intervention’ applies to evaluation. Financial and administrative resources employed by the Commission and the Member States in the implementation of the funds in relation to the selection of indicators, evaluations, management and control system and reporting shall be proportional to the total amount of expenditure allocated to an operational programme (32). |
| --- | --- |

—

The principle of ‘proportional intervention’ applies to evaluation. Financial and administrative resources employed by the Commission and the Member States in the implementation of the funds in relation to the selection of indicators, evaluations, management and control system and reporting shall be proportional to the total amount of expenditure allocated to an operational programme (32).

32

| — | There is no compulsory performance reserve. Member States may instead choose to establish a national contingency reserve and/or a national performance reserve (33). (The procedures for allocating any national performance reserve resemble those applying to the performance reserve in 2000-2006). |
| --- | --- |

—

There is no compulsory performance reserve. Member States may instead choose to establish a national contingency reserve and/or a national performance reserve (33). (The procedures for allocating any national performance reserve resemble those applying to the performance reserve in 2000-2006).

33

| — | As for 2000-2006 mid-term revision of programmes is not compulsory for all programmes. (Revision of the central programming guidelines agreed by the Council does not require Member States to change their operational programmes or national strategic reference framework). Two cases in which Member States might decide to carry out a review are however identified in the Regulation: 1. significant socio-economic changes and to take greater or different account of major changes in Community, national or regional priorities, 2. in the light of a significant departure from the goals initially set or following implementation difficulties. |
| --- | --- |

—

As for 2000-2006 mid-term revision of programmes is not compulsory for all programmes. (Revision of the central programming guidelines agreed by the Council does not require Member States to change their operational programmes or national strategic reference framework). Two cases in which Member States might decide to carry out a review are however identified in the Regulation: 1. significant socio-economic changes and to take greater or different account of major changes in Community, national or regional priorities, 2. in the light of a significant departure from the goals initially set or following implementation difficulties.

| 46. | The ‘N + 2’ rule will still apply, but will (for a limited period) become an ‘N + 3’ rule for Member States whose GDP per head from 2001 to 2003 was below 85 % of the EU-25 average. |
| --- | --- |

46.

The ‘N + 2’ rule will still apply, but will (for a limited period) become an ‘N + 3’ rule for Member States whose GDP per head from 2001 to 2003 was below 85 % of the EU-25 average.

CONCLUSIONS

| 47. | Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes. |
| --- | --- |

47.

Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes.

a fortiori

| 48. | The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption (see paragraph 33). |
| --- | --- |

48.

The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption (see paragraph 33).

| 49. | In a number of cases the mid-term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant effect of the mid-term revision was to give priority to activities where money could be spent. |
| --- | --- |

49.

In a number of cases the mid-term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant effect of the mid-term revision was to give priority to activities where money could be spent.

| 50. | Serious efforts were made to carry out the mid-term processes as set out in the Structural Funds Regulation. On the other hand, since the mid-term evaluations were carried out at a relatively early stage in the programme, the mid-term evaluations and revisions could provide little assurance that Structural Fund spending would have the effects and impacts desired. The Member States and the Commission also made use of the performance reserve and the mid-term revision to promote spending that could be carried out within the permitted time-frame. |
| --- | --- |

50.

Serious efforts were made to carry out the mid-term processes as set out in the Structural Funds Regulation. On the other hand, since the mid-term evaluations were carried out at a relatively early stage in the programme, the mid-term evaluations and revisions could provide little assurance that Structural Fund spending would have the effects and impacts desired. The Member States and the Commission also made use of the performance reserve and the mid-term revision to promote spending that could be carried out within the permitted time-frame.

| 51. | The Court is of the view that it remains appropriate to include in large scale expenditure programmes, running for as long as 7 years, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision.






  — | Expenditure should be efficient, effective and economic. On the other hand the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as ceilings on spending (34). Absorption is itself an implicit objective for EU spending on cohesion. |
| --- | --- | --- |

51.

The Court is of the view that it remains appropriate to include in large scale expenditure programmes, running for as long as 7 years, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision.

—

Expenditure should be efficient, effective and economic. On the other hand the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as ceilings on spending (34). Absorption is itself an implicit objective for EU spending on cohesion.

34

| — | The N + 2 rule provides a discipline to ensure that spending is well planned, but creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease in allocation of any reserve. |
| --- | --- |

—

The N + 2 rule provides a discipline to ensure that spending is well planned, but creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease in allocation of any reserve.

| — | Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure. A corollary is that, if evaluation is to play a significant part in the allocation of the funds, it is important to carry out a high quality ex ante evaluation. |
| --- | --- |

—

Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure. A corollary is that, if evaluation is to play a significant part in the allocation of the funds, it is important to carry out a high quality ex ante evaluation.

ex ante

| 52. | The particular context of the 2000-2006 programme period exacerbated some of these problems. Expenditure for the 1994-1999 period was not subject to the N + 2 rule, and was significantly delayed (indeed the 1994-1999 programmes are still not closed). This contributed to the delay in starting programmes for the 2000-2006 period. Some of the other limitations of mid-term evaluation are more fundamental: it is intrinsically difficult to assess the net outcome of infrastructure investments, or of employment initiatives. |
| --- | --- |

52.

The particular context of the 2000-2006 programme period exacerbated some of these problems. Expenditure for the 1994-1999 period was not subject to the N + 2 rule, and was significantly delayed (indeed the 1994-1999 programmes are still not closed). This contributed to the delay in starting programmes for the 2000-2006 period. Some of the other limitations of mid-term evaluation are more fundamental: it is intrinsically difficult to assess the net outcome of infrastructure investments, or of employment initiatives.

RECOMMENDATIONS

| 53. | The Court recommends the following steps to improve the usefulness of medium term evaluation, revision and reallocation during the 2007-2013 and future programme periods:






  — | ‘Medium term’ evaluation and revision of programmes are more likely to be successful, the greater the extent to which expenditure takes place during the programme period, rather than following some years in arrears. In order to reduce delays in getting the 2007-2013 programme underway, the Commission and Member States should ensure that closure of the 2000-2006 programmes takes place speedily and efficiently. |
| --- | --- | --- |

53.

The Court recommends the following steps to improve the usefulness of medium term evaluation, revision and reallocation during the 2007-2013 and future programme periods:

—

‘Medium term’ evaluation and revision of programmes are more likely to be successful, the greater the extent to which expenditure takes place during the programme period, rather than following some years in arrears. In order to reduce delays in getting the 2007-2013 programme underway, the Commission and Member States should ensure that closure of the 2000-2006 programmes takes place speedily and efficiently.

| — | A particular difficulty found by evaluators of the 2000-2006 programmes was that monitoring systems were operating poorly. The Commission should take steps to ensure that performance indicators are monitored from the outset in the new programme period. |
| --- | --- |

—

A particular difficulty found by evaluators of the 2000-2006 programmes was that monitoring systems were operating poorly. The Commission should take steps to ensure that performance indicators are monitored from the outset in the new programme period.

| — | Evaluation and reassessment of expenditure should take place late enough for there to be sufficient experience of a programme to permit useful appraisal to take place, but early enough to permit changes in priorities. |
| --- | --- |

—

Evaluation and reassessment of expenditure should take place late enough for there to be sufficient experience of a programme to permit useful appraisal to take place, but early enough to permit changes in priorities.

| — | The Mid-Term processes in 2000-2006 constituted a lengthy and complex procedure. The Commission should consider ways in which the process can be made more rapid and responsive. |
| --- | --- |

—

The Mid-Term processes in 2000-2006 constituted a lengthy and complex procedure. The Commission should consider ways in which the process can be made more rapid and responsive.

| — | While the Commission should not be prescriptive in defining the method to be used by evaluators, it should identify how programmes can be amended so as to ensure that impacts on cohesion are identifiable, that evaluations are a source of best practice, and that, where performance reserves are established, they are used in a way which gives proper weight to the effective and efficient implementation of programmes. |
| --- | --- |

—

While the Commission should not be prescriptive in defining the method to be used by evaluators, it should identify how programmes can be amended so as to ensure that impacts on cohesion are identifiable, that evaluations are a source of best practice, and that, where performance reserves are established, they are used in a way which gives proper weight to the effective and efficient implementation of programmes.

| — | All guidance provided by the Commission should be made available in good time. |
| --- | --- |

—

All guidance provided by the Commission should be made available in good time.

| — | The Commission and the Member States should give special attention to the programming and management arrangements for expenditure on EU policy priorities. |
| --- | --- |

—

The Commission and the Member States should give special attention to the programming and management arrangements for expenditure on EU policy priorities.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 8 March 2007.

For the Court of Auditors

Hubert WEBER

President

---

(1)  About 16 billion euro Structural Funds for the years 2004-2006 were allocated to around 200 programmes of the new Member States.

1

(2)  See Title II Programming — Article 13-19 of Council Regulation (EC) No 1260/1999 of 21 June 1999 laying down general provisions on the Structural Funds (OJ L 161, 26.6.1999, p. 1).

2

(3)  Article 10(3) of Regulation (EC) No 1260/1999.

3

(4)  Article 18 and 19 of Regulation (EC) No 1260/1999. In cases where Community allocation did not exceed 1 billion euro the assistance could take the form of a ‘Single Programming Document’ (SPD), containing the same information to be found in a CSF and OP.

4

(5)  Article 14(2) of Regulation (EC) No 1260/1999.

5

(6)  Article 35(2) of Regulation (EC) No 1260/1999.

6

(7)  Performance reserve, see Article 44, rule ‘N + 2’ Article 31(2) of Regulation (EC) No 1260/1999.

7

(8)  The total amount of Structural Funds allocated to an Objective 1 programme is, according to its financing plan, committed in yearly tranches from 2000 to 2006. Payments may take the form of payments on account (representing 7 % of the total envelope of each programme, providing at the beginning a relatively long protection against the application of N + 2) or the rembursement of expenditure actually paid out to final recipients. In order to avoid decommitment Member States have to submit to the Commission by the end of the second year following the year of commitment a payment application, covering for each year the amounts forseen in the financing plan (Articles 31(2) and 32(3) of Regulation (EC) No 1260/1999).

8

(9)  Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 on the Financial Regulation applicable to the EC general budget (OJ L 248, 16.9.2002, p. 1), Article 27. Economy means that recources shall be made available in due time, in appropriate quantity and quality and at the best price. Effectiveness means attaining the objectives set and achieving the intended results; efficiency means the best relationship between resources employed and results achieved. For the evaluation of the effectiveness of the SF refer also to Article 40 of Regulation (EC) No 1260/1999.

9

(10)  Article 41 of Regulation (EC) No 1260/1999.

10

(11)  Article 42 of Regulation (EC) No 1260/1999.

11

(12)  Article 43 of Regulation (EC) No 1260/1999.

12

(13)  Special Report No 7/2003 on the implementation of assistance programming for the period 2000-2006 within the framework of the Structural Funds (OJ C 174, 23.7.2003).

13

(14)  See Special Report No 10/2006 on ex post evaluations of Objectives 1 and 3 programmes 1994-1999.

14

ex post

(15)  Article 42(2) of Regulation (EC) No 1260/1999.

15

(16)  Article 44(1) of Regulation (EC) No 1260/1999.

16

(17)  Article 44(2) of Regulation (EC) No 1260/1999.

17

(18)  Article 14(2) in combination with Title II Programming and Article 10(3) of Regulation (EC) No 1260/1999.

18

(19)  Article 42(3) of Regulation (EC) No 1260/1999.

19

(20)  Belgium SPD: Flanders Objective 3; France: SPD's Corsica and Objective 3; Germany: CSF and OP's Brandenburg, Transport; Italy: CSF and OP's Calabria, Ponsil, Transport; SPD Lombardy Objective 3; Ireland: CSF and OP's South-eastern, productive sector; Spain: CSF and OP's Andalusia, Galicia; UK: SPD Merseyside, OP Objective 3.

20

The processes of mid-term evaluations, allocation of performance reserve and revision concerning the CSF Portugal, the OP Algarve (P), the SPD Burgenland (A), the SPD Hainaut (B), the SPD Réunion (F), the SPD Flevoland (NL) were examined at Commission level.

(21)  At the draft stage, the Commission assessed one third of the reports as good or excellent. The assessment of the final report was much more positive: two-thirds of the reports were assessed as good or excellent. Where weaknesses were identified, the main cause was that evaluators were stretched by:

21

| — | the number of evaluation questions; |
| --- | --- |

—

the number of evaluation questions;

| — | the number of projects involved; |
| --- | --- |

—

the number of projects involved;

| — | the complexity of some large programmes, and the limited resources of some evaluators. |
| --- | --- |

—

the complexity of some large programmes, and the limited resources of some evaluators.

(22)  Growing Evaluation Capacity, European Commission, November 2004 (page 35).

22

(23)  There are a number of empirical studies and evaluations showing these effects. In general, there is insufficient data with regard to the real needs, including the kind of infrastructure in order to optimise the effects of the Structural funds.

23

(24)  A ‘deadweight’ effect exists where an activity or investment would have been undertaken in the absence of financing.

24

(25)  European Commission, the new programming period 2000-2006: methodological Working Paper 4 — Implementation of the performance reserve.

25

(26)  These indicators are related: financial absorption and output are strongly correlated.

26

(27)  The Court's Special Report No 7/2003 pointed out that in practice, the performance reserve entailed a complex procedure for reallocating funds, and could lead to inconsistencies and ineffectiveness in the use of the reserve.

27

(28)  In one case, the Dutch Objective 3 programme, the Commission had no option other than to allocate the performance reserve despite the fact that significant decommitments had already taken place because funds had not been spent.

28

(29)  This corresponded to the requirements of the Union's concept of sustainable development of the Amsterdam Treaty and anticipated largely the conclusions of the Lisbon European Council (March 2000: ‘to become the most competitive and knowledge-based economy in the world’) to which the Gothenburg European Council (June 2001) added the environmental strategy of sustainable development.

29

(30)  See Articles 10(3) and 14 of Regulation (EC) No 1260/1999. ‘... in accordance with this title’. This ‘title’ refers to the programming provisions of Articles 15 to 19.

30

(31)  Council Regulation (EC) No 1083/2006 of 11 July 2006 laying down general provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund and repealing Regulation (EC) No 1260/1999 (OJ L 210, 31.7.2006, p. 25), Article 47.

31

(32)  Regulation (EC) No 1083/2006, Article 13.

32

(33)  Regulation (EC) No 1083/2006, Articles 50 and 51.

33

(34)  According to paragraph 12, third subparagraph of the Interinstitutional Agreement (including the financial perspective) between the European Parliament and the Council and the Commission of 6 May 1999 (OJ C 172, 18.6.1999, p. 1) the European Parliament and the Council undertake to respect the allocations of commitment appropriations provided in the financial perspective for structural operations.

34

---

THE COMMISSION'S REPLIES

SUMMARY

| III. | First indent — The aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. The late or slow start-up of many programmes made such analysis difficult, although it was carried out in some cases (see Chapter 5 of the Commission's report Growing Evaluation Capacity). Nevertheless, the other aspects examined in the mid-term evaluation provided useful input to the decisions on the allocation of the performance reserve and the adaptation of operational programmes.
  Second indent — The performance reserve was used to allocate resources to programmes on several criteria for which the Commission recommended three groups of indicators. Financial absorption was an important factor, but the criteria varied between Member States and were often wider subject to availability of robust data (see points 28 and 32).
  Third indent — The Commission considers that the mid-term revision had a significant effect on the general performance of programmes and the way they took account of policy priorities. At the same time, the Commission wanted to avoid allocating additional resources to fields which were already having difficulties in spending the existing resources. |
| --- | --- |

III.

First indent — The aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. The late or slow start-up of many programmes made such analysis difficult, although it was carried out in some cases (see Chapter 5 of the Commission's report Growing Evaluation Capacity). Nevertheless, the other aspects examined in the mid-term evaluation provided useful input to the decisions on the allocation of the performance reserve and the adaptation of operational programmes.

Second indent — The performance reserve was used to allocate resources to programmes on several criteria for which the Commission recommended three groups of indicators. Financial absorption was an important factor, but the criteria varied between Member States and were often wider subject to availability of robust data (see points 28 and 32).

Third indent — The Commission considers that the mid-term revision had a significant effect on the general performance of programmes and the way they took account of policy priorities. At the same time, the Commission wanted to avoid allocating additional resources to fields which were already having difficulties in spending the existing resources.

| IV. | The Commission agrees that it is appropriate to reconsider programme objectives, measures and budgetary allocations during the programme period. It has built on the experience of 2000-2006 in proposing revised arrangements for 2007-2013.
  First indent — It is the task of the Commission to ensure that due account is taken of the effectiveness and value for money as well as just absorption. The Commission endeavours to discharge this responsibility.
  Second indent — The tension the n + 2 rule creates between fast absorption and quality need not mean that quality is sacrificed. The Commission endeavours to prevent this.
  Third indent — The monitoring and evaluation arrangements for the 2007-2013 programme period are designed to be responsive to the practical challenges faced by programmes. |
| --- | --- |

IV.

The Commission agrees that it is appropriate to reconsider programme objectives, measures and budgetary allocations during the programme period. It has built on the experience of 2000-2006 in proposing revised arrangements for 2007-2013.

First indent — It is the task of the Commission to ensure that due account is taken of the effectiveness and value for money as well as just absorption. The Commission endeavours to discharge this responsibility.

Second indent — The tension the n + 2 rule creates between fast absorption and quality need not mean that quality is sacrificed. The Commission endeavours to prevent this.

Third indent — The monitoring and evaluation arrangements for the 2007-2013 programme period are designed to be responsive to the practical challenges faced by programmes.

| V. | First indent — The Commission issued a detailed guidance document for the 2000-2006 period closure in August 2006. The closure requirements are now more familiar than before the 1994-1999 period closure.
  Second indent — The Commission will devote close attention to the setting up of monitoring systems and the definition of indicators in the new programmes.
  Third indent — In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach can be better tailored to the demands of the users of the evaluation.
  The proportionality introduced by the regulations and the streamlining of the programming process will also increase flexibility in evaluation arrangements.
  Fourth indent — The more flexible arrangements will allow this.
  Fifth indent — For the 2007-2013 period the Commission provided guidance documents in 2006.
  Sixth indent — Co-financed activities are required to promote European Union (EU) policy priorities, including the Growth and Jobs agenda. EU-15 Member States have to meet specified targets for spending on the latter. |
| --- | --- |

V.

First indent — The Commission issued a detailed guidance document for the 2000-2006 period closure in August 2006. The closure requirements are now more familiar than before the 1994-1999 period closure.

Second indent — The Commission will devote close attention to the setting up of monitoring systems and the definition of indicators in the new programmes.

Third indent — In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach can be better tailored to the demands of the users of the evaluation.

The proportionality introduced by the regulations and the streamlining of the programming process will also increase flexibility in evaluation arrangements.

Fourth indent — The more flexible arrangements will allow this.

Fifth indent — For the 2007-2013 period the Commission provided guidance documents in 2006.

Sixth indent — Co-financed activities are required to promote European Union (EU) policy priorities, including the Growth and Jobs agenda. EU-15 Member States have to meet specified targets for spending on the latter.

INTRODUCTION

| 6. | Box 1. — The Commission had initially proposed a performance reserve that could be shifted between Member States, but the proposal was amended during the negotiations on the future legislation. |
| --- | --- |

6.

Box 1. — The Commission had initially proposed a performance reserve that could be shifted between Member States, but the proposal was amended during the negotiations on the future legislation.

| 9. | First indent — The Commission considered that progress had been made in the programming of the Structural Funds to ensure their implementation in accordance with a clearly laid down development strategy and the quest for effectiveness had not been neglected in favour of the consumption of funds.
  Second indent — All Member States carried out ex ante evaluations, which led to a greater coherence of strategies and greater knowledge of the basis of the choices made and significantly improved quantification of objectives. In those Member States where the ex ante evaluation examined alternative strategies, the intention was to use this evaluation as part of their national planning process. |
| --- | --- |

9.

First indent — The Commission considered that progress had been made in the programming of the Structural Funds to ensure their implementation in accordance with a clearly laid down development strategy and the quest for effectiveness had not been neglected in favour of the consumption of funds.

Second indent — All Member States carried out ex ante evaluations, which led to a greater coherence of strategies and greater knowledge of the basis of the choices made and significantly improved quantification of objectives. In those Member States where the ex ante evaluation examined alternative strategies, the intention was to use this evaluation as part of their national planning process.

| 10. | The Commission agrees that future ex post evaluations should be more focused and takes full account of data and resource constraints. It will prepare the exercise carefully. |
| --- | --- |

10.

The Commission agrees that future ex post evaluations should be more focused and takes full account of data and resource constraints. It will prepare the exercise carefully.

ASSESSMENT OF MID-TERM EVALUATION

| 20. | Box 4. — The Commission's report on the Mid-Term Evaluation — Growing Evaluation Capacity also noted that some evaluations began an analysis of unit costs which can be built on in the future. |
| --- | --- |

20.

Box 4. — The Commission's report on the Mid-Term Evaluation — Growing Evaluation Capacity also noted that some evaluations began an analysis of unit costs which can be built on in the future.

| 23. | First indent — The Commission, too, recognised in its report ‘Growing Evaluation Capacity’ (point 4.1.1) that the low level of spending had limited the potential of the evaluations.
  Second indent — The evaluations' findings with respect to the Lisbon-Gothenburg agenda were reviewed in point 5.2 of the Commission's report ‘Growing Evaluation Capacity’.
  Third indent — The Commission's report ‘Growing Evaluation Capacity’ did not deal with methodological problems of assessing impacts, as the low level of spending and the resulting relatively small outputs were a bigger limitation. In the Working Paper on the mid-term evaluation, however, it had made clear that the aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. This also applies to the assessment of efficiency in terms of employment effects. |
| --- | --- |

23.

First indent — The Commission, too, recognised in its report ‘Growing Evaluation Capacity’ (point 4.1.1) that the low level of spending had limited the potential of the evaluations.

Second indent — The evaluations' findings with respect to the Lisbon-Gothenburg agenda were reviewed in point 5.2 of the Commission's report ‘Growing Evaluation Capacity’.

Third indent — The Commission's report ‘Growing Evaluation Capacity’ did not deal with methodological problems of assessing impacts, as the low level of spending and the resulting relatively small outputs were a bigger limitation. In the Working Paper on the mid-term evaluation, however, it had made clear that the aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. This also applies to the assessment of efficiency in terms of employment effects.

| 25. | Third indent — The balance between addressing longer-term needs, such as for R&D and innovation, and financing faster-spending demand-driven measures is hard to achieve.
  Fourth indent — The problem of lack of comparability of different modelling results and evaluations is discussed in the Commission's report Growing Evaluation Capacity, point 5.1.2.1 — (see also point 22 above). |
| --- | --- |

25.

Third indent — The balance between addressing longer-term needs, such as for R&D and innovation, and financing faster-spending demand-driven measures is hard to achieve.

Fourth indent — The problem of lack of comparability of different modelling results and evaluations is discussed in the Commission's report Growing Evaluation Capacity, point 5.1.2.1 — (see also point 22 above).

| 26. | As noted in the reply to point 23, final indent, the aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and impact were limited.
  Third indent — The Working Paper on the mid-term evaluation pointed out that the estimation of these effects, too, would depend on data availability.
  Fourth indent — See reply to box 4 and to point 23, last indent. |
| --- | --- |

26.

As noted in the reply to point 23, final indent, the aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and impact were limited.

Third indent — The Working Paper on the mid-term evaluation pointed out that the estimation of these effects, too, would depend on data availability.

Fourth indent — See reply to box 4 and to point 23, last indent.

| 27. | Another important objective of the mid-term evaluations was to assess the relevance of the strategies, which could be done in the time-frame set for the mid-term evaluation despite low expenditure (see working paper). Regarding the limited aims of the assessment of effectiveness and impact, see point 23. |
| --- | --- |

27.

Another important objective of the mid-term evaluations was to assess the relevance of the strategies, which could be done in the time-frame set for the mid-term evaluation despite low expenditure (see working paper). Regarding the limited aims of the assessment of effectiveness and impact, see point 23.

ALLOCATION OF PERFORMANCE RESERVE

| 30. | The mid-term evaluation provided information for the allocation of the performance reserve on matters other than spending. Some of the indicators recommended by the Commission for the mid-term evaluation did not depend on the level of expenditure, but measured different aspects of programme implementation.
  It should be noted that some infrastructure projects do contribute to the Lisbon and Gothenburg objectives. |
| --- | --- |

30.

The mid-term evaluation provided information for the allocation of the performance reserve on matters other than spending. Some of the indicators recommended by the Commission for the mid-term evaluation did not depend on the level of expenditure, but measured different aspects of programme implementation.

It should be noted that some infrastructure projects do contribute to the Lisbon and Gothenburg objectives.

| 31. | The Commission's working document recommended the use of output indicators, as data is available with only a small time-lag, rather than results indicators where data is available only after a longer period. Impact indicators are not available until years after completion of the intervention. |
| --- | --- |

31.

The Commission's working document recommended the use of output indicators, as data is available with only a small time-lag, rather than results indicators where data is available only after a longer period. Impact indicators are not available until years after completion of the intervention.

| 33. | The reason for the Commission's stating that it would pay particular attention to absorption was that it would not be sensible to allocate additional resources to a programme or priority which was not absorbing the initial allocation.
  The Commission agrees that fast spending by itself is not an indication of success of a programme. But efficient spending is a necessary condition if a programme is to achieve the objectives set. Although financial absorption was an important factor in the allocation of the performance reserve, the criteria varied between Member States and were often wider where data were sufficiently robust (see points 28 and 32).
  First indent — Although railway investment was environmentally desirable, the managing authority had to take account of the fact that it had used only 14,5 % of available funds.
  Second indent — The financial allocation to the environment was one of the most contentious parts of the negotiations. The outcome was that cuts for some environmental projects (such as a waste incinerator whose construction had been postponed) were partly compensated for by increases in the allocation for other environmentally-friendly projects. While there was no new allocation of funding to the equal opportunities measure, the discussion led to intensified activity in this area.
  Third indent — The Commission was not happy with the high allocation of funding by Brandenburg to transport infrastructure, especially road infrastructure, and has regularly raised the matter with the Land, which defends its decision on n + 2 grounds. The Commission intends to take up this point in the negotiations for the next programme period. |
| --- | --- |

33.

The reason for the Commission's stating that it would pay particular attention to absorption was that it would not be sensible to allocate additional resources to a programme or priority which was not absorbing the initial allocation.

The Commission agrees that fast spending by itself is not an indication of success of a programme. But efficient spending is a necessary condition if a programme is to achieve the objectives set. Although financial absorption was an important factor in the allocation of the performance reserve, the criteria varied between Member States and were often wider where data were sufficiently robust (see points 28 and 32).

First indent — Although railway investment was environmentally desirable, the managing authority had to take account of the fact that it had used only 14,5 % of available funds.

Second indent — The financial allocation to the environment was one of the most contentious parts of the negotiations. The outcome was that cuts for some environmental projects (such as a waste incinerator whose construction had been postponed) were partly compensated for by increases in the allocation for other environmentally-friendly projects. While there was no new allocation of funding to the equal opportunities measure, the discussion led to intensified activity in this area.

Third indent — The Commission was not happy with the high allocation of funding by Brandenburg to transport infrastructure, especially road infrastructure, and has regularly raised the matter with the Land, which defends its decision on n + 2 grounds. The Commission intends to take up this point in the negotiations for the next programme period.

| 34. | The allocation of the performance reserve was intended to be independent of the particular Structural Fund from which the funding would be paid.
  It responded to longer-term development priorities as well as high demand for funding from particular sectors. In some of the cases referred to by the Court, programmes were eliminated as candidates for the reserve because of an imminent risk of decommitment of funding under the n + 2 rule. |
| --- | --- |

34.

The allocation of the performance reserve was intended to be independent of the particular Structural Fund from which the funding would be paid.

It responded to longer-term development priorities as well as high demand for funding from particular sectors. In some of the cases referred to by the Court, programmes were eliminated as candidates for the reserve because of an imminent risk of decommitment of funding under the n + 2 rule.

| 36. | As noted in the reply to point 34, the allocation was by no means always based on absorption, although this was one of indicators for allocation of the reserve, but also took account of effectiveness in terms of outputs and results, subject to availability of robust data, and of longer-term needs. |
| --- | --- |

36.

As noted in the reply to point 34, the allocation was by no means always based on absorption, although this was one of indicators for allocation of the reserve, but also took account of effectiveness in terms of outputs and results, subject to availability of robust data, and of longer-term needs.

IMPACT OF MID-TERM REVISION

| 42. | On the question of the role of absorption in the mid-term revision, the Commission refers to its replies to point 34 in relation to the performance reserve.
  Second indent — As well as financial performance, the effectiveness of expenditure, in particular for job creation, was a criterion. The risk of environmental projects overshooting the time limit for expenditure was a factor in the reduction of funding for this type of investment.
  Fifth indent — The Commission monitors the eligibility of such projects for EU funding through audit work and the relevant Monitoring Committee. For example, audits of projects funded by the European Social Fund (ESF) under the Calabria programme are to take place in March 2007.
  Sixth indent — The removal of private funding was partly because private funding had been included in the financial tables by mistake (it had not been intended to include it in the co-financing base) and partly because of its volatility which meant that financial plans had to be regularly revised, whereas public funding was more stable. The European Regional Development Fund (ERDF) allocations to the measures concerned remained unchanged and private funding continued to be mobilised to implement the measures, although no longer part of the co-financing base. |
| --- | --- |

42.

On the question of the role of absorption in the mid-term revision, the Commission refers to its replies to point 34 in relation to the performance reserve.

Second indent — As well as financial performance, the effectiveness of expenditure, in particular for job creation, was a criterion. The risk of environmental projects overshooting the time limit for expenditure was a factor in the reduction of funding for this type of investment.

Fifth indent — The Commission monitors the eligibility of such projects for EU funding through audit work and the relevant Monitoring Committee. For example, audits of projects funded by the European Social Fund (ESF) under the Calabria programme are to take place in March 2007.

Sixth indent — The removal of private funding was partly because private funding had been included in the financial tables by mistake (it had not been intended to include it in the co-financing base) and partly because of its volatility which meant that financial plans had to be regularly revised, whereas public funding was more stable. The European Regional Development Fund (ERDF) allocations to the measures concerned remained unchanged and private funding continued to be mobilised to implement the measures, although no longer part of the co-financing base.

| 43. | The Commission refers to its reply at point 33. |
| --- | --- |

43.

The Commission refers to its reply at point 33.

ARRANGEMENTS FOR 2007-2013

| 45. | In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach emphasises the need for stronger links between monitoring, evaluation and decision-making, and can be better tailored to the demands of the users of the evaluation. An evaluation will be triggered by actual or potential difficulties revealed by the monitoring system but can be undertaken to ensure regular review of strategic or operational aspects which cannot be solely analysed on the basis of monitoring data.
  First indent — The Commission provided guidance on this new approach in 2006 (Working Document No 5 ‘Indicative Guidelines on Evaluation Methods: Evaluation during the Programme period — 2007-2013’). |
| --- | --- |

45.

In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach emphasises the need for stronger links between monitoring, evaluation and decision-making, and can be better tailored to the demands of the users of the evaluation. An evaluation will be triggered by actual or potential difficulties revealed by the monitoring system but can be undertaken to ensure regular review of strategic or operational aspects which cannot be solely analysed on the basis of monitoring data.

First indent — The Commission provided guidance on this new approach in 2006 (Working Document No 5 ‘Indicative Guidelines on Evaluation Methods: Evaluation during the Programme period — 2007-2013’).

CONCLUSIONS

| 47. | The aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. The late or slow start-up of many programmes made such analysis difficult, although it was carried out in some cases (see Chapter 5 of the Commission's report Growing Evaluation Capacity). Nevertheless, the other aspects examined in the mid-term evaluation provided useful input to the decisions on the allocation of the performance reserve and the adaptation of operational programmes. |
| --- | --- |

47.

The aims of the mid-term evaluation with regard to the assessment of effectiveness, efficiency and expected impacts were limited to analysis of initial outputs and results. The late or slow start-up of many programmes made such analysis difficult, although it was carried out in some cases (see Chapter 5 of the Commission's report Growing Evaluation Capacity). Nevertheless, the other aspects examined in the mid-term evaluation provided useful input to the decisions on the allocation of the performance reserve and the adaptation of operational programmes.

| 48. | The performance reserve was used to allocate resources to programmes on several criteria for which the Commisssion recommended three groups of indicators. Financial absorption was an important factor, but the criteria varied between Member States and were often wider subject to availability of robust data (see points 28 and 32). |
| --- | --- |

48.

The performance reserve was used to allocate resources to programmes on several criteria for which the Commisssion recommended three groups of indicators. Financial absorption was an important factor, but the criteria varied between Member States and were often wider subject to availability of robust data (see points 28 and 32).

| 49. | The Commission considers that the mid-term revision had a significant effect on the general performance of programmes and the way they took account of policy priorities. At the same time, the Commission wanted to avoid allocating additional resources to fields which were already having difficulties in spending the existing resources. |
| --- | --- |

49.

The Commission considers that the mid-term revision had a significant effect on the general performance of programmes and the way they took account of policy priorities. At the same time, the Commission wanted to avoid allocating additional resources to fields which were already having difficulties in spending the existing resources.

| 50. | Given the low level of spending, the mid-term evaluation could only provide an indication of an appropriate redirection of funds |
| --- | --- |

50.

Given the low level of spending, the mid-term evaluation could only provide an indication of an appropriate redirection of funds

| 51. | The Commission agrees that it is appropriate to reconsider programme objectives, measures and budgetary allocations during the programme period 2000-2006. It has built on the experience of 2000-2006 in proposing revised arrangements for 2007-2013.
  First indent — It is the task of the Commission to ensure that due account is taken of effectiveness and value for money as well as just absorption. The Commission endeavours to discharge this responsibility.
  Second indent — The tension the n + 2 rule creates between fast absorption and quality need not mean that quality is sacrificed. The Commission endeavours to prevent this.
  Third indent — The monitoring and evaluation arrangements for the 2007-2013 programme period are designed to be responsive to the practical challenges faced by programmes. |
| --- | --- |

51.

The Commission agrees that it is appropriate to reconsider programme objectives, measures and budgetary allocations during the programme period 2000-2006. It has built on the experience of 2000-2006 in proposing revised arrangements for 2007-2013.

First indent — It is the task of the Commission to ensure that due account is taken of effectiveness and value for money as well as just absorption. The Commission endeavours to discharge this responsibility.

Second indent — The tension the n + 2 rule creates between fast absorption and quality need not mean that quality is sacrificed. The Commission endeavours to prevent this.

Third indent — The monitoring and evaluation arrangements for the 2007-2013 programme period are designed to be responsive to the practical challenges faced by programmes.

| 52. | The Commission agrees that a smooth start of implementation will increase the usefulness of evaluations carried out mid-term. The role of the mid-term evaluation in the 2000-2006 period was to see how well programmes were performing and on the basis of initial results make any changes that were appropriate, rather than continuing on a course which would limit positive outcomes. Apart from commitments remaining open because of legal proceeding the 1994-1999 programmes have largely been closed and the outstanding commitments as at 31 December 2006 were down to under 0,6 % (917 million euros) of the original allocations. |
| --- | --- |

52.

The Commission agrees that a smooth start of implementation will increase the usefulness of evaluations carried out mid-term. The role of the mid-term evaluation in the 2000-2006 period was to see how well programmes were performing and on the basis of initial results make any changes that were appropriate, rather than continuing on a course which would limit positive outcomes. Apart from commitments remaining open because of legal proceeding the 1994-1999 programmes have largely been closed and the outstanding commitments as at 31 December 2006 were down to under 0,6 % (917 million euros) of the original allocations.

RECOMMENDATIONS

| 53. | First indent — The Commission issued a detailed guidance document for the 2000-2006 period closure in August 2006. The closure requirements are now more familiar than before the 1994-1999 period closure.
  Second indent — The Commission will devote close attention to the setting-up of monitoring systems and the definition of indicators in the 2007-2013 programmes.
  Third indent — The new evaluation arrangements increase flexibility. In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach can be better tailored to the demands of the users of the evaluation.
  The proportionality introduced by the regulations and the streamlining of the programming process will also increase flexibility in evaluation arrangements.
  Fourth indent — The arrangements for the 2007-2013 period are more flexible.
  Fifth indent — For the 2007-2013 period, the Structural Funds Regulation provides a flexible way to link evaluation and reprogramming. Evaluations are a source of valuable recommendations and can identify good and bad practices.
  Sixth indent — The main guidance for the 2000-2006 period was provided in good time. For the 2007-2013 period the Commission provided guidance documents in 2006.
  Seventh indent — Co-financed activities are required to promote EU policy priorities, including the Growth and Jobs agenda. EU-15 Member States have to meet specified targets for spending on the latter. |
| --- | --- |

53.

First indent — The Commission issued a detailed guidance document for the 2000-2006 period closure in August 2006. The closure requirements are now more familiar than before the 1994-1999 period closure.

Second indent — The Commission will devote close attention to the setting-up of monitoring systems and the definition of indicators in the 2007-2013 programmes.

Third indent — The new evaluation arrangements increase flexibility. In the 2007-2013 period the previous general requirement for mid-term evaluation will be replaced by need-based ongoing evaluations to assess the implementation of a programme and react to changes in its external environment. This approach can be better tailored to the demands of the users of the evaluation.

The proportionality introduced by the regulations and the streamlining of the programming process will also increase flexibility in evaluation arrangements.

Fourth indent — The arrangements for the 2007-2013 period are more flexible.

Fifth indent — For the 2007-2013 period, the Structural Funds Regulation provides a flexible way to link evaluation and reprogramming. Evaluations are a source of valuable recommendations and can identify good and bad practices.

Sixth indent — The main guidance for the 2000-2006 period was provided in good time. For the 2007-2013 period the Commission provided guidance documents in 2006.

Seventh indent — Co-financed activities are required to promote EU policy priorities, including the Growth and Jobs agenda. EU-15 Member States have to meet specified targets for spending on the latter.

---