EN Official Journal of the European Union C 124/1

SPECIAL REPORT No 1/2007

concerning the implementation of the mid-term processes on the Structural Funds 2000-2006 together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, EC)

(2007/C 124/01)

TABLE OF CONTENTS

I-V SUMMARY

1-12 INTRODUCTION

1-2 The Structural Funds

3-6 The Structural Funds 2000-2006

7-10 Evaluation and appraisal of the Structural Funds

11-12 Mid-term evaluation and review and the allocation of the performance reserve

13-15 SCOPE OF THE AUDIT

16-17 TIMETABLE

18-27 ASSESSMENT OF MID-TERM EVALUATION

20 The results of the evaluations

21-22 Commission assessment of evaluations

23-27 Assessment by the Court

28-36 ALLOCATION OF PERFORMANCE RESERVE

30-36 Assessments by Member States and allocation of the performance reserve

37-44 IMPACT OF MID-TERM REVISION

45-46 ARRANGEMENTS FOR 2007-2013

47-52 CONCLUSIONS

53 RECOMMENDATIONS

The Commission's replies

SUMMARY

I. The Structural Funds Regulation provided for a series of steps roughly half way through the 2000-2006 programme period: — a mid-term evaluation of programmes; — allocation of the performance reserve; and — mid-term revision of expenditure plans.

II. The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds: in particular, whether — mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure; — the performance reserve was allocated so as to favour effective and well-managed spending programmes; — the mid-term revision updated priorities and improved the targeting of spending programmes.

III. The Court concluded as follows: — Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes. — The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption. — In a number of cases the mid term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant impact of mid-term revision was to give priority to activities where money could be spent.

IV. The Court considers that it remains appropriate to include, in expenditure programmes such as the Structural Funds, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds, which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision: — Expenditure should be efficient, effective and economic. On the other hand, the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as limits on spending. Absorption is an implicit objective for EU spending on cohesion. — The N + 2 rule provides a discipline to ensure that spending is well planned, but also creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease. — Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure.

V. The Court makes a number of recommendations designed to increase the usefulness of mid-term review and revision in the context of the more flexible arrangements for the Structural Funds which will apply for the 2007-2013 programme period: — speedy and efficient closure of the 2000-2006 programmes; — better performance indicators; — careful choice of timing for evaluation of expenditure (taking advantage of the increased flexibility of the reformed system); — shorter and quicker mid-term revision; — timely and relevant guidance from the Commission; — special attention to the programmes and management arrangements for expenditure for EU policy priorities.

INTRODUCTION

The Structural Funds

1. The Structural Funds are the Union's principal policy instrument in support of the Treaty objectives of economic and social cohesion.

2. There are four Structural Funds: the European Regional Development Fund (ERDF), the European Social Fund (ESF), the European Agricultural Guidance and Guarantee Fund, ‘Guidance section’ (EAGGF-Guidance), and the Financial Instrument for Fisheries Guidance (FIFG). Expenditure is planned and carried out on a multiannual basis in ‘programme periods’. It supports investment in physical and human capital within Member States. Member States pay part of the cost of (co-finance) the investment projects concerned.

The Structural Funds 2000-2006

3. The European Council agreed at its meeting in Berlin (March 1999) that 195 billion euro (219 billion euro in 2005 prices) should be made available from the Structural Funds for the period 2000-2006. Over 90 % of spending from the funds was to be used in pursuit of three objectives: — Objective 1: promoting the development and structural adjustment of regions whose development is lagging behind (regions whose per capita GDP is less than 75 % of the Community average). About 70 % of total spending was to be devoted to Objective 1; — Objective 2: supporting the economic and social conversion of areas (outside Objective 1 regions) facing structural difficulties (industrial, rural, urban areas and areas dependent on fisheries); — Objective 3: supporting (outside Objective 1 regions) the adaptation and modernisation of policies and systems of education, training and employment.

4. The planned expenditure totals agreed at Berlin were allocated to some 400 national and regional programmes (1) through a complex 'programming' process, whose main elements were as follows (2): — The Commission established (3), for the initial programming exercise and subsequently for the mid-term review, indicative programming guidelines in relation to the objectives of the Funds; — Member States negotiated with the Commission ‘Community Support Frameworks’ (CSF), which set out the strategy and priorities for action of the Funds and the Member State; — Each CSF was implemented by means of ‘operational programmes’ (OP) (4) approved by the Commission; — Regions or Member States then approved individual projects to implement operational programmes; — There was provision for programmes to be revised midway through the programme period (mid-term revision) (5).

5. Expenditure was to be supervised by ‘Monitoring Committees’ set up by the Member State concerned and including Commission representatives (6) (in an advisory capacity).

6. Two innovations of relevance to this report were introduced into the management of the Structural Funds for the 2000-2006 programme period: a performance reserve and the ‘N + 2’ rule (7). See boxes 1 and 2. Box 1The performance reserveThe Structural Fund Regulations for the 2000-2006 programme period required the Commission, when allocating Structural Fund resources, to withhold 4 % from each operational programme. In order to encourage the efficient and effective use of Community funds, this reserve was to be allocated among and within each Member State's operational programmes in 2004, on the basis of the comparative success of these programmes.The performance reserve was a limited measure. Funds could not be switched among Member States nor between progammes coming under different objectives. Nor could any part of the reserve be withheld further.

Box 2The N + 2 ruleThe N + 2 rule is designed to prevent an excessive build up of unused commitments for an operational programme, which can complicate closure and interfere with successor programmes, by providing an incentive to ‘spend the money’ in line with targets. It provides that if a commitment has not led to payments within two years of being entered in the budget (8), the commitment is cancelled (decommitted) and not replaced, i.e. spending falls short of the totals agreed at the Berlin European Council.

Evaluation and appraisal of the Structural Funds

7. The Financial Regulation, which sets out rules for the management of the Community budget, provides that EU spending must be economical, efficient and effective. Objectives must be set for all sectors of activity covered by the budget, and their achievement must be monitored by performance indicators. Expenditure must also be subject to ex ante and ex post evaluations (9).

8. The Structural Funds Regulations for the 2000-2006 programme period set out how these principles were to be applied to the Structural Funds. In particular there were to be: — an ex ante evaluation, for each CSF or OP, carried out within Member States, to provide a basis for preparing the development plans (10); — a mid-term evaluation, carried out by an independent assessor under the responsibility of the authority managing the expenditure, and in cooperation with the Commission and the Member State; its purpose was to examine, in the light of the ex ante evaluation, initial results, their relevance, the achievement of targets, the use of financial resources, monitoring and implementation (11); — an ex post evaluation, covering the use of resources, and the effectiveness and efficiency of intervention, carried out by the Commission in collaboration with the Member State and the authority managing the expenditure concerned (12).

9. The Court audited the arrangements for ex ante evaluation and programming of the 2000-2006 generation of Structural Funds in its Special Report No 7/2003 (13). The Court noted, inter alia, that: — budgetary allocations were determined less by a well-established development strategy and the effectiveness of the Structural Funds than by maximising likely take-up of funding, — ex ante evaluation had little impact on the programming process. Evaluation was confined to justifying the programming as proposed by Member States and only rarely considered its validity and the underlying strategy. There was no critical appraisal or analysis of alternative strategies.

10. The Court has also audited the Commission's ex post evaluations of most Structural Fund expenditure for the previous programme period (1994-1999) (14). The Court found a number of significant weaknesses in the evaluations carried out and in the Commission's oversight of them; and recommended ensuring that available data and allocated resources were in line with set objectives, which should be realistic, and some changes in scope and approach for future evaluation exercises.

Mid-term evaluation and review and the allocation of the performance reserve

11. The Structural Funds Regulation thus provided for a series of procedures roughly half way through the 2000-2006 programme period: — Member States were required to carry out a mid-term evaluation of programmes and to submit the results to the Commission by the end of 2003 (15). — By the same date, under the arrangements for the performance reserve, they were required to assess the performance of operational programmes (16). — By 31 March 2004 the Commission was to allocate the performance reserve on the basis of a proposal from the Member State concerned (17). — Spending plans could, if necessary, be subject to a subsequent mid-term revision (18).

12. The Structural Funds Regulation provided that mid-term evaluations were intended as an input into both the allocation of the performance reserve and any mid-term revision of programmes: the Commission was therefore under an obligation to examine whether the MTEs were suitable for those purposes (19).

SCOPE OF THE AUDIT

13. The Court's audit examined whether these three 'mid-term processes' were carried out effectively and how they affected spending from the Structural Funds.

14. In particular, the audit aimed to establish whether — mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure; — the performance reserve was allocated so as to favour effective and well-managed spending programmes; — the mid-term revision updated priorities and improved the targeting of spending programmes.

15. The audit concentrated on the ERDF and ESF, as the main components of the Structural Funds. In the course of the audit, the Court: — reviewed Commission guidelines for the three processes, — examined evaluation reports and proposals for use of the performance reserve and for mid-term revision for 5 CSFs and 20 operational programmes/single programming documents (20), — considered the Commission's own assessment of the evaluation exercise, — analysed the allocation of the performance reserve and the impact of mid-term revision.

TIMETABLE

16. These 'mid-term' processes occurred at an early stage in the 2000-2006 programme period in terms of expenditure undertaken. — Community budgets since 2000 have been consistent with the Berlin European Council agreement that the Union should enter into commitments during 2000-2006 to spend 195 billion euro from the Structural funds. As is normal for multiannual programmes, however, payments — spending the money — have lagged behind commitments. — Operational programmes were adopted more slowly than planned (see Special Report No 7/2003, paragraphs 18-24): most of them in 2001, some only in 2002. — In 2000 and 2001 the primary concern of Member States and regions was to complete the implementation of the 1994-1999 generation of Structural Fund programmes, which slowed down the implementation of new programmes. — Since Member States had to submit the mid-term evaluation by the end of 2003, evaluators were obliged to base their assessment on information and data available at the end of 2002 or, at the latest, autumn 2003.

17. At the end of 2002, almost the entire Berlin totals had been allocated to programmes, but only about 20 billion euro (about 1/10 of the total) had been spent. Even at the end of 2003, only about 60 billion euro (about one third) had been spent.

ASSESSMENT OF MID-TERM EVALUATION

18. As explained in paragraph 8, the Structural Funds Regulation itself set out limited aims for the mid-term evaluation. A Commission working paper (Working Paper No 8: The Mid-Term Evaluation of Structural Fund Intervention, 5.12.2000) provided more extensive guidance and more ambitious goals to Member States for the mid-term evaluation. The core of the evaluation was to be the assessment of how well the form of assistance was performing in reality, judged on the basis of its continued relevance and the effectiveness and efficiency of its implementation. The working paper set out the evaluation questions and the content of an evaluation report (See box 3). Box 3Role of the MTE according to Working Paper No 8 — To assess whether the overall form of assistance remains the appropriate means to address the issues confronting the region or sector. — To review whether the strategic axes, priorities and objectives are coherent and still relevant, how far progress has been made towards the achievement of these objectives and the extent to which they can actually be achieved. — To assess the quantification of objectives, specifically the extent to which they have facilitated monitoring and evaluation. — To assess the extent to which horizontal priorities — equal opportunities and the environment in particular — have been integrated into the forms of assistance. — To analyse the adequacy of the implementation and monitoring arrangements. — To present the results against the indicators agreed for the performance reserve. Source: Commission Working Document No 8 (pages 4 and 5).

19. The same Commission Working Paper indicated that the scale of the evaluation (and therefore its cost) should be proportional to the scale of the programme — giving the example that macro-economic modelling would be appropriate for large Objective 1 CSFs, but not for small Objectives 2 or 3 programmes. The fees for the evaluation were financed partly from EU funds. The final cost of evaluation fees amounted to 40 million euro, and thus remained below 0,1 % of annual expenditure on the Structural Funds. (In addition the Commission incurred administrative costs in dealing with evaluations: these are not readily quantifiable).

The results of the evaluations

20. Almost all evaluations concluded that the initial development strategy remained relevant but that it was too early to assess effectiveness. The Commission summary used the findings of the evaluations in its report ‘The Mid-Term Evaluation in Objective 1 and 2 Regions: Growing Evaluation Capacity’ (November 2004). The Commission's summary is set out in Box 4. Box 4Evaluators' response to the evaluation questions — In nearly all cases the evaluators concluded that the strategies adopted in the programmes were still appropriate. — It was too early to assess effectiveness in many cases, given the late or slow start-up of programmes. A difficulty was presented by the monitoring system which often required revision of the definition of indicators and their targets. — It was likewise too early to measure impact, although evaluations did in some cases recommend the revision of targets based on the experience of the early years of implementation. Macro economic modelling undertaken for large CSFs suggests that programmes are on course to achieve their global objectives. — Efficiency was more often defined as financial absorption rather than the cost per output or result achieved. Financial absorption is clearly improved (i.e. money was spent more quickly than in the past), driven by the requirements of the N + 2 rule. — Evaluations found an urgent need to quantify objectives better. — Implementation systems were found to fulfil the regulatory requirements and to be significantly improved on the past. Source: Commission Report: ‘The Mid-Term Evaluation in Objective 1 and 2 Regions — Growing Evaluation Capacity’ — November 2004 (page 19).

Commission assessment of evaluations

21. Member States, monitoring committees and the Commission all played a role in evaluating the quality of evaluations. The services of the Commission examined the evaluation reports at draft and final stage. The Commission had to operate within the ambitious deadlines set by the Regulation and assess the evaluation reports on about 400 programmes simultaneously from January to March 2004. The Commission hired outside evaluators in addition to its own staff in order to finish the task on time.

22. The Commission review of the findings of evaluators (21), is set out in ‘Growing Evaluation Capacity’. The Commission found that: — evaluations were in general based upon appropriate methodology; — poor monitoring systems had hampered many evaluations; — primary research into the reality of programmes required improvement; — some evaluations were weak on analysis, or evaluations often did not produce sufficiently operational conclusions and recommendations; — results of economic modelling, generally limited to larger programmes, could not be compared because of great variations in the approaches applied; — some evaluators overemphasised financial analysis, with a limited focus on what was being achieved by the programmes because of the unavailability of monitoring data, but also because of the strong focus on spending due to the demands of N + 2. This led to recommendations which were purely finance-driven, rather than reviewing the strategic implications of such reallocations (22).

Assessment by the Court

23. In its audit, the Court identified a number of problems faced by, and difficulties identified by, evaluators in giving a clear answer as to whether the overall form of assistance remained appropriate to address the issues confronting the region or sector, and whether objectives could be achieved. — All evaluators had to cope with the problem that only limited spending had been made at the time of evaluation. — Almost half of the evaluators of the programmes which the Court examined identified important issues of concern linked to concrete development needs and proposed stronger emphasis on certain policy fields, especially linked to the Lisbon and Gothenburg strategies. — Other evaluators drew attention to the intrinsic difficulty of evaluation, such as the difficulty of determining the impact of additional investment in road construction or of establishing the impact of ESF intervention on net employment levels. For example commenting on investment in roads, the German evaluator suggested that further investment in economies with good existing infrastructure provision could even lead to slower growth than without it (23). The evaluators for the UK and Gibraltar programmes commented that, when considering the employment impact of the ESF, only gross job creation could be monitored.

24. The Commission's summary report ‘Growing Evaluation Capacity’ gave less attention to these problems and difficulties, than to more positive evaluation outcomes.

25. Some evaluators were also able to identify significant problems in the way programmes were operating. For example: — the evaluator of the Italian OP PONSIL (financial aid for enterprises) estimated a deadweight (24) effect of 50 %; — in the UK, evaluators commented that the programme had had only very modest success in securing sizeable net movements of individuals into the labour market who were previously outside it, and noted that ESF support tended to reinforce existing patterns of training within companies, in which well-qualified individuals generally received more training; However the report also stated that the programme had been ‘disproportionately’ successful in helping the most disadvantaged beneficiaries find work; — the evaluators of the Austrian Burgenland operational programme observed that the N + 2 rule encouraged higher spending rather than effective allocation; — where evaluators used a macroeconomic model (Germany, Spain, Portugal and Italy) estimates of the impact of the funds (in particular on GDP growth) were often significantly reduced as compared with the ex ante appraisal, suggesting that programmes were unlikely to meet their objectives; — out of the 25 selected CSFs, OPs and SPDs more than half of the MTEs noted weaknesses with regard to project selection and the integration of horizontal principles such as the environment, equal opportunities and the information society. In many cases the evaluations however only described weaknesses without proposing operational recommendations.

26. In some other respects, however, the Court's audit confirmed the Commission's analysis. Evaluators responded to the range of questions set out in Working Paper No 8, and attempted to assess the continuing coherence and validity of the strategies underlying CSFs and OPs (in most cases re-examining social and economic factors and preparing a revised analysis of programmes' strengths, weaknesses, opportunities and threats). The Court's audit showed that implementation was more robustly assessed than effectiveness, efficiency and impact: — the evaluations contained clear assessments of the physical and financial progress of OPs and CSFs; — estimates of effectiveness and impact, on the other hand, were not always carried out, or were carried out in a partial manner (e.g. gross jobs created were in almost all cases presented as equivalent to the employment effects, concerning both ERDF and ESF measures); — deadweight and substitution effects were seldom considered; very often evaluations did not attempt directly to assess these effects; — efficiency was not always analysed, and where it was, no conclusions or benchmarks were identified, nor recommendations made on policy; — in some cases the validity of the underlying strategy was not properly addressed.

27. The Court's overall assessment is that the evaluation process provided a check on the progress being made with programmes and confirmed that implementation systems were improved from the previous generation of Structural Funds. On the other hand, there were significant weaknesses in monitoring, and it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes.

ALLOCATION OF PERFORMANCE RESERVE

28. The Regulation from the outset provided a limited number of quantified indicators. The Commission provided guidance to Member States in a working paper issued in 1999 (25) on how to assess programmes against the criteria for allocation of the performance reserve: effectiveness, the quality of management and financial implementation. Performance assessment was to be based on monitoring indicators measuring the mid-term results in relation to their specific initial targets.

29. The Commission's working paper included examples of indicators (26) to be taken into account in carrying out the review of performance. For effectiveness: — a basket of some outputs covering at least half the value of the programme; — comparison of actual and planned results for employment (temporary/permanent jobs created or maintained) or employability of target groups. For management: — four types of indicators, measuring monitoring, financial control, project selection and evaluation systems. For financial performance: — absorption and rate of private funding.

Assessments by Member States and allocation of the performance reserve

30. Member States were required to evaluate performance for a period when on average only 20 % of expenditure had been undertaken. Some evaluators drew attention to this problem. For example, the Italian evaluator pointed out that it was too early to carry out a general analysis of effectiveness. There were however significant differences in the speed with which programme priorities could be implemented. Overall, the programmes examined by the Court showed faster spending at mid-term for physical infrastructure than for some priorities and measures related to the Lisbon and Gothenburg objectives.

31. Member States tried to assess performance against indicators, though the process was handicapped by the limitations of some of the indicators developed in the ex ante assessment of 2000-2006 programmes. For example, for the German and Italian programmes, effectiveness was only monitored through output indicators (27).

32. Member States' proposals for allocation of the performance reserve were based on the criteria listed in paragraph 29 above. In the case of the Spanish, Portuguese and Irish Community Support Frameworks, Member States went further and assessed performance by reference to a wider set of indicators than recommended by the Commission.

33. Nevertheless, Member States' proposals for the allocation of the performance reserve were significantly constrained by a perceived need to maximise absorption of EU funds. The Commission itself emphasised the importance of absorption. For example, in a Commission note of 16 October 2003 to the members of the CDCR ‘preparation for the allocation of the performance reserve’, communicated also to the Member States, it was stressed that ‘the Commission will pay particular attention to the absorption of funds in the case of significant losses under the N + 2 mechanism and the quality of the mid-term evaluation (28).’ Examples include the following: — the German transport OP aimed to spend approximately equal amounts on road and rail. But on absorption grounds it was proposed to allocate all of the performance reserve to road construction. When submitting the proposal to the Commission, the managing authority stated that the performance reserve was not employed in line with environmental or transport policy objectives, but according to progress in spending; — SPD Réunion: Some environmental and equal opportunities measures did not receive any allocation of the performance reserve. The managing authority underlined that it was in some cases difficult to reconcile certain Community objectives with the need to spend funds; — the proposal of Brandenburg, Germany, made mainly in the light of the analysis of absorption, excluded environmental investment measures from the allocation. Transport infrastructure received the highest amount. The evaluators of the programme also concluded that the Commission's indicators were rather based on absorption than on effectiveness.

34. In addition there were several allocations of the performance reserve exclusively to the ERDF, mainly on the grounds of absorption, for example in Germany (OP Saxony), Ireland (the performance reserve of all Irish OPs was entirely allocated to one ERDF financed programme), Italy (OP Sviluppo Imprenditoriale Locale) and some Objective 2 Austrian, German, French, and Spanish and UK OPs.

35. The time-table for allocation of the performance reserve was tight. The proposals were received in January 2004 and agreement in principle had to be concluded by early February.

36. Some use was made of the performance reserve to allocate money to programmes on the basis of effectiveness and quality of management. Overall, however, the performance reserve was primarily used to maximise spending rather than to concentrate spending on activities which were assessed as particularly effective.

IMPACT OF MID-TERM REVISION

37. The 2000-2006 reform of the Structural Funds modified and strengthened the Funds' objectives. The intention was to promote growth and employment in a way which took account of and combined interrelated economic, social and environmental policy objectives (29). In particular, expenditure from the funds was to take account of new technologies and innovation, research, the information society, clean and efficient use of energy (including renewable energy), better protection of the environment, social inclusion and life-long learning.

38. Allocation of the performance reserve and shifts of funds following the mid-term revision had to take account of the programming principles set out in the Commission's programming guidelines (30). The Commission however issued its revised programming guidelines only in August 2003. This was too late for most Member States and regions to take the revised guidelines into account in the MTR. On the other hand the revised guidelines involved only a number of qualitative shifts in the initial guidelines, for instance in the consolidation of the Lisbon strategy, in the fields of transport (emphasis on intermodality, shift from road to rail and waterways) and energy.

39. All operational programmes and Community support frameworks had to be revised as a result of the allocation of the performance reserve. This involved the adaptation of each programme through a similar process as applied during the initial programming, including consultation of relevant Commission Directorates General and formal adoption by the Commission.

40. There was no obligation on Member States to revise OPs/CSFs for reasons other than allocation of the performance reserve. Of the 25 OPs and CSFs examined by the Court, revisions for other purposes were made in 15 cases.

41. In many cases, the mid-term revision improved the way in which OPs took account of policy priorities or changes in EU policies. In some cases performance indicators were improved and changes made to management structures. Changes of these types were for example made in Ireland, Portugal and Italy.

42. As already noted in respect of the performance reserve, however, priority was given to activities where money could be spent, for instance: — SPD Burgenland, Austria. Main changes: the reduction of planned spending on research, technology and development and landscape conservation because of absorption difficulties, — SPD Hainaut, Belgium: absorption played an important role, for instance planned spending on the ‘management of the environment’ was substantially reduced on the grounds of low absorption and transferred to aid for redevelopment of industrial sites, — OP Brandenburg, Germany: planned spending on environmental measures was reduced substantially, the measure ‘Renewable Energy’ was abandoned, — OP Transport, Germany: the allocation of the performance reserve only to road projects substantially changed the ratio of spending on roads to spending on rail, — OP Calabria: A large part of the revised programme concerns ‘compatible’ projects. These are public projects, which were already financed with national funds prior to the current programme period. They were not foreseen in the initial programme. This approach was mainly applied in order to avoid decommitments, — CSF Italy, S&E OP Ireland: The financial plans were substantially changed because private funding initially foreseen was reduced (CSF Italy by 95 %, S&E OP by 70 %) and partly replaced by national public funds. The result was to increase nationally funded spending and thus avoid decommitments.

43. The Court's examination found that transfers of funds for environmental measures were made mainly in favour of aid for enterprises and transport infrastructure (especially roads). This was the case, for example, in Germany, Spain, Italy and Belgium, amounting to around 800 million euros (about 1 % of Structural Fund allocations for these Member States).

44. Overall the Court found that, in respect of a number of Structural Fund programmes, mid-term revision led to improvements (for example taking account of policy priorities or changes in EU policies or improving performance indicators). A significant impact in practice however was to shift budgets away from underspending measures.

ARRANGEMENTS FOR 2007-2013

45. Revised arrangements for evaluation, review and revision of programmes have been agreed for the next Structural Fund programme period 2007-2013. They differ in a number of respects from the arrangements which applied in 2000-2006. In general, where changes have been made, they provide for a greater degree of flexibility than under the 2000-2006 arrangements and should assist in ‘fine tuning’ mid-term evaluation and the revision of spending plans to the particular circumstances of individual programmes. — There is still a requirement for evaluation before, during and after the programme period. Evaluations must, in particular, take account of the objective of sustainable development and of the relevant Community legislation concerning environmental impact and strategic environmental assessment (31). — On the other hand there is no obligation to carry out, for all programmes, a mid-term review at a specific point in the programme period. — The principle of ‘proportional intervention’ applies to evaluation. Financial and administrative resources employed by the Commission and the Member States in the implementation of the funds in relation to the selection of indicators, evaluations, management and control system and reporting shall be proportional to the total amount of expenditure allocated to an operational programme (32). — There is no compulsory performance reserve. Member States may instead choose to establish a national contingency reserve and/or a national performance reserve (33). (The procedures for allocating any national performance reserve resemble those applying to the performance reserve in 2000-2006). — As for 2000-2006 mid-term revision of programmes is not compulsory for all programmes. (Revision of the central programming guidelines agreed by the Council does not require Member States to change their operational programmes or national strategic reference framework). Two cases in which Member States might decide to carry out a review are however identified in the Regulation: 1. significant socio-economic changes and to take greater or different account of major changes in Community, national or regional priorities, 2. in the light of a significant departure from the goals initially set or following implementation difficulties.

46. The ‘N + 2’ rule will still apply, but will (for a limited period) become an ‘N + 3’ rule for Member States whose GDP per head from 2001 to 2003 was below 85 % of the EU-25 average.

CONCLUSIONS

47. Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes and confirmed that implementation systems were significantly improved from the previous generation of Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a fortiori, the impact of programmes.

48. The performance reserve was used primarily to maximise spending rather than to concentrate spending on areas which were shown to be particularly effective. The Commission emphasised the importance of absorption (see paragraph 33).

49. In a number of cases the mid-term revision of Structural Fund programmes improved the way in which operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant effect of the mid-term revision was to give priority to activities where money could be spent.

50. Serious efforts were made to carry out the mid-term processes as set out in the Structural Funds Regulation. On the other hand, since the mid-term evaluations were carried out at a relatively early stage in the programme, the mid-term evaluations and revisions could provide little assurance that Structural Fund spending would have the effects and impacts desired. The Member States and the Commission also made use of the performance reserve and the mid-term revision to promote spending that could be carried out within the permitted time-frame.

51. The Court is of the view that it remains appropriate to include in large scale expenditure programmes, running for as long as 7 years, provision for reconsideration of programme objectives, measures and budgetary allocations during the course of the programme. But the audit illustrated tensions inherent in the planning and management of the Structural Funds which need to be borne in mind when designing and applying arrangements for mid-term evaluation and revision. — Expenditure should be efficient, effective and economic. On the other hand the commitment totals agreed for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as well as ceilings on spending (34). Absorption is itself an implicit objective for EU spending on cohesion. — The N + 2 rule provides a discipline to ensure that spending is well planned, but creates an incentive for the Member States and the Commission to favour activities where spending can be undertaken with relative ease in allocation of any reserve. — Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the effectiveness and impact of expenditure. A corollary is that, if evaluation is to play a significant part in the allocation of the funds, it is important to carry out a high quality ex ante evaluation.

52. The particular context of the 2000-2006 programme period exacerbated some of these problems. Expenditure for the 1994-1999 period was not subject to the N + 2 rule, and was significantly delayed (indeed the 1994-1999 programmes are still not closed). This contributed to the delay in starting programmes for the 2000-2006 period. Some of the other limitations of mid-term evaluation are more fundamental: it is intrinsically difficult to assess the net outcome of infrastructure investments, or of employment initiatives.

RECOMMENDATIONS

53. The Court recommends the following steps to improve the usefulness of medium term evaluation, revision and reallocation during the 2007-2013 and future programme periods: — ‘Medium term’ evaluation and revision of programmes are more likely to be successful, the greater the extent to which expenditure takes place during the programme period, rather than following some years in arrears. In order to reduce delays in getting the 2007-2013 programme underway, the Commission and Member States should ensure that closure of the 2000-2006 programmes takes place speedily and efficiently. — A particular difficulty found by evaluators of the 2000-2006 programmes was that monitoring systems were operating poorly. The Commission should take steps to ensure that performance indicators are monitored from the outset in the new programme period. — Evaluation and reassessment of expenditure should take place late enough for there to be sufficient experience of a programme to permit useful appraisal to take place, but early enough to permit changes in priorities. — The Mid-Term processes in 2000-2006 constituted a lengthy and complex procedure. The Commission should consider ways in which the process can be made more rapid and responsive. — While the Commission should not be prescriptive in defining the method to be used by evaluators, it should identify how programmes can be amended so as to ensure that impacts on cohesion are identifiable, that evaluations are a source of best practice, and that, where performance reserves are established, they are used in a way which gives proper weight to the effective and efficient implementation of programmes. — All guidance provided by the Commission should be made available in good time. — The Commission and the Member States should give special attention to the programming and management arrangements for expenditure on EU policy priorities.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 8 March 2007. For the Court of Auditors Hubert WEBER President

(1) About 16 billion euro Structural Funds for the years 2004-2006 were allocated to around 200 programmes of the new Member States.

(2) See Title II Programming — Article 13-19 of Council Regulation (EC) No 1260/1999 of 21 June 1999 laying down general provisions on the Structural Funds (OJ L 161, 26.6.1999, p. 1).

(3) Article 10(3) of Regulation (EC) No 1260/1999.

(4) Article 18 and 19 of Regulation (EC) No 1260/1999. In cases where Community allocation did not exceed 1 billion euro the assistance could take the form of a ‘Single Programming Document’ (SPD), containing the same information to be found in a CSF and OP.

(5) Article 14(2) of Regulation (EC) No 1260/1999.

(6) Article 35(2) of Regulation (EC) No 1260/1999.

(7) Performance reserve, see Article 44, rule ‘N + 2’ Article 31(2) of Regulation (EC) No 1260/1999.

(8) The total amount of Structural Funds allocated to an Objective 1 programme is, according to its financing plan, committed in yearly tranches from 2000 to 2006. Payments may take the form of payments on account (representing 7 % of the total envelope of each programme, providing at the beginning a relatively long protection against the application of N + 2) or the rembursement of expenditure actually paid out to final recipients. In order to avoid decommitment Member States have to submit to the Commission by the end of the second year following the year of commitment a payment application, covering for each year the amounts forseen in the financing plan (Articles 31(2) and 32(3) of Regulation (EC) No 1260/1999).

(9) Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 on the Financial Regulation applicable to the EC general budget (OJ L 248, 16.9.2002, p. 1), Article 27. Economy means that recources shall be made available in due time, in appropriate quantity and quality and at the best price. Effectiveness means attaining the objectives set and achieving the intended results; efficiency means the best relationship between resources employed and results achieved. For the evaluation of the effectiveness of the SF refer also to Article 40 of Regulation (EC) No 1260/1999.

(10) Article 41 of Regulation (EC) No 1260/1999.

(11) Article 42 of Regulation (EC) No 1260/1999.

(12) Article 43 of Regulation (EC) No 1260/1999.

(13) Special Report No 7/2003 on the implementation of assistance programming for the period 2000-2006 within the framework of the Structural Funds (OJ C 174, 23.7.2003).

(14) See Special Report No 10/2006 on ex post evaluations of Objectives 1 and 3 programmes 1994-1999.

(15) Article 42(2) of Regulation (EC) No 1260/1999.

(16) Article 44(1) of Regulation (EC) No 1260/1999.

(17) Article 44(2) of Regulation (EC) No 1260/1999.

(18) Article 14(2) in combination with Title II Programming and Article 10(3) of Regulation (EC) No 1260/1999.

(19) Article 42(3) of Regulation (EC) No 1260/1999.

(20) Belgium SPD: Flanders Objective 3; France: SPD's Corsica and Objective 3; Germany: CSF and OP's Brandenburg, Transport; Italy: CSF and OP's Calabria, Ponsil, Transport; SPD Lombardy Objective 3; Ireland: CSF and OP's South-eastern, productive sector; Spain: CSF and OP's Andalusia, Galicia; UK: SPD Merseyside, OP Objective 3.

The processes of mid-term evaluations, allocation of performance reserve and revision concerning the CSF Portugal, the OP Algarve (P), the SPD Burgenland (A), the SPD Hainaut (B), the SPD Réunion (F), the SPD Flevoland (NL) were examined at Commission level.

(21) At the draft stage, the Commission assessed one third of the reports as good or excellent. The assessment of the final report was much more positive: two-thirds of the reports were assessed as good or excellent. Where weaknesses were identified, the main cause was that evaluators were stretched by:

— the number of evaluation questions;

— the number of projects involved;

— the complexity of some large programmes, and the limited resources of some evaluators.

(22) Growing Evaluation Capacity, European Commission, November 2004 (page 35).

(23) There are a number of empirical studies and evaluations showing these effects. In general, there is insufficient data with regard to the real needs, including the kind of infrastructure in order to optimise the effects of the Structural funds.

(24) A ‘deadweight’ effect exists where an activity or investment would have been undertaken in the absence of financing.

(25) European Commission, the new programming period 2000-2006: methodological Working Paper 4 — Implementation of the performance reserve.

(26) These indicators are related: financial absorption and output are strongly correlated.

(27) The Court's Special Report No 7/2003 pointed out that in practice, the performance reserve entailed a complex procedure for reallocating funds, and could lead to inconsistencies and ineffectiveness in the use of the reserve.

(28) In one case, the Dutch Objective 3 programme, the Commission had no option other than to allocate the performance reserve despite the fact that significant decommitments had already taken place because funds had not been spent.

(29) This corresponded to the requirements of the Union's concept of sustainable development of the Amsterdam Treaty and anticipated largely the conclusions of the Lisbon European Council (March 2000: ‘to become the most competitive and knowledge-based economy in the world’) to which the Gothenburg European Council (June 2001) added the environmental strategy of sustainable development.

(30) See Articles 10(3) and 14 of Regulation (EC) No 1260/1999. ‘… in accordance with this title’. This ‘title’ refers to the programming provisions of Articles 15 to 19.

(31) Council Regulation (EC) No 1083/2006 of 11 July 2006 laying down general provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund and repealing Regulation (EC) No 1260/1999 (OJ L 210, 31.7.2006, p. 25), Article 47.

(32) Regulation (EC) No 1083/2006, Article 13.

(33) Regulation (EC) No 1083/2006, Articles 50 and 51.

(34) According to paragraph 12, third subparagraph of the Interinstitutional Agreement (including the financial perspective) between the European Parliament and the Council and the Commission of 6 May 1999 (OJ C 172, 18.6.1999, p. 1) the European Parliament and the Council undertake to respect the allocations of commitment appropriations provided in the financial perspective for structural operations.