Special Report No 1/2007 concerning the implementation of the mid-term processes on the Structural Funds 2000-2006 together with the Commission's replies
Date: 2007-03-08
Institutions: European Court of Auditors
Subjects: EU financing, Structural Funds, audit, financial control, financial management, impact study, operational programme, project evaluation, report

5.6.2007

EN

Official Journal of the European Union

C 124/1

IV

(Notices)

NOTICES FROM EUROPEAN UNION INSTITUTIONS AND BODIES

COURT OF AUDITORS

SPECIAL REPORT No 1/2007

concerning the implementation of the mid-term processes on the
Structural Funds 2000-2006 together with the Commission’s replies

(pursuant to Article 248(4), second subparagraph, EC)

(2007/C 124/01)

TABLE OF CONTENTS

Paragraphs

Page

SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Structural Funds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Structural Funds 2000-2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Evaluation and appraisal of the Structural Funds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mid-term evaluation and review and the allocation of the performance reserve . . . . . . .

SCOPE OF THE AUDIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

TIMETABLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ASSESSMENT OF MID-TERM EVALUATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The results of the evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commission assessment of evaluations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Assessment by the Court

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ALLOCATION OF PERFORMANCE RESERVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Assessments by Member States and allocation of the performance reserve . . . . . . . . . . .

IMPACT OF MID-TERM REVISION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ARRANGEMENTS FOR 2007-2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

I-V

1-12

1-2

3-6

7-10

11-12

13-15

16-17

18-27

20

21-22

23-27

28-36

30-36

37-44

45-46

47-52

53

The Commission’s replies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

3

3

3

4

5

5

5

6

6

6

7

8

8

9

10

11

11

13

C 124/2

EN

Official Journal of the European Union

5.6.2007

SUMMARY

The Structural Funds Regulation provided for a series of steps roughly half way through the 2000-2006

I.
programme period:

— a mid-term evaluation of programmes;

— allocation of the performance reserve; and

— mid-term revision of expenditure plans.

The Court’s audit examined whether these three ’mid-term processes’ were carried out effectively and

II.
how they affected spending from the Structural Funds: in particular, whether

— mid-term evaluations provided an appropriate and useful appraisal of Structural Fund expenditure;

— the performance reserve was allocated so as to favour effective and well-managed spending programmes;

— the mid-term revision updated priorities and improved the targeting of spending programmes.

III.

The Court concluded as follows:

— Mid-term evaluation provided a check on the progress made in implementing Structural Fund programmes
and confirmed that implementation systems were significantly improved from the previous generation of
Structural Funds. On the other hand it was in many cases too early to assess the effectiveness and, a for-
tiori, the impact of programmes.

— The performance reserve was used primarily to maximise spending rather than to concentrate spending
on areas which were shown to be particularly effective. The Commission emphasised the importance of
absorption.

— In a number of cases the mid term revision of Structural Fund programmes improved the way in which
operational programmes took account of policy priorities or changes in EU policies (e.g. as reflected in
the Lisbon and Gothenborg European Councils) or improved performance indicators. A significant impact
of mid-term revision was to give priority to activities where money could be spent.

IV.
The Court considers that it remains appropriate to include, in expenditure programmes such as the
Structural Funds, provision for reconsideration of programme objectives, measures and budgetary allocations
during the course of the programme. But the audit illustrated tensions inherent in the planning and manage-
ment of the Structural Funds, which need to be borne in mind when designing and applying arrangements for
mid-term evaluation and revision:

— Expenditure should be efficient, effective and economic. On the other hand, the commitment totals agreed
for the Structural and Cohesion Funds and set out in successive EU financial perspectives are targets as
well as limits on spending. Absorption is an implicit objective for EU spending on cohesion.

— The N + 2 rule provides a discipline to ensure that spending is well planned, but also creates an incentive
for the Member States and the Commission to favour activities where spending can be undertaken with
relative ease.

— Mid-term revision poses problems of timing. The later the revision, the less the chance of making changes
to a programme; but the earlier any revision, the more difficult it is to form a robust judgment about the
effectiveness and impact of expenditure.

5.6.2007

EN

Official Journal of the European Union

C 124/3

The Court makes a number of recommendations designed to increase the usefulness of mid-term review
V.
and revision in the context of the more flexible arrangements for the Structural Funds which will apply for the
2007-2013 programme period:

— speedy and efficient closure of the 2000-2006 programmes;

— better performance indicators;

— careful choice of timing for evaluation of expenditure (taking advantage of the increased flexibility of the

reformed system);

— shorter and quicker mid-term revision;

— timely and relevant guidance from the Commission;

— special attention to the programmes and management arrangements for expenditure for EU policy

priorities.

INTRODUCTION

The Structural Funds

4.
The planned expenditure totals agreed at Berlin were allo-
cated to some 400 national and regional programmes (1) through
a complex ’programming’ process, whose main elements were as
follows (2):

1.
The Structural Funds are the Union’s principal policy
instrument in support of the Treaty objectives of economic and
social cohesion.

2.
There are four Structural Funds: the European Regional
Development Fund (ERDF), the European Social Fund (ESF), the
European Agricultural Guidance and Guarantee Fund, ‘Guidance
section’ (EAGGF-Guidance), and the Financial Instrument for
Fisheries Guidance (FIFG). Expenditure is planned and carried out
on a multiannual basis in ‘programme periods’. It supports invest-
ment in physical and human capital within Member States. Mem-
ber States pay part of the cost of (co-finance) the investment
projects concerned.

— The Commission established (3), for the initial programming
exercise and subsequently for the mid-term review, indica-
tive programming guidelines in relation to the objectives of
the Funds;

— Member States negotiated with the Commission ‘Community
Support Frameworks’ (CSF), which set out the strategy and
priorities for action of the Funds and the Member State;

The Structural Funds 2000-2006

— Each CSF was implemented by means of ‘operational pro-

grammes’ (OP) (4) approved by the Commission;

3.
The European Council agreed at its meeting in Berlin
(March 1999) that 195 billion euro (219 billion euro in 2005
prices) should be made available from the Structural Funds for the
period 2000-2006. Over 90 % of spending from the funds was to
be used in pursuit of three objectives:

— Objective 1: promoting the development and structural
adjustment of regions whose development is lagging behind
(regions whose per capita GDP is less than 75 % of the Com-
munity average). About 70 % of total spending was to be
devoted to Objective 1;

— Objective 2: supporting the economic and social conversion
of areas (outside Objective 1 regions) facing structural diffi-
culties (industrial, rural, urban areas and areas dependent on
fisheries);

— Objective 3: supporting (outside Objective 1 regions) the
adaptation and modernisation of policies and systems of
education, training and employment.

— Regions or Member States then approved individual projects

to implement operational programmes;

— There was provision for programmes to be revised midway
through the programme period (mid-term revision) (5).

(1) About 16 billion euro Structural Funds for the years 2004-2006 were
allocated to around 200 programmes of the new Member States.
(2) See Title II Programming — Article 13-19 of Council Regulation (EC)
No 1260/1999 of 21 June 1999 laying down general provisions on
the Structural Funds (OJ L 161, 26.6.1999, p. 1).
(3) Article 10(3) of Regulation (EC) No 1260/1999.
(4) Article 18 and 19 of Regulation (EC) No 1260/1999. In cases where
Community allocation did not exceed 1 billion euro the assistance
could take the form of a ‘Single Programming Document’ (SPD), con-
taining the same information to be found in a CSF and OP.

(5) Article 14(2) of Regulation (EC) No 1260/1999.

C 124/4

EN

Official Journal of the European Union

5.6.2007

Expenditure was to be supervised by ‘Monitoring Commit-
5.
tees’ set up by the Member State concerned and including Com-
mission representatives (1) (in an advisory capacity).

their achievement must be monitored by performance indicators.
Expenditure must also be subject
to ex ante and ex post
evaluations (4).

6.
Two innovations of relevance to this report were intro-
duced into the management of the Structural Funds for the 2000-
2006 programme period: a performance reserve and the ‘N + 2’
rule (2). See boxes 1 and 2.

Box 1

The performance reserve

The Structural Fund Regulations for the 2000-2006 programme
period required the Commission, when allocating Structural Fund
resources, to withhold 4 % from each operational programme. In
order to encourage the efficient and effective use of Community
funds, this reserve was to be allocated among and within each Mem-
ber State’s operational programmes in 2004, on the basis of the com-
parative success of these programmes.

The performance reserve was a limited measure. Funds could not be
switched among Member States nor between progammes coming
under different objectives. Nor could any part of the reserve be with-
held further.

Box 2

The N + 2 rule

The N + 2 rule is designed to prevent an excessive build up of unused
commitments for an operational programme, which can complicate
closure and interfere with successor programmes, by providing an
incentive to ‘spend the money’ in line with targets. It provides that if
a commitment has not led to payments within two years of being
entered in the budget (3), the commitment is cancelled (decommitted)
and not replaced, i.e. spending falls short of the totals agreed at the
Berlin European Council.

Evaluation and appraisal of the Structural Funds

7.
The Financial Regulation, which sets out rules for the man-
agement of the Community budget, provides that EU spending
must be economical, efficient and effective. Objectives must be set
sectors of activity covered by the budget, and
for all

(1) Article 35(2) of Regulation (EC) No 1260/1999.
(2) Performance reserve, see Article 44, rule ‘N + 2’ Article 31(2) of Regu-

lation (EC) No 1260/1999.

(3) The total amount of Structural Funds allocated to an Objective 1 pro-
gramme is, according to its financing plan, committed in yearly
tranches from 2000 to 2006. Payments may take the form of pay-
ments on account (representing 7 % of the total envelope of each pro-
gramme, providing at the beginning a relatively long protection
against the application of N + 2) or the rembursement of expenditure
actually paid out to final recipients. In order to avoid decommitment
Member States have to submit to the Commission by the end of the
second year following the year of commitment a payment application,
covering for each year the amounts forseen in the financing plan
(Articles 31(2) and 32(3) of Regulation (EC) No 1260/1999).

8.
The Structural Funds Regulations for the 2000-2006 pro-
gramme period set out how these principles were to be applied
to the Structural Funds. In particular there were to be:

— an ex ante evaluation, for each CSF or OP, carried out within
Member States, to provide a basis for preparing the develop-
ment plans (5);

— a mid-term evaluation, carried out by an independent asses-
sor under the responsibility of the authority managing the
expenditure, and in cooperation with the Commission and
the Member State; its purpose was to examine, in the light of
the ex ante evaluation, initial results, their relevance, the
achievement of targets, the use of financial resources, moni-
toring and implementation (6);

— an ex post evaluation, covering the use of resources, and the
effectiveness and efficiency of intervention, carried out by the
Commission in collaboration with the Member State and the
authority managing the expenditure concerned (7).

9.
The Court audited the arrangements for ex ante evaluation
and programming of the 2000-2006 generation of Structural
Funds in its Special Report No 7/2003 (8). The Court noted, inter
alia, that:

— budgetary allocations were determined less by a well-
established development strategy and the effectiveness of the
Structural Funds than by maximising likely take-up of
funding,

— ex ante evaluation had little impact on the programming pro-
cess. Evaluation was confined to justifying the programming
as proposed by Member States and only rarely considered its
validity and the underlying strategy. There was no critical
appraisal or analysis of alternative strategies.

(4) Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 on
the Financial Regulation applicable to the EC general budget (OJ L 248,
16.9.2002, p. 1), Article 27. Economy means that recources shall be
made available in due time, in appropriate quantity and quality and at
the best price. Effectiveness means attaining the objectives set and
achieving the intended results; efficiency means the best relationship
between resources employed and results achieved. For the evalua-
tion of the effectiveness of the SF refer also to Article 40 of Regula-
tion (EC) No 1260/1999.

(5) Article 41 of Regulation (EC) No 1260/1999.
(6) Article 42 of Regulation (EC) No 1260/1999.
(7) Article 43 of Regulation (EC) No 1260/1999.
(8) Special Report No 7/2003 on the implementation of assistance pro-
gramming for the period 2000-2006 within the framework of the
Structural Funds (OJ C 174, 23.7.2003).

5.6.2007

EN

Official Journal of the European Union

C 124/5

The Court has also audited the Commission’s ex post
10.
evaluations of most Structural Fund expenditure for the previous
programme period (1994-1999) (1). The Court found a number
of significant weaknesses in the evaluations carried out and in the
Commission’s oversight of them; and recommended ensuring
that available data and allocated resources were in line with set
objectives, which should be realistic, and some changes in scope
and approach for future evaluation exercises.

Mid-term evaluation and review and the allocation of the
performance reserve

The Structural Funds Regulation thus provided for a
11.
series of procedures roughly half way through the 2000-2006
programme period:

— Member States were required to carry out a mid-term evalu-
ation of programmes and to submit the results to the Com-
mission by the end of 2003 (2).

— By the same date, under the arrangements for the perfor-
mance reserve, they were required to assess the performance
of operational programmes (3).

— the performance reserve was allocated so as to favour effec-

tive and well-managed spending programmes;

— the mid-term revision updated priorities and improved the

targeting of spending programmes.

15.
The audit concentrated on the ERDF and ESF, as the main
components of the Structural Funds. In the course of the audit,
the Court:

— reviewed Commission guidelines for the three processes,

— examined evaluation reports and proposals for use of the
performance reserve and for mid-term revision for 5 CSFs
and 20 operational programmes/single programming
documents (7),

— considered the Commission’s own assessment of the evalua-

tion exercise,

— analysed the allocation of the performance reserve and the

impact of mid-term revision.

— By 31 March 2004 the Commission was to allocate the per-
formance reserve on the basis of a proposal from the Mem-
ber State concerned (4).

TIMETABLE

— Spending plans could, if necessary, be subject to a subsequent

mid-term revision (5).

The Structural Funds Regulation provided that mid-term
12.
evaluations were intended as an input into both the allocation of
the performance reserve and any mid-term revision of pro-
grammes: the Commission was therefore under an obligation to
examine whether the MTEs were suitable for those purposes (6).

SCOPE OF THE AUDIT

13.
The Court’s audit examined whether these three ’mid-
term processes’ were carried out effectively and how they affected
spending from the Structural Funds.

14.

In particular, the audit aimed to establish whether

— mid-term evaluations provided an appropriate and useful

appraisal of Structural Fund expenditure;

16.
These ’mid-term’ processes occurred at an early stage in
the 2000-2006 programme period in terms of expenditure
undertaken.

— Community budgets since 2000 have been consistent with
the Berlin European Council agreement that the Union
should enter into commitments during 2000-2006 to spend
195 billion euro from the Structural funds. As is normal for
multiannual programmes, however, payments — spending
the money — have lagged behind commitments.

— Operational programmes were adopted more slowly than
planned (see Special Report No 7/2003, paragraphs 18-24):
most of them in 2001, some only in 2002.

— In 2000 and 2001 the primary concern of Member States
and regions was to complete the implementation of the
1994-1999 generation of Structural Fund programmes,
which slowed
implementation of new
programmes.

down the

(1) See Special Report No 10/2006 on ex post evaluations of Objectives 1

and 3 programmes 1994-1999.

(2) Article 42(2) of Regulation (EC) No 1260/1999.
(3) Article 44(1) of Regulation (EC) No 1260/1999.
(4) Article 44(2) of Regulation (EC) No 1260/1999.
(5) Article 14(2)

in combination with Title II Programming and

Article 10(3) of Regulation (EC) No 1260/1999.
(6) Article 42(3) of Regulation (EC) No 1260/1999.

(7) Belgium SPD: Flanders Objective 3; France: SPD’s Corsica and Objec-
tive 3; Germany: CSF and OP’s Brandenburg, Transport; Italy: CSF and
OP’s Calabria, Ponsil, Transport; SPD Lombardy Objective 3; Ireland:
CSF and OP’s South-eastern, productive sector; Spain: CSF and OP’s
Andalusia, Galicia; UK: SPD Merseyside, OP Objective 3.
The processes of mid-term evaluations, allocation of performance
reserve and revision concerning the CSF Portugal, the OP Algarve (P),
the SPD Burgenland (A), the SPD Hainaut (B), the SPD Réunion (F), the
SPD Flevoland (NL) were examined at Commission level.

C 124/6

EN

Official Journal of the European Union

5.6.2007

— Since Member States had to submit the mid-term evaluation
by the end of 2003, evaluators were obliged to base their
assessment on information and data available at the end of
2002 or, at the latest, autumn 2003.

EU funds. The final cost of evaluation fees amounted to 40 mil-
lion euro, and thus remained below 0,1 % of annual expenditure
on the Structural Funds. (In addition the Commission incurred
administrative costs in dealing with evaluations: these are not
readily quantifiable).

17.
At the end of 2002, almost the entire Berlin totals had
been allocated to programmes, but only about 20 billion euro
(about 1/10 of the total) had been spent. Even at the end of 2003,
only about 60 billion euro (about one third) had been spent.

The results of the evaluations

ASSESSMENT OF MID-TERM EVALUATION

18.
As explained in paragraph 8, the Structural Funds Regu-
lation itself set out limited aims for the mid-term evaluation. A
Commission working paper (Working Paper No 8: The Mid-Term
Evaluation of Structural Fund Intervention, 5.12.2000) provided
more extensive guidance and more ambitious goals to Member
States for the mid-term evaluation. The core of the evaluation was
to be the assessment of how well the form of assistance was per-
forming in reality, judged on the basis of its continued relevance
and the effectiveness and efficiency of its implementation. The
working paper set out the evaluation questions and the content
of an evaluation report (See box 3).

Box 3

Role of the MTE according to Working Paper No 8

— To assess whether the overall form of assistance remains the
appropriate means to address the issues confronting the region
or sector.

— To review whether the strategic axes, priorities and objectives are
coherent and still relevant, how far progress has been made
towards the achievement of these objectives and the extent to
which they can actually be achieved.

— To assess the quantification of objectives, specifically the extent
to which they have facilitated monitoring and evaluation.

— To assess the extent to which horizontal priorities — equal
opportunities and the environment in particular — have been
integrated into the forms of assistance.

— To analyse the adequacy of the implementation and monitoring

arrangements.

20.
Almost all evaluations concluded that the initial develop-
ment strategy remained relevant but that it was too early to assess
effectiveness. The Commission summary used the findings of the
evaluations in its report ‘The Mid-Term Evaluation in Objective 1
and 2 Regions: Growing Evaluation Capacity’ (November 2004).
The Commission’s summary is set out in Box 4.

Box 4

Evaluators’ response to the evaluation questions

— In nearly all cases the evaluators concluded that the strategies

adopted in the programmes were still appropriate.

— It was too early to assess effectiveness in many cases, given
the late or slow start-up of programmes. A difficulty was pre-
sented by the monitoring system which often required revision
of the definition of indicators and their targets.

— It was likewise too early to measure impact, although evalu-
ations did in some cases recommend the revision of targets
based on the experience of the early years of implementation.
Macro economic modelling undertaken for large CSFs suggests
that programmes are on course to achieve their global objec-
tives.

— Efficiency was more often defined as financial absorption rather
than the cost per output or result achieved. Financial absorp-
tion is clearly improved (i.e. money was spent more quickly
than in the past), driven by the requirements of the N + 2 rule.

— Evaluations found an urgent need to quantify objectives

better.

— Implementation systems were found to fulfil the regulatory

requirements and to be significantly improved on the past.

Source:

Commission Report: ‘The Mid-Term Evaluation in Objec-
tive 1 and 2 Regions — Growing Evaluation Capacity’ —
November 2004 (page 19).

— To present the results against the indicators agreed for the per-

formance reserve.

Commission assessment of evaluations

Source:

Commission Working Document No 8 (pages 4 and 5).

The same Commission Working Paper indicated that the
19.
scale of the evaluation (and therefore its cost) should be propor-
tional to the scale of the programme — giving the example that
macro-economic modelling would be appropriate for large
Objective 1 CSFs, but not for small Objectives 2 or 3 pro-
grammes. The fees for the evaluation were financed partly from

21.
Member States, monitoring committees and the Com-
mission all played a role in evaluating the quality of evaluations.
The services of the Commission examined the evaluation reports
at draft and final stage. The Commission had to operate within
the ambitious deadlines set by the Regulation and assess the
evaluation reports on about 400 programmes simultaneously
from January to March 2004. The Commission hired outside
evaluators in addition to its own staff in order to finish the task
on time.

5.6.2007

EN

Official Journal of the European Union

C 124/7

The Commission review of the findings of evaluators (1),
22.
is set out in ‘Growing Evaluation Capacity’. The Commission
found that:

— Almost half of the evaluators of the programmes which the
Court examined identified important issues of concern linked
to concrete development needs and proposed stronger
emphasis on certain policy fields, especially linked to the Lis-
bon and Gothenburg strategies.

— evaluations were in general based upon appropriate

methodology;

— poor monitoring systems had hampered many evaluations;

— primary research into the reality of programmes required

improvement;

— some evaluations were weak on analysis, or evaluations often
did not produce sufficiently operational conclusions and
recommendations;

— results of economic modelling, generally limited to larger
programmes, could not be compared because of great varia-
tions in the approaches applied;

— some evaluators overemphasised financial analysis, with a
limited focus on what was being achieved by the programmes
because of the unavailability of monitoring data, but also
because of the strong focus on spending due to the demands
of N + 2. This led to recommendations which were purely
finance-driven, rather than reviewing the strategic implica-
tions of such reallocations (2).

Assessment by the Court

23.
In its audit, the Court identified a number of problems
faced by, and difficulties identified by, evaluators in giving a clear
answer as to whether the overall form of assistance remained
appropriate to address the issues confronting the region or sec-
tor, and whether objectives could be achieved.

— All evaluators had to cope with the problem that only lim-
ited spending had been made at the time of evaluation.

(1) At the draft stage, the Commission assessed one third of the reports
as good or excellent. The assessment of the final report was much
more positive: two-thirds of the reports were assessed as good or
excellent. Where weaknesses were identified, the main cause was that
evaluators were stretched by:
— the number of evaluation questions;
— the number of projects involved;
— the complexity of some large programmes, and the limited

resources of some evaluators.

— Other evaluators drew attention to the intrinsic difficulty of
evaluation, such as the difficulty of determining the impact
of additional investment in road construction or of establish-
ing the impact of ESF intervention on net employment lev-
els. For example commenting on investment in roads, the
German evaluator suggested that further investment in
economies with good existing infrastructure provision could
even lead to slower growth than without it (3). The evalua-
tors for the UK and Gibraltar programmes commented that,
when considering the employment impact of the ESF, only
gross job creation could be monitored.

24.
The Commission’s summary report ‘Growing Evaluation
Capacity’ gave less attention to these problems and difficulties,
than to more positive evaluation outcomes.

25.
Some evaluators were also able to identify significant
problems in the way programmes were operating. For example:

— the evaluator of the Italian OP PONSIL (financial aid for
enterprises) estimated a deadweight (4) effect of 50 %;

— in the UK, evaluators commented that the programme had
had only very modest success in securing sizeable net move-
ments of individuals into the labour market who were pre-
viously outside it, and noted that ESF support tended to rein-
force existing patterns of training within companies, in which
well-qualified individuals generally received more train-
ing; However the report also stated that the programme had
been ‘disproportionately’ successful in helping the most dis-
advantaged beneficiaries find work;

— the evaluators of the Austrian Burgenland operational pro-
gramme observed that the N + 2 rule encouraged higher
spending rather than effective allocation;

— where evaluators used a macroeconomic model (Germany,
Spain, Portugal and Italy) estimates of the impact of the funds
(in particular on GDP growth) were often significantly
reduced as compared with the ex ante appraisal, suggesting
that programmes were unlikely to meet their objectives;

(3) There are a number of empirical studies and evaluations showing these
effects. In general, there is insufficient data with regard to the real
needs, including the kind of infrastructure in order to optimise the
effects of the Structural funds.

(2) Growing Evaluation Capacity, European Commission, November

(4) A ‘deadweight’ effect exists where an activity or investment would

2004 (page 35).

have been undertaken in the absence of financing.

C 124/8

EN

Official Journal of the European Union

5.6.2007

— out of the 25 selected CSFs, OPs and SPDs more than half of
the MTEs noted weaknesses with regard to project selection
and the integration of horizontal principles such as the envi-
ronment, equal opportunities and the information society. In
many cases the evaluations however only described weak-
nesses without proposing operational recommendations.

26.
In some other respects, however, the Court’s audit con-
firmed the Commission’s analysis. Evaluators responded to the
range of questions set out in Working Paper No 8, and attempted
to assess the continuing coherence and validity of the strategies
underlying CSFs and OPs (in most cases re-examining social and
economic factors and preparing a revised analysis of programmes’
strengths, weaknesses, opportunities and threats). The Court’s
audit showed that implementation was more robustly assessed
than effectiveness, efficiency and impact:

performance reserve: effectiveness, the quality of management
and financial implementation. Performance assessment was to be
based on monitoring indicators measuring the mid-term results in
relation to their specific initial targets.

29.
The Commission’s working paper included examples of
indicators (2) to be taken into account in carrying out the review
of performance.

For effectiveness:

— a basket of some outputs covering at least half the value of

the programme;

— the evaluations contained clear assessments of the physical

and financial progress of OPs and CSFs;

— comparison of actual and planned results for employment
jobs created or maintained) or

(temporary/permanent
employability of target groups.

— estimates of effectiveness and impact, on the other hand,
were not always carried out, or were carried out in a partial
manner (e.g. gross jobs created were in almost all cases pre-
sented as equivalent to the employment effects, concerning
both ERDF and ESF measures);

For management:

— four types of indicators, measuring monitoring, financial

control, project selection and evaluation systems.

— deadweight and substitution effects were seldom considered;
very often evaluations did not attempt directly to assess these
effects;

For financial performance:

— absorption and rate of private funding.

— efficiency was not always analysed, and where it was, no con-
clusions or benchmarks were identified, nor recommenda-
tions made on policy;

— in some cases the validity of the underlying strategy was not

properly addressed.

27.
The Court’s overall assessment is that the evaluation pro-
cess provided a check on the progress being made with pro-
grammes and confirmed that implementation systems were
improved from the previous generation of Structural Funds. On
the other hand, there were significant weaknesses in monitoring,
and it was in many cases too early to assess the effectiveness and,
a fortiori, the impact of programmes.

ALLOCATION OF PERFORMANCE RESERVE

28.
The Regulation from the outset provided a limited num-
ber of quantified indicators. The Commission provided guidance
to Member States in a working paper issued in 1999 (1) on how
to assess programmes against the criteria for allocation of the

(1) European Commission, the new programming period 2000-2006:
methodological Working Paper 4 — Implementation of the perfor-
mance reserve.

Assessments by Member States and allocation of
performance reserve

the

Member States were required to evaluate performance for
30.
a period when on average only 20 % of expenditure had been
undertaken. Some evaluators drew attention to this problem. For
example, the Italian evaluator pointed out that it was too early to
carry out a general analysis of effectiveness. There were however
significant differences in the speed with which programme pri-
orities could be implemented. Overall, the programmes examined
by the Court showed faster spending at mid-term for physical
infrastructure than for some priorities and measures related to the
Lisbon and Gothenburg objectives.

31.
Member States tried to assess performance against indi-
cators, though the process was handicapped by the limitations of
some of the indicators developed in the ex ante assessment of
2000-2006 programmes. For example, for the German and Ital-
ian programmes, effectiveness was only monitored through out-
put indicators (3).

(2) These indicators are related: financial absorption and output are

strongly correlated.

(3) The Court’s Special Report No 7/2003 pointed out that in practice, the
performance reserve entailed a complex procedure for reallocating
funds, and could lead to inconsistencies and ineffectiveness in the use
of the reserve.

5.6.2007

EN

Official Journal of the European Union

C 124/9

Member States’ proposals for allocation of the perfor-
32.
mance reserve were based on the criteria listed in paragraph 29
above. In the case of the Spanish, Portuguese and Irish Commu-
nity Support Frameworks, Member States went further and
assessed performance by reference to a wider set of indicators
than recommended by the Commission.

Some use was made of the performance reserve to allo-
36.
cate money to programmes on the basis of effectiveness and qual-
ity of management. Overall, however, the performance reserve
was primarily used to maximise spending rather than to concen-
trate spending on activities which were assessed as particularly
effective.

33.
Nevertheless, Member States’ proposals for the allocation
of the performance reserve were significantly constrained by a
perceived need to maximise absorption of EU funds. The Com-
mission itself emphasised the importance of absorption. For
example, in a Commission note of 16 October 2003 to the mem-
bers of the CDCR ‘preparation for the allocation of the perfor-
mance reserve’, communicated also to the Member States, it was
stressed that ‘the Commission will pay particular attention to the
absorption of funds in the case of significant losses under the
N + 2 mechanism and the quality of the mid-term evaluation (1).’
Examples include the following:

— the German transport OP aimed to spend approximately
equal amounts on road and rail. But on absorption grounds
it was proposed to allocate all of the performance reserve to
road construction. When submitting the proposal to the
Commission, the managing authority stated that the perfor-
mance reserve was not employed in line with environmental
or transport policy objectives, but according to progress in
spending;

— SPD Réunion: Some environmental and equal opportunities
measures did not receive any allocation of the performance
reserve. The managing authority underlined that it was in
some cases difficult to reconcile certain Community objec-
tives with the need to spend funds;

— the proposal of Brandenburg, Germany, made mainly in the
light of the analysis of absorption, excluded environmental
investment measures from the allocation. Transport infra-
structure received the highest amount. The evaluators of the
programme also concluded that the Commission’s indicators
were rather based on absorption than on effectiveness.

34.
In addition there were several allocations of the perfor-
mance reserve exclusively to the ERDF, mainly on the grounds of
absorption, for example in Germany (OP Saxony), Ireland (the
performance reserve of all Irish OPs was entirely allocated to one
ERDF financed programme), Italy (OP Sviluppo Imprenditoriale
Locale) and some Objective 2 Austrian, German, French, and
Spanish and UK OPs.

IMPACT OF MID-TERM REVISION

37.
The 2000-2006 reform of the Structural Funds modified
and strengthened the Funds’ objectives. The intention was to pro-
mote growth and employment in a way which took account of
and combined interrelated economic, social and environmental
policy objectives (2). In particular, expenditure from the funds was
to take account of new technologies and innovation, research, the
information society, clean and efficient use of energy (including
renewable energy), better protection of the environment, social
inclusion and life-long learning.

Allocation of the performance reserve and shifts of funds
38.
following the mid-term revision had to take account of the pro-
gramming principles set out in the Commission’s programming
guidelines (3). The Commission however issued its revised pro-
gramming guidelines only in August 2003. This was too late for
most Member States and regions to take the revised guidelines
into account in the MTR. On the other hand the revised guide-
lines involved only a number of qualitative shifts in the initial
guidelines, for instance in the consolidation of the Lisbon strat-
egy, in the fields of transport (emphasis on intermodality, shift
from road to rail and waterways) and energy.

39.
All operational programmes and Community support
frameworks had to be revised as a result of the allocation of the
performance reserve. This involved the adaptation of each pro-
gramme through a similar process as applied during the initial
programming, including consultation of relevant Commission
Directorates General and formal adoption by the Commission.

40.
There was no obligation on Member States to revise
OPs/CSFs for reasons other than allocation of the performance
reserve. Of the 25 OPs and CSFs examined by the Court, revisions
for other purposes were made in 15 cases.

35.
The time-table for allocation of the performance reserve
was tight. The proposals were received in January 2004 and
agreement in principle had to be concluded by early February.

(1) In one case, the Dutch Objective 3 programme, the Commission had
no option other than to allocate the performance reserve despite the
fact that significant decommitments had already taken place because
funds had not been spent.

(2) This corresponded to the requirements of the Union’s concept of sus-
tainable development of the Amsterdam Treaty and anticipated largely
the conclusions of the Lisbon European Council (March 2000: ‘to
become the most competitive and knowledge-based economy in the
world’) to which the Gothenburg European Council (June 2001) added
the environmental strategy of sustainable development.

(3) See Articles 10(3) and 14 of Regulation (EC) No 1260/1999. ‘… in
accordance with this title’. This ‘title’ refers to the programming pro-
visions of Articles 15 to 19.

C 124/10

EN

Official Journal of the European Union

5.6.2007

In many cases, the mid-term revision improved the way
41.
in which OPs took account of policy priorities or changes in EU
policies. In some cases performance indicators were improved
and changes made to management structures. Changes of these
types were for example made in Ireland, Portugal and Italy.

42.
As already noted in respect of the performance reserve,
however, priority was given to activities where money could be
spent, for instance:

— SPD Burgenland, Austria. Main changes: the reduction of
planned spending on research, technology and development
absorption
and
difficulties,

conservation because of

landscape

— SPD Hainaut, Belgium: absorption played an important role,
for instance planned spending on the ‘management of the
environment’ was substantially reduced on the grounds of
low absorption and transferred to aid for redevelopment of
industrial sites,

— OP Brandenburg, Germany: planned spending on environ-
mental measures was reduced substantially, the measure
‘Renewable Energy’ was abandoned,

— OP Transport, Germany: the allocation of the performance
reserve only to road projects substantially changed the ratio
of spending on roads to spending on rail,

— OP Calabria: A large part of the revised programme concerns
‘compatible’ projects. These are public projects, which were
already financed with national funds prior to the current pro-
gramme period. They were not foreseen in the initial pro-
gramme. This approach was mainly applied in order to avoid
decommitments,

— CSF Italy, S&E OP Ireland: The financial plans were substan-
tially changed because private funding initially foreseen was
reduced (CSF Italy by 95 %, S&E OP by 70 %) and partly
replaced by national public funds. The result was to increase
nationally funded spending and thus avoid decommitments.

43.
The Court’s examination found that transfers of funds for
environmental measures were made mainly in favour of aid for
enterprises and transport infrastructure (especially roads). This
was the case, for example, in Germany, Spain, Italy and Belgium,
amounting to around 800 million euros (about 1 % of Structural
Fund allocations for these Member States).

ARRANGEMENTS FOR 2007-2013

45.
Revised arrangements for evaluation, review and revision
of programmes have been agreed for the next Structural Fund
programme period 2007-2013. They differ in a number of
respects from the arrangements which applied in 2000-2006. In
general, where changes have been made, they provide for a greater
degree of flexibility than under the 2000-2006 arrangements and
should assist in ‘fine tuning’ mid-term evaluation and the revision
of spending plans to the particular circumstances of individual
programmes.

— There is still a requirement for evaluation before, during and
after the programme period. Evaluations must, in particular,
take account of the objective of sustainable development and
of the relevant Community legislation concerning environ-
mental impact and strategic environmental assessment (1).

— On the other hand there is no obligation to carry out, for all
programmes, a mid-term review at a specific point in the
programme period.

— The principle of ‘proportional intervention’ applies to evalu-
ation. Financial and administrative resources employed by
the Commission and the Member States in the implementa-
tion of the funds in relation to the selection of indicators,
evaluations, management and control system and reporting
shall be proportional to the total amount of expenditure allo-
cated to an operational programme (2).

— There is no compulsory performance reserve. Member States
may instead choose to establish a national contingency
reserve and/or a national performance reserve (3). (The pro-
cedures for allocating any national performance reserve
resemble those applying to the performance reserve in
2000-2006).

— As for 2000-2006 mid-term revision of programmes is not
compulsory for all programmes. (Revision of the central pro-
gramming guidelines agreed by the Council does not require
Member States to change their operational programmes or
national strategic reference framework). Two cases in which
Member States might decide to carry out a review are how-
ever identified in the Regulation: 1. significant socio-
economic changes and to take greater or different account of
major changes in Community, national or regional priorities,
2. in the light of a significant departure from the goals ini-
tially set or following implementation difficulties.

Overall the Court found that, in respect of a number of
44.
Structural Fund programmes, mid-term revision led to improve-
ments (for example taking account of policy priorities or changes
in EU policies or improving performance indicators). A signifi-
cant impact in practice however was to shift budgets away from
underspending measures.

(1) Council Regulation (EC) No 1083/2006 of 11 July 2006 laying down
general provisions on the European Regional Development Fund, the
European Social Fund and the Cohesion Fund and repealing Regula-
tion (EC) No 1260/1999 (OJ L 210, 31.7.2006, p. 25), Article 47.

(2) Regulation (EC) No 1083/2006, Article 13.
(3) Regulation (EC) No 1083/2006, Articles 50 and 51.

5.6.2007

EN

Official Journal of the European Union

C 124/11

The ‘N + 2’ rule will still apply, but will (for a limited
46.
period) become an ‘N + 3’ rule for Member States whose GDP per
head from 2001 to 2003 was below 85 % of the EU-25 average.

financial perspectives are targets as well as ceilings on spend-
ing (1). Absorption is itself an implicit objective for EU spend-
ing on cohesion.

CONCLUSIONS

47.
Mid-term evaluation provided a check on the progress
made in implementing Structural Fund programmes and con-
firmed that implementation systems were significantly improved
from the previous generation of Structural Funds. On the other
hand it was in many cases too early to assess the effectiveness
and, a fortiori, the impact of programmes.

48.
The performance reserve was used primarily to maximise
spending rather than to concentrate spending on areas which
were shown to be particularly effective. The Commission empha-
sised the importance of absorption (see paragraph 33).

49.
In a number of cases the mid-term revision of Structural
Fund programmes improved the way in which operational pro-
grammes took account of policy priorities or changes in EU poli-
cies (e.g. as reflected in the Lisbon and Gothenborg European
Councils) or improved performance indicators. A significant
effect of the mid-term revision was to give priority to activities
where money could be spent.

50.
Serious efforts were made to carry out the mid-term pro-
cesses as set out in the Structural Funds Regulation. On the other
hand, since the mid-term evaluations were carried out at a rela-
tively early stage in the programme, the mid-term evaluations and
revisions could provide little assurance that Structural Fund
spending would have the effects and impacts desired. The Mem-
ber States and the Commission also made use of the performance
reserve and the mid-term revision to promote spending that could
be carried out within the permitted time-frame.

51.
The Court is of the view that it remains appropriate to
include in large scale expenditure programmes, running for as
long as 7 years, provision for reconsideration of programme
objectives, measures and budgetary allocations during the course
of the programme. But the audit illustrated tensions inherent in
the planning and management of the Structural Funds which need
to be borne in mind when designing and applying arrangements
for mid-term evaluation and revision.

— Expenditure should be efficient, effective and economic. On
the other hand the commitment totals agreed for the Struc-
tural and Cohesion Funds and set out in successive EU

— The N + 2 rule provides a discipline to ensure that spending
is well planned, but creates an incentive for the Member
States and the Commission to favour activities where spend-
ing can be undertaken with relative ease in allocation of any
reserve.

— Mid-term revision poses problems of timing. The later the
revision, the less the chance of making changes to a pro-
gramme; but the earlier any revision, the more difficult it is
to form a robust judgment about the effectiveness and impact
of expenditure. A corollary is that, if evaluation is to play a
significant part in the allocation of the funds, it is important
to carry out a high quality ex ante evaluation.

52.
The particular context of the 2000-2006 programme
period exacerbated some of these problems. Expenditure for the
1994-1999 period was not subject to the N + 2 rule, and was sig-
nificantly delayed (indeed the 1994-1999 programmes are still
not closed). This contributed to the delay in starting programmes
for the 2000-2006 period. Some of the other limitations of mid-
term evaluation are more fundamental: it is intrinsically difficult
to assess the net outcome of infrastructure investments, or of
employment initiatives.

RECOMMENDATIONS

53.
The Court recommends the following steps to improve
the usefulness of medium term evaluation, revision and realloca-
tion during the 2007-2013 and future programme periods:

— ‘Medium term’ evaluation and revision of programmes are
more likely to be successful, the greater the extent to which
expenditure takes place during the programme period, rather
than following some years in arrears. In order to reduce
delays in getting the 2007-2013 programme underway, the
Commission and Member States should ensure that closure
of the 2000-2006 programmes takes place speedily and
efficiently.

— A particular difficulty found by evaluators of the 2000-2006
programmes was that monitoring systems were operating
poorly. The Commission should take steps to ensure that
performance indicators are monitored from the outset in the
new programme period.

(1) According to paragraph 12, third subparagraph of the Interinstitu-
tional Agreement (including the financial perspective) between the
European Parliament and the Council and the Commission of
6 May 1999 (OJ C 172, 18.6.1999, p. 1) the European Parliament and
the Council undertake to respect the allocations of commitment
appropriations provided in the financial perspective for structural
operations.

C 124/12

EN

Official Journal of the European Union

5.6.2007

— Evaluation and reassessment of expenditure should take
place late enough for there to be sufficient experience of a
programme to permit useful appraisal to take place, but early
enough to permit changes in priorities.

— The Mid-Term processes in 2000-2006 constituted a lengthy
and complex procedure. The Commission should consider
ways in which the process can be made more rapid and
responsive.

impacts on cohesion are identifiable, that evaluations are a
source of best practice, and that, where performance reserves
are established, they are used in a way which gives proper
weight to the effective and efficient implementation of
programmes.

— All guidance provided by the Commission should be made

available in good time.

— While the Commission should not be prescriptive in defin-
ing the method to be used by evaluators, it should identify
how programmes can be amended so as to ensure that

— The Commission and the Member States should give special
attention to the programming and management arrange-
ments for expenditure on EU policy priorities.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 8 March 2007.

For the Court of Auditors
Hubert WEBER
President

5.6.2007

EN

Official Journal of the European Union

C 124/13

THE COMMISSION’S REPLIES

V.

SUMMARY

III.

First indent — The aims of the mid-term evaluation with regard
to the assessment of effectiveness, efficiency and expected impacts
were limited to analysis of initial outputs and results. The late or
slow start-up of many programmes made such analysis difficult,
although it was carried out in some cases (see Chapter 5 of the
Commission’s report Growing Evaluation Capacity). Nevertheless,
the other aspects examined in the mid-term evaluation provided
useful input to the decisions on the allocation of the performance
reserve and the adaptation of operational programmes.

Second indent — The performance reserve was used to allocate
resources to programmes on several criteria for which the Com-
mission recommended three groups of indicators. Financial
absorption was an important factor, but the criteria varied
between Member States and were often wider subject to availabil-
ity of robust data (see points 28 and 32).

Third indent — The Commission considers that the mid-term
revision had a significant effect on the general performance of
programmes and the way they took account of policy priorities.
At the same time, the Commission wanted to avoid allocating
additional resources to fields which were already having difficul-
ties in spending the existing resources.

IV.
The Commission agrees that it is appropriate to recon-
sider programme objectives, measures and budgetary allocations
during the programme period. It has built on the experience of
2000-2006 in proposing revised arrangements for 2007-2013.

First indent — It is the task of the Commission to ensure that due
account is taken of the effectiveness and value for money as well
as just absorption. The Commission endeavours to discharge this
responsibility.

First indent — The Commission issued a detailed guidance docu-
ment for the 2000-2006 period closure in August 2006. The clo-
sure requirements are now more familiar than before the 1994-
1999 period closure.

Second indent — The Commission will devote close attention to
the setting up of monitoring systems and the definition of indi-
cators in the new programmes.

Third indent — In the 2007-2013 period the previous general
requirement for mid-term evaluation will be replaced by need-
based ongoing evaluations to assess the implementation of a pro-
gramme and react to changes in its external environment. This
approach can be better tailored to the demands of the users of the
evaluation.

The proportionality introduced by the regulations and the stream-
lining of the programming process will also increase flexibility in
evaluation arrangements.

Fourth indent — The more flexible arrangements will allow this.

Fifth indent — For the 2007-2013 period the Commission pro-
vided guidance documents in 2006.

Sixth indent — Co-financed activities are required to promote
European Union (EU) policy priorities, including the Growth and
Jobs agenda. EU-15 Member States have to meet specified targets
for spending on the latter.

INTRODUCTION

6.
Box 1. — The Commission had initially proposed a per-
formance reserve that could be shifted between Member States,
but the proposal was amended during the negotiations on the
future legislation.

Second indent — The tension the n + 2 rule creates between fast
absorption and quality need not mean that quality is sacrificed.
The Commission endeavours to prevent this.

9.

Third indent — The monitoring and evaluation arrangements for
the 2007-2013 programme period are designed to be responsive
to the practical challenges faced by programmes.

First indent — The Commission considered that progress had
been made in the programming of the Structural Funds to ensure
their implementation in accordance with a clearly laid down
development strategy and the quest for effectiveness had not been
neglected in favour of the consumption of funds.

C 124/14

EN

Official Journal of the European Union

5.6.2007

Second indent — All Member States carried out ex ante evalua-
tions, which led to a greater coherence of strategies and greater
knowledge of the basis of the choices made and significantly
improved quantification of objectives. In those Member States
where the ex ante evaluation examined alternative strategies, the
intention was to use this evaluation as part of their national plan-
ning process.

As noted in the reply to point 23, final indent, the aims
26.
of the mid-term evaluation with regard to the assessment of effec-
tiveness, efficiency and impact were limited.

Third indent — The Working Paper on the mid-term evaluation
pointed out that the estimation of these effects, too, would
depend on data availability.

The Commission agrees that future ex post evaluations
10.
should be more focused and takes full account of data and
resource constraints. It will prepare the exercise carefully.

Fourth indent — See reply to box 4 and to point 23, last indent.

ASSESSMENT OF MID-TERM EVALUATION

20.
Box 4. — The Commission’s report on the Mid-Term
Evaluation — Growing Evaluation Capacity also noted that some
evaluations began an analysis of unit costs which can be built on
in the future.

23.

First indent — The Commission, too, recognised in its report
‘Growing Evaluation Capacity’ (point 4.1.1) that the low level of
spending had limited the potential of the evaluations.

27.
Another important objective of the mid-term evaluations
was to assess the relevance of the strategies, which could be done
in the time-frame set for the mid-term evaluation despite low
expenditure (see working paper). Regarding the limited aims of
the assessment of effectiveness and impact, see point 23.

ALLOCATION OF PERFORMANCE RESERVE

30.
The mid-term evaluation provided information for the
allocation of the performance reserve on matters other than
spending. Some of the indicators recommended by the Commis-
sion for the mid-term evaluation did not depend on the level of
expenditure, but measured different aspects of programme
implementation.

Second indent — The evaluations’ findings with respect to the
Lisbon-Gothenburg agenda were reviewed in point 5.2 of the
Commission’s report ‘Growing Evaluation Capacity’.

It should be noted that some infrastructure projects do contrib-
ute to the Lisbon and Gothenburg objectives.

Third indent — The Commission’s report ‘Growing Evaluation
Capacity’ did not deal with methodological problems of assessing
impacts, as the low level of spending and the resulting relatively
small outputs were a bigger limitation. In the Working Paper on
the mid-term evaluation, however, it had made clear that the aims
of the mid-term evaluation with regard to the assessment of effec-
tiveness, efficiency and expected impacts were limited to analysis
of initial outputs and results. This also applies to the assessment
of efficiency in terms of employment effects.

25.

Third indent — The balance between addressing longer-term
needs, such as for R&D and innovation, and financing faster-
spending demand-driven measures is hard to achieve.

31.
The Commission’s working document recommended the
use of output indicators, as data is available with only a small
time-lag, rather than results indicators where data is available only
after a longer period. Impact indicators are not available until
years after completion of the intervention.

33.
The reason for the Commission’s stating that it would
pay particular attention to absorption was that it would not be
sensible to allocate additional resources to a programme or pri-
ority which was not absorbing the initial allocation.

The Commission agrees that fast spending by itself is not an indi-
cation of success of a programme. But efficient spending is a nec-
essary condition if a programme is to achieve the objectives set.
Although financial absorption was an important factor in the
allocation of the performance reserve, the criteria varied between
Member States and were often wider where data were sufficiently
robust (see points 28 and 32).

Fourth indent — The problem of lack of comparability of differ-
ent modelling results and evaluations is discussed in the Commis-
sion’s report Growing Evaluation Capacity, point 5.1.2.1 —
(see also point 22 above).

First indent — Although railway investment was environmentally
desirable, the managing authority had to take account of the fact
that it had used only 14,5 % of available funds.

5.6.2007

EN

Official Journal of the European Union

C 124/15

Second indent — The financial allocation to the environment was
one of the most contentious parts of the negotiations. The out-
come was that cuts for some environmental projects (such as a
waste incinerator whose construction had been postponed) were
partly compensated for by increases in the allocation for other
environmentally-friendly projects. While there was no new allo-
cation of funding to the equal opportunities measure, the discus-
sion led to intensified activity in this area.

Sixth indent — The removal of private funding was partly because
private funding had been included in the financial tables by mis-
take (it had not been intended to include it in the co-financing
base) and partly because of its volatility which meant that finan-
cial plans had to be regularly revised, whereas public funding was
more stable. The European Regional Development Fund (ERDF)
allocations to the measures concerned remained unchanged and
private funding continued to be mobilised to implement the mea-
sures, although no longer part of the co-financing base.

Third indent — The Commission was not happy with the high
allocation of funding by Brandenburg to transport infrastructure,
especially road infrastructure, and has regularly raised the matter
with the Land, which defends its decision on n + 2 grounds. The
Commission intends to take up this point in the negotiations for
the next programme period.

34.
The allocation of the performance reserve was intended
to be independent of the particular Structural Fund from which
the funding would be paid.

It responded to longer-term development priorities as well as high
demand for funding from particular sectors. In some of the cases
referred to by the Court, programmes were eliminated as candi-
dates for the reserve because of an imminent risk of decommit-
ment of funding under the n + 2 rule.

43.

The Commission refers to its reply at point 33.

ARRANGEMENTS FOR 2007-2013

45.
In the 2007-2013 period the previous general require-
ment for mid-term evaluation will be replaced by need-based
ongoing evaluations to assess the implementation of a pro-
gramme and react to changes in its external environment. This
approach emphasises the need for stronger links between moni-
toring, evaluation and decision-making, and can be better tailored
to the demands of the users of the evaluation. An evaluation will
be triggered by actual or potential difficulties revealed by the
monitoring system but can be undertaken to ensure regular
review of strategic or operational aspects which cannot be solely
analysed on the basis of monitoring data.

36.
As noted in the reply to point 34, the allocation was by
no means always based on absorption, although this was one of
indicators for allocation of the reserve, but also took account of
effectiveness in terms of outputs and results, subject to availabil-
ity of robust data, and of longer-term needs.

First indent — The Commission provided guidance on this new
approach in 2006 (Working Document No 5 ‘Indicative Guide-
lines on Evaluation Methods: Evaluation during the Programme
period — 2007-2013’).

IMPACT OF MID-TERM REVISION

CONCLUSIONS

42.
On the question of the role of absorption in the mid-term
revision, the Commission refers to its replies to point 34 in rela-
tion to the performance reserve.

Second indent — As well as financial performance, the effective-
ness of expenditure, in particular for job creation, was a criterion.
The risk of environmental projects overshooting the time limit for
expenditure was a factor in the reduction of funding for this type
of investment.

47.
The aims of the mid-term evaluation with regard to the
assessment of effectiveness, efficiency and expected impacts were
limited to analysis of initial outputs and results. The late or slow
start-up of many programmes made such analysis difficult,
although it was carried out in some cases (see Chapter 5 of the
Commission’s report Growing Evaluation Capacity). Nevertheless,
the other aspects examined in the mid-term evaluation provided
useful input to the decisions on the allocation of the performance
reserve and the adaptation of operational programmes.

Fifth indent — The Commission monitors the eligibility of such
projects for EU funding through audit work and the relevant
Monitoring Committee. For example, audits of projects funded by
the European Social Fund (ESF) under the Calabria programme
are to take place in March 2007.

The performance reserve was used to allocate resources
48.
to programmes on several criteria for which the Commisssion
recommended three groups of indicators. Financial absorption
was an important factor, but the criteria varied between Member
States and were often wider subject to availability of robust data
(see points 28 and 32).

C 124/16

EN

Official Journal of the European Union

5.6.2007

The Commission considers that the mid-term revision
49.
had a significant effect on the general performance of pro-
grammes and the way they took account of policy priorities. At
the same time, the Commission wanted to avoid allocating addi-
tional resources to fields which were already having difficulties in
spending the existing resources.

50.
Given the low level of spending, the mid-term evaluation
could only provide an indication of an appropriate redirection of
funds

51.
The Commission agrees that it is appropriate to recon-
sider programme objectives, measures and budgetary allocations
during the programme period 2000-2006. It has built on the
experience of 2000-2006 in proposing revised arrangements for
2007-2013.

First indent — It is the task of the Commission to ensure that due
account is taken of effectiveness and value for money as well as
just absorption. The Commission endeavours to discharge this
responsibility.

Second indent — The tension the n + 2 rule creates between fast
absorption and quality need not mean that quality is sacrificed.
The Commission endeavours to prevent this.

Third indent — The monitoring and evaluation arrangements for
the 2007-2013 programme period are designed to be responsive
to the practical challenges faced by programmes.

52.
The Commission agrees that a smooth start of imple-
mentation will increase the usefulness of evaluations carried out
mid-term. The role of the mid-term evaluation in the 2000-2006
period was to see how well programmes were performing and on
the basis of initial results make any changes that were appropri-
ate, rather than continuing on a course which would limit posi-
tive outcomes. Apart from commitments remaining open because
of legal proceeding the 1994-1999 programmes have largely
been closed and the outstanding commitments as at 31 Decem-
ber 2006 were down to under 0,6 % (917 million euros) of the
original allocations.

RECOMMENDATIONS

53.

First indent — The Commission issued a detailed guidance docu-
ment for the 2000-2006 period closure in August 2006. The clo-
sure requirements are now more familiar than before the
1994-1999 period closure.

Second indent — The Commission will devote close attention to
the setting-up of monitoring systems and the definition of indi-
cators in the 2007-2013 programmes.

Third indent — The new evaluation arrangements increase flex-
ibility. In the 2007-2013 period the previous general requirement
for mid-term evaluation will be replaced by need-based ongoing
evaluations to assess the implementation of a programme and
react to changes in its external environment. This approach can
be better tailored to the demands of the users of the evaluation.

The proportionality introduced by the regulations and the stream-
lining of the programming process will also increase flexibility in
evaluation arrangements.

Fourth indent — The arrangements for the 2007-2013 period are
more flexible.

Fifth indent — For the 2007-2013 period, the Structural Funds
Regulation provides a flexible way to link evaluation and repro-
gramming. Evaluations are a source of valuable recommendations
and can identify good and bad practices.

Sixth indent — The main guidance for the 2000-2006 period was
provided in good time. For the 2007-2013 period the Commis-
sion provided guidance documents in 2006.

Seventh indent — Co-financed activities are required to promote
EU policy priorities,
including the Growth and Jobs agenda.
EU-15 Member States have to meet specified targets for spending
on the latter.