Special report No 1/2008 concerning the procedures for the preliminary examination and evaluation of major investment projects for the 1994-1999 and 2000-2006 programming periods together with the Commission's replies
Date: 2008-02-14
Institutions: European Court of Auditors
Subjects: Cohesion Fund, EU Member State, EU financing, European Regional Development Fund, Structural Funds, audit, investment project, multiannual financial framework

1.4.2008

EN

Official Journal of the European Union

C 81/1

IV

(Notices)

NOTICES FROM EUROPEAN UNION INSTITUTIONS AND BODIES

COURT OF AUDITORS

SPECIAL REPORT No 1/2008

concerning the procedures for the preliminary examination and
evaluation of major investment projects for the 1994-1999 and 2000-2006
programming periods together with the Commission’s replies

(presented pursuant to Article 248(4), second subparagraph, EC)

(2008/C 81/01)

CONTENTS

Paragraph

Page

GLOSSARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Definitions and approval framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The volume of funds involved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

THE SCOPE OF THE AUDIT AND OF THIS REPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DOES THE COMMISSION PREPARE COMMUNITY FINANCING OF MAJOR PROJECTS
ADEQUATELY? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Preliminary examination framework and audit criteria . . . . . . . . . . . . . . . . . . . . . . . . . . .

Project examination: improvements in the methodological framework but procedures to
be strengthened . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The cost-benefit analysis: varying quality of applications . . . . . . . . . . . . . . . . . . . . . . . . .

Setting Community financial assistance: criteria inconsistently interpreted and applied . . .

DO EX POST EVALUATIONS OF MAJOR PROJECTS GENERATE RESULTS THAT ARE HELP-
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FUL FOR SUBSEQUENT DECISION-MAKING?

Evaluation framework and audit criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Several wide-ranging evaluations but very few individual evaluations of major projects . . .

I-VII

1-8

1-7

8

9-13

14-34

14-16

17-20

21-25

26-34

35-55

35-39

40-41

3

5

5

5

7

7

7

7

8

9

10

11

11

12

C 81/2

EN

Official Journal of the European Union

1.4.2008

Paragraph

Page

Factors that impeded the ex post evaluation of major projects . . . . . . . . . . . . . . . . . . . . .

Incomplete aid applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Inadequate monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The 2000-2006 period: evaluations at risk of being compromised . . . . . . . . . . . . . . . . .

THE NEW REGULATIONS FOR THE 2007-2013 PERIOD . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONCLUSIONS AND RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Recommendations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

42-49

43-46

47-49

50-55

56-58

59-65

59-62

63-65

Annex I Samples I and II by Member State concerned — List of projects examined . . . . . . . . . . . . . . . . . . .

Annex II 2000-2006 major projects evaluated — Discount rate and time horizons . . . . . . . . . . . . . . . . . . .

The Commission’s replies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

13

13

14

14

14

14

15

16

18

20

1.4.2008

EN

Official Journal of the European Union

C 81/3

GLOSSARY

Cohesion Fund

Cost-benefit analysis

Instrument designed to promote economic and social cohesion by financ-
ing large projects in the fields of the environment and transport in Mem-
ber States with a per capita GNP of less than 90 % of the Community
average. The Cohesion Fund was originally implemented in Spain, Greece,
Ireland and Portugal. Since 1 January 2004 Ireland has no longer been eli-
gible. The new Member States have profited from Cohesion Fund support
since their accession to the Community.

A technique for comparing all the costs and all the benefits of an inter-
vention to determine whether the benefits outweigh the costs, and if so,
by what proportion.

Deadweight effect

An effect that becomes apparent and would have emerged even if a pub-
lic intervention had not taken place.

Effectiveness

Measures the relationship between the results obtained and the objectives
set.

Efficiency

Measures the relationship between the resources employed and the results
obtained.

ERDF (European Regional
Development Fund)

Financial instrument designed to promote economic and social cohesion
between the countries of the EU by correcting the main regional imbal-
ances and by reconversion of regions whose development is lagging
behind. ERDF interventions are mainly implemented through operational
programmes encompassing a large number of projects.

Impact

Socioeconomic consequences that may be attributed to an intervention.

IRR (internal rate of return) Rate at which an investment becomes profitable.

— Financial IRR: a rate of return calculated exclusively from the

amounts of financing relating directly to the project.

— Economic IRR: a rate of return calculated taking into account exter-

nal economic factors.

NPV (net present value)

Value obtained by indexing at the present time the value of all cash bal-
ances which will be generated by a project during its life. This method
enables various projects to be compared.

Operational programmes

A document approved by the Commission, it takes the form of a coher-
ent set of priorities comprising multiannual measures. The priorities may
be implemented through recourse to one or more Structural Funds, or,
alternatively, to other financial instruments and the EIB.

Programming period

The multiannual framework within which Structural Fund expenditure is
planned and implemented.

C 81/4

EN

Official Journal of the European Union

1.4.2008

Risk analysis

Study to determine the probability that the value of the underlying vari-
ables originally calculated for a project will actually be confirmed.

Sensitivity analysis

Study of the stability of the expected result of a project when exposed to
changes in the underlying variables.

Structural Funds

Community funds financing actions with the aim of reducing disparities
in development between the regions of the EU. ERDF is the main finan-
cial instrument for these actions.

1.4.2008

EN

Official Journal of the European Union

C 81/5

SUMMARY

I.
The regulations define ‘major’ infrastructure and productive investment projects essentially in terms of
the volume of finance involved. In view of the economic significance and potential impact of these projects,
they are subject to special preliminary examination procedures prior to approval. For the same reasons, rel-
evant information must be collected so that the effects actually achieved can be assessed in comparison to those
which had been expected (see paragraphs 1-8).

II.
The Court’s audit centred on ERDF major projects and, by analogy, those under the Cohesion Fund for
the 1994-1999 and 2000-2006 programming periods. This report is constructed around two main themes:
the process of preparation for approval, by the Commission, of major projects and the process of evaluation
after their implementation (see paragraphs 9-13).

The Court found several weaknesses in the preliminary examination procedure in the two periods in
III.
question. Of particular concern were the procedures for project appraisal and for setting the rate of Commu-
nity co-financing. The quality of the cost-benefit analyses presented by project promoters in support of appli-
cations for assistance was extremely variable, and the same was true of the use to which the Commission put
those analyses. These findings lead to doubts about the real value added of the special procedure for approv-
ing major projects (see paragraphs 14-34).

IV.
The Court recommends that the Commission should apply more efficient preliminary examination
procedures based on the tools it has itself developed, should ensure that the cost-benefit analysis is rigorously
applied, particularly in respect of the data provided, and should continue its efforts to take into account those
factors which allow the rate of Community participation to be adjusted (see paragraph 63).

V.
As regards the ex post evaluation of major projects from the 1994-1999 period (the only such evalu-
ation that the Commission had been able to carry out at the time of the audit), it proved to be severely handi-
capped by the lack of pertinent and accurate information on anticipated effects and monitoring. Given these
circumstances, the evaluations did not produce any information that was useful from the point of view of
decision-making for the following period, with the exception of one case concerning projects cofinanced by
the Cohesion Fund. The evaluation of ERDF major projects from the 2000-2006 period, which has yet to be
carried out, will be able to rely on stricter rules, especially in the area of monitoring. However, not much has
yet been done to apply those rules in practice (see paragraphs 35-55).

VI.
The Structural Funds regulations for the 2007-2013 period do not require the Commission to carry
out individual ex post evaluations of major projects. However, the Court recommends that, when completed,
the major projects should be evaluated individually, in relation to their specific objectives, by the national man-
agement authorities and the beneficiaries, for whom the information obtained from this exercise would be use-
ful. To this end, the Commission must ensure that the applications for assistance contain the information
required in respect of indicators and the initial situation, and that this information is rigorously monitored. As
regards the information provided by the evaluation, which must be forwarded to it, the Commission must anal-
yse its own requirements so as to ensure that it has relevant, reliable data available on the effects produced by
the major projects (see paragraph 64).

VII.
give thought to the purpose of the special arrangements for major projects (see paragraph 65).

In the longer term, the Court invites the Commission, the European Parliament and the Council to

INTRODUCTION

Definitions and approval framework

For the 1994-1999 programming period, ‘major projects’
2.
were ‘those the total cost of which taken into account in deter-
mining the amount of Community assistance is, as a general rule,
greater than ECU 25 million for infrastructure investments or
greater than ECU 15 million for productive investments’ (2).

1.
The concept of ‘major projects’ is defined, in essence, in
the Community regulations on the ERDF (1), and depends on the
volume of finance involved in the projects.

(1) The concept already existed in Council Regulation (EEC) No 4253/88
of 19 December 1988 (OJ L 374, 31.12.1988, p. 1). It also applies in
the rules for the new period (2007-2013) (Articles 39-41 of Council
Regulation (EC) No 1083/2006 of 11 July 2006 (OJ L 210, 31.7.2006,
p. 25)).

For the 2000-2006 programming period, the definition of
3.
‘major projects’ covers projects ‘whose total cost taken into

(2) Article 16(2) of Council Regulation (EEC) No 4253/88 as amended by
Regulation (EEC) No 2082/93 of 20 July 1993 (OJ L 193, 31.7.1993,
p. 20).

C 81/6

EN

Official Journal of the European Union

1.4.2008

account in determining the contribution of the Funds exceeds
50 million euro’ (1). In addition, these projects must comprise an
economically indivisible series of works fulfilling a precise tech-
nical function and which have clearly identified aims. As in the
previous period, these may be investments in infrastructure or
productive investments (2).

As regards the approval framework, ERDF major projects
5.
for the 1994-1999 period could be submitted individually or
within the framework of an operational programme. In any case,
the Commission was informed about major projects, with further
details to be supplied (4), but no formal approval decision was
required.

4.
The Cohesion Fund regulations, on the other hand, con-
tain no definition of ‘major projects’ for either one of the periods
under consideration (3). However, since, in the cohesion coun-
tries, similar transport infrastructure or environment projects
may be financed by either the Cohesion Fund or the ERDF, the
audit also examined, by analogy, a number of Cohesion Fund
projects that were selected on the basis of total cost, applying the
same criteria as for the ERDF major projects.

6.
All ERDF major projects for 2000-2006 were financed
within a wider programme of assistance (5). However, the regu-
lations required the Commission to approve them individually,
unlike other projects, which were approved only at Member State
level (6).

7.
Under the Cohesion Fund, in both periods, only individual
projects could be financed, each project forming the subject of a
specific decision.

Table 1

Definition of ‘major projects’

FUND

1994-1999 period

2000-2006 period

EDRF ‘major projects’

(infrastructure and productive invest-
ment projects)

Total cost greater than:

Total cost greater than:

— ECU 25 million for infrastruc-

50 million euro for all types of project

tures

— ECU 15 million for productive

investments

In addition:

indivisibility of work from economical
and technical points of view and clearly
identified objectives

Cohesion Fund ‘major projects’

(transport infrastructure and environ-
ment projects)

Individual project or project included
in larger intervention

Project always part of larger interven-
tion

No definition in the regulations

No definition in the regulations

For the purposes of the audit, total cost
greater than ECU 25 million

For the purposes of the audit, total cost
greater than 50 million euro

Individual project

Individual project

(1) Article 25 of Council Regulation (EC) No 1260/1999 (OJ L 161,

26.6.1999, p. 1).

(2) Article 26 of Regulation (EC) No 1260/1999.
(3) The Cohesion Fund exists to finance larger projects in the fields of
transport and environment. Article 10(3) of Council Regulation (EC)
No 1164/94 of 16 May 1994 (OJ L 130, 25.5.1994, p. 1) specified,
for the 1994-1999 programming period, that ‘the total cost of projects
or groups of projects may in principle not be less than ECU 10 mil-
lion’. For the following programming period, Council Regulation (EC)
No 1264/1999 of 21 June 1999 (OJ L 161, 26.6.1999, p. 57) replaces
‘ECU’ with ‘euro’.

(4) Article 5 of Regulation (EEC) No 4254/88 as amended by Council
Regulation (EEC) No 2083/93 of 20 July 1993 (OJ L 193, 31.7.1993,
p. 34).

(5) Mainly Operational Programmes, Single Programming Documents

and Community Initiative Programmes.

(6) Article 26(3) of Regulation (EC) No 1260/1999.

1.4.2008

EN

Official Journal of the European Union

C 81/7

The volume of funds involved

8.
Table 2 shows the total cost for the two programming
periods of the major projects which received assistance from the
EDRF and the Cohesion Fund:

Table 2

Total cost of the ‘major projects’ which benefited from
Community intervention

1994-1999 period

2000-2006 period

Turning to the subject of ex post evaluation, only those
13.
evaluations concerning the projects of the 1994-1999 period
could be taken into account, since, at the time of the Court’s
audit, implementation of the projects of the 2000-2006 period
had not yet been completed and the ex post evaluations were still
to be done (2). Nevertheless, the report also examines whether
improvements can be hoped for in respect of the 2000-2006
period. The observations that were drawn up on conclusion of
the audit may have an impact on the evaluations for the 2000-
2006 period, which are yet to be carried out, but chiefly on those
for the 2007-2013 period which has just begun.

ECU
1 000 mil-
lion

Number of
projects

1 000 mil-
lion euro

Number of
projects

DOES THE COMMISSION PREPARE COMMUNITY FINANCING
OF MAJOR PROJECTS ADEQUATELY?

ERDF MPs

Cohesion Fund
MPs

Total

36

12

48

676

147

823

48

23

71

279

154

437

THE SCOPE OF THE AUDIT AND OF THIS REPORT

Given the impact that single projects of a financial volume
9.
exceeding a certain amount may have on regional development,
major investment projects are bound by preliminary examination
procedures that are more than usually complex. As of the 2000-
2006 period, at least, they are also subject to a separate approval
decision. For the same reasons, a retrospective appraisal of the
actual effects generated by the allocated resources is required in
the interest of obtaining guidance for project design and selection
in future programming periods.

The Court therefore sought replies to the following

10.
questions:

— Is Community financing of major projects adequately pre-

pared by the Commission?

— Does the ex post evaluation of major projects generate results

that are useful for future decision-making?

11.
The administrative checks and checks carried out in the
Member States covered a sample of 20 closed major projects
from the 1994-1999 period, and 30 major projects from the
2000-2006 period. In addition, a supplementary sample of 36
major projects from 2000-2006 was taken for administrative
checks at the Commission (see Annex I).

12.
With regard to the system that was set up for the prelimi-
nary examination of major projects, the audit focused on the
Commission’s procedures for the two periods, 1994-1999
and 2000-2006. The progressive improvements made are indi-
cated. The recommendations in this part will probably have an
impact on implementation in the 2007-2013 period, for which
the concept of ‘major projects’ has been retained for all the Struc-
tural Funds (1).

(1) See in particular Articles 39-41 of Regulation (EC) No 1083/2006.

Preliminary examination framework and audit criteria

14.
It is the responsibility of the Member States to present
major projects for Community co-financing, and this is carried
out on the basis of their own preliminary examination procedures
and rules, taking into account the stated priorities of the relevant
ERDF Operational Programmes and of the Cohesion Fund.

15.
The Commission is not expected to make a comparative
assessment of the different Member States’ projects, as the vol-
umes of finance earmarked for each State are fixed in advance
during the preparations for each programming period. Neverthe-
less, the Commission must examine the quality of the projects to
be financed, i.e. whether the projects selected have the potential
to achieve results with regard to regional development, and
whether financial participation by the Community is required to
carry them out and would deliver real added value.

16.
The Court scrutinised the Commission’s preliminary
examination phase, on the basis that the following factors were
necessary for this phase to lead to the adoption of worthwhile
projects:

(a) a common framework for the examination of applications
for assistance to ensure that the project has been correctly
prepared and contains the information which can form the
basis for a decision on whether it merits assistance;

(b) a decision process based on well-tried methods, understood
and applied in a similar way by the Member States and the
Commission;

(c) determination of the rate of Community financial assistance
on the basis of rules which ensure the public intervention is
put to most efficient use.

(2) Eight projects adopted between 2000 and 2002 were nevertheless
examined in the context of an ex post evaluation carried out on behalf
of the Commission (see paragraph 41).

C 81/8

EN

Official Journal of the European Union

1.4.2008

Project examination:
framework but procedures to be strengthened

improvements in the methodological

Box 1

The Court’s audit demonstrated that, with time, the man-
17.
agement tools put in place by the Commission have been refined.
Although there are differences between the application forms for
assistance for ERDF and Cohesion Fund major projects for the
2000-2006 period, they are more comprehensive and suitable
than the forms used in the preceding period. Likewise, the Guide
to the cost-benefit analysis (1) made available to the Member
States by the Commission has been improved both in terms of its
presentation of the theoretical
framework and its practical
application (2).

However, the examination of the applications varied in

18.
quality:

— as several managing departments are responsible for the pre-
liminary examination (the Commission’s geographical units
assess the projects as a whole, while the Evaluation Unit
examines the cost-benefit analysis), for a long time there was
no common approach, for example in the form of a check-
list. Apart from the vade mecums and procedural handbooks
for Cohesion Fund and ERDF projects in general, it was not
until 2003 that appraisal summary tables and checklists for
major projects in particular were laid down,

— interdepartmental consultation sometimes leads to conflict-
ing views concerning, above all, the appropriateness and
level of assistance. The geographical units responsible do not
always heed these unless an explanation is put on file. Some-
times, too, requests for additional information from other
services are not followed up (see Box 1),

— there is no clear rule, in the cohesion countries, for assigning
major transport and environment projects to the ERDF or
the Cohesion Fund, apart from the fact that the Cohesion
Fund must prioritise financing for the Trans-European Trans-
port Networks (TEN-T). As the Cohesion Fund applies higher
ceilings to project finance, this has led to cases where similar
projects are allocated different financial contribution rates.
What is more, the assignment of a project to the ERDF or the
Cohesion Fund also has different implications for monitor-
ing and ex post evaluation (see Box 2).

(1) See the Guide to cost-benefit analysis of ERDF major projects in the
context of Community regional policy, which was finalised in 1994.
A revised Guide published in 1997 also covered Cohesion Fund major
projects. For the following period, the former guidance instrument
was replaced in 2002 by the Guide to cost-benefit analysis of invest-
ment projects under the Structural Funds (ERDF), Cohesion Fund and
ISPA. These documents are available on the Commission’s website at:
http://ec.europa.eu/regional_policy/sources/docoffic

(2) In particular, the Guide published in 2002 specifies the financial and
economic rates of return that should be used and the ‘time horizons’
that should be applied in each investment field in order to assess the
return on projects. However, it fails to provide any details or forecast-
ing techniques concerning impacts on the environment or job cre-
ation.

With regard to productive projects, one Commission department had
pointed out that the investments were likely to prove extremely prof-
itable and therefore did not require assistance. For one transport infra-
the
structure project, another department had proposed that
requested 60 % contribution rate be reduced because the project was
located in the wealthiest region of the Member State and the authori-
ties were applying lower rates in regions that were less well-off. A rate
cut was also sought in the case of another project where the invest-
ment return was extremely good. In the end, the reduction in the
requested contribution rates was not made for any of these projects.
In the case of the construction of an underground railway, it was con-
verted into an environmental project under the Cohesion Fund when
DG TREN refused to view it as transport infrastructure, despite res-
ervations expressed by DG ENV on the grounds that only 1-3 % of the
benefits to be derived from the project would be environmental in
nature.

Box 2

In Greece, assignment to one or the other fund, following consulta-
tion with the Commission, depends on the projects’ level of prepara-
tion and planning and the availability of resources. This has led to the
somewhat illogical situation where one extension project for a major
transport project was financed by the Cohesion Fund at the rate of
66,3 % and another by the ERDF under an operational programme at
50 %.

19.
There were substantial delays in the examination process.
The two-month time limit set for Commission approval (3) fol-
lowing the receipt of application forms is usually exceeded. The
average time needed for the ERDF and Cohesion Fund projects
that were audited on the spot was 187 days, with the approval
period ranging from 46 to 1 001 days. These delays are attribut-
able to the fact that the Commission is obliged to seek further
details on the vast majority of major projects, largely owing to the
poor quality — or absence — of environmental impact studies,
which are compulsory in many cases, and to specific aspects of
the financial and economic project analyses (4). Furthermore, the
Member States are often very slow in replying. The shortcomings
which affected project management for a considerable time may,
in part, explain these delays (see paragraph 17).

The Commission’s examination for the approval of

20.
projects proved to be too lenient:

— projects were approved before they were entirely ready, as
indicated by the fact that budgets were overrun and imple-
mentation deadlines were missed. Even where it was
informed of this problem, the Commission approved them
nonetheless,

(3) Article 26(3) of Regulation (EC) No 1260/1999; the time limit is three

months if the EIB has to be consulted.

(4) In this connection, see also paragraphs 22-24.

1.4.2008

EN

Official Journal of the European Union

C 81/9

— some projects, on the other hand, were almost completed by
the date they were submitted. For some productive projects,
the projects were close to physical completion by the date on
which the application file was sent to the Commission
(although this was ahead of the closure of the accounts). In
such circumstances the project could no longer be modified
and the Commission could only either approve or refuse the
application,

— in the case of productive investments, application of the
principles relating to justification of public intervention was
only rarely verified. Consideration of the criteria relating to
compensating for market weaknesses and avoiding the
co-financing of deadweight effects did not play a major role
in the financing decisions,

— some major projects did not satisfy the criterion whereby
major projects are supposed to comprise an economically
indivisible series of works fulfilling a precise technical func-
tion and with clearly identified aims (see paragraph 3). A
number of ERDF projects that were adopted as major projects
were in fact made up by combining a number of separate
individual projects. This is important in view of the fact that
submission of an application for a major project grants the
de facto right to benefit from extended application of the
‘n + 2 rule’ while the project is being examined, which may
take a considerable amount of time (1).

The cost-benefit analysis: varying quality of applications

21.
The cost-benefit analysis required by the legislation, in
order to substantiate applications for assistance, is the main tool
at the Commission’s disposal for its decision to grant Commu-
nity financing or not. Its satisfactory application is essential to
evaluate the quality of the projects submitted. Its objective is to
compare the resources deployed with the socioeconomic benefits
which may be obtained in the medium term from the proposed
action (2). The Commission may ask for the analysis submitted in
support of the application to be revised or expanded, or may itself
carry out an appraisal of the project, seeking the views of inde-
pendent experts, where necessary, and the assistance of the EIB.

22.
Despite publication by the Commission of a Guide to the
cost-benefit analysis of major projects (see paragraph 17), great
disparity was noted in the quality of the financial and economic
analyses presented by the promoters.

The Commission takes an equivocal attitude to this. In
23.
some cases it tolerates numerous weaknesses and approves
projects without requiring further details, while in other cases it
suspends approval until it has received essential items of finan-
cial information.

The Court’s audit has brought to light the following
24.
shortcomings and inconsistencies which impair the appraisal of
the projects:

— inadequate information to assess the soundness of the analy-
sis carried out; for a number of projects the financial and eco-
nomic analyses are nothing more than a statement of the
investment amount, costs and revenue, with no breakdown
of these figures; for other projects only the results of the
analyses are presented,

— missing components: in some cases the economic analysis
and the cost-benefit report are missing; the contents of the
risk and sensitivity analyses are often incomplete; rates and
time horizons are not specified (see Annex II); direct environ-
mental impacts are simply described or referred to without
being assigned a financial value, or only a partial value. In the
case of a number of major transport infrastructure projects
that form part of wider projects, no overview of the entire
project is provided although this is necessary for evaluation
of the part presented for cofinancing (see Box 3). In the case
of major industrial projects the results and impact indicators
are rarely quantified, with the exception of employment
(direct
job creation, maintenance of
employment),

indirect

and

— conflicting or erroneous practices: different financial and
economic discount rates have been applied in the analyses,
even in one and the same Member State, either to the public
infrastructure projects or the productive investments (see
Annex II). It is not always clear whether rate-of-return calcu-
lations are inclusive or exclusive of assistance; depreciation is
taken into account when calculating the financial IRR and
NPV, which is contrary to the generally acknowledged prin-
ciples of financial analysis. Lastly, a relatively high propor-
tion of the productive investments examined gave identical
financial and economic IRR and NPV data.

(1) This rule provides for the automatic decommitment of commitments
that have not been settled within two years of the date on which the
corresponding appropriations were entered in the Community bud-
get. It constitutes an incentive to rapid implementation of the projects.
(2) Article 26(1) of Regulation (EEC) No 4253/88, as modified by Regu-
lation (EEC) No 2082/93 for ERDF projects from the 1994-
1999 period, Articles 25 and 26 of Regulation (EC) No 1260/1999 for
ERDF projects from 2000-2006. Article 10 of Regulation (EC)
No 1164/94 and Regulation (EC) No 1264/1999 for Cohesion Fund
projects from 1994-1999 and 2000-2006 respectively.

Box 3

In the case of the construction of a cross-border tunnel, the applica-
tion, as a major project, was submitted by one of the two Member
States involved. An overview of the entire tunnel project would have
been useful for an evaluation of the applicant Member State’s section.
The same applies to the cross-border motorways. For the national
motorways and railways a separate economic analysis was made in
many cases for each phase, with each phase constituting a major
project, but there was no overall economic analysis.

C 81/10

EN

Official Journal of the European Union

1.4.2008

Generally speaking, the quality of the analyses is weaker in the
case of major productive investment projects.

25.
The poor quality of the analyses accompanying the aid
applications shows that, in the examination phase, the Commis-
sion did not always carry out an effective examination of the cost-
infrastructure
benefit
investments or productive investments, under these circum-
stances conclusive rejection of a project was a very rare occur-
rence, as rare as requests for projects to be revised after
examination by the Commission.

the projects. Whether

analyses of

Setting Community financial assistance: criteria inconsistently
interpreted and applied

26.
The purpose of the administrative examination under-
taken by the Commission is to ‘confirm or amend the rate of
Community assistance’ (1). Introduced to ensure that awards are
the financed project
effective,
receives the resources needed for its achievement without the
beneficiary acquiring any unjustified advantage. This is particu-
larly important where the beneficiaries are private businesses, as
Community funding may distort market conditions.

this instrument ensures that

Community assistance is subject to certain regulatory
27.
ceilings. Lower regulatory ceilings are provided for investments in
infrastructure that generate substantial net revenue and for invest-
ments in firms (2). In addition, Community rules and State aid
ceilings must be complied with.

Within the limits of these ceilings the only precise factor
28.
used by the Commission to determine the rate of Community
assistance has been the financing gap ratio, which is only applied
to revenue-generating infrastructure projects. It consists of calcu-
lating that part of the discounted initial investment cost of the
project which is not covered by the net present value of operat-
ing revenues (including the residual value) (3). This calculation
allows the maximum level of public intervention to be fixed. In
2003 the Commission introduced an amended method of

(1) See Article 26(3) of Regulation (EC) No 1260/1999.
(2) See Article 13 of Regulation (EEC) No 2052/88 as amended by Coun-
cil Regulation (EEC) No 2081/93 of 20 July 1993 (OJ L 193,
31.7.1993, p. 5) and Article 29 of Regulation (EC) No 1260/1999 for
the ERDF; Article 17 of Regulation (EEC) No 4253/88 as modified by
Regulation (EEC) No 2082/93 and Article 7 of Regulation (EC)
No 1264/1999 for the Cohesion Fund respectively.

(3) The ‘financing gap’ ratio formula is as follows: r = (C-R)/C, where C
represents the total cost of the discounted initial investment and R the
net present value of operating revenues.

calculation for major sanitation and solid waste management
projects (4), which takes account of revenue generation while
seeking to avoid penalising beneficiaries who rigorously observe
the ‘polluter pays’ principle (5).

29.
In practice, in certain cases examined by the Court, the
application of the financing gap ratio principle has either resulted
in fixing too high a rate of Community contribution or there is
doubt as to the correct application of this principle.

30.
Apart from the financing gap method, no guidelines indi-
cate any other factors to be taken into account for the purpose of
adjusting Community assistance rates, which helps to explain cer-
tain differences in the views expressed during the interdepartmen-
tal consultation process (see paragraph 18). In the absence of
previously established criteria, the rate of assistance is adjusted in
a random manner and its consistent application cannot be
verified.

31.
The Court’s audit has shown that as regards public infra-
structure investment projects which do not generate revenue,
socioeconomic aspects (‘context’ factors such as the degree of
industrialisation of a region or its relative prosperity) are not
taken into account for the purpose of reducing assistance rates.
The same applies to such criteria as complementarity with other
Community operations. The Commission also did not question
the structure of the financial arrangement submitted by the
project initiator.

As regards productive investments, the review of the
32.
projects has shown the method of setting assistance rates to be
relatively automatic:

— in most cases the rate granted is merely the rate applied for,

— financial cost-benefit analysis (which is mainly constructed
around the financial IRRs and NPVs and risk and sensitivity
analyses) was not taken into account for the adjustment of
the assistance rate. Some highly profitable projects with a
high financial IRR attracted high co-financing even though
Commission departments other than the examining geo-
graphical unit had emphasised the deadweight effect of assis-
tance. The audit has indicated that in certain cases deadweight
effects actually appeared after the projects were implemented
(see Box 4),

(4) ‘Simplification, transparency and coherence in the evaluation of major
public infrastructure projects: application of four elementary rules’,
DG Regional Policy, Note of 14.4.2003 formally addressed to the
competent authorities of the Member States and accession countries.
A note on the new rules for the application of the ‘financing gap’ for-
mulae had already been sent to the Committee for the Development
and Reconversion of Regions in July 2002.

(5) For both types of projects the principle that the polluter pays requires
setting a higher charging level to cover the operating and investment
costs, which reduces the financing gap and consequently the amount
of Community assistance. The modifications applied in relation to the
traditional formula correct this ‘distortion’ by providing that the for-
mula should not take account of the total net value of operating rev-
enues but only the part calculated as a proportion of the investment
assistance granted by the Commission. The formula is slightly differ-
ent for the Cohesion Fund and for the ERDF.

1.4.2008

EN

Official Journal of the European Union

C 81/11

— within the limits of the regulatory ceilings and the maximum
levels determined by the rules governing State aid, socioeco-
nomic concepts (such as value added, the creation or main-
tenance of employment, leverage effects) were not generally
taken into account in fixing the assistance rate. Nevertheless,
good practice was observed in the case of productive invest-
ments in Spain, where the competent national departments
also drew on non-financial criteria when specifying the rate,
and the co-financing rate was calculated by reference to
employment, the use of regional natural resources, value
added (especially in terms of increased productivity), techno-
logical innovation and knock-on potential,

— the Commission’s departments did not consider the use of
loans rather than grants. However, loans ought to be consid-
ered, particularly for productive investments with high finan-
cial
IRRs or for businesses with good financial ratios.
Generally speaking, aspects such as the financial situation of
the business or market conditions are not taken into account
in determining the level of assistance.

Box 4

In the case of one company generating substantial profits, the project
would have succeeded without Community assistance. The same
applies to another industrial production project, for which the busi-
ness concerned had almost no competitors in the European market.
However, in certain cases the effects of locating a project at a particu-
lar site might justify subsidies for projects with favourable IRRs. In the
case of a major project of another company, which, at the time, had
substantial shares of the world market for several projects, the cofi-
nanced investment was at a company site in an Objective 1 region
and Community grants played a significant role in the choice of site.

33.
The Commission is aware of these weaknesses. In 2004
it made an attempt at rationalisation by seeking to adjust Com-
munity assistance rates by reference to certain criteria. This pro-
duced indicative assistance rates weighted by each sector’s
anticipated rate of return, calculated on the basis of financial IRRs
from a sample of 200 infrastructure and productive projects. This
is a useful tool, although still very limited. In any case, it came too
late for examination of the files.

34.
The new guidelines published in 2006 (1) propose includ-
ing in the financing gap calculation factors that are not purely
financial, such as the existence of a public-private partnership and
equity (that is, the relative prosperity of the region). They also deal

(1) See Working Document No 4 ‘The new programming period 2007-
2013. Guidance on the Methodology for carrying out cost-benefit
analysis’, August 2006.

with the manner of combining subsidies and loans on the basis
of the profitability normally expected per sector. Even though
‘context’ factors are still not fully taken into account, these
improvements are in line with the approach favoured by this
report of seeking a wider and more objective framework for dif-
ferentiating the amount of aid. They must be applied consistently
and transparently to the financing decisions of the new period.

DO EX POST EVALUATIONS OF MAJOR PROJECTS GENERATE
RESULTS
SUBSEQUENT
DECISION-MAKING?

THAT ARE HELPFUL

FOR

Evaluation framework and audit criteria

Generally speaking, the aim of evaluating a project after
35.
completion is to account for the use of resources and the effec-
tiveness and efficiency of the operation undertaken. It is of par-
ticular interest to political
leaders, managing authorities and
beneficiaries. It must allow lessons to be drawn about the need for
supplementary or corrective measures to be taken in the future.
As the results generated by such evaluations concern the effects
produced by a project, they will also be used in quantifying the
objectives of future projects.

36.
The Structural Funds regulations stipulate that Commu-
nity interventions shall be subject to such a type of evaluation. It
is characterised as an ‘ex post evaluation’ and must be undertaken
by the Commission in collaboration with the Member States (2).
As an ERDF major project that is submitted outside an opera-
tional programme is regarded as an intervention, it is in principle
subject to ex post evaluation. There is no explicit regulatory
requirement as regards the individual major projects that make up
an intervention (3).

However, as regards the Cohesion Fund, there is express

37.
provision for ex post evaluation of each project (4).

(2) Article 6(2) of Regulation (EEC) No 2052/88 as amended by Regula-
tion (EEC) No 2081/93 and Article 43 of Regulation (EC)
No 1260/1999.

(3) Nevertheless, a separate ex post evaluation is possible. Article 27(4) of
the 2002 Financial Regulation provides for the evaluation of all pro-
grammes and activities which entail significant spending. Meanwhile,
Article 21 of the rules implementing the Financial Regulation requires
an interim and/or ex post evaluation of any programme or activity that
exceeds five million euro. Without prejudice to these provisions, the
Commission has asserted its right to chose the level of exclusiveness
at which an evaluation is to be carried out, taking into account a cer-
tain number of criteria including, in particular, financial size and
expected impact (see the recent Communication to the Commission
from Mrs Grybauskaitė, Responding to strategic needs: Reinforcing the use
of evaluation, SEC(2007) 213 of 14 February 2007, paragraph 2.1).
(4) Article 13 of Regulations (EC) No 1164/94 and (EC) No 1264/1999.

C 81/12

EN

Official Journal of the European Union

1.4.2008

The basic data used in the ex post evaluations are mainly
38.
collected by the Member States through the monitoring system.
During the 1994-1999 period, only Cohesion Fund projects were
subject to individual monitoring and annual
implementation
reports. Structural Funds projects were explicitly monitored at a
higher level (measures, priorities, operational programmes, Com-
munity support frameworks, etc. (1)).

At Commission level, the evaluations that specifically tar-
41.
get the major projects of the 1994-1999 period (the only ones to
have been evaluated up to now) (see paragraph 13), or concern
some aspect of them, have very heterogeneous aims and meth-
ods. They were not used by the national managing authorities and
beneficiaries, with one exception (see (d)):

The Court examined the evaluation phase from the point

39.
of view that:

(a)

the evaluation of major projects must serve to guide the
implementation of future projects. The national authorities
and the beneficiaries are primarily concerned by these evalu-
ations, which provide information for improving the effec-
tiveness and efficiency of operations on the ground;

(b)

the tools for monitoring project implementation and for
comparing their achievement must be defined from the
outset;

(c) data must be obtained for these tools to be used.

Several wide-ranging evaluations but very few individual
evaluations of major projects

The Member States only rarely carry out ex post evalua-
40.
tions of major projects (2). Only one of the twenty projects
audited by the Court had been evaluated (see Box 5). Naturally,
studies of the sectors to which the major projects belong — such
as, for example, the port and automotive sectors — may be con-
ducted by beneficiaries, managing administrations or external
bodies (universities or research centres). However, such studies do
not target the projects directly and are not in the nature of ex post
evaluations. Furthermore, they are not always forwarded to the
departments responsible for managing the Community funds.

Box 5

Only one of the projects in the Court’s sample had undergone indi-
vidual ex post evaluation. However, the macroeconomic impact was
assessed on the basis of factors which differed from those initially
provided for in the ex ante evaluation presented in support of the
application. As a result, the impact cannot be compared with the one
anticipated. What is more, there was no recalculation of the expected
financial return.

(a) a thematic evaluation of the impact of the Structural Funds
on transport infrastructures, which essentially considered the
strategies per country and per method of transport at pro-
gramme level. This evaluation provides little data with regard
to the efficiency, effectiveness and impact of the individual
projects;

(b) an ex post evaluation of Objective 1 that includes an exami-
nation of sixty major ERDF and Cohesion Fund projects. It
used a wide-ranging approach that included an assessment of
the appropriateness of the strategy, of effectiveness, effi-
ciency and impact, management and implementation sys-
tems and Community value added. However, only a small
number of major projects were considered from the point of
view of certain aspects of effectiveness;

(c) an evaluation of the macroeconomic impact of individual
Cohesion Fund projects — the only evaluation to consider
this aspect — by means of specially adapted macroeconomic
models. However, the real effects of the projects were not
examined, as the only data used was that drawn from the
ex-ante cost-benefit analyses (3);

(d) an ex post evaluation of projects (4) on the basis of a sample
of 200 Cohesion Fund projects. The evaluation assessed the
adequacy of the strategic planning, project design and project
selection procedures, management and implementation sys-
tems, effectiveness and efficiency. A sub-group of 60 projects
was later given a more in-depth examination from the point
of view of socioeconomic impact. Using the Guide to cost-
benefit analysis as a benchmark, the internal economic rate
of return was recalculated. This study’s recommendations
were discussed by the Member States concerned and led to
improvements in methodology.

(1) Article 25 of Regulation (EEC) No 2082/93.
(2) By way of example, in Ireland Forfás performs ex post evaluations of a
sample of productive investments every year. In Portugal, the Depar-
tamento de Prospectiva e Planeamento (DPP) used econometric mod-
els to carry out the macroeconomic ex post evaluation of certain
infrastructure investments.

(3) The models were applied especially to investments in nine major
transport infrastructures in environmental projects in Spain. The
application of
these models was reviewed in Special Report
No 15/2000.

(4) The projects were from the 1994-1999 programming period, with the
exception of eight projects adopted by the Commission between 2000
and 2002.

1.4.2008

EN

Official Journal of the European Union

C 81/13

Factors that impeded the ex post evaluation of major projects

Inadequate monitoring

The Court has on several occasions (1) highlighted the
42.
weaknesses of the evaluation exercises for the 1994-1999 period,
which were particularly marked by the absence of data or its
unreliability. The Court’s reports invited the Commission to con-
sider the possibility of collecting information on the individual
projects, without which the methodologies used are too theoreti-
cal or inapplicable. The audit of the major projects of that period
corroborated these findings.

Incomplete aid applications

43.
The ex post evaluation of major projects was at first hin-
dered by missing information in the aid applications. This was
noted both by the studies undertaken of major projects and by
the Court’s audit.

One of the evaluations carried out highlighted, in par-

44.
ticular, that:

(a)

the objectives of the projects were not always quantified;

(b) at the level of cost-benefit analysis, benefits which were dif-
ficult to evaluate — in particular environmental benefits —
were not quantified or were taken into account in the calcu-
lation of the ex ante economic internal rate of return without
adequate concrete information and justification having been
obtained. In numerous cases the anticipated results were not
quantified;

(c)

the distinction was not always drawn between permanent
jobs and temporary jobs; the methodological hypotheses for
ex ante estimates of
impact on employment were not
explained or different methodologies were applied.

45.
In addition to the inadequacy of the cost-benefit analysis
that has already been highlighted (see paragraphs 21-25), the
audit carried out by the Court confirmed the shortcomings found
in the earlier reports, in particular:

(a)

the absence, weakness or inadequacy of the indicators;

(b)

the absence of any reliable information concerning the ini-
tial situation, that is, the situation existing before the project
was implemented.

47.
Even in those cases where a monitoring system was
introduced at project level, the studies undertaken and the audit
carried out by the Court have shown that the monitoring process
did not ensure that adequate data was collected for the purposes
of evaluation.

48.

One of the evaluations carried out found that:

(a)

the collection of statistics relating to the use of transport
infrastructures did not focus on the economic implications of
this use, such as reductions in the cost of travel;

(b) analyses of unit costs did not result in a comparison of the

projects on the basis of benchmarks;

(c)

(d)

information on the revenue generated and on the application
of the polluter pays principle were inadequate in a number
of cases;

the ex post recalculation of the economic internal rate of
return was hampered by a lack of information concerning
actual results. It often had to be based on hypotheses and, in
the absence of any contradictory information, on the data
contained in the ex ante evaluation;

(e)

information concerning the creation of direct employment
and permanent employment was sometimes not available or
of poor quality.

The Court’s audit of major projects for the 1994-1999

49.
period also showed weaknesses in monitoring:

(a) as regards investments in infrastructures, with the exception
of one project, monitoring focused only on effects that were,
technically, easy to trace, i.e. creation of direct employment,
goods and passenger traffic, number of connections, areas
sold, number of beds per 1 000 inhabitants. Other direct
effects that are more difficult to monitor (petrol consump-
tion, comfort, environmental effects, quality of healthcare)
were not subject to any follow-up. The same applied a for-
tiori to indirect and more long-term effects (such as produc-
tivity, the creation of businesses and employment, prices,
etc.). Furthermore, those aspects used by the beneficiary to
calculate the estimated economic return (for example, for
road infrastructure: time savings, reduction of the cost of
transport, reduction of accidents, effects on the flow of traf-
fic generated and on additional traffic, etc.) were generally
not considered to be indicators that needed monitoring;

46.
As these factors are necessary in order to evaluate the
operations, the deficiencies discovered impede ex post evaluations.

(1) See in particular Special Reports No 15/2000 on the Cohesion Fund,
No 12/2001 concerning certain structural measures to improve the
employment situation, No 7/2003 on the implementation of assis-
tance programming for the period 2000 to 2006 and No 10/2006 on
ex post evaluations of Objectives 1 and 3 programmes for the 1994-
1999 period.

(b) as regards productive investments, monitoring was limited to
the physical and financial aspects of the investments and,
generally, to the direct creation of employment, since the
final reports drawn up in connection with the final payment
had to comply with the contractual requirements concern-
ing payment. Generally speaking, the managing body does
nothave information about such factors as the creation of
enterprises and indirect employment, productivity increases
or the environment.

C 81/14

EN

Official Journal of the European Union

1.4.2008

The 2000-2006 period:
compromised

evaluations at

risk of being

(d)

the annual report and the final report on implementation
must provide a clear view of the physical progress and financ-
ing of major projects.

50.
For the 2000-2006 period, the ERDF legislation stipu-
lates that, in future, for the major projects, the annual implemen-
tation reports on interventions must contain a separate chapter
on progress made and financing. In addition, financial and
progress indicators must be designed to provide separate infor-
mation for the major projects (1).

Nevertheless, although the annual

51.
implementation
reports contain a chapter on major projects, these chapters are
often restricted to a simple description of the projects and do not
present the physical progress made, or separate results and impact
indicators.

52.
In the same way, the priorities and measures concerning
major projects do not always include the indicators allocated to
them, or the indicators at priority and measure level are not con-
sistent with those for the major projects.

53.
At the level of the projects examined on-the-spot, while
a number of major projects have an adequate list of implemen-
tation, results and impact indicators, other projects either do not
refer to any indicator at all, or have only been allocated physical
and financial implementation indicators.

54.
Sometimes no clear link exists between the overall objec-
tives, the individual objectives and the designated indicators. A
number of objectives have not been allocated indicators at all.
Likewise, indicators put in place for projects which are similar but
cofinanced by the ERDF or the Cohesion Fund respectively are
not identical.

55.
Future ex post evaluations will be affected for the same
reasons as in the preceding period, i.e. the weaknesses found con-
cerning monitoring and indicators. The contribution of major
projects to the more global objectives of regional development
will therefore remain very difficult to define.

THE NEW REGULATIONS FOR THE 2007-2013 PERIOD

The new framework regulation for the 2007-2013
56.
period (2), which applies both to the ERDF and the Cohesion
Fund, introduces certain improvements as regards major projects:

(a)

the concept is redefined (3), with ERDF projects and Cohe-
sion Fund projects being treated equally;

(b)

the information to be submitted by the Member States is
clearly defined. In particular, there must be explicit justifica-
tion for a public contribution to the funding;

(c)

there is a clearer definition of revenue-generating projects
and how they are to be treated;

In addition, in 2006 the Commission published guide-
57.
lines on the preparation of major projects in the new period,
which made
(see
paragraph 34).

certain number of

improvements

a

58.
Finally, the rules provide for ex post evaluations to be car-
ried out by the Commission at programme level only. There is
therefore no provision for either EDRF or Cohesion Fund projects
to be evaluated individually.

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

There are a number of shortcomings in the preliminary

59.
examination procedures for major projects:

(a)

the project examinations varied in quality. Not until 2003
was a common approach provided for the managing depart-
ments (see paragraphs 17-20);

(b) many weaknesses were tolerated by the Commission. Great
disparities exist in the quality of the financial and socioeco-
nomic analyses presented in support of the projects. In par-
ticular, the quality of the analyses presented in support of
productive investments projects is poor. As a result, major
projects are adopted although the data provided do not per-
mit an adequate assessment (see paragraphs 21-25);

(c)

in spite of the improvements made by the Commission, the
factors used to adjust the rate of Community intervention are
not yet sufficiently precise. For productive investments the
possibility of granting a loan instead of a subsidy was not
examined (see paragraphs 26-34).

60.
Taking these factors into consideration, the Court enter-
tains doubts concerning the real added value of the approval pro-
cess for major projects as implemented by the Commission. The
exercise is cumbersome and does not exclude projects of inferior
quality or adjust the rate of intervention in a satisfactory manner.
Neither does it guarantee adequate preparation for the subsequent
evaluation. The weaknesses noted in the preliminary examination
procedure had repercussions on the ex post evaluation process.

Weaknesses were also discovered in the ex post

61.
evaluations:

(1) Article 36(3) of Regulation (EC) No 1260/1999.
(2) Regulation (EC) No 1083/2006. These aspects are referred to in

Articles 39-41, Article 37, Article 55 and Article 67(2)(g).

(3) New thresholds now apply: 25 million euro for environmental

projects and 50 million euro for other projects.

(a) hardly any individual ex post evaluations are carried out on
the initiative of the Member States. At Commission level
there is no global approach to the ex post evaluation of major
projects. Those evaluations carried out had very heteroge-
neous aims and methods (see paragraphs 40-41);

1.4.2008

EN

Official Journal of the European Union

C 81/15

(b)

the ex post evaluation of major projects for the 1994-1999
period was hindered by substantial shortcomings in the aid
applications,
in particular concerning anticipated effects,
indicators, and the initial situation. These deficiencies pre-
vented an evaluation of the real effects of the interventions
and the drawing of lessons from them for subsequent
decision-making (see paragraphs 43-46);

(c)

the manner in which the monitoring was carried out in this
period hampered the ex post evaluations, as numerous aspects
were neglected (see paragraphs 47-49).

62.
The Commission adopted a number of measures in 2003
in the light of the weaknesses found at the end of the ex post
evaluation of the 1994-1999 period. However, no substantial
progress was made with the adoption and monitoring procedures
for the major projects of the 2000-2006 period and the ex post
evaluation exercise for this period, which is still to be carried out,
is at risk of being compromised (see paragraphs 50-55).

Recommendations

63.
As regards the 2007-2013 period, the Community regu-
lations provide that, as in the preceding periods, the Commission
should approve the granting of Community assistance for each
major project. To avoid financing decisions being reduced to an
administrative exercise with no influence on the quality of the
projects adopted, the Commission should:

(a) apply more efficient preliminary examination procedures

based on the tools that it has itself developed;

(b) ensure that the cost-benefit analysis is applied rigorously,
particularly as regards information that can be used to check
the reliability of the hypotheses used;

(c)

continue its efforts to take into account the socioeconomic
factors that allow the rate of Community participation,

which at present is based mainly on financial aspects, to be
adjusted. Consideration of such factors should be based on
explicit and less random rules applied in a consistent manner.

64.
The Structural Funds regulations for the 2007-2013
period do not provide for the ex post evaluation of major projects
by the Commission. However, the Court considers that the evalu-
ation of major projects after completion is essential for the
national administrations and the beneficiaries, who need to verify
the real efficiency and effectiveness of the expenditure in order to
improve management and for whom the evaluation should pro-
vide real assistance with regard to programming. As a result:

(a)

it is important for major projects to be evaluated individu-
ally, against their own objectives, by the national administra-
tions and beneficiaries;

(b) as regards the information provided by the evaluation, which
must be communicated to it, the Commission must analyse
its requirements in respect of the information needed to
undertake the evaluation and, in particular, must review the
information needed to assess the impact of the major projects
on regional development objectives, to which the projects
are supposed to make a very substantial contribution. The
Court considers that this evaluation could be based on data
generated at major project level. To this end, and to ensure
the evaluation is feasible, the requirements regarding the
information to be collected and the reports to be drawn up
should be rationalised;

(c)

(d)

in order to guarantee the quality of subsequent evaluations,
the Commission must ensure, when examining the files, that
there are no lacunae in the applications submitted as regards
indicators and the initial situation;

for the same purpose, subsequent to the Commission’s
examination, monitoring must focus on the objectives and
on the implementation and results indicators set out in the
applications for assistance.

65.
In the longer term, the Court invites the Commission, the
European Parliament and the Council to give thought to the pur-
pose of the special arrangements for authorising major projects.

This report was adopted by the Court of Auditors in Luxembourg at
14 February 2008.

its meeting of

For the Court of Auditors
Vítor Manuel DA SILVA CALDEIRA
President

SAMPLES I AND II BY MEMBER STATE CONCERNED — LIST OF PROJECTS EXAMINED

ANNEX I

Checks on files and in the Member States (50 projects)

Additional checks on files at the Commission (36 projects)

Germany
— Buna Sow — project 25153584
— Logistik-Center Rheinhausen ‘Logport’ Infrastruktur
— Ausbau der Schienenstrecke (ABS) Berlin-Frankfurt/Oder
— Otto GmbH&Co KG
— Phönix West

Germany
/

Greece
— Athens Metro (OP Urban development 1994-1999)
— General hospital of Chania — project 18861 (OP Crete 1994-1999)
— Extension of Athens Metro from Panagoulis to the old airport at Elliniko (CF)
— Water supply of Irakleion and Agios Nikolaos from the Aposelemi dam (2nd. phase) (CF)
— Renewal and extension of the bus and trolley fleet of Athens public transport companies ETHEL-ILPAP

(OP SAAS 2000-2006)

— Extensions of Athens Metro and transfer stations (OP OALAA)

Spain
— Túnel de Somport — 941165005
— Autopista Madrid-Valencia — 961165008
— Linea Valencia-Tarragona-Acond 200-220 km/h entre ppkk 30 y 50-10.6
— CN-VI. Autovia Madrid-A Coruna. Tramo: Villartelin — Nadela — 1.1.1
— Fabricacion de Automoviles Renault Espana S.A. (Construction of an environmentally friendly car

engine) — VA/198/P07

— Ceramicas Belcaire, S.A. (Reconstruction of a manufacturing plant) — CS/265/P12
— Ampliación del Puerto de Alicante — C(2003) 4394 (2003 ES 16 CPT 018)
— Actuaciones Línea alta velocidad Madrid-Barcelona-frontera francesa: Tramo Madrid-Lleida

C(2000) 2939 (2000 ES 16 CPT 002)

— Arteria aductora del Campo de Pozos de Guadarrama y estación de tratamiento-2002

C(2004) 3565 (2002 ES 16 CPT 035)

— Saneamiento y depuración de diversas zonas de Cantabria — C(2000) 4307 (2000 ES 16 CPT 001)
— Ampliación Puerto de Ferrol (puerto exterior) — C(2004) 1862 (2002 ES 161 PR 001)
— Ampliación del Puerto de Sagunto — C(2003) 4232 (2002 ES 161 PR010)
— Glapilk, AIE — C(2003) 118 (2002 ES 161 PR 016)
— Petroquimica Espanola SA — C(2004) 145 (2003 ES 161 PR026)
— Tableros Tradema SA — C(2003) 5090 (2003 ES 161 PR 020)
— Construcciones aeronáuticas — Cádiz (2004 ES 161 PR 010)

Greece
— Développement du tram d’Athènes — C(2003)/122
— Sections d’Egnatia Odos-Ioannina-Metsovo — C(2004)/4367

Spain
— Dupont Ibérica SL — C(2003) 1342
— Fibras del Noroeste SA — C(2003) 180
— Bioetanol Galicia SA — C(2003) 181
— BP Oil Refinería de Castellón — C(2003) 3974
— Biocarburantes de Castilla y León — C(2004) 658
— Intercontinental Química SA — C(2004) 2306
— Airbus España SL, Fabricación automatizada Toledo — C(2003) 4736
— Airbus España SL, centro excelencia montajes Cádiz — C(2003) 4738
— Construcciones Aeronáuticas Sevilla — C(2004) 2441
— Ford España — C(2003) 4229 et C(2004) 2303
— Renault España — C(2004) 2304, C(2004) 3444 et C(2004) 3702
— Autovia de la Plata — C(2004) 275, C(2004) 0141, C(2004) 245 et C(2004) 2440
— Autovia del Mediterraneo — C(2003) 1336 et C(2003) 3809
— Autovia del Cantabrico — C(2003) 209, C(2003) 211 et C(2003) 2594

C
8
1
/
1
6

E
N

O

f
f
i
c
i
a
l

J
o
u
r
n
a
l

o
f

t
h
e

E
u
r
o
p
e
a
n
U
n
o
n

i

.

.

1
4
2
0
0
8

Checks on files and in the Member States (50 projects)

Additional checks on files at the Commission (36 projects)

France
— RN 20 — Déviation de Foix
— RCEA — Route Centre Europe Atlantique — C(2003) 2265
— Sevelnord — C(2003) 53
— Pilkington — C(2002) 3478

Ireland
— Gateway 2000 Ireland Ltd Dublin
— N11 Rathnew/Ashford by-pass
— M1 Cloghran/Lissenhall (stage 2)

Italy
— ESSO Italiana SpA
— SGS-Thomson microelectronics
— Produzione di pompe ad altissima pressione per motori diesel — C(2004) 5697
— RFI SpA — Potenziamento infrastrutturale e tecnologico della Caserta-Foggia — C(2005) 1268

Portugal
— Autoeuropa
— Auto-estrada A12
— Sistema multimunicipal de saneamento do Tejo-Trancão — C(2003) 2951
— Porto do Caniçal — C(2003) 4458
— Linha do Algarve II et V — C(2000) 3486 et C(2001) 3590
— Linha do Douro (troço Cête — Caíde) — C(2001) 2124
— Infineon technologies — C(2004) 3220
— Mabor Continental — C(2004) 661

France
— Tunnel du Lioran — C(2002) 4154
— Tramway de Clermont-Ferrand — C(2004) 4349
— Grand Halle d’Auvergne — C(2002) 1765
— Electrification des lignes vosgiennes — 2ème tranche — C(2003) 3131

Ireland
— Luas Line A (Light rail), C(2001)/4330
— N22 Ballincollig By-pass, C(2002)/4617

Italy
— Metropolitana di Napoli — Dante/Garibaldi — C(2005) 5235
— Metropolitana di Napoli — Vantivelli/Dante — C(2005) 5241
— Autostrada Siracusa-Gela — C(2005) 5233
— FRIEL — C(2005) 2839
— IVECO — C(2005) 1481

Portugal
— IC 10 — Ponte sobre o Tejo em Santarém — C(2001) 2727
— Armazenamento subterrâneo do Canriço — C(2001) 2011
— EPCOS — C(2003) 183
— CIMPOR — C(2003) 5099
— MST — Metropolitano do Sul do Tejo — C(2003) 3078
— Aeroporto Sá Carneiro (2003 PT 161 PR 004)

United Kingdom
— Phoenix natural gas network
— Liverpool Airport
— Merseyside Special Investment Fund (MSIF) — C(2001)2059 et C(2003)5386
— Yorkshire and Humber Partnership Investment Fund (Y&H PIF) — C(2003)206 et C(2003)5386

United Kingdom
— Frenchgate Interchange, Doncaster
— Combined universities in Cornwall (CUC)
— Edinburgh Biomedical Research Institute

.

.

1
4
2
0
0
8

E
N

O

f
f
i
c
i
a
l

J
o
u
r
n
a
l

o
f

t
h
e

E
u
r
o
p
e
a
n
U
n
o
n

i

C
8
1
/
1
7

C 81/18

EN

Official Journal of the European Union

1.4.2008

2000-2006 MAJOR PROJECTS EVALUATED — DISCOUNT RATE AND TIME HORIZONS

ANNEX II

Project

Sector

Financial discount
rate

Economic discount
rate

Time horizons

Germany

Ausbau der Schienenstrecke (ABS) Berlin-Frankfurt/Oder

Infrastructure

Phoenix west

Otto GmbH

Greece

Extension Metro Athens (CF)

Extension Metro Athens (OP AA)

Aposelemi dam (CF)

Bus and trolley Renewal (OP SAAS)

Extensions Metro Athens (OP OALAA)

Sections d’Egnatia Odos-Ioannina-Metsovo

France

RCEA

Sevelnord

Pilkington

Tunnel de Lioran

Grande Halle d’Auvergne

Tramway de Clermont-Ferrand

Electrification lignes vosgiennes

Déviation Sainte-Marie

Electrification Ligne Tours-Saincaize

ATMEL II

ATMEL III

Philips France Semi-Conducteurs

Spain

Ampliación del Puerto de Alicante

Actuaciones de Línea de alta velocidad Madrid-Barcelona-
frontera francesa: Tramo Madrid-Lleida

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Productif

Prod.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Prod.

Prod.

Pord.

Infr.

Infr.

Arteria aductora del Campo de Pozos de Guadarrama y est-
ación de tratamiento — 2002

Infr.-Env.

Saneamiento y depuración de diversas zonas de Cantabria

Infr.-Env.

Ampliación Puerto de Ferrol (puerto exterior)

Ampliación del Puerto de Sagunto

Glapilk, AIE

Tableros Tradema SA

EADS Petroquimica Española-Andalucia

EADS Construcciones aeronáuticas — Cádiz

Dupont Ibérica SL — C(2003) 1342

Fibras del Noroeste SA — C(2003) 180

Bioetanol Galicia SA — C(2003) 181

BP Oil Refinería de Castellón — C(2003) 3974

Biocarburantes de Castilla y León — C(2004) 658

Intercontinental Química SA — C(2004) 2306

Airbus España SL, Fabricación automatizada Toledo —
C(2003) 4736

Airbus España SL, centro excelencia montajes Cádiz —
C(2003) 4738

Infr.

Infr.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

7,5

6 et 5

5

6

6

5

6

Unspecified

6

—

—

—

—

4 and 5,2

6

6

—

6

5,5

5,5

6

6

6

5

5

6

6

5

10

5

8

3,5

6

Unspecified

Unspecified

Unspecified

Unspecified

Unspecified

Unspecified

3

—

5

6

6

3

5

6

6

8

—

—

8

—

6

6

8

8

—

—

6

6

6

5

5

6

6

—

—

—

—

—

—

—

—

—

—

—

—

6 + 30

15

14

30

30

45

12

Unspecified

25

25

—

—

20

31

35

31

23

30

5

5

5

30

20

25

25

20

20

25

10

5

10

Unspecified

Unspecified

5

10

5

Unspecified

Unspecified

Unspecified

1.4.2008

EN

Official Journal of the European Union

C 81/19

Financial discount
rate

Economic discount
rate

Project

Construcciones Aeronáuticas Sevilla — C(2004) 2441

Ford España — C(2003) 4229

Ford España — C(2004) 2303

Renault España — C(2004) 2304

Renault España — C(2004) 3444

Renault España — C(2004) 3702

Autovia de la Plata — C(2004) 275

Autovia de la Plata — C(2004) 0141

Autovia de la Plata — C(2004) 245

Autovia de la Plata — C(2004) 2440

Autovia del Mediterraneo — C(2003) 1336

Autovia del Mediterraneo — C(2003) 3809

Autovia del Mediterraneo — C(2003) 209

Autovia del Cantabrico — C(2003) 211

Autovia del Cantabrico — C(2003) 2594

Ireland

N11 Rathnew/Ashford By-Pass

M1 Cloghran — Lissenhall Stage 2

N22 Ballincollig By-Pass

Luas Line A

Italy

Produzione di pompe ad altissima pressione Bosch —
C(2004) 5697

RFI SpA — Ligne Caserta-Foggia — C(2005) 1268

Metropolitana di Napoli — Dante/Garibaldi — C(2005) 5235

Metropolitana di Napoli — Vanitelli/Dante — C(2005) 5241

Autostrada Siracusa-Gela — C(2005) 5233

FRIEL — C(2005) 2839

IVECO — C(2005) 1481

Portugal

Sistema multimunicipal de saneamento do Tejo-Trancão —
C(2003) 2951

Porto do Caniçal — C(2003) 4458

Modernização da linha do Algarve II-C(2000) 3486

Linha do Algarve V (troço Ermidas-Faro) — C(2001) 3590

Linha do Douro (troço Cête-Caíde) — C(2001) 2124

Infineon technologies — C(2004) 3220

Mabor Continental — C(2004) 661

IC 10 — Ponte sobre o Tejo em Santarém — C(2001) 2727

Sector

Prod.

Prod.

Prod.

Prod.

Prod.

Prod.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Infr.

Prod.

Infr.

Infr.

Infr.

Infr.

Infr.-Energy

Prod.

Infr.-Env.

Infr.

Infr.

Infr.

Infr.

Prod.

Prod.

Infr.

Armazenamento subterrâneo do Caniço — C(2001) 2011

Infr.-Energy

EPCOS — C(2003) 183

CIMPOR — C(2003) 5099

MST — Metropolitano do Sul do Tejo — C(2003) 3078

Aeroporto Sá Carneiro

United Kingdom

Merseyside Special Investment Fund (MSIF)

Yorkshire and Humber Partnership Investment (Y&H PIF)

Frenchgate Interchange, Doncaster

Combined universities in Cornwall (CUC)

Edinburgh Biomedical Research Institute

Prod.

Prod.

Infr.

Infr.

Prod.

Prod.

Unspecified

4,5

4,5

Unspecified

Unspecified

—

—

—

—

—

—

—

—

—

—

6

6

6

6

5

5

6

6

6

5

5

5

3

3

—

8,4

10

8

7

6,62

Unspecified

Unspecified

6

5

3,5

—

—

—

—

—

—

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

6

6

5

5

6

6

6

5

5

—

—

10

—

—

—

7

12

Time horizons

Unspecified

11

10

7

10

5

30

30

30

30

30

30

30

30

30

30

30

20

30

10

30

30

30

35

16

4 + 7

30

25

25

25

35

10

10

23

30

10

6

27

25/27

Unspecified

9/10

5

6

5

3,5

10

25

25

25

C 81/20

EN

Official Journal of the European Union

1.4.2008

THE COMMISSION REPLIES

SUMMARY

I.
Given the very significant amounts of funds involved in Major Projects, the Commission believes it is
important to subject them to a more rigorous application process and a closer examination. This helps to
ensure maximum impact on regional development and efficient use of Community resources.

III.
The Commission agrees that Major Project applications from Member States have been of mixed qual-
ity. To improve the quality of dossiers, it has sought clarifications and additional information from the project
promoters during the appraisal.

The Commission has taken steps to improve both the preparation of project applications by Member States
and its own appraisal. These will have their full impact in the 2007-2013 programming period.

The Commission agrees with the Court’s recommendations and intends to ensure that the improved

IV.
regulatory provisions are properly implemented.

For the 2007-2013 programming period the Commission has published a working document on cost-benefit
analysis (CBA), which should improve the quality of CBAs and provide a common framework for project
appraisal. Also, the document sets out the calculation method for determining the level of the EU grant based
on the provisions of Regulation 1083/2006.

V.
The Commission will use Major Project information to address specific evaluation questions, such as
unit costs in infrastructure projects, in the context of its wider European Regional Development Fund (ERDF)
2000-2006 ex post evaluation.

The Commission agrees that Member States could benefit from ex post evaluations of Major Projects

VI.
and has therefore always encouraged the practice.

The new application forms specifically ask for a description of the starting position, together with relevant indi-
cators of outputs and of the contribution to programmes objectives. The new tool for electronic exchange of
information will facilitate the collection of structured data on projects which are required by the Commission
implementing Regulation.

Annual implementation reports will have to include information on the progress and financing of Major
Projects.

VII.
ments in procedures for the 2007-2013 period.

Any review of the requirements for Major Projects should take account of the impact of the improve-

DOES THE COMMISSION PREPARE COMMUNITY FINANCING
OF MAJOR PROJECTS ADEQUATELY?

15.
The Commission examines the project applications as
thoroughly as possible, with reference to the factors and criteria
listed in Article 26(2) of Regulation (EC) No 1260/1999, and car-
ries out the necessary internal consultations, and if necessary
enlists the help of the European Investment Bank (EIB), in order
to make a full appraisal.

16.
The Commission agrees with the Court’s statement of
principles as to the standards to be aimed for in examining Major
Projects. However it is important to bear in mind the constraints
of the regulatory environment:

(a)

the Commission has to rely on the information sup-
plied by Member States;

(b) and (c) different rules applied for ERDF and the Cohesion
Fund which were interpreted differently by the Mem-
ber States. The Commission had no clear legal crite-
ria it could use to require Member States to accept a
given financial contribution.

17.
In addition, a Guide to the Cohesion Fund 2000-2006
was sent to the Member States in 2000. It contained annexes on
the ‘polluter pays’ principle and the financing of revenue-
generating projects.

The 2002 CBA guide is currently being updated to reflect the new
regulatory framework and include the latest methodological
developments. The new version will be issued in the second quar-
ter of 2008. It will strengthen the sections dealing with the inclu-
sion of environmental impacts and employment effects in CBA
results.

1.4.2008

EN

Official Journal of the European Union

C 81/21

18.

20.

First indent — Most 2000-2006 projects were in fact submitted
after the detailed guidance was provided in 2003.

Second indent — The results of the inter-service consultation are
shown in a fiche accompanying the proposed decision so that the
Commission is aware of any dissenting views before it takes the
decision. The Directorate General for Regional Policy endeavours
to procure any further documentation requested by the other ser-
vices to clarify uncertain points.

Third indent — There are no ground rules in the regulations
about the criteria governing the allocation of projects to the one
or the other fund. The range of projects to which the Funds can
be applied is determined by their regulations, namely Article 2 of
Regulation 1783/1999 for ERDF and Article 3 of Regulation
1164/94 for the Cohesion Fund. The Cohesion Fund has a much
narrower scope, being limited to TEN-T and environmental
projects.

Box 1 — In productive investment projects considerations of
regional development may outweigh arguments against funding
based on the financial profitability of projects. State aid is allowed
in order to compensate for the disadvantages of location.

With regard to the metro project, it was considered that the lack
of an immediate link with the TEN-T should not disqualify the
project from funding by the Cohesion Fund. The Environment
Directorate-General’s concern was that the acceptance of this
project should not be to the detriment of classic environmental
projects financed by the Cohesion Fund, which proved not to be
the case. The quantification of the environmental effects of an
individual project is very difficult, in view of the number of fac-
tors influencing air pollution and energy use in a major
conurbation.

Box 2 — For a project eligible for financing from either fund,
maturity is a valid consideration as the completion deadlines for
Cohesion Fund projects are somewhat more flexible than those
under ERDF programmes. The co-financing rate for the Cohesion
Fund project of 66 % was justified, given the higher permitted
co-financing rates for the Cohesion Fund and the restriction of
ERDF co-financing of projects generating substantial revenue
located in Member States covered by the Cohesion Fund to 50 %
(Article 29(4)(a) of Regulation 1260/1999).

The limited period for processing applications begins to
19.
run from the receipt of complete information. Therefore, the pro-
cessing periods referred to by the Court do not necessarily repre-
sent a breach of the deadline. As the Court itself notes, the reason
for the delays has mainly been the need to request clarifications
or missing information from the managing authorities.

First indent — The Commission encourages Member States to
present funding applications for projects that are sufficiently
mature. To avoid the presentation of funding applications for
immature projects, the Commission in 2002 decided that Cohe-
sion Fund project decisions would only be able to be amended
once in the project’s lifetime. The JASPERS facility now set up for
the 2007-2013 period (see reply to point 57) is also intended to
assist Member States to improve project preparation. Cost over-
runs can occur even for well prepared, mature projects, especially
as a result of inflation of raw material costs, and especially as
implementation is spread over several years. Delays are some-
times due to administrative or legal procedures in the Member
State, which are also independent of the state of maturity of the
project when presented.

Second indent — The regulations allow Member States to present
funding applications for projects which are virtually complete.
For productive investment projects the EU grant is often awarded
to ‘top-up’ the national aid within the allowable state aid limit; the
decision by the national authorities is taken well before the
project is completed. In any case, irrespective of the time of sub-
mission, the information required with applications imposes a
beneficial discipline on Member States in the justification of
projects and increases transparency.

Third indent — Productive investment projects are subject to the
rules on state aid schemes which include criteria for assessing eli-
gibility for aid and the compatibility of the aid package proposed
with the state aid rules. The standard application form moreover
requires information on how the ERDF contribution accelerates
or is essential for the implementation of the project. The Court is
also referred to the reply to point 32.

Fourth indent — Cases of groups of projects being accepted as an
ERDF Major Project are rare and justified by special circum-
stances. The advantage of Major Project status under the ‘n + 2’
rule is counterbalanced by the more stringent information
requirements. This reduces the risk of spurious Major Project
applications. In the Commission’s experience, there is a greater
risk of Member States failing to notify projects that meet the cri-
teria for Major Projects, and this risk was behind the definition in
the regulations as an ‘economically indivisible series of works.’

22.
The Commission agrees that the CBAs submitted were of
mixed quality. Use of the CBA guide was not compulsory, but the
Directorate-General for Regional Policy has encouraged national
authorities to follow it and has contributed to training actions.
Above all, however, the Commission has sought to promote
internal consistency of CBA methodological approaches across
projects at single Member State level because CBA is used to
assign financial resources to projects and therefore needs to be
applied consistently.

C 81/22

EN

Official Journal of the European Union

1.4.2008

In order to improve the quality and the comparability of CBAs for
the 2007-2013 programming period, the Directorate-General for
Regional Policy has produced a working document providing
guidance for carrying out CBA (working document No 4 — see
reply to point 24).

23.
The degree of detail required in the analysis advanced in
justification of funding proposals is also a matter of judgment,
especially where the analysis submitted is based on national cri-
teria for infrastructure investments or on a state aid examination.

24.
First paragraph — It is true that Major Project applica-
tions sometimes suffered from the weaknesses listed by the Court
and that the Commission has in some cases been unable to obtain
all the additional information necessary to fill gaps before approv-
ing projects.

For the 2007-2013 period working document No 4 containing
guidance for carrying out CBA provides a set of working rules and
indicative values for key parameters for CBA (e.g., discount rates,
reference periods). In the document the Commission recom-
mends that the Member States develop their own CBA guidelines
to take into account their specific institutional and economic con-
ditions. JASPERS (see reply to point 57) is currently assisting
Member States to develop national guidelines.

First indent — As from 2007, following the inclusion in the
Commission implementing regulation of a structured list of data
required for Major Project applications (Annex XX of Regulation
1828/2006), it will not be possible to omit key information like
the rates of return or the reference period, so that projects lack-
ing basic elements of CBA will not in principle be considered as
admissible.

Second indent — For the 2007-2013 programme period the
Commission recommends use of standard reference time hori-
zons by sector (working document No 4). The quantification of
environmental impacts is not straightforward; however, the CBA
guide makes it clear that this should be done and recommends
use of any of the various techniques that have been developed.

Third indent — The discount rates recommended by the Com-
mission are intended as benchmarks from which national rates
can diverge, but the Commission recommends internal consis-
tency across similar projects in a Member State or region. The
application forms for the 2007-2013 period distinguish clearly
between financial profitability with and without the EU grant.
Concerning the inclusion of depreciation, both the CBA guide
and working document No 4 make it clear that this is an account-
ing item and not a cash-flow and therefore it must be excluded
from the discounted cash-flow analysis.

Second paragraph — The Commission agrees on the relative
weakness of the CBAs for productive investments compared to
infrastructure projects. To improve the guidance to the Member
States on CBA for productive investments, the Commission
intends to add a case study on a productive investment in the
update of the CBA guide. The differences between the financial
analysis and the economic analysis for industrial projects will be
highlighted.

Box 3 — The Commission agrees that as far as possible informa-
tion on the impact of linked projects on their respective localities
should be taken into account when preparing a Major Project
application. It should be noted that network effects are typically
dealt with in the traffic and demand analysis and are therefore
already indirectly included in the CBAs.

The Commission has generally carried out a thorough
25.
appraisal. The relatively lengthy period for adoption of project
applications is typically due to the ‘requests for additional infor-
mation that the Commission has to make for a large majority of
Major Projects’ (see point 19).

While the Commission very seldom rejects projects, in some cases
Member States withdraw projects following observations from
the Commission.

27.
The purpose of the state aid examination is to ensure that
State aid is allowed only up to the level necessary to compensate
for the disadvantages of a particular location or reward the per-
formance of certain activities in the public interest (such as
research, training or environmental protection).

29.
In the cases referred to by the Court the cost-benefit or
financial analyses were, often after considerable internal discus-
sion, considered to justify the co-financing rates proposed.

The Evaluation Unit has developed CBA software which can be
used by desk officers to check the correctness of the analyses car-
ried out by the Member States, including the financing-gap cal-
culation done to determine the EU grant. The software has been
available since the end of 2004 and was recently updated to
incorporate the new co-financing rules for 2007-2013.

The 2000-2006 regulations afforded considerable flex-
30.
ibility in setting co-financing rates. In the Cohesion Fund the flex-
ibility allowed by the Regulation was even greater than for ERDF.
The Commission’s guidance (for example, the ‘appraisal summary
tables’ referred to in point 18) strives for maximum consistency
in the methods applied, especially internally within a given Mem-
ber State, but cannot impose completely standard practices.

For the 2007-2013 programme period, clearer rules for modu-
lating contributions from the Funds are provided by Article 55 of
Regulation 1083/2006. The calculation method for modulating
the EU grant has been explained in working document No 4 (see
reply to point 24). The application forms have also been adapted
to ensure provision of relevant information.

1.4.2008

EN

Official Journal of the European Union

C 81/23

Socio-economic conditions such as the relative prosper-
31.
ity of the region where the investment is made can play an indi-
rect role in the determination of the grant through the demand
analysis of the CBA.

The economic analysis mainly plays a role in the decision on
whether or not to fund an infrastructure project. However, once
a project is selected based on its socio-economic impact, the grant
is generally determined on the basis of the financial analysis. The
Commission believes that this is a correct approach. Giving less
money to economically very desirable projects could endanger
them.

For public-private partnership projects working docu-
34.
ment No 4 suggests that the use of a higher discount rate may be
appropriate to reflect the higher opportunity cost of capital to the
private investor. Regarding considerations of relative prosperity,
the working document recommends that tariff affordability is
explicitly taken into account when determining project revenues,
which are an input into the financing-gap formula.

DO EX POST EVALUATIONS OF MAJOR PROJECTS GENERATE
RESULTS
SUBSEQUENT
DECISION-MAKING?

THAT ARE HELPFUL

FOR

The Commission considers that it cannot decide on the structure
of the overall financing package proposed by the Member State
but only on the level of the Community grant.

36.
The ERDF ex post evaluation was carried out for the
1994-1999 period for the whole ‘Objective’ and therefore encom-
passed interventions at different levels, though mainly aggregate
programme level. Concerning 2000-2006 ex post evaluations, see
reply to point 55.

32.
Where a productive investment has been the subject of
an ad-hoc state aid approval or is in line with an approved state
aid scheme, it is normal to rely on this assessment in considering
an EU contribution requested within the agreed public aid pack-
age (see also replies to points 20 and 27).

38.
Information from the monitoring system does not form
the only basis for the ex post evaluation. Such data needs to be
supplemented with, for example, analysis of network effects,
longer term demand and impacts after the investment phases, etc.

As regards taking into account socio-economic factors to modu-
late the level of grant, see reply to point 31.

The Commission agrees that procedures for the comparative
appraisal of aid applications for productive investment projects at
national level are desirable in order to consider wider economic
issues and avoid over-financing.

Box 4 — The financial status of a company contemplating an
investment at a particular location is not a determining factor in
the decision on the award of aid but the expected financial prof-
itability of the (incremental) investment at the location. Dead-
weight is a typical risk in state aid schemes, since a large
proportion of Major Projects may be assumed to have positive
financial net present values and financial rates of return higher
than the financial discount rate and sometimes the very objective
of the aid is to attract to a less developed area an investment that
would have been carried out anyway in other areas.

33.
In addition to the guidance referred to by the Court, the
Commission had earlier provided the guidance referred to at
points 17 and 18 in 1999-2001, a guidance note on the deter-
mination of the rate of assistance for revenue-generating invest-
ments in the Structural Funds in 2002, and in 2003 had
recommended Member States to apply common principles for
discount rates and other parameters in financial analyses (see
point 28).

39.
The Commission agrees with the Court on the impor-
tance of evaluation as an input into future decisions on financing
and implementation and on the need for tools to determine and
collect the data required to carry out evaluations. It has made con-
siderable efforts to improve the arrangements for evaluation and
to foster an evaluation ‘culture’ among the actors responsible for
structural actions expenditure.

40.
Other exceptions have been the ex post evaluation of
CBAs of environmental projects in Ireland in 2002/2003 and a
recently published ex post evaluation of 1993-1999 and 2000-
2006 Cohesion Fund projects in Portugal. For the 1994-1999
period the regulations did not explicitly assign responsibility for
ex post evaluation (as is the case for the 2000-2006 and 2007-
2013 periods). Eventually it was the Commission (not the Mem-
ber States) which assumed the responsibility for ex post evaluation.
There was no regulatory requirement for carrying out ex post
evaluations at project level, although this is encouraged.

The performance of Major infrastructure Projects is often reported
on for the purpose of further development, as in the case of the
Athens Metro project with its ‘Metro Development Study’ and 20-
year business plan. These studies contain much information
about previous stages of the project similar to that in an ex post
evaluation.

41.
in para-
The evaluation referred to by the Court
graph 41(d) provided useful information for future decision-
making and the debate on the findings led to improvements in
the new regulatory framework for the 2007-2013 programming
period.

C 81/24

EN

Official Journal of the European Union

1.4.2008

The Commission has fully accepted the weaknesses due
42.
to lack of data in relation to the 1994-1999 period and has taken
steps to improve the gathering of data in the 2000-2006 period
and still more so in that for 2007-2013.

(e)

It is worth noting the difficulties in estimating indirect job
creation due to infrastructure projects. For this reason the
Commission has decided not to ask for this information in
infrastructural Major Project application for the 2007-2013
period.

43.
With the new tool for electronic exchange of informa-
tion, ‘SFC2007’, it will not be possible to submit a Major Project
application until all the structured data (Annex XX of Regulation
(EC) No 1828/2006) are encoded in the system.

49.

(a)

44.
The Commission has followed up the findings of the
evaluation referred to by the Court, as described in the reply to
point 41. See also the various actions taken to improve the data
available for evaluation referred in the replies to points 24, 42
and 43.

Indirect impacts forecast by the CBAs of infrastructure
projects such as indirect employment or reduction of road
congestion following a railway extension cannot be mea-
sured through monitoring. Financial and physical outputs
can be monitored during the investment phase, but the same
is not true for economic impacts which occur in the opera-
tional phase and more generally in the longer term.

(c) The Commission has recently developed a working docu-
ment on methods for measuring employment effects. It is
encouraging Member States to use this methodology in order
to improve the information on job creation.

(b) The Commission believes that the evaluation of broader eco-
nomic impacts at regional level (business creation, induced
job creation, productivity, and environment) can be better
assessed at programme rather than project level.

45.

(b)

It is true that in many funding applications the starting posi-
tion is not explicitly or comprehensively described. Never-
theless, as CBA is carried out on an incremental basis, i.e.
based on the difference between a scenario with the project
and the scenario without the project, the latter is of course
modelled on the existing situation. The new application
forms are more specific on this point. They also ask for the
relevant indicators on output and on the contribution to the
programmes objectives.

46.

See reply to point 42.

48.

(b) The Commission intends to explore the issue of unit costs in
more depth starting with the ERDF 2000-2006 ex post evalu-
ation. One of the evaluations will assess the efficiency of
ERDF funded Major Projects. It will also develop and test a
methodology for investigating unit costs which will later be
applied in future evaluations, such as the planned ex post
evaluations of Cohesion Fund and ISPA projects. The ulti-
mate objective is to create a database of unit costs which
include both operations funded under cohesion policy and
available international benchmarks.

(d) At the time of the ex post recalculation of the projects’ eco-
nomic rate of return, actual figures were mostly only avail-
able for the investment phases (i.e. investment cost amounts
and timing) and only in part for the operational phases. This
is why the ex post recalculation needed in several cases to use
the ex ante figures for measuring the project’s net benefits
which occur in the operational phase.

51.
It is intended to increase the attention paid to this require-
ment in the new period. At all events, such data are generally
found in the final reports after completion of the projects.

52-53.
indicators
Although they should be consistent,
selected for monitoring Major Projects should not necessarily be
the same as those set at priority level. This is particularly the case
for results or impact which can be meaningfully measured at pri-
ority level but not at project level. For instance, it would be hard
to measure improved accessibility due to a single project and the
improvement of accessibility measured at priority axis level would
not stem from the mere sum of the individual project impacts
because network effects would probably play an important role.

54.
While there are certainly weaknesses and inconsistencies
in the data being collected for the 2000-2006 period, the Com-
mission considers that the situation has improved compared to
the past and that there is a considerable amount of relevant mate-
rial that is being gathered and could be used for evaluation pur-
poses (see reply to point 55). For the 2007-2013 period, to
increase the standardisation and comparability of information,
the Commission has defined a set of ‘core indicators’ (Annex I to
working document No 2 on monitoring and evaluation indica-
tors). The Commission encourages Member States to use these
key output and result indicators at programme, not at project
level.

The ERDF 2000-2006 ex post evaluation is being organ-
55.
ised in 11 work packages assessing the overall effects of the policy
including selected thematic aspects. One of these has been
designed specially to collect the available information on indica-
tors across all programming documents. The data gathering work
package will facilitate the work of the consultants working on the
other work packages which will focus on filling the data gaps and
will carry out the actual evaluations. While in the ex post evalua-
tion the Commission will mainly evaluate
investments
co-financed by the ERDF at aggregate policy level, not at indi-
vidual project level, one of the work packages will focus on the

1.4.2008

EN

Official Journal of the European Union

C 81/25

impact of Major transport Projects, which will be analysed indi-
vidually. Another work package will aggregate information on
the employment effects of Major Projects. Work is also to be done
on unit costs in Major infrastructure Projects (see reply to
point 48(b)).

Given the very significant amounts of funds involved in
60.
Major Projects, the Commission believes it is important to sub-
ject them to a more rigorous application process and a closer
examination. This helps to ensure maximum impact on regional
development and efficient use of Community resources.

The Commission will carry out an ex post evaluation of 2000-
2006 period Cohesion Fund projects in 2009.

The Commission has taken steps to improve both the prepara-
tion of project applications by Member States and its own
appraisal. These will have their full impact in the 2007-2013 pro-
gramming period.

THE NEW REGULATIONS FOR THE 2007-2013 PERIOD

61.

57.
The Commission also refers to the improved application
forms, whose data will be recorded on the central database
‘SFC 2007’, the updating of internal instructions (via a special sec-
tion of the aide-mémoire for desk officers), the use of software to
check the coherence and consistency of CBA parameters and
results, and the introduction of a facility for technical assistance
of Member States with project preparation called ‘JASPERS’ (Joint
Assistance for Projects in European Regions). See also the replies
to points 22, 24, 29, 43 and 44.

58.
While the main emphasis will be on ex post evaluation at
programme level, the Commission will draw on data from Major
Projects as it is doing in the ex post evaluation for the 2000-2006
period (see reply to point 55). It will also continue to encourage
Member States to carry out evaluations, inter alia at project and
sector level, on a voluntary basis.

CONCLUSIONS AND RECOMMENDATIONS

The constraints of the regulatory environment for 2000-

59.
2006 should be borne in mind.

(a) The Commission has generally carried out a thorough
appraisal. Most 2000-2006 projects were submitted after
2003.

(b) The Commission agrees that funding applications were of
mixed quality and were relatively weak for productive invest-
ment projects. It generally made efforts to improve the qual-
ity of dossiers by requesting clarifications. For the 2007-
2013 period,
the
Commission’s examination of Major Project applications
have been improved.

the legal basis and the tools

for

(c) The 2000-2006 regulations afforded considerable flexibility

in setting co-financing rates.

(a) Since 2000 there has been no regulatory basis for requiring
Member States to carry out ex post evaluations at project level
although this is encouraged. Ex post evaluation for projects
was only required for the Cohesion Fund and was under-
taken by the Commission.

(b) The Commission recognised the weaknesses of the data pro-
vided with applications in the 1994-99 period and has taken
steps to improve the gathering of data in the 2000-2006
period and still more so in that for 2007-2013.

(c) There were deficiencies in monitoring data which handi-
capped evaluation. Steps have been taken to remedy these
deficiencies, especially for 2007-2013. However, the Com-
mission does not consider that indirect impacts can be moni-
tored at project level (see reply to point 49).

62.
While there are certainly weaknesses and inconsistencies
in the data being collected for the 2000-2006 period, the situa-
tion has improved and the limitations of previous evaluations
have been taken into account in planning those for the 2000-
2006 period.

The Commission agrees with the Court’s recommenda-
63.
tions and intends to ensure that the improved regulatory provi-
sions are properly implemented.

(a) As well as stricter requirements in the regulations on the
information required with applications and clarification of
the rules for determining the EU-grant, internal instructions
have been updated and procedures are being computerised.

(b) Together with the stricter requirements for applications and
the computerisation of processing, a working document on
cost-benefit analysis and an updated CBA guide should
improve the quality of CBAs and provide a common frame-
work for project appraisal.

(c) The regulations for 2007-2013 lay down clearer rules for
determining the EU contribution for revenue-generating
projects. The working document has provided guidance on
the application of the new rules. The Commission believes
that the modulation of the grant should be based on the
financial analysis which already indirectly takes into account
some socio-economic aspects.

C 81/26

EN

Official Journal of the European Union

1.4.2008

64.

(a) The Commission agrees that Member States could benefit
from ex post evaluations of Major Projects and has therefore
always encouraged the practice. It will continue to do so, but
believes that broader economic impacts at regional level can
in many cases be better assessed at programme level.

Annual implementation reports will have to include infor-
mation on the progress and financing of Major Projects.

(c) The new application forms specifically ask for a description
of the starting position, together with relevant indicators of
outputs and of the contribution to programmes objectives.

(b) The Commission’s recommendations to Member States on
monitoring take account of the needs of future evaluations,
in which project data will continue to play a major part. The
new tool for electronic exchange of information will facili-
tate the collection of the structured project data required by
the Commission implementing regulation.

65.
The Commission believes it is important to subject Major
Projects to additional requirements and scrutiny in view of the
large amount of resources they employ. Any such review as pro-
posed by the Court should take account of the impact of the
improvements in procedures proposed by the Commission for
the 2007-2013 period.