----

----

I Information

Court of Auditors

----

SPECIAL REPORT No 10/2006

on ex post evaluations of Objectives 1 and 3 programmes 1994 to 1999 (Structural Funds), together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, EC)

(2006/C 302/01)

TABLE OF CONTENTS

Glossary

ANNEXES:

Annex I — Breakdown of aid under the Structural Funds

Annex II — GDP and the employment rate

Annex III — GDP growth and the Structural Funds

Annex IV — HERMIN simulations and unemployment

The Commission's replies

GLOSSARY
ADAPT

Adaptation of the Workforce to Industrial Change (a Community initiative to promote employment and the adaptation of the workforce to industrial change)

ALMP

Active Labour Market Policies

CSF

Community Support Framework

Deadweight

Change observed among direct addressees following the public intervention, or reported by direct addressees as a consequence of the public intervention, that would have occurred, even without the intervention.

Displacement effect

Effect obtained in an eligible area at the cost of another area.

ERDF

European Regional Development Fund

ESF

European Social Fund

FIFG

Financial Instrument for Fisheries Guidance

HERMIN model

Macroeconomic model used to simulate the impact of SF interventions

Inception report

Report containing methodological questions such as the evaluation objectives, the methodological framework, the questionnaires and the organisations to be contacted

Monitoring Committee

Committee to oversee the strategic implementation of the funding programme

OP

Operational programme

RTD

Research and technical development

SFs

Structural Funds

SME

Small and medium-sized enterprises

Soft outcome

Result difficult to measure quantitatively

SPD

Single programming document

Synthesis Report

Final reports of DG REGIO (concerning Objective 1) and of DG EMPL (concerning Objective 3) about the ex post evaluation of operations financed by the ERDF and the ESF

ToR

Terms of reference

SUMMARY

INTRODUCTION

Framework for the evaluation

The basic objectives established for each evaluation

AUDIT SCOPE

EX POST ASSESSMENT OF OBJECTIVE 1 BY DG REGIO

The Court’s assessment of the quality of the evaluation

Preliminary remarks

Evolution of issues confronting the regions since 1994

Appropriateness of strategies adopted

Table 1

Summary of discrepancies noted in paragraph 24

Table 2

Actual expenditure (16)

Evaluation of effectiveness

Evaluation of efficiency

Evaluation of the impact of SFs

Evaluation of the management systems

Adequacy of the framework for evaluation

The terms of reference

Adequacy of supervision by the Commission

Relevance of evaluation recommendations for future programming periods

EX POST ASSESSMENT OF ESF OPERATIONS (UNDER OBJECTIVES 1 AND 3) BY DG EMPL

The Court’s assessment of the quality of the evaluation

Review of shortcomings in the data and in the analysis

Assessing the impact of ESF operations

Table 3

Observations on the hypotheses addressed in the examination of cases of impact on individuals

Evaluation of equal opportunities

Other aspects examined

Comments on the overall assessment by the Commission

Other insights from the national reports

Adequacy of the framework for evaluation

Recommendations concluding the evaluation

CONCLUSIONS

RECOMMENDATIONS

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 20 July 2006.

For the Court of Auditors

Hubert WEBER

President

----

(1)  SEC(2000) 1051 of 26 July 2000: Communication to the Commission ‘Focus on Results: Strengthening Evaluation of Commission Activities.’

(2)  The MEANS programme was initiated by the Commission as a response to the demand for developing evaluation methodology. The MEANS Collection consists of six volumes on the evaluation of socio-economic programmes.

(3)  MEANS Collection, No 1, p. 53.

(4)  Comprising the European Regional Development Fund (ERDF), the European Social Fund (ESF), the European Agriculture Guidance and Guarantee Fund (EAGGF) and the Financial Instrument for Fisheries Guidance (FIFG).

(5)  Synthesis Report, p. 53.

(6)  Hereafter referred to as ‘national evaluators’.

(7)  National evaluations examined relate to: Objective 1 Belgium, Greece, Spain, Germany, Italy, Ireland, Portugal, and Objectives 1 and 3 (ESF): Spain, France, Italy, Germany, United Kingdom.

(8)  See paragraph 5 for the overall figures.

(9)  Synthesis Report, p. 61.

(10)  Synthesis Report, p. 62.

(11)  Here referring to the active labour force as a percentage of the working age population.

(12)  Synthesis Report, Table 2.1, p. 75, Tables 2.5 and 2.6, p. 88 and 89.

(13)  Synthesis Report, p. 75.

(14)  Per capita figures, in this context, have been calculated with reference to the whole national population.

(15)  Million euro.

(16)  The same data as published in the report has been used in the compilation of Table 2. As a result, it can be seen that, for the Member States listed in Table 1, the sum of the three columns in this table do not add up to the total column.

(17)  This assessment needs identification and examination within a global framework of the causal links between the needs, the strategic objectives, the objectives of programmes/sub-programmes/measures and the allocation of resources. It needs also in principle a ranking of the needs and unsatisfactory socio-economic situations which the CSF aim to address and a hierarchy of inherent factors to be considered in order to ensure that resources are optimally targeted towards meeting needs.

(18)  Chapter 3.

(19)  A thematic evaluation is an evaluation which transversally analyses a particular point (a theme) in the context of several interventions within a single programme or of several programmes implemented in different countries or regions.

(20)  Thematic Evaluation of the Impact of Structural Funds on Transport Infrastructures, Final Report, November 2000, p. 8.

(21)  For example, in the case of Ireland, this result means that for every 100 euro invested in terms of construction and operating costs, the value of journey time savings, vehicle operating cost savings and safety benefits, was 13 euro for each year. Thus, the investment effectively paid for itself in eight years. (Thematic Evaluation of the Impact of Structural Funds on Transport Infrastructures, p. 8).

(22)  See section on impact.

(23)  Synthesis Report, p. 105.

(24)  The specific comment made was: ‘Whilst this would seem a logical recommendation, the context in which it is applied emerges as an important consideration’. Synthesis Report, p. 105.

(25)  Synthesis Report, p. 108.

(26)  Synthesis Report, p. 130.

(27)  To quote again from the study: ‘The most effective mechanism for the creation and maintenance of jobs is argued to be direct support for productive investment, although as this takes no account of deadweight and substitution effects, its overall effectiveness cannot be judged’. Synthesis Report, p. 130.

(28)  The implications of weak data and some general comments on this will be discussed in a later section.

(29)  ToR, p. 90.

(30)  Synthesis Report, p. 111.

(31)  ToR, p. 11.

(32)  Greater than 25 million euro for infrastructure and greater than 15 million euro for productive investment projects.

(33)  ToR, p. 13.

(34)  Synthesis Report, p. 143.

(35)  Synthesis Report, Tables 4.6, 4.7 and 4.8.

(36)  As noted above.

(37)  Synthesis Report, p. 71.

(38)  Synthesis Report, p. 72.

(39)  ‘Inclusion of private sector co-financing is of questionable value for an impact evaluation that is based on the use of a formal macro model and since considerable uncertainty and ambiguity surrounds the driving mechanisms behind the private sector CSF co-finance expenditures, we exclude them from our analysis’. Synthesis Report, p. 71.

(40)  UK-Speke Garston project; in this particular example, additional private sector investment of 223 million GBP was attracted, generating 4 600 jobs in direct employment. Synthesis Report, p. 204.

(41)  The four countries which qualify, in their totality, as Objective 1 regions.

(42)  In the case of Italy the planned construction of a HERMIN model foreseen in the ToR, was not done due to ‘contracting difficulties’. For the remaining regions, which did not have a HERMIN model available, a so-called ‘bottom-up’ estimation was undertaken.

(43)  The Report states: ‘Ideally we should use the actual ex post realised CSF expenditures. But these were not available for every country or region, disaggregated by priority and on an annual basis. In the interests of uniformity, we have used the planned CSF expenditure data as contained in the CSF 94 to 99 treaty documents. While these give a fairly accurate total for expenditure, they do not always give an accurate picture of the ex post scheduling of expenditures.’ (Synthesis Report, p. 150).

(44)  Synthesis Report, p. 161.

(45)  As a matter of fact, some studies claim that EU funding crowds out national investment such that the growth path of the economy without CSF funding would look better than the picture obtained from the HERMIN simulations.

(46)  This sensitivity analysis was carried out for eastern Germany, Spain, Greece, Ireland and Portugal.

(47)  Synthesis Report, p. 183.

(48)  These demonstrate the positive influence of the SFs on the development of multiannual programming, the application of the programme concept and of objective-based management, the establishment of management structures and procedures and in particular systems of monitoring and indicators, systems of quality assurance in public works, tendering procedures for government aid, the development of evaluation systems, capacity and culture, the project selection procedures and control systems. SFs stimulate the development of more appropriate structures and capacities to handle development projects.

(49)  Synthesis Report, section 9.1.

(50)  Synthesis Report, p. 196.

(51)  Opinion No 2/2005 on the proposal for a Council Regulation laying down general provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund (COM(2004) 492 final of 14 July 2004) (OJ C 121, 20.5.2005, p. 18); Special Report No 7/2003 on the implementation of assistance programming for the period 2000 to 2006 within the framework of the Structural Funds, paragraph 90 (OJ C 174, 23.7.2003, p. 22).

(52)  These included: greater effectiveness in programme development and monitoring, more effective project selection, greater legitimacy and transparency in decisions and decision-making processes, greater commitment and ownership of programme outputs, etc. See Thematic Evaluation of the Partnership Principle, p. II.

(53)  Synthesis Report, p. 188.

(54)  Synthesis Report, p. 218.

(55)  Annual Report concerning the financial year 2003, paragraphs 5.22, 5.32(d), 5.33 and 5.34 (OJ C 293, 30.11.2004, p. 162). Annual Report concerning the financial year 2004, paragraph 5.30 (OJ C 301, 30.11.2005, p. 143).

(56)  Synthesis Report, p. 224.

(57)  Synthesis Report, p. 219.

(58)  Synthesis Report, p. 221.

(59)  Dealing with ongoing evaluations in this context.

(60)  Synthesis Report, p. 222.

(61)  For example: ‘There is clear evidence that the partnership arrangements have deepened and widened and that there has been increasing delegation of responsibility for decision making to the programme partnership’ (Synthesis Report, p. 246).

(62)  ToR, p. 16.

(63)  Methodology Guide, p. 16.

(64)  ‘Net’ effects are estimated by subtracting from gross effects any deadweight and displacement/substitution effects.

(65)  Especially the links between the results of the analyses of impact and the analysis of effectiveness and appropriateness of the strategy.

(66)  Inception Report, p. 7.

(67)  ToR, p. 102.

(68)  Inception Report, page 13. Similarly, the Methodology Guide includes key questions as: ‘What was the overall impact of the structural funds on supporting regional convergence during the period 1994 to 1999? What were the economic effects of the Structural Funds investments in terms of convergence of income levels?’ (p. 24).

(69)  ToR, p. 94.

(70)  For example, a case was made for: ‘Providing much greater discretion to those Member States which have effective domestic monitoring systems to utilise their own monitoring requirements in relation to EU-supported initiatives. This would reduce the costs involved and the data which emerges would probably be more meaningful’. Synthesis Report, p. 249.

(71)  Synthesis Report, p. 249.

(72)  Synthesis Report, p. 251.

(73)  For example, increased labour force ‘flexibility’ as a result of training.

(74)  Synthesis Report, p. 124.

(75)  Synthesis Report, p. 125.

(76)  For instance, one can read in these notes that the figures for France, which amount to 17 million, were being ‘double-checked’ for ‘double counting’ while the figures for the Netherlands, which amount to 8,7 million, are also being ‘double checked with national teams’. Furthermore, it is indicated that data are largely missing for Germany, Ireland and the United Kingdom — although the later is listed as having 7,9 million beneficiaries and Ireland as having 680 000 beneficiaries.

(77)Table 3 reproduces part of Table 4.8 in the Synthesis Report.

(78)  Here refers primarily to gender equality.

(79)  Synthesis Report, p.173.

(80)  Combined measures enclose intensive counselling and training, work experience and training and employment incentives and training.

(81)  Synthesis Report, p. 122.

(82)  Paragraph 3.2.8.

(83)  Synthesis Report, p. 86.

(84)  Synthesis Report, p. 144.

(85)  For example, the achievement of qualifications and the soft outcomes presented in the UK evaluation report (representing ‘distance travelled toward the labour market’).

(86)  This was also stated by some evaluators (Spain and Italy).

(87)  For example, in Spanish cases.

(88)  One such example was quoted in the case of Germany.

(89)  In particular, the EES promotes stronger links between EU programme funding mechanisms and other policy instruments, regulations and national programmes including passive labour market policies.

(90)  For example, in the case of Germany: support to the objectives that target disadvantaged sections of the public, sustainable development, equal opportunities, financial leverage effects for the new Länder. Similarly, various other topics were highlighted in the other national reports.

(91)  National report, p. 95.

(92)  Methods Report, p. 24.

(93)  Methods Report, p. 44.

(94)  See evaluations concerning France, Spain and Italy.

(95)  Examples are: the need for a stronger link between the planned objectives and the local socio-economic context (Italy); a better targeting of beneficiaries (France); the expansion of integrated intervention approaches (United Kingdom); to widen the scope of the programmes to more innovative and risky actions (Spain); to strengthen anticipation actions in the training domain (Spain); to pay more attention to the less-favoured regions by putting more weight on variables such as unemployment, long term unemployment and social exclusion (Spain).

(96)  Synthesis Report, p. 188 for Objectives 1 and 3.

(97)  Synthesis Report, p. 198.

----

ANNEX I

Breakdown of aid under the Structural Funds — Available resources related to CSF 1994-1999

Breakdown of aid under the Structural Funds — Available resources related to Community Initiatives 1994 to 1999 (1)

----

(1)  Names of programmes related to Structural Funds; further references can be found in the Commission budget, subsection B2.

Source: Special Report No 16/98 (OJ C 347, 16.11.1998).

----

ANNEX II

GDP and the employment rate

Table 1 (2)

Regression of the change in GDP per capita on change in employment (CEMPL) and change in unemployment (CUNEMP)

Table 2 (3)

Regression of average change in GDP per capita on average change in employment (ACEMPL) and average change in unemployment (ACUNEMP)

----

(1)  Synthesis Report, p. 62.

(2)  50 observations used for estimation. BE, IR and DX are dummy variables for Belgium, Ireland and Germany. C1 is the intercept.

(3)  145 observations used for estimation. ID, LD, BED, DED are dummy variables for Ireland, Luxembourg, Belgium and Germany. C is the intercept.

----

ANNEX III

GDP growth and the structural funds

Table 1

Regression of average change in GDP per capita on average change in EU structural funds per capita (EUSFPC)

----

ANNEX IV

HERMIN simulations and unemployment

Table 1

HERMIN simulations

----

(1)  Evaluation report for Germany (O1), (pp. 42, 48, 70, 72, 85, 86).

(2)  Evaluation report for Ireland (pp. 61, 92).

(3)  Department for Forecasts and Planning (DPP).

(4)  National report, p. 210.

----

THE COMMISSION'S REPLIES

SUMMARY

INTRODUCTION

EX POST ASSESSMENT OF OBJECTIVE 1 BY DG REGIO

EX POST ASSESSMENT OF ESF OPERATIONS (UNDER OBJECTIVES 1 AND 3) BY DG EMPL

CONCLUSIONS

RECOMMENDATIONS

----

----

OPINION No 6/2006

on a proposal for a decision by the European Parliament and the Council amending and extending Decision No 804/2004/EC of the European Parliament and of the Council of 21 April 2004 establishing a Community action programme to promote activities in the field of the protection of the Community's financial interests (Hercule II programme)

(pursuant to Article 248(4), second subparagraph, EC)

(2006/C 302/02)

THE COURT OF AUDITORS OF THE EUROPEAN COMMUNITIES

Having regard to the Treaty establishing the European Community, and in particular Article 280(4) thereof,

Having regard to the proposal for a decision by the European Parliament and the Council amending and extending the Community action programme to promote activities in the field of the interest of the Community's financial interests (1),

Having regard to the Council's request for an opinion on this proposal, which was received by the Court of Auditors on 14 July 2006,

Having regard to Opinion No 8/2003 of the Court of Auditors of 11 November 2003 on the original Community action programme,

Whereas the proposed decision is based on Article 280(4) of the Treaty, which provides that: ‘The Council ... after consulting the Court of Auditors, shall adopt the necessary measures in the fields of the prevention of and fight against fraud affecting the financial interests of the Community’,

Whereas the main purpose of the Commission's proposal is to extend the Hercule (2) programme for a further period of seven years (2007 to 2013). The indicative allocation of 98,5 million euro results in average annual instalments of approximately 14 million euro. The programme provides OLAF with the financial resources needed to operate the service platform offered to the Member States for the purpose of coordinating the fight against fraud at European Union level (3),

HAS ADOPTED THE FOLLOWING OPINION:

This opinion was adopted by the Court of Auditors in Luxembourg at its meeting of 26 October 2006.

For the Court of Auditors

Hubert WEBER

President

----

(1)  COM(2006) 339 final of 28 June 2006.

(2)  Decision No 804/2004/EC of the European Parliament and of the Council (OJ L 143, 30.4.2004, p. 9).

(3)  Special Report No 1/2005 of the Court of Auditors, paragraphs 50 and 76 (OJ C 202, 18.8.2005).

(4)  Philip Morris Group (PMI).

----