# Official Journal C 119/2006

| | ISSN 1725-2423 |
| --- | --- |
| Official Journal
  of the European Union | C 119 |
| | |
| English edition | Information and Notices | Volume 4919 May 2006 |

ISSN 1725-2423

Official Journal

of the European Union

C 119

English edition

Information and Notices

Volume 4919 May 2006

---

| Notice No | Contents | page |
| --- | --- | --- |
| | I   Information |
| | Court of Auditors |
| 2006/C 119/1 | Special report No 2/2006 concerning the performance of projects financed under TACIS in the Russian Federation together with the Commission's replies | 1 |
| 2006/C 119/2 | Report by the Court of Auditors on the audit of the operational efficiency of the management of the European Central Bank for the financial year 2004 together with the replies of the European Central Bank | 17 |

Notice No

Contents

page

I   Information

Court of Auditors

2006/C 119/1

Special report No 2/2006 concerning the performance of projects financed under TACIS in the Russian Federation together with the Commission's replies

1

2006/C 119/2

Report by the Court of Auditors on the audit of the operational efficiency of the management of the European Central Bank for the financial year 2004 together with the replies of the European Central Bank

17

| EN | |
| --- | --- |

EN

---

I Information

Court of Auditors

| 19.5.2006 | EN | Official Journal of the European Union | C 119/1 |
| --- | --- | --- | --- |

19.5.2006

EN

Official Journal of the European Union

C 119/1

---

SPECIAL REPORT No 2/2006

concerning the performance of projects financed under TACIS in the Russian Federation together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, EC)

(2006/C 119/01)

TABLE OF CONTENTS

ABBREVIATIONS

| I-VII | SUMMARY |
| --- | --- |

I-VII

| 1-16 | INTRODUCTION |
| --- | --- |

1-16

| 1-8 | The TACIS Programme |
| --- | --- |

1-8

| 9 | TACIS in the Russian Federation |
| --- | --- |

9

| 10-11 | The EU and the Russian Federation |
| --- | --- |

10-11

| 12-16 | Audit scope and approach |
| --- | --- |

12-16

| 17-53 | OBSERVATIONS |
| --- | --- |

17-53

| 18-19 | Objectives not achieved or achieved only partially |
| --- | --- |

18-19

| 20-22 | Programming framework |
| --- | --- |

20-22

| 23-34 | Project identification and planning |
| --- | --- |

23-34

| 23-24 | Poor application of the Project Cycle Management System |
| --- | --- |

23-24

| 25 | Long project-planning schedule |
| --- | --- |

25

| 26-28 | Unrealistic underlying assumptions and objectives |
| --- | --- |

26-28

| 29-31 | Imprecise or missing objectives |
| --- | --- |

29-31

| 32-34 | Unsuccessful selection and involvement of beneficiaries |
| --- | --- |

32-34

| 35-38 | Project financing |
| --- | --- |

35-38

| 35-36 | Problems related to the determination of the project size |
| --- | --- |

35-36

| 37-38 | Rare national co-financing |
| --- | --- |

37-38

| 39-49 | Project Implementation |
| --- | --- |

39-49

| 39-40 | Delays in implementation |
| --- | --- |

39-40

| 41 | Contractors' performance was generally acceptable |
| --- | --- |

41

| 42 | Ineffective steering committees |
| --- | --- |

42

| 43-46 | Equipment not used for the purposes of the project |
| --- | --- |

43-46

| 47 | Purchases free of VAT from domestic suppliers impossible |
| --- | --- |

47

| 48-49 | The monitors performed well |
| --- | --- |

48-49

| 50-53 | Dissemination, sustainability and evaluation |
| --- | --- |

50-53

| 50-51 | Poor dissemination |
| --- | --- |

50-51

| 52 | Poor sustainability |
| --- | --- |

52

| 53 | Lack of evaluation |
| --- | --- |

53

| 54-56 | CONCLUSIONS AND RECOMMENDATIONS |
| --- | --- |

54-56

| 54-55 | Conclusions |
| --- | --- |

54-55

| 56 | Recommendations |
| --- | --- |

56

The Commission's replies

AP

| AP | Action Programme |
| --- | --- |
| CBC | Cross-Border Cooperation |
| CIS | Commonwealth of Independent States |
| DG | Directorate-General |
| EuropeAid | Europe Aid Cooperation Office |
| MIP | Multi-annual Indicative Programme |
| PCA | Partnership and Cooperation Agreement |
| PCM | Project Cycle Management |
| RCoA | Russian Chamber of Accounts |
| RELEX | DG External Relations |
| SME | Small and Medium Enterprise |
| SP | Strategy Paper |
| TACIS | Technical Assistance for the Commonwealth of Independent States |
| ToR | Terms of Reference |

Action Programme

CBC

Cross-Border Cooperation

CIS

Commonwealth of Independent States

DG

Directorate-General

EuropeAid

Europe Aid Cooperation Office

MIP

Multi-annual Indicative Programme

PCA

Partnership and Cooperation Agreement

PCM

Project Cycle Management

RCoA

Russian Chamber of Accounts

RELEX

DG External Relations

SME

Small and Medium Enterprise

SP

Strategy Paper

TACIS

Technical Assistance for the Commonwealth of Independent States

ToR

Terms of Reference

SUMMARY

| I. | From 1991 to 2006 over 7 000 million euro have been allocated by the European Union for the TACIS (1) programme devoted to assist the newly created Commonwealth of Independent States (CIS) after the break-up of the Soviet Union. The objective of this programme was and still is to promote the transition to market economy and to reinforce democracy and the rule of law in the recipient countries. The European Commission services are in the process of devising a new concept and legal basis for assistance from 2007 onwards (see paragraphs 1 to 8). |
| --- | --- |

I.

From 1991 to 2006 over 7 000 million euro have been allocated by the European Union for the TACIS (1) programme devoted to assist the newly created Commonwealth of Independent States (CIS) after the break-up of the Soviet Union. The objective of this programme was and still is to promote the transition to market economy and to reinforce democracy and the rule of law in the recipient countries. The European Commission services are in the process of devising a new concept and legal basis for assistance from 2007 onwards (see paragraphs 1 to 8).

| II. | The Russian Federation is the largest beneficiary of the TACIS programme, having received about 40 % of all funding under the programme. In total the TACIS funding allocated to the Russian Federation has been about 200 million euro annually. TACIS aims to help create the conditions for sound economic growth and develop a true partnership between the Russian Federation and the EU (see paragraphs 9 to 11). |
| --- | --- |

II.

The Russian Federation is the largest beneficiary of the TACIS programme, having received about 40 % of all funding under the programme. In total the TACIS funding allocated to the Russian Federation has been about 200 million euro annually. TACIS aims to help create the conditions for sound economic growth and develop a true partnership between the Russian Federation and the EU (see paragraphs 9 to 11).

| III. | The objective of the Court's audit was to assess whether the Commission had managed the TACIS projects in the Russian Federation in such a way that these projects had been effective. The Court examined 29 contracts (projects) out of a total of 275 from the contracting year 2000 or later and with an expiry date no later than the end of 2003, i.e. the most recent projects for which the sustainability could be assessed. These projects were financed under the Actions Programmes 1997 to 2000, and implemented mainly in 2002 and 2003. The total value of the contracts audited was over 56 million euro out of a total population of 109 million euro (see paragraphs 12 to 16). |
| --- | --- |

III.

The objective of the Court's audit was to assess whether the Commission had managed the TACIS projects in the Russian Federation in such a way that these projects had been effective. The Court examined 29 contracts (projects) out of a total of 275 from the contracting year 2000 or later and with an expiry date no later than the end of 2003, i.e. the most recent projects for which the sustainability could be assessed. These projects were financed under the Actions Programmes 1997 to 2000, and implemented mainly in 2002 and 2003. The total value of the contracts audited was over 56 million euro out of a total population of 109 million euro (see paragraphs 12 to 16).

| IV. | Overall the Court concluded that nine of the audited projects achieved their objectives. In eight cases the objectives were partially met and in twelve cases they were not achieved. The results of five of the projects were sustainable. Considering the above audit results, which show that the effectiveness of the use of TACIS funds has been low, the Court cannot assess the performance of TACIS projects in the Russian Federation positively. However, the audit revealed that the performance of contractors and monitors met the requirements of their contracts (see paragraph 55). |
| --- | --- |

IV.

Overall the Court concluded that nine of the audited projects achieved their objectives. In eight cases the objectives were partially met and in twelve cases they were not achieved. The results of five of the projects were sustainable. Considering the above audit results, which show that the effectiveness of the use of TACIS funds has been low, the Court cannot assess the performance of TACIS projects in the Russian Federation positively. However, the audit revealed that the performance of contractors and monitors met the requirements of their contracts (see paragraph 55).

| V. | The Court observed that the Commission had exhaustive programming systems. However, the programming process took long and, despite the Commission's efforts, there was a lack of a real dialogue between the Commission and the Russian authorities and beneficiaries. As a consequence there were some cases where the Russian authorities accepted assistance, which the final beneficiaries did not actually want. This had a negative impact on the effectiveness of the projects (see paragraphs 20 to 22). |
| --- | --- |

V.

The Court observed that the Commission had exhaustive programming systems. However, the programming process took long and, despite the Commission's efforts, there was a lack of a real dialogue between the Commission and the Russian authorities and beneficiaries. As a consequence there were some cases where the Russian authorities accepted assistance, which the final beneficiaries did not actually want. This had a negative impact on the effectiveness of the projects (see paragraphs 20 to 22).

| VI. | Other observations were that projects’ objectives were sometimes imprecise and not measurable, or sometimes missing, and in some cases underlying assumptions were unrealistic (see paragraphs 26 to 31). The selection and involvement of beneficiaries on some occasions was not successful (see paragraphs 32 to 34). There were problems related to the determination of the project size. In some cases individual projects were incorporated into one set of ToR with loose connection. Co-financing was hardly used (see paragraphs 37 and 38). In twelve cases (of 29) there were major delays in implementation, but the devolution of responsibilities to the Delegation resulted in more effective project management procedures (see paragraphs 39 and 40). Steering committees did not work well. Equipment was often not used for the purpose of the project (see paragraphs 43 to 46). Ex post assessment (evaluations) of the impacts of the projects did not take place (see paragraphs 48 and 53). |
| --- | --- |

VI.

Other observations were that projects’ objectives were sometimes imprecise and not measurable, or sometimes missing, and in some cases underlying assumptions were unrealistic (see paragraphs 26 to 31). The selection and involvement of beneficiaries on some occasions was not successful (see paragraphs 32 to 34). There were problems related to the determination of the project size. In some cases individual projects were incorporated into one set of ToR with loose connection. Co-financing was hardly used (see paragraphs 37 and 38). In twelve cases (of 29) there were major delays in implementation, but the devolution of responsibilities to the Delegation resulted in more effective project management procedures (see paragraphs 39 and 40). Steering committees did not work well. Equipment was often not used for the purpose of the project (see paragraphs 43 to 46). Ex post assessment (evaluations) of the impacts of the projects did not take place (see paragraphs 48 and 53).

| VII. | The main recommendations of the Court are the following (see paragraph 56): |
| --- | --- |

VII.

The main recommendations of the Court are the following (see paragraph 56):

| — | All planning of external action programmes should be based on a real dialogue between the Commission and the beneficiaries. Only such projects should be financed where there are clear objectives which are shared by the Commission and the beneficiaries. |
| --- | --- |

—

All planning of external action programmes should be based on a real dialogue between the Commission and the beneficiaries. Only such projects should be financed where there are clear objectives which are shared by the Commission and the beneficiaries.

| — | To improve the working of its systems the Commission should apply the Project Cycle Management (PCM) including the logical framework whenever appropriate, and it should avoid consolidating small projects with different objectives and beneficiaries. |
| --- | --- |

—

To improve the working of its systems the Commission should apply the Project Cycle Management (PCM) including the logical framework whenever appropriate, and it should avoid consolidating small projects with different objectives and beneficiaries.

| — | The Commission should appraise realistically the potential for sustainability during the planning phase of the projects; ensure the ex post assessment, or evaluation, of the completed projects in order to learn the good and bad practices; and provide information on the projects and their results for dissemination purposes. |
| --- | --- |

—

The Commission should appraise realistically the potential for sustainability during the planning phase of the projects; ensure the ex post assessment, or evaluation, of the completed projects in order to learn the good and bad practices; and provide information on the projects and their results for dissemination purposes.

INTRODUCTION

The TACIS Programme

| 1. | After the break-up of the Soviet Union, the European Union devised a technical assistance programme for the countries of the newly created Commonwealth of Independent States (CIS). The objective of this programme, named TACIS, was to promote the transition to market economics and to reinforce democracy and the rule of law in the recipient countries. |
| --- | --- |

1.

After the break-up of the Soviet Union, the European Union devised a technical assistance programme for the countries of the newly created Commonwealth of Independent States (CIS). The objective of this programme, named TACIS, was to promote the transition to market economics and to reinforce democracy and the rule of law in the recipient countries.

| 2. | Two Council Regulations (2) covering the period from 1991-1999 made available approximately 4 221 million euro for assistance to the partner countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Mongolia, the Russian Federation, Tajikistan, Turkmenistan, Ukraine and Uzbekistan. The current Council Regulation ((EC, Euratom) No 99/2000 of 29 December 1999 (3)) earmarks 3 138 million euro for TACIS assistance during the period from 2000 to 2006. The Commission services are in the process of devising a new concept and legal basis for assistance to these countries from 2007 onwards. |
| --- | --- |

2.

Two Council Regulations (2) covering the period from 1991-1999 made available approximately 4 221 million euro for assistance to the partner countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Mongolia, the Russian Federation, Tajikistan, Turkmenistan, Ukraine and Uzbekistan. The current Council Regulation ((EC, Euratom) No 99/2000 of 29 December 1999 (3)) earmarks 3 138 million euro for TACIS assistance during the period from 2000 to 2006. The Commission services are in the process of devising a new concept and legal basis for assistance to these countries from 2007 onwards.

| 3. | The aim of TACIS is to transfer know-how through technical assistance, including undertaking studies, preparation of plans, and giving advice and training. Consultants from the EU countries work with the beneficiaries with the aim of transferring know-how and skills, which are intended to be replicated by other projects and organisations in the recipient country. The aid is financed through payments from the Community budget being made directly to the European contractors providing the technical assistance. |
| --- | --- |

3.

The aim of TACIS is to transfer know-how through technical assistance, including undertaking studies, preparation of plans, and giving advice and training. Consultants from the EU countries work with the beneficiaries with the aim of transferring know-how and skills, which are intended to be replicated by other projects and organisations in the recipient country. The aid is financed through payments from the Community budget being made directly to the European contractors providing the technical assistance.

| 4. | In addition to the Council Regulation (see paragraph 2), relations between the European Union and the TACIS countries are based on Partnership and Cooperation Agreements (PCAs) signed between the EU, Member States and individual partner countries. The External Relations Directorate-General (RELEX) is responsible for the programming of TACIS in cooperation with the partner countries. The implementation of the TACIS programme is subsequently defined in multi-annual Country and/or Regional Strategy Papers (SPs) and Multi-annual Indicative Programmes (for the national programme as well as the regional or multi-country programmes) (MIPs). Before establishing the MIPs the Commission has to discuss the priorities with the Committee for assistance to the New Independent States (TACIS Committee) (4). |
| --- | --- |

4.

In addition to the Council Regulation (see paragraph 2), relations between the European Union and the TACIS countries are based on Partnership and Cooperation Agreements (PCAs) signed between the EU, Member States and individual partner countries. The External Relations Directorate-General (RELEX) is responsible for the programming of TACIS in cooperation with the partner countries. The implementation of the TACIS programme is subsequently defined in multi-annual Country and/or Regional Strategy Papers (SPs) and Multi-annual Indicative Programmes (for the national programme as well as the regional or multi-country programmes) (MIPs). Before establishing the MIPs the Commission has to discuss the priorities with the Committee for assistance to the New Independent States (TACIS Committee) (4).

| 5. | The Europe Aid Cooperation Office (EuropeAid) currently transforms the MIPs into annual or bi-annual Action Programmes (APs) in dialogue with the TACIS national coordinators, relevant ministries and other organisations in the beneficiary countries and after internal consultation within the Commission. The TACIS Committee is required to give its opinion on the contents of APs, including the list of projects. These documents are completed by financing memoranda (agreements between the EU and the beneficiary countries on the projects to be financed) and planning documents pertaining to the individual projects (Terms of Reference, ToR). The TACIS Cross-Border Cooperation (CBC) programme and the TACIS nuclear safety activities have their own strategy papers and indicative and action programmes. |
| --- | --- |

5.

The Europe Aid Cooperation Office (EuropeAid) currently transforms the MIPs into annual or bi-annual Action Programmes (APs) in dialogue with the TACIS national coordinators, relevant ministries and other organisations in the beneficiary countries and after internal consultation within the Commission. The TACIS Committee is required to give its opinion on the contents of APs, including the list of projects. These documents are completed by financing memoranda (agreements between the EU and the beneficiary countries on the projects to be financed) and planning documents pertaining to the individual projects (Terms of Reference, ToR). The TACIS Cross-Border Cooperation (CBC) programme and the TACIS nuclear safety activities have their own strategy papers and indicative and action programmes.

| 6. | The planning of the TACIS intervention is intended to be dialogue-driven. This means that at all levels of the programming and project identification stages the Commission and the beneficiary countries should agree on the priorities and the projects. Therefore all documents mentioned above from the MIPs down to the ToRs have to be discussed and signed by representatives of the Commission and of the beneficiary countries. |
| --- | --- |

6.

The planning of the TACIS intervention is intended to be dialogue-driven. This means that at all levels of the programming and project identification stages the Commission and the beneficiary countries should agree on the priorities and the projects. Therefore all documents mentioned above from the MIPs down to the ToRs have to be discussed and signed by representatives of the Commission and of the beneficiary countries.

| 7. | EuropeAid has overall responsibility for the management of the project cycle, from project identification to project appraisal, financing, project implementation and evaluation. |
| --- | --- |

7.

EuropeAid has overall responsibility for the management of the project cycle, from project identification to project appraisal, financing, project implementation and evaluation.

| 8. | In 2000, the Commission embarked on a major reform of the management of its external aid programmes (5). The main objective was to make significant improvements to the speed and quality of EC external aid. A key component of this reform was the extensive devolution of aid management tasks and responsibilities to the Commission's Delegations. As a result of this devolution, the Delegations became responsible for project preparation, contracting, and financial and technical implementation. The role of the central services in Brussels has developed from direct management of projects towards monitoring and supporting Delegations. |
| --- | --- |

8.

In 2000, the Commission embarked on a major reform of the management of its external aid programmes (5). The main objective was to make significant improvements to the speed and quality of EC external aid. A key component of this reform was the extensive devolution of aid management tasks and responsibilities to the Commission's Delegations. As a result of this devolution, the Delegations became responsible for project preparation, contracting, and financial and technical implementation. The role of the central services in Brussels has developed from direct management of projects towards monitoring and supporting Delegations.

TACIS in the Russian Federation

| 9. | The Russian Federation is the largest beneficiary of the TACIS programme, having received about 40 % of all funding under the programme. In total the TACIS funding allocated to the Russian Federation has been about 200 million euro annually. |
| --- | --- |

9.

The Russian Federation is the largest beneficiary of the TACIS programme, having received about 40 % of all funding under the programme. In total the TACIS funding allocated to the Russian Federation has been about 200 million euro annually.

The EU and the Russian Federation

| 10. | The European Union's policy towards the Russian Federation is geared towards: contributing to strengthening the rule of law through the development of efficient institutions as well as effective legislative, executive and judicial systems; improving the investment climate; enhancing legislative harmonisation with the EU; and cooperating in the fields of justice and home affairs, environment and nuclear safety. These measures should help create the conditions for sound economic growth in the Russian Federation, and developing a true partnership with the EU. |
| --- | --- |

10.

The European Union's policy towards the Russian Federation is geared towards: contributing to strengthening the rule of law through the development of efficient institutions as well as effective legislative, executive and judicial systems; improving the investment climate; enhancing legislative harmonisation with the EU; and cooperating in the fields of justice and home affairs, environment and nuclear safety. These measures should help create the conditions for sound economic growth in the Russian Federation, and developing a true partnership with the EU.

| 11. | The PCA between the European Union and the Russian Federation regulates the political, economic and cultural relations between the EU and the Russian Federation. It was signed in 1994 and entered into force on 1 December 1997 for an initial period of ten years. The Commission and the Russian authorities are in the process of devising a new instrument for the period after the expiry of the present PCA. |
| --- | --- |

11.

The PCA between the European Union and the Russian Federation regulates the political, economic and cultural relations between the EU and the Russian Federation. It was signed in 1994 and entered into force on 1 December 1997 for an initial period of ten years. The Commission and the Russian authorities are in the process of devising a new instrument for the period after the expiry of the present PCA.

Audit scope and approach

| 12. | The audit objective was to assess whether the Commission had managed the TACIS projects in the Russian Federation in such a way that these projects had been effective. To achieve this the Court audited a sample of projects in order to determine (i) the extent to which the projects' objectives had been achieved and (ii) how far projects were sustainable. In its audit the Court followed the stages of the Commission's system of Project Cycle Management (PCM): programming; project identification and planning; project financing; project implementation and evaluation. |
| --- | --- |

12.

The audit objective was to assess whether the Commission had managed the TACIS projects in the Russian Federation in such a way that these projects had been effective. To achieve this the Court audited a sample of projects in order to determine (i) the extent to which the projects' objectives had been achieved and (ii) how far projects were sustainable. In its audit the Court followed the stages of the Commission's system of Project Cycle Management (PCM): programming; project identification and planning; project financing; project implementation and evaluation.

| 13. | As the audit concentrated on the project level, the assessment of the programming process was limited to aspects which had a direct impact on the planning of the projects audited. The audit did not cover other general non-measurable political objectives of the projects. |
| --- | --- |

13.

As the audit concentrated on the project level, the assessment of the programming process was limited to aspects which had a direct impact on the planning of the projects audited. The audit did not cover other general non-measurable political objectives of the projects.

| 14. | In the framework of this audit, the Court was cooperating with the Russian Chamber of Accounts (RCoA). The RCoA performed a parallel audit on the side of relevant Russian authorities on TACIS projects. The report of RCoA will be published at the beginning of 2006. |
| --- | --- |

14.

In the framework of this audit, the Court was cooperating with the Russian Chamber of Accounts (RCoA). The RCoA performed a parallel audit on the side of relevant Russian authorities on TACIS projects. The report of RCoA will be published at the beginning of 2006.

| 15. | The Court examined 29 contracts (projects) out of a total of 275 from the contracting year 2000 or later and with an expiry date no later than the end of 2003 which were the most recent projects for which the sustainability could be assessed. These projects were financed under the Action Programmes 1997-2000, and implemented mainly in 2002 and 2003. The total value of the contracts audited was over 56 million euro out of a total population of 109 million euro (6). The Nuclear Safety projects as well as TACIS CBC projects were excluded from the sample, as the Court has published Special Reports based on audits of these areas in 1998 (7) (with a follow up in the Annual Report 2000) and 2001 (8). |
| --- | --- |

15.

The Court examined 29 contracts (projects) out of a total of 275 from the contracting year 2000 or later and with an expiry date no later than the end of 2003 which were the most recent projects for which the sustainability could be assessed. These projects were financed under the Action Programmes 1997-2000, and implemented mainly in 2002 and 2003. The total value of the contracts audited was over 56 million euro out of a total population of 109 million euro (6). The Nuclear Safety projects as well as TACIS CBC projects were excluded from the sample, as the Court has published Special Reports based on audits of these areas in 1998 (7) (with a follow up in the Annual Report 2000) and 2001 (8).

| 16. | The sample covered different kinds of actions and areas of intervention. In addition to large TACIS projects with a budget of at least 2 million euro, there were three smaller Bistro (9) projects, three smaller extensions of earlier projects and one feasibility study contract. |
| --- | --- |

16.

The sample covered different kinds of actions and areas of intervention. In addition to large TACIS projects with a budget of at least 2 million euro, there were three smaller Bistro (9) projects, three smaller extensions of earlier projects and one feasibility study contract.

OBSERVATIONS

| 17. | Most of the projects faced problems during one or several phases of the project cycle. These problems are elaborated below. Table 1 shows the number of projects affected by the main problems revealed by the audit.
  Table 1
  Number of projects affected by the main problems revealed by the audit | Number of projects with problems | Number of projects without problems |
| --- | --- | --- | --- |
| Total number of projects audited = 29 | | |
| Project identification and planning |
| Lengthy project-planning schedule | 8 | 21 |
| Unrealistic assumptions | 12 | 17 |
| Unrealistic objectives | 9 | 20 |
| Imprecise or missing objectives | 16 | 13 |
| Unsuccessful choice and involvement of the beneficiary | 11 | 18 |
| Project financing |
| Unclear project size determination | 8 | 21 |
| Project implementation |
| Delays in implementation | 12 | 17 |
| Problems with contractors' performance | 6 | 23 |
| Monitoring was ineffective | 3 | 26 |
| Objectives were not/only partly met | 12/8 | 9 |
| Conditionalities not set or not respected | 8 | 21 |
| Dissemination and sustainability |
| Poor dissemination | 17 | 12 |
| Poor sustainability | 24 | 5 |
| Lack of evaluation | 27 | 2 |

17.

Most of the projects faced problems during one or several phases of the project cycle. These problems are elaborated below. Table 1 shows the number of projects affected by the main problems revealed by the audit.

Table 1

Number of projects affected by the main problems revealed by the audit

Number of projects with problems

Number of projects without problems

Total number of projects audited = 29

Project identification and planning

Lengthy project-planning schedule

8

21

Unrealistic assumptions

12

17

Unrealistic objectives

9

20

Imprecise or missing objectives

16

13

Unsuccessful choice and involvement of the beneficiary

11

18

Project financing

Unclear project size determination

8

21

Project implementation

Delays in implementation

12

17

Problems with contractors' performance

6

23

Monitoring was ineffective

3

26

Objectives were not/only partly met

12/8

9

Conditionalities not set or not respected

8

21

Dissemination and sustainability

Poor dissemination

17

12

Poor sustainability

24

5

Lack of evaluation

27

2

Objectives not achieved or achieved only partially

| 18. | Nine projects fully achieved their objectives. In eight cases the objectives were partially reached and in twelve cases the objectives were not achieved.



  Box 1

  One project had several specific objectives. One of them was to ensure harmonisation of legal road standards between the EU and the Russian Federation. This objective could not be reached because there is no such EU standard. The second one was to enable comparative tests of road materials by providing a Russian agency with testing equipment. The equipment was delivered but it was still unused almost two years after the project's end. The third objective, training of the beneficiary's staff, was cancelled in the middle of the project. |
| --- | --- |

18.

Nine projects fully achieved their objectives. In eight cases the objectives were partially reached and in twelve cases the objectives were not achieved.

Box 1

  One project had several specific objectives. One of them was to ensure harmonisation of legal road standards between the EU and the Russian Federation. This objective could not be reached because there is no such EU standard. The second one was to enable comparative tests of road materials by providing a Russian agency with testing equipment. The equipment was delivered but it was still unused almost two years after the project's end. The third objective, training of the beneficiary's staff, was cancelled in the middle of the project.

| 19. | In general, the most successful projects in terms of meeting their objectives were the small ones, where the objectives were well and clearly identified. Also projects implemented in the regions generally performed better than projects targeting ministries at central level because of a greater level of commitment and feeling of ownership by the beneficiary, and more staff motivation. |
| --- | --- |

19.

In general, the most successful projects in terms of meeting their objectives were the small ones, where the objectives were well and clearly identified. Also projects implemented in the regions generally performed better than projects targeting ministries at central level because of a greater level of commitment and feeling of ownership by the beneficiary, and more staff motivation.

Programming framework

| 20. | Over the years the Commission has developed a system of programming external actions. In theory the system is comprehensive and logical. It has been stable already for many years. In line with other external action programmes many documents are drafted at various levels of TACIS programming and implementation (see paragraphs 4 and 5). |
| --- | --- |

20.

Over the years the Commission has developed a system of programming external actions. In theory the system is comprehensive and logical. It has been stable already for many years. In line with other external action programmes many documents are drafted at various levels of TACIS programming and implementation (see paragraphs 4 and 5).

| 21. | However, the programming process including the internal and external consultations is long. This put pressure on the project identification and preparation of the planning documents for the individual projects and finally when the deadlines for submission of projects were approaching, resulted in ToRs of insufficient quality (see paragraph 26). |
| --- | --- |

21.

However, the programming process including the internal and external consultations is long. This put pressure on the project identification and preparation of the planning documents for the individual projects and finally when the deadlines for submission of projects were approaching, resulted in ToRs of insufficient quality (see paragraph 26).

| 22. | The results of the projects audited, and the analysis of the problems faced by them, show that, despite the apparent efforts of the Commission during the formal consultations, the programming and subsequent project identification including preparation of financing memoranda and individual ToRs, were carried out in the absence of an adequate dialogue (see paragraph 6) with the various levels, i.e. Russian authorities administering the TACIS programme and beneficiaries (ministries and other organisations). For some areas of intervention the reform process in the Russian Federation had not advanced as hoped or the political environment on the beneficiary side had not matured enough for a constructive dialogue with the Commission. As a consequence there were some cases where the Russian authorities accepted assistance which the final beneficiaries did not actually want. This lack of dialogue and the length of the planning process (see paragraph 21) had a negative impact on the effectiveness of the projects. |
| --- | --- |

22.

The results of the projects audited, and the analysis of the problems faced by them, show that, despite the apparent efforts of the Commission during the formal consultations, the programming and subsequent project identification including preparation of financing memoranda and individual ToRs, were carried out in the absence of an adequate dialogue (see paragraph 6) with the various levels, i.e. Russian authorities administering the TACIS programme and beneficiaries (ministries and other organisations). For some areas of intervention the reform process in the Russian Federation had not advanced as hoped or the political environment on the beneficiary side had not matured enough for a constructive dialogue with the Commission. As a consequence there were some cases where the Russian authorities accepted assistance which the final beneficiaries did not actually want. This lack of dialogue and the length of the planning process (see paragraph 21) had a negative impact on the effectiveness of the projects.

Project identification and planning

Poor application of the Project Cycle Management System

| 23. | In 1992 the Commission adopted the PCM methodology setting out the format and contents of Financing Proposals and including a requirement to draw up a logical framework (10) in order to improve its project management. However, the use of the PCM methodology and logical frameworks was made obligatory by the Commission only from 2001. |
| --- | --- |

23.

In 1992 the Commission adopted the PCM methodology setting out the format and contents of Financing Proposals and including a requirement to draw up a logical framework (10) in order to improve its project management. However, the use of the PCM methodology and logical frameworks was made obligatory by the Commission only from 2001.

| 24. | In 2001 the Court published Special Report No 21/2000 on the management of the Commission's external aid programmes. One of the findings in the report was that the logical frameworks as part of the PCM were often not well prepared (11). This TACIS audit showed that the Commission still did not use the PCM, including the logical frameworks, systematically. The audit also showed that in most of the cases where the logical frameworks were used effectively, the projects achieved better results. The use of the PCM framework forced the planning to be done more thoroughly. |
| --- | --- |

24.

In 2001 the Court published Special Report No 21/2000 on the management of the Commission's external aid programmes. One of the findings in the report was that the logical frameworks as part of the PCM were often not well prepared (11). This TACIS audit showed that the Commission still did not use the PCM, including the logical frameworks, systematically. The audit also showed that in most of the cases where the logical frameworks were used effectively, the projects achieved better results. The use of the PCM framework forced the planning to be done more thoroughly.

Long project-planning schedule

| 25. | The time-span from the original idea or request for project funding to the inception of the actual project was very long. It may take up to three years to launch a consultancy project, which is too long, as the needs and situations of the beneficiaries change. Due to the delays, the projects, especially those related to policy development or crisis management, were already outdated when they eventually started. Consequently, as time had solved the original problems the contractors had to change the projects' approaches which further delayed their implementation.



  Box 2

  Prior to providing assistance in the area of anti-money laundering activities the Commission insisted on a feasibility study be produced. Mainly due to internal coordination difficulties between Commission services but also because it took the beneficiary organisation unduly long to approve its ToR, the start of this preparatory project itself was delayed by two years. Eventually, this delayed the technical assistance project — which was politically and economically important and also urgent for the Russian Federation — by five years. |
| --- | --- |

25.

The time-span from the original idea or request for project funding to the inception of the actual project was very long. It may take up to three years to launch a consultancy project, which is too long, as the needs and situations of the beneficiaries change. Due to the delays, the projects, especially those related to policy development or crisis management, were already outdated when they eventually started. Consequently, as time had solved the original problems the contractors had to change the projects' approaches which further delayed their implementation.

Box 2

  Prior to providing assistance in the area of anti-money laundering activities the Commission insisted on a feasibility study be produced. Mainly due to internal coordination difficulties between Commission services but also because it took the beneficiary organisation unduly long to approve its ToR, the start of this preparatory project itself was delayed by two years. Eventually, this delayed the technical assistance project — which was politically and economically important and also urgent for the Russian Federation — by five years.

Unrealistic underlying assumptions and objectives

| 26. | The long planning period did not always result in ToRs of good quality. In nine cases insufficient background investigations in the project design phase led to unrealistic objectives. Often, the project designs did not take into account the legal constraints in implementing the project, or the objectives were outside the competence of the beneficiary, or they were just too extensive and abstract to be reached by a single project. For example, most of the projects with objectives or tasks related to policy formulation, including enacting of legislation, were unsuccessful.



  Box 3

  A pilot project was established to test a new model for facilitating the start of new businesses. However, a similar model had already been established in the Russian Federation with the help of Canadian technical assistance in the middle of the nineties. |
| --- | --- |

26.

The long planning period did not always result in ToRs of good quality. In nine cases insufficient background investigations in the project design phase led to unrealistic objectives. Often, the project designs did not take into account the legal constraints in implementing the project, or the objectives were outside the competence of the beneficiary, or they were just too extensive and abstract to be reached by a single project. For example, most of the projects with objectives or tasks related to policy formulation, including enacting of legislation, were unsuccessful.

Box 3

  A pilot project was established to test a new model for facilitating the start of new businesses. However, a similar model had already been established in the Russian Federation with the help of Canadian technical assistance in the middle of the nineties.

| 27. | The participation of the Russian counterparts, as well as the line directorates-general at the Commission, in the processes of drafting and approval of the planning documents varied greatly and can be deemed to have been insufficient to secure realistic planning in some cases.



  Box 4

  In one case a city was involved in a district heating and power project but not nominated as a primary beneficiary. It did not endorse the ToR and as a result, the project was delayed and faced other managerial problems caused by a lack of commitment by the city authorities. |
| --- | --- |

27.

The participation of the Russian counterparts, as well as the line directorates-general at the Commission, in the processes of drafting and approval of the planning documents varied greatly and can be deemed to have been insufficient to secure realistic planning in some cases.

Box 4

  In one case a city was involved in a district heating and power project but not nominated as a primary beneficiary. It did not endorse the ToR and as a result, the project was delayed and faced other managerial problems caused by a lack of commitment by the city authorities.

| 28. | Examples of unrealistic assumptions and objectives include: the institutional framework of a beneficiary was not adequate or was non-existent, the target group had no interest in the services provided, the cooperation potential of the local administration was over-estimated or the beneficiary was not, by legislation, allowed to provide the services or information foreseen by the project. |
| --- | --- |

28.

Examples of unrealistic assumptions and objectives include: the institutional framework of a beneficiary was not adequate or was non-existent, the target group had no interest in the services provided, the cooperation potential of the local administration was over-estimated or the beneficiary was not, by legislation, allowed to provide the services or information foreseen by the project.

Box 5

  A remote and entirely local road management project had an inadequate overall objective, ‘to facilitate trade between the EU and the Russian Federation’.
  Another example of an unrealistic objective is presented in the box of paragraph 18 concerning the harmonisation of road standards.

Imprecise or missing objectives

| 29. | Thirteen projects audited had clear objectives. However, in sixteen of the projects audited, the initial project objectives in the ToR either described only the intended activities or input instead of setting clear and measurable objectives; or they were formulated in the same general way as in the political programming documents, forcing the contractors to define the actual purpose of the projects during implementation. The projects did not have objectives that would have identified the problems to be solved nor did they have such objectives against which the achievements of the projects could have been measured. |
| --- | --- |

29.

Thirteen projects audited had clear objectives. However, in sixteen of the projects audited, the initial project objectives in the ToR either described only the intended activities or input instead of setting clear and measurable objectives; or they were formulated in the same general way as in the political programming documents, forcing the contractors to define the actual purpose of the projects during implementation. The projects did not have objectives that would have identified the problems to be solved nor did they have such objectives against which the achievements of the projects could have been measured.

| 30. | For most of the projects audited objectively verifiable indicators had not been defined. In other cases, where the objectives were clear and intelligible, there were differences between the actual project activities and the project's initial objectives as described in the ToR. In one case the project was executed without specific objectives or objectively verifiable indicators. Instead it was directed towards providing ad hoc services to a ministry without being clear about the exact objective of the work. In the case of another project the beneficiaries in the private sector received assistance to develop their commercial activities which in fact was outside the scope of the overall objective of the project. |
| --- | --- |

30.

For most of the projects audited objectively verifiable indicators had not been defined. In other cases, where the objectives were clear and intelligible, there were differences between the actual project activities and the project's initial objectives as described in the ToR. In one case the project was executed without specific objectives or objectively verifiable indicators. Instead it was directed towards providing ad hoc services to a ministry without being clear about the exact objective of the work. In the case of another project the beneficiaries in the private sector received assistance to develop their commercial activities which in fact was outside the scope of the overall objective of the project.

| 31. | For some projects the design was overly complex, including too many separate and not interlinked tasks and objectives.



  Box 6

  One project had 23 different major outputs or objectives integrated into 8 different components. These covered themes like environmental protection in oil production, energy saving measures and studies of fish hatcheries. As a consequence of this complexity several of the foreseen objectives, including the two most significant ones, were not fully achieved. |
| --- | --- |

31.

For some projects the design was overly complex, including too many separate and not interlinked tasks and objectives.

Box 6

  One project had 23 different major outputs or objectives integrated into 8 different components. These covered themes like environmental protection in oil production, energy saving measures and studies of fish hatcheries. As a consequence of this complexity several of the foreseen objectives, including the two most significant ones, were not fully achieved.

Unsuccessful selection and involvement of beneficiaries

| 32. | Selecting suitable beneficiaries as project partners was very problematic because the project objectives were sometimes outside the competence of the project partner, or the beneficiary was not ready to receive or able to benefit from the support. In one case the Commission wanted to support a federal agency for political reasons, in view of the forthcoming cooperation negotiations between the EU and the Russian Federation. However, this agency was not in the end involved in the actual negotiations. |
| --- | --- |

32.

Selecting suitable beneficiaries as project partners was very problematic because the project objectives were sometimes outside the competence of the project partner, or the beneficiary was not ready to receive or able to benefit from the support. In one case the Commission wanted to support a federal agency for political reasons, in view of the forthcoming cooperation negotiations between the EU and the Russian Federation. However, this agency was not in the end involved in the actual negotiations.

| 33. | When a project had several beneficiaries, implementation faced a number of problems. Lack of cooperation between the federal and regional authorities or lack of cooperation between different federal and between different regional authorities was usual. This failing, although apparent, had not been taken into consideration during the planning.



  Box 7

  In one case the initiating beneficiary was operating only at the level of a region. To meet the minimum size requirements of the project budget it had to be expanded to cover several regions. To solve the problem another beneficiary was created but only on paper. It did not have legal identity, nor did it otherwise exist outside the project planning documents. |
| --- | --- |

33.

When a project had several beneficiaries, implementation faced a number of problems. Lack of cooperation between the federal and regional authorities or lack of cooperation between different federal and between different regional authorities was usual. This failing, although apparent, had not been taken into consideration during the planning.

Box 7

  In one case the initiating beneficiary was operating only at the level of a region. To meet the minimum size requirements of the project budget it had to be expanded to cover several regions. To solve the problem another beneficiary was created but only on paper. It did not have legal identity, nor did it otherwise exist outside the project planning documents.

| 34. | In five cases audited the beneficiaries were not involved in the project planning, or not early enough, to have an influence on the project design. |
| --- | --- |

34.

In five cases audited the beneficiaries were not involved in the project planning, or not early enough, to have an influence on the project design.

Project financing

Problems related to the determination of the project size

| 35. | According to the programming guidelines, the minimum budget size of TACIS projects in the Russian Federation should be 2 million euro in order to achieve sufficient impact and visibility. However, the programme may finance a number of small-scale initiatives, including Bistro projects. Most projects in the audited sample had a budget of between 2 and 3 million euro (see paragraph 16). In eight cases it was not clear from the project documentation how the project budgets had been arrived at. When the projects’ objectives are not clear, defining a well-founded budget is difficult. |
| --- | --- |

35.

According to the programming guidelines, the minimum budget size of TACIS projects in the Russian Federation should be 2 million euro in order to achieve sufficient impact and visibility. However, the programme may finance a number of small-scale initiatives, including Bistro projects. Most projects in the audited sample had a budget of between 2 and 3 million euro (see paragraph 16). In eight cases it was not clear from the project documentation how the project budgets had been arrived at. When the projects’ objectives are not clear, defining a well-founded budget is difficult.

| 36. | In some cases this minimum size led to problems. Instead of launching several, well-defined and focused projects, multiple tasks were incorporated into one set of ToR with loose connection or no synergy at all (see paragraph 31). The audit results show that such a budget-driven approach, which artificially creates larger projects, is unlikely to enhance the projects' impact and visibility.



  Box 8

  In the case of an industrial restructuring project 30 subprojects were planned. The budget was set in order to reach the minimum size rather than to satisfy the need of the subprojects. |
| --- | --- |

36.

In some cases this minimum size led to problems. Instead of launching several, well-defined and focused projects, multiple tasks were incorporated into one set of ToR with loose connection or no synergy at all (see paragraph 31). The audit results show that such a budget-driven approach, which artificially creates larger projects, is unlikely to enhance the projects' impact and visibility.

Box 8

  In the case of an industrial restructuring project 30 subprojects were planned. The budget was set in order to reach the minimum size rather than to satisfy the need of the subprojects.

Rare national co-financing

| 37. | According to the TACIS Regulation, national co-financing of projects was to be actively encouraged by the Commission. However, co-financing requirements are rare in the TACIS projects in the Russian Federation, even though they are intended to have a positive impact on the beneficiary's and contractor's commitment to achieve the expected results. There were only two cases among the audited projects where the beneficiary was co-financing the activities. The co-financing had a positive impact on the ownership of the projects by the beneficiaries which led to achievement of the projects' objectives. |
| --- | --- |

37.

According to the TACIS Regulation, national co-financing of projects was to be actively encouraged by the Commission. However, co-financing requirements are rare in the TACIS projects in the Russian Federation, even though they are intended to have a positive impact on the beneficiary's and contractor's commitment to achieve the expected results. There were only two cases among the audited projects where the beneficiary was co-financing the activities. The co-financing had a positive impact on the ownership of the projects by the beneficiaries which led to achievement of the projects' objectives.

| 38. | Most of the projects include in-kind contributions from the beneficiaries such as commitments to make local counterpart staff available or to provide the contractor with premises. In eight cases there were problems regarding this indirect co-financing because the commitments were not honoured, particularly for projects in Moscow. The ministries in the Russian Federation generally are not legally allowed to provide the contractors with office accommodation within their premises. Although the Commission was aware of the problem, it continued to insert such conditions into the ToRs. |
| --- | --- |

38.

Most of the projects include in-kind contributions from the beneficiaries such as commitments to make local counterpart staff available or to provide the contractor with premises. In eight cases there were problems regarding this indirect co-financing because the commitments were not honoured, particularly for projects in Moscow. The ministries in the Russian Federation generally are not legally allowed to provide the contractors with office accommodation within their premises. Although the Commission was aware of the problem, it continued to insert such conditions into the ToRs.

Project Implementation

Delays in implementation

| 39. | Seventeen projects were implemented within the time set. In the remaining twelve cases there were major delays in implementation. In this respect the most critical part was the inception phase, which usually took more time than planned. Those delays were mostly linked to an inadequate quality of the preparation of the projects. Due to the poor ToR, the projects needed many approvals for changes during inception and implementation. The procedure to change the project approach through addenda is time-consuming. Even a change of personnel requires the same procedure as modifications to the project's actual substance. |
| --- | --- |

39.

Seventeen projects were implemented within the time set. In the remaining twelve cases there were major delays in implementation. In this respect the most critical part was the inception phase, which usually took more time than planned. Those delays were mostly linked to an inadequate quality of the preparation of the projects. Due to the poor ToR, the projects needed many approvals for changes during inception and implementation. The procedure to change the project approach through addenda is time-consuming. Even a change of personnel requires the same procedure as modifications to the project's actual substance.

| 40. | The devolution of project management responsibilities from Commission headquarters to the Delegation resulted in more effective management procedures through more direct contacts between the project managers and project stakeholders; in fact, more proactive management by the Delegation even salvaged several of the projects from serious difficulties. These observations are in line with those of the Court's audit of the devolution of EC external aid management to the Commission Delegations published in Special Report No 10/2004 (12). However, the implementation of projects which had been programmed and prepared centrally in Brussels and which in 2002/2003 were transferred to the EC Delegation in Moscow (about half of all the projects audited) encountered one-off delays and obvious managerial deficiencies because of this transfer process. |
| --- | --- |

40.

The devolution of project management responsibilities from Commission headquarters to the Delegation resulted in more effective management procedures through more direct contacts between the project managers and project stakeholders; in fact, more proactive management by the Delegation even salvaged several of the projects from serious difficulties. These observations are in line with those of the Court's audit of the devolution of EC external aid management to the Commission Delegations published in Special Report No 10/2004 (12). However, the implementation of projects which had been programmed and prepared centrally in Brussels and which in 2002/2003 were transferred to the EC Delegation in Moscow (about half of all the projects audited) encountered one-off delays and obvious managerial deficiencies because of this transfer process.

Contractors' performance was generally acceptable

| 41. | Although the objectives were often not achieved or achieved only partially (see paragraph 18), in most of the cases the contractors had delivered the output (deliverables) as foreseen either in the original ToR or in the ToR amended through addenda. A typical problem for the contractors was to find and maintain suitable staff in more remote project locations when the staff originally allocated was not available. Only one small project was a failure due to a lack of output by the contractor. Another contractor faced financial problems during project implementation, which led to significant delays in implementation. |
| --- | --- |

41.

Although the objectives were often not achieved or achieved only partially (see paragraph 18), in most of the cases the contractors had delivered the output (deliverables) as foreseen either in the original ToR or in the ToR amended through addenda. A typical problem for the contractors was to find and maintain suitable staff in more remote project locations when the staff originally allocated was not available. Only one small project was a failure due to a lack of output by the contractor. Another contractor faced financial problems during project implementation, which led to significant delays in implementation.

Ineffective steering committees

| 42. | Where steering committees had been established they usually did not work effectively or efficiently. This hampered the implementation of the projects. The powers of the committees were limited by the Commission so the Russian members did not see what kind of an impact the steering committees could have and were, thus, not actively participating in the meetings. The contractors' project managers had to take over (part of) the tasks of the steering committees. |
| --- | --- |

42.

Where steering committees had been established they usually did not work effectively or efficiently. This hampered the implementation of the projects. The powers of the committees were limited by the Commission so the Russian members did not see what kind of an impact the steering committees could have and were, thus, not actively participating in the meetings. The contractors' project managers had to take over (part of) the tasks of the steering committees.

Equipment not used for the purposes of the project

| 43. | According to the TACIS Regulation (Euratom, EC) No 1279/96 the programme shall cover reasonable costs of supplies required in support of the implementation of the technical assistance (13). TACIS Regulation (EC, Euratom) No 99/2000 states that a maximum of 20 % of the annual budget can be allocated to investment (14). In practice the service contracts included a supply component which obliged the technical assistance contractors, in cooperation with the beneficiaries, to define the technical specifications of the supplies, and to manage the procurement process in accordance with the TACIS procurement rules. |
| --- | --- |

43.

According to the TACIS Regulation (Euratom, EC) No 1279/96 the programme shall cover reasonable costs of supplies required in support of the implementation of the technical assistance (13). TACIS Regulation (EC, Euratom) No 99/2000 states that a maximum of 20 % of the annual budget can be allocated to investment (14). In practice the service contracts included a supply component which obliged the technical assistance contractors, in cooperation with the beneficiaries, to define the technical specifications of the supplies, and to manage the procurement process in accordance with the TACIS procurement rules.

| 44. | There was not always clear synergy between the technical assistance and equipment provided under the projects. Nor was there any evidence that the equipment component had addressed the specific areas described (15) in the Regulation or Guidelines. The beneficiaries treated the equipment component (mostly office and IT equipment) more as an additional bonus given for accepting the consultancy. |
| --- | --- |

44.

There was not always clear synergy between the technical assistance and equipment provided under the projects. Nor was there any evidence that the equipment component had addressed the specific areas described (15) in the Regulation or Guidelines. The beneficiaries treated the equipment component (mostly office and IT equipment) more as an additional bonus given for accepting the consultancy.

| 45. | In many cases the equipment was delivered at the end of the project, due to the long procurement procedure, and therefore could not be used for the project activities. |
| --- | --- |

45.

In many cases the equipment was delivered at the end of the project, due to the long procurement procedure, and therefore could not be used for the project activities.

| 46. | The equipment needs maintenance (including the supply of consumables) after the project has ended. However, this maintenance did not always take place.



  Box 9

  In one case, the customs offices did not have any budget to buy consumables for the laboratory. Another laboratory remained mainly unused, as the beneficiary did not have adequate premises in which to use the equipment.
  One of the federal beneficiaries refused to accept IT equipment for its own bureaucratic reasons (as it did not pay for the equipment it did not know how to book it into the balance sheet).
  In another case some of the equipment was sold by the beneficiary because it did not know how to make use of it. Part of the equipment could not be used because the needed internet connection was not available. Some fitness equipment was not suited for the target group, children, but was used by soldiers from a local military academy. |
| --- | --- |

46.

The equipment needs maintenance (including the supply of consumables) after the project has ended. However, this maintenance did not always take place.

Box 9

  In one case, the customs offices did not have any budget to buy consumables for the laboratory. Another laboratory remained mainly unused, as the beneficiary did not have adequate premises in which to use the equipment.
  One of the federal beneficiaries refused to accept IT equipment for its own bureaucratic reasons (as it did not pay for the equipment it did not know how to book it into the balance sheet).
  In another case some of the equipment was sold by the beneficiary because it did not know how to make use of it. Part of the equipment could not be used because the needed internet connection was not available. Some fitness equipment was not suited for the target group, children, but was used by soldiers from a local military academy.

Purchases free of VAT from domestic suppliers impossible

| 47. | Contractors indicated problems in domestic procurement of equipment and services as well. According to the TACIS Regulation (16), taxes and duties are not eligible expenditure. Purchases free of VAT from domestic suppliers in the Russian Federation were impossible because the Russian secondary legislation does not allow it. This is not in line with the agreement between the EU and the Russian Federation (17). |
| --- | --- |

47.

Contractors indicated problems in domestic procurement of equipment and services as well. According to the TACIS Regulation (16), taxes and duties are not eligible expenditure. Purchases free of VAT from domestic suppliers in the Russian Federation were impossible because the Russian secondary legislation does not allow it. This is not in line with the agreement between the EU and the Russian Federation (17).

The monitors performed well

| 48. | An external consulting company was charged with the monitoring of the projects, during their lifetime. The consultants (monitors) form an opinion about design, efficiency, effectiveness, impact and sustainability of the projects. The financial side of the project is excluded. Also, assessment of the impact of the projects after the expiry of the projects' activities was and is not part of the contract between the Commission and the monitors. The lack of this assessment was not compensated for by evaluations. |
| --- | --- |

48.

An external consulting company was charged with the monitoring of the projects, during their lifetime. The consultants (monitors) form an opinion about design, efficiency, effectiveness, impact and sustainability of the projects. The financial side of the project is excluded. Also, assessment of the impact of the projects after the expiry of the projects' activities was and is not part of the contract between the Commission and the monitors. The lack of this assessment was not compensated for by evaluations.

| 49. | In general the monitors performed according to the requirements of their contracts. However, in a few cases the monitors' reports repeated statements expressing future intentions as if they were merits of the projects. Such reports did not reveal clearly enough the implementation problems, which may partly explain the lack of corrective actions. |
| --- | --- |

49.

In general the monitors performed according to the requirements of their contracts. However, in a few cases the monitors' reports repeated statements expressing future intentions as if they were merits of the projects. Such reports did not reveal clearly enough the implementation problems, which may partly explain the lack of corrective actions.

Dissemination, sustainability and evaluation

Poor dissemination

| 50. | Typical dissemination activities were seminars, a web-page and brochures. Even though some projects had some dissemination activities during project implementation, those ceased when the project activities finished. For example, the web pages created as part of the activities of six projects were not maintained by the beneficiary after the expiry of the project. To some degree, this also demonstrates the perceived value of the results achieved, as insufficient funds were devoted by the Commission and Russian side to the dissemination of the results. |
| --- | --- |

50.

Typical dissemination activities were seminars, a web-page and brochures. Even though some projects had some dissemination activities during project implementation, those ceased when the project activities finished. For example, the web pages created as part of the activities of six projects were not maintained by the beneficiary after the expiry of the project. To some degree, this also demonstrates the perceived value of the results achieved, as insufficient funds were devoted by the Commission and Russian side to the dissemination of the results.

| 51. | Nor had the Commission proper systems and instructions for effectively disseminating the results and lessons learned from the TACIS projects. For example, there is no consolidated, public database on TACIS projects. Nor are the Russian authorities fully aware of the results of TACIS projects. |
| --- | --- |

51.

Nor had the Commission proper systems and instructions for effectively disseminating the results and lessons learned from the TACIS projects. For example, there is no consolidated, public database on TACIS projects. Nor are the Russian authorities fully aware of the results of TACIS projects.

Poor sustainability

| 52. | As noted above, there were several problems in achieving the results of the projects and disseminating information about the results achieved. Furthermore, the beneficiaries have only rarely used or further developed the outputs such as reports and models. The results of only five of the projects audited were sustainable. |
| --- | --- |

52.

As noted above, there were several problems in achieving the results of the projects and disseminating information about the results achieved. Furthermore, the beneficiaries have only rarely used or further developed the outputs such as reports and models. The results of only five of the projects audited were sustainable.

Lack of evaluation

| 53. | The Commission carries out sector or countrywide evaluations, but targets rather seldomly individual projects. Only two of the 29 projects audited were evaluated. This is inadequate considering the intended pilot nature of many of the projects as well as the amount of money involved, and it is not at the level required to improve the design of future projects. |
| --- | --- |

53.

The Commission carries out sector or countrywide evaluations, but targets rather seldomly individual projects. Only two of the 29 projects audited were evaluated. This is inadequate considering the intended pilot nature of many of the projects as well as the amount of money involved, and it is not at the level required to improve the design of future projects.

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

| 54. | The Russian Federation is an important trading and political partner of the EU. Effective EU assistance aiming at creating conditions for democracy, rule of law and sound economic growth in the Russian Federation and to develop a true partnership between the EU and the Russian Federation can be an important and meaningful instrument. |
| --- | --- |

54.

The Russian Federation is an important trading and political partner of the EU. Effective EU assistance aiming at creating conditions for democracy, rule of law and sound economic growth in the Russian Federation and to develop a true partnership between the EU and the Russian Federation can be an important and meaningful instrument.

| 55. | About one third of the audited projects fully achieved their objectives. In eight cases the objectives were partially reached and in twelve cases they were not reached (see paragraph 18). The results of five of the projects audited were sustainable (see paragraph 52). Considering the above audit results, which show that the effectiveness of the use of TACIS funds has been low, the Court cannot assess the performance of TACIS projects in the Russian Federation positively. However, the audit revealed that the performance of contractors and monitors met the requirements of their contracts. The main failings are the following:






  (a) | for the purposes of planning and management of TACIS and other similar programmes the Commission has developed systems of programming and project cycle management. This system framework is aimed at ensuring effective use of the funds allocated to an activity although the need to prepare and approve the programming documents and the related consultations made the programming process long. Despite the exhaustive programming systems and the Commission's efforts, a real dialogue with the Russian authorities and beneficiaries often did not take place resulting in some cases where the Russian authorities accepted assistance, which the final beneficiaries did not actually want. Under these circumstances the documents had to be drafted at a very general level, not reflecting the real needs of the beneficiaries (see paragraphs 20 to 22); |
| --- | --- | --- |

55.

About one third of the audited projects fully achieved their objectives. In eight cases the objectives were partially reached and in twelve cases they were not reached (see paragraph 18). The results of five of the projects audited were sustainable (see paragraph 52). Considering the above audit results, which show that the effectiveness of the use of TACIS funds has been low, the Court cannot assess the performance of TACIS projects in the Russian Federation positively. However, the audit revealed that the performance of contractors and monitors met the requirements of their contracts. The main failings are the following:

(a)

for the purposes of planning and management of TACIS and other similar programmes the Commission has developed systems of programming and project cycle management. This system framework is aimed at ensuring effective use of the funds allocated to an activity although the need to prepare and approve the programming documents and the related consultations made the programming process long. Despite the exhaustive programming systems and the Commission's efforts, a real dialogue with the Russian authorities and beneficiaries often did not take place resulting in some cases where the Russian authorities accepted assistance, which the final beneficiaries did not actually want. Under these circumstances the documents had to be drafted at a very general level, not reflecting the real needs of the beneficiaries (see paragraphs 20 to 22);

| (b) | this was reflected in the design of many projects. Objectives were sometimes imprecise and not measurable and in some cases underlying assumptions were unrealistic. This resulted in ill-founded project budgets. Some apparently larger scale projects in fact comprised many different small projects with many different objectives and several beneficiaries. The selection of beneficiaries and the cooperation between various beneficiaries was problematic. During the implementation the beneficiaries often failed to meet their obligations to contribute in-kind (counterpart staff, office accommodation). A specific aspect illustrating the insufficient dialogue between the Commission and the Russian authorities is the impossibility of purchases free of VAT from domestic suppliers (see paragraphs 25 to 34); |
| --- | --- |

(b)

this was reflected in the design of many projects. Objectives were sometimes imprecise and not measurable and in some cases underlying assumptions were unrealistic. This resulted in ill-founded project budgets. Some apparently larger scale projects in fact comprised many different small projects with many different objectives and several beneficiaries. The selection of beneficiaries and the cooperation between various beneficiaries was problematic. During the implementation the beneficiaries often failed to meet their obligations to contribute in-kind (counterpart staff, office accommodation). A specific aspect illustrating the insufficient dialogue between the Commission and the Russian authorities is the impossibility of purchases free of VAT from domestic suppliers (see paragraphs 25 to 34);

| (c) | the potential sustainability was not given sufficient consideration by the Commission when planning the projects, and further, when a project achieved its results, they were not effectively disseminated. Thus the replication of activities and learning from the results remained limited (see paragraph 52). Co-financing was practically not used to get more ownership (see paragraph 37); |
| --- | --- |

(c)

the potential sustainability was not given sufficient consideration by the Commission when planning the projects, and further, when a project achieved its results, they were not effectively disseminated. Thus the replication of activities and learning from the results remained limited (see paragraph 52). Co-financing was practically not used to get more ownership (see paragraph 37);

| (d) | the assessment of the impact of the projects after the expiry of the projects' activities was not part of the monitors' duties and this was not compensated for by evaluations. The Commission could therefore not systematically learn from the implemented projects in order to develop more effective future interventions (see paragraphs 48 and 53). |
| --- | --- |

(d)

the assessment of the impact of the projects after the expiry of the projects' activities was not part of the monitors' duties and this was not compensated for by evaluations. The Commission could therefore not systematically learn from the implemented projects in order to develop more effective future interventions (see paragraphs 48 and 53).

Recommendations

| 56. | To facilitate improved effectiveness of the future assistance to the Russian Federation the Court makes the following recommendations.






  (a) | The Commission should find ways to shorten the programming process and base all planning of external action programmes on a real dialogue between the Commission and the recipient leading to a real commitment of the latter. |
| --- | --- | --- |

56.

To facilitate improved effectiveness of the future assistance to the Russian Federation the Court makes the following recommendations.

(a)

The Commission should find ways to shorten the programming process and base all planning of external action programmes on a real dialogue between the Commission and the recipient leading to a real commitment of the latter.

| (b) | Projects should only be financed where all the objectives are specific, measurable, realistic, timed and achievable, and are truly shared by the Commission and the recipient/beneficiary, and in which the steering committees, where appropriate, are made into an important management instrument through giving them the right powers and instituting membership at decision making level; and the investment components (if any) would be co-financed by the beneficiary; and realistic arrangements concerning VAT exemption and indirect in-kind co-financing are made. |
| --- | --- |

(b)

Projects should only be financed where all the objectives are specific, measurable, realistic, timed and achievable, and are truly shared by the Commission and the recipient/beneficiary, and in which the steering committees, where appropriate, are made into an important management instrument through giving them the right powers and instituting membership at decision making level; and the investment components (if any) would be co-financed by the beneficiary; and realistic arrangements concerning VAT exemption and indirect in-kind co-financing are made.

| (c) | The Court stresses the need for a stable organisation and for robust and rigorous action in line with the established rules and guidelines: the PCM including the logical framework should be applied whenever appropriate; the Commission should avoid putting small projects together in one larger-scale project with multiple objectives and different beneficiaries; the Commission should promptly react to shortcomings in project implementation; and the Commission should set benchmarks for the speed of its administrative procedures for approvals of and other reactions to the programming and contracting documents and reports. |
| --- | --- |

(c)

The Court stresses the need for a stable organisation and for robust and rigorous action in line with the established rules and guidelines: the PCM including the logical framework should be applied whenever appropriate; the Commission should avoid putting small projects together in one larger-scale project with multiple objectives and different beneficiaries; the Commission should promptly react to shortcomings in project implementation; and the Commission should set benchmarks for the speed of its administrative procedures for approvals of and other reactions to the programming and contracting documents and reports.

| (d) | In order to have better overview of the sustainability of the projects the Commission should appraise realistically the potential for sustainability during the planning phase of the projects; ensure the ex post assessment, or evaluation, of the completed projects in order to improve the future projects; and ensure that information on the projects results is made accessible to the public and actively look for the best practices achieved by its and other donors' projects. |
| --- | --- |

(d)

In order to have better overview of the sustainability of the projects the Commission should appraise realistically the potential for sustainability during the planning phase of the projects; ensure the ex post assessment, or evaluation, of the completed projects in order to improve the future projects; and ensure that information on the projects results is made accessible to the public and actively look for the best practices achieved by its and other donors' projects.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 9 March 2006.

For the Court of Auditors

Hubert WEBER

President

---

(1)  TACIS: Technical Assistance for the Commonwealth of Independent States.

1

(2)  1991-1995: (EEC, Euratom) No 2157/91 of 15 July 1991, 1996-1999: (Euratom, EC) No 1279/96 of 25 June 1996.

2

(3)  OJ L 12, 18.1.2000, p. 1.

3

(4)  The Committee is composed of representatives of the Member States and chaired by a representative of the Commission.

4

(5)  Communication to the Commission on the reform of the management of external assistance, 16 May 2000, SEC(2000) 814/5.

5

(6)  The contracts audited were selected using the method of Monetary Unit Sampling. As a consequence, the selection includes more contracts with big amounts than contracts with small amounts.

6

(7)  Special Report No 25/1998 concerning operations undertaken by the European Union in the field of nuclear safety in central and eastern Europe (CEEC) and in the new Independent States (NIS) (1990 to 1997). Follow up in 2000, the Court's Annual Report, paragraphs 5.45-5.76.

7

(8)  Special Report 11/2001 concerning the TACIS cross-border cooperation programme.

8

(9)  Bistro is a TACIS facility designed to respond quickly to requests for support for small scale projects (as a rule up to 100 000 euro).

9

(10)  The logical framework is a tool which makes explicit the underlying logic of the project design, i.e. how particular inputs should produce certain outputs which will help meet objectives which will contribute to wider goals.

10

(11)  OJ C 57, 22.2.2001, paragraph 37.

11

(12)  OJ C 72, 22.3.2005.

12

(13)  Article 3.

13

(14)  Article 6.

14

(15)  Cross-border cooperation, promotion of Small and Medium Enterprises (SMEs), environmental infrastructure and networks of strategic importance to the Community.

15

(16)  Article 9 of Regulation (EC, Euratom) No 99/2000.

16

(17)  General Rules to the technical Assistance of the European Communities signed by the European Commission and the Government of the Russian Federation on 18 July 1997.

17

---

THE COMMISSION'S REPLIES

SUMMARY

| I. | The European Neighbourhood and Partnership Instrument, proposed by the Commission in September 2004, which will govern the future cooperation with Russia, is currently under consideration in the Council and the European Parliament. A Country Strategy Paper and a National Indicative Programme covering the period 2007 to 2010 will be put to Member States for consideration when appropriate.
  The Strategy will guide the European Commission in the implementation of financial cooperation at levels decided by the Council. |
| --- | --- |

I.

The European Neighbourhood and Partnership Instrument, proposed by the Commission in September 2004, which will govern the future cooperation with Russia, is currently under consideration in the Council and the European Parliament. A Country Strategy Paper and a National Indicative Programme covering the period 2007 to 2010 will be put to Member States for consideration when appropriate.

The Strategy will guide the European Commission in the implementation of financial cooperation at levels decided by the Council.

| II. | The financial funds for Russia have been on a decreasing trend. Russia has achieved much economic progress in recent years and the capacity of the Russian authorities to finance reform measures itself has grown significantly at a time when the country has a budget surplus and is repaying its international debts. The overall funding level, including also horizontal thematic budget headings, fell to EUR 150 million in 2005 and is expected to decrease further to around EUR 120 to 130 million in 2006. |
| --- | --- |

II.

The financial funds for Russia have been on a decreasing trend. Russia has achieved much economic progress in recent years and the capacity of the Russian authorities to finance reform measures itself has grown significantly at a time when the country has a budget surplus and is repaying its international debts. The overall funding level, including also horizontal thematic budget headings, fell to EUR 150 million in 2005 and is expected to decrease further to around EUR 120 to 130 million in 2006.

| IV. | The Commission has itself recognised these weaknesses, which arose partly as a result of shortcomings in the project management, but also as a result of lack of involvement and ownership by the Russian beneficiaries. Moreover, in some cases the impact of the projects was hampered by the diversity of interest within the Russian society, materialised in diverging interest between the various authorities.
  A majority of the projects achieved their objectives fully or partially, and despite the low effectiveness of some of the projects the Commission believes that its programmes of assistance have helped to broaden the general level of cooperation between the EU and the Russian Government.
  Nevertheless, the Commission will draw the lessons of experience, and of the new economic situation prevailing in Russia, when preparing its new country strategy. |
| --- | --- |

IV.

The Commission has itself recognised these weaknesses, which arose partly as a result of shortcomings in the project management, but also as a result of lack of involvement and ownership by the Russian beneficiaries. Moreover, in some cases the impact of the projects was hampered by the diversity of interest within the Russian society, materialised in diverging interest between the various authorities.

A majority of the projects achieved their objectives fully or partially, and despite the low effectiveness of some of the projects the Commission believes that its programmes of assistance have helped to broaden the general level of cooperation between the EU and the Russian Government.

Nevertheless, the Commission will draw the lessons of experience, and of the new economic situation prevailing in Russia, when preparing its new country strategy.

| V. | A constructive political environment is vital for the successful implementation of financial cooperation. In the context of assistance to Russia, an occasional lack of such an environment has from time to time had a negative impact on programming and implementation. Moreover, the programming was affected by the financial crisis in Russia at the end of the 1990s, when general disintegration made dialogue difficult, thereby diminishing local ownership.
  The Commission has already addressed the weaknesses in the management of the past programmes by introducing the Reform of External Assistance (the RELEX reform), notably by means of the devolution of EC external aid management to the Commission Delegations. The devolution ensures a greater involvement of the partner country from the preparatory phase to the final implementation, and it also strengthens the Commission involvement in donor coordination on the spot. |
| --- | --- |

V.

A constructive political environment is vital for the successful implementation of financial cooperation. In the context of assistance to Russia, an occasional lack of such an environment has from time to time had a negative impact on programming and implementation. Moreover, the programming was affected by the financial crisis in Russia at the end of the 1990s, when general disintegration made dialogue difficult, thereby diminishing local ownership.

The Commission has already addressed the weaknesses in the management of the past programmes by introducing the Reform of External Assistance (the RELEX reform), notably by means of the devolution of EC external aid management to the Commission Delegations. The devolution ensures a greater involvement of the partner country from the preparatory phase to the final implementation, and it also strengthens the Commission involvement in donor coordination on the spot.

| VI. | Since the design of these projects in 1997 to 2000, significant improvements have been introduced, including the development of clear and measurable objectives for actions envisaged. The Court's observations also relate to the difficult environment in Russia at the time. The 2005 Monitoring Report of Tacis projects revealed a significant increase in quality of projects in terms of relevance, efficiency, effectiveness, impact and sustainability. Moreover, the Commission systematically carries out sector and country-wide evaluations.
  Whilst sharing the Court's analysis that larger-scale projects should not be sought by pooling together a variety of small projects with multiple objectives and many different beneficiaries, the Commission still intends to move towards larger-scale projects in the TACIS region as a whole. The Commission also intends examining the feasibility of new implementing tools to improve the cooperation programme with Russia when the TACIS Regulation expires and a new instrument will be introduced.
  Through building on the Common Space political dialogue with Russia the aim is to maximise ownership by the partner country. |
| --- | --- |

VI.

Since the design of these projects in 1997 to 2000, significant improvements have been introduced, including the development of clear and measurable objectives for actions envisaged. The Court's observations also relate to the difficult environment in Russia at the time. The 2005 Monitoring Report of Tacis projects revealed a significant increase in quality of projects in terms of relevance, efficiency, effectiveness, impact and sustainability. Moreover, the Commission systematically carries out sector and country-wide evaluations.

Whilst sharing the Court's analysis that larger-scale projects should not be sought by pooling together a variety of small projects with multiple objectives and many different beneficiaries, the Commission still intends to move towards larger-scale projects in the TACIS region as a whole. The Commission also intends examining the feasibility of new implementing tools to improve the cooperation programme with Russia when the TACIS Regulation expires and a new instrument will be introduced.

Through building on the Common Space political dialogue with Russia the aim is to maximise ownership by the partner country.

| VII. | (first indent) The Commission intends to base future cooperation on the Common Spaces agreed between Russia and the European Union, and will support initiatives emanating from the roadmaps; financial cooperation will seek to meet policy objectives already agreed between the two sides.
  However, despite a generally improved political environment following agreement on the joint Common Spaces policy in recent years, engaging the large Russian bureaucracy in sustained dialogue over the strategy for future financial cooperation is not straightforward. Furthermore, the Russian authorities remain hesitant about the form of the future cooperation. Therefore there will be even more emphasis in the future on funding actions on which both sides agree. The Commission will be prepared to use financial cooperation to drive forward the EU-Russia policy agenda.
  (second indent) The Commission made PCM and use of logical framework obligatory as of 2001. Amongst others the External Assistance Reform has also strengthened the quality of the projects, i.e. by introducing systematic review of projects/programmes at identification and formulation stages by Quality Support Groups, as well as by systematic use of result-oriented monitoring. On project level also systematic use of logical framework with objectively verifiable indicators is put in place.
  (third indent) The Commission systematically carries out sector or countrywide evaluations. It will consider increasing the number of project-based evaluations. Best practice is to publish all evaluations, including project evaluations.
  There is already a database of results-oriented monitoring of ongoing projects, used as a management tool by the Commission, which allows learning from good and bad practices. Results are also shared in general with the beneficiary country. |
| --- | --- |

VII.

(first indent) The Commission intends to base future cooperation on the Common Spaces agreed between Russia and the European Union, and will support initiatives emanating from the roadmaps; financial cooperation will seek to meet policy objectives already agreed between the two sides.

However, despite a generally improved political environment following agreement on the joint Common Spaces policy in recent years, engaging the large Russian bureaucracy in sustained dialogue over the strategy for future financial cooperation is not straightforward. Furthermore, the Russian authorities remain hesitant about the form of the future cooperation. Therefore there will be even more emphasis in the future on funding actions on which both sides agree. The Commission will be prepared to use financial cooperation to drive forward the EU-Russia policy agenda.

(second indent) The Commission made PCM and use of logical framework obligatory as of 2001. Amongst others the External Assistance Reform has also strengthened the quality of the projects, i.e. by introducing systematic review of projects/programmes at identification and formulation stages by Quality Support Groups, as well as by systematic use of result-oriented monitoring. On project level also systematic use of logical framework with objectively verifiable indicators is put in place.

(third indent) The Commission systematically carries out sector or countrywide evaluations. It will consider increasing the number of project-based evaluations. Best practice is to publish all evaluations, including project evaluations.

There is already a database of results-oriented monitoring of ongoing projects, used as a management tool by the Commission, which allows learning from good and bad practices. Results are also shared in general with the beneficiary country.

INTRODUCTION

| 2. | The European Neighbourhood and Partnership Instrument, proposed by the Commission in September 2004, which will govern the future cooperation with Russia, is currently under consideration in the Council and the European Parliament. A Country Strategy Paper and National Indicative Programme 2007 to 2010 will be put to Member States for consideration when appropriate. |
| --- | --- |

2.

The European Neighbourhood and Partnership Instrument, proposed by the Commission in September 2004, which will govern the future cooperation with Russia, is currently under consideration in the Council and the European Parliament. A Country Strategy Paper and National Indicative Programme 2007 to 2010 will be put to Member States for consideration when appropriate.

| 9. | Financial cooperation with Russia has been on a decreasing trend, as continued high commodity prices have substantially improved Russia's financial position. The overall funding level fell to EUR 150 million in 2005 and is expected to decrease further to around EUR 120-130 million in 2006. |
| --- | --- |

9.

Financial cooperation with Russia has been on a decreasing trend, as continued high commodity prices have substantially improved Russia's financial position. The overall funding level fell to EUR 150 million in 2005 and is expected to decrease further to around EUR 120-130 million in 2006.

| 10. | EU policy towards Russia is now expressed by the four Common Spaces (Common Economic Space, Common Spaces of Justice, Liberty and Security, Common Space of External Security, Common Space of Research and Education, including Culture) — with detailed policy commitments set out in the accompanying roadmaps (agreed in May 2005). |
| --- | --- |

10.

EU policy towards Russia is now expressed by the four Common Spaces (Common Economic Space, Common Spaces of Justice, Liberty and Security, Common Space of External Security, Common Space of Research and Education, including Culture) — with detailed policy commitments set out in the accompanying roadmaps (agreed in May 2005).

OBSERVATIONS

| 17. | The projects examined do not reflect the improvements in project design procedures introduced since the reform of the EC external assistance, which has taken place over the 2002 to 2005 period, in particular the devolution of responsibilities to Delegations. This has and will continue to improve the implementation and monitoring of bilateral projects on the ground. |
| --- | --- |

17.

The projects examined do not reflect the improvements in project design procedures introduced since the reform of the EC external assistance, which has taken place over the 2002 to 2005 period, in particular the devolution of responsibilities to Delegations. This has and will continue to improve the implementation and monitoring of bilateral projects on the ground.

| 18. | The Commission recognises certain weaknesses in project implementation at that time. They have been addressed through the Reform of External Assistance.

  Box 1 — See reply to paragraph 18. |
| --- | --- |

18.

The Commission recognises certain weaknesses in project implementation at that time. They have been addressed through the Reform of External Assistance.

Box 1 — See reply to paragraph 18.

| 21. | A distinction should be made between the two types of consultations: internal consultations within the Commission take into account the need to consult, in the case of an important partner such as Russia, most if not all DGs and services. The consultation procedure with Member States takes time, but this is a regulatory obligation. External consultations with the Russian side are important to ensure dialogue and a real sense of ownership. Internal and external consultations in any case proceed, to the greatest extent possible, in parallel.
  In some cases the Russian authorities have subsequently delayed the signature of Action Programmes, therefore delaying the implementation of the projects themselves. |
| --- | --- |

21.

A distinction should be made between the two types of consultations: internal consultations within the Commission take into account the need to consult, in the case of an important partner such as Russia, most if not all DGs and services. The consultation procedure with Member States takes time, but this is a regulatory obligation. External consultations with the Russian side are important to ensure dialogue and a real sense of ownership. Internal and external consultations in any case proceed, to the greatest extent possible, in parallel.

In some cases the Russian authorities have subsequently delayed the signature of Action Programmes, therefore delaying the implementation of the projects themselves.

| 22. | The Reform of External Assistance (the RELEX reform), notably by means of the devolution of EC external aid management to the Commission Delegations, aims at ensuring a greater involvement of the partner country from the preparatory phase to the final implementation, and it also strengthens the Commission involvement in donor coordination on the spot.
  Attempts have been made, by the Delegation in particular but also by RELEX Headquarters during a recent programming mission (October 2005), to engage the Russian Government in dialogue over the future strategy, which is firmly based on the Common Spaces, and at least the Commission has formal Russian agreement to these (since May 2005).
  The dialogue has improved with the Russian authorities since devolution. Regular dialogue is maintained with the National Coordination Unit (NCU). However, contacts could be developed further; they are currently limited by the lack of its prioritisation by the Russian authorities illustrated by only up to four people working on the Tacis programme in the NCU. The Russian authorities remain hesitant about the form of the future cooperation. |
| --- | --- |

22.

The Reform of External Assistance (the RELEX reform), notably by means of the devolution of EC external aid management to the Commission Delegations, aims at ensuring a greater involvement of the partner country from the preparatory phase to the final implementation, and it also strengthens the Commission involvement in donor coordination on the spot.

Attempts have been made, by the Delegation in particular but also by RELEX Headquarters during a recent programming mission (October 2005), to engage the Russian Government in dialogue over the future strategy, which is firmly based on the Common Spaces, and at least the Commission has formal Russian agreement to these (since May 2005).

The dialogue has improved with the Russian authorities since devolution. Regular dialogue is maintained with the National Coordination Unit (NCU). However, contacts could be developed further; they are currently limited by the lack of its prioritisation by the Russian authorities illustrated by only up to four people working on the Tacis programme in the NCU. The Russian authorities remain hesitant about the form of the future cooperation.

| 24. | Since the reform of EC external assistance, all aspects of Project Cycle Management (PCM), including the use of the logical frameworks, are fully institutionalised. The use of the logical framework approach provides objectively verifiable indicators and predefined means to assess the effectiveness of the expenditure. Independent and systematic results-oriented monitoring, as well as mandatory mid-term and final evaluations, are then used to verify this. Moreover, in order to improve the design of individual operations, EuropeAid Office Quality Support Groups (oQSGs) review projects and programmes twice before the financing decision is taken — namely, at identification and formulation stages. Quality of projects/programmes is assessed by the means of checklists based on a Quality Frame which is coherent with OECD/DAC criteria (the Quality Frame is a system based on three quality attributes — relevance, feasibility, and effectiveness and good management — and 12 criteria indicating the key issues to be assessed in order to make a judgement about quality). In addition, the Delegation in Russia has established a system of ‘peer group perusal’ to review project outlines and terms of reference before they are approved. |
| --- | --- |

24.

Since the reform of EC external assistance, all aspects of Project Cycle Management (PCM), including the use of the logical frameworks, are fully institutionalised. The use of the logical framework approach provides objectively verifiable indicators and predefined means to assess the effectiveness of the expenditure. Independent and systematic results-oriented monitoring, as well as mandatory mid-term and final evaluations, are then used to verify this. Moreover, in order to improve the design of individual operations, EuropeAid Office Quality Support Groups (oQSGs) review projects and programmes twice before the financing decision is taken — namely, at identification and formulation stages. Quality of projects/programmes is assessed by the means of checklists based on a Quality Frame which is coherent with OECD/DAC criteria (the Quality Frame is a system based on three quality attributes — relevance, feasibility, and effectiveness and good management — and 12 criteria indicating the key issues to be assessed in order to make a judgement about quality). In addition, the Delegation in Russia has established a system of ‘peer group perusal’ to review project outlines and terms of reference before they are approved.

| 25. | As a result of this reform, including the establishment of EuropeAid (January 2001) and the devolution of aid management to the Commission Delegations, together with the New Financial Regulation (January 2003), project delays will be limited and contained.
  The Policy Advice Programme has been introduced to ensure a rapid response to Russian policy advice requests. In addition, new tools have been introduced (e.g. General Technical Assistance Facility, Accompanying Measures, etc.) enabling also swift replies where there is an urgent need. |
| --- | --- |

25.

As a result of this reform, including the establishment of EuropeAid (January 2001) and the devolution of aid management to the Commission Delegations, together with the New Financial Regulation (January 2003), project delays will be limited and contained.

The Policy Advice Programme has been introduced to ensure a rapid response to Russian policy advice requests. In addition, new tools have been introduced (e.g. General Technical Assistance Facility, Accompanying Measures, etc.) enabling also swift replies where there is an urgent need.

| 26. | In the meantime devolution has led to a significant increase in staff in Delegations, which should facilitate the execution of background studies as well as investigations of the current (though quickly changing) legal environment. The increasing role of the Delegation should also contribute to avoiding problems of the kind mentioned in boxes 3, 4 and 5. |
| --- | --- |

26.

In the meantime devolution has led to a significant increase in staff in Delegations, which should facilitate the execution of background studies as well as investigations of the current (though quickly changing) legal environment. The increasing role of the Delegation should also contribute to avoiding problems of the kind mentioned in boxes 3, 4 and 5.

| 27. | Harmonised programming guidelines have been developed since 2000 and programming documents are now systematically screened for quality and consistency. As regards the Country Strategy Paper 2007 to 2013 a consultative mission took place in October 2005 to ensure the participation of the Russian authorities in the programming process. See also reply to paragraph 21. |
| --- | --- |

27.

Harmonised programming guidelines have been developed since 2000 and programming documents are now systematically screened for quality and consistency. As regards the Country Strategy Paper 2007 to 2013 a consultative mission took place in October 2005 to ensure the participation of the Russian authorities in the programming process. See also reply to paragraph 21.

| 28. | See reply to paragraph 26. |
| --- | --- |

28.

See reply to paragraph 26.

| 29. | Since 2000 the Commission has put much effort into developing clear and measurable objectives for action envisaged. The newly established internal quality support process at the headquarter focuses, among others, on the appropriateness of objectives. |
| --- | --- |

29.

Since 2000 the Commission has put much effort into developing clear and measurable objectives for action envisaged. The newly established internal quality support process at the headquarter focuses, among others, on the appropriateness of objectives.

| 30 and 31. | Project design has considerably improved through the reform of EC external assistance as outlined in the Commission reply to paragraph 24.

  Box 6 — The Commission confirms that the project was complex. However, the project results gave start to activities in the field of sustainable and environmentally friendly use of energy resources. |
| --- | --- |

30 and 31.

Project design has considerably improved through the reform of EC external assistance as outlined in the Commission reply to paragraph 24.

Box 6 — The Commission confirms that the project was complex. However, the project results gave start to activities in the field of sustainable and environmentally friendly use of energy resources.

| 32. | It is the Russian authorities that propose the beneficiaries. This process is at times complicated due to Russian internal coordination difficulties and changes in government structures. |
| --- | --- |

32.

It is the Russian authorities that propose the beneficiaries. This process is at times complicated due to Russian internal coordination difficulties and changes in government structures.

| 33. | Lack of cooperation between the federal and regional authorities or lack of cooperation between different federal and between different regional authorities is a constant feature of the Russian political environment. The Delegation has nevertheless made every effort to ensure the effective implementation of planned projects. |
| --- | --- |

33.

Lack of cooperation between the federal and regional authorities or lack of cooperation between different federal and between different regional authorities is a constant feature of the Russian political environment. The Delegation has nevertheless made every effort to ensure the effective implementation of planned projects.

| 34. | In all cases beneficiaries must sign Statements of Endorsement for Terms of Reference, so they always have the chance to propose changes to the design of the project. |
| --- | --- |

34.

In all cases beneficiaries must sign Statements of Endorsement for Terms of Reference, so they always have the chance to propose changes to the design of the project.

| 35. | A consensus will have to be reached, for the period from 2007, to avoid excessive dispersion of modest funding and to concentrate and maintain financial cooperation on a limited number of actions. |
| --- | --- |

35.

A consensus will have to be reached, for the period from 2007, to avoid excessive dispersion of modest funding and to concentrate and maintain financial cooperation on a limited number of actions.

| 36. | The quality of project design has considerably improved as explained under paragraph 24. The newly establish internal quality support process at the headquarter focuses, among others, on the appropriateness of objectives. |
| --- | --- |

36.

The quality of project design has considerably improved as explained under paragraph 24. The newly establish internal quality support process at the headquarter focuses, among others, on the appropriateness of objectives.

| 37. | Co-financing is included in the current dialogue with the Russian authorities. It is a complicated issue and can only be evolved at a gradual pace. |
| --- | --- |

37.

Co-financing is included in the current dialogue with the Russian authorities. It is a complicated issue and can only be evolved at a gradual pace.

| 38. | Inclusion of office premises within Terms of References only happens as a result of discussions with the Russian beneficiary. This is part of the discussion on asking the beneficiary to show material commitment to the project. Normally the beneficiaries (including ministries) commit themselves to providing accomodation (regardless of whether this is inside or outside a specific ministry). For the future, the Commission intends to give a number of specific suggestions for ways in which the Russian side may demonstrate commitment on their side. For example, the authorities should facilitate the implementation of actions funded. Russian commitment could be further engaged through requests to provide co-financing or in-kind contributions where appropriate. |
| --- | --- |

38.

Inclusion of office premises within Terms of References only happens as a result of discussions with the Russian beneficiary. This is part of the discussion on asking the beneficiary to show material commitment to the project. Normally the beneficiaries (including ministries) commit themselves to providing accomodation (regardless of whether this is inside or outside a specific ministry). For the future, the Commission intends to give a number of specific suggestions for ways in which the Russian side may demonstrate commitment on their side. For example, the authorities should facilitate the implementation of actions funded. Russian commitment could be further engaged through requests to provide co-financing or in-kind contributions where appropriate.

| 39. | As outlined in paragraph 24, the quality of project design has considerably improved in the context of the reform of EC external assistance.
  As far as the speed of implementation is concerned there are serious delays with regard to contracting due to the lengthy signing procedure by the Russian authorities. As regards the time period between contracting and finalisation, projects are in general implemented as planned. |
| --- | --- |

39.

As outlined in paragraph 24, the quality of project design has considerably improved in the context of the reform of EC external assistance.

As far as the speed of implementation is concerned there are serious delays with regard to contracting due to the lengthy signing procedure by the Russian authorities. As regards the time period between contracting and finalisation, projects are in general implemented as planned.

| 40. | The ‘Qualitative Assessment of the Reform of External Assistance’ of July 2005 (1) confirms that assistance is now delivered with more impact and is managed more efficiently. Improvements are still needed and continue to be made. |
| --- | --- |

40.

The ‘Qualitative Assessment of the Reform of External Assistance’ of July 2005 (1) confirms that assistance is now delivered with more impact and is managed more efficiently. Improvements are still needed and continue to be made.

| 41. | Indeed the Commission pays much attention to the process of selecting (and managing) the best consultants. |
| --- | --- |

41.

Indeed the Commission pays much attention to the process of selecting (and managing) the best consultants.

| 42. | The Commission believes that Steering Committees are an important tool in the guidance of projects. There might be some shortcomings in the functioning of steering committees, given that interests of its members may diverge. However, overall the advantages outweigh the drawbacks. |
| --- | --- |

42.

The Commission believes that Steering Committees are an important tool in the guidance of projects. There might be some shortcomings in the functioning of steering committees, given that interests of its members may diverge. However, overall the advantages outweigh the drawbacks.

| 43. | Since the introduction of the new Financial Regulation, all contracting of supplies has been done in-house in the EC Delegation. |
| --- | --- |

43.

Since the introduction of the new Financial Regulation, all contracting of supplies has been done in-house in the EC Delegation.

| 44. | The EC Delegation attempts where possible to minimise the inclusion of the purchase of equipment in projects. However at times there may be a strong need expressed by the beneficiary in which case the Commission negotiates. |
| --- | --- |

44.

The EC Delegation attempts where possible to minimise the inclusion of the purchase of equipment in projects. However at times there may be a strong need expressed by the beneficiary in which case the Commission negotiates.

| 45. | The Commission has to make every effort to ensure the transparancy and regularity in the application of the procurement procedures. In addition Russian VAT and Customs procedures also cause additional delays. |
| --- | --- |

45.

The Commission has to make every effort to ensure the transparancy and regularity in the application of the procurement procedures. In addition Russian VAT and Customs procedures also cause additional delays.

| 46. | Office and IT equipment (see paragraph 44) is standard; beneficiaries are expected to maintain it. |
| --- | --- |

46.

Office and IT equipment (see paragraph 44) is standard; beneficiaries are expected to maintain it.

| 47. | The legislation is already in force at federal level (General Rules are compulsory, the Taxation law refers to the humanitarian aid/technical assistance), but there is a lack of implementation mechanisms (and willingness) at local level. The regional services of the tax department do not have the internal instructions needed to apply the international agreements and to accept the exemption of the VAT.
  Thus, in practice, although the necessary legal base exists and the projects are all reported to the relevant Commission (2), it is technically not possible to obtain the exemption at the source, nor the refunding of the VAT on the goods and the services acquired in Russia.
  This problem is regularly highlighted at the political level, unsuccessfully as of today. |
| --- | --- |

47.

The legislation is already in force at federal level (General Rules are compulsory, the Taxation law refers to the humanitarian aid/technical assistance), but there is a lack of implementation mechanisms (and willingness) at local level. The regional services of the tax department do not have the internal instructions needed to apply the international agreements and to accept the exemption of the VAT.

Thus, in practice, although the necessary legal base exists and the projects are all reported to the relevant Commission (2), it is technically not possible to obtain the exemption at the source, nor the refunding of the VAT on the goods and the services acquired in Russia.

This problem is regularly highlighted at the political level, unsuccessfully as of today.

| 48. | The results-oriented monitoring programme is deemed to be an important contribution to the project management work and it has been extended to other areas (outside of Tacis). See also paragraph 53. |
| --- | --- |

48.

The results-oriented monitoring programme is deemed to be an important contribution to the project management work and it has been extended to other areas (outside of Tacis). See also paragraph 53.

| 49. | The 2005 Monitoring Report of Tacis projects revealed a significant increase in quality of all four components (relevance, efficiency, effectiveness, impact and sustainability) monitored for projects that were already monitored in 2004. This proves that corrective action has been taken. |
| --- | --- |

49.

The 2005 Monitoring Report of Tacis projects revealed a significant increase in quality of all four components (relevance, efficiency, effectiveness, impact and sustainability) monitored for projects that were already monitored in 2004. This proves that corrective action has been taken.

| 51. | Within the Commission, lessons learnt are systematically taken into account in the establishment of each project fiche for Tacis projects. The establishment of the Directorate ‘Quality Support’ also fosters the build-up of institutional knowledge on results of previous projects. |
| --- | --- |

51.

Within the Commission, lessons learnt are systematically taken into account in the establishment of each project fiche for Tacis projects. The establishment of the Directorate ‘Quality Support’ also fosters the build-up of institutional knowledge on results of previous projects.

| 52. | Monitoring and evaluation of projects has been enhanced by obligatory use of the logical framework tool. This will help to further improve sustainability. |
| --- | --- |

52.

Monitoring and evaluation of projects has been enhanced by obligatory use of the logical framework tool. This will help to further improve sustainability.

| 53. | The Commission will consider increasing the number of project-based evaluations. |
| --- | --- |

53.

The Commission will consider increasing the number of project-based evaluations.

CONCLUSIONS AND RECOMMENDATIONS

| 54. | For the future, the Commission intends to stress the potential of successful financial cooperation in helping to consolidate the EU-Russian partnership based on the Common Spaces agreed in 2003. |
| --- | --- |

54.

For the future, the Commission intends to stress the potential of successful financial cooperation in helping to consolidate the EU-Russian partnership based on the Common Spaces agreed in 2003.

| 55. | The Commission has itself recognised these weaknesses, which arose, as indicated by the Court, partly as a result of shortcomings in various phases of the project management cycle and partly as a result of lack of involvement and ownership by the Russian beneficiaries. Moreover, in some cases the impact of the projects was hampered by the diversity of interest within the Russian society, materialised in conflicting interest between the various authorities.
  A majority of the projects achieved their objectives fully or partially, and despite the low effectiveness of some of the projects the Commission believes that its programmes of assistance have helped to broaden the general level of cooperation between the EU and the Russian Government.
  Nevertheless, the Commission will draw the lessons of experience, and of the new economic situation prevailing in Russia, when preparing its new Country strategy.






  (a) | The Commission intends to use the new generation Country Strategy Paper 2007 to 2013 and accompanying National Indicative Programme 2007 to 2010 to set out three bold priority areas for financial cooperation, largely based on the roadmaps to the four Common Spaces, officially agreed with the Russian Federation in May 2005. The priority areas give rise to a number of possible specific objectives and a series of examples of possible actions arising from the roadmaps are also enumerated in the annex to the new National Indicative Programme. The new CSP and NIP should therefore give rather detailed guidance while leaving room for dialogue. |
| --- | --- | --- |

55.

The Commission has itself recognised these weaknesses, which arose, as indicated by the Court, partly as a result of shortcomings in various phases of the project management cycle and partly as a result of lack of involvement and ownership by the Russian beneficiaries. Moreover, in some cases the impact of the projects was hampered by the diversity of interest within the Russian society, materialised in conflicting interest between the various authorities.

A majority of the projects achieved their objectives fully or partially, and despite the low effectiveness of some of the projects the Commission believes that its programmes of assistance have helped to broaden the general level of cooperation between the EU and the Russian Government.

Nevertheless, the Commission will draw the lessons of experience, and of the new economic situation prevailing in Russia, when preparing its new Country strategy.

(a)

The Commission intends to use the new generation Country Strategy Paper 2007 to 2013 and accompanying National Indicative Programme 2007 to 2010 to set out three bold priority areas for financial cooperation, largely based on the roadmaps to the four Common Spaces, officially agreed with the Russian Federation in May 2005. The priority areas give rise to a number of possible specific objectives and a series of examples of possible actions arising from the roadmaps are also enumerated in the annex to the new National Indicative Programme. The new CSP and NIP should therefore give rather detailed guidance while leaving room for dialogue.

| (b) | All aspects of the project cycle management have been improved since 2001 in the context of the reform of EC external assistance. Moreover, as part of the reform, the devolution of the management of aid to the Commission Delegation ensures a greater involvement of the beneficiaries in the preparation and planning of the projects.
  The dialogue with the Russian authorities has sometimes been difficult in the past. There has also been intensive dialogue on the VAT issue. However no effective solution has been found by the Russian government so far. |
| --- | --- |

(b)

All aspects of the project cycle management have been improved since 2001 in the context of the reform of EC external assistance. Moreover, as part of the reform, the devolution of the management of aid to the Commission Delegation ensures a greater involvement of the beneficiaries in the preparation and planning of the projects.

The dialogue with the Russian authorities has sometimes been difficult in the past. There has also been intensive dialogue on the VAT issue. However no effective solution has been found by the Russian government so far.

| (c) | The sustainability of projects should be improved in the meantime, as monitoring and evaluation of projects has been enhanced by obligatory use of the logical framework tool. Co-financing is included in the current dialogue with the Russian authorities and its importance is witnessed through its mentioning in the Country Strategy Paper for Russia for 2007 to 2013. |
| --- | --- |

(c)

The sustainability of projects should be improved in the meantime, as monitoring and evaluation of projects has been enhanced by obligatory use of the logical framework tool. Co-financing is included in the current dialogue with the Russian authorities and its importance is witnessed through its mentioning in the Country Strategy Paper for Russia for 2007 to 2013.

| (d) | The Commission systematically carries out sector and country-wide evaluations (see paragraph 53). Furthermore, lessons learnt are systematically taken into account in the establishment of each project fiche for Tacis projects. Finally, the establishment of the Directorate ‘Quality Support’ also foster the build-up of institutional knowledge on results of previous projects. |
| --- | --- |

(d)

The Commission systematically carries out sector and country-wide evaluations (see paragraph 53). Furthermore, lessons learnt are systematically taken into account in the establishment of each project fiche for Tacis projects. Finally, the establishment of the Directorate ‘Quality Support’ also foster the build-up of institutional knowledge on results of previous projects.

| 56. | (a) | The Commission would like to shorten the programming process but without compromising external dialogue and prejudicing a sense of ownership. Dialogue is important, and engaging the large Russian bureaucracy in sustained dialogue is not straightforward, despite a generally improved political environment following agreement on the joint Common Spaces policy.
  A programming mission did take place in October 2005 and the Delegation is in constant dialogue with its Russian counterparts.
  The regulatory obligation to consult Member States takes time. The Commission will endeavour to finalise the National Indicative Programme 2007 to 2010 sufficiently early to allow timely project preparation for the 2007 action programme. |
| --- | --- | --- |

56.

(a)

The Commission would like to shorten the programming process but without compromising external dialogue and prejudicing a sense of ownership. Dialogue is important, and engaging the large Russian bureaucracy in sustained dialogue is not straightforward, despite a generally improved political environment following agreement on the joint Common Spaces policy.

A programming mission did take place in October 2005 and the Delegation is in constant dialogue with its Russian counterparts.

The regulatory obligation to consult Member States takes time. The Commission will endeavour to finalise the National Indicative Programme 2007 to 2010 sufficiently early to allow timely project preparation for the 2007 action programme.

| (b) | The Commission shares the assessment that projects should only be financed when their objectives are ‘smart’ (specific, measurable, achievable, realistic, timed) and ownership of the beneficiary exists. Co-financing is already an important element with the Russian authorities. |
| --- | --- |

(b)

The Commission shares the assessment that projects should only be financed when their objectives are ‘smart’ (specific, measurable, achievable, realistic, timed) and ownership of the beneficiary exists. Co-financing is already an important element with the Russian authorities.

| (c) | Amongst others the External Assistance reform has also strengthened the quality of the projects, i.e. by introducing systematic review of projects/programmes at identification and formulation stages by Quality Support Groups, as well as by systematic use of result-oriented monitoring. On project level also systematic use of logical frameworks with objectively verifiable indicators is put in place. PCM and logical framework have been systematically used by the Commission since 2001.
  Since devolution, due to the Commission's presence on the ground, monitoring and consequently reacting to shortcomings in project implementation can be carried out more easily and efficiently compared to the period under investigation by the Court. |
| --- | --- |

(c)

Amongst others the External Assistance reform has also strengthened the quality of the projects, i.e. by introducing systematic review of projects/programmes at identification and formulation stages by Quality Support Groups, as well as by systematic use of result-oriented monitoring. On project level also systematic use of logical frameworks with objectively verifiable indicators is put in place. PCM and logical framework have been systematically used by the Commission since 2001.

Since devolution, due to the Commission's presence on the ground, monitoring and consequently reacting to shortcomings in project implementation can be carried out more easily and efficiently compared to the period under investigation by the Court.

| (d) | As for sustainability please see response above (c). To date, The Commission systematically carries out sector or countrywide evaluations. It will consider increasing the number of project-based evaluations (paragraph 53). The Delegation in Moscow also puts a lot of effort into donor coordination and, consequently, tries to implement best practices wherever appropriate and possible. |
| --- | --- |

(d)

As for sustainability please see response above (c). To date, The Commission systematically carries out sector or countrywide evaluations. It will consider increasing the number of project-based evaluations (paragraph 53). The Delegation in Moscow also puts a lot of effort into donor coordination and, consequently, tries to implement best practices wherever appropriate and possible.

---

(1)  Commission staff working paper ‘Qualitative Assessment of the Reform of External Assistance’, SEC(2005) 963 of 11 July 2005.

(2)  ‘Commission for International Humanitarian and Technical Assistance’ established by decision of the Government of the Russian Federation in July 2004.

---

| 19.5.2006 | EN | Official Journal of the European Union | C 119/17 |
| --- | --- | --- | --- |

19.5.2006

EN

Official Journal of the European Union

C 119/17

---

Report by the Court of Auditors on the audit of the operational efficiency of the management of the European Central Bank for the financial year 2004 together with the replies of the European Central Bank

(2006/C 119/02)

| 1. | The Court's audit of the operational efficiency of the European Central Bank (ECB) is a requirement of Article 27(2) of the Protocol on the Statute of the European System of Central Banks (ESCB) and of the European Central Bank (1). The audit subject selected for the 2004 financial year was the efficiency of human resources policy at the ECB. The Court also did a follow-up audit on the monitoring of projects and on the procurement and management of external services. |
| --- | --- |

1.

The Court's audit of the operational efficiency of the European Central Bank (ECB) is a requirement of Article 27(2) of the Protocol on the Statute of the European System of Central Banks (ESCB) and of the European Central Bank (1). The audit subject selected for the 2004 financial year was the efficiency of human resources policy at the ECB. The Court also did a follow-up audit on the monitoring of projects and on the procurement and management of external services.

HUMAN RESOURCES POLICY

| 2. | The ECB's 2004 budget amounted to 376,4 million euro, of which staff costs were 145,9 million euro (39 %) (2). |
| --- | --- |

2.

The ECB's 2004 budget amounted to 376,4 million euro, of which staff costs were 145,9 million euro (39 %) (2).

| 3. | Since its inception in 1999 the ECB has experienced a period of rapid growth, from 518 staff members in 1999 to 1 309 staff members at the end of 2004. In its start-up phase, the Directorate Human Resources (D-HR) was mainly occupied with recruitment, day-to-day administrative tasks and coping with short-term constraints regarding human resources policy. In the current consolidation phase, the ECB has identified (3) several areas for improvement and fields for action in the HR (Human Resources) area. |
| --- | --- |

3.

Since its inception in 1999 the ECB has experienced a period of rapid growth, from 518 staff members in 1999 to 1 309 staff members at the end of 2004. In its start-up phase, the Directorate Human Resources (D-HR) was mainly occupied with recruitment, day-to-day administrative tasks and coping with short-term constraints regarding human resources policy. In the current consolidation phase, the ECB has identified (3) several areas for improvement and fields for action in the HR (Human Resources) area.

| 4. | The objective of the Court's audit was to assess the operational efficiency of the ECB's HR policy in respect of recruitment and mobility, remuneration and compensation, and absence management. The Court assessed whether the ECB had: |
| --- | --- |

4.

The objective of the Court's audit was to assess the operational efficiency of the ECB's HR policy in respect of recruitment and mobility, remuneration and compensation, and absence management. The Court assessed whether the ECB had:

| — | developed a comprehensive and coherent HR policy addressing its organisational needs, |
| --- | --- |

—

developed a comprehensive and coherent HR policy addressing its organisational needs,

| — | established adequate mechanisms at the central level for monitoring the implementation of its HR policy, |
| --- | --- |

—

established adequate mechanisms at the central level for monitoring the implementation of its HR policy,

| — | properly implemented its HR policy at all levels. |
| --- | --- |

—

properly implemented its HR policy at all levels.

| 5. | The completeness of the ECB's human resources policy in the selected areas was compared with other international organisations, these being the European Investment Bank (EIB), the European Bank for Reconstruction and Development (EBRD) and the EU institutions. In addition, the Court examined the management information available to the central services of the ECB to monitor the implementation of HR policy. In a sample of Business Areas (4) the Court then reviewed how the HR policy in the selected areas was being implemented. |
| --- | --- |

5.

The completeness of the ECB's human resources policy in the selected areas was compared with other international organisations, these being the European Investment Bank (EIB), the European Bank for Reconstruction and Development (EBRD) and the EU institutions. In addition, the Court examined the management information available to the central services of the ECB to monitor the implementation of HR policy. In a sample of Business Areas (4) the Court then reviewed how the HR policy in the selected areas was being implemented.

| 6. | Over the years, the ECB has developed a coherent HR policy addressing its main organisational needs. However, there is still room for improvement in the areas of recruitment, grading, disciplinary action and the prevention of bullying and harassment. For some of these areas, full compliance with Council Directives has not yet been achieved. |
| --- | --- |

6.

Over the years, the ECB has developed a coherent HR policy addressing its main organisational needs. However, there is still room for improvement in the areas of recruitment, grading, disciplinary action and the prevention of bullying and harassment. For some of these areas, full compliance with Council Directives has not yet been achieved.

| 7. | While mechanisms have been established at the central level for monitoring the implementation of HR policy, the monitoring role of D-HR still shows weaknesses, particularly in the areas of recruitment, training organised by Business Areas and leave and absence management. |
| --- | --- |

7.

While mechanisms have been established at the central level for monitoring the implementation of HR policy, the monitoring role of D-HR still shows weaknesses, particularly in the areas of recruitment, training organised by Business Areas and leave and absence management.

| 8. | Overall, the ECB has implemented most of its HR policy at all levels according to the policy it had established, although weaknesses relating to consistency and uniformity were found in the areas of salary setting, bonus distribution, recruitment and training. However, the most significant weakness concerned the low level of compliance with the leave and absence rules. |
| --- | --- |

8.

Overall, the ECB has implemented most of its HR policy at all levels according to the policy it had established, although weaknesses relating to consistency and uniformity were found in the areas of salary setting, bonus distribution, recruitment and training. However, the most significant weakness concerned the low level of compliance with the leave and absence rules.

| 9. | The Court recommends that the ECB should: |
| --- | --- |

9.

The Court recommends that the ECB should:

| — | make its HR policy more comprehensive and review it more regularly to keep it updated, having regard to developments in comparable international organisations, |
| --- | --- |

—

make its HR policy more comprehensive and review it more regularly to keep it updated, having regard to developments in comparable international organisations,

| — | strengthen the central monitoring function in the HR area in order to ensure that HR policy is equally implemented throughout the ECB and introduce performance measurement in the HR area by using key performance indicators and critical success factors, |
| --- | --- |

—

strengthen the central monitoring function in the HR area in order to ensure that HR policy is equally implemented throughout the ECB and introduce performance measurement in the HR area by using key performance indicators and critical success factors,

| — | address the problems found in the leave and absence area by improved monitoring, clearer communication and a better definition of responsibilities. |
| --- | --- |

—

address the problems found in the leave and absence area by improved monitoring, clearer communication and a better definition of responsibilities.

FOLLOW-UP OF PREVIOUS OBSERVATIONS

| 10. | The Court carried out a follow-up audit on the monitoring of projects (5) and on the procurement and management of external services (6). |
| --- | --- |

10.

The Court carried out a follow-up audit on the monitoring of projects (5) and on the procurement and management of external services (6).

| 11. | Central monitoring is only performed for projects with IT involvement and only encompasses IT resources. Other projects are still only monitored locally by the Business Areas concerned without an independent second level of control for these activities at prioritisation, approving and monitoring level. Furthermore, nearly half of the projects monitored at the central level in 2004 were subject to delays of more than 20 % compared to the initial planning. |
| --- | --- |

11.

Central monitoring is only performed for projects with IT involvement and only encompasses IT resources. Other projects are still only monitored locally by the Business Areas concerned without an independent second level of control for these activities at prioritisation, approving and monitoring level. Furthermore, nearly half of the projects monitored at the central level in 2004 were subject to delays of more than 20 % compared to the initial planning.

| 12. | A new Administrative Circular on the ECB's Procurement Rules was issued in September 2003 and a prototype database has been developed to record all exemptions to the Procurement Rules with an amount above 10 000 euro and all large purchases (amount above 250 000 euro). However, the ECB has still not implemented quality control of work performed by consultants and suppliers, neither at central monitoring level nor at the decentralised level. |
| --- | --- |

12.

A new Administrative Circular on the ECB's Procurement Rules was issued in September 2003 and a prototype database has been developed to record all exemptions to the Procurement Rules with an amount above 10 000 euro and all large purchases (amount above 250 000 euro). However, the ECB has still not implemented quality control of work performed by consultants and suppliers, neither at central monitoring level nor at the decentralised level.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 5 April 2006.

For the Court of Auditors

Hubert WEBER

President

---

(1)  Article 27(2) stipulates 'The provisions of Article 248 of the Treaty shall only apply to an examination of the operational efficiency of the management of the ECB'. The institutional provisions relating to the European Central Bank are included in Article 110 of the EC Treaty.

1

(2)  Comprising 133,2 million euro of direct salary costs and 12,7 million euro of indirect human resources costs, such as recruitment and removal allowances.

2

(3)  Within the context of an organisational change programme called ‘ECB in Motion’, the Executive Board launched a number of surveys conducted among staff in order to improve the effectiveness and efficiency of the ECB.

3

(4)  The term ‘Business Areas’ is used in the text for ‘Directorates General’ and ‘Directorates’ of the ECB.

4

(5)  OJ C 286, 23.11.2004 and OJ C 341, 4.12.2001.

5

(6)  OJ C 45, 20.2.2004.

6

---

REPLY OF THE EUROPEAN CENTRAL BANK

The European Central Bank (ECB) welcomes the report of the European Court of Auditors for the financial year 2004 and expresses its appreciation of the Court's acknowledgment that the ECB has developed a coherent HR policy addressing its main organisational needs.

The ECB takes note of the findings and recommendations for improvement made by the Court. In this respect, the ECB has prioritised the development of a diversity policy and a framework for preventing bullying and harassment. Work on these issues is now in its final stage. Although the ECB's practice complies with the substance of the European Union Directives, the ECB will address further improvements to ensure that its rules and regulations fully reflect the Directives. In all policy development projects, the ECB takes account of developments not only in comparable international organisations, but also in the other national central banks of the European System of Central Banks. Furthermore, as regards compliance with leave and absence rules, particular attention will be given to improving the monitoring of absence management in the near future. With regard to performance measurement in the HR area, the ECB has decided on a process to gradually introduce key performance indicators.

Major IT- and non-IT-related projects have always been centrally monitored by the ECB's Project Steering Committee. Since the third quarter of 2005, the Project Steering Committee's central monitoring has been extended to cover all IT related projects thereby covering the vast majority of the ECB's project undertaking. The quality control of work performed by consultants and suppliers is carried out as an integrated part of line management responsibility.

---