EN Official Journal of the European Union C 285/1

SPECIAL REPORT No 5/2007

on the Commission's Management of the CARDS programme together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, EC)

(2007/C 285/01)

CONTENTS

ABBREVIATIONS

I-VIII SUMMARY

1-7 INTRODUCTION

8-10 AUDIT SCOPE AND APPROACH

11-60 OBSERVATIONS

11-27 Weaknesses in strategic guidance in the programming process

11-19 CSPs too broadly formulated, MIPs without clear priorities, lack of effective guidance of aid

20 Weaknesses in strategic guidance lead to implementation problems

21-27 Redirecting assistance from investment support to institution building

28-41 Efficient management structures ensured speedy delivery of aid but national administrative capacity challenged

28-37 Management structures not always appropriate

38-41 Aid delivered rapidly but without taking sufficiently into account recipient countries' absorption capacity

42-54 Commission's management approach not conducive to increasing the responsibility of recipient countries

45-52 Limited recipient country responsibilities in the project identification and tendering phases

53-54 Lack of country ownership of donor coordination

55-60 Inconsistent approach to monitoring and evaluation

61-70 CONCLUSIONS AND RECOMMENDATIONS

61 How effective was the Commission's management of the CARDS programme?

62-64 How far was the strategic guidance directing the programmes appropriate and coherent?

65-67 How far were the management approaches established and used by the Commission appropriate and efficient?

68-69 To what extent did the projects financed encourage recipient country ownership?

70 How far was monitoring and evaluation effective in identifying weaknesses and disseminating information, and was timely subsequent action taken by the Commission?

The Commission's replies

ABBREVIATIONS ADF Albanian Development Fund BiH Bosnia and Herzegovina CARDS Community Assistance for Reconstruction, Development and Stabilisation CFCU Central Finance and Contracting Unit CSP Country Strategy Paper DG ELARG Directorate-General for Enlargement DG RELEX Directorate-General for External Relations EAR European Agency for Reconstruction EC European Commission ECPA EC Police Assistance mission in Albania EU European Union EURALIUS European Assistance Mission to the Albanian Justice System FRY Federal Republic of Yugoslavia FYROM Former Yugoslav Republic of Macedonia HQ Headquarter IPA Instrument for Pre-Accession KFOR NATO led Kosovo Force MIP Multiannual Indicative Programme OSCE Organization for Security and Cooperation in Europe OVI Objectively Verifiable Indicators ROM Result Oriented Monitoring SAI Supreme Audit Institution SAp Stabilisation and Association process SR Special Report of the European Court of Auditors UN United Nations UNMIK United Nations Mission in Kosovo WB World Bank

SUMMARY

I. The main objectives of the ‘Community Assistance for Reconstruction, Development and Stabilisation’ (CARDS) programme are to contribute to the recipients' participation in the Stabilisation and Association Process (SAp) and to increase the responsibility of the western Balkan countries and entities in relation to that process. CARDS was designed to encourage sustainable and EU compatible reform, consolidate stability and to bring the region closer to the European Union. For the period of 2000 to 2006 the programme has had a budget allocation of more than 5 000 million euro (see paragraphs 1 to 7).

II. The Court audited the CARDS programme to conclude on the effectiveness of the Commission's management. This was done by assessing (see paragraph 8): (a) whether the strategic guidance directing the programme was appropriate and coherent; (b) whether the management approaches established and used by the Commission were appropriate and efficient; (c) whether projects financed by the programme encouraged recipient country ownership; and (d) whether monitoring and evaluation were effective in identifying weaknesses and disseminating their findings, and the subsequent action taken by the Commission was timely.

III. The Commission's strategic guidance was very general. This was adequate for the reconstruction phase but not for the more challenging institution building objectives. The country strategy papers did not provide sufficient orientation, particularly as they were not supported by timely sectoral strategies. The multi-annual indicative programmes did not prioritise between or within the different areas of intervention. The European partnerships introduced in 2004 were not used to improve guidance for selection of actions (see paragraphs 11 to 27).

IV. In the past the EU assistance to the western Balkans suffered from the co-existence of three different legal bases, slow implementation of programmes and excessive centralisation. The CARDS Regulation unified the legal bases. The Commission was also successful in ensuring that aid was delivered quickly although this was sometimes at the expense of the capacity of the recipient countries to benefit from the aid. The Commission used four different management approaches. The two main approaches — devolved Delegations and the European Agency for Reconstruction (EAR) — were mostly effective and appropriate (see paragraphs 28 to 41).

V. While recipient country ownership was considered a priority by the Commission, there was no consistent approach to achieving this aim and the management approaches used did not encourage such ownership (see paragraphs 42 to 54).

VI. Monitoring and evaluation was good in parts, but inconsistencies exist between the Delegations and the EAR. An absence of adequate project indicators compromised the effectiveness of monitoring (see paragraphs 55 to 60).

VII. The Court concludes that the Commission's management of the CARDS programme was largely successful in contributing to stabilisation and reconstruction in the recipient countries. However, the aid was less effective in improving administrative capacities, due to an initially low concentration on institution building and insufficient recipient country capacity to absorb the assistance provided (see paragraphs 61 to 70).

VIII. The CARDS Regulation came to an end in 2006 and the new Instrument for Pre-accession Assistance came into effect in 2007. The Court recommends that the Commission: improves its strategic guidance; ensures that implementation speed takes full account of beneficiaries' ability to absorb the assistance; establishes an effective strategy for ensuring recipient country ownership; and extends best practice in monitoring and evaluation (see paragraphs 61 to 70).

INTRODUCTION

1. CARDS is the acronym for ‘Community Assistance for Reconstruction, Development and Stabilisation’. The CARDS programme (1) covers the following countries of the western Balkans: Albania, Bosnia and Herzegovina, Croatia, the Federal Republic of Yugoslavia (FRY) and the former Yugoslav Republic of Macedonia. Concerning the assistance to the FRY and its successor the State Union of Serbia and Montenegro (2), the CARDS programme was split into three annual programmes: Serbia (3), Kosovo and Montenegro.

2. CARDS is part of the European Union's Stabilisation and Association strategy towards the western Balkans, which emerged in response to the wars in former Yugoslavia during the 1990s. The Stabilisation and Association strategy marked the end of the military conflict, the start of physical reconstruction and economic re-launch of the countries (i.e. stabilisation) and increasingly closer ties with the European Union (i.e. association). The European Council meeting in June 2000 was explicit that this integration could ultimately mean membership of the European Union.

3. The preamble of the CARDS Regulation describes the main objective of the assistance: ‘The existing Community assistance should be expanded and redirected to adjust it to the European Union's political objectives for the region and, particularly, to contribute to the stabilisation and association process and increase the responsibility of the recipient countries and entities in relation to that process’. Also, according to the Regulation the regional dimension of Community assistance should be given special attention.

4. The Stabilisation and Association process (SAp) is a progressive partnership in which the EU offers a mixture of trade concessions (Autonomous Trade Measures), economic and financial assistance (CARDS Programme) and contractual relationships (Stabilisation and Association Agreements).

5. In the period from 2000 to 2006 the CARDS programme had a total budget of over 5 000 million euro, mainly for investment support and institution building. The total population in the western Balkans region is around 24 million. Thus the CARDS allocation represents an annual support of 30 euro per capita demonstrating the political importance that the EU attaches to the region (4).

6. The main areas of intervention of the CARDS programme are: (a) reconstruction, democratic stabilisation, reconciliation and the return of refugees; (b) institutional and legislative development, including harmonisation with European Union norms and approaches, the underpinning of democracy and the rule of law, human rights, civil society and the media, and the operation of a free market economy; (c) sustainable economic and social development, including structural reform; (d) promotion of closer relations and regional cooperation among countries and between them, the EU and the (then) candidate countries of central Europe.

7. In the European Commission the management of the CARDS programme, until early 2005, was the joint responsibility of the Directorate-General for External Relations (DG RELEX) and the EuropeAid Co-operation Office. DG RELEX had the responsibility for political relations and strategic development such as preparation of the country strategy papers and multi-annual indicative programmes. The EuropeAid Co-operation Office was responsible for the management of the annual programmes. In early 2005 the Directorate-General for Enlargement (DG ELARG) took over the responsibility for the full project cycle from policy formulation and programming to the implementation.

AUDIT SCOPE AND APPROACH

8. The purpose of the European Court of Auditors audit was to assess on the effectiveness of the Commission's management of the CARDS programme. The main objectives of the audit were to conclude on the extent to which: (a) the strategic guidance directing the programme was appropriate and coherent; (b) the management approaches established and used by the Commission were appropriate and efficient; (c) the projects financed by the programme encouraged recipient country ownership; (d) monitoring and evaluation were effective in identifying weaknesses and disseminating their findings, and the subsequent action taken by the Commission was timely.

9. The audit work included a review of strategy and planning documents, guidelines and reports. In addition a judgemental sample of CARDS projects (Annex) over the period 2002 to 2005 was examined.

10. The audit also examined the relevant evaluation reports, prepared on behalf of the Commission — including the mid-term evaluation of the CARDS programme (5), which was completed in June 2004 — as well as the follow-up actions taken by the Commission. The Court concluded that they provided a sufficient basis to use the results as corroborative evidence for its own audit findings (6).

OBSERVATIONS

Weaknesses in strategic guidance in the programming process

CSPs too broadly formulated, MIPs without clear priorities, lack of effective guidance of aid

11. The CARDS regulation provides the legal basis for the main strategy papers used for CARDS assistance: ‘The Community assistance should be governed by a strategy framework and by annual and multi-annual programming, which will be put to the management committee set up by this regulation for an opinion’.

12. The Court assessed the extent to which the CARDS programme was governed by adequate strategic guidance and directed towards the specific need of the countries involved while being coherent with the regulatory-defined objectives of the programme and the SAp in its wider sense.

13. In accordance with the CARDS regulation the Commission defined a strategy framework for the western Balkan countries, with country strategy papers (CSPs) for each individual country, and a regional strategy paper, drafted taking into account all the recipient countries. As required by the regulatory framework the Commission has produced two multi-annual indicative programmes (MIPs), one for each country and one for the regional inter-state programme for the periods 2002 to 2004 and 2005 to 2006.

14. In 2004 — at the end of the first MIP period — European partnership documents were approved by the European Council covering Albania, Bosnia and Herzegovina, the former Yugoslav Republic of Macedonia, and Serbia and Montenegro including Kosovo (7), with the aim of intensifying the stabilisation and association process (8). The partnerships were drawn up for each country in the region and identified areas for action and obligations to be fulfilled by the partner countries in order to move them closer to the European Union. EU assistance was to be directed towards the priorities set out in the partnerships.

15. The audit revealed weaknesses in the strategic guidance of CARDS programming mechanisms. Strategies as described in Country Strategy Papers (CSPs), although formally compliant with the Commission CSP guidelines, were too broadly formulated to be useful for the programming exercise. CSPs were not updated to reflect major changes such as the Council's European partnership documents.

16. The MIPs did not prioritise between and within sectors as required by Commission guidelines (9) nor indicated timing for the foreseen actions. The first European partnership in 2004 offered an opportunity for the Commission to focus its actions more precisely in the MIP covering 2005 to 2006 (10). Even though the MIPs for 2005 to 2006 contained references to the European Partnership priorities, they still gave insufficient guidance for the selection of actions to be financed.

17. As provided in the CARDS Regulation the CARDS Management Committee (11) was required to give its opinion on the CARDS programming guidelines, the Multi-annual Indicative Programmes and the Annual Programmes, with the intention of enhancing strategic guidance in the programming process. In reality no evidence was found that the Committee's work contributed to improving the quality of the proposed documents.

18. Detailed sectoral analyses by the Commission and/or recipient country national strategies mostly did not exist or were produced too late to enhance the strategic guidance. As a result, projects were not selected on the basis of objective criteria and, in certain cases, they were selected in response to the beneficiaries' ad hoc request.

19. Although the identification process was complex, it did not involve a clear methodology for optimising project selection. For instance, there was no cost-benefit analysis (12) or other comparative analysis of possible alternative projects before selecting the projects. Some projects were simply continuations of earlier support. There were no guidelines to help decide which intervention would best support the stabilisation and association process.

Weaknesses in strategic guidance lead to implementation problems

20. The audit identified differences between the planned and implemented projects. There were essential projects not implemented due to the resistance of the recipient country or projects for which the initial plans were disregarded (Box 1). Box 1Examples of non-implemented projectsCase 1: No political support to the Serbian Supreme Audit Institution (SAI)As early as 2001 the EAR had identified the necessary building blocks for significant government institutional reform in Serbia, including support to the treasury, the internal audit function as well as a project concerning the creation of a Supreme Audit Institution. However, the necessary backing of the Serbian Parliament did not materialise for the SAI and the project was not taken forward until the 2006 programme.Case 2: The European Commission Police Assistance mission in AlbaniaThe project fiche of the Justice and Home Affairs sector in Albania for the programming year 2002 envisaged for the Police and Public Order component ‘to contribute to the establishment of rule of law and to the establishment of a functioning police force able to contribute to democratic stability’. It established the following: ‘The project will be executed through supporting the EC Police Assistance Mission (ECPA), training and related investment support’. Out of the 8 million euro allocated to the Police and Public order component, 4 million euro were planned for institution building support (financing ECPA mission), 2,5 million euro for training and support component and 1,5 million euro for the investment component.In reality the project fiche was completely disregarded. Not one euro was spent in order to finance the ECPA mission. 4,7 million euro were spent on a Data Communication Network, 1,9 million euro were spent on the power supply equipment and 1 million euro were spent on a grant to the United Nations in order to support the Security Sector Reform programme.

Redirecting assistance from investment support to institution building

21. Because the recipient countries were recovering from war the early years of the CARDS programme focused more on reconstruction (through investment projects) than institution building. However, as the programme matured a shift towards more institution building was required.

22. The redirection of assistance was of interest to the discharge authority. In its report on the European Agency for Reconstruction (EAR) published in May 2002, the European Parliament considered that the EAR should significantly increase EU assistance in the two countries (i.e. the Federal Republic of Yugoslavia and the former Yugoslav Republic of Macedonia) in favour of institution building.

23. The Court analysed the CARDS projects in order to determine and assess the proportion of institution building projects compared with investment projects (see Graph 1).

24. In general between 2001 and 2005 the emphasis on institution building projects increased. From the very beginning the focus on institution building was more pronounced in countries where assistance was managed by the Delegations (14) than in countries where it was managed by the EAR (15).

25. When strengthening the administrative capacity of a country, twinning is recognised as an important instrument (16). However, twinning had a slow start in the western Balkans particularly in the former Yugoslav Republic of Macedonia where the EAR only started to use twinning in 2005. An EAR evaluation of twinning (17) concluded that there was sometimes only limited support from the Agency's Programme Managers for twinning and that EAR staff were not always trained sufficiently in the use of that instrument.

26. As the countries of western Balkans recovered from the war, the draft programmes and the related Project Fiches (18) as well as the projects under implementation took into account the Parliament's request to increase the EU assistance from investment support to institution building. However, in some cases the changes to the planned projects led to decreased emphasis on institution building (see Box 2). Box 2Redirecting an institution building project — Integrated Border Management in the former Yugoslav Republic of MacedoniaThe Annual Programme for CARDS 2003 in the former Yugoslav Republic of Macedonia envisaged, among other things, to strengthen control of the green border and allocated 2,8 million euro for this purpose. According to the Annual Programme, there were a number of short-term actions needed in this area: (1) set-up a dedicated border guard service, (2) improve recruitment procedures, (3) provide basic training and in-depth course on border control and (4) supply equipment for detection of forged or falsified documents and for high-tech vehicle searches, together with corresponding training tools.Objectives for this initiative were twofold: (1) To enhance the co-operational capacity of government border services within the existing organisational framework, to fulfil immediate requirements, and (2) to develop a new organisational structure for the border police and operational capacities in compliance with European Union standards.Instead of consistently addressing the above needs, objectives and priorities, the EAR used the whole amount to purchase vehicles.

27. The Court found no evidence of on-going monitoring of projects by the Management Committee after the approval stage. In other words the Management Committee did not concern itself with changes to projects once implementation had started.

Efficient management structures ensured speedy delivery of aid but national administrative capacity challenged

Management structures not always appropriate

28. In July 1998 the Court published Special Report No 5/98 on reconstruction in former Yugoslavia (period 1996 to 1997). The audit covered two countries: Bosnia and Herzegovina and to a lesser extent Croatia. The report criticised that despite the EU's clear declarations of intent, the Commission was not successful in deploying sufficient human resources to the task of reconstruction in former Yugoslavia, neither at its headquarters nor in the areas under reconstruction. The Court concluded the following: (a) Implementation of the appropriations allocated by the EU for reconstruction was slowed by the lack of an implementation programme, shortage of staff, excessive headquarter centralisation and the institutional weaknesses of the local authorities. (b) In 1997, the disbursement rate of the EU's reconstruction programme (29 %) was relatively low compared with the average of other donors (52 %). (c) The co-existence of three separate regulations governing the budget articles used to finance the reconstruction measures hampered the Commission's management.

29. To overcome the deficiencies in EU aid delivery, the Court recommended the Commission should devolve management for its aid effort to its Delegations and focus on a smaller number of sectors.

30. While taking into account the above-mentioned weaknesses, the Court's current audit assessed the extent to which the CARDS management structures were efficient and whether the aid was delivered with sufficient rapidity.

31. The audit revealed significant improvement in the efficiency of EU operations in the western Balkans since 1998. Major changes took place. Firstly, the CARDS regulation unified the legal basis for similar operations. Secondly the devolution process was introduced to improve the speed and quality of EC external aid (19). Thirdly the Commission delegated part of the implementation of the assistance to a new organisation specially created for this purpose the European Agency for Reconstruction. All these actions speeded up aid delivery and improved the efficiency of the assistance (see Box 3). Box 3Example of high contracting rates under CARDSEAR was created as a response to the need to act quickly and efficiently when addressing the urgent post-conflict needs of the population in Kosovo in the immediate aftermath of the 1999 crisis. The Court has in its earlier audits concluded that EAR actions in Kosovo immediately after the crisis met the ambitious objectives in the fields of basic services. The high contracting rates have been maintained in Kosovo throughout the CARDS programme. As of November 2006, the EAR has in Kosovo managed over 1 100 million euro on behalf of the Commission, some 91 % of that money has been contracted and some 86 % disbursed. Along with the KFOR troops, the EU's assistance helped normalise life in Kosovo.

32. In total CARDS was implemented using four different methods and structures (see Graph 2), most of which left little room for involvement of the recipient states. This was done in order to ensure speedy delivery of aid and reflected the weaknesses of the national administrations.

33. The main characteristics of the different management structures are as follows: (a) Centralised management by DG Enlargement (and earlier EuropeAid): DG ELARG manages the regional programmes directly from the headquarters. Regional programmes focus on strengthening administrative capacity e.g. in customs, the fight against organised crime and supporting civil society. (b) Devolved management: The EC Delegations in Albania, Bosnia and Herzegovina and Croatia were devolved (20). As a result of devolution, Delegations became responsible for project preparation, contracting, and financial and technical implementation. (c) Decentralised implementation: involves the Commission delegating implementation of some programmes to the recipient country Government in Albania and in Croatia. (d) Indirect centralised management: The European Agency for Reconstruction (EAR) (21), an independent agency of the EU, manages the programme in Kosovo (22), Serbia, Montenegro and in the former Yugoslav Republic of Macedonia. The EAR is administratively accountable to the European Council and to the European Parliament, but not to the European Commission.

34. The Court found that devolved and indirect centralisation (EAR) ensured efficient delivery of aid, whereas weaknesses were noted with the other two structures (i.e. centralised and decentralised) (23).

35. In both Delegations and the EAR, the speed of project management benefited from the increased capacity in the operational units and from having finance and contracts staff available on the ground. The increased contacts with the beneficiary and other relevant parties and better understanding of local conditions, risks and opportunities improved performance while reducing the time needed to solve problems (24).

36. The two other implementation structures — although less significant in monetary terms — were less effective. For centralised management — centrally managed regional inter-state programme — there were several shortcomings: (a) funds allocated to regional cooperation comprised a very small proportion of total expenditure (around 5 %). Some of this was managed at national rather than regional level, e.g. integrated border management; (b) similarly most of the refugee return programmes which, by definition, have cross-border characteristics were managed separately in each country; (c) while steps were taken in 2003 to involve beneficiary countries more in the programming of CARDS regional allocations, countries generally did not receive sufficient information of the regional projects.

37. The decentralised implementation structures in Albania and in Croatia also had problems: (a) in Albania, programmes involving the Albanian Development Fund (ADF) continued without the Commission having assessed whether the ADF was capable of meeting its responsibilities, according to the provisions of the Financial Regulation applicable in 2003. Due to the problems encountered with the ADF, the Commission decided to strongly reduce the decentralised part of the Albanian CARDS programme and devolved it to the Delegation; (b) in 2002, in Croatia, specific steps were taken towards formal decentralisation. However, the speed at which EuropeAid considered it appropriate to decentralise did not take into account practical operational realities in the Croatian Ministry of Finance. As a result the decentralisation process was halted between 2003 to 2005. Once the CARDS programme moved from EuropeAid to DG ELARG in early 2005, the Commission re-evaluated the situation and decided in February 2006 to restart decentralisation in Croatia (25).

Aid delivered rapidly but without taking sufficiently into account recipient countries' absorption capacity

38. The European Court of Auditors has in its reports concerning external actions (for instance Special Report No 6/2006 on the environmental aspects of the Commission's development cooperation (26)) repeatedly criticised the Commission of delays in contracting and implementation. In contrast the CARDS programme had a very high contracting speed (see Graph 3) in Kosovo, Bosnia and Herzegovina, and Serbia but due to the back-log of earlier programmes a relatively slow start in Albania and Croatia.

39. In Albania the need to compensate this delay led, later on, to a very high concentration of contract signatures: 63 % of the CARDS contracts signed in the period from 2001 to 2005 were signed in 2004, thus leading to the simultaneous implementation of a high number of contracts and an increased challenge to the beneficiaries' absorption capacity (27) (see Box 4). Delays in the 2001 annual programme meant that in Croatia two annual programmes were implemented at the same time. Box 4Overstretched national administrative capacity in AlbaniaIn Albania in addition to high concentration of projects in a short period of time, most of the assistance was concentrated in the Justice and Home Affairs sector. At the time of the audit, there were at least 48 different, simultaneously on-going projects in the Justice and Home Affairs' area. 13 of them were financed by the EU and 35 by other donors.For example in the Euralius project in Albania (European Assistance Mission to the Albanian Justice System), the audit found out that the level of ambition of the project design was very high. Ten specific objectives had to be achieved involving 15 different types of project partners simultaneously. Each of the ten objectives involved numerous different project partners. As a result, the Ministry of Justice, the principal beneficiary, would be dealing with 78 different foreseen activities.

40. One of the main unintended side effects of the high contracting speed, encouraged by the Commission was that beneficiaries might not fully benefit from the projects, thus reducing their effectiveness and impact.

41. To ensure project effectiveness Community assistance under the stabilisation and association process is conditional and failure to respect the conditions could lead the Commission to take appropriate measures. However, there are projects which were implemented in cases where conditionalities were not respected by the beneficiary, thus providing less added value than originally expected. An example in this respect was the support to the Croatian Court System described in Box 5. Box 5Support to the Croatian Court SystemThe Twinning project financed by the 2002 programme was implemented despite the fact that none of the conditionalities defined in the project form were fulfilled at the time of the project start-up. One of the conditionalities presumed the existence of a strategy for the reform of the judiciary. Given that the project was an absolute priority for the Commission, it was imposed on the recipient. For the above reasons recommendations made to the recipient, although accepted and recognised by the Croatian Ministry of Justice, have not produced major actions in all areas.

Commission's management approach not conducive to increasing the responsibility of recipient countries

42. In addition to the contribution to the SAp, the preamble of the CARDS Regulation describes the other main objective of the assistance as: ‘The existing Community assistance should … increase the responsibility of the recipient countries and entities in relation to that [SAp] process.’

43. As set out in paragraph 31, the main reason behind the management structures established was to ensure quick delivery of aid. The downside of this approach is reduced recipient state involvement. This approach was appropriate in the reconstruction phase but less so as the assistance moved from infrastructure projects to institution building projects.

44. The regulation does not define any specific responsibilities for the recipient countries. Despite the apparent political will to bring the countries as rapidly as possible into the European Union, this has not been matched by any increase in the countries' management responsibilities. The Court did not find any indication of a formal aid management strategy on the Commission's part — general or country specific — to increase progressively the responsibility of the recipients.

Limited recipient country responsibilities in the project identification and tendering phases

45. The Court assessed to what degree the Commission's management of the CARDS programme contributed to increasing the responsibility of recipient countries in certain critical steps of the project cycle management, and assessed the consequences of insufficient ownership.

46. The Commission's guidelines for the identification of projects and subsequent drafting of the Annual Programmes (28) did not promote beneficiary participation in the decision-making. The instructions issued by EuropeAid for the 2001 and 2002 annual programming did not specify any kind of beneficiary involvement. The guidelines for the 2003 and 2004 annual programmes specified some involvement — albeit limited — requiring initial contacts to be taken with potential beneficiaries during the preparatory work, and draft project fiches to be sent by the Delegation or EAR to the beneficiaries at least two weeks before finalising the annual programme. The audit assessed the extent of the involvement of the beneficiaries through on-the-spot interviews with around 30 beneficiaries.

47. The Court found some examples, especially in Bosnia and Herzegovina, Croatia and Serbia, where the Commission and the Agency had tried to increase recipient country ownership (see Box 6). Box 6Example of good practices in ownershipCase 1: Strengthening the Directorate of European Integration in Bosnia and Herzegovina (BiH)The decision-making process for the CARDS 2006 programming in BiH is a good illustration of ownership and accountability development. Already in the CARDS 2004 and 2005 annual programme, the Commission started to build capacity with the recipient country, for example by introducing senior programming officer posts in BiH line ministries and agencies and by supporting the Directorate for European Integration. Because of the efforts made, in the last year of the CARDS programme, project fiches were prepared by the BiH authorities.Case 2: Nikola Tesla project in SerbiaAlthough the Agency's legal framework (i.e. indirect centralised management) is not the most appropriate for enhancing the ownership of the recipient country, the Agency was still able to find an innovative solution in the case of a power plant project in Serbia (Nikola Tesla A5).The innovative feature of the implementation of the project was the use of a ‘Grant Agreement’. Through this arrangement the responsibility for the planning, including the definition of the technical specifications and management of the project was entrusted to the beneficiary, the Serbian electricity utility.The technical specifications were checked and approved by the contracted supervisor and the tendering procedure and control of payments was maintained by the Agency. All parties, including the beneficiary, were very satisfied with the arrangement and thus it can be concluded that the ownership was as complete as can be expected under the present arrangements.

48. However, in most cases the Commission was not consistent in promoting recipient country responsibility when developing the so-called project fiches and later the terms of reference. Participation varied from country to country on the basis of initiatives taken by the Delegations or the EAR depending on the capacities of the countries concerned. While the Court noted increasing input from recipient countries, ownership remained limited.

49. The co-financing of projects with national funds is one indicator of recipient country ownership and interest. Although the CARDS Guidelines encourage the use of national co-financing, it was hardly used (29). A positive exception, i.e. the only one out of the 31 non-grant projects audited, in this respect is the Croatian land cadastre project, which was significantly co-financed and where the donor coordination was effectively led by the Croatian authorities. Similarly the commitment of the beneficiaries of grants was also high (30).

50. One way of increasing recipient country ownership is to involve national authorities in tender evaluation committees. However, the EuropeAid instructions binding the Delegations and acting as guidance for the EAR, require that the majority of the voting members of the evaluation committees represent the Commission (see Box 7). This contrasts with many other areas of external actions where the Commission is not represented as a voting member on evaluation committees, but retains the right of veto. Box 7Composition of tender evaluation committeesThe Court assessed 177 tender evaluation committees covering 2002 to 2005. On average 55 % percent of the voting members were task managers from the Delegations or EAR operational centres and 13 % of the voting members were independent external experts. The average representation of beneficiary organisations was 32 % with the exception of Croatia, where the percentage increased significantly to 52 % and Bosnia and Herzegovina where the trend of beneficiary voting members decreased to 24 %.

51. The EuropeAid instructions require evaluation committees to comprise a minimum of four EC Delegation/EAR staff (i.e. chairman, secretary and two evaluators) for one external evaluator and an additional two EC Delegation/EAR staff for every additional beneficiary evaluator. This draws excessively on the human resources of the Delegations while preventing improved country ownership.

52. One of the obstacles to decentralisation was a lack of knowledge by recipient countries of the European Commission's contract procedures. In Croatia, representatives from national authorities were systematically invited by the Delegation to participate in the evaluation committee meetings as non-voting observers. This represented good practice by allowing national authorities to become familiar with the detailed tendering procedures, but was not applied systematically in other countries. For example, in Albania, the former Yugoslav Republic of Macedonia and Serbia the practice was applied only in very rare cases.

Lack of country ownership of donor co-ordination

53. According to the CARDS guidelines all CARDS support should be co-ordinated with other donors. Donor co-ordination is particularly challenging in the western Balkans because of the high number of donors and the existence of two agendas: the accession agenda of the EU; and the development, poverty-reduction agenda of the World Bank, UN and some agencies of EU Member States.

54. The ‘Paris Declaration on Aid Effectiveness’ (31) confirmed the principle that the recipient countries themselves should take the lead in coordinating aid at all levels, while the donors should pledge to respect partner country leadership. In the CARDS countries the audit found almost no evidence of such leadership by the recipient countries or of any actions by donors to encourage beneficiary country leadership. The only initiative was visible in Albania (see Box 8). Box 8Donor co-ordination in AlbaniaIn order to improve donor coordination in Albania, the four multilaterals (EC, OSCE, UN and WB) set up a sectoral coordination mechanism in December 2003. They agreed on the division of responsibilities between themselves in four thematic areas. In order to manage the coordination process, they formed the Donor Technical Secretariat. This contributes to a more effective complementarity and more effective division of labour. The donors are expected to make use of their respective comparative advantages at sector level and to delegate authority to the assigned lead donors for the execution of programmes.The donor community contributed to establishing an Integrated Planning System, which was launched by the Government of Albania in 2005 and which should help the country to lead its own development process. This aims to integrate the various policy frameworks into a single medium-term development plan in 2007 and to set up a new architecture for donor coordination and management of external assistance.

Inconsistent approach to monitoring and evaluation

55. Delegations and the EAR have different monitoring procedures: (a) the Commission HQ and Delegations use outsourced monitoring services following a standard methodology called Result-Oriented Monitoring (ROM). It includes the use of a rating system for the performance of the projects, but does not provide a formal record of the follow-up; (b) the EAR has its own monitoring teams in each Operational Centre, as well as a coordinator in the HQ. These teams used the ROM system from the beginning of 2005. The follow-up is formally recorded in the monitoring documents.

56. Effective monitoring requires projects to have objectives and objectively verifiable indicators (OVIs) to monitor against. However, OVIs are either not provided, or when provided, they are not measurable or time-bound. For instance in a Police Academy project one of the OVIs was ‘adequate teaching of staff’ without specifying the number of participants foreseen or the curriculum. In another project, there were indicators such as ‘enhanced operational capacity’ without specific criteria.

57. In accordance with the CARDS regulation the Commission evaluated the programme by the end of June 2004. According to the evaluation terms of reference, its objective was to provide the European Commission with an independent and reliable assessment of the assistance strategy under CARDS and its evolution with its wider political setting. It covered the 2001 to 2003 period (see also paragraph 10).

58. In order to improve financial management a critical part of the evaluation process is the follow-up on the recommendations made. In principle, the Commission has one year to report on its follow-up actions. In this case the Commission reported over two years later. This may have been partly due to the Commission not being satisfied with the quality of some of the recommendations made.

59. While there was no formal requirement to evaluate individual projects or sectors, this is good practice as it would allow focused results and recommendations. The EAR systematically carried out evaluations at sector and project level. Both mid-term evaluations and ex-post evaluations were made. The executive summaries of these evaluations were published on the EAR website, thus contributing to a dissemination of the findings. In contrast the Delegations rarely carried out project or sector evaluations and the few reports carried out by the Delegations were not published on their websites.

60. The dissemination of the results of monitoring and evaluation exercises also differs between the Delegations and the EAR. The EAR organises regular meetings and workshops for the Task and Programme Managers from different operational centres in order to exchange experience. However, the same level of dissemination does not take place between Delegation countries. There are some contacts between Delegations and EAR staff but these are not systematic.

CONCLUSIONS AND RECOMMENDATIONS

How effective was the Commission's management of the CARDS programme?

61. The Court concludes that the Commission's management of the CARDS programme was largely successful in contributing to stabilisation and reconstruction in the recipient countries. However, the aid was less effective in improving administrative capacities, due to an initially low concentration on institution building and insufficient recipient country capacity to absorb the assistance provided.

How far was the strategic guidance directing the programmes appropriate and coherent?

62. The Commission's strategic guidance was very general and not targeted at specific areas. It was adequate for the reconstruction phase, but not for the more challenging institution building objectives.

63. The country strategy papers were broadly formulated with the result that they provided insufficient orientation at a detailed level, particularly as they were not supported by timely sectoral strategies. The multi-annual indicative programmes — which form the basis for annual planning and the identification of projects — did not prioritise between or within the different areas of intervention. As such there was no basis for optimising project selection resulting in a number of uncoordinated and isolated projects.

64. Early in the programmes, aid was mostly directed towards reconstruction investment-type projects, rather than institution building. As the programmes progressed, the focus naturally changed towards institution building which now represents the large majority of expenditure. The Court notes that from the very beginning the focus on institution building was more pronounced in countries where assistance was managed by the Delegations than in countries where it was managed by the EAR. In the latter case it delayed the introduction of twinning projects. Recommendation 1The strategic guidance for implementing the new Instrument for Pre-Accession Assistance (IPA) (32) should give sufficient attention to individual areas of intervention to ensure an appropriate focus of aid, a systematic approach to key areas and an appropriate basis for prioritising and selecting projects.

How far were the management approaches established and used by the Commission appropriate and efficient?

65. The management approaches used by the Commission were largely, but not always, appropriate for recipient countries' circumstances and needs. They were effective in ensuring quick delivery of aid, but gave insufficient consideration to maximising the benefit from the aid.

66. Four different management approaches were used, with the aim of reflecting the circumstances of the recipient countries concerned. Two of the approaches (i.e. devolved Delegations and the EAR) — covering the majority of expenditure — were mostly effective and appropriate. The other two (i.e. centralised and decentralised management) were less efficient and responsive to the specific needs.

67. The Commission was successful in responding to previous criticisms of slow implementation by ensuring that aid was delivered quickly. This was of particular advantage during the reconstruction phase. However, for institution building, the focus on speed was not balanced by sufficient consideration of the ability of recipient countries to maximise the benefit from the aid or comply with all preconditions. Recommendation 2The Commission should maintain its focus on rapid implementation of aid, but also pay sufficient attention to aspects of effectiveness such as the ability of beneficiaries to fully absorb the assistance provided (particularly in the context of institution building projects).

To what extent did the projects financed encourage recipient country ownership?

68. While recipient country ownership was considered a priority by the Commission, there was no consistent approach to achieving this aim and the management approaches used did not encourage such ownership.

69. Recipient countries had only a limited involvement in the project identification and tendering phases. Towards the end of the programme the input from recipient countries increased, although ownership remained limited. Despite the recipient countries committing themselves to co-ordinating the many different donors in the area — an important indicator of ownership — the audit found no evidence that such coordination had taken place. Recommendation 3For the IPA programme the Commission should address the important aspect of recipient country ownership through specific strategies in all areas of management and implementation, in particular tendering.

How far was monitoring and evaluation effective in identifying weaknesses and disseminating information, and was timely subsequent action taken by the Commission?

70. Monitoring and evaluation was good in parts, but of inconsistent quality. Monitoring procedures differed between the Delegations and the European Agency for Reconstruction, although a common methodology was applied from 2005. An absence of adequate project indicators compromised the effectiveness of monitoring. An evaluation of the CARDS programme was carried out as foreseen, but the follow-up by the Commission was late. Unjustified inconsistencies exist between the Delegations and the EAR in respect of the intensity of project or sector evaluations and the dissemination of the results of monitoring and evaluation. Recommendation 4Monitoring and evaluation practices should be harmonised, including extending to the Delegations the EAR's practice of evaluating individual projects/sectors and of disseminating results and lessons learnt.The Commission should ensure that it follows up evaluations promptly.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 28 June 2007. For the Court of Auditors Hubert WEBER President

(1) Council Regulation (EC) No 2666/2000 (OJ L 306, 7.12.2000, p. 1).

(2) In February 2003 the FRY was restructured into a State Union of two republics ‘Serbia and Montenegro’. The Constitutional Charter allowed, after three years, either republic to hold a referendum for their independence. Montenegro held an independence referendum in the spring of 2006 in line with international standards. The vote for severing ties with Serbia exceeded the 55 % threshold, allowing Montenegro to formally declare its independence on 3 June 2006. Two days later, Serbia declared that it was the successor state to the union of Serbia and Montenegro.

(3) Throughout the report reference to Serbia therefore excludes the province of Kosovo.

(4) By way of comparison the Tacis Regulation covering the same period as CARDS foresees an amount of less than 2 euro per capita per annum.

(5) Article 13 of the CARDS Regulation established that ‘not later than 30 June 2004, the Commission shall submit to the Council an evaluation report, together with proposals for the future of this Regulation and for any amendments to be made to it.’

(6) The main conclusions of the evaluation of the assistance to Balkan countries under CARDS Regulation (EC) No 2666/2000 (contract No B7-6510/2002/005) were:

(a) the CARDS programme was highly relevant for ensuring financial and technical support to the Stabilisation and Association process;

(b) the centralised nature and approach might become an obstacle to the programme ownership and effectiveness;

(c) balancing support for institution building and development investment was a key challenge;

(d) the regional dimension of the programme was weak.

(7) On the basis of Council Regulation (EC) No 533/2004(OJ L 86, 24.3.2004, p. 1).

(8) The European Partnership for Croatia was approved by Council Decision 2004/648/EC (OJ L 297, 22.9.2004, p. 19).

(9) Reference made to point 3.1.4 of the guidelines for implementation of the common framework for country strategy papers.

(10) Since becoming a candidate country in 2004, Croatia has benefited from the pre-accession instruments. A revision of the CARDS MIP for 2005 to 2006 was therefore not necessary.

(11) The CARDS Management Committee is composed of the representations of the Member States and chaired by the Commission.

(12) Cost-benefit analysis allows a measure of the economic efficiency of the programme or project, as well as providing a basis for comparison with alternatives.

(13) Projects of each programming year n are implemented in the year n and the following years.

(14) The EC Delegations are implementing CARDS programmes in Albania, Bosnia and Herzegovina and in Croatia.

(15) The EAR has operational centres in former Yugoslav Republic of Macedonia, Kosovo, Montenegro and Serbia.

(16) The Commission launched twinning as the principal tool of pre-accession assistance for institution building already in May 1998. The Thessaloniki Summit, in June 2003 underlined the significance of twinning in the western Balkans.

(17) Reference made to EAR evaluation EU/13/040/05 of November 2005.

(18) A project fiche is a short description (4 to 5 pages in average) without specific terms of reference. The initial description includes the foreseen project intervention logic, project objectives, results and activities. It assesses the project risks and defines objectively verifiable indicators (OVIs) and means to verify these indicators.

(19) As a result of devolution EC Delegations are responsible for project preparation, contracting, and financial and technical implementation. As a consequence the role of the central services in Brussels evolved away from direct management of projects towards monitoring and supporting Delegations.

(20) The devolution process had started in Bosnia and Herzegovina in 1998, even before the Commission announcement in May 2000 of the major reform of the management of its external aid programme. The Delegation in Croatia was devolved in 2001 and the Delegation in Albania in 2003.

(21) The EAR was established in February 2000, its predecessor, the task force for Kosovo, started operation in 1999. In 2001, the agency's mandate was extended to Serbia and Montenegro and in 2002 to the former Yugoslav Republic of Macedonia.

(22) Kosovo is presently under UNMIK administration as defined by UN Security Council Resolution 1244.

(23) The shortcomings are also confirmed by the mid-term CARDS evaluation.

(24) Reference is made to Special Report No 10/2004 concerning the devolution of EC external aid to the Commission delegations (OJ C 72, 22.3.2005).

(25) Reference made to Article 2 in Commission Decision C(2006) 264 (E/148/2006): ‘For CARDS, this Decision shall apply to a number of projects agreed between the Commission and Croatia in the CARDS Financing Agreement 2003 (signed on 2 September 2003) and in the CARDS Financing Agreement 2004 (signed on 29 November 2004).’

(26) OJ C 235, 29.9.2006.

(27) The Albanian contracting peak of 2004 is at least partly a consequence of a Commission HQ devolution verification mission carried out in December 2003. The mission concluded that the final confirmation of the devolution sub-Delegation was conditional to achieving a tender target of 25 million euro within a delay of six months.

(28) The annual programmes contain a detailed list of projects — so called project fiches — to be financed and specify the relevant amounts. Since 2005 project fiches are being grouped and consolidated in sector fiches. Formally the project fiches were not part of the Commission financing decision therefore not legally binding unlike to sector fiches.

(29) Except with the grant contracts where the co-financing is an obligation. A grant is a direct payment of a non-commercial nature by the European Community to a specific beneficiary to implement an action forming part of EU policy.

(30) Projects audited included for example a tourist centre in the Travnik fortress in Bosnia and Herzegovina and a mobile library in the city of Zadar in Croatia.

(31) The Paris Declaration on Aid Effectiveness was signed after the High Level Forum which was held in Paris in March 2005. Among the participants there was the European Commission and 91 countries, two of which represented the CARDS area (Albania and Serbia-Montenegro).

(32) Council Regulation (EC) No 1085/2006 (OJ L 210, 31.7.2006, p. 82).

ANNEX CARDS PROJECTS AUDITED (33 in total) Annual Programme Date of signature Sector EUR ALBANIA 1 Local Community Development Fund 2002 10/2003 Regional Development 7 400 000 2 EURALIUS — European Assistance Mission to the Albanian Justice System 2002 05/2005 Judiciary and Police 4 498 686 3 PAMECA II — Police Assistance Mission of the European Community to Albania 2004 12/2004 Judiciary and Police 3 800 000 4 Korça and Vlora Courts of Appeal 2003 05/2005 Judiciary and Police 1 630 395 5 ASYCUDA II — Technical Assistance and Training 2002 04/2005 Judiciary and Police 399 815 Total 17 751 366 BOSNIA AND HERZEGOVINA 1 Constructing of Karakaj Border Crossing Point 2001 02/2004 Integrated Border Management 3 427 967 2 Technical Assistance to the Establishment of Single Economic Space II 2002 04/2003 Economic Development 1 939 802 3 Basic Functional and Financial Audit of the Institutions of BiH 2001 04/2003 Public Administration 146 513 4 Training of Prosecutors in BiH on the new Criminal Procedure Codes 2002 04/2003 Judiciary and Police 146 255 5 Assessment, design, supervision of blocking illegal border crossings towards Croatia 2001 04/2003 Integrated Border Management 144 900 6 Rebuild part of the Travnik fortress for educational tourism (part of EU-RED) 2002 12/2004 Regional Development 57 433 Total 5 862 870 CROATIA 1 Modernisation of state border surveillance 2004 05/2005 Integrated Border Management 5 000 000 2 Support to land cadastre and land registry reform 2002 05/2003 Economic Development 5 000 000 3 Registration of maritime domain 2003 02/2004 Economic Development 2 000 000 4 Agriculture and Food products 2002 03/2005 Agriculture 1 749 864 5 Support to the Croatian court system. Assistance in the court reform 2002 02/2004 Judiciary and Police 1 000 000 6 Supply for Equipment for the Criminal Intelligence System 2002 12/2004 Judiciary and Police 248 944 7 Zadar City-Mobile Library 2002 07/2005 Democratic Stabilisation 238 630 Total 15 237 438 FORMER YUGOSLAV REPUBLIC OF MACEDONIA 1 Local self-governance. Construction of local roads in 9 municipalities 2004 11/2004 Regional Development 3 474 623 2 Provision of vehicles for border police — Lot 1 2003 10/2004 Integrated Border Management 2 486 726 3 Development of operational capacity to combat money laundering 2002 09/2004 Judiciary and Police 700 000 4 Development of administrative capacity of the Courts 2003 09/2004 Judiciary and Police 148 495 5 TA to support the implementation of IBM 2002 11/2004 Integrated Border Management 55 540 Total 6 865 384 KOSOVO 1 Public Administration Reform — Municipal Development and Social Infrastructure, Technical Assistance 2003 12/2004 Economic Development 11 200 000 2 Programme for minority rights and returns — Reintegration programme in Prizren municipality (Prizren 615 000 EUR) 2004 08/2004 Refugees and IDPs 5 000 000 3 Institutional support to the Ministry of Trade and Industry 2003 04/2004 Economic Development 2 235 760 Total 18 435 760 SERBIA 1 Major Overhaul of Nikola Tesla Unit A5 Power Plant (Grant agreement) 2003 12/2003 Energy 42 721 021 2 Supporting the National Strategy for the Closure of Collective Centres in Serbia 2004 12/2004 Refugees and IDPs 4 443 000 3 Strengthening Quality Management, Capabilities and Infrastructure in the State Union 2003 12/2003 Public Administration Reform 3 559 520 4 Assistance in Establishing an Electricity Market in Serbia 2004 01/2005 Energy 2 723 637 5 Internal Audit and PIFC Phase 2 2003 04/2005 Public Administration Reform 1 870 680 6 Assistance to European Integration Office 2002 04/2002 Public Administration Reform 1 500 000 7 Support of Cross Boundary Initiatives for the IDPs from Kosovo 2004 08/2004 Refugees and IDPs 449 983 Total 57 267 841 OVERALL TOTAL 121 420 659