ACRONYMS AND ABBREVIATIONS

ACP countries : African, Caribbean, Pacific countries

CRIS : Common RELEX Information System

CSP : Country strategy paper

EAMR : External Assistance Management Report

EDF : European Development Fund

EEAS : European External Action Service

ENPI : European Neighbourhood and Partnership Instrument

EuropeAid : EuropeAid Cooperation Office

FLEX : EDF support for fluctuation in export earnings

ROM : Results oriented monitoring

SPSP : Sector policy support programme

EXECUTIVE SUMMARY

I. The European Court of Auditors carried out an audit to assess whether the major devolution of responsibilities for the management of external assistance from European Commission headquarters to its delegations around the world had resulted in improved aid delivery. The audit focused on three central objectives of devolution: to increase the speed of aid delivery, to improve the quality of aid, and to make financial management procedures more robust.

II. The audit covered the period since the completion of the devolution process in 2004 until 2009. The timing of the audit is particularly appropriate as the European Union is currently in the process of another major reform which has important implications for the Commission‘s management of European Union external assistance, the creation of the European External Action Service (EEAS) in accordance with the Treaty of Lisbon.

III. The Court found that devolution has contributed to an improvement in aid delivery. In relation to the three specific objectives:

(a) The Commission has speeded up its delivery of aid, both committing an increased amount of funds to new aid interventions and spending this additional funding sufficiently quickly to avoid the build-up of a backlog of unspent funds.

(b) There are some indications that devolution has contributed to improving the quality of aid in terms of better results. In particular, Commission assessments suggest that the number of poorly performing projects has fallen and the relevance of projects has increased. However, assessing the quality of aid is a difficult process and the Commission’s current systems are not yet sufficiently developed for this purpose.

(c) The Commission has steadily introduced important improvements to its financial management since devolution, even if some weaknesses remain.

IV. Despite these improvements the audit highlighted a number of issues which prevent the Commission from realising the full benefits of devolution.

(a) There is still scope for the Commission to improve its allocation and use of human resources in delegations. In this respect, key issues identified by the audit included the right balance between staff working on aid and staff working on political and trade matters, the skills profile of staff, and the high number of vacancies among contract staff. The number of sectors in which delegations continued to be involved, despite steps to strengthen donor coordination, increases the workload of delegations.

(b) The role of the EuropeAid Cooperation Office (EuropeAid) in providing support to delegations is hampered by the high turnover among its contract staff which makes up 40 % of its workforce. This makes it difficult to build up the required expertise, causes a loss of institutional memory and reduces the efficiency of operations.

(c) Delegations have not carried out sufficient on-the-spot technical and financial monitoring of aid interventions, although the greater opportunity for such monitoring was one of the potential benefits of devolution. Reporting by delegations to EuropeAid does not adequately provide feedback on results or on the soundness of financial management systems.

V. The report concludes with a series of recommendations, which are designed to improve the benefits the Commission can derive from devolution.

INTRODUCTION

1. The European Union is the largest aid donor in the world and the European Commission manages a significant part of this aid, its spending on global aid operations in 2009 amounting to 8440 million euro [1]. Approximately 80 % of its aid is implemented through a devolved management system. "Devolution" involves the delegation of tasks and responsibilities for the management of European Commission financed cooperation activities from the European Commission’s headquarters to 111 offices ("delegations") in partner countries. Devolution was introduced over the period 2002 to 2004 to support a wider reform launched in 2000 to improve the Commission’s management of external assistance. The objectives of devolution and the overall reform process were principally to speed up implementation, increase quality and ensure robust procedures.

2. In 2004 the Court issued a Special Report on "The devolution of EC external aid management to the Commission’s delegations" [2]. The report focused on two main issues: firstly, the Commission’s management of the devolution process and, secondly, whether the devolution of management had achieved its objectives. The report made a positive assessment of the Commission’s management of the devolution process but concluded that in 2004 it was still too early to assess whether the intended results of devolution had been achieved.

3. The main features of devolution, which has now been in operation for more than five years, are as follows (see also Figure 1).

(a) Delegations are responsible for the identification, formulation and implementation of aid interventions, including the contracting and payment stages. Heads of delegations have been made sub-authorising officers and delegations have both operations and finance sections.

(b) EuropeAid remains responsible for the finalisation and modification of financing decisions and financing agreements while the preparatory work is carried out by delegations.

(c) EuropeAid supports delegations by carrying out reviews of the quality of proposed aid interventions at the end of the identification and formulation stages and provides ongoing support at the request of the delegations throughout the lifetime of each aid intervention.

(d) While delegations are responsible for the technical and financial monitoring and evaluation of individual aid interventions, EuropeAid is responsible for the overall monitoring and evaluation of the implementation of the Commission’s development assistance.

(e) The programming of assistance — the establishment of multiannual cooperation strategies with partner countries and the allocation of resources to support them — has not been devolved. The Commission headquarters have continued to be responsible for this [3] although delegations play a greater part in the process than before devolution.

4. Since the major devolution of responsibilities from Commission headquarters to delegations from 2002 to 2004, there have only been minor further transfers of responsibilities. Nevertheless in line with the 2008 Accra Agenda for Action [4] the Commission, along with other donors has committed itself to the principle of further devolution of responsibilities from headquarters to their offices in partner countries to make aid more responsive to the needs of the latter.

5. In the most significant reform to the management of external assistance since devolution, under the Treaty of Lisbon, the European Union is creating a new European External Action Service (EEAS) [5]. Preparations for the setting up of the EEAS were continuing at the time of the audit. Delegations will report to the EEAS and will have a stronger political and diplomatic role than in the past, while remaining responsible for the implementation of development assistance from the EU budget. For the development assistance part of their responsibilities, heads of delegations will continue as subdelegated authorising officers to report to EuropeAid while reporting to the EEAS for their other functions.

FIGURE 1

DIVISION OF RESPONSIBILITIES BETWEEN HEADQUARTERS AND DELEGATIONS IN THE COMMISSION'S DEVOLVED MANAGEMENT SYSTEM

+++++ TIFF +++++

Source: European Court of Auditors’ analysis based on Commission’s strategic documents.

AUDIT SCOPE AND APPROACH

6. The audit set out to answer the overall question:

"Has the devolution of the Commission‘s management of external assistance from its headquarters to its delegations led to improved aid delivery?"

This overall question was addressed by examining three more specific questions.

(a) Has devolution led to improvements in the speed of aid delivery?

(b) Has devolution led to improvements in the quality of aid?

(c) Have robust financial management procedures been implemented?

7. The audit focused on the period since 2004, the year in which the process of introducing devolution was completed and the year of the Court’s previous report on devolution. The audit was carried out from mid-2009 to mid-2010. It focused on the role of EuropeAid and the delegations and did not cover the programming phase and the roles of the Development DG and External Relations DG [6]. It examined the Commission’s devolved management system through documentary review and interviews with key staff in EuropeAid. To assess the functioning of the system at delegation level, the audit also involved visits to five partner countries (Colombia, Ecuador, Ethiopia, Indonesia, Tunisia). In addition, the audit drew on observations relating to the devolved management system contained in the Court’s own Annual Reports and Special Reports produced since 2004.

8. The first part of the report examines whether and to what extent aid delivery has improved. While it is not feasible to assess exactly how far any improvements can be specifically attributed to devolution, the audit sought to identify aspects of the devolved management system which had particularly contributed to improvements.

9. The second part of the report addresses aspects of the devolution which the audit identified as areas where the Commission management needed to be strengthened. The final section of the report presents recommendations on how to do this. The planned reorganisation of EuropeAid following the establishment of the EEAS provides an opportunity for implementing these recommendations.

OBSERVATIONS

DEVOLUTION HAS CONTRIBUTED TO AN OVERALL IMPROVEMENT IN AID DELIVERY

DEVOLUTION HAS CONTRIBUTED TO IMPROVING THE SPEED OF DELIVERY, THE COMMISSION BEING ABLE TO BOTH COMMIT AND ALSO DISBURSE INCREASING VOLUMES OF FUNDING

10. During the 1990s the Commission struggled to spend its development assistance budget which more than doubled over the period. This led to a substantial backlog of unspent funding. Improving the speed of aid delivery was therefore a key objective of devolution.

11. To assess whether the speed of aid delivery had increased, the audit examined the Commission’s performance using the key financial indicators established by EuropeAid (see Box 1). The audit’s assessment included whether the indicators were based on reliable data and together gave a comprehensive picture of the speed of aid delivery. The audit also sought to identify factors which had influenced the speed of aid delivery.

BOX 1

KEY FINANCIAL DATA INDICATORS USED BY EUROPEAID

The following speed-related indicators form part of the EuropeAid Indicators Report which is presented three times a year to the External Relations Commissioners’ group:

(a) the annual amount of aid funding committed, contracted and paid;

(b) level of "old" and "dormant" commitments. This indicator measures how far interventions have significant delays. "Old" commitments are those commitments which have been open for more than six years. "Dormant" commitments are those commitments which are less than five years old which have had no financial transactions over the last 24 months;

(c) contracting on financing agreements subject to the so-called "D + 3" rule. This indicator also relates to the speed with which aid interventions are implemented, focusing on the risk that interventions will lose funds if they are not contracted within three years of the financing agreement signature.

12. The audit concluded that the financial information used for the indicators was reliable and that, overall, there had been an increase in the speed of aid delivery. The audit’s conclusion on the reliability of the financial information is largely based on the work carried out in the framework of the Court’s financial audits. The following paragraphs set out the audit’s assessment concerning the increase in speed.

13. The volume of aid under implementation was significantly higher in 2009 than in 2004. The amount of development aid committed in 2009 was 42 % higher than in 2004, the amount contracted in 2009 increased by 45 % compared with 2004 and payments rose by 30 % over the same period. While there have been some fluctuations in amounts of commitments, contracts and payments between the years (see Figure 2), the underlying trend has been clearly upwards.

FIGURE 2

AMOUNT OF EXTERNAL ASSISTANCE COMMITTED, CONTRACTED AND PAID 2004–09

+++++ TIFF +++++

Source: EuropeAid Indicators Reports.

14. Despite the increasing volume of aid, the Commission was able in 2009 to bring down the volume of old commitments to the level of 2004. While the figure had been approximately 50 % higher in the years 2006 to 2008 than in 2004, this build-up of uncompleted aid interventions was eliminated in 2009. The volume of dormant commitments fell over the same period by almost 25 % (see Figure 3). This indicates that fewer new aid interventions committed are experiencing significant delays.

15. There has nevertheless been a small decline in the speed at which commitments have been contracted since 2004 (see key indicator, Box 1 (c)). Whereas approximately 4 % of the amount of 2004 commitments were uncontracted by 2007 and therefore could no longer be used, the corresponding figure in 2009 for 2006 commitments was 6 %.

FIGURE 3

AMOUNT OF OLD AND DORMANT COMMITMENTS 2004–09

+++++ TIFF +++++

Source: EuropeAid Indicators Reports.

16. The Court’s recent Annual Reports largely confirm the overall improvement in the speed of aid delivery compared with the situation before devolution where the Court was particularly critical of the build-up in unspent funding, notably in relation to the EDF.

17. The Court’s main concern with the comprehensiveness of the indicators used is that they largely focus on the speed of budget execution. However, budget execution indicators overstate the speed of actual aid implementation and insufficient use is made of other indicators to monitor the time required for aid implementation. This is a particularly important issue for some of the Commission’s aid modalities which involve funds being paid into intermediate accounts before final utilisation. For example:

(a) As much as 45 % of the development assistance from the EDF and 25 % from the general budget is programmed in country strategy papers to be disbursed as budget support over the period 2007–13. Although for the Commission its transfer of this funding to the treasury account of the partner country is the final payment, some time may elapse before it is actually fully spent by the national authorities as part of their budget. In addition, there may be delays in the country implementing its annual recurrent budget or in it implementing funds it has earmarked for the capital budget to finance investment projects.

(b) Increasingly, funds are being transferred by the Commission under joint management arrangements to international organisations or for international initiatives to finance programmes which these organisations then often implement through intermediaries such as NGOs. Commission funding is generally made in the form of large advances, and it can take some time before these advances are fully utilised. The Court’s Special Report on health assistance to Sub-Saharan Africa highlighted problems in this area in relation to the Global Fund against Aids, Tuberculosis and Malaria [7]

(c) For many projects, particularly in African, Caribbean, Pacific countries (ACP) [8], the Commission makes advance payments to finance the annual work programmes of partner countries under decentralised management. The implementation and therefore disbursement of these programmes can be subject to delays.

18. Devolution has facilitated the speeding up of aid implementation in several ways. Having Commission finance and contract staff in the delegations means that they, as well as the increased numbers of operational staff, are able to work more closely with national administrations to prevent and solve procedural problems more quickly. The existence of finance and contracts sections alongside operations sections also increases the speed with which financial management issues arising within the Commission services can be resolved. The significant measures taken by EuropeAid to streamline and standardise procedures have also facilitated financial management in delegations.

19. Nevertheless, other factors apart from devolution have also been important in speeding up aid delivery. The Commission’s increasing use of budget support, a generally faster disbursing instrument than the traditional project approach, has been one aspect of this. The Commission’s greater use of international organisations has similarly been a factor. The stricter requirements of the 2002 financial regulation concerning deadlines for the contracting of aid assistance have also provided further incentives for aid to be delivered quickly to avoid significant amounts being cancelled. It is not possible to quantify the degree to which devolution and the degree to which these other factors have contributed to speeding up delivery.

ALTHOUGH SOME INDICATORS POINT TO IMPROVEMENTS IN THE QUALITY OF AID THE COMMISSION’S SYSTEMS FOR ASSESSING QUALITY DO NOT ALLOW FIRM CONCLUSIONS TO BE DRAWN

20. The overarching objective of the Commission’s development assistance is poverty reduction. The contribution it makes in this respect in terms of outcomes and impact and their long-term sustainability are the ultimate determinants of the quality of its aid.

21. To assess whether devolution had improved the quality of aid, the audit examined the Commission’s performance using the key quality indicators established by EuropeAid (see Box 2). The audit’s assessment included whether the indicators were reliable and together gave a comprehensive picture of the quality of aid. The audit also sought to identify factors relating to devolution which had influenced the quality of aid delivery.

22. The audit found that some Commission indicators pointed to improvements in the quality of aid in terms of results. However, the assessment of quality is an inherently difficult exercise and the Court considers there are shortcomings in the Commission’s assessment system both in relation to the usefulness of the data as well as the coverage and relevance of the indicators.

BOX 2

KEY QUALITY DATA INDICATORS USED BY EUROPEAID

EuropeAid uses four indicators for assessing the quality of aid interventions:

(a) quality measuring during implementation: overall assessment of project performance based on scores for five standard evaluation criteria (relevance, efficiency, effectiveness, impact, sustainability) [9];

(b) quality measuring during implementation: detailed assessment of projects according to five standard evaluation criteria. The indicator represents the percentage of projects, which are rated as "very good" or "good" for each criterion;

(c) quality measuring during project preparation: percentage of project proposals assessed as requiring redesign;

(d) amount of funds committed to general budget support and sector policy support programmes.

23. The main tool used by the Commission for assessing the overall quality of its interventions is the Results Oriented Monitoring System (ROM). The ROM system involves external monitors under contract with the Commission visiting projects on one or more occasions during a project’s lifetime. The monitors assess the overall performance of the projects and also score their performance in relation to the five standard evaluation criteria (see first two indicators in Box 2). The ROM results indicate that there has been an improvement in the overall performance of projects since 2004 on the basis that the percentage of non-performing projects has fallen steadily from 11 % in 2004 to 6 % in 2009 (see Figure 4).

FIGURE 4

PERCENTAGE OF NON-PERFORMING PROJECTS ACCORDING TO ROM ASSESSMENTS 2004–09

+++++ TIFF +++++

Source: EuropeAid Indicators Reports.

24. The ROM assessments also point to a general improvement in relation to some of the standard evaluation criteria (see Figure 5) [10].

(a) Relevance, the criterion where projects scored the highest, increased from 68 % to 82 % between 2005 and 2009 although most of this increase took place from 2008 to 2009. Impact also increased significantly from 69 % in 2005 to 78 % in 2009.

(b) Efficiency and sustainability remained at or slightly above their 2005 levels.

(c) Effectiveness was rated in 2009 below the 2005 level, falling from 65 % to 61 %, reversing a trend of steady improvement up to 2008. It is difficult to reconcile this decrease in effectiveness with the assessment that there has been a significant rise in impact for the same projects (see paragraph 24 (a)).

(d) The significant changes in the scores for relevance and effectiveness from 2008 to 2009 are due in part at least to a change in the methodology for carrying out the ROM assessments. This highlights that the assessment of quality within the ROM system is necessarily dependent on the specific methodology used.

FIGURE 5

ROM ASSESSMENT OF PROJECT RELEVANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY 2005–09

+++++ TIFF +++++

Source: EuropeAid Indicators Reports.

25. However, there are several general factors which limit the use to which the ROM system can be put for measuring improvements in the overall effectiveness of aid delivery. These largely stem from the fact that ROM was not originally designed for the purpose of measuring the overall performance of the Commission’s aid portfolio. Rather it was intended to monitor individual projects during their implementation with a view to identifying timely corrective actions to help ensure they achieved their planned results. Key limitations reducing ROM’s usefulness as an indicator for the overall aid portfolio are as follows.

(a) Since ROM very largely involves monitoring visits to ongoing projects, its reliability varies according to the criterion assessed. Its assessment of relevance is likely to be reliable since this criterion can be assessed before and during project implementation. Efficiency too can be assessed during implementation. However, for the most important area of results the ROM system can only assess the potential effectiveness, impact and sustainability of projects, not their actual performance in these areas which can only be judged after the end of the project.

(b) It is not a reliable instrument for making comparisons between years. This is because the sample of projects selected for monitoring in a given year is not established on a statistical but rather on a judgemental basis.

(c) ROM’s coverage of aid interventions is limited. The ROM methodology was designed for projects and remains very largely focused on this type of intervention. However, only approximately half the Commission’s aid is still delivered through projects because of the growth in new forms of aid, in particular budget support which is not covered by the ROM. The Commission is in the process of testing a ROM methodology for sector policy support programmes (SPSP), including sector budget support, but as yet has no ROM methodology for assessing general budget support.

26. To address the problem that the ROM assessments, because they are undertaken during implementation, only address potential effectiveness, impact and sustainability, the Commission aims to carry out approximately 10 % of its ROM assessments after project operations have closed. However, these assessments are not used to provide a further quality indicator.

27. The limitations of the ROM system as a tool for assessing quality have not been adequately compensated for by the use of other tools. The Commission’s headquarters’ aid evaluation unit carries out through independent consultants a large number of geographical, sectoral, thematic and instrument evaluations. The delegations also contract consultants to carry out detailed evaluations of individual aid interventions. However, neither the assessments in the evaluations carried out by the Commission’s headquarters’ aid evaluation unit nor by delegations are quantified which means they cannot be used as part of the quality measuring system. In addition, there is no systematic assessment and scoring of programmes and projects by delegation staff at the end of the programme/project (see also paragraph 56).

28. The EuropeAid quality indicator for the project preparation stage (third indicator in Box 2) shows that the percentage of project proposals submitted by delegations which were assessed by EuropeAid as requiring redesign before they could be financed by the Commission fell from 20 % in 2005 to 15 % in 2009.

29. The Commission also uses the percentage of commitments allocated to general budget support and sector policy support programmes, which are mainly implemented through sector budget support, as a quality indicator (fourth indicator in Box 2). This is because they are the aid delivery mechanisms which the 2005 Paris Declaration [11] considers to be most suitable for promoting the key principles likely to improve aid effectiveness. The indicator shows a strong upward trend since 2004 in the proportion of aid being channelled through budget support. However, this indicator does not demonstrate the ultimate quality of aid in terms of its results and impact on poverty reduction [12].

30. The Court’s audit found that devolution had contributed to improved quality in the following ways.

(a) It has enabled the Commission to develop through the delegations a greater knowledge and understanding of partner country circumstances and has increased opportunities for dialogue with the national authorities and other local stakeholders, including donors represented in the country. This has contributed to improving the relevance of Commission interventions, including building up more ownership of the interventions by the partner countries, and also improving their efficiency through better coordination with other donors.

(b) The Commission’s strengthened in-country presence has made it easier to monitor its projects, enabling better identification of underperforming projects and the necessary corrective actions.

(c) EuropeAid’s quality support group mechanism and thematic support units have been an important complement to the devolved management of assistance by delegations. They reflect the importance paid by EuropeAid to quality issues and have provided support to, and control over, delegations during the crucial preparatory phases of projects and programmes as well as assistance during subsequent implementation. Since devolution, screening of project proposals has steadily increased to cover all proposed aid interventions eligible for such screening.

31. Nevertheless, as observed in the second part of the report, there remains a need for the Commission to improve quality by strengthening its delegations’ capacity and further improving their dialogue and monitoring as well as the support provided by EuropeAid. This is confirmed by the Court’s own performance audits of development assistance carried out in recent years (see Annex) which also show that despite some improvements there is still considerable scope for making Commission aid interventions more effective.

THE FINANCIAL MANAGEMENT PROCEDURES IMPLEMENTED SINCE DEVOLUTION ARE INCREASINGLY ROBUST

32. One of the major initiatives of devolution was the transfer of much of the financial management of aid from the Commission headquarters to delegations. Heads of delegations were made subdelegated authorising officers and a "Finance and contracts section" was established within delegations.

33. To assess how far the Commission had established and implemented robust financial management procedures for the devolved management of aid, the audit used the Court’s Annual Reports since 2004. A key objective of these reports is to assess the soundness of the Commission’s supervisory and control systems, including the level of legality and regularity errors in commitments and payments made by the Commission.

34. EuropeAid has made significant progress in improving the supervisory and control systems in relation to its devolved management since 2004, even if the Court considers that overall these systems are still only "partially effective". In 2008 the Commission established a comprehensive control strategy and it has continued to introduce improvements as it implements the strategy.

BOX 3

CRITERIA FOR ASSESSING THE ROBUSTNESS OF FINANCIAL MANAGEMENT PROCEDURES

Each year the Court assesses the supervisory and control systems of EuropeAid and all other Commission Directorates-General. Systems are classified as being "effective" in mitigating the risk of error in transactions, "partially effective" (when there are some weaknesses affecting operational effectiveness) or "not effective" (when weaknesses are pervasive and thereby completely undermine operating effectiveness).

35. The Court’s Annual Reports since 2004 have found that the Commission has made particular progress in relation to the following aspects of the supervisory and control systems.

(a) The Commission has made significant efforts in establishing clear, harmonised financial procedures and in organising guidance manuals and training for both financial and operational sections of delegations.

(b) A key feature of the Commission’s control system is its use of external audits for projects [13]. The Court found that both the quantity and quality of these audits had significantly increased since 2004. Moreover, the Commission has developed a special module within its computerised information system for external assistance (CRIS [14]) for delegations to communicate the results of audits to Commission headquarters for information and subsequent analysis.

(c) The Commission’s risk assessment framework has progressively improved, partly thanks to a greater involvement of delegations in this exercise and to better analysis of reports issued by EuropeAid’s Internal Audit Capability, the Commission’s overall Internal Audit Service and contracted external auditors.

(d) The Court’s reports have tended to be critical of the way the Commission has assessed the eligibility of partner countries for budget support programmes and disbursements. Nevertheless, the reports also recognise that there has been a steady improvement both in the clarity of the frameworks used by the Commission for assessing the eligibility criteria and in the quality of the actual assessments of eligibility made by the delegations.

36. Despite the progress made, the Court has continued to find weaknesses in supervisory and control systems, notably at the level of delegations. These primarily concern how the delegations address shortcomings in the financial management and controls of key actors, involved in EU development assistance, in partner countries. These include the national organisations that implement the aid, the supervisors of infrastructure projects, and, for the EDF, the national authorising officers’ services (see also paragraph 57).

37. Weaknesses in these areas have been the main source of errors identified by the Court in its audit of the legality and regularity of the Commission’s financial transactions. For the Commission policy area which includes development assistance financed from the EU general budget, the Court has consistently had to report a material level of error for payments estimated at between 2 % and 5 %. This has generally been the case for the EDF too although in 2009 the Court concluded that payments were free from material error.

38. A key aspect of the Commission’s work on further improving its control strategy and tackling these weaknesses is an assessment of the strategy’s costs and benefits. This has proved a significant challenge for several reasons. The estimates of costs thus far produced by the Commission have been hampered by the inherent difficulties in defining what actually constitutes a control and then assessing the staff time spent on implementing those controls.

WHILE DEVOLUTION HAS CONTRIBUTED TO IMPROVING THE AID DELIVERY, IT IS NOT BRINGING ALL THE BENEFITS IT COULD, ESPECIALLY IN RELATION TO IMPROVING QUALITY

39. The first part of the report concluded that devolution has contributed to improving aid delivery but also found that there was still a need to strengthen aid delivery, particularly in relation to the quality of aid. This second part of the report highlights areas identified by the audit which need to be addressed in order to better realise the potential benefits of devolution.

THERE IS STILL SCOPE FOR THE COMMISSION TO IMPROVE ITS ALLOCATION AND USE OF HUMAN RESOURCES IN DELEGATIONS

40. The transfer under devolution of major new responsibilities from the Commission headquarters required that delegations be granted adequate resources, particularly staff, to carry out these responsibilities. The Commission recognised the need to significantly increase staffing levels in delegations and over the period 2001 to 2005 assigned approximately 1500 additional staff to delegations. This was done by transferring around 450 staff from EuropeAid and recruiting more than 1000 contract staff [15]. However, in 2007 the Commission committed itself to a zero growth policy for its staff until 2013 which has increased the importance of making the best allocation and use of existing resources if the full benefits of devolution are to be realised.

41. EuropeAid has highlighted the risks presented by its human resources situation to the effective delivery of aid in its recent annual activity reports. The Court has also raised the issue of the adequacy of staff resources and its impact on the quality and financial management of aid on several occasions, notably in its Annual Reports.

42. Over the period 2005 to 2008 the composition of delegation staffing changed as a result of a greater priority being given to strengthening delegation’s political and trade functions. Heads of delegation themselves are increasingly preoccupied with political and trade issues and this tendency is likely to grow with the establishment of the EEAS. In a major redeployment exercise in 2008, 56 posts of officials were transferred from operations sections to political and trade sections, which reduced the aid management capacity of delegations. Following the 2008 redeployment exercise, posts for officials in the aid operations sections were replaced by less senior posts for contract and local staff.

43. At the end of 2009, approximately 20 % of delegation staff working on aid management were officials, 30 % were contract staff, and approaching 50 % were local staff. Contract and local staff have a valuable role to play in supplying specialist and local knowledge although both staff groups tended to have less access to training opportunities which reduced their potential contribution. At the end of 2009 the overall vacancy rate for contract agents in delegations was 14 % and was recorded in EuropeAid’s central risk register as a critical risk for 2010. The high vacancy rate was mainly due to difficulties recruiting contract staff with appropriate expertise, particularly for working in hardship countries.

44. The skill composition of staff is a further issue facing delegations. Delegations are still adjusting to the new skills requirements entailed by the Commission’s shift from the traditional project approach to aid delivery to the use of budget support which requires expertise in macroeconomics and public finance management. The necessary skills to conduct policy dialogue are also an increasingly important requirement for all aid modalities but particularly for budget support programmes. While the Commission has made significant efforts to develop training courses for its staff on new aid mechanisms, these courses are only of a short duration. In practice they mainly serve to raise the awareness of staff on key topics but do not give them the in-depth expertise required to bring a significant value added. Recent performance audits by the Court on health and education have also pointed to limited expertise in these more traditional aid areas despite their centrality to the millennium development goals and poverty reduction.

45. While EuropeAid has undertaken various quite detailed workload analyses, these have largely been used to assess individual requests of heads of delegation for additional staff and have not concluded on the overall adequacy of staffing levels between delegations. There has also been no assessment, in the build-up to the establishment of the EEAS, to determine the optimal balance between political/trade staff posts and aid management posts in delegations. Similarly the Commission has also not mapped the available existing expertise in delegations in order to make sure that staff with specific training and experience are assigned to the delegations where they can be put to best use.

46. The Commission has taken a number of steps intended to allow aid to be managed more efficiently through existing staff resources. For example, it has sought to increase the size of the individual projects/programmes it funds, in order to reduce the administrative burden of managing a large number of small projects. Increased use of budget support programmes has facilitated large disbursements although the main benefit of budget support is often the opportunities it provides for dialogue in relation to the programme objectives. Effective dialogue requires intensive staff inputs. For general budget support programmes, the objectives of which typically cover macroeconomic stability, public finance and health and education, delegations often do not have the resources to be involved in dialogue in all these areas.

47. The Commission’s approach of requiring the funding from its country strategy papers (CSPs) to be concentrated on no more than two "focal" sectors in order to achieve greater impact in principle has also the benefit of reducing the need for diverse expertise in the delegations. On the other hand, in addition to these two focal sectors, delegations typically also manage aid interventions in a number of additional areas which place a significant additional workload on them (see Figure 6).

(a) Delegations in ACP countries also have to manage general budget support programmes when countries are eligible for them.

(b) CSPs generally also include "non-focal" sectors to finance smaller interventions outside the two main focal sectors.

(c) Delegations are frequently involved in the supervision of regional projects.

(d) Delegations also have to implement a series of smaller projects funded from the horizontal thematic programmes [16].

(e) Delegations are often also involved in the management of other interventions funded from other specific aid instruments (see Figure 6).

FIGURE 6

AREAS OF AID INTERVENTION OF A TYPICAL DELEGATION

+++++ TIFF +++++

Source: European Court of Auditors’ analysis based on Commission’s documents.

48. The continued involvement in a large number of sectors of intervention runs contrary to the 2007 EU Code of Conduct on Complementarity and the Division of Labour in Development Policy [17]. This calls on EU donors to limit their active involvement in a partner country to a maximum of three sectors [18] and to redeploy their other activities, possibly through delegated cooperation/partnership arrangements with other donors. It also provides a framework for drawing on the expertise of designated "lead donors" for individual sectors. The Commission drew up the policy document [19] which formed the basis for the Code of Conduct and has been seeking to promote its implementation. Nevertheless, considerable progress remains to be made in this respect including through greater joint programming. It is clear that reducing the number of sectors the Commission is actively involved in and further strengthening coordination with other donors, in particular other EU Member States, can help make the best use of existing staff resources.

49. Given the challenges faced by delegations in meeting their current levels of responsibilities and the increased volumes of aid being managed, there appears little scope for further devolution of responsibilities, even if the Commission is in principle favourable towards this on the basis of the Accra Agenda for Action.

THE COMMISSION’S HEADQUARTERS HAVE ONLY A LIMITED CAPACITY TO SUPPORT DELEGATIONS IN IMPROVING QUALITY DESPITE THE SIGNIFICANT EFFORTS MADE BY EUROPEAID

50. A central feature of the devolved management system is that EuropeAid should provide the expertise to enable the delegations to carry out their responsibilities. EuropeAid has taken important measures to fulfil this role. These include establishing a directorate specifically for "Quality of Operations" while its geographical directorates also provide support. The quality support group mechanism which screens proposed interventions at the end of the identification and formulation stages is chaired by each geographical director and includes both the geographical services and the Quality of Operations Directorate.

51. However, EuropeAid faces significant difficulties in building up the necessary expertise to properly play its central support role because of the high turnover rate amongst its staff. This is because approximately 40 % of its staff are contract agents [20] all of whom have a maximum three-year non-renewable contract as required by the Staff Regulations. In fact many staff leave before the three years have expired because of the need to find new employment. The high turnover rate does not provide a sound foundation for a strong central support function and weakens both institutional memory and the efficiency of operations.

FIGURE 7

EUROPEAID QUALITY OF OPERATIONS DIRECTORATE THEMATIC SUPPORT UNITS

+++++ TIFF +++++

Source: European Court of Auditors’ analysis based on Commission’s documents.

52. The Quality of Operations Directorate includes six thematic units which between them cover more than 20 different sectors (see Figure 7). The wide range of sectors covered does not reflect the European Consensus on Development that the Commission should focus on fewer sectors based on an analysis of its comparative advantage. There are only normally between two and six staff working on each sector to provide support to the 111 EU delegations worldwide. This means that the directorate has only limited capacity both for participating in the formal quality support review mechanism and for additional ad-hoc support to delegations at the latter’s specific requests. The low budget available for travelling to delegations to provide support represents a further constraint on the directorate in fulfilling its role.

53. EuropeAid has made significant efforts to develop a range of useful guidelines, manuals and other materials as another way to support delegations. The main area which has not received sufficient attention was guidance on policy dialogue. This is despite the fact that policy dialogue is a key means to improve aid results and that a major advantage of devolution is the increased possibilities it gives the Commission for policy dialogue. This was especially the case in relation to the Commission’s guidelines on general budget support despite the particular opportunities presented by budget support for policy dialogue. The Commission’s project cycle management guidelines, which date back to 2004, scarcely address the role which policy dialogue can also play in the context of project interventions. The Court’s audit on the new European Neighbourhood and Partnership Instrument (ENPI) also pointed out the lack of specific guidance for structured dialogue with the partner country [21].

MONITORING, REPORTING AND EVALUATION ARE NOT YET SUFFICIENTLY DEVELOPED IN RELATION TO AID QUALITY AND FINANCIAL MANAGEMENT

54. Devolution led to delegations being made responsible for the technical and financial monitoring of individual programmes and projects. As already observed (see paragraph 29), having operational and financial sections in country has given the Commission considerably increased opportunities for monitoring, particularly for on-the-spot project visits, with a view to improving the quality and financial management of its interventions. As the Commission headquarters remain responsible for the overall monitoring of aid delivery, it is essential that an effective reporting system from delegations to EuropeAid is in operation. Evaluations undertaken after aid interventions have been completed also have an important role to play in generating information for both headquarters and delegations on the quality of aid interventions.

55. In practice, delegations have not taken full advantage of the greater opportunities provided by devolution for on-the-spot monitoring of the technical and financial implementation of projects and programmes. The Court’s audit visits to delegations indicated that monitoring visits are not systematic but tend to be more on an ad-hoc basis, depending on the time and budget available. Frequently the processing of documentation relating to contracts and payments takes priority for both operational and financial staff and field visits are thus neglected. This tendency is exacerbated by the fact that delegations have not developed strategic monitoring plans to ensure that projects are visited regularly.

56. As far as on-the-spot monitoring of the quality of aid in terms of results is concerned, the ROM system is the main means used by the Commission. The ROM system is based on external consultants carrying out short missions to the partner countries to make on-the-spot visits to projects and reporting back to the delegation staff responsible for the project. The Commission has not established a system for its own staff to visit projects during and after implementation and to draw up structured, quantified results focused reports. This is despite the Commission stressing the importance of its staff having a results oriented approach and despite delegation staff having the best knowledge of the aid interventions and the country circumstances.

57. As far as on-the-spot monitoring of financial management is concerned, the Court’s Annual Reports regularly point to the financial management weaknesses of implementing organisations and the resultant legality and regularity errors they make. The reports also refer to weaknesses in the control of infrastructure projects by works supervisors. In both cases the limited on-the-spot monitoring visits by delegation staff, particularly financial staff, mean that financial management is not sufficiently monitored and the remedial action necessary to address weaknesses or errors is delayed or not taken at all.

58. This audit and the recent Court audit of EDF regional cooperation have identified particular weaknesses in the technical and financial on-the-spot monitoring by delegations of thematic projects (see paragraph 46) and regional projects [22]. This reflects the difficulties that delegations face in meeting all priorities given the staff constraints. In the case of regional cooperation it also reflects the lack of clear definition of responsibilities between delegations.

59. The main reporting tool used by the Commission between delegations and EuropeAid is the External Assistance Monitoring Report (EAMR) which delegations are required to prepare every six months. The drawing up of these reports requires a considerable time input from the delegations. However, these reports largely describe activities carried out and implementation problems encountered. Moreover, information provided to a certain extent overlaps with information available in CRIS. They provide little indication on the actual results of the aid, either in terms of assessments made by the delegation itself or by reporting on the results of evaluations made after aid interventions have closed. At the same time they do not adequately address financial management issues for which the head of delegation as subdelegated authorising officer is responsible.

60. The evaluations carried out by the Commission’s headquarters’ aid evaluation unit (see paragraph 27) are available to delegations through EuropeAid’s public website. However, the findings of the evaluations contracted by delegations are not systematically made use of by Commission headquarters because there is no central database for holding and analysing such evaluations.

CONCLUSIONS AND RECOMMENDATIONS

OVERALL CONCLUSIONS

61. Devolution has contributed to improving aid delivery as a key part of the Commission’s wider reform of its management of external assistance over the last decade. The improvements are most evident in the areas of the speed of delivery and the robustness of the financial management procedures. Some indicators also point to improvements in the quality of aid, although the Commission system for measuring the quality of aid is not yet sufficiently developed to allow firm conclusions to be drawn.

62. The audit, nevertheless, also found that several shortcomings in the working of the devolved management system prevent the full potential benefits of devolution from being realised. Delegations face a number of challenges to make the best allocation and use of existing resources. At the same time EuropeAid has difficulty in providing adequate support to delegations to ensure aid interventions are of a high quality, despite the significant efforts it has made in this respect. The Commission has not fully taken advantage of the greater opportunities arising from devolution for improving aid through in-country dialogue and on-the-spot monitoring.

63. The reorganisation of EuropeAid following the establishment of the EEAS provides an opportunity for the Commission to tackle the issues highlighted in this report.

SPECIFIC CONCLUSIONS AND RECOMMENDATIONS

64. The Commission has been able to both commit and disburse more aid since the advent of devolution. As a result the Commission has avoided the risk that larger allocations of aid might lead to a backlog of unspent funding. However, some time can elapse between the Commission payment and the actual implementation of the aid, and this is something not sufficiently addressed by the Commission’s current indicators.

RECOMMENDATION 1

The Commission should complement budget execution as an indicator of speed with other indicators to allow it to better assess the speed of actual aid implementation.

65. The Commission has already made significant efforts to improve the quality of its aid and some indicators point to the achievement of better results. Nevertheless, the Commission’s systems for measuring quality are not very robust and the Commission places too much reliance on its external monitoring system (ROM) for this purpose. Delegations themselves do not carry out assessments of the effectiveness of their aid interventions and the evaluations contracted by delegations are not adequately exploited for this purpose.

RECOMMENDATION 2

The Commission should take steps to improve its system for measuring the quality of its external aid as follows:

(a) delegations should carry out assessments of the effectiveness of all completed aid interventions;

(b) the independent evaluations contracted by the Commission should include quantitative assessments to enable the effectiveness of aid interventions to be more clearly assessed.

66. EuropeAid has made significant progress in improving the robustness of its financial management procedures even if some weaknesses remain. Since 2008 it has been implementing a comprehensive control strategy and is further refining it through an ongoing analysis of the costs and benefits of its controls.

RECOMMENDATION 3

The Commission should monitor closely the implementation of its new control strategy, both at the level of EuropeAid and the delegations, in order to obtain reliable and complete information to assess its costs and benefits.

67. The Commission faces a number of challenges to ensure the best allocation and use of existing staff resources in its delegations in a context of a zero growth policy for human resources. These challenges include the balance of staffing between aid management and other functions, the high vacancy rate amongst contract staff, and the need for its staff to develop skills for managing new aid modalities. The aid managed by individual delegations continues to cover a wide range of areas, despite the efforts the Commission has made to improve coordination with other donors, and this puts further pressure on delegations’ resources.

RECOMMENDATION 4

In order to help tackle the human resources challenge faced by EU delegations, the Commission should:

(a) carry out an analysis of:

(i) the workload within delegations to conclude on the adequacy of staffing levels between delegations;

(ii) the balance of delegations’ staffing between aid management and other functions;

(iii) the expertise of delegation staff in order to assign staff to delegations where their skills are most needed and identify skill gaps to be filled;

(b) make further efforts to reduce the number of areas of intervention in which it is actively involved, in particular through closer coordination with EU Member States;

(c) make greater use of Member States’ expertise in partner countries.

68. EuropeAid has made significant efforts to support delegations in improving the quality of aid interventions. However, it is hindered in this by a high turnover rate due to the fact that 40 % of its staff are on three-year non-renewable contracts. This makes it difficult to build up and retain a solid foundation of expertise and is inefficient. EuropeAid has developed a wide range of guidance for delegations although more support is still required in the key area of policy dialogue.

RECOMMENDATION 5

In order to strengthen support to delegations:

(a) the Commission should take steps to reduce the high rate of turnover among staff in EuropeAid;

(b) EuropeAid should develop improved guidance for policy dialogue.

69. Delegations have not made sufficient use of the opportunities provided by devolution for on-the-spot monitoring visits to aid interventions with a view to improving their quality and financial management and reporting on results. In general, delegation reporting to EuropeAid has focused too much on activities and duplicates reporting in CRIS. Commission headquarters do not make sufficient use of the external, independent evaluations contracted by delegations.

RECOMMENDATION 6

The Commission should strengthen the monitoring, reporting and evaluation within the devolved management system by:

(a) requiring delegations to systematically carry out technical and financial monitoring visits to projects within the framework of a monitoring strategy for each delegation;

(b) focusing the internal reporting system more on the results achieved by the aid interventions;

(c) in the context of setting up the EEAS, ensuring the effective implementation of the new legal provisions proposed for holding heads of delegations accountable as subdelegated authorising officers.

This Report was adopted by Chamber III, headed by Jan KINŠT, Member of the Court of Auditors, in Luxembourg at its meeting of 1 February 2011.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] 5371 million euro was spent from the EU general budget and 3069 million euro from the European Development Funds (EDFs).

[2] Special Report No 10/2004 on the devolution of EC external aid management to the Commission delegations (OJ C 72, 22.3.2005).

[3] The Development DG has been responsible for programming of assistance in ACP countries and the External Relations DG for programming of assistance in other countries.

[4] The Accra Agenda for Action was drawn up at a meeting of partner countries and donors in Accra in September 2008 as part of the review of progress in implementing the commitments of the 2005 Paris Declaration on Aid Effectiveness.

[5] Article 27(3) of the Treaty on European Union as amended by the Treaty of Lisbon.

[6] The Enlargement DG was not included within the scope of this audit.

[7] See Special Report No 10/2008: EC development assistance to health services in Sub-Saharan Africa (paragraphs 26 and 27).

[8] In 2008 approximately 16 % (300 million euro) of the EDF was contracted to be disbursed through this mechanism (see AIDCO Annual Activity Report for 2008, Annex 5).

[9] These five criteria are standard evaluation criteria for development as recommended by the OECD Development Assistance Committee (DAC).

[10] It should be noted that there was a change in the ROM methodology in 2009 (see paragraph 24 (d)).

[11] The 2005 Paris Declaration on Aid Effectiveness identified country ownership, alignment, harmonisation, managing for results and mutual accountability as five key principles for making aid more effective. The Declaration was signed by more than a hundred donors and partner countries, including the European Commission.

[12] See the Court’s Special Report No 11/2010 on the Commission's management of general budget support in ACP, Latin American and Asian countries, paragraph 83.

[13] External audits are principally contracted by delegations at the end of projects before the final payment is made to ensure that project expenditure has been legal and regular.

[14] Common RELEX Information System.

[15] In 2001, delegation staff amounted to approximately 900, while staff in EuropeAid amounted to approximately 1350.

[16] There are five thematic programmes under the Development and Cooperation Instrument: non-state actors and local authorities, investing in people, food security, environment and sustainable management of natural resources, and migration and asylum.

[17] EU Code of Conduct on Complementarity and Division of Labour in Development Policy. Conclusions of the Council and Representatives of the Governments of the Member States 9558/07, Brussels, 15.5.2007.

[18] The three sectors do not include general budget support programmes.

[19] Communication from the Commission to the Council and European Parliament: "EU Code of Conduct on Division of Labour in Development Policy", COM(2007) 72 final, Brussels, 28.2.2007.

[20] In some geographical directorates the proportion is as high as 50 %, in the Quality of Operations Directorate the level is 35 %. This compares with an average of 15 % across the Commission as a whole.

[21] See Court’s Special Report No 13/2010: "Is the new European Neighbourhood and Partnership Instrument successfully launched and achieving results in the Southern Caucasus (Armenia, Azerbaijan and Georgia)?".

[22] See Special Report No 18/2009 on the effectiveness of EDF support for regional economic integration in East Africa and West Africa, in particular Box 2.

--------------------------------------------------

ANNEX

RECENT PERFORMANCE AUDITS (2007–10) OF DEVELOPMENT ASSISTANCE BY THE EUROPEAN COURT OF AUDITORS

Special Report No 6/2007 on the effectiveness of technical assistance in the context of capacity development.

Special Report No 6/2008 on European Commission rehabilitation aid following the Tsunami and Hurricane Mitch.

Special Report No 9/2008 on the effectiveness of EU support in the area of freedom, security and justice for Belarus, Moldova and Ukraine.

Special Report No 10/2008 on the EC development assistance to health services in Sub-Saharan Africa.

Special Report No 4/2009 on the Commission’s management of non-state actors’ involvement in EC development cooperation.

Special Report No 15/2009 on the EU assistance implemented through United Nations organisations: decision-making and monitoring.

Special Report No 18/2009 on the effectiveness of EDF support for regional economic integration in East Africa and West Africa.

Special Report No 11/2010 on the Commission’s management of general budget support in ACP, Latin American and Asian countries.

Special Report No 12/2010 on the EU development assistance for basic education in Sub-Saharan Africa and South Asia.

Special Report No 13/2010 on the audit ‘Is the new European Neighbourhood and Partnership Instrument successfully launched and achieving results in the Southern Caucasus (Armenia, Azerbaijan and Georgia)?’.

--------------------------------------------------