# EUR-Lex - 52011SA0003 - EN

*|*

# 52011SA0003

**Special Report No 3/2011 ‘The efficiency and effectiveness of EU contributions channelled through United Nations Organisations in conflict-affected countries’ **

*

*

ABBREVIATIONS AND GLOSSARY

EC : European Commission

EU : European Union

EuropeAid : Cooperation Office of the European Commission

FAFA : Financial and Administrative Framework Agreement between the European Commission and the United Nations

SMART : Specific, measurable, attainable, realistic and time-based

TFEU : Treaty on the Functioning of the European Union

UN : United Nations

UNDP : United Nations Development Programme

UNRWA : United Nations Relief and Works Agency for Palestine Refugees in the Near East

Effectiveness : Attaining the specific objectives set and achieving the intended results

Efficiency : The best relationship between resources employed and results achieved

Sustainability : Results are maintained after the EU funding has ended

Input : Resources needed to carry out an activity e.g. implementing a project

Output : The products, capital goods and services which result from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes

Outcome : The longer-term impact usually expressed in terms of broad socioeconomic consequences

Direct costs : Costs directly attributable to an activity

Indirect costs : Costs which are not directly attributable to an activity, like overheads

Trust fund : Funding instruments through which donors pool resources

Joint management : The implementation of activities through international organisations

EXECUTIVE SUMMARY

I. The European Commission has intensified its cooperation with the United Nations as part of its commitment to the better coordination of aid. The amount of EuropeAid funds channelled through United Nations Organisations increased from 144 million euro in 2001 to 935 million euro in 2009, reaching a peak of over 1 billion in 2006.

II. This is the second phase of a two-part audit. The overall audit question was whether the Commission achieves value for money when channelling funds through the United Nations. Part one looked at decision-making and monitoring. It concluded that the process for deciding to implement aid through the UN did not demonstrate that that is the most effective and efficient option and that the monitoring systems should be further improved. Special Report No 15/2009 for phase 1 was published in January 2010.

III. The second phase evaluates the achievement of objectives. It complements the first phase by assessing whether EU contributions channelled through United Nations organisations are an effective, efficient and sustainable way of delivering aid in conflict-affected countries. The audit examined a sample of projects in Afghanistan, Iraq and Sudan covering the period 2006–08.

IV. The Court concludes that:

(a) The audit of a sample of projects confirmed a key observation of phase 1, that the Commission does not receive sufficient timely information from UN reports. A large proportion of reports are still delayed, not detailed enough and focus on activities rather than results.

(b) Frequent weaknesses in project design were noted which had negative consequences for the implementation and assessment of projects.

(c) By channelling funds through the UN the Commission has delivered aid in areas which would otherwise have been very difficult to target. Given the high inherent risk in conflict-affected countries, the activities funded had an overall positive impact, with 10 out of 19 projects in the sample achieving all or at least the main objectives.

(c) For the majority of interventions in Afghanistan, Iraq and Sudan the planned implementation period was exceeded, in general due to the unrealistic timeframe set out in the contribution agreements, which underestimated the difficult situation in these countries. Four out of 19 projects were implemented within the initial timeframe, and for several projects the original duration was more than doubled.

(d) Given the challenging circumstances in which the projects were implemented, sustainability of results is in general difficult to guarantee. However, the ​ majority of the results achieved were assessed as having reasonable chances of sustainability.

(e) The Commission has not sufficiently focused on efficiency aspects given that the cost assessments carried out were limited and not systematically documented. In addition, the Court identified shortcomings in this area in almost half (nine out of 19) of the sampled projects while for seven projects sufficient information was not available.

V. On the basis of its observations, the Court makes the following recommendations to further improve the effectiveness and efficiency of the interventions:

(a) The contribution agreements should include objectives which are well adapted to the specific circumstances, quantified indicators to measure the achievement of these objectives, where possible, and a realistic timeframe.

(b) The reporting of the funded activities should be further improved in order to enable the Commission to assess the effectiveness and efficiency of the projects financed on a timely basis. The Commission should follow up all instances where reporting is judged to be unsatisfactory.

(c) A systematic assessment of the efficiency of funded projects should be carried out and benchmarks should be developed for standard costs where feasible.

MAP SHOWING THE COUNTRIES COVERED BY THIS AUDIT

+++++ TIFF +++++

Source: Eurostat.

INTRODUCTION

AUDIT AREA

1. Over the last two decades efforts have been made to coordinate and harmonise development cooperation with the objective of further increasing aid effectiveness. This focus on results was most notably reflected in the 2000 millennium development goals.

2. The Commission continued this policy with its commitment in 2005 in the Paris Declaration on Aid Effectiveness as well as in the European Consensus on Development to coordinate aid in close cooperation with international organisations like the United Nations.

3. As a result of the decision to intensify cooperation with the UN, but also due to the major crises in Afghanistan, Iraq and Sudan, Commission aid implemented through UN organisations increased considerably. For the Commission’s EuropeAid Cooperation Office (EuropeAid) alone the funds channelled through the UN rose from 144 million euro in 2001 to 935 million euro in 2009, reaching a peak of over 1 billion euro in 2006 (see Figure 1).

FIGURE 1

EUROPEAID CONTRIBUTIONS TO UN ORGANISATIONS 2001–09 (MILLION EURO)

+++++ TIFF +++++

Source: European Commission.

POLICY FRAMEWORK

4. The method of implementing the EU budget through international organisations like the UN is called joint management [1]. The basic principle of joint management is to delegate certain implementation tasks to international organisations by relying on their control systems. However, the overall responsibility for the management of the budget and for ensuring value for money remains with the Commission [2].

5. EU funds channelled through United Nations organisations can be implemented in the following ways:

(a) contributions to specific projects either as the only donor or together with other donors;

(b) contributions to Multi-Donor Trust Funds [3], typically used in crisis (either post-conflict or post-disaster) situations. The basic idea of trust funds is that donors contribute to the fund without allocating the money to specific activities or objectives (no "earmarking"). General objectives are then set for the fund as a whole, which might be further specified in objectives for the underlying projects or activities. About 26 % of the contributions to UN organisations are through Multi-Donor Trust Funds [4]. Examples are the interventions in Afghanistan and in particular Iraq, where Commission funding has, until recently, almost exclusively been channelled through Multi-Donor Trust Funds;

(c) contributions to the general budget of a UN organisation. This specific approach is used for the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) in the occupied Palestinian territory. The Commission’s annual contribution of 66 million euro represents about a quarter of the UNRWA general budget.

6. The EU contributions are governed by the following regulations and agreements:

(a) the Financial Regulation of the European Communities sets out the basic principles for channelling funds through international organisations. It specifies the cases in which joint management can be used and states as a condition for joint management that the international organisation shall apply internationally accepted standards for accounting, auditing, internal control and procurement;

(b) the 2003 Financial and Administrative Framework Agreement (FAFA) between the EU and the UN together with the other related agreements, aim to translate the requirements of the Financial Regulation into contractual terms. The FAFA applies to all funding agreements between the Commission and the UN and sets out a framework intended to enhance cooperation by allowing UN organisations to manage contributions in accordance with their own procedures;

(c) a contribution agreement is signed for each UN action funded by the Commission, stating the amount of funding, the objectives of the action, the activities to be carried out, their timing and the reporting requirements.

AUDIT SCOPE AND APPROACH

7. The subject of this audit is Commission aid channelled through United Nations organisations. It is the second part of an audit divided into two phases [5].

8. The overall audit question for phase 1 and phase 2 was whether the Commission achieves value for money when channelling funds through the United Nations.

9. The first phase addressed decision-making and monitoring systems. The main findings were:

(a) the process for deciding to implement aid through the UN did not demonstrate that that is the most efficient and effective option;

(b) monitoring arrangements did not provide adequate information on the robustness of financial procedures and on the achievement of objectives.

10. The second phase evaluates the achievement of objectives. It complements the first phase by assessing whether EU contributions channelled through United Nations organisations are an effective, efficient and sustainable way of delivering aid in conflict-affected countries [6]. It does not review the action taken by the Commission to address the issues raised in the first phase.

11. The audit focused on conflict-affected countries mainly for reasons of materiality. Seven of the 10 countries which received most funding channelled through the UN in the period 2006–08 were conflict-affected: Iraq, Palestine, Afghanistan, Sudan, Côte d’Ivoire, Somalia and Sri Lanka. These seven countries received 39 % of total EuropeAid funding channelled through the UN, whilst the remaining 61 % was spread across more than 80 countries (see Figure 2).

12. The audit question was addressed by examining a sample of projects in Afghanistan, Iraq and Sudan during the period 2006–08. These three countries roughly represent one quarter of all EU funds channelled through UN organisations during this period. The projects were selected on the basis of materiality. They represented about 80 % of the total contributions to the three countries concerned during the period 2006-08 (see Figure 3).

13. The most common activities of the projects in the sample were de-mining, support to refugees, the preparation of elections, rehabilitation and capacity building.

FIGURE 2

COUNTRIES RECEIVING EU FUNDS CHANNELLED THROUGH THE UN 2006–08 (MILLION EURO AND %)

+++++ TIFF +++++

Source: European Commission.

14. The audit results are based on:

(a) an analysis of general documentation concerning funds channelled through the UN including the Commission’s evaluation of its cooperation with the UN;

(b) a documentary review of a sample of 23 contributions to 19 projects (including three trust funds) on the basis of contribution agreements and financial, progress and audit reports (see Annex);

(c) interviews with Commission and UN staff;

(d) an on-the-spot audit in Sudan;

(e) meetings at the UNDP’s Multi-Donor Trust Fund Office in New York.

FIGURE 3

TOTAL FUNDING 2006–08 AND SAMPLE SIZES

+++++ TIFF +++++

Source: European Commission.

15. The decision to fund projects in conflict-affected countries entails a substantial level of risks, which can only to a certain extent be mitigated. The question of whether these inherent risks are acceptable is a political decision and therefore not the subject of this audit.

16. No on-the-spot visits could be carried out for two out of the three countries (Iraq and Afghanistan), due to the fact that these countries were in a post-conflict if not conflict situation at the time of the audit. This limited the assessment of effectiveness, efficiency and in particular sustainability.

OBSERVATIONS

MONITORING SYSTEMS AND PROJECT DESIGN AS KEY FACTORS FOR EFFECTIVENESS AND EFFICIENCY

17. This part of the report outlines the audit findings relating to the key aspects of efficiency, effectiveness and sustainability. To ensure that these are attainable it is essential to put in place the necessary building blocks, two of the most important being solid project design and adequate monitoring and reporting. These influence all stages of the process from commencement to ensuring that the funded actions continue into the foreseeable future.

18. The subject of monitoring was treated in phase 1 of the audit from the point of view of assessing the strength of the systems in place. This, phase 2 of the audit, complements the previous one, as indicated in paragraph 10.

THE MONITORING OF PROJECTS NEEDS IMPROVEMENT

19. The decision of the Commission to direct aid towards conflict-affected countries implies the acceptance of a substantial level of inherent risks in terms of the results. Furthermore the decision to intervene in these countries often requires the use of intermediaries, like the UN, and results in limited possibilities to monitor the interventions funded on-the-spot. To a larger extent than for other activities financed by the EU, the Commission therefore depends on information produced by third parties, with hardly any possibility to corroborate it.

20. In these circumstances, the Commission has, to a large extent, to rely on UN reports. In line with the FAFA, reports for each contribution agreement are required at least annually and a final report should be provided within six months after the end of the implementation period.

21. The information to be provided in reports is set out in the contribution agreement for each activity financed [7]. In order to improve the quality of UN reports, guidelines on reporting were introduced in 2007 [8]. In line with these guidelines project reports should focus on results rather than activities and therefore allow a comparison between the objectives in the contribution agreement and actual achievements.

22. The sample of projects assessed during the second phase confirmed the main findings of phase 1 of the audit because a large proportion of reports were:

(a) considerably delayed;

(b) focusing on activities rather than interim and final results;

(c) not detailed enough to assess effectiveness and efficiency.

Photo 1 — Building works in the context of the project "Enhancing the capacity of Khartoum State in the formulation and implementation of urban planning policies" (Sudan)

+++++ TIFF +++++

Source: European Court of Auditors.

BOX 1

REPORTING WEAKNESSES IN THE SAMPLED PROJECTS

Delayed reports

Support to the fifth Sudan population census:

The implementation period for a census project in Sudan was April 2006 to June 2009. At the time of the audit the only available report was from 2007, which means that neither the 2008 report nor the final report, which was due in December 2009, had been provided.

Emergency infrastructural support to the Ministry of Legal Affairs and Constitutional Development and the Judiciary of Southern Sudan:

The final report, although due in September 2009, was not yet available at the time of the audit.

Insufficient information

Interim disarmament, demobilisation and reintegration programme (IDDRP):

Although the final report shows substantial alterations in terms of the scope of work (expenditure reduced by more than 70 % and several activities not carried out) there was no indication of these modifications in previous reports or any other evidence that the Commission had been informed.

Capacity building for the new administration of Southern Sudan:

65 % of the expenditure in the final report was declared as "previous expenditure" without providing any further explanations on the nature or content of the costs involved.

Further support to elections in Afghanistan:

There was no specific report related to this project, although all payments had been made. The only available documents were in fact general reports on the election.

WEAKNESSES IN PROJECT DESIGN IMPAIR EFFECTIVENESS, EFFICIENCY AND THEIR ASSESSMENT

23. An adequate project design is the basis not only for effectiveness but also for efficiency. The Court examined whether:

- the objectives set out in the contribution agreement were clear and realistic;

- indicators were defined where possible;

- the time frame was well adapted to the specific circumstances and the budget sufficiently detailed to assess in particular the efficiency [9].

24. The large majority of contribution agreements in the sample (18 out of 19) had one or more of the following weaknesses:

(a) Unspecific objectives: have the effect that performance cannot be measured as it is unclear what should have been achieved. This also applies for objectives that are so vague that they will always be achieved (typically characterised by formulations like "to support..."). In addition the results and in particular the impact of a project are more likely to be limited if the objectives to be achieved are not clear from the beginning.

(b) Missing indicators: in the absence of indicators neither the Commission nor the UN can measure performance and track results, monitor activities and broader outputs adequately and thus take corrective actions in time.

(c) No baseline criteria: without baseline criteria improvements cannot be measured because there is no information on what the situation was like before the intervention started and therefore what the results should be measured against.

(d) Weak logical link between activities and objectives: the funds used to finance the project’s activities should result in the achievement of the project’s objectives. However, the logical link between the activities and the overall objectives, in particular those that are longer term, is not always clear. Consequently, although activities may have been successfully completed, the projects’ intended impact may not be achieved.

(e) An unrealistic timeframe: results in an extension of the project at a later stage, which requires additional work and therefore resources from both, the Commission and the UN.

(f) Underestimated risks: although risks are particularly high and difficult to assess in conflict-affected countries it is of importance that they are given due consideration at the planning stage.

EFFECTIVENESS AND SUSTAINABILITY OF THE ACTIVITIES FUNDED

THE COMMISSION-FUNDED ACTIVITIES ACHIEVED SOME GOOD RESULTS IN DIFFICULT CIRCUMSTANCES

25. Effectiveness in the context of this audit is, in line with the Financial Regulation, defined as the attainment of objectives and the achievement of results. The Court assessed the effectiveness of a sample of 19 projects by comparing the intended objectives in the contribution agreement with the actual achievements. The effectiveness was rated as:

Good | If all objectives in terms of output as well as outcome have been achieved or are likely to be achieved |

Minor weaknesses | If only the main objective(s) are likely to be achieved |

Serious weaknesses | If the project will only partially achieve the main objective(s) |

Unsatisfactory | If the main objectives were not or will not be achieved |

26. The cooperation with organisations of the UN has made aid possible in a very difficult operational environment. Whereas two projects achieved their objectives in full and were therefore rated as "good", eight showed "minor weaknesses" and five "serious weaknesses". Two projects were not fully implemented due to specific circumstances and therefore they did not achieve their main objective and are thus considered "unsatisfactory". Examples of the weaknesses noted can be found in Box 2 and for the overall result see Figure 4 and the Annex.

27. Due to insufficient information, the results could not be assessed in the following cases:

(a) for one project, although all payments had been made, there was no specific report for the funded project;

(b) one major project was a trust fund for which the objectives stated in the contribution agreement concerned the fund as a whole and were therefore of a more general nature. Although more specific objectives were defined for the underlying projects, there can be no clear link between the achievement of the project objectives and the general objectives in the contribution agreement. Consequently the effectiveness can only be assessed when the fund itself will be closed, which was not yet the case at the time of the audit.

FIGURE 4

THE EFFECTIVENESS OF SAMPLED PROJECTS (BY NUMBER OF PROJECTS AND AMOUNTS REPRESENTED)

+++++ TIFF +++++

Source: European Court of Auditors.

BOX 2

EFFECTIVENESS EXAMPLES

Project likely to achieve the intended results

Support to the return and reintegration of returnees and internally displaced people inside Iraq:

The project was still ongoing at the time of the audit. However, according to the September 2009 report the programme was progressing according to plan with several activities already achieved, like provision of shelters and safe drinking water, and others even exceeding the expected results.

Projects which have not achieved the intended results

Mine clearance in Afghanistan:

The project has not achieved its intended results as only 1,6 km2 out of the 26,6 km2 as stated in the contribution agreement were actually de-mined.

Emergency infrastructural support to the Ministry of Legal Affairs and Constitutional Development and the Judiciary of Southern Sudan:

The project ended in March 2009 without providing the main output, to construct buildings for the Ministry of Legal Affairs and for the Judiciary of Southern Sudan. After several delays and extensions of the contract, the sub-contracted building company became insolvent and the bank guarantees turned out to be falsified. The amount to be recovered was still under discussion at the time of the audit.

Project for which the impact is unclear

International Reconstruction Fund Facility for Iraq:

Apart from the objective to halve the infant mortality rate, where targets were set and the information available suggests that it has not been achieved, there was no means of assessing the extent to which the other long-term objectives were achieved as no indicators or baselines were defined and no reliable statistics were available.

28. For the sample audited, no connection has been noted between the size of the project or the type of the activity financed and its effectiveness.

29. The serious weaknesses noted in several projects were in general a result of deficiencies in project design and the lack of monitoring at a later stage.

Photo 2 — Construction of offices for the "Emergency infrastructural support to the Ministry of Legal Affairs and Constitutional Development and the Judiciary of Southern Sudan"

+++++ TIFF +++++

Source: European Court of Auditors.

RESULTS ARE OFTEN NOT ACHIEVED WITHIN THE PLANNED IMPLEMENTATION PERIOD

30. The time frame for each project is defined in its contribution agreement. The respect of this timeframe impacts effectiveness as well as efficiency due to the fact that:

(a) delays result in extensions of the projects, which require additional work for the Commission staff;

(b) time extensions result in the increase of certain costs categories, like staff costs, which have to be funded from the Commission’s fixed contribution. Less funding will then be available to finance the core activities of the project. Unless other funding is available, project activities will thus have to be curtailed;

(c) delays in the implementation lead to a lack of confidence among the concerned parties and can affect the reputation of the Commission, as was clearly observed for two projects in Sudan.

31. For the vast majority of projects in the sample the initial implementation period was considerably exceeded. This was to a large extent due to the fact that the time needed to achieve certain results had been underestimated in the contribution agreements, as not enough consideration was given to the difficult circumstances in the countries.

32. Four out of the 19 projects were implemented within the timeframe initially foreseen. In 12 cases the timeframe was exceeded and for five of these the implementation period was doubled or more than doubled. Notably two out of the four projects which were implemented within the initial timeframe were de-mining projects. For details on the delays noted see Annex and Figure 5.

33. For the remaining three projects the assessment of timely implementation is of limited significance for the following reasons:

(a) for two of the three trust funds (in Sudan and Iraq) the implementation periods set out in the contribution agreements had, although legally binding, no practical effect as the relevant time schedules were those for the funds as such;

(c) for one other project the assessment of the timeframe is still preliminary as the project was at an early stage of implementation.

FIGURE 5

EXTENSION OF THE INITIAL IMPLEMENTATION PERIOD BY PROJECT

+++++ TIFF +++++

(1) For these two trust funds, although implementation periods were set in legally binding documents, they hardly had any practical effects.

(2) At the time of the audit, the implementation of this project was still at an early stage.

(3) At the time of the audit, a 12-month extension was under discussion but not yet approved.

Source: European Court of Auditors.

MOST RESULTS ARE LIKELY TO BE SUSTAINED

34. Sustainability in the context of this audit is defined as the extent to which results have been maintained after the Commission funding has ended.

35. As the majority of projects in the sample had not yet been closed, the audit examined the likely sustainability of project results. The basis for this assessment was the extent to which the necessary conditions for sustainability, i.e. ownership by the beneficiary, further funding, the necessary expertise and the political situation in the country or region, were fulfilled. The sustainability was rated as follows:

Good | If the results were or will most likely be sustained as all conditions for sustainability are fulfilled |

Minor weaknesses | If not all conditions for sustainability are fulfilled but nevertheless the main results are likely to be sustained |

Serious weaknesses | If several or very important conditions for sustainability are not fulfilled and therefore there are reasonable doubts about the sustainability of the main project results |

Unsatisfactory | In case there was evidence that the results will not be maintained |

36. Four projects in the sample were rated as "good" in terms of sustainability. Five projects had "minor weaknesses", two "serious weaknesses" and one was considered "unsatisfactory". For seven projects no assessment could be made as they were either still in an early phase of implementation (three cases), the information available was insufficient (two cases) or they had not achieved their main objectives and therefore there were no results to be sustained (two cases). For an overview of the results see Annex and Figure 6 and examples can be found in Box 3.

37. The sample of projects audited showed that the conditions for sustainability vary in importance. Those projects which were characterised by a strong ownership in combination with political stability were in fact the ones most likely to be sustainable.

FIGURE 6

SUSTAINABILITY OF RESULTS (BY NUMBER OF PROJECTS AND AMOUNTS REPRESENTED)

+++++ TIFF +++++

Source: European Court of Auditors.

BOX 3

SUSTAINABILITY EXAMPLES

Projects which are likely to be sustained

Support to the fifth population census, Sudan:

Although there was no exit strategy, there is adequate ownership from the beneficiary and future funding foreseen in the budget. Therefore the results had been sustained at least until the time of the audit which was one year after the implementation period had ended.

Enhancing the capacity of Khartoum State in the delivery of vocational training services, Sudan:

The training centres financed under this project are not yet operational. However, it is likely that the main results will be sustained, in particular due to the strong ownership by the beneficiary.

Projects with weaknesses in terms of sustainability

Law and Order Trust Fund, Afghanistan:

The main objective of the Law and Order Trust Fund is to pay salaries of police officers in Afghanistan. Due to the limited financial capacity of Afghanistan the fund will, in the medium to longer term, remain dependent on further donor funding to continue its activities.

Photo 3 — Equipment for a training centre financed under the project "Enhancing the capacity of Khartoum State in the delivery of vocational training services, Sudan"

+++++ TIFF +++++

Source: European Court of Auditors.

EFFICIENCY OF THE ACTIVITIES FUNDED

38. One of the principles of sound financial management, as specified in the Financial Regulation, is efficiency [10]. Efficiency is also one of the objectives agreed upon in the "European Consensus on Development" [11].

39. In line with the Financial Regulation, efficiency in the context of this audit is defined as the best relationship between resources employed and results achieved.

40. The Court aimed to establish to what extent the Commission’s cost assessments and monitoring ensured that the costs included in budgets and financial reports were reasonable in relation to the services provided. In this context the Court observed examples of projects which show indications of suboptimal cost / result relations, in particular instances of high administrative costs.

INADEQUATE ASSESSMENT OF PROJECT COSTS

41. The level of detail in budgets, specified in the contribution agreements, is often not adequate to properly assess the reasonableness of costs and to facilitate the assessment of efficiency.

42. The Commission’s cost assessments, during the lifespan of a project, focus on the eligibility of costs with considerably less attention being devoted to aspects of efficiency. They are generally neither systematic nor adequately documented.

43. No systematic use is made of available cost comparison mechanisms such as:

(a) comparing the same cost categories of different projects (e.g. staff costs for similar functions);

(b) calculation and comparison of cost ratios (e.g. proportion of international staff costs to total staff costs, maintenance in relation to equipment);

(c) comparison with benchmarks (e.g. average costs of de-mining a certain area and average cost per capita for a census).

SHORTCOMINGS HAVE BEEN IDENTIFIED IN THE SAMPLED PROJECTS

44. In nine out of the 19 projects examined, high levels of costs were identified without evidence that these resulted in the improvement of the services provided (examples are given in Box 4). No assessment could be made for seven projects as the information available in financial reports was not sufficient. A summary of the results can be found in the Annex and Figure 7.

FIGURE 7

THE EFFICIENCY OF SAMPLED PROJECTS (BY NUMBER OF PROJECTS AND AMOUNTS REPRESENTED)

+++++ TIFF +++++

Source: European Court of Auditors.

45. Incidences of high costs were particularly noted in the following areas:

(a) Indirect costs: in line with the FAFA the Commission contributes up to 7 % of direct costs in respect of the UN organisation’s indirect costs. The FAFA also requires in Article 5.1 that subcontracting should not lead to higher costs. However, the audit identified projects in which implementing partners, often other UN organisations, charged indirect costs that were additional to the 7 % of direct costs provided for in the FAFA agreement, and subcontractors were engaged without evidence that this did not lead to higher costs.

(b) Reduction of the scope of activities not reflected in the costs: the follow-up of instances where problems arise due to the curtailment of the level of activities, or where there is insufficient information in the UN reports, was inadequate. In two projects a reduction of the scope of the work has not led to an equivalent reduction of the EU contribution.

Photo 4 — Offices constructed under the "Sudan productive capacity recovery programme"

+++++ TIFF +++++

Source: European Court of Auditors.

BOX 4

EXAMPLES OF SHORTCOMINGS IN RELATION TO PROJECT COSTS

Indirect costs

Capacity building for the new administration of Southern Sudan:

This trust fund, to which the EU contributed 1,9 million euro, was implemented by a UN organisation, which received a 3 % contribution for indirect costs. However the actual (financial) management of the project was subcontracted to a private company which charged another 10 % of the total costs, resulting in total administrative costs of more than 13 %.

Sudan productive capacity recovery programme (SPCRP):

In addition to the 7% indirect costs, a management fee (187 338 USD) was paid to another UN organisation for services provided in the context of this project (e.g. building of offices).

Reduction of the scope

Mine clearance in Afghanistan:

Although the implementation period was reduced from 12 to three months and the project has in fact only partly achieved its objectives, the amount charged to the Commission was not reduced.

Interim disarmament demobilisation and reintegration programme (IDDRP):

The final report states a reduction of the scope of work and with it a decrease of the total costs by 70 %. However, this has not resulted in an equivalent reduction of the Commission’s contribution, which has only decreased by 12 % and now represents 65 % of the total costs instead of the planned 20 %.

Photo 5 — A communication and information centre for the "Sudan community-based recovery and rehabilitation programme (RRP)"

+++++ TIFF +++++

Source: European Court of Auditors.

CONCLUSIONS AND RECOMMENDATIONS

46. The presence and activities of the UN, in conflict-affected areas, provide the Commission with the opportunity to channel its aid in difficult circumstances. However, providing aid in conflict-affected areas presents special difficulties, only some of which can be overcome. As the amount of checking that can be done by the Commission on the spot, is severely curtailed, more reliance must be placed on the implementing body to compensate for the lack of direct monitoring by the Commission.

47. The Court concludes that in the circumstances some good results were achieved and the majority of the results of the projects concerned have reasonable prospects of being sustainable. Inadequate project design and deficiencies in monitoring and reporting have contributed, however, to the weaknesses identified in projects. Regarding efficiency, the Commission does not focus sufficiently on this element and on ensuring that it receives the necessary feedback from the UN reports.

PROJECT DESIGN

48. The project design is generally inadequate, as 18 out of the 19 projects examined were affected by deficiencies in this planning process. As a result, there was a lack of clarity as to what was to be achieved and how the success of the projects’ activities could be assessed. Poor project design was also a factor contributing to the weaknesses noted in the projects examined.

RECOMMENDATION 1

The Commission should ensure that clear practical objectives are set for the projects to which its funds are committed. Objectives should be quantified where possible to facilitate the execution and monitoring of the projects and to provide useful feedback for the Commission. The link between the projects’ activities, the projects’ objectives and the wider objectives should be clearly specified.

In addition, the time frame set out in the contribution agreement should be better adapted to the project environment to avoid timely and costly extensions of the implementation period at a later stage.

As the level of detail in the budget forms the basis for subsequent reports, it should include all information necessary to assess in particular the efficiency of the activities funded.

EFFECTIVENESS AND SUSTAINABILITY

49. The Annex illustrates that approximately half of the projects achieved at least their main objectives and that there is a reasonable likelihood that the majority of the results obtained will be sustained, provided there is no further deterioration in the security and political situation. The Annex also shows, however, that major weaknesses were noted in certain of the projects. The results are best for Sudan, where due to the improved security situation the Commission is able to take a more active role in the monitoring of its funded projects. However, for the majority of projects in the sample the planned implementation period was considerably exceeded.

50. As the Commission is generally dependent on the UN for its information on the progress and achievements of the projects it needs reports that are on time and that give sufficient information, particularly on the achievement of results. However, for the projects examined, many reports were either late or were not comprehensive.

RECOMMENDATION 2

The Commission should insist on receiving the necessary reports on time and continue its efforts to ensure that these reports give the information that it needs for assessing the progress and success of the projects.

The follow-up by the Commission should be prompt, systematic, clearly evidenced and comprehensive throughout the project duration.

Following each project the Commission should identify the lessons learnt for future interventions having special regard to the aspect of sustainability.

EFFICIENCY

51. The lack of information for seven out of the 19 projects considerably hampered the Court’s assessment of efficiency. For most of the projects it was not possible to ascertain what work the Commission did to assess efficiency as this exercise was neither systematic nor adequately documented. The emphasis was more on the eligibility of costs rather than the need to incur certain of these. The lack of detail in the individual budgets relating to the contribution agreements reduced the extent to which the reasonableness of costs could be assessed and monitored. Very little use was made of comparative information in the cost assessment process.

52. In nine out of the 19 projects in the sample the Court identified incidences of high costs without evidence that these resulted in an increase or improvement of the services provided.

RECOMMENDATION 3

A systematic assessment of the costs should be performed and the results should be adequately documented. More focus should be put on efficiency and the assessment of costs should not be limited to questions of eligibility.

Where feasible, benchmarks should be developed for common cost items in order to facilitate the assessment of costs in project proposals and financial reports.

This report was adopted by Chamber III, headed by Mr Jan KINŠT, Member of the Court of Auditors, in Luxembourg at its meeting of 8 March 2011.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 on the Financial Regulation applicable to the general budget of the European Communities (OJ L 248, 16.9.2002, p. 1). Article 163 of Title IV relating to external actions states that actions may be implemented either on a centralised basis by the Commission or on a decentralised basis by the beneficiary third country or countries, or jointly with international organisations.

[2] Article 317 TFEU states that the Commission shall implement the budget on its own responsibility and having regard to the principles of sound financial management.

[3] Multi-Donor Trust Funds as defined by UNDP are "a funding instrument through which donors pool resources to support national priorities and facilitate UN agencies to work and deliver in close coordination and collaboration" (see http://www.undp.org/mdtf/trustfunds.shtml).

[4] European Commission: Information Note for the Committee on Budgetary Control of the European Parliament on Multi-Donor Trust Funds supported by the European Community General Budget since 2003 ( 10.2.2009).

[5] The first phase has been completed with the Special Report published in January 2010 (European Court of Auditors, Special Report No 15/2009 "EU assistance implemented through United Nations organisations: decision-making and monitoring").

[6] The Financial Regulation states in Article 27 that "Budget appropriations should be used in accordance with the principle of sound financial management, namely in accordance with the principles of economy, efficiency and effectiveness.".

[7] FAFA Article 2.1 states that "the contribution-specific agreement sets out the information to be provided in reports by the United Nations to the Commission.".

[8] See "Joint Guidelines on reporting obligations under the FAFA". The purpose of the guideline is to provide practical guidance in addition to the relevant articles in the FAFA.

[9] The FAFA states in Article 1.1: "UN submissions of proposals will include objectives and indicators of achievement to be agreed in contribution specific agreements. Performance measures will be based on objectives that are specific, measurable, attainable, realistic and time based."

[10] The Financial Regulation states in Article 27 that "Budget appropriations should be used in accordance with the principle of sound financial management, namely in accordance with the principles of economy, efficiency and effectiveness.".

[11] "The European Consensus on Development" (OJ C 46, 24.2.2006) states in paragraph 25 that "As well as more aid, the EU will provide better aid. Transaction costs of aid will be reduced and its global impact will improve.".

--------------------------------------------------

ANNEX

Number | Title | Effectiveness | Likely sustainability | Respect of the timeframe | Efficiency | Amount (million euro) |

Afgh 1 | Anti-personnel mine and ammunition stockpile destruction | Serious weaknesses | Minor weaknesses | No | Shortcomings identified | 6 |

Afgh 2 | Support to governance in the Islamic Republic of Afghanistan — Census | Uunsatisfactory | N/A (main objectives not achieved) | No | Shortcomings identified | 15 |

Afgh 3 | Further support to elections in Afghanistan | Insufficient evidence | Insufficient evidence | No | Insufficient evidence | 9 |

Afgh 4-6 | Law and Order Trust Fund for Afghanistan (LOTFA) | Serious weaknesses | Unsatisfactory | Yes | Insufficient evidence | 100 |

Afgh 7 | Mine clearance in Afghanistan | Serious weaknesses | Good | Yes | Shortcomings identified | 2 |

Afgh 8 | Regional programme in support of Afghan refugees in Iran and Pakistan, and of returnees in Afghanistan | Minor weaknesses | Good | Yes | Insufficient evidence | 4 |

Afgh 9 | Support to the mine action sector in Afghanistan | Minor weaknesses | Good | Yes | Insufficient evidence | 20 |

Iraq 1-3 | International Reconstruction Fund Facility for Iraq (IRRFI) | Insufficient evidence | Minor weaknesses | N/A (fund) | Insufficient evidence | 198 |

Iraq 4 | Support to the return and reintegration of returnees and internally displaced people inside Iraq | Good | N/A (project still at an early stage) | N/A (project still at an early stage) | Insufficient evidence | 6 |

Sudan 1 | Interim disarmament, demobilisation and reintegration programme (IDDRP) | Serious weaknesses | Serious weaknesses | No | Shortcomings identified | 12 |

Sudan 2 | Capacity development for aid management and coordination | Minor weaknesses | Minor weaknesses | No | No shortcomings identified | 2 |

Sudan 3 | Support to the fifth population census Sudan | Good | Good | No | Insufficient evidence | 4 |

Sudan 4 | Enhancing the capacity of Khartoum State in the formulation and implementation of urban planning policies | Minor weaknesses | Minor weaknesses | No | No shortcomings identified | 2 |

Sudan 5 | Capacity-building for the new administration of Southern Sudan | Serious weaknesses | Insufficient evidence | N/A (fund) | Shortcomings identified | 2 |

Sudan 6 | Enhancing the capacity of Khartoum State in the delivery of vocational training services | Minor weaknesses | Minor weaknesses | No | No shortcomings identified | 11 |

Sudan 7 | Emergency infrastructural support to the Ministry of Legal Affairs and Constitutional Development and the Judiciary of Southern Sudan | Unsatisfactory | N/A (main objectives not achieved) | No | Shortcomings identified | 4 |

Sudan 8 | De-mining in Southern Sudan | Minor weaknesses | Serious weaknesses | No | Shortcomings identified | 5 |

Sudan 9 | Sudan productive capacity recovery programme (SPCRP) | Minor weaknesses | N/A (project still at an early stage) | No | Shortcomings identified | 39 |

Sudan 10 | Community-based recovery and rehabilitation programme (RRP) | Minor weaknesses | N/A (project still at an early stage) | No | Shortcomings identified | 51 |

--------------------------------------------------