# **Introduction**

How should aid quality be evaluated and compared across donors? A number of recent papers have proposed rankings of bilateral and multilateral donors, on the assumption that donor performance can be measured and ranked globally in a meaningful and robust way (Mosley 1985; Roodman 2006 and 2009; Center for Global Development 2007; Easterly and Pfutze 2008). These public rankings are intended to inspire both bilateral and multilateral donors to improve their effort, effectiveness, and efficiency in ways that will move them up in the rankings. And there is evidence that donors do in fact pay attention to these rankings and care about public perceptions (see, for example, Ramankutty et al. 2009). A sustained and focused ―peer pressure‖ campaign within the DAC appears to have contributed to a marked decline in the share of aid that is tied to purchases of good and services in donor countries (OECD 2009a).<sup>2</sup> Whether this approach is successful or not is likely to depend on the perceived validity and robustness of the rankings. While the ranking of aid agencies on well-defined individual components is hard to argue with, this paper investigates the robustness of these overall rankings to inclusion of new indicators and to different weightings of the components. If the various indicators of donor performance that have been put forward are not highly correlated with each other, then the rankings will be sensitive to decisions on how to weight them in the index. Indeed, an example from Easterly and Pfutze (2009) shows how weightings of different components of the index – in this case, policy selectivity vs. poverty selectivity – can lead to an institution being ranked differently:

[The Global Environment Facility] has a comparatively low share of its ODA going to corrupt or non-free countries, but at the same time spends relatively little on the poorest countries. As we freely admit in the paper, _one has to trade off one aspect of selectivity against the other_ [emphasis added] _. . ._ In assessing aid quality by donors, this paper extends and enlarges upon the work of Roodman (2006 and 2009) and Easterly and Pfutze (2008) in several respects. First, it includes a more comprehensive set of indicators that have been used to assess donor performance, combining existing indicators with new ones that reflect the content of the Paris Declaration in a more comprehensive and representative way. In particular, we draw heavily on the OECD-DAC Survey on Monitoring the Paris Declaration (SMPD), a rich new source of indicators on the use of country systems, aid predictability, and coordination with other donors in country (OECD 2008). We also include an indicator on bilaterals' share of aid funneled through multilateral donors. One advantage of including more indicators is that an overall ranking of donors should be less sensitive to the inclusion of (or weight assigned to) any one indicator.

Second, most of our aid quality indicators adjust for important factors that are not directly under the control of the donor agencies, since one goal of the ranking is to assess how effectively the aid agencies are managing. For example, from a management perspective, it is pointless to castigate the EBRD for not focusing on the world’s poorest countries, given that the institution is allowed to operate only in the largely middle-income Europe and Central Asia region. So for our ranking exercise, we measure poverty selectivity using only the set of countries in which the EBRD is mandated to work. In effect, we are therefore assessing whether within its mandated region, the

> 2 This approach has also been tried at the country level; in Rwanda, for example, donors participate in a Donor Performance Assessment Framework that rates their progress toward certain targets (Rwanda Development Partners 2009).

2