Development Bank nearly meets that criterion, but the other three rank poorly on Easterly-Pfutze. At the bottom of the rankings, the CGD rankings line up reasonably well with ours, but there are substantial discrepancies with Easterly-Pfutze: among large donors, the US, Japan, and France all rank at least 15 slots better in their index than in ours. Those three donors all rank fairly high on the Easterly-Pfutze transparency and overhead cost indexes, which we omit from our core index.

The modest correlation of our index with Roodman’s, and weak correlation of ours with EasterlyPfutze, does not by itself suggest that ours is a more valid index of aid quality. It is notable, however, that Roodman and Easterly-Pfutze are even more weakly correlated (at .19) with each other. One explanation for this low correlation is that Easterly and Pfutze are measuring aspects of donor performance that are go beyond the usual aid-quality dimensions; the large weight that they give to their new transparency and overhead cost indexes is likely to lead to weak correlations between their index and other reasonable rankings.

In comparison with the alternatives, our index better meets the criteria of ―content validity‖ (Carmines and Zeller 1979). Namely, the content of our 18 indicators reflects a much more comprehensive and representative sample of ―best‖ aid practices contained in the Paris Declaration and in the research literature on aid effectiveness.<sup>27</sup> The adjustments we make to control for factors unrelated to performance (for example, donor size in measuring geographic concentration of aid) also add to the validity of our indicators and indexes produced from them.

Finally, our index and sub-indexes have a much greater degree of internal reliability than those of Easterly-Pfutze. Higher inter-correlations among indicators and among sub-indexes do not prove that they are all good measures of the same underlying and broader concept of aid quality, but low inter-correlations would suggest more doubt. The mean inter-item correlation of our four subindexes is a modest .37, but the corresponding value for the five sub-indexes in Easterly-Pfutze is only .14.<sup>28</sup> The alpha coefficient of index reliability is .70 for our index and .44 for Easterly-Pfutze. The lowest among the mean inter-item correlations among our four sub-indexes is .26, while two of the five sub-indexes in Easterly-Pfutze have negative mean inter-item correlations.

How is aid quality correlated with quantity of aid delivered, relative to the donor country’s GNI (both measured in 2007)? As the bottom right panel of Figure 2 shows, they have a strong positive correlation among the bilaterals, at 0.66. In other words, countries that show a strong commitment to aid quantitatively also deliver that aid in ways that is consistent with the quality standards agreed to by the international community.

# **Conclusion**

In this paper, we have presented a new index of the quality of aid provided by bilateral and multilateral donor agencies. The index is constructed from a more comprehensive set of indicators that cover the four major areas around which the greatest consensus has developed in the aid community: selectivity, alignment, harmonization, and specialization. It combines indicators that have been used in previous indexes with rich new data collected through the recent monitoring survey for the Paris Declaration of the OECD-DAC, as well as the new AidData projects database. Raw indicators were adjusted for the influence of exogenous factors where appropriate, but in fairly simple and intuitive ways that substantially enhance validity at a modest cost in transparency. We also construct sub-indexes for each of the four dimensions of quality noted above, and show

> 27 Roodman’s ability to make use of many more aid quality indicators is constrained by the need in his approach to identify a plausible estimate of the degree to which a ―bad‖ aid practice reduces the effective value of a dollar of aid. 28 Roodman’s method does not lend itself to computing measures of reliability.

19