One potential concern is that rankings could be overly sensitive to findings from a single data source, the SMPD, as it provides all three indicators in the sub-index on harmonization and 5 of the 6 indicators in the alignment sub-index. However, the untied aid indicator is correlated almost as strongly (at .60) as the 5 SMPD indicators are on average (.63) with the alignment sub-index scores. Moreover, the two SMPD-dominated sub-indexes are not as strongly correlated (at .77) with the overall index scores as the selectivity sub-index is (at .83). The single indicator most strongly correlated with the overall index is not one of the 8 Paris Survey indicators, but rather poverty selectivity (at .82). As mentioned above, we cannot entirely rule out the possibility that some donors may have interpreted the survey guidance more liberally than others to improve their apparent performance, but if so, any resulting bias is unlikely to affect the overall ranking very much.

In Table 4, we further investigate the sensitivity of the overall rankings to alternative assumptions about indicator weightings or regarding the purpose of an overall index. The first rankings column replicates the ―Overall‖ rankings from Table 3, while the next three columns use other weighting schemes. Index_18 and Index_20 dispense with the sub-indexes altogether. Rather than averaging sub-index scores, they average the individual indicators. Index_18 uses the 18 indicators included in our sub-indexes, while Index_20 adds two more—the Transparency and Overhead indicators provided by Easterly and Pfutze (2008)—that we do not include in our sub-indexes or in the Overall index constructed from them. Relative to the latter index, Index_18 and Index_20 give greater weight to aid quality dimensions for which we have more indicators, namely Specialization and Alignment. While these changes in methodology and content affect the rankings, the general contours remain the same: the top five donors were all previously in the top 8, while all of the bottom five were previously in the bottom eight. Among bilaterals, only two smaller donors (Hungary and Luxembourg) have ranks that move more than six spots as a result of the reweightings. The EBRD is the only multilateral that moves much: shifting to Index_20 from Overall drops it by 10 spots, because it ranks poorly on specialization (which has an effective weight of 7/20 in Index_20) and on the transparency index in Easterly and Pfutze (2009).

The fourth column ranks donors on what we call the Allocation Index. As discussed above, the Allocation Index is not an alternative version of the overall index (which could be used for ―naming and shaming‖), but an index aimed at a different goal: helping to guide allocations of aid resources. In calculating it, we reverse several adjustments made in the first three indexes for factors that were beyond the control of the aid agencies themselves (most notably, geographic mandate). We also code untied aid at 100 percent for the multilateral donors, because the fact that none of their aid is tied is a relevant criterion for allocating resources between a bilateral agency, the EC, and a multilateral. Despite the different in purpose, the Allocation Index rankings do not differ much from the Overall index rankings. The only donors that move more than 5 spots in the rankings are two regional banks, the IDB and the EBRD. This result is not surprising, given that both banks serve largely middle-income regions, which reduces their poverty selectivity when we do not adjust for geographical mandate. Among the sub-indexes, the selectivity sub-index is the most affected, but even it has a rank correlation of 0.84 with the original sub-index; for the other three sub-indexes, the correlations range from 0.94 to 1.0. In summary, moving to an Allocation Index approach does not dramatically affect the rankings, but it does advance several bilaterals past two of the regional development banks in the rankings.<sup>23</sup>

> 23 Some readers may be uncomfortable with comparing bilaterals and multilaterals (both development banks and vertical funds) on the same ranking, given the inherent differences in their structures and mandates.  From a rankings perspective, it is likely that each group has advantages in different areas (such as untied aid for multilaterals and specialization for bilaterals and vertical funds), and it is not clear which would have the greater net advantage.  But a reader who prefers to compare only within donor groups can interpret the rankings in that way, by comparing bilaterals against only other bilaterals and multilaterals against only other multilaterals, with or without vertical funds included.

17