In attempting to shed light on the validity of CPIA ratings, IEG (2010) reports correlations between some of the CPIA ratings and other related indicators. It does not test for possible regional biases, however. Nor does IEG (2010) test for an IDA country bias, despite noting repeatedly that the use of CPIA scores for IDA allocations created incentives for country teams to inflate ratings. If it is common for regional staff to inflate ratings proposals for the purpose of increased IDA allocations, then we would expect to observe a positive, significant coefficient on an IDA country dummy included in regressions of CPIA ratings on related indicators.

There is never perfect conceptual overlap between any single CPIA question (or cluster of questions) and related indicators produced by other organizations. The lack of a perfect or even close relationship empirically does not necessarily indicate that the CPIA (or the related indicator) is invalid. For any individual country, a higher (or lower) ranking in the CPIA than on the related indicator could reflect less-than-perfect conceptual overlap, or measurement error in the CPIA, or measurement error in the related indicator. For a large group of countries, however, it is more difficult to dismiss systematic discrepancies between the CPIA and related indicators that tend to favor IDA countries, or countries from a particular region. If CPIA ratings are significantly higher than predicted – based on the values of a related indicator – for IDA countries or for an entire region, it cannot plausibly be attributed to random measurement error in the full sample of more than 130 countries covered by the CPIA. Conceivably, a source of comparator data may contain its own regional biases that would contaminate tests for bias in CPIA. For this reason our tests use a range of related indicators, from several largely independent data sources. It is highly unlikely that these sources would all exhibit the same regional biases.

Table 6 presents a pair of OLS regressions for each of the four cluster D questions (12, 14, 15 and 16). The dependent variable in the first of each pair is the region’s proposed rating in the 2012 CPIA exercise, and in the second of each pair is the final 2012 rating. Independent variables include a conceptually related “comparator” indicator, log of per capita income, an IDA dummy, and an ECA dummy. Preliminary tests showed that ECA’s coefficients were consistently larger (more positive) than those for other regions and variations among the other five regions were small. We therefore test only one regional dummy in Table 6, focusing on the issue of whether ECA ratings are significantly higher than those of countries in the other regions combined, controlling for their ratings on comparator indicators and per capita incomes.

The regressions within each pair are identical other than using proposed ratings and final ratings as alternative dependent variables. Reporting these tests side-by-side shows the extent to which any IDA or regional biases in proposed ratings are dampened or eliminated by the network review.<sup>10</sup> Equations 6.1 and 6.2 analyze question 12 ratings. The related comparator indicator is an index of “guidepost indicators” constructed for purposes of the network review, and designed to match

> 10 This method captures only the immediate direct effects of the network review.  From a longer-term perspective the existence of the reviews can deter regions from proposing ratings that are higher than justified.

18