criteria. Only one tenth of EAP write-ups are graded “C” or below (meaning they address less than one half of the criteria in the question), compared to over one half for LCR. Strengthening the guidance provided by central units to regional staff regarding the cluster D questions could help somewhat in steering the write-ups toward provision of more pertinent information. For example, certain data sources (such as the World Bank’s own Enterprise Surveys) are underutilized, while others (such as TI’s Corruption Perceptions Index) are often cited in ways that do not provide meaningful comparative information across countries or over time. However, the payoff to strengthening guidance notes may be limited, as it is unclear whether many of the relevant staff members currently make use of the existing guidance.

Second, the network reviews could also more assertively request additional information in a much larger number of cases. The network with the lead role in reviewing question 13 (on budgetary management) employed this strategy with some success over the course of several years. It might be overly ambitious to implement this approach simultaneously for all four cluster D questions for which the Public Sector Governance Group has the lead review role. However, it might be feasible to implement it more gradually, e.g. focusing initially on the question with the lowest average quality grades (14, on revenue mobilization), or starting with the grade “D” cases (where most criteria go unaddressed) for all four questions and moving on the following year to the “C” cases.

Third, the network reviews should be more assertive about questioning proposed ratings even when no increase is being proposed, and in borderline cases. Reviews that are overly focused on rating changes produce a status quo bias in the ratings, and reduce the scope for eliminating regional or other biases when they appear in the ratings.

Finally, the process would benefit from increased checks and balances within each region, including an enhanced role in at least some regions for sector specialists. The most striking results from the analyses above involve regional comparisons. On quality of write-ups, EAP is the positive outlier, and could serve as a useful model for other regions to follow. In contrast, LCR is a negative outlier as the region with the least pertinent write-ups.

On realism of proposed ratings, ECA stands out as the region that appears to be most overoptimistic in its assessments. The network reviews can play a useful role in minimizing any regional or other biases that emerge in the ratings, as well as in alerting regional staff on the need to improve the quality of write-ups in more instances. The regional chief economists’ offices arguably can achieve the same or better results, however, at lower cost<sup>16</sup> and via interactions with regional colleagues that country teams might find more credible.

> 16 Disagreements by network reviewers typically occur after in-depth analysis of individual outliers identified from cross-country statistical analyses.  Conducting these analyses and crafting the written arguments to challenge 42 proposed ratings is a fairly time-intensive task for network reviewers.  If other regions were as assertive as ECA in proposing increases, and network reviewers disagreed with them at the same rate, there would have been about 75 disagreements instead of 42.

23