MDA [ministry, department, agency] level). Based on this definition, 1,097 out of the 3,202 projects are PSM projects.

This definition of PSM draws on the Bank’s classification of project themes and sectors.<sup>24</sup> It comprises all projects that have at least a 25 percent PSM component—whether by sector or theme. The delineation between PSM and non-PSM projects is thus based on a—somewhat arbitrary—25 percent cutoff value. In other words, more than half of the projects classified as PSM actually contain more than 50 percent nonPSM components (550 out of 1,097 PSM projects). These projects’ performance ratings may therefore not necessarily reflect the performance of their PSM components, and the resulting estimates may be subject to significant noise.<sup>25</sup> Within PSM projects, four PSM reform content areas can be distinguished based on theme codes: (i) civil service and administrative reform; (ii) public expenditure, financial management, and procurement; (iii) decentralization, and (iv) tax policy and administration.<sup>26</sup> Assigning PSM projects to a single (largest) PSM content category is problematic, because such projects are often designed as packages comprising several components relating to different PSM content categories (for example, both CSR and PFM). Rather than attempt to identify projects with a single major theme, this paper relies primarily on the percentages assigned to each public sector theme as a more accurate measure of a project’s actual PSM reform content (see annex table A.5 for the average share of the six themes across projects both in and outside the public sector).

# **4.3 Project Performance Measures**

This review uses the IEG’s project outcome rating—available across Bank projects—as the dependent variable to measure project performance. This is the main tool used to analyze the performance of the Bank’s lending portfolio, and has been employed in several related studies, including Isham, Kaufmann, and Pritchett (1995), Cruz and Keefer (2010), and Denizer, Kaufmann, and Kraay (2011).<sup>27</sup>

> 24 When preparing new projects, Bank project managers assign percentage shares for up to five of these theme and sector classifications, each totaling 100 percent. “Theme” codes are meant to reflect “the goals/objectives of Bank activities,” whereas sector codes are meant to reflect the “high-level grouping of economic activities based on the types of goods or services produced” and are “used to indicate which part of the economy is supported by the Bank intervention” (http://go.worldbank.org/CVGJVWWDF0). 25

> This holds particularly true as the thematic classifications of Bank projects only approximate the actual project content. Theme and sector classifications are entered during project preparation—but are rarely amended if later in the project cycle they no longer reflect the actual project focus (for example, after restructuring). Also, different sector units may have different ways of classifying projects, such that projects with actual public sector content may or may not be classified as such. 26

> The theme codes used for identifying PSM projects include codes 25 to 30. The PSM-related sector classifications include “Central Government Administration,” “General Public Administration,” and “Sub-National Government Administration.” For detailed definitions of these theme and sector codes, please refer to http://go.worldbank.org/CVGJVWWDF0.

> 27 While the IEG provides a number of other project ratings, this paper focuses on IEG outcome ratings. For details on the definitions of other project ratings, please refer to World Bank (2005). In particular, “IEG Institutional Development Impact Ratings” could be employed, which reflect the extent to “which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources” (Cruz and Keefer 2010). As Cruz and Keefer (2010) argue, for public sector reforms, “institutional development is precisely the point of the reforms” and these ratings may thus better reflect the performance of a project’s PSM components than the IEG’s overall outcome rating. But these ratings have several shortcomings. First, they are likely much less accurate measures than

18