# **Annex 2 – Further details on data coverage and sourcing**

Despite initial data collection for 4,348 Investment Project Finance (IPF) projects closed between fiscal year (FY) 1995 and FY2017, the paper’s analysis concentrates only on those approved between FY1995 and FY2009. This interval minimizes both missingness in select measures, which is more prevalent in projects that were approved before FY1995, and censoring, which is an issue for projects approved after 2009 as many of them had yet to close or be rated at the time of data collection. Accordingly, the paper focuses more on relatively recent World Bank IPF lending, and the analytical data set comprises 90% of projects approved between FY1995 and FY2009, which is 89% of projects that exited between FY2001 and FY2015.<sup>1</sup> Table A2.1 summarizes the distribution of projects by fiscal year.

As this paper has as a central objective, the intention to focus further on the design and preparation phase of the project cycle, an extensive data collection exercise was required in order to capture elements of the preparation task team not systematically stored in the Bank’s centralized data systems, and match them to their corresponding projects. By design, the World Bank’s administrative and project-level data systems store several fields of information in its current format only, overwriting historical data. This includes the TTL who prepares and takes a project to Board, and their Practice Manager and Country Director at that time. For example, if, as is often the case, the project’s Practice Manager at closing is not the same as the project’s Practice Manager at entry, then system data cannot be used as it will only provide information for the last known Practice Manager. To circumvent this issue, data collection turned to text scraping to avoid a manual data collection of 4,348 projects’ preparation staffing teams from the projects’ Project Appraisal Documents (PADs).

Doing so clearly proved more efficient than manual collection, however it was not without its own set of challenges. While uncommon, sometimes the challenge was simply that a project’s PAD could not always be found in electronic format in the Bank’s main document directories. This occurred more frequently for older projects whose preparation documents were scanned into the system from hard copy. A number of different templates that the document has used over the years increased the programming variations necessary to text read and locate the correct staffing information. Not only the changing layouts of documents caused problems, but also changing tendencies as to who is listed in a PAD as part of the preparation team. For example, a majority of PADs listed the Country Manager, a position that reports to the Country Director. Information for the latter was eventually sourced from the World Bank’s service “Ask an Archivist.” Sector Manager as opposed to the more elevated Sector Director uncovered a separate set of issues. Quality checks of the data collated by text scraping found cases where the person listed as Sector Manager for the project had in fact, at that time, been in the role of Sector Director, and the potential effect of these positions on project quality are not necessarily interchangeable. Therefore, first, Sector Manager at approval data had to be collected manually, and then cross-checked with HR historical reports to ensure the name collected had the title of Sector Manager and not Sector Director.

Given duplication and the number of ways in which staff may choose to have their names recorded (full name on HR’s staffing record, but only first and last name on PAD – for example), the most reliable identifier is a staff’s Unique Personal Identification number (UPI), which the data set required in order to

> 1 These shares are determined as of July 2017 when the data set was created. The missing 10% of the universe of IPF lending approvals between FY1995 and 2009 is mainly due to projects that while approved, were cancelled within permitted timeframes and without sufficient disbursement to warrant an IEG evaluation.

40