away from the precise rate‐of‐return calculations toward broader sectoral analyses and public expenditure reviews. They noted three critical areas for proper project appraisal, namely: the rationale for public intervention, fiscal impact of the project and contribution to poverty reduction. These aspects have been incorporated into recent World Bank guidance on doing EA and CBA for investment project financing (IPFs).

The analysis presented in this note suggests that in addition to the elements noted by Devarajan et al (1997), project EA should be seen as a means to think through design and implementation issues with a focus on assessing ways to improve performance and manage risks to achieve targeted goals. This is a particularly practical approach given that the estimated level of ERR is rarely a factor for deciding whether to proceed with a project or not (World Bank 2010). Also, the Bank is putting more attention on how to use project risk ratings to improve the performance of the project portfolio.

The EA could be used to strengthen the discussion in the risk section of the PAD and the rationale for the project risk ratings (or the SORT). In order to achieve this, it will be important to pay closer attention to the assumptions used in calculating ERRs. There may also be a need to more systematically track ERRs reported in the PADs and ICRs to ensure these are systematically and consistently captured, which, along with other readily available country and project level variables, could greatly facilitate gaining a deeper understanding of the factors that determine the ERR.

A number of recommendations could help strengthen the link between the EA and the SORT and thereby more strongly inform project design.

- I. **Automate the process for systematically tracking ERRs reported in PADs and ICRs** .  In the course of this review, it was noted that some of the ERRs estimated and reported in the PADs and ICRs were not always systematically captured in the IEG database.  For example, some PADs/ICRs reported multiple ERRs (sometimes by project component) that were often reflected as an average in the IEG database or not captured at all.  Systematically capturing estimated ERRs would provide more refinement to the data and provide a more robust estimate of the levels of rates of returns for different sectors/projects and sub‐components of a project.  This system could be captured through the Bank’s management information system (or Ops Portal).  This would enable the systematic capture of ERRs (even when this is done only for specific components or involves multiple estimates or for project restructuring).   This could then be downloaded automatically to the IEG ICR Review templates and validated by IEG at the time of their independent review.

- II. **Place more emphasis on the EA as a way to systematically think through project design and how risks will impact design** . This would also include devising an approach to risk analysis that focuses on informing the design of the project or the approach to implementation.  The aim would be to use the EA to devise approaches to increase the project’s expected return and reduce the risk to the PDO. Risk analysis could also be used to prioritize and make explicit in the results framework what variables to monitor more closely during implementation.  This analysis would then be used to discuss risk mitigation in the Risk section of the PAD and inform the project (SORT) ratings.  There could also be more systematic discussions during the PCN and Decision Review meetings by requiring that reviewers comment specifically on whether the EA adequately underpins the design or approach outlined in the project; or whether it provides sufficient evidence to demonstrate that the chosen design or implementation approach is the best to maximize the project’s expected return.

- III. **Strengthen the database on EA in order to facilitate better analysis and identify flags, etc. for managing the portfolio** .  Collect project‐by‐project information on cost overruns and implementation

23