# **ASSESSMENT OF WORLD BANK IMPACT EVALUATIONS CONTAINING VFM ANALYSES**

We developed a framework to analyze the frequency, type and transparency of VFM analyses found in the sample of 168 IEs produced by the World Bank. Information from the subset of 30 IE’s found to contain any kind of VFM IE’s was tabulated in a Microsoft Excel spreadsheet for analysis by Sector Board, business line, method, and transparency of reporting.

The analysis generated three outcome variables corresponding to the type of VFM analysis observed in the studies: CBA, CEA, or general efficiency discussion (includes some VFM, but no indicators meeting the CBA or CEA criteria, and no VFM (a small number of tagged studies were found to contain no VFM analysis).

An impact evaluation was classified as having a cost-benefit analysis (CBA) if it included a comparison of estimates of Costs and Benefits (with data of costs and benefits) and one or several of the following indicators: Benefit to Cost Ratio, Cost to Benefit Ratio, Economic Rate of Return/Internal Rate of Return, Financial Internal Rate of Return, Present Value, or Net Present Value. The studies classified as CBA vary in the extent of their reporting from quick, one paragraph “back of the envelope” calculations, to more elaborate exercises. For example, we classified as a CBA the analysis in “Evaluating Preschool Programs when Length of Exposure to the Program Varies: A Nonparametric Approach” implemented in Bolivia. This IE includes benefit-cost ratios in the range of 2.28 and 3.66 resulting from sensitivity analysis of the program’s benefits under different assumptions. We classified “Does Management Matter? Evidence from India” as a “back of the envelope” CBA. Firms included in the study did not want to report internal accounts information. The evaluation’s authors reported a 130 percent rate of return in one year based on an analysis of the firm’s main cost (cost of the consultancy firm’s services) and an estimate of other costs. In a third example, we classified as CBA “Contracting-Out Dialysis in Romania: What Was the Impact?” which compares the total program Cost with the total savings of the benefitting public entity.

An impact evaluation was classified as having a cost-effectiveness analysis when it compared two or more alternative programs measuring the same outcome. Cost comparisons to programs with different outcomes were excluded from CEA classification. We classified as CEA two Cambodian experiments that presented the cost per unit increase in student promotion, dropout, and achievement in literacy and numeracy to the same impacts in a control group. We classified less direct comparisons as CEA if a comparison was present. For example, one CEA compared the marginal cost per child of an intervention that provided information to parents in Pakistan to the cost of programs with the same outcome in lowincome countries.

The general efficiency analysis (GEA) designation was applied when the IE included an efficiency discussion that did not qualify as either CBA or CEA or as another VFM method.<sup>5</sup> We classified IEs that reported average intervention costs, cost per beneficiary, the share of

> 5 Cost utility analysis, social return on investment, basic efficiency analysis, multi-criteria appraisal, rank correlation of cost versus impact, cost minimization analysis.

9