Even if coordination of methodological standards was resolved, challenges to external validity remain. As pointed out by Evans and Popova (2016), program Costs differ greatly across contexts even for the same type of intervention. These Cost differences naturally result in large variations in efficiency estimates. Uncertainty in effect estimates, recall bias in expenditures, and scale can substantially influence cost-effectiveness estimates (Evans and Popova 2016). Even so, transparency in Cost reporting and assumptions can improve crosscontext comparisons and usefully quantify the nature of the variation in implementation costs. Better still, multi-arm evaluations within the same context can largely evade such thorny issues.

Impact evaluators in general seem hungry for templates, checklists and third parties to coordinate and guide their efforts. Such tools have the potential to effectively lower the effort required to build and justify new models and assuage the coordination concerns of evaluators. However, there is a lack of harmonization between the institutions that commonly utilize such tools, leaving still unanswered the coordination failure of having consistent and comparable methods.

Even so, if developed with a core consensus, a revised toolkit has the potential to crystalize accepted standards. To address the concerns about generalizability and sensitivity to specifications, these tools will likely need to be focused on a particular sector (or even intervention or outcome); allow users to explicitly model uncertainty; choose between reasonable methodological alternatives; and to transparently adjust the parameter values of the model’s assumptions to more closely align with the particulars of a specified target context and scale.<sup>10</sup> Until such models are available, systematic reviews and league tables such as those done at JPAL<sup>11</sup> and in the innovative work in the state of Washington, USA<sup>12</sup> can provide policymakers with useful information on relative cost effectiveness.

Beyond methodological considerations, data access and quality problems were also cited as significant barriers. Evaluators frequently cite difficulty in extracting reliable expense information—even from administrative data—to reflect the financial cost considerations most relevant to policymakers. Though hardly limited to the World Bank, low levels of baseline financial data collection at the World Bank impedes the ability to conduct CBA at either the start or the end of projects (IEG 2010). Economic costs are even more challenging. Data on opportunity costs are rare, notwithstanding their centrality to economic analysis. Even more challenging than these considerations of basic cost data are the important issues of apportioning Cost (and often benefit) components when an evaluated project is part of a larger multi-arm effort with multiple outcomes and benefits.

These data and methodological challenges reflect an apparent lack of up-front planning to do VFM types of analysis. While the measurement of benefits is now approached in impact evaluations with careful planning well before implementation begins, it is unusual for Cost

> 10 The second paper in the VFM Component 2 series is a case study that pioneered a tool to address many of these concerns while generating locally relevant, user-informed valuations of the environmental effects of World Bank projects across the globe.

> 11 http://www.povertyactionlab.org/policy-lessons/education/student-participation.

> 12 http://www.wsipp.wa.gov/BenefitCost.

19