REPORT TO CONGRESS ON 

EVALUATION STANDARDS AND PRACTICES AT THE 

MULTILATERAL DEVELOPMENT BANKS 

A Report to Congress 

in response to 

Section 7029(a) of the 

Consolidated and Further Continuing Appropriations Act of 2015 

United States Department of the Treasury 

October 2015 
Report of the U.S. Department of the Treasury Pursuant to Section 7029(a) of the 

Consolidated and Further Continuing Appropriations Act of 2015 (P.L. 113-235) 

October 2, 2015 

Executive Summary 

This report has been prepared pursuant to section 7029(a) of the Consolidated and Further 

Continuing Appropriations Act of 2015 (the “FY 15 Act”). This section requires the Secretary of 

the Treasury to “instruct the United States executive director of each international financial 

institution to seek to require that such institution adopts and implements a publicly available 

policy, including the strategic use of peer reviews and external experts, to conduct independent, 

in-depth evaluations of the effectiveness of at least 25 percent of all loans, grants, programs, and 

significant analytical non-lending activities in advancing the institution’s goals of reducing 

poverty and promoting equitable economic growth, consistent with relevant safeguards, to ensure 

that decisions to support such loans, grants, programs, and activities are based on accurate data 

and objective analysis.” 

Treasury is to report to the Committees on Appropriations not later than 180 days after 

enactment of the Act on steps taken by the U.S. executive directors (the “USEDs”) and the 

international financial institutions (the “IFIs”) consistent with section 7029(a). Following 

enactment of the FY 15 Act on December 16, 2014, Treasury instructed the USED of each IFI to 

seek to require that such institution complies with section 7029(a). 

This report details actions consistent with section 7029(a) taken by Treasury and the USED 

offices at the five major multilateral development banks (MDBs) in which the United States is a 

shareholder: the World Bank, the African Development Bank (AfDB), the Asian Development 

Bank (AsDB), the European Bank for Reconstruction and Development (EBRD), and the InterAmerican Development Bank (IDB). 

The first section of the report provides an overview of Treasury’s efforts on evaluation since 

submitting its last report on evaluation to Congress in September 2014 (“Treasury’s 2014 report”), 

copy of which is attached for reference. The second section addresses each MDB’s compliance 

with the four components of the FY 15 Act’s section 7029(a). In brief, Treasury finds that: 

(i) Adoption and implementation of a publicly available evaluation policy. All MDBs 

have adopted publicly-available evaluation policies, but we have identified 

improvements that we will encourage at the AsDB and IDB. Treasury also believes 

that the MDBs should implement their evaluation policies in a way that reflects a riskbased, strategic approach to evaluation, which we will encourage at all MDBs. 

(ii) Strategic use of peer reviews and external experts. All MDBs use peer reviews and 

external experts to some extent, but their use should be more systematic and strategic. 

Treasury will seek increased use of external reviews, especially at the IDB, and the 

establishment of standard procedures and criteria for their use at all MDBs. 

1 
 

 

(iii) Conducting independent, in-depth evaluations of the effectiveness of at least 25 

percent of all loans, grants, programs, and significant analytical non-lending 

activities. Treasury’s research on evaluation best practices and its analysis of MDBs’ 

programs have resulted in broadening our understanding of the scope of what 

constitutes an “independent, in-depth evaluation.” Treasury’s 2014 report defined 

“independent, in-depth evaluation” as one that involves a visit to the project field site 

after the project is completed. This kind of in-depth evaluation, or project 

performance assessment as it is commonly known, is conducted on about four to 18 

percent of all MDB loans, grants, programs, and significant analytical non-lending 

activities. 

However, the percentage of projects that are independently evaluated through an onsite field visit is markedly higher when we include broader evaluations, such as 

country program evaluations and sector and thematic reviews. In these instances, a 

portfolio of projects is evaluated (e.g., across a theme, like water, or a group of 

specific countries). This approach to evaluation has proved to be more effective for 

learning purposes and more cost-efficient. As such, Treasury believes that it is more 

accurate to define an “independent, in-depth evaluation” as any evaluation conducted 

by the MDBs’ independent evaluation office that includes an in-depth field visit to one 

or more project sites. Under this definition, Treasury finds that the MDBs conduct 

independent, in-depth evaluations of about 38 percent of all projects. Treasury is 

encouraging the MDBs to increase coverage of all types of independent, in-depth 

evaluations, rather than seeking increased use of any single evaluation type (e.g., 

project performance assessments). 

(iv) Ensuring that decisions to support such loans, grants, programs, and activities are 

based on accurate data and objective analysis. Treasury finds that all MDBs are 

making efforts to strengthen feedback loops to ensure that evidence supports 

programmatic decision-making. However, there is room for improvement at all 

institutions. Promoting evidence-based decision-making and strengthening feedback 

loops within MDB evaluation systems are major priorities of Treasury’s enhanced 

engagement with the MDBs. 

MDB evaluation policies address governance and accountability issues related to the 

independent evaluation units, but do not address operational issues such as the strategic use of 

peer reviews or the percentage of project coverage by in-depth evaluations. Such operational 

practices are established in the evaluation units’ annual work plans, which require approval of 

the MDBs’ Boards of Directors. Thus, Treasury and the USED offices have focused their efforts 

on advocating for the principles of section 7029(a) in the context of the MDB Boards’ review of 

annual work plans, rather than seeking to modify the overarching MDB evaluation policies. 

The report concludes with a brief description of Treasury’s broader agenda on evaluation, 

particularly with regard to pressing the MDBs to make progress on the six recommendations 

highlighted in Treasury’s 2014 report. 

2 
 

 

I. Overview of Treasury’s Efforts 

In our September 2014 comprehensive review of the evaluation standards and practices at the 

five MDBs, Treasury found that, while the MDBs are leaders on evaluation in the development 

community, there are several areas in which they can continue to strengthen their standards and 

practices. Treasury believes that many of the areas for improvement are well identified in the 

language of section 7029(a). 

The following section details actions taken by Treasury and the USEDs to enhance our 

engagement on evaluation with the MDBs’ independent evaluation units, senior MDB 

management, shareholders, and other relevant stakeholders, consistent with section 7029(a). 

 Engagement with Independent Evaluation Offices. In December 2014, Treasury’s 

Principal Deputy Assistant Secretary (PDAS) for International Development Policy 

Alexia Latortue explained the implications of section 7029(a) in bilateral meetings with 

the heads of evaluation from all five major MDBs.1

 Also in December, PDAS Latortue 

gave a presentation on section 7029(a) to the Evaluation Cooperation Group (ECG), a 

consortium of chief evaluators from the five major MDBs and several other IFIs. A 

lively discussion followed the presentation, focusing on opportunities for taking concrete 

steps for improvements and acknowledging some of the constraints and cultural barriers 

to change. During late 2014 and early 2015, Treasury staff visited each MDB’s 

independent evaluation office to have further conversations on the implications of section 

7029(a) and specific steps the MDBs could take in response. In early 2015, Treasury also 

met with the team of external evaluators that was conducting an independent, external 

review of the World Bank’s Independent Evaluation Group (IEG). The review team 

ultimately incorporated many of the findings from Treasury’s 2014 report into their 

recommendations for IEG. 

 Engagement with MDB Management. Treasury has also sought to raise the profile of 

independent evaluation at the MDBs through outreach with senior management and 

engagement with all parts of the institutions that have a role in strengthening 

accountability and learning. For any progress to occur, the buy-in, leadership, and strong 

support of the MDBs’ top management are critical. Treasury’s Assistant Secretary for 

International Markets and Development Marisa Lago directly pressed AfDB President 

Donald Kaberuka for improvements to the AfDB’s recently revised evaluation policy, 

and Treasury staff played a key role in recommending specific amendments that helped 

strengthen the policy. PDAS Latortue participated in AsDB evaluation events with 

President Takehiko Nakao in Manila. Treasury also engaged with the units responsible 

for other aspects of MDB evaluation, such as the departments that design, monitor, and 

adapt quality standards for projects and track corporate results. These units are integral 

to ensuring that projects are well-designed using evaluation evidence, embedding 

evaluation into the institutional culture of the MDBs, and using evaluation to strengthen 

1

 Caroline Heider of the World Bank’s Independent Evaluation Group (IEG), Rakesh Nangia of the AfDB’s 

Independent Development Evaluation (IDEV) Department, Vinod Thomas of the AsDB’s Independent Evaluation 

Department (IED), Joe Eichenberger of the EBRD’s Evaluation Department (EvD), and Cheryl Gray of the IDB’s 

Office of Evaluation and Oversight (OVE). 

3 
 

the quality of projects. In early 2015, Treasury staff discussed the principles of section 

7029(a) with the heads of these units at each MDB and with the chairperson of the 

Managing for Development Results (MfDR) Working Group (a consortium of MDB 

results units similar to ECG). 

 Engagement with MDB Shareholders. Treasury conducted direct outreach with the 

Boards of Directors at each MDB. The USED offices highlighted section 7029(a) and 

Treasury’s 2014 report during Board discussions, particularly when reviewing the 

evaluation units’ annual reviews. The AfDB USED’s office provided all Board members 

with Treasury’s 2014 report and discussed section 7029(a) with other Board members. 

Finally, Treasury conducted outreach on the elements of section 7029(a) during bilateral 

meetings with officials from other donor country capitals throughout the year. 

 Engagement with Other Relevant Stakeholders. Treasury has also conducted outreach 

on evaluation issues with the U.S. government interagency, think tanks, and civil society. 

We have engaged with the evaluation teams from the State Department, USAID, and the 

Millennium Challenge Corporation (MCC) to share lessons learned and discuss best 

practices in evaluation. We have engaged with and participated in evaluation working 

groups hosted by leading think tanks such as the Center for Global Development and the 

Aspen Institute. Civil society organizations (CSOs) are also focused on evaluation at the 

MDBs, particularly as it relates to strengthening environmental and social safeguards, 

and Treasury continues to engage extensively with the CSOs on these issues. 

 Treasury’s Internal Efforts. Treasury’s International Affairs division has significantly 

bolstered its work on evaluation since 2014. Monitoring MDB evaluation standards and 

practices has been added to the work portfolios of two desk officers in the International 

Development Policy deputate, which oversees U.S. engagement with the MDBs. These 

desk officers consult regularly with the USED offices at each MDB to seek opportunities 

to strengthen MDB evaluation systems in accordance with the principles of section 

7029(a). They also engage regularly with other relevant stakeholders, such as external 

experts in the broader evaluation community. Treasury’s International Development 

Policy deputate has also created an Office of Development Results and Accountability 

that focuses in major part on MDB project results, evaluations, and feedback loops. 

II. Assessment of MDBs’ Compliance with Section 7029(a) 

The following section details steps taken since December 2014 by the MDBs, with strong 

support from the USED offices and Treasury, in relation to the four central components of 

section 7029(a). 

(i) Adoption and Implementation of a Publicly Available Evaluation Policy 

Overview. Formal, publicly available evaluation policies enhance the independence and 

accountability of the evaluation function. Treasury’s assessment found that each MDB has 

adopted and implemented a publicly available evaluation policy that details the principal 

responsibilities and governance structures of the independent evaluation offices, including their 

4 
 

relations with MDB management and the Boards of Directors. At all MDBs, the independent 

evaluation offices are managed under the oversight of the Boards, to which they submit all 

evaluation products, annual budgets and work programs, as well as periodic reports on actions 

taken by the MDBs in response to evaluation findings. The Boards also appoint the head of 

evaluation at all MDBs and oversee their selection process, performance review, and remuneration. 

These policies vary in their formality and the level of detail with which they address operational 

matters, such as specific evaluation practices and implementation arrangements. However, in all 

MDBs, operational matters are addressed annually in the evaluation units’ work programs, which 

require Board approval. While the MDB evaluation policies themselves do not address the 

conduct of “independent, in-depth evaluations” or establish a threshold coverage level of at least 

25 percent of all MDB projects, the USED offices each year review the work programs of the 

independent evaluation offices. Treasury is using these reviews as an opportunity to request the 

MDBs to conduct more independent, in-depth evaluations. 

Coverage of Evaluation Policies. The World Bank has a mandate for the Director-General of 

IEG and terms of reference for the independent evaluation function that collectively function as 

an evaluation policy and encompass the World Bank, the International Finance Corporation 

(IFC), and the Multilateral Investment Guarantee Agency (MIGA).2

 The mandate and terms of 

reference clearly define the principal functions of IEG, including assessments of the 

effectiveness of World Bank activities, validations of project completion reports, and 

dissemination of evaluation findings. They establish that IEG’s functions and staff are 

organizationally independent and describe governance arrangements, including IEG’s 

relationship with the World Bank’s Boards of Directors. They establish that IEG will also have 

unrestricted access to World Bank staff as well as policy and operational records. 

The AfDB’s Board approved the Independent Development Evaluation (IDEV) Department’s 

latest evaluation policy3 in March 2015, incorporating lessons and recommendations raised in 

IDEV’s 2012 self-assessment of the AfDB’s evaluation system. The new evaluation policy 

includes IDEV’s missions and core objectives, guiding principles (independence, credibility, 

impartiality, transparency, usefulness, and partnership), and governance structure, including the 

respective roles of the Board, the Board’s Committee on Development Effectiveness, and 

Management. The policy establishes that operational activities will be guided by three-year 

rolling work programs and budgets approved by the Board. Treasury engaged extensively with 

IDEV and AfDB leadership while the policy was being drafted, and succeeded in strengthening 

key components such as establishing IDEV’s full independence from AfDB Management. 

The AsDB’s Independent Evaluation Department (IED) issued a revised evaluation policy4

 in 

2009 as part of an independent review of the ASDB’s evaluation system. The policy establishes 

IED’s mandate, responsibilities, guiding principles (impartiality, independence, credibility, 

usefulness, partnership, and learning) and governance arrangements. The responsibilities of key 

stakeholders, including the Board, the Board’s Committee on Development Effectiveness, IED’s 

2

 Available at http://ieg.worldbank.org/Data/dge_mandate_tor.pdf. 

3 AfDB Evaluation Policy will be published online in fall 2015. 

4

 Available at http://www.adb.org/site/evaluation/policy. 

5 
 

 

 

Director General, Management, and AsDB staff are summarized, including principles around 

reporting, disclosure, and follow-up. 

The EBRD’s Board approved its most recent evaluation policy in 2013.5 The EBRD first issued 

an evaluation policy in 1992 and subsequently revised it several times to reflect changing 

organizational expectations and good practice. The policy sets out the evaluation-related 

activities and responsibilities of the EBRD’s Evaluation Department (EvD), as well as those of 

EBRD Management and the Board of Directors. It provides an operational framework for 

evaluation procedures and methods, and directs EvD to submit work plans (with proposed 

evaluation activities) to the Board each year. It requires Management response, internal 

circulation, and external disclosure of all evaluation findings, and sets out specific provisions 

pertaining to EvD’s chief evaluator, budget, and staff. 

The IDB has terms of reference for the Director of Office of Evaluation and Oversight (OVE) and 

an operational and governance framework document, which together function as an evaluation 

policy6

 and were developed through an independent review of the IDB’s evaluation system in 

2010. The terms of reference and framework document establish the purpose of evaluation at IDB, 

the responsibilities for OVE, its reporting relationship to the Board through the Board’s Policy and 

Evaluation Committee, and its relationship with Management. The documents also confirm that 

OVE is responsible for its work plan, budget, and quarterly and annual reports. 

Next Steps. Treasury will recommend that the IDB and the AsDB improve the transparency of 

their evaluation systems by publicly posting the documents mentioned above as standalone 

evaluation policies on their websites. We will also continue to work with USED offices and 

MDB management and staff to identify opportunities to strengthen each MDB’s ability to 

strategically guide the deployment of evaluation resources, including enhanced risk-based 

criteria for selecting when to evaluate different parts of each MDB’s portfolio and in what order. 

(ii) Strategic Use of Peer Reviews and External Experts 

Overview. Peer reviews strengthen the independence and quality of MDB evaluations. A key 

recommendation of Treasury’s 2014 report, in line with section 7029(a)’s language, was for the 

MDBs to increase the strategic use of peer reviews as part of the overall evaluation process. 

Each MDB’s independent evaluation office conducts two types of peer reviews on its evaluation 

products: (i) internal review by internal peer reviewers from the evaluation unit and from MDB 

management and operational staff, and (ii) external review by external evaluation experts, peers 

from the evaluation units of other MDBs, and stakeholders from client country governments. 

Review activities are typically conducted at two different stages: the inception report stage (e.g., 

terms of reference for the evaluation) and the draft summary stage (e.g., prior to delivery of the 

evaluation report to MDB management for its review and response). Typically, each significant 

evaluation product is independently reviewed by at least one to three external reviewers, 

including technical experts or evaluation experts, depending on the complexity of the evaluation 

product and its topic. The number and types of external reviewers needed are specified in each 

5

 Available at http://www.ebrd.com/documents/evaluation/evaluation-policy-2013.pdf. 

6

 Not available online. Treasury has requested public disclosure of these documents. 

6 
 

 

 

evaluation product’s terms of reference, which are typically available for review by the MDBs’ 

Boards of Directors. During Treasury’s consultations, some MDB evaluation units reported that 

more than three dozen different external reviewers were consulted for various evaluation 

activities in 2014. 

Coverage of Peer Reviews. Treasury’s assessment found that all evaluations completed in 2014 

by the MDBs’ independent evaluation offices underwent internal review, and an average of 56 

percent underwent external review (see table below for coverage at each MDB). This coverage 

ratio applies to all significant evaluation types: in-depth evaluations, including country program 

and strategy evaluations, sector and thematic reviews, and impact evaluations; corporate 

evaluations; and evaluation annual reports. (See Treasury’s 2014 report for definitions and 

taxonomy of each evaluation type.) This coverage ratio excludes the independent “validations” 

of MDB self-evaluations, which are typically only internally reviewed due to their large number. 

Most independent evaluation units validate 100 percent of self-evaluations, the most complex of 

which can include site visits to the project site.7 

Evaluations Subject to Peer Review 

(2014) 

MDB Internal Reviews External 

Reviews 

World Bank 100% 77% 

AfDB 100% 90% 

AsDB 100% 47% 

EBRD 100% 42% 

IDB 100% 29% 

The evaluation units report that both internal and external reviews add tremendous value to their 

work and often substantially improve the final versions of evaluation products. The MDB 

evaluation units also engage with external evaluation experts for a range of other activities, 

including: providing input on annual work plans and future evaluation products; providing 

advice, guidance, and quality control on evaluation approaches, methods, and instruments; 

providing training and workshops for evaluation staff; making presentations and providing 

trainings to evaluation and MDB staff; and conducting assessments and training of member 

country governments’ national evaluation systems. 

Some MDBs have established standing review committees of prominent academics and 

policymakers to provide advice to senior management (e.g., the Economic Policy Research 

Advisor Council at the EBRD commissions in-depth reports and evaluations of EBRD activities 

in particular sectors). 

7

 Some MDBs’ evaluation units conduct project site visits for validations of self-evaluations, particularly for private 

sector projects that pose significant risks. The World Bank’s IEG, for example, seeks to visit approximately 10 

percent of the projects that it validates for the IFC and MIGA. These site visits are selected based on risk-based 

criteria, including any projects rated as having significant environmental or social impacts (i.e., “Category A” 

projects). 

7 
 

Next Steps. Each of the evaluation units consulted indicated its support for further formalizing 

the role of the peer review function, and this will be a key area of future Treasury engagement. 

Treasury will encourage the establishment of standard procedures and criteria to guide decisions 

on when to use internal and external peer review. Treasury will also continue to advocate for 

increased usage of external reviews at all MDBs, with a special focus on the IDB, which 

currently conducts external peer reviews on less than a third of its evaluations. 

(iii) Conducting Independent, In-depth Evaluations of the Effectiveness of at Least 25 Percent 

of All Loans, Grants, Programs, and Significant Analytical Non-lending Activities 

Overview. As noted in Treasury’s 2014 report, the MDBs’ independent evaluation offices 

produce a broad range of evaluation products, from project evaluations to assessments of internal 

corporate processes. The MDBs’ independent evaluation offices use different definitions for 

what constitutes an independent, in-depth evaluation. For the purposes of this report, Treasury 

defines an “independent, in-depth evaluation” as an evaluation conducted by the independent 

evaluation office that includes a field visit to the project. Such field visits provide an additional 

level of depth to an independent evaluation. During the course of the review, Treasury 

determined that this definition encompasses a wide range of evaluation types, and that these 

multiple types enhance the overall quality and rigor of the MDBs’ evaluation portfolios. There 

are two main categories of independent, in-depth evaluations: 

 Project Performance Assessments: The most well-known type of in-depth evaluation is 

the project performance assessment, involving travel to the field site and interviews with 

project staff and beneficiaries following project completion. While useful from an 

accountability perspective (each project performance assessment is a lengthy report on 

the effectiveness of a single MDB project), they are time- and resource-intensive (costing 

$35,000-$45,000 each and requiring several months of work). Additionally, there is a 

perception that they are not the most effective learning tools, as MDB staff are unlikely to 

read lengthy reports on single projects from which it is difficult to generalize larger 

lessons. While Treasury believes that project performance assessments are an important 

element of the MDB evaluation ecosystem, they are not the only type of independent, indepth evaluation conducted by the MDB evaluation units, and we do not advocate 

increased usage of any single type of evaluation product. Treasury believes that the 

MDB should prioritize evaluations with high learning value and cost efficiency. 

 Country Program Evaluations, and Thematic and Sector Reviews: Other types of 

evaluations that include in-depth review and site visits are country program evaluations 

and thematic and sector reviews. For these evaluations, the MDB evaluation offices 

select specific themes (e.g., fragile states, safeguards), sectors (e.g., transport 

infrastructure, agriculture), or regions or countries (e.g., Central America, sub-Saharan 

Africa, Indonesia) and evaluate a group of projects under those subjects. For each of 

these evaluations, several projects are visited to conduct in-depth assessments of results, 

which provides the opportunity to draw cross-cutting lessons from different types of 

projects. Such evaluations are read widely by MDB staff and perceived to be more 

effective and actionable learning tools as they point to success and failure patterns, and 

are thus more likely to result in operational changes and increased development 

8 
 

 

 

 

 

 

effectiveness. Treasury believes that such evaluations leverage the MDBs’ independent 

evaluation offices to generate lessons and produce meaningful change, and do so in a 

more cost-effective manner. 

Impact evaluations attempt to measure the causal effects of a project using a more experimental 

approach than other types of in-depth evaluations. Impact evaluations of MDB projects are not 

typically conducted by the MDBs’ independent evaluation offices and rely heavily on operational 

teams with specialized technical expertise. While very “in-depth” and useful for learning, impact 

evaluations are not always “independent,” and so they are not included in this report. 

Coverage of Independent, In-Depth Evaluations. The overall coverage ratio was calculated by 

dividing the number of “independent, in-depth evaluations” (as described above) completed in 

2014 by the number of projects completed annually. Treasury’s assessment found an overall 

average coverage ratio of about 38 percent across the MDBs (see table below for coverage at 

each MDB). A coverage ratio for project performance assessments, which provide the most 

project-level scrutiny, was calculated by dividing the number of project performance assessments 

conducted in 2014 by the total number of projects completed annually. Treasury’s assessment 

found an overall average project performance assessment coverage ratio of about 5.5 percent 

across the MDBs.8

 Each independent evaluation office has its own criteria for selecting projects 

for in-depth evaluation, such as projects that are particularly risky, innovative, problematic, or 

relevant to institutional priorities. 

Independent, In-Depth Evaluations 

(2014 Coverage Ratio) 

MDB Project performance assessments All independent, in-depth 

evaluations 

World Bank9 7.8% 28.2% 

AfDB 18.2%10 67.5% 

AsDB 13.7% 56.1% 

EBRD11 6.1% 26.1% 

IDB12 4.2%13 39.5% 

8

 If impact evaluations were to be included, the coverage would be slightly higher. For example, since 2007, the 

World Bank has completed 92 impact evaluations and 79 are ongoing (representing approximately 7 percent of all 

projects). In the same period, the IDB has completed 20 impact evaluations and 62 are ongoing (representing 11 

percent of all projects). 

9 World Bank data does not include non-lending activities of the World Bank, due to both their sheer volume (e.g., 

the World Bank completed 1,215 non-lending activities in 2014) and methodological difficulties around what 

constitutes a “significant” and “analytical” non-lending activity, as described in the FY 15 Act. The data do, 

however, include non-lending activities of IFC. 

10 Rather than conduct project performance assessments of individual projects, the AfDB’s evaluation unit conducts 

in-depth evaluations on clusters of similar projects (e.g., projects in the same sector or region), which are perceived 

to be more relevant and useful for AfDB staff. 

11 The EBRD’s evaluation unit selects from a list of pre-identified “evaluation-ready” projects (e.g., projects that 

have been completed for at least 18-25 months and are not in corporate recovery). While the list does not include all 

completed projects (e.g., while the EBRD completed 251 projects in 2014, the list included 115), Treasury believes 

this is the appropriate approach for EBRD, where most projects are investment operations. If all completed projects 

were included, the EBRD’s coverage ratios in the table above would decrease to 2.8% and 12.0%. 

9 
 

 

 

 

Next Steps. Treasury will continue to urge the evaluation units to conduct a larger number of indepth evaluations at all the MDBs, with a special focus on the World Bank, EBRD, and IDB. 

We will especially use Board reviews of annual evaluation work plans as an opportunity to seek 

such increases. 

Qualifying Considerations. Treasury notes that a reliance on the use of “independent, in-depth 

evaluation” as the only criterion for assessing the effectiveness of MDB evaluation systems 

poses several issues. 

 First, the criterion does not consider valuable monitoring and evaluation work that is not 

“independent,” such as MDB operational staff efforts to improve project implementation 

quality through rigorous monitoring, or impact evaluations that, as discussed, are 

rigorously in-depth but not always conducted by an independent entity. Moreover, strong 

monitoring is the foundation of any monitoring, evaluation, and learning system. 

 Second, measuring the coverage of in-depth evaluations does not consider the influence 

of other types of institutional evaluations, such as process evaluations or corporate 

evaluations, which rarely involve project-level assessments and yet can have 

considerable impact on the MDBs’ activities and development effectiveness. For 

example, a 2014 IDEV evaluation of operational procurement at the AfDB provided key 

inputs to an ongoing procurement reform. Similarly, a 2012 IEG evaluation of the World 

Bank’s use of impact evaluations led to the adoption of a more streamlined approach to 

their use. 

 Finally, calculating the coverage ratio of any type of evaluation over an entire MDB 

portfolio is complicated by the fact that MDB projects are active for long and variable 

periods of time. If MDB projects were more standard, well-defined, and cyclical, it 

would be easier and more effective to set – and hold MDBs accountable to – a target 

coverage ratio. However, the active life of an MDB loan can span decades and can vary 

considerably from one project to the next, which makes such standard-setting difficult 

and impractical. 

(iv) Ensuring that Decisions to Support such Loans, Grants, Programs, and Activities are 

Based on Accurate Data and Objective Analysis 

Overview. To keep the MDBs accountable to their shareholders, clients, and beneficiaries, 

Treasury has long sought to enhance the MDBs’ ability to determine whether development 

programs are delivering on their intended objectives. The fourth component of section 7029(a) 

is, thus, a core aspect of Treasury’s work on evaluation. 

12 For each evaluation, the IDB’s evaluation unit establishes a “universe” of evaluable projects, which can include a 

large number of projects completed over varying time periods. The entire IDB evaluation universe for 2014 

comprised 2,327 projects, far more than were actually completed in 2014. 

13 Similar to the AfDB, the IDB’s evaluation unit conducts in-depth evaluations on clusters of similar projects. 

10 
 

 

Coverage of Use of Evidence in Project Design and Feedback Loops. While the independent 

evaluation offices are integral to the overall MDB evaluation systems, the MDBs’ operational 

staff, results departments, and internal control units typically have front-line responsibility for 

ensuring that evidence is used in decision-making. Such efforts occur at all stages of the MDB 

project cycle. 

 Using Evidence in Project Design. Each MDB has established standard requirements 

for integrating evaluation components upfront in project proposals. These standard 

requirements include: strong results measurement frameworks, with key performance 

indicators to measure and evaluate results during and after project implementation to 

determine whether specific and tangible results are achieved; evaluation plans, including 

ensuring that baseline data are collected at project inception; risk assessments; and 

“lessons learned” sections of the proposal documents that describe the evidence from 

previous evaluations that was used to inform the solutions being proposed, including 

assessments of the MDB’s prior experience in the topic/sector of the project. 

The quality control on whether these requirements are effectively met is uneven across 

the MDBs, but each institution is making efforts to strengthen its results measurement 

frameworks. To help project designers make greater use of evidence, for example, the 

MDBs have developed project database systems with information on development 

outcomes and evaluation findings. Prior to Board approval, the MDBs’ internal control 

units conduct “readiness reviews,” rate the project’s “quality-at-entry” and anticipated 

development outcomes, and ensure that the project meets threshold standards of 

preparedness for ex-post evaluation. Project teams are expected to address deficiencies 

before the project can be brought to the Board for approval. Such up-front planning helps 

embed evaluation into the institutional culture of the MDBs. It is most effective when it 

is tied to employee performance management, which creates incentives for operational 

staff to take evaluation seriously and recognize its value in strengthening project quality. 

 Using Evidence to Improve Implementation: Strengthening Feedback Loops. 

During project implementation, evidence can provide essential real-time data to improve 

a project’s efficacy. Each MDB has developed, and is seeking to strengthen and 

mainstream, a range of evidence-based monitoring and evaluation tools to improve the 

effectiveness of project implementation. A critical component of such efforts is finding 

ways to better incentivize project staff to enhance real-time results reporting and use midstream data to undertake mid-course corrections when and where appropriate. 

Next Steps. Treasury believes that these aspects of MDB evaluation systems can be 

strengthened, including by: increasing staff training on managing for development results; 

linking the use of evaluation evidence to staff performance management and compensation; 

improving the quality and strategic use of impact evaluations; conducting a greater number of 

evaluations to build a more comprehensive evidence base for learning and knowledge; assessing 

the results and sustainability of projects several years after completion, in order to generate more 

useful learning on how to make future projects more sustainable; and upgrading existing learning 

and knowledge systems to strengthen feedback loops for project design. 

11 
 

Treasury and the USEDs will continue to advocate for these objectives through engagement with 

the MDBs’ senior management and during Board review of major evaluation and results reports. 

Most recently, each MDB’s Board reviewed the 2014 annual reports conducted by the 

independent evaluation units and results departments. Other recent opportunities for USED 

engagement on these topics included Board review of IEG’s FY 2016 Work Plan, the World 

Bank FY 2014 Results and Performance Report, and the 2014 IEG Client Survey. 

Treasury will continue to seek opportunities to urge the MDBs to strengthen evaluation feedback 

loops and enhance the use of evidence to inform decision-making, including by linking the use of 

evaluation evidence in MDB activities to staff performance management and compensation. We 

also recognize that recipient countries have an important role in the development of feedback 

loops. Therefore, Treasury is also working with the MDBs and other relevant stakeholders to 

encourage greater attention to data and statistical capacity building in developing countries, as 

part of the implementation of the Financing for Development outcomes and the Sustainable 

Development Goals. 

III. Conclusion 

Treasury’s assessment has found that all MDBs meet the standards of section 7029(a), but work 

remains to be done to enhance the role of evaluation in MDB activities. Given our view of the 

evaluation function’s importance at the MDBs, Treasury has welcomed the opportunity to 

redouble our commitment to strengthening standards and practices, and we are implementing a 

proactive agenda based on the four elements of section 7029(a) and the six priority areas 

identified in our September 2014 report. As explained in section I, Treasury has already 

enhanced its engagement with the MDBs on each of these priority areas, and will continue our 

efforts over the next year. The priority areas are: 

1) Adopting risk-based, strategic portfolio approaches to evaluation. Treasury 

highlighted this recommendation in consultations with senior management and during 

Board review of each MDB evaluation unit’s 2015 work plan. 

2) Increasing the use of external and peer reviews as part of the overall evaluation 

process. As noted, Treasury is encouraging each MDB to increase the strategic use of 

external and peer reviews, particularly at institutions where current coverage lags behind 

others. For example, Treasury has received a commitment from the AfDB’s IDEV to 

systematize the process for increasing the use of external reviews. 

3) Instituting regular external evaluations of the MDBs’ independent evaluation 

offices. Treasury highlighted this recommendation during Board review of each MDB 

evaluation unit’s work plan, underscoring that such external evaluations should be 

aligned with the tenure of the independent evaluation office’s director. We consulted 

extensively with the panel conducting the recent external review of the World Bank’s 

IEG, and the panel’s findings were largely consistent with Treasury’s six priority areas. 

In the coming year, the USED office at the World Bank will work with other 

shareholders to formalize several of the review’s recommendations. 

12 
4) Creating incentives and feedback loops to strengthen monitoring and evaluation 

(M&E) and the generation, access to, and use of learning and knowledge in MDB 

programming. Treasury highlighted this recommendation with the evaluation units and 

managements of each MDB, underscoring that strengthening learning and creating 

institutional cultures around evaluation require joint efforts by multiple MDB units. 

Treasury is assessing the strengths and weaknesses of the current knowledge and learning 

ecosystems at each MDB, in order to identify institution-specific recommendations. 

5) Strengthening public disclosure of data in transparent and usable forms. Treasury 

highlighted this recommendation at Board reviews and in consultations with 

management, and is focusing on opportunities to strengthen the user-friendliness of 

evaluation data and findings. 

6) Bolstering inter-MDB collaboration (and peer pressure) on evaluation. Treasury 

presented our recommendations to the ECG and provided suggestions on how its role 

could be enhanced, especially to help those institutions that are lagging behind to catch 

up with best practices. Throughout 2015, Treasury will push forward this 

recommendation with current ECG Chair Caroline Heider, Director General of the World 

Bank’s IEG. Treasury is also consulting with the MfDR Working Group on how its role 

can be bolstered to strengthen inter-MDB collaboration on results measurement and 

evidence-based decision-making. 

Throughout 2015, Treasury and the USED’s offices at each MDB will continue to advocate for 

stronger evaluation standards and practices. 

13