REPORT TO CONGRESS ON

EVALUATION STANDARDS AND PRACTICES AT THE 

MULTILATERAL DEVELOPMENT BANKS

A Report to Congress 

in response to

Section 7029(a) of the 

Consolidated Appropriations Act, 2018 (P.L. 115-141)

United States Department of the Treasury

August 2018
Report of the U.S. Department of the Treasury Pursuant to Section 7029(a) of the

Consolidated Appropriations Act, 2018 (P.L. 115-141)

August 2018

Executive Summary

As part of its focus on high-quality development projects that achieve tangible results, Treasury

prioritizes evaluation issues in its engagement with the MDBs. Evaluation of development

programs helps to a) provide accountability by assessing program performance and b) support

learning by determining why a program performed as it did and how it can be improved. The

five major multilateral development banks (MDBs) of which the United States is a shareholder –

the World Bank, the African Development Bank (AfDB), the Asian Development Bank (AsDB),

the European Bank for Reconstruction and Development (EBRD), and the Inter-American

Development Bank (IDB) – are viewed as standard-setters among development organizations in

employing evaluation to assess and improve their programs.

1

 Treasury seeks further

strengthening of the MDBs’ already robust evaluation functions, while utilizing evaluation

lessons to improve projects, strategies, and operations in its position on the MDBs’ Boards of

Directors. 

Section 7029(a) of the Consolidated Appropriations Act, 2018 (the “FY 18 Act”) requires the 

Secretary of the Treasury to “instruct the United States executive director of each international 

financial institution to seek to require that such institution adopts and implements a publicly 

available policy, including the strategic use of peer reviews and external experts, to conduct 

independent, in-depth evaluations of the effectiveness of at least 25 percent of all loans, grants, 

programs, and significant analytical non-lending activities in advancing the institution’s goals of 

reducing poverty and promoting equitable economic growth, consistent with relevant safeguards, 

to ensure that decisions to support such loans, grants, programs, and activities are based on 

accurate data and objective analysis[.]” Section 7029(a) also requires the Secretary of the 

Treasury to submit a report on steps taken in FY 17 by the U.S. Executive Directors (USEDs)

and the international financial institutions consistent with 7029(a) compared to the previous 

fiscal year. 

In accordance with the above, this report details actions taken by Treasury, the USEDs, and the 

five aforementioned MDBs in FY 17.2, 3 Section I of the report reviews the MDBs’ performance 

on the four central components of section 7029(a). Section II provides an overview of Treasury 

and the USEDs (hereafter referred to collectively as Treasury)’s engagement with the MDBs on 

 

1 For an overview of evaluation practices at the MDBs, and discussion of these practices relative to other development 

organizations, see Treasury’s 2014 report to Congress on MDB evaluation. 

2 When referring to a specific MDB, this report is based on data for that MDB’s FY 17, as opposed to the U.S. 

government’s FY 17. The World Bank’s FY 17 ran from July 2016-June 2017; at the other four MDBs FY 17 was 

the 2017 calendar year. 

3 The focus on FY 17 in this year’s reporting language means there is overlap between sections I(i), I(iv), and II of 

this report and last year’s comparable report, which included data through October 2017 for the aforementioned 

sections. 
evaluation issues. The report finds that with input from Treasury, the MDBs strengthened their 

performance on three of the four central components of section 7029(a) in FY 17, as follows. 

(i) Adoption and implementation of a publicly available evaluation policy. The World Bank

began revising the mandate for its independent evaluation office (IEG) to better articulate the

office’s role and independence, and began developing a World Bank-wide evaluation

framework. 

(ii) Strategic use of peer reviews and external experts. In FY 17, almost all evaluations

completed by the MDBs’ independent evaluation offices underwent internal review, and, across

the MDBs, an average of 59 percent underwent external review. The latter is 7 percentage points

higher than FY 16. 

(iii) Conducting independent, in-depth evaluations of the effectiveness of at least 25 percent of

all loans, grants, programs, and significant analytical non-lending activities. In FY 17, the

average coverage ratio of independent, in-depth evaluations4across the MDBs was 43 percent. 

Four of the five MDBs maintained coverage levels distinctly above the 25 percent threshold, but

EBRD’s coverage fell below the threshold. While closer analysis indicates that the EBRD’s FY

17 performance is an anomaly based on specific factors, Treasury is focused on ensuring it does

not repeat in FY 18. 

(iv) Ensuring that decisions to support such loans, grants, programs, and activities are based 

on accurate data and objective analysis. Treasury has made learning from evaluation results a 

priority in its engagement with the MDBs, and thus welcomes the MDBs’ progress in this area. 

For example, the World Bank continued to strengthen feedback loops from project beneficiaries; 

the AfDB, EBRD, IDB, and AsDB deployed or improved tracking systems to hold themselves 

accountable for implementing evaluation recommendations; and the AsDB took a holistic 

approach to enhancing communication of evaluation knowledge. 

I. Assessment of the MDBs’ Performance Consistent with Section 7029(a)

(i) Adoption and Implementation of a Publicly Available Evaluation Policy

Overview. Each MDB has an evaluation policy that details the principal responsibilities and 

governance structure of its independent evaluation office, including its relationship with the 

MDB’s management and the Board of Directors.5 These policies are publicly available for all 

MDBs other than the IDB. 

Actions in FY 17. The World Bank was the only MDB that revised its evaluation policy in FY

17. Specifically, the World Bank began revising the mandate for its independent evaluation

 

4 As explained in section II(iii), Treasury defines an “independent, in-depth evaluation” as an evaluation conducted 

by the independent evaluation office that includes a field visit to the project site. 

5 At each MDB, the independent evaluation office is under the oversight of the Board of Directors, to which it 

submits evaluation products, its annual budget and work program, and periodic reports on actions taken by the MDB 

in response to evaluation findings. The Board also appoints the head of the independent evaluation office and

oversees performance review and remuneration.
office (IEG) – which serves as the office’s policy – and developing a World Bank-wide

evaluation framework. Among other things, the revised mandate seeks to better articulate IEG’s

independence and dual accountability and learning role. Key objectives of the evaluation

framework include providing common principles for evaluations conducted by different actors

across the World Bank Group, and clarifying the roles and responsibilities of these actors. 

Priorities for FY 18. Treasury’s top priorities in this area include advocating for the IDB’s

independent evaluation office (OVE) to publish its evaluation policy on its website, and working

with the World Bank to finalize development of its new evaluation framework. 

(ii) Strategic Use of Peer Reviews and External Experts

Overview. MDB evaluation products receive two types of peer review: (i) internal review by

peer reviewers from the evaluation office as well as management and operational staff from the

MDB; and (ii) external review by peers from the evaluation offices of other MDBs, stakeholders

from shareholder governments, and other evaluation experts outside the MDB. The majority of

significant evaluation products are independently reviewed by at least one to three external

reviewers, including technical experts or evaluation experts, depending on the complexity of the

evaluation product and its topic. 

Coverage of Peer Reviews. Treasury finds that almost all significant evaluations6that the

MDBs’ independent evaluation offices completed in FY 17 underwent internal review, and

across the MDBs, an average of 59 percent benefitted from external review. As Table 1

indicates, there was wide variation in external review coverage, with the IDB lagging behind the

other MDBs. IDB staff indicate that they use external reviews selectively – while they did not

use external reviews for evaluations completed in FY 17, they intend to use them in the years

ahead. OVE also utilizes external reviews in a number of other ways – from conducting

trainings to providing technical expertise on evaluations and even assessing the quality of IDB

evaluations conducted by offices other than OVE.

 

6 This includes project evaluations, sector and thematic reviews, country program evaluations, impact evaluations,

corporate evaluations, and evaluation annual reports. It excludes the independent “validations” of self-evaluations

of projects by MDB staff, which are typically only reviewed internally due to their large number.
Table 1. Evaluations Subject to Peer Review

(FY 17 Coverage Ratio)

MDB

Internal 

Reviews

External 

Reviews

World Bank 100% 67%

AfDB 56% 78%

AsDB 100% 100%

EBRD 100% 50%

IDB 100% 0%

MDB Average 

(non-weighted) 91% 59%

Chart 1, beneath Table 1, shows that external review coverage across the MDBs improved in FY

17, driven by a significant increase in external reviews at the AsDB. 

Priorities for FY 18. Treasury will encourage IDB OVE to increase its use of external peer

reviews. While Treasury does not believe it is necessary or cost-effective to conduct external

reviews for all significant evaluations, they provide an important function. 

(iii) Conducting Independent, In-depth Evaluations of the Effectiveness of at Least 25

Percent of All Loans, Grants, Programs, and Significant Analytical Non-lending

Activities. 

Overview. The MDBs’ independent evaluation offices produce a broad range of evaluation 

products, from project evaluations to assessments of internal corporate processes, and use 

0%

20%

40%

60%

80%

100%

120%

Average

(non-weighted)

World Bank AfDB AsDB EBRD IDB

Chart 1. MDB External Review Coverage

FY 17 vs. FY 16

2016 2017
different definitions for what constitutes an independent, in-depth evaluation. For the purposes 

of this report, as with Treasury’s previous reports, Treasury defines an “independent, in-depth 

evaluation” as an evaluation conducted by the independent evaluation office that includes a field 

visit to the project site. Field visits can add an additional level of depth to an independent 

evaluation. 

Coverage of Independent, In-Depth Evaluations. Calculating the coverage of independent, indepth evaluations is complicated by the fact that the MDBs’ independent evaluation offices 

produce both evaluations of individual projects and broader evaluations on country programs and 

specific themes and sectors (e.g., fragile states or infrastructure) that include multiple projects. 

To account for this, Treasury calculates each MDB’s overall evaluation coverage ratio by 

dividing the number of projects that received a field visit during an evaluation completed in FY 

17 by the number of projects completed annually. 

As Table 2 indicates, Treasury finds an average coverage ratio of 43 percent across the MDBs, 

with all MDBs other than the EBRD achieving a solid coverage ratio. Closer analysis, and 

discussions with staff from the EBRD’s independent evaluation office (EvD), confirm that

EBRD’s FY 17 coverage ratio is an anomaly that does not capture EvD’s commitment to “indepth evaluations,” as measured by project site visits. EvD reported a low number of site visits 

for FY 17 due to a confluence of factors, including the types of evaluations completed (e.g., 

corporate evaluations that do not require site visits) and the fact the Treasury’s methodology only 

counts site visits for evaluations completed in FY 17 (during the year, EvD conducted a number 

of site visits for evaluations scheduled to be completed in FY 18). Treasury expects EBRD’s 

coverage ratio to rise in 2018. 

Table 2. Projects Subject to 

Independent, In-Depth Evaluation

(FY 17 Coverage Ratio)

MDB Coverage Ratio

World Bank 31%

AfDB 87%

AsDB 63%

EBRD 3%

IDB 31%

MDB Average 

(non-weighted) 43%

Chart 2 shows that the MDB average coverage ratio of 43 percent average is lower than the 

average for 2016 and 2015, though higher than 2014. It illustrates the link between the decline 

in the overall coverage ratio and the fall off in coverage at the EBRD, while confirming that the 

EBRD’s FY 17 coverage ratio is an outlier. 
Priorities for FY 18. Treasury will work closely with EBRD EvD to ensure that its coverage 

ratio rises as expected, and will advocate for the independent evaluation offices at the other 

MDBs to continue to conduct a large number of in-depth evaluations. Treasury will also review 

its methodology for calculating the coverage of “independent in-depth, evaluations” due to the 

methodological challenges illustrated by the EBRD in FY 17 and the qualifying considerations 

below. 

Qualifying Considerations. Treasury notes that there are some issues with relying on the 

coverage ratio of “independent, in-depth evaluations” as the only or even the main criterion for 

assessing the effectiveness of MDB evaluation systems. First, the criterion does not consider 

valuable monitoring and evaluation work that may not be considered “independent.” For 

example, it excludes impact evaluations, such as randomized control trials, which attempt to 

measure the causal effects of projects using more experimental approaches than other types of indepth evaluations. While rigorously “in-depth” and useful for learning, the independent 

evaluation offices do not conduct most impact evaluations at the MDBs. Second, it does not 

adequately consider institutional evaluations such as process evaluations or corporate 

evaluations, which rarely involve project-level assessments and yet can have considerable impact 

on the MDBs’ activities and development effectiveness. Third, the number of site visits involved 

in an independent, in-depth evaluation varies based on its subject and type. As the MDBs’ 

independent evaluation offices do different types of evaluations on different subjects each year, 

the number of site visits fluctuates on an annual basis. An MDB’s coverage ratio in a given year 

is thus less telling than the trend line over a longer period of time. 

(iv) Ensuring that Decisions to Support such Loans, Grants, Programs, and Activities are 

based on accurate Data and Objective Analysis

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2014 2015 2016 2017

Chart 2. Coverage of Independent, 

In-Depth Evaluations: FY 14-FY 17

World Bank

AfDB

AsDB

EBRD

IDB

MDB Avg,
Overview. The MDBs have robust requirements and systems to facilitate the feedback of 

evaluation results into project design and implementation, strategies, and policies. For project 

design, the MDBs require that evaluation components, such as results measurement frameworks

and the identification of evidence from previous evaluations that was used to inform the design 

of the project, are included upfront in project proposals. The MDBs also have an array of 

feedback loops to improve projects implementation in real-time, including regularized progress 

monitoring reports, and prepare project completion reports. And the MDBs have systems that 

track the implementation of recommendations from the independent evaluation offices, thereby 

holding themselves accountable for follow through. 

Actions in FY 17. The MDBs continued to improve learning from evaluation results in FY 17. 

At the World Bank Group, among many other actions, the International Finance Corporation 

(IFC) developed a new framework for assessing and tracking project development impacts, and 

the World Bank continued to strengthen project beneficiary feedback mechanisms, so that 

project implementation can be adjusted in real-time based on beneficiary experience. The 

AfDB’s independent evaluation office (BDEV) began evaluations of the AfDB’s project quality 

and monitoring systems, and also implemented the Bank’s new evaluation recommendation 

tracking system. EBRD EvD also deployed its new recommendation tracking system, and 

engaged in a range of activities to communicate evaluation lessons learned to project staff and 

the Board (e.g., training classes). Similarly, IDB OVE conducted its first validation of the data 

in IDB’s recommendation tracking system, while continuing to use an array of tools to 

disseminate evaluation results. Finally, the AsDB’s independent evaluation office (IED) 

developed an overarching strategic plan to improve how it shares evaluation knowledge, 

including events, publications, systems (including the AsDB’s recommendation tracking 

system), and trainings. 

Priorities for FY 18. Learning was a top priority in Treasury’s engagement at the MDBs on 

evaluation in FY 17, and it will continue to be a priority in FY 18. In particular, Treasury will 

work closely with the MDBs as they refine their recommendation tracking systems to ensure 

they provide robust accountability over the longer-term, and continue its efforts to strengthen 

incentivizes and requirements to incorporate results from previous evaluations in project design. 

II. Summary of Treasury’s Engagement with the MDBs on Evaluation

Evaluation remained a key component of Treasury’s engagement with the MDBs in FY 17. 

Treasury’s approach was guided by the four components of section 7029(a), with a special focus,

as mentioned above, on learning. Other key priorities included helping the independent

evaluation offices develop highly strategic work plans -- e.g., focusing on evaluations in priority

areas and that are timed to influence key Board decisions -- and working to ensure the offices

retain their independence and are well-resourced. 

Treasury also honed in on specific priorities at each of the MDBs, including the following:

 World Bank -- Treasury helped develop the revised IEG mandate, increasing the

emphasis on learning in the mandate, and the World Bank-wide evaluation framework.
 AfDB – Treasury advocated for timely follow through on implementation on evaluation

recommendations, most notably recommendations from the 2016 comprehensive

evaluation of the AfDB’s development results.

 AsDB – Treasury engaged heavily on evaluations of key institutional tools and lending

instruments, and oversaw an external assessment of the AsDB’s overall evaluation

function (including IED). 

 EBRD – Treasury guided and oversaw an external assessment at the EBRD, while also

pushing for enhanced follow through on evaluation recommendations and dissemination

of evaluation knowledge.

 IDB – Like at AsDB and EBRD, Treasury guided and oversaw an external assessment of

IDB’s evaluation function. Treasury also served on the selection panel for the new OVE

Director, ensuring that a strong individual with an appropriate mandate was selected. 

Looking ahead, the external assessments at the AsDB and IDB are now complete, the EBRD’s is

ongoing, and a fourth external assessment, at the AfDB, will be conducted in 2018. These

external assessments, which are conducted from time to time, provide a valuable opportunity to

enhance the effectiveness of the MDBs’ evaluation functions. The assessments produce specific

recommendations, and Treasury will engage closely in follow-up discussions to implement

appropriate recommendations. Treasury will also engage in the process for selecting a new

Director for the AfDB’s evaluation office.