a

GAO

United States General Accounting Office

Report to the Chairman, Committee on 

Financial Services, and to the Vice 

Chairman, Joint Economic Committee, 

House of Representatives

June 2003 INTERNATIONAL 

FINANCIAL CRISES

Challenges Remain in 

IMF’s Ability to 

Anticipate, Prevent, 

and Resolve Financial 

Crises

GAO-03-734
While the Fund’s new vulnerability framework is more comprehensive than 

its previous efforts, it is too early to assess whether it will improve Fund 

efforts to anticipate crises. The new framework uses the Fund’s major 

forecasting tools, the World Economic Outlook (WEO) and the Early 

Warning System (EWS), which have not performed well in anticipating prior 

crises. The forecasting of crises has been historically difficult for all 

forecasters. 

The Fund, with the World Bank, has made progress in implementing 

initiatives to prevent crises, but several challenges remain. To obtain better 

information about country financial sector weaknesses, the Fund and Bank 

introduced the Financial Sector Assessment Program (FSAP) to report on 

member countries’ financial sectors and the Reports on the Observance of 

Standards and Codes (ROSC) to assess member countries’ adherence to 12 

standards. Assessments have not been completed in some major emerging 

market countries primarily because participation is voluntary, and use of this 

information has been mixed. For example, some private sector market 

participants have found the reports untimely, outdated, and dense. 

Participation Gaps by 33 Major Emerging Market Countries in Key Assessments, 1999-2003 

Number of assessments 

The Fund is considering two approaches to restructuring unsustainable 

sovereign debt; however, there are significant challenges to implementing 

them. One approach involves creating an international legal framework that 

would allow a specified majority of a country’s external creditors to 

restructure most private sector loans. Under the second approach, the Fund 

is encouraging members to include renegotiation clauses in individual 

bonds. Many private sector representatives wish to maintain the existing 

process in which the Fund assists resolution by providing loans to some 

eligible members. In response to concerns that its resources may have 

unintended negative impacts during a crisis, the Fund has clarified and 

strengthened its criteria for lending to members experiencing crises. 

Building on reform initiatives 

instituted after the Mexican 

financial crisis, the IMF 

implemented new initiatives in the 

mid-1990s to better anticipate, 

prevent, and resolve sovereign 

financial crises. GAO was asked to 

assess (1) the IMF’s framework for 

anticipating financial crises, (2) the 

status of key IMF reform initiatives 

to prevent financial crises, and (3) 

new IMF proposals to resolve 

future financial crises. 

GAO recommends that the 

Secretary of the Treasury instruct 

the U.S. Executive Director of the 

Fund to work with other Executive 

Board members to encourage the 

Fund to 

• improve the timeliness of 

FSAP and ROSC reports; 

• expand the coverage, 

frequency, and publication of 

updates of participants’ 

implementation of FSAP and 

ROSC recommendations; 

• improve the FSAP and ROSC 

reports’ readability; and 

• increase participation in the 

FSAP and all standards of the 

ROSC and consider making 

participation mandatory. 

Treasury, IMF, and the World Bank 

generally agreed with the report’s 

recommendations. The IMF stated 

that we gave WEO and EWS 

forecasts greater importance than 

is warranted in anticipating crises. 

However, we focused on the only 

mature and quantifiable elements 

of the vulnerability framework. 

www.gao.gov/cgi-bin/getrpt?GAO-03-734.

To view the full report, including the scope 

and methodology, click on the link above. 

For more information, contact Joseph A. 

Christoff at (202) 512-8979 or 

christoffj@gao.gov. 

Highlights of GAO-03-734, a report to 

Chairman, House Committee on Financial 

Services and Vice Chairman, House Joint 

Economic Committee 

June 2003

INTERNATIONAL FINANCIAL CRISES

Challenges Remain in IMF's Ability to 

Anticipate, Prevent, and Resolve 

Financial Crises 
Page i GAO-03-734 International Financial Crises

Contents

Letter 1

Background 2

Results in Brief 4

Too Early to Determine If Fund’s New Process for Anticipating 

Financial Crises Improves on Past Efforts 7

Some Progress in Implementing Crisis Prevention Initiatives, but 

Challenges Remain 15

The Fund’s Efforts to Better Resolve Future Financial Crises Face 

Significant Challenges 24

Conclusion 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 31

Appendixes

Appendix I: Objectives, Scope, and Methodology 34

Appendix II: Private Sector Structured Interview Participants 37

Appendix III: Assessment of IMF Forecasting 38

GAO Analysis Used Standard Econometric Techniques for Forecast 

Evaluation 38

Appendix IV: Eighty-seven Emerging Market Countries 50

Appendix V: Standards, Codes, and Principles Assessed under IMF and 

World Bank Reports on the Observance of Standards and 

Codes 53

Appendix VI: Update on the International Monetary Fund’s Safeguards 

Assessments 56

The Fund Introduces Safeguards Assessments for Member 

Countries That Currently Borrow 56

Implementation of the Fund’s Assessment Recommendations 57

Appendix VII: Fund/World Bank FSAP and ROSC Process 59

Appendix VIII: Country Participation in and Publication of FSAPs and 

ROSCs 60

Appendix IX: Fund Resources Provided under the Supplemental Reserve 

Facility 64

Appendix X: Comments from the Department of the Treasury 65

Appendix XI: Comments from the International Monetary Fund 67

GAO Comments 70
Contents

Page ii GAO-03-734 International Financial Crises

Appendix XII: Comments from The World Bank 72

GAO Comments 74

Appendix XIII: GAO Contacts and Staff Acknowledgments 75

GAO Contacts 75

Staff Acknowledgments 75

Tables Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial 

Crises, 1990–2001 12

Table 2: Forecast Quality for 87 Emerging Market Countries, 1990–

2001 43

Table 3: Forecast Quality for G-7 Countries, 1990–2001 45

Table 4: Comparison of Year-Ahead WEO Forecasts for Program 

and Nonprogram Countries, 1990–2001 46

Table 5: Comparison of Program and WEO Year-Ahead Forecasts, 

for 52 Countries in the Initial Years on Program, 

1990–2001 47

Table 6: IMF Supplemental Reserve Facility Loans, 1997–2002 64

Figures Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve 

Financial Crises 4

Figure 2: IMF Vulnerability Assessment Framework 8

Figure 3: Participation of 33 Major Emerging Markets in Key ROSC 

Assessments, 1999–2003 20

Figure 4: FSAP and ROSC Participation and Publication by Major 

Emerging Market Countries 61

Figure 5: FSAP and ROSC Participation and Publication by Other 

Countries 62

Abbreviations

BCP Basel Core Principles 

BMA Bond Market Association

CAC Collective Action Clauses

CCFF Compensatory Contingency Finance Facility

CPI consumer price index

CPSS Committee on Payments and Settlements Systems

DCSD Developing Country Studies Division

EFF Extended Fund Facility

EMCA Emerging Market Creditors Association
Contents

Page iii GAO-03-734 International Financial Crises

EMTA Emerging Market Traders Association

EWS Early Warning System 

FATF Financial Action Task Force

FSA Financial Sector Assessment

FSAP Financial Sector Assessment Program

FSSA Financial System Stability Assessment

GDDS General Data Dissemination Standard

GDP gross domestic product

IAIS International Association of Insurance Supervisors 

IASB International Accounting Standards Board 

IFAC International Federation of Accountants

IMF International Monetary Fund

IOSCO International Organization of Securities Commissions

IPMA International Primary Market Association

ISMA International Securities Market Association

KLR Kaminsky, Lizondo, and Reinhart

MSE mean square error

OECD Organization for Economic Cooperation and Development

RMSE root mean square error

ROSC Reports on the Observance of Standards and Codes

RSS Residual Sum of Squares

SBA Stand-By Arrangements

SDDS Special Data Dissemination Standard 

SDRM Sovereign Debt Restructuring Mechanism

SEC U.S. Securities and Exchange Commission

SIA Securities Industry Association

SRF Supplemental Reserve Facility

STF Systemic Transformation Facility

UNCITRAL United Nations Commission on International Trade Law

WEO World Economic Outlook

This is a work of the U.S. Government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. It may contain copyrighted graphics, images or other materials. 

Permission from the copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAO’s product.
Page 1 GAO-03-734 International Financial Crises

United States General Accounting Office

Washington, D.C. 20548

A

June 16, 2003 Lret

The Honorable Michael G. Oxley

Chairman

Committee on Financial Services

House of Representatives

The Honorable Jim Saxton

Vice Chairman

Joint Economic Committee

House of Representatives

In May 2000, in the aftermath of the Asian financial crisis, the newly 

appointed Managing Director of the International Monetary Fund (IMF or 

the Fund) announced his plans to strengthen the Fund’s reform initiatives 

to enable the Fund to more effectively safeguard the stability of the 

international financial system. Building on reform initiatives instituted in 

the mid-1990s, after the Mexican financial crisis of 1994–95, these new 

initiatives were designed to better anticipate, prevent, and resolve 

sovereign financial crises. A sovereign financial crisis can occur when a 

country is unable or unwilling to honor its debt obligations and investors 

lose confidence in that country’s financial markets. Recent crises have 

occurred primarily in “emerging market” countries, larger and more 

economically advanced developing countries such as Argentina, Brazil, and 

Turkey. According to World Bank (the Bank) estimates, the financial costs 

to countries that experienced crises in the 1980s and 1990s exceeded $1 

trillion—greater than the total amount of all donors’ assistance to 

developing countries in the 1980s and 1990s.

This letter responds to your request that we examine the Fund’s efforts to 

better safeguard the stability of the international financial system. In this 

report, we assess (1) IMF’s framework for anticipating financial crises, (2) 

the status of key IMF initiatives to prevent financial crises, and (3) new IMF 

proposals to resolve future financial crises. In addition, we analyzed the 

quality of IMF’s forecasts produced by the World Economic Outlook 

(WEO), the Fund’s primary forecasting tool (see appendix III). 

As part of our assessment, we reviewed documents from the U.S. 

government, international organizations, and private firms, including 

testimonies, reports, and relevant laws. We used statistical models that we 

developed to assess the Fund’s WEO forecasts and program projections for 

selected emerging market countries. We interviewed key officials at the 
Page 2 GAO-03-734 International Financial Crises

Fund, World Bank, Department of the Treasury (Treasury), and emerging 

market bond and equity funds. We also interviewed some representatives 

of private sector firms that specialize in international finance, economics, 

and law. Appendix I provides a more detailed description of our objectives, 

scope, and methodology. Appendix II provides a list of the private sector 

firms we interviewed.

Background In 1994–95, Mexico faced a severe financial crisis when a shift in market 

sentiment led to sudden large capital outflows. Investors also temporarily 

removed their funds from other emerging market countries, an effect 

known as contagion. In response to the crisis, Mexico quickly adopted a 

strong and ultimately successful program of adjustment and reform. To 

support the program, the Fund approved a loan of $17.8 billion to Mexico—

one of the largest loan commitments it had ever made to a country. One of 

the major reasons cited for the crisis was the lack of timely, reliable, and 

publicly available economic, financial, and sociodemographic data for 

Mexico. Beginning in 1996, to correct this weakness, the Fund created data 

standards to guide countries in disseminating better data to the public. 

However, as we reported in 1997, the Fund needed to address a number of 

other financial, economic, and political challenges, in addition to data 

limitations, to better anticipate, prevent, and resolve financial crises.1

Before the Fund could fully address these challenges, the Asian financial 

crisis of 1997–98 occurred. After the Asian financial crisis, the Fund 

assessed the effects of its responses to the crisis and reassessed its role in 

safeguarding the stability of the international financial system, including 

rethinking its core mission, operations, and lending activities. The Fund 

also recognized that it needed to improve its ability to anticipate financial 

crises; monitor countries’ activities; and increase public awareness, 

particularly that of the investment community. Recognizing its inability to 

anticipate past crises, the Fund instituted a quarterly vulnerability 

assessment framework in 2001 to identify countries that may be 

susceptible to crisis. 

1

U.S. General Accounting Office, International Financial Crises: Efforts to Anticipate, 

Avoid, and Resolve Sovereign Crises, GAO/GGD/NSIAD-97-168 (Washington, D.C.: July 7, 

1997).
Page 3 GAO-03-734 International Financial Crises

To improve its ability to prevent future crises, the Fund and the World Bank 

in 1999 began performing joint assessments of member countries’ financial 

sectors to help identify and monitor existing and potential weaknesses. In 

addition, the Fund and the World Bank began to work with countries to 

promote adherence to voluntary standards to reassure the international 

community that the countries’ policies and practices conform to standards 

and codes of good practice. These include standards to improve 

transparency in government economic data; fiscal, monetary, and financial 

policies; and guidelines on strengthening the financial and corporate 

sectors. 

The Fund acknowledges that it would be almost impossible to anticipate or 

prevent all crises. According to the Fund, past efforts to resolve financial 

crises during the 1990s were lengthy and very costly to debtor countries. 

The Fund is encouraging the adoption of agreements that would allow a 

quicker, more orderly, and predictable restructuring of countries’ debts. 

The Fund’s ultimate goal is to maintain investor confidence and stability in 

the international financial system. Figure 1 shows the Fund’s key initiatives 

for better anticipating, preventing, and resolving financial crises. Treasury 

through the U.S. Executive Director to the Fund has the lead responsibility 

for monitoring the IMF’s progress in addressing these issues. 
Page 4 GAO-03-734 International Financial Crises

Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve Financial Crises

Results in Brief While it is too early to assess the effectiveness of the Fund’s new 

vulnerability framework for anticipating crises, this framework builds on 

key elements that have been unable to anticipate past financial crises. The 

new framework is more comprehensive than the Fund’s previous efforts, 

bringing together country-specific knowledge and financial expertise 

within the Fund to better identify weaknesses in emerging market 

economies that could lead to a crisis. However, the new framework uses 

the Fund’s major forecasting tools, the WEO and the Early Warning System 

(EWS), which have not performed well in anticipating prior crises. The 

WEO has not successfully anticipated past financial crises, and the Fund’s 

EWS models have had a high false alarm rate, having predicted many crises 

that did not occur. The forecasting of crises has been historically difficult 

for all forecasters due to complex underlying factors, including concerns 

about the reliability of important macroeconomic data on emerging market 

countries.
Page 5 GAO-03-734 International Financial Crises

The Fund, in collaboration with the World Bank, has made progress in 

implementing initiatives to prevent financial crises, but several challenges 

remain. In the late 1990s, the Fund and the World Bank adopted two key 

initiatives designed to assist four parties: Fund staff, World Bank staff, 

member country governments, and private sector participants. First, 

financial sector assessments were introduced to provide reports on aspects 

of member countries’ financial sectors such as banking systems and crisis 

management capacity. Second, the standards initiative was adopted to 

assess member countries’ adherence to 12 standards in areas such as 

banking supervision and economic data dissemination. Use of the 

information provided by these two initiatives by various parties has been 

mixed, and several significant challenges remain. Fund staff frequently 

incorporate information from the assessments into their policy advice to 

member countries. However, assessments have not been completed in 

some important emerging market countries, such as China and Thailand, 

primarily because participation is voluntary. World Bank staff’s use of the 

assessments to inform country development assistance programs is also 

affected by the lack of participation by member countries and by borrower 

countries’ competing development needs. Member country governments 

sometimes use the assessments in prioritizing their reform agendas but 

often find that the reforms are too difficult to implement. Some private 

sector participants find the published reports outdated, untimely, and too 

dense to be useful in making investment decisions. The Fund and the World 

Bank acknowledge that the assessments cannot prevent all crises because 

reforms require many years to be fully implemented, and crises can be 

caused by factors outside the scope of the reforms.

The Fund is considering two approaches to achieve a more orderly and 

predictable restructuring of unsustainable debt between a country and its 

creditors; however, there are significant challenges to implementing both 

approaches. Fund officials assert that the current process for renegotiating 

the terms of member countries’ loans with external private sector creditors 

is lengthy and costly. The Sovereign Debt Restructuring Mechanism 

(SDRM), is a proposed international legal framework that would allow a 

majority of a country’s external creditors to quickly restructure most 

private sector loans. The Fund is also encouraging members to include 

Collective Action Clauses (CACs) in individual bonds, which would allow a 

majority of bondholders to renegotiate the bond’s terms. Although some 

elements of both approaches are acceptable to the private sector and to 

governments, a number of political, legal, and technical challenges stand in 

the way of implementing the SDRM; it seems unlikely that these issues will 

be resolved in the immediate future. While private sector officials expect 
Page 6 GAO-03-734 International Financial Crises

that many restructurings need only involve the private sector and the 

debtor country, under some circumstances, voluntary debt restructurings 

will not adequately resolve all financial crises. In those cases, they said the 

Fund should provide loans to eligible countries to help fill their external 

financing gaps. However, some financial experts and government 

representatives have raised concerns that such loans have the potential to 

increase the probability of future crises. In response, the Fund has clarified 

and strengthened its policy of lending in crisis situations.

The Fund’s crisis prevention initiatives are hindered by several factors that 

limit their effectiveness. To strengthen the effectiveness of these initiatives, 

we are recommending that the Secretary of the Treasury instruct the U.S. 

Executive Director of the Fund to work with other Executive Board 

members to encourage the Fund to improve the readability, timeliness, 

coverage, and frequency of updates of assessment reports. Additionally, the 

Fund should pursue strategies for increasing participation in the 

assessment process including the possibility of making participation 

mandatory for all members of the IMF. 

In responding to our draft report, Treasury, IMF, and the World Bank 

generally agreed with the report’s recommendations. However, the IMF 

stated that we mischaracterized the role of WEO forecasts and EWS 

models in IMF crisis anticipation efforts by saying that they have a greater 

importance than is warranted. We disagree with this depiction. Our 

assessment examined all six components of the IMF’s vulnerability 

assessment framework, including the WEO and EWS. As the only mature 

and quantifiable elements of the framework, our analysis focused more 

heavily on the track records of the WEO and EWS. The World Bank 

expressed concern with the report’s suggestion that consideration be given 

in making participation in the FSAP and ROSC assessments mandatory. 

While we are not suggesting that the assessments should be made 

mandatory, the voluntary nature of these assessments has posed an 

obstacle to full participation by important emerging market countries. 
Page 7 GAO-03-734 International Financial Crises

Too Early to Determine 

If Fund’s New Process 

for Anticipating 

Financial Crises 

Improves on Past 

Efforts

In May 2001, the IMF implemented a new vulnerability assessment 

framework for emerging market countries to strengthen the Fund’s ability 

to anticipate financial crises. This framework brings together countryspecific knowledge and financial expertise within the Fund to better 

identify weaknesses in emerging market economies that could lead to a 

crisis. Although the new vulnerability assessment framework is more 

comprehensive than the Fund’s previous efforts, it is new and still evolving. 

It is too early to tell whether this new framework will successfully 

anticipate future crises. The new framework uses the Fund’s major 

forecasting tools, the WEO and the EWS, which have not performed well in 

anticipating prior crises. The WEO has not successfully anticipated past 

financial crises, and the Fund’s EWS models have had a high false alarm 

rate, having predicted many crises that did not occur. The forecasting of 

crises has been historically difficult for all forecasters due to complex 

underlying factors, including concerns about the reliability of important 

macroeconomic data on emerging market countries.

The IMF Has Recently 

Implemented a New 

Vulnerability Assessment 

Framework 

The Fund has attempted for many years to identify countries vulnerable to 

financial crisis; however, their existing tools were insufficient to anticipate 

the financial crises of the 1990s and led the Fund in 2001 to develop the 

new vulnerability assessment framework. This comprehensive framework 

brings together detailed, country-specific knowledge and financial 

expertise of various IMF departments, including those with regional, 

macroeconomic, or forecasting expertise. The new framework monitors 

the vulnerability of key emerging market countries that borrow 

significantly from international capital markets. This information is 

provided in a quarterly report on crisis vulnerability. Fund staff report 

monthly on countries identified as vulnerable and provide more frequent ad 

hoc analyses during volatile periods.

To conduct the vulnerability assessment, the Fund integrates six 

independent inputs that represent the analyses and perspectives of 

different departments of the Fund (see figure 2).
Page 8 GAO-03-734 International Financial Crises

Figure 2: IMF Vulnerability Assessment Framework

• World Economic Outlook: The WEO is a twice yearly publication that 

provides analyses of global economic developments. Through the WEO, 

the IMF provides current and following year forecasts for countries and 

regions of key economic variables such as economic growth, inflation, 

and current account.2

 According to Fund staff, WEO forecasts use the 

best available information and represent the most realistic estimate of 

key economic variables, including those that could help anticipate a 

financial crisis. The IMF uses these forecasts as an input in the 

vulnerability assessment to gauge the impact of unanticipated adverse 

changes in the global environment. For example, the WEO forecasts for 

selected countries may be recalculated to examine the impact of sudden 

2

The current account represents a country’s trade balance (exports less imports of goods 

and services) plus the net interest income (or payments) on outstanding international 

investments (or debts) plus net transfers (grants, workers’ remittances).
Page 9 GAO-03-734 International Financial Crises

increases in oil prices or an unanticipated recession in the advanced 

economies. 

• Early Warning System models: The Fund uses internal and private 

sector EWS models that compute the probability of a country having a 

crisis over the following 12 to 24 months. The model examines a series 

of vulnerability indicators, including whether a country’s real exchange 

rate is overvalued, or whether the country has significantly depleted its 

foreign exchange reserves.3

 The output of the EWS models helps the 

Fund focus on specific areas of vulnerability. For example, if one 

variable, such as the exchange rate, signals a crisis, the Fund would 

more closely examine related components of the vulnerability 

assessment, such as a country’s external financing requirements.

• Country external financing requirements: On a quarterly basis, the 

Fund produces an internal assessment of a country’s ability to meet its 

total external debt obligations and estimates whether that country has 

sufficient foreign exchange to avoid a crisis. This assessment includes 

estimating a country’s ratio of foreign exchange reserves to short-term 

external debt, estimating the magnitude of its current account deficit, 

and considering whether and how it manages its exchange rate. 

• Market information: On an ongoing basis, the Fund analyzes most 

countries’ cost of borrowing on the international market and whether 

the country is paying a higher interest rate than similar countries. The 

Fund uses this information to provide an internal analysis of the private 

sector’s expectations of a country’s likelihood of default on its foreign 

debt and to identify possible evidence that financial problems are 

spreading across countries.

• Financial sector vulnerability: The Fund assesses the strengths and 

weaknesses of a country’s financial sector, including the banking 

system. IMF staff evaluate the financial sector’s vulnerability to changes 

in market conditions, such as fluctuations in interest and exchange 

rates. Although the detailed results of these assessments are used 

internally by the Fund, summaries of key findings are frequently 

published.

3

Foreign exchange reserves are those external assets that are readily available to and 

controlled by monetary authorities. Reserves can be used for direct financing of payment 

imbalances, for indirectly regulating the exchange rate, and/or for other purposes.
Page 10 GAO-03-734 International Financial Crises

• Country expert perspectives: IMF country experts examine the data 

produced by the above analyses, supplementing those results with 

country-specific details such as the political risks of implementing 

certain government policies or the relevance of certain market 

information. 

Until 1999, the Fund used the WEO as the primary forecasting tool to help 

identify country risks and vulnerability to crises. The new vulnerability 

framework, which has been in operation for about 2 years, is a more 

comprehensive process. According to the Fund, the quarterly integration of 

detailed information from country experts who continuously monitor 

developments in their countries is a great strength of the new vulnerability 

framework. Effective analysis by Fund staff of the framework’s six 

elements could better allow the Fund to give timely advice to authorities in 

vulnerable countries. It is too early to tell whether this framework will be 

successful in anticipating future crises. We assess the performance of the 

WEO and EWS models, the Fund’s primary tools for anticipating crises 

prior to the implementation of the new framework in May 2001.4

Forecasting Elements of the 

Vulnerability Framework 

Have Performed Poorly in 

the Past 

The new vulnerability assessment framework uses the Fund’s two major 

forecasting tools—the WEO and the EWS models—which have not 

performed well in anticipating prior crises. The WEO has not successfully 

anticipated the severe financial crises of the past decade. The Fund’s EWS 

models have had a high false alarm rate, having predicted many crises that 

did not occur.

The WEO Is Not a Reliable Tool 

for Anticipating Crises

Severe financial crises are characterized by a number of negative economic 

outcomes, including large declines in gross domestic product (GDP), also 

known as recessions. We found that the WEO had a poor record of 

forecasting such declines in GDP, tending instead to follow existing 

positive economic growth trends. In addition, the IMF indicates that the 

current account is a key variable in explaining financial crises. We found 

that for the current account, the accuracy of 75 percent of WEO country 

forecasts was worse than simply assuming that next year’s value is the 

same as this year. The WEO’s difficulty forecasting both GDP and current 

4

Data on the performance of the other four components of the framework were not made 

available to us because these elements are considered part of the staff level deliberative 

process and are not provided to the Executive Board.
Page 11 GAO-03-734 International Financial Crises

account demonstrate poor performance in anticipating the severe financial 

crises of the past decade. 

WEO Has Performed Poorly at Forecasting Recessions

In most cases, countries experiencing a financial crisis also experience a 

severe recession in which their GDP declines significantly. Although most 

recessions do not involve a major financial crisis, successful anticipation of 

recessions, especially the most severe ones, would greatly increase the 

likelihood of anticipating impending financial crisis. However, we found 

that the WEO has a poor track record of forecasting recessions, including 

those directly associated with a financial crisis. 

The WEO did not forecast most of the recessions that occurred in emerging 

market countries in the last 10 years. During the 1991–2001 forecast period, 

134 recessions occurred in all 87 emerging market countries. We found that 

the WEO correctly forecast only 15, or 11 percent, of those recessions, 

while predicting an increase in GDP in the other 119 actual recessions.5

 The 

WEO is considerably more likely to forecast a recession when a recession 

has occurred in the prior year. However, a prior year recession did not 

occur in two-thirds of the recessions that the WEO failed to forecast. Thus, 

WEO forecasts generally follow the existing growth trend within a country, 

making it unlikely that the WEO would correctly forecast an unanticipated 

recession. Furthermore, this tendency to follow the current growth trend 

makes it especially difficult for the WEO to anticipate a financial crisis 

because nearly all of the crisis-related recessions of the last decade 

occurred in years following positive economic growth.

Further illustrating our point, the WEO was unable to anticipate large 

declines in GDP, also known as recessions, that corresponded to 14 major 

financial crises of the past decade, including the Mexican and Asian 

financial crises (see table 1).6

5

In one case, the WEO forecast zero growth.

6

Several studies examining international financial crises from 1990–2001 identified 14 major 

financial crises in 12 prominent emerging market countries that were considered to be 

fundamentally sound before the crisis. Balance of payments problems, large GDP 

contractions, and financial panic commonly characterize these crises. 
Page 12 GAO-03-734 International Financial Crises

Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial Crises, 1990–2001

Sources: Various crisis studies and IMF.

a

Actual GDP growth rate is from the WEO published 1 year later.

b

The Asian crisis began in Thailand in the summer of 1997 and spread to the other Asian countries in 

the latter part of 1997. The full effects of the crisis on the GDP of other Asian countries and Russia 

occurred in 1998. Thus we are using 1998 actual GDP growth rate data for the Asian countries—

Indonesia, Malaysia, Philippines, South Korea, and Thailand. The crisis in Brazil extended into1999.

The WEO’s failure to identify these recessions demonstrates that it did not 

anticipate the corresponding financial crises. In 14 cases, we found that the 

WEO forecast strong economic growth, averaging a 4.4 percent increase in 

GDP, despite large declines in actual GDP in 13 of 14 cases.7 In fact, actual 

GDP declined by an average of 5.7 percent during the first full year of these 

14 financial crises. Indonesia presents the most startling disparity, in which 

the WEO forecast a growth of 6.2 percent in its GDP, when in fact 

Indonesia’s GDP declined by almost 14 percent in the first full year of its 

financial crisis. 

Country Crisis years

WEO forecast

GDP growth rate

(percent)

Actual GDP

growth ratea

(percent)

Argentina 1995 4.0 -4.4

Argentina 2001 3.7 -4.4

Brazil 1998 4.0 0.1

Ecuador 1999 2.5 -7.3

Indonesiab 1998 6.2 -13.7

Malaysiab 1998 6.5 -6.7

Mexico 1995 4.0 -6.9

Philippinesb 1998 5.0 -0.5

Russia 1998 5.1 -4.6

South Koreab 1998 6.0 -5.8

Thailandb 1998 3.5 -9.4

Turkey 1994 3.0 -5.5

Turkey 2001 5.5 -7.4

Venezuela 1994 2.5 -3.3

Average growth rates 4.4 -5.7

7

Brazil was the sole exception, having experienced a small 0.1 percent GDP annual growth 

rate during the 1998 financial crisis.
Page 13 GAO-03-734 International Financial Crises

WEO Does a Poor Job in Forecasting the Current Account

According to the Fund, a country’s current account (primarily exports 

minus imports of goods and services) is a key variable in anticipating 

crises. Crises are associated with problems of external financing that result 

from a country having difficulty obtaining foreign exchange. Since exports 

are an important source of foreign exchange for developing countries, 

projections of a country’s current account balance provide information 

about the country’s ability to earn foreign exchange and to service its 

external debt.8

 According to the Fund, an unsustainably large current 

account deficit can contribute to or precipitate a crisis.9

 

We found that WEO forecasts for current account were inaccurate most of 

the time. Our analysis for the 87 emerging market countries shows that, for 

more than 75 percent of the countries, the WEO current account forecasts 

were less accurate than if the Fund had simply assumed that the next year’s 

current account would be the same as this year’s. The results are even more 

dramatic for G-7 countries; a forecast of no change was a better predictor 

than the WEO forecast for six of the seven countries. This demonstrates 

that, even in stable economies with excellent data, the WEO has done a 

poor job of forecasting this key crisis anticipation variable. (See appendix 

III for more detailed explanation on our methodology and findings).

8

A deficit in the current account can also be financed by capital inflows, such as foreign 

private investment, a drawing down of a country’s international reserves, by bilateral or 

multilateral loans, or by provision of exceptional finance, such as debt service relief.

9

Fund studies indicate that the current account deficit, as a percentage of GDP, is 

significantly larger before a crisis than during relatively stable periods, which suggests that 

unsustainable current account deficits tended to be part of the general overheating of the 

economy before a crisis.
Page 14 GAO-03-734 International Financial Crises

The Fund’s Early Warning 

System Models Indicate Many 

Crises That Do Not Occur

Since 1999, the Fund has analyzed the results from internal and private 

sector EWS models in its crisis anticipation efforts. The Fund’s internal 

efforts focused on two EWS models to systematically identify countries 

vulnerable to crises: the Kaminsky, Lizondo, and Reinhart (KLR) model, 

which monitors a set of 15 monthly variables that signal a crisis whenever 

any cross a certain threshold; and the Developing Country Studies Division 

(DCSD) model, which uses five variables to compute the probability of a 

crisis occurring in the next 24 months.10 The Fund’s models use a variableby-variable approach that allows economists to determine which variables 

are signaling the crisis. Internal assessments of the Fund’s EWS models 

show that they are weak predictors of actual crises. While the models 

worked reasonably well in anticipating Turkey’s recent financial crisis, they 

did not successfully anticipate Argentina’s financial crisis in 2002. 

According to the Fund, the models’ most significant limitation is that they 

have high false alarm rates; that is, they predict many crises that did not 

occur. In about 80 percent of the cases where a crisis was predicted over 

the next 24 months, no crisis occurred. Furthermore, in about 9 percent of 

the cases where no crisis was predicted, there was a crisis.11

Forecasting Crises Has 

Been Historically Difficult 

for All Parties

Financial crises have been historically difficult to anticipate because of a 

number of complex underlying factors. Economic outcomes are often 

influenced by unanticipated events such as conflicts and natural disasters. 

Many factors, in addition to weaknesses in a country’s financial structure, 

can lead to a crisis. These include economic disturbances, such as an 

unanticipated drop in export prices, political events, and changes in 

investor sentiment leading to sudden withdrawals of foreign capital. 

Furthermore, data may be inadequate, particularly in developing countries 

where data are often not timely and are of poor quality. 

10More specifically, the DCSD model is a multivariate probit regression. The five variables 

used are real exchange rate overvaluation, current account, foreign exchange reserve 

losses, export growth, and the ratio of short-term debt to foreign exchange reserves. Also, 

the model defines a currency crisis as the weighted average of 1-month changes in exchange 

rate and reserves more than 3 (country-specific) standard deviations above country average.

11The DCSD model had a cutoff probability of 23 percent and the KLR model had a cutoff 

probability of 15 percent. A forecast probability above these cutoff points is deemed to 

signal a crisis.
Page 15 GAO-03-734 International Financial Crises

Forecasters consistently fail to foresee crises and recessions. Forecasts 

produced by private sector economic forecasters, governments, and 

multinational agencies including the IMF and the Organization for 

Economic Cooperation and Development, routinely fail to foresee the 

coming of crises and recessions, and often fail to outperform the naive 

model, which simply assumes that next year’s outcome will be the same as 

this year’s. This is true even for evaluations of recent U.S. forecasts of GDP 

and inflation. Our review of a number of forecast evaluation studies 

confirms that the inability to predict recessions is a common feature of 

growth forecasts for both industrialized and developing countries. The 

studies also showed that forecast accuracy improves as the time horizon 

gets shorter, and that there is little difference in forecast accuracy between 

private sector and WEO forecasts.

Some Progress in 

Implementing Crisis 

Prevention Initiatives, 

but Challenges Remain

In the late 1990s, the Fund and the World Bank began implementing two 

crisis prevention initiatives designed to assist four parties: IMF staff, World 

Bank staff, member country governments, and private sector participants. 

The first initiative, the financial sector assessments, provides reports on 

aspects of member countries’ financial sectors such as banking systems 

and crisis management capacity. The second, the standards initiative, 

assesses member countries’ adherence to 12 standards in areas such as 

banking supervision and economic data dissemination. Parties’ use of the 

information provided by these two initiatives has been mixed, and several 

significant challenges remain. Fund staff frequently incorporate 

information from the assessments into their policy advice when 

assessments are available; however, assessments have not been completed 

in some important emerging market countries primarily because 

participation is voluntary. Bank staff’s use of the assessments to inform 

country development assistance programs is also affected by gaps in the 

completion of some assessments and by borrower countries’ competing 

development demands. Member country governments sometimes use the 

assessments in prioritizing their reform agendas but often find the reforms 

too difficult to implement. Some private sector participants find the 

published reports untimely, outdated, and too dense to be useful in making 

investment decisions. The IMF and the Bank acknowledge that the 

initiatives cannot prevent all crises because recommended reforms require 

many years to be fully implemented, and crises can be caused by factors 

outside the scope of the reforms.
Page 16 GAO-03-734 International Financial Crises

Two Key Crisis Prevention 

Initiatives Target Four 

Parties 

In the wake of the Mexican and Asian financial crises of the 1990s, the 

Fund and the World Bank became increasingly aware of the importance of 

transparent financial data and policies, stronger financial systems, and 

better-functioning markets as a complement to member country 

governments’ sound macroeconomic policies. Fund evaluations 

acknowledge that the institution failed to collect information that could 

have enabled it to detect financial and corporate sector vulnerabilities and 

to provide appropriate policy advice to the affected countries before the 

crises occurred. 

In response to this, in the late 1990s, the Fund and the Bank jointly 

launched two initiatives to prevent the long-term likelihood of financial 

crises. The first initiative, the Financial Sector Assessment Program 

(FSAP), consists of in-depth assessments of key elements of member 

countries’ financial sectors. These elements include the structure of 

financial markets, financial systems’ response to changes in key variables 

such as exchange rates, legal arrangements for crisis management, and the 

quality of financial sector supervision. The second initiative, the Reports on 

the Observance of Standards and Codes (ROSC), consists of assessments 

of member countries’ adherence to 12 standards12 related to transparency 

in government policy making and operations,13 financial sector regulation, 

and corporate sector practices (see appendix V).14 Building on earlier 

efforts to assess transparency, in 1999, the IMF and the Bank began 

conducting joint assessments of observance of standards related to 

12According to the Fund, 11 of these 12 standards are internationally accepted. The World 

Bank’s corporate insolvency and creditor rights standard is still in draft form.

13In addition to the ROSC, the Fund launched an experimental program in 2000 to help 

prevent possible misuse of Fund resources. This program assesses weaknesses in central 

banks’ ability to safeguard Fund resources through internal controls, accounting, reporting, 

auditing systems, and legal structure. In 2002, the Executive Board made assessments of 

central banks’ safeguards a permanent policy. See appendix VI for an update on progress in 

this area. 

14In February and March 2003, the IMF and Bank published reviews of the FSAP and ROSC 

initiatives. See International Monetary Fund and World Bank, Financial Sector Assessment 

Program—Review, Lessons, and Issues Going Forward (Washington, D.C.: 2003); 

International Monetary Fund and World Bank, International Standards: Strengthening 

Surveillance, Domestic Institutions, and International Markets (Washington, D.C.: 2003); 

International Monetary Fund, International Standards: Background Paper on 

Strengthening Surveillance, Domestic Institutions, and International Markets

(Washington, D.C.: 2003); World Bank, Reports on the Observance of Standards and Codes 

Background Paper on Standards Review: Assessing Progress and Lessons Learned from 

Bank-Led ROSC Modules (Washington, D.C.: 2003). 
Page 17 GAO-03-734 International Financial Crises

financial sector regulation, covering areas such as banking supervision and 

securities regulation. The Bank began performing assessments of 

standards related to corporate sector practices, including private sector 

accounting rules and corporate governance principles, in 2000. Some 

transparency, financial sector, and corporate sector standards may be 

assessed under the FSAP. Country participation in both initiatives is 

voluntary. The Fund and the Bank initially considered making participation 

in the ROSC assessments mandatory for member countries after 

determining that the Fund’s Articles of Agreement could allow such a 

requirement. 

IMF or World Bank staff lead FSAP and ROSC assessment teams, with 

participation by experts from national central banks and supervisory 

agencies and standard-setting bodies such as the Basel Committee and the 

International Organization of Securities Commissions (IOSCO). Before 

undertaking an FSAP or ROSC, the Fund and the Bank work with country 

governments to choose areas on which to focus. During the assessment 

process, FSAP and ROSC teams conduct at least one in-country visit, 

allowing team members to work with government officials from the finance 

ministry, the central bank, and regulatory bodies to collect information for 

the assessment.15 For example, an FSAP team in Russia analyzed financial 

information for the largest banks and single largest corporation to 

determine how changes in economic variables such as oil prices might 

affect the banking system. The South Korean ROSC team interviewed 

government officials at financial sector regulatory entities and private 

sector representatives to determine how closely regulatory practices 

conform to standards and to identify weaknesses that could put the 

financial sector at risk. 

The two initiatives provide information to assist four parties that play a role 

in crisis prevention: the Fund, the World Bank, member country 

governments, and private sector investors. In-depth information on 

member countries’ financial sectors and adherence to standards of good 

practice is intended to help the IMF and the Bank fulfill their missions. 

Fund staff identify countries’ vulnerabilities and develop appropriate 

advice to redress them; Bank staff identify long-term financial sector 

development needs and formulate relevant lending and nonlending 

responses. Member countries can use these assessments to help prioritize 

reform agendas and win domestic support for difficult policy decisions that 

15Appendix VII contains a diagram of the FSAP and ROSC assessment process.
Page 18 GAO-03-734 International Financial Crises

may make their financial sectors and institutions more resistant to crisis. 

The Fund and the Bank often provide technical assistance to help 

governments build capacity to implement reforms. The financial crises of 

the 1990s also raised awareness of the private sector’s role in crisis 

prevention. Thus, the Fund and the Bank expect the assessments to help 

private sector participants make sounder investment decisions, thereby 

reducing volatility in capital markets.

Parties’ Use of Assessments 

Is Mixed, and Significant 

Challenges Remain

The use of FSAP and ROSC assessments in crisis prevention efforts is 

mixed, and significant limitations remain. Fund staff use the assessments, 

when available, as inputs for the policy advice they provide to member 

country governments. However, the Fund lacks crucial information about 

vulnerabilities to financial crisis because some major emerging market 

countries have not participated in the assessments. World Bank staff’s use 

of the assessments to inform development assistance priorities is also 

affected by these gaps in participation and by borrower countries’ 

competing development needs. Many member country governments face 

limitations in using assessments to make policy decisions because the 

reforms recommended in the assessments are often difficult to implement. 

Finally, some private sector participants find assessments of limited use 

because they are untimely, outdated, and dense. 

The Fund Uses Assessments, 

When Available, to Inform Policy 

Advice

The IMF uses FSAP and ROSC assessments, when available, as inputs for 

the policy advice it provides member country governments. According to 

Fund officials, these assessments highlight issues such as weak banking 

supervision or high levels of debt held in foreign currency that could make 

countries vulnerable to crisis. The assessments also provide 

recommendations to address these issues. The findings and 

recommendations inform the discussions of policy issues that Fund 

officials have with member country authorities during Article IV 

consultations.16 For example, when an FSAP was performed in Mexico, 

Mexican authorities had begun replacing a system where the government 

fully insured all bank deposits with one that covers deposits up to a certain 

16In accordance with Article IV of the IMF’s Articles of Agreement, the IMF holds regular 

consultations, normally every year, with each of its members. These consultations focus on 

the member’s exchange rate, fiscal, and monetary policies; its balance of payments and 

external debt developments; the influence of its policies on the country’s external accounts; 

the international and regional implications of those policies; and on the identification of 

potential vulnerabilities. (“IMF Surveillance: A Factsheet,” http://www.imf.org, downloaded 

4/23/03).
Page 19 GAO-03-734 International Financial Crises

limit. The FSAP team was concerned because this reform was undertaken 

before Mexico had developed a well-defined framework for closing 

unprofitable banks. Without a clear framework for bank closures, the 

introduction of limited deposit insurance could damage depositor 

confidence in the banking system and precipitate a banking crisis. 

According to Fund officials, the FSAP team and Mexican authorities 

discussed the need to create such a framework, and a subsequent Article IV 

mission reviewed the government’s progress in this area. In Poland, an 

FSAP team discovered that Polish households and small businesses had 

high levels of debt held in foreign currency. The team was concerned that a 

depreciation of Polish currency could raise the cost of these loans and 

cause widespread repayment difficulties, which could in turn lead to a 

banking crisis. FSAP team members raised this issue with Polish central 

bank authorities and followed up again during the next Article IV 

consultation. In both countries, government officials followed the Fund’s 

advice and implemented reforms. The Mexican government began 

developing a framework for closing banks, and Poland’s central bank 

established a team to monitor household and small business debt. 

Since 1999, FSAP assessments have been conducted in more than 40 

member countries and ROSC assessments in about 90 member countries. 

However, we found that assessments have not been completed for some 

major emerging market countries, limiting the Fund’s awareness of crisis 

vulnerabilities in certain countries. Appendix VIII contains a record of 

country participation in and publication of FSAP and ROSC assessments. 

Fund and Bank staff encourage participation in FSAP and ROSC 

assessments by countries whose economies have worldwide or regional 

implications or have known vulnerabilities to a financial crisis, but officials 

acknowledge that some governments have persistently resisted their 

efforts. According to our analysis, between 1999 and 2003,17 45 percent of 

33 major emerging market countries participated in an FSAP.18 However, 

the Fund has not performed FSAPs in important countries such as China 

and Thailand because their authorities have not agreed to participate. 

These gaps in participation limit the Fund’s ability to develop policy advice 

based on in-depth knowledge of their financial sectors. 

17Our analysis is based on Fund and Bank data through March 2003.

18These 33 countries, a subset of the 87 we analyzed in the previous section, represent the 

most significant emerging market participants in the international capital markets, as 

identified by J.P. Morgan.
Page 20 GAO-03-734 International Financial Crises

According to the Fund, the Mexican and Asian financial crises were caused, 

in part, by vulnerabilities in areas covered by the ROSCs. Our analysis 

found gaps in participation in assessments of several key standards that the 

Fund identifies as contributing factors to past crises (see figure 3). For 

example, only one-third of major emerging market countries have 

participated in assessments of their adherence to standards for 

dissemination of economic and financial data. About half have participated 

in the fiscal, monetary and financial policy formulation assessments and 

the banking supervision assessment. In addition, Fund documents point to 

limited progress in assessing adherence to the four World Bank-led 

corporate sector standards19 (accounting, auditing, corporate governance, 

and insolvency regimes), which play a key role in the effective operation of 

domestic and international financial systems. Less than one-third of the 33 

major emerging market countries have participated in one or more 

assessments related to accounting and auditing. 

Figure 3: Participation of 33 Major Emerging Markets in Key ROSC Assessments, 

1999–2003 

19The Bank performs accounting and auditing assessments simultaneously and publishes 

them as a single report. An agreed-upon standard for insolvency regimes has not yet been 

finalized. The Bank’s draft standard for insolvency regimes is under review by the Bank and 

the Fund.
Page 21 GAO-03-734 International Financial Crises

The Fund asserts that its delayed response in preventing or mitigating the 

Mexican and Asian crises was partially caused by insufficient information 

on these vulnerabilities. For example, according to the IMF, the Mexican 

government’s publicly available data was outdated and incomplete in 1993–

94, which contributed to significant delays in responding to the country’s 

excessive indebtedness. The Fund also was unaware of some Asian 

countries’ unsound corporate accounting practices, which contributed to 

the Asian financial crisis. Continued participation gaps in these 

assessments suggest that the Fund still lacks crucial information about 

some countries’ potential vulnerability to crisis. 

The World Bank’s Use of 

Assessments in Formulating 

Country Development Assistance 

Programs Is Mixed 

The World Bank acknowledges the importance of FSAP and ROSC 

assessments in formulating its financial sector development programs, but 

limited participation in corporate sector assessments (described earlier) 

affects the Bank’s ability to respond to weaknesses in borrower countries’ 

financial sectors. According to the Bank, country participation in corporate 

sector assessments has been lower than in areas related to transparency 

and financial sector regulation because the Bank has experienced delays in 

finalizing standards and methodologies for evaluating the corporate sector. 

For example, the methodologies for performing assessments of the 

accounting and auditing standards were not finalized until October 2000. 

Bank officials acknowledge that even when assessments are available, 

Bank staff do not always incorporate the issues raised as a key priority in 

formulating its country development assistance programs. In justifying 

their limited prioritization of FSAPs and ROSCs, Bank officials cited 

competing development demands and timing issues. First, Bank officials 

stated that they must balance borrower countries’ financial sector reform 

needs with other demands for development assistance. Most borrower 

country governments have multiple concerns, and Bank staff may 

determine that its resources will have more impact in areas other than 

financial sector development. Second, Bank officials cited the scheduling 

of the FSAP and ROSC assessments as a reason for their limited use since 

the timing of many assessments does not coincide with the Bank’s 

preparation of Country Assistance Strategies.20 

20The Country Assistance Strategy describes the Bank’s assessment of development 

priorities in each borrower country and identifies the level and composition of assistance to 

be provided based on this assessment. These strategies are currently prepared every 3 years.
Page 22 GAO-03-734 International Financial Crises

Member Country Governments’ 

Use of Assessments Is Limited

Although member country authorities sometimes use FSAP and ROSC 

assessments to inform policy decisions, reforms recommended in the 

assessments are often difficult to implement. Some member country 

governments have faced obstacles to implementing reforms, including 

political opposition, legal constraints, and lack of technical capacity. For 

example, IMF officials stated that political opposition has limited the South 

Korean government’s progress in eliminating extrabudgetary funds, a key 

recommendation of the fiscal transparency ROSC. Extrabudgetary funds 

diminish transparency because they are exempt from rules that require 

scrutiny and prioritization of expenditures for most of South Korea’s 

budget. Fund officials cited Peru as a case where legal constraints delayed 

reform efforts. The FSAP and banking supervision ROSC found that 

protecting bank supervisors from the political pressures of the powerful 

bankers’ lobby would strengthen Peru’s banking supervision. According to 

Fund officials, existing legislation precluded awarding supervisors greater 

independence, and passage of new legislation was delayed. In Russia, 

limited technical capacity interfered with the government’s ability to 

implement reform recommendations. For example, the FSAP team 

reviewed the government’s proposal to stimulate competition in the 

banking sector by introducing a deposit insurance system for household 

deposits. However, Fund staff noted that Russia’s bank supervisory agency 

lacks the capacity to implement a deposit insurance system for a large 

number of banks. 

Private Sector Participants Find 

Assessments Difficult to Use

The Fund claims that private sector participants increasingly use the 

results of FSAPs and ROSCs to inform investment decisions and risk 

management. However, representatives of major international investment 

firms and ratings agencies we interviewed stated that the reports were 

untimely, outdated, and too dense to be useful. For example, several 

respondents indicated that delays in publishing ROSC assessments reduced 

their usefulness. Some private sector participants stated that ROSC reports 

and FSAP summaries, known as Financial System Stability Assessments 

(FSSAs), should be published within 6 months of performing the 

assessments. However, our analysis of the 58 ROSC reports published for 

major emerging market countries found that in one-third of the cases, 9 

months or more elapsed between assessment and publication. Several 

private sector participants we interviewed stated that outdated ROSC 

reports are unreliable for decision making. The Fund acknowledges that 

assessments must be current for private sector participants to use them. 

According to Fund data, 13 countries have published an update of at least 

one ROSC module. However, IMF officials estimate that, of the more than 

40 FSAPs performed to date, only 4 have been fully updated. Some private 
Page 23 GAO-03-734 International Financial Crises

sector participants also stated that FSSA and ROSC reports are not clearly 

written. Representatives of one multinational investment bank stated that 

the assessments are written in a way that is difficult to understand, which 

limits the reports’ usefulness for investment decisions. While these 

interviews were limited in number and may not be representative of all 

private sector participants, they do provide an indication of the problems 

these individuals may currently have in using FSAPs and ROSCs.

Fund and Bank outreach sessions and a 2002 Fund survey corroborated our 

findings on private sector participants’ difficulties in using ROSC 

assessments. The Fund reports that private sector participants place high 

priority on timely publication and frequent updates of ROSC assessments. 

For example, several participants observed that ROSC reports for 

Argentina had not been updated since their publication in 1999. Moreover, 

respondents to the Fund’s survey commented that ROSC assessments 

should state more clearly the deficiencies in a country’s adherence to a 

standard. In a March 2003 review of the standards initiative, the Fund and 

the Bank concluded that ROSC reports would be more useful if they stated 

the main findings and their significance clearly and prioritized 

recommendations more explicitly. 

Crisis Prevention Initiatives 

Cannot Prevent all Crises 

The Fund and the World Bank acknowledge that FSAP and ROSC 

assessments cannot prevent all crises because recommended reforms may 

require many years to be fully implemented and because crises can be 

caused by factors outside the reforms’ scope. For example, Argentina 

participated in four ROSC assessments in 1999 to improve economic data 

dissemination; banking supervision; and transparency in the formulation of 

fiscal, monetary, and financial policies. According to senior IMF officials, 

the Argentine government followed many of the recommendations 

generated by these assessments, but their actions did not address 

vulnerabilities related to weak fiscal policy and a fixed exchange rate 

regime that contributed to Argentina’s 2001 crisis. Fund officials cite 

Turkey as another example of a country that made considerable progress in 

improving transparency and data provision based on reforms 

recommended by the fiscal transparency and economic data dissemination 

ROSC assessments of 2000–2001. However, according to the Fund, these 

reforms could not have prevented Turkey’s 2001 crisis, which originated 

with declines in its exchange rate.
Page 24 GAO-03-734 International Financial Crises

The Fund’s Efforts to 

Better Resolve Future 

Financial Crises Face 

Significant Challenges

Fund officials assert that the current process for renegotiating the terms of 

member countries’ loans with external private sector creditors is lengthy 

and costly. In 2001, the Fund began considering the SDRM, an international 

legal framework that would allow a majority of a country’s external 

creditors to approve a restructuring of most private sector loans. The Fund 

is also encouraging members to include CACs in bonds, which would allow 

a majority of bondholders to renegotiate the terms of that bond. Although 

some elements of both approaches are acceptable to the private sector and 

governments, a number of political, legal, and technical challenges stand in 

the way of implementing the SDRM; it seems unlikely that these issues will 

be resolved in the immediate future. While private sector officials expect 

that many restructurings need only involve the private sector and the 

debtor country, under some circumstances, voluntary debt restructurings 

will not adequately resolve all financial crises. These officials stated that, in 

those cases, the Fund should provide short-term loans to eligible countries 

to help fill their external financing gaps. However, concerns have been 

raised by some financial experts and government representatives that such 

Fund loans have the potential to increase the probability of future crises. 

In response to these concerns, the Fund has clarified and strengthened its 

policy of lending into crisis situations.

Efforts to Resolve Future 

Sovereign Debt Crises Face 

Significant Challenges 

According to the Fund, countries facing severe liquidity problems often go 

to extraordinary lengths to avoid renegotiating or restructuring the terms 

of their loans. They do so because, in the past, restructuring damaged the 

economy and the banking system of participating countries. In some cases, 

even when a voluntary restructuring process is initiated, individual 

creditors may hold out for the best possible terms or sue in an attempt for 

better terms. Additionally, countries believe that creditors also may be 

unwilling to make future loans if they default on their existing debt. 

The SDRM approach is an attempt to create a more orderly, predictable, 

and comprehensive restructuring process and to lower the costs of 

restructuring for both the debtor and creditors. The approach sought to 

reduce the duration of the restructuring process from years to months and 

to provide incentives to restructure debt before default to better protect 

debtor and creditor interests. In the case of debtors, the Fund maintains 

that an orderly restructuring process could reduce the likelihood of a 

reduction in future capital flows. For creditors, it could provide more 

favorable repayment terms from the restructured debt. 
Page 25 GAO-03-734 International Financial Crises

The SDRM is a proposed international legal framework that would allow a 

member country to declare its debt unsustainable and invoke a process to 

restructure most of its external private sector loans.21 A specified majority 

of the country’s external creditors would vote to approve the terms for 

restructuring, which would bind all eligible private sector creditors. The 

framework is designed to increase the incentives for the Fund’s member 

countries and their creditors to reach a rapid and collaborative agreement 

on restructuring unsustainable debt. 

A number of political, legal, and technical challenges stand in the way of 

implementing the SDRM, and it seems unlikely that these issues will be 

resolved in the immediate future. According to the Fund, successful 

implementation of SDRM will require overcoming certain political 

constraints. The SDRM could be put into practice either by countries 

adopting a new international treaty or by amending the Fund’s Articles of 

Agreement. Both options would be difficult to implement since a number of 

countries have indicated opposition to the SDRM. The draft framework 

recommended that the SDRM be created through an amendment to the 

Fund’s Articles of Agreement because the SDRM is closely related to the 

role already assigned to the Fund under the Articles in the resolution of its 

members’ external financial obligations. However, the Fund acknowledged 

that, given the opposition of some countries, changing the Articles could be 

difficult to achieve since it requires acceptance by three-fifths of the 

members, having 85 percent of the total voting representation. The United 

States, for example, could unilaterally veto any proposed amendment to 

the Fund’s Articles given its 17 percent voting representation.

21The SDRM proposal has undergone several revisions. Our report discusses the proposal 

presented to the International Monetary and Financial Committee in April 2003. See

http://www.imf.org/external/np/omd/2003/040803.htm.
Page 26 GAO-03-734 International Financial Crises

A key legal challenge to the implementation of SDRM is the need for most 

countries to change their domestic laws to conform to the requirements of 

any new Fund articles. Before a member country can vote to accept an 

amendment, it must take all the necessary steps needed under its own 

domestic law to ensure that the amendment will be given full force and 

effect under its domestic law. However, some Fund members have raised 

concerns over whether the domestic legal systems of some member 

countries could accommodate a new legal framework that applied to 

preexisting claims. The proposed SDRM approach also faces technical 

challenges. For example, the proposed framework does not specify how 

the claims of official bilateral creditors and some guaranteed domestic 

debts would be treated. The Fund is consulting with the Paris Club on how 

the Club’s practices may be modified to better facilitate coordination 

between official bilateral and the private creditors in a debt restructuring 

process.22

The Fund Has Encouraged 

the Adoption of Collective 

Action Clauses

CACs are terms in individual bonds that permit a specified majority of 

sovereign bondholders to agree on a debt restructuring that would bind all 

holders of that bond. In June 2002, the Fund’s Executive Board endorsed 

the use of certain CAC provisions in new bonds and agreed to encourage 

member countries to incorporate CACs into their sovereign bonds in future 

restructurings. Inclusion of these clauses into new bonds would be 

voluntary.

22The Paris Club is an informal group of official creditors whose role is to find solutions to 

the payment difficulties experienced by debtor countries. Paris Club creditors agree to 

provide a country with debt relief through a postponement and, in the case of concessional 

rescheduling, a reduction in debt service obligations.
Page 27 GAO-03-734 International Financial Crises

The Fund views CACs and SDRM as complementary instruments in 

resolving future financial crises. According to the Fund, the existence of 

CACs in certain bond agreements inspired the development of the SDRM 

framework. Although the Fund has not created its own CAC framework, it 

has endorsed the use of two key features from a G-10 Working Group 

Report23 and an Industry Associations draft proposal.24 These features 

include the following:

• Majority restructuring provision25 enables a qualified majority of 

bondholders to bind all holders of a particular bond to the financial 

terms of a restructuring, both before and after a default. Although 

majority restructuring provisions have generally been included in bonds 

governed by the laws of the United Kingdom or Japan, they have not 

been included in bonds governed by the laws of the United States or 

Germany. 

• Majority enforcement provisions prevent a minority of creditors from 

pursuing disruptive legal action after a default and before reaching a 

restructuring agreement. Many international sovereign bonds governed 

by both U.S. and English law contain these provisions. Specifically, the 

Fund supports the requirement that (1) an affirmative vote of a 

minimum percentage of bondholders is necessary to approve claims 

following a default and (2) a specified majority of bondholders can 

reverse an approval of a claim that has already occurred. 

In early 2003, Mexico became the first emerging market country to issue a 

public, SEC-registered global bond with CACs under New York law. 

Previous issues under New York law by Lebanon, Qatar, and Egypt had 

been placed privately to institutional investors and included only a limited 

23The G-10 Working Group on Contractual Clauses (the Working Group) was formed in June 

2002 at the behest of ministers and governors. The mandate of the Working Group was to 

consider how sovereign debt contracts could be modified to resolve debt crises in a more 

orderly fashion.

24The proposal was put forward to Fund members for consideration by the Institute of 

International Finance and six other financial industry trade associations. The six financial 

industry associations are the Emerging Market Traders Association (EMTA), the 

International Primary Market Association (IPMA), the Bond Market Association (BMA), the 

Securities Industry Association (SIA), the International Securities Market Association 

(ISMA), and the Emerging Markets Creditors Association (EMCA).

25These provisions are referred to in the G-10 Working Group Report as “majority 

amendment provisions.”
Page 28 GAO-03-734 International Financial Crises

range of CACs. For example, the bonds issued by Egypt and Qatar 

included a very limited form of majority enforcement provisions, while 

Lebanese bonds did not contain them at all. Since Mexico’s successful 

issue, Brazil, South Africa, and Korea have issued bonds with CACs. 

Uruguay included CACs in the bonds resulting from its debt exchange. The 

details of the Brazil, South Africa, Korea, and Uruguay bond provisions 

were not available at the time we conducted our review.

Some countries criticize CACs because they would only apply to new bond 

offerings and not existing bonds. Accordingly, during a restructuring of a 

country’s bond obligations, not all creditors would be bound by the CAC 

provisions. Borrowing countries also contend that inclusion of CACs in 

bond offerings could suggest to creditors that countries anticipate having 

difficulty repaying their loans. In response, creditors may charge a higher 

interest rate. However, a May 2000 academic study26 compared interest 

rates on bonds issued in the United States (where CACs are not used) with 

the United Kingdom (where CACs are used) and found that CACs do not 

contribute to higher rates in United Kingdom bonds. 

While Private Sector 

Officials Prefer Voluntary 

Debt Restructuring, They 

Expect Continued Fund 

Loans in Exceptional 

Circumstances

To date, officials from the private sector, including lenders, have expressed 

preference for continuing the current voluntary process, which only 

involves the private sector and borrowing countries, in the efforts to 

restructure sovereign debts. Many private sector officials we interviewed 

oppose the proposed SDRM approach and the Fund’s attempts to integrate 

CACs into new bond issues, partly because they would interfere with the 

normal bargaining process. They maintain that a voluntary approach to the 

restructurings that took place from 1998 to 2001 in Ecuador, Pakistan, 

Russia, and Ukraine were successful.27 Private sector officials assert that 

these and other experiences have worked well enough, and that a 

substantive change in current market practice is unnecessary. In contrast 

to the Fund’s assertion that new approaches are needed to make 

restructurings shorter and less expensive, private sector officials note that 

26See Eichengreen, Barry and Ashoka Mody. “Would Collective Action Clauses Raise 

Borrowing Costs?” Working paper, Center for International and Development Economics 

Research, Berkeley, CA, 2000.

27The restructurings in Russia involved domestic debt and Soviet-era foreign debt owed to 

commercial banks. In 1998, Pakistan froze withdrawals in foreign currency from all 

nonresident foreign currency deposits and subsequently reached restructuring agreements 

with these nonresident investors.
Page 29 GAO-03-734 International Financial Crises

most recent voluntary restructurings successfully concluded in 1 year or 

less and that creditor holdouts or litigation did not significantly delay the 

restructurings.28 

While private sector officials expect that many restructurings need only 

involve the private sector and the debtor country, under some 

circumstances, voluntary debt restructurings will not adequately resolve all 

financial crises. In those cases, they said the Fund should provide loans to 

eligible countries to help fill their external financing gaps. Such loans 

would assist the restructuring process and facilitate efforts at 

implementing necessary reforms. However, large Fund loans, such as those 

given during the Asian financial crises, have received substantial criticism 

from financial experts and government representatives, including U.S. 

government officials. One concern is that the possibility of receiving 

substantial financial assistance provides an incentive for debtor countries 

to adopt unsustainable economic policies to forestall needed reform. 

Another concern is that these large loans may encourage private sector 

creditors to continue providing large capital flows to countries with 

unsustainable economic policies because these otherwise risky 

investments have the potential of being “bailed out” by future Fund loans. 

This condition is referred to as “moral hazard.” According to these critics, 

efforts to help resolve existing financial crises through large Fund loans 

may increase the probability of future crises due to these two concerns. 

The Fund has advocated the SDRM framework and CACs to replace the 

current voluntary approach, partially in response to concerns over the 

potential adverse affects of its lending. 

The Fund Has Clarified and 

Strengthened its Lending 

Practices to Address 

Concerns Over Exceptional 

Lending

To reduce the risk that Fund loans would increase the probability of future 

financial crises, the Fund clarified and strengthened its policy of lending in 

crisis situations. The Fund has clarified elements of its Lending into 

Arrears Policy and strengthened its criteria for requesting large short-term 

loans under the Supplemental Reserve Facility (SRF). Since 1997, nine 

countries have received loans under the two mechanisms. 

The Fund’s Lending into Arrears Policy permits the IMF to provide 

resources to countries that are unable to repay their external creditors and 

are thus considered in default. Conceived in the late 1980s and amended in 

28For some countries, the negotiating process entailed several years and more than one 

restructuring agreement. 
Page 30 GAO-03-734 International Financial Crises

the late 1990s, the policy is designed to protect the value of creditor assets 

while providing creditors with incentives to enter rapidly into restructuring 

negotiations with countries. The Lending into Arrears Policy increases the 

likelihood that a country’s private sector lenders would agree to reduce the 

value of their loans because Fund resources reduce short-term fiscal 

pressures experienced by the country while in default. A country is eligible 

for Fund resources while in default if the Fund determines that the debt 

burden is unsustainable, and the country is making satisfactory progress in 

implementing reforms. Additionally, the country must have demonstrated a 

good faith effort to reach a restructuring agreement with creditors to 

restore its ability to repay its debt. In 2002, the Fund clarified the criteria to 

be used to determine whether the debtor country is making a good faith 

effort. For example, the Fund would consider how quickly the debtor 

engaged in negotiations with its creditors after it defaulted. To date, the 

Fund has lent into arrears on international bonds on four occasions—

Ukraine, Ecuador, Moldova, and Argentina.

Introduced in 1997, the SRF provides large short-term loans to members 

experiencing exceptional balance of payments difficulties prior to a 

default. The interest rates on these loans are much higher than standard 

Fund loans. The increased cost of these loans is expected to reduce the 

probability that countries consider Fund resources a viable means for 

underwriting unsustainable economic policies. Higher loan terms also 

increase incentives for early repayment and compensate for additional 

repayment risks to the Fund. Countries are expected to repay SRF loans 

within 2 to 21⁄2 years, but they may request extensions of up to 6 months. All 

SRF loans carry a substantial surcharge of 3–5 percentage points. In 2003, 

the Fund strengthened its criteria for providing large short-term loans 

under the SRF. For example, countries requesting SRF loans must provide a 

more extensive justification for their repayment difficulties. Additionally, 

the member has to demonstrate good prospects of regaining access to 

private capital markets within the time period that Fund resources are 

outstanding to minimize long-term reliance on Fund resources. To date, the 

Fund has provided SRF loans on nine occasions to six countries, including 

Korea, Russia, Brazil, Turkey, Argentina, and Uruguay (see appendix IX).

Conclusion In accordance with its goal of strengthening the international financial 

system, the Fund has undertaken a number of reforms to better anticipate, 

prevent, and resolve sovereign financial crises. The Fund’s new 

vulnerability assessment process is more comprehensive than its previous 

crisis anticipation efforts, but it is too soon to judge its effectiveness. The 
Page 31 GAO-03-734 International Financial Crises

Fund’s proposed approaches to better resolve financial crises have met 

considerable resistance, and it is unclear whether they will ultimately be 

adopted. The Fund and the Bank have made progress in their crisis 

prevention efforts by performing assessments of member countries’ 

financial sectors and adherence to standards. However, the effectiveness of 

these crisis prevention efforts is hindered by (1) private sector participants’ 

limited use of published assessments, which they find untimely, outdated, 

and too dense to be useful and (2) gaps in crucial information about crisis 

vulnerabilities in some important emerging market countries due to 

voluntary participation in the assessments. These limitations prevent 

multilateral institutions, national policy makers, and private sector 

participants from making sound decisions, thus reducing the likelihood 

that these reforms will help prevent crises.

Recommendations for 

Executive Action 

To help strengthen the Fund’s crisis prevention initiatives we recommend 

that the Secretary of the Treasury instruct the U.S. Executive Director of 

the Fund to work with other Executive Board members to encourage the 

Fund to

• improve the timeliness of publication of Financial System Stability 

Assessments and Reports on the Observance of Standards and Codes; 

• expand the coverage, frequency, and publication of reports on member 

countries’ progress on implementing assessment recommendations;

• improve the assessment reports’ readability, for example, by creating a 

standardized format in which to present executive summaries and key 

findings; and

• pursue strategies for increasing participation in the Financial Sector 

Assessment Program and all modules of the Reports on the Observance 

of Standards and Codes, including the possibility of making 

participation mandatory for all members of the IMF. 

Agency Comments and 

Our Evaluation

We received written comments on this report from the Department of the 

Treasury, the International Monetary Fund, and the World Bank. These 

comments and GAO’s evaluation of them are reprinted in appendixes X, XI, 

and XII. The organizations also separately provided technical comments 
Page 32 GAO-03-734 International Financial Crises

that GAO discussed with relevant officials and included in the text of the 

report where appropriate.

Treasury agreed with the report’s recommendations. Treasury recognized 

that some important countries have not volunteered to participate in the 

FSAP and ROSC and that there should be a shorter turnaround between the 

completion of an assessment and its public release. Treasury also pointed 

out that the acceptance of collective action clauses in some recent bond 

offerings serves as an important signal to investors that official financing is 

limited and that they cannot expect to be protected from risks.

The IMF broadly agreed with the report’s recommendations. However, the 

IMF stated that we mischaracterized the role of the WEO forecasts and 

EWS models in IMF crisis anticipation efforts by saying that they have a 

greater importance than is warranted. We disagree with this depiction. Our 

assessment examined all six components of the IMF’s vulnerability 

assessment framework, including the WEO and the EWS. As the only 

mature and quantifiable elements of the framework, our analysis focused 

more heavily on the track records of the WEO and EWS. The IMF also 

stated that its responsibility to maintain financial stability could make its 

predictions less accurate so as not to contribute to a crisis. The IMF’s 

comment not only validates our finding on the WEO’s weakness but also 

raises questions regarding the purpose and credibility of the WEO 

forecasts. 

The World Bank generally agreed with the report’s recommendations. 

However, the Bank expressed concern with the report’s suggestion that 

consideration be given to making participation in the FSAP and ROSC 

assessments mandatory. While we are not suggesting that the assessments 

should be made mandatory, the voluntary nature of the FSAP and ROSC 

has posed an obstacle to full participation by important emerging market 

countries. 

We are sending copies of this report to the Secretary of the Treasury, the 

International Monetary Fund, the World Bank, and interested congressional 

committees. We also will make copies available to other interested parties 

upon request. In addition, the report will be available at no charge on the 

GAO Web site at http://www.gao.gov.
Page 33 GAO-03-734 International Financial Crises

If you or your staff have any questions regarding this report, please call me 

at (202) 512-8979. Other GAO contacts and staff are acknowledged in 

appendix XIII.

Joseph A. Christoff, Director

International Affairs and Trade
Page 34 GAO-03-734 International Financial Crises

Appendix I

Objectives, Scope, and Methodology AAppenendxIdeisi

The Chairman of the House Financial Services and the Vice Chairman of 

the Joint Economic Committee requested that we assess (1) the 

International Monetary Fund’s (IMF’s or the Fund’s) framework for 

anticipating financial crises, (2) the status of key IMF initiatives to prevent 

financial crises, and (3) new IMF proposals to resolve future financial 

crises. They requested that our review cover the period after the Mexican 

financial crisis of 1994–95.

To assess IMF’s framework for anticipating financial crises, we examined 

prior and new IMF mechanisms for anticipating crises. Our analysis 

focused on the World Economic Outlook (WEO) forecasts and the IMF’s 

Early Warning System (EWS) models that were the IMF’s primary 

forecasting tools prior to the implementation of the new vulnerability 

assessment framework in May 2001. Data on the performance of the other 

four components of the framework were not made available to us because 

these elements are considered part of the staff level deliberative process, 

and are not provided to the Executive Board. We obtained near-term data 

from the WEO forecasts, including real gross domestic product (GDP) 

growth rate and current account balance for 87 emerging market countries 

for the period 1990–2001 (see appendix III). We focused on the 81 middleincome countries and an additional 6 low-income countries listed by J.P. 

Morgan as emerging markets. To evaluate the WEO and program forecasts, 

we used standard econometric techniques based on methods commonly 

found in the forecasting literature. The formal methodology of our forecast 

evaluations was based on several expert publications, the replication of our 

summary statistics with another author’s results, and discussions with a 

forecasting expert. To describe the performance of the IMF’s EWS models 

in anticipating crises, we reviewed and summarized the results of IMF 

evaluations. We interviewed IMF staff, including country desk economists 

and staff from several departments, to discuss the IMF’s new framework 

for vulnerability assessment, the EWS models, and the WEO methodology. 

We also interviewed 23 major private sector emerging market participants 

to discuss whether and how they use IMF forecasts in their investment 

decisions.

To assess the status of key IMF initiatives to prevent financial crises, we 

reviewed Fund and World Bank documents published between 1999 and 

2003 on the creation of the Financial Sector Assessment Program (FSAP) 

and Reports on the Observance of Standards and Codes (ROSC) and 

evaluations of progress in implementing these reforms. We interviewed 

senior Fund officials, including staff from the Monetary Affairs and 

Exchange Department, the Policy Development and Review Department, 
Appendix I

Objectives, Scope, and Methodology

Page 35 GAO-03-734 International Financial Crises

and the Fiscal Affairs Department. We also met with senior advisers at the 

World Bank (the Bank) who oversee the Bank’s participation in the FSAP 

and ROSC initiatives. To gain a better understanding of how the Fund uses 

FSAP and ROSC assessments to inform the policy advice it provides to 

member country governments and challenges it faces in using these 

assessments, we interviewed Fund officials in nine area departments 

(Argentina, Brazil, Korea, Mexico, Peru, Poland, Russia, Turkey, and 

Uruguay). We also spoke with Fund officials in these area departments 

about member country governments’ use of the assessments in shaping 

their reform agendas and the obstacles that member country authorities 

encounter in implementing the reforms recommended in the assessments. 

To assess the extent to which emerging market countries have participated 

in and authorized publication of FSAP and ROSC assessments, we 

examined Fund and World Bank data on country participation in the FSAP 

and 12 ROSC modules and publication of the resulting reports between 

May 1999 and March 2003. To obtain views on the private sector’s use of 

Fund and World Bank assessments, we conducted structured interviews 

with 13 representatives of private sector firms, including ratings agencies, 

investment banks, and pension funds. We focused on 33 countries (a subset 

of the 87 we analyzed in the previous section) identified as the major 

emerging market countries by J.P. Morgan.1

 

To describe new proposals to resolve future financial crises and their 

potential challenges, we obtained the most current Fund documentation 

for the two key proposals, the Sovereign Debt Restructuring Mechanism 

(SDRM) and Collective Action Clauses (CACs). We examined the purpose, 

goals, requirements, and status of implementation. To obtain views on the 

private sector’s understanding of the components of the new proposals and 

potential implementation challenges, we conducted structured interviews 

with 22 representatives of private sector firms, including ratings agencies, 

investment banks, and pension funds. We also met with government, 

private sector emerging market participants, and nongovernmental 

officials at several conferences. We also interviewed Department of the 

Treasury officials and experts in international finance and law. The IMF did 

not meet with us on these proposals because they were still under 

negotiation at the time of our review. 

1

These 33 countries represent the most significant emerging market participants in the 

international capital markets, as identified by J.P. Morgan. 
Appendix I

Objectives, Scope, and Methodology

Page 36 GAO-03-734 International Financial Crises

We conducted fieldwork in Washington, D.C., and in New York. We 

performed our work from May 2002 to May 2003 in accordance with 

generally accepted government auditing standards.
Page 37 GAO-03-734 International Financial Crises

Appendix II

Private Sector Structured Interview 

Participants ApenIdxi

Source: GAO.

Perspectives obtained on IMF efforts in crisis

Company Anticipation Prevention Resolution

Citicorp x xx

Cleary Gottlieb & Hamilton x x

Credit Suisse x xx

Darby’s Overseas Investments x x x

Elliot Associates x x

Emerging Market Traders Association x x

eStandards Forum x x x

Eurasia Group x x x

Fitch Ratings x xx

HBK x x

HSBC x xx

International Primary Markets Association x x

Institute for International Finance x x

Japan Center for International Finance x x

J.P. Morgan x x

Lehman Brothers x x x

Mass Mutual x x

Moody’s Investor Service x x x

Securities Industry Association x x x

Societe Generale x x x

Standard & Poor’s x x x

Straus & Boies x x

Wilshire Associates x x
Page 38 GAO-03-734 International Financial Crises

Appendix III

Assessment of IMF Forecasting ApenIdxi

Congress expressed concerns regarding the accuracy of the International 

Monetary Fund’s (IMF’s) growth rate projections and asked us to examine 

them. In response, we analyzed the quality of the forecasts produced by the 

World Economic Outlook (WEO), the Fund’s primary forecasting tool. 

Using econometric techniques common to forecast evaluation studies, we 

examined the basic measures of forecast accuracy, bias, and efficiency. 

This assessment supplements our finding on WEO’s efforts to anticipate 

crises reported earlier. We found that WEO forecasts of gross domestic 

product (GDP) growth and inflation perform somewhat better than an 

assumption that next year’s rate will be the same as this year’s (called a 

“naive” forecast). However, there is evidence of an optimistic bias in the 

forecasts of GDP growth and inflation. In addition, we found that the naive 

forecast of the current account generally performed better than the WEO 

forecast. Moreover, WEO forecasts for the major industrialized countries 

were superior to emerging market forecasts, and forecasts for emerging 

market countries that had been on an IMF program were better than for 

those countries that were not. The shortcomings we observed in WEO 

forecasting are similar to those encountered by other private sector and 

official forecasters.

GAO Analysis Used 

Standard Econometric 

Techniques for 

Forecast Evaluation

We evaluated IMF forecasts for 87 emerging market countries. Our analysis 

focused on WEO forecasts of GDP, inflation, and the current account. Our 

measures of forecast quality relied on generally accepted econometric 

measures of accuracy, bias, and efficiency.

Overall Approach To evaluate the quality of IMF forecasts, we analyzed the near-term and 

year-ahead WEO forecasts for 87 middle-income emerging market 

countries.1

 Appendix IV lists the 87 emerging market countries used in the 

analysis. Our analysis focused on three WEO forecast variables: (1) the 

growth rate of real GDP, (2) consumer price index (CPI) inflation (average 

over period), and (3) current account balance in billions of U.S. dollars. Our 

evaluation methodology is based on standard econometric techniques 

commonly found in the forecast literature, including the work of 

1

Our analysis focused on 81 middle-income countries and an additional 6 low-income 

countries listed by J.P. Morgan as emerging markets. 
Appendix III

Assessment of IMF Forecasting

Page 39 GAO-03-734 International Financial Crises

forecasting experts such as Stekler (1991), Artis (1996), and Loungani 

(2001).2

 

We also compared the quality of WEO forecasts of emerging market 

countries with WEO forecasts of the G-7 countries and we compared 

borrowers of IMF resources with those that were not.3

 Our comparison 

with the G-7 countries4 allowed us to informally assess whether income 

level or data quality mattered in forecast quality. Our analysis of forecasts 

for program countries permitted us to assess whether WEO forecasts differ 

from forecasts contained within program documents, which are produced 

under conditions of greater staff scrutiny.5

 We also reviewed a number of 

forecast evaluation studies to see how our results compared to previous 

reviews and to contrast IMF forecast quality with other forecasting efforts.

Basic Definitions and Methods of 

Evaluation

Our analysis focused on the WEO’s near-term and year-ahead forecasts. 

Near-term forecasts originate from the May WEO for each year, and they 

project for the remainder of the existing year (approximately 6 months 

ahead). The year-ahead forecasts come from the October WEO of the 

preceding year. Thus, a near-term forecast for 2000 would come from the 

May 2000 WEO, and a year-ahead forecast for 2000 would come from the 

October 1999 WEO. We compared these WEO forecasts to the “first settled 

estimate,” which comes from the October WEO of the following year for 

which the forecast is made. Thus, we compared both the near-term and

2

Herman Stekler, “Macroeconomic Forecast Evaluation Techniques,” International Journal 

of Forecasting 7 (1991): 375-384; Michael J. Artis, “How Accurate are the IMF’s Short-Term 

Forecasts? Another Examination of the World Economic Outlook,” WP/96/89 (Washington, 

D.C.: International Monetary Fund [IMF], August 1996); Prakash Loungani, “How Accurate 

are Private Sector Forecasts? Cross-Country Evidence from Consensus Forecasts of Output 

Growth,” International Journal of Forecasting 17 (2001): 419-432.

3

We include the following General Resources Account-funded IMF programs: Stand-By 

Arrangements (SBA), Extended Fund Facility (EFF), Systemic Transformation Facility 

(STF), Compensatory Contingency Finance Facility (CCFF). Under the CCFF, countries can 

borrow resources on a stand-alone basis, i.e., outside of an IMF program. Macedonia and 

South Africa borrowed funds under the CCFF.

4

The G-7 include Canada, France, Germany, Italy, Japan, United Kingdom, and United States.

5

Of the 87 emerging market countries analyzed, 57 were on an IMF program for some part of 

the 1990–2001 period.
Appendix III

Assessment of IMF Forecasting

Page 40 GAO-03-734 International Financial Crises

year-ahead forecasts for 2000 with the “first settled estimate” from the 

October 2001 WEO.6

Most of the econometric tools we used to assess the quality of WEO 

forecasts analyze the errors deriving from the forecasts. Our econometric 

tools examined these errors for certain qualities and patterns. We defined 

the forecast error, as the difference between the forecasted, and 

realized, value of an indicator. Hence, we have

Our examination of the errors in the Fund’s forecasts focused on three 

measures of “goodness”: accuracy, bias, and efficiency. We performed 

these tests separately for the 87 countries over the 11-year forecasting 

period.

 Accuracy The credibility of a forecast is established by its accuracy. The ultimate test 

of any forecast is whether it can predict future events accurately. Accuracy 

assesses whether forecasts tend, by some standard, to be close to actual 

outcomes. Although there is no objective standard of accuracy, 

comparisons to alternative forecasting methods, such as a naive model that 

uses historical trend data, is one way to judge relative accuracy. The 

accuracy measure we used is Theil’s U-statistic (U1), based on the naive 

model that assumes this year’s growth rate will be the same as last year’s.7

The Theil U-statistic is based on an examination of the forecast’s root mean 

square error (RMSE). To compute RMSE, the forecast errors are squared 

and averaged over the sample to get the mean square error (MSE). RMSE is 

the square root of MSE. 

6

A persistent issue in the forecasting literature concerns which “actual” value to use to 

evaluate the accuracy of the forecasts: the first available estimate (available in May of the 

following year), the first settled estimate (available in October of the following year), or a 

later revision. We have taken the middle ground, as suggested by Loungani (2001), on the 

basis that forecasters are not attempting to predict later revisions, which incorporate 

information such as revisions of weights and changes in methods of construction that 

forecasters would not have been aware of at the time of the forecast. 

, t e , t f

, t a

7

That is, the naive model assumes there is no change in the growth rate between t-1 and t.
Appendix III

Assessment of IMF Forecasting

Page 41 GAO-03-734 International Financial Crises

The Theil U-statistic allows us to scale the RMSE by the variability of the 

underlying data. The standard Theil statistic, commonly denoted as , 

compares the RMSE of forecast series to the RMSE of the actual series,

where = - . A statistic greater than one denotes that the 

naive model performed better than the model being tested.

Bias Bias determines whether forecast errors in one direction tend to be larger 

and/or more numerous than errors in the opposite direction. Forecast 

errors can be divided into two parts. One part is the “random error,” which 

varies unsystematically, or randomly, from one forecast to the next. The 

other part is the “bias error,” which remains constant for any particular 

forecasting procedure. Bias happens when factors other than random 

events influence the forecast results, resulting in an upward or downward 

tendency. An unbiased forecast means that forecast errors are 

approximately zero on average over time. However, an unbiased forecast 

does not guarantee that a forecast will be accurate enough to be useful if 

the errors are large.

To assess the extent of bias in the forecasts, we regress the forecast error 

on a constant term and then carry out a t-test for the coefficient. For a time 

series of forecasts (1990–2001 in our analysis), we have the set of forecast 

errors . The regression test for bias involves estimating the 

intercept coefficient, , for the simple regression

U1

∆at at t −1 a U1

{ }N e ,e , ,e 1 2 K

β 0
Appendix III

Assessment of IMF Forecasting

Page 42 GAO-03-734 International Financial Crises

This reflects a partition of the forecast error into an estimate of the 

systematic component and a random component ( ). A t-test with 

degrees of freedom is then performed to test versus the twosided alternative We perform a t-test to determine whether 

the average bias is significantly different from zero. If the p-value is less 

than .05, we reject the hypothesis that there is no bias at the 95 percent 

level of significance. This means that there is less than a 5 percent chance 

that we are making a false rejection, that is saying the forecast is biased 

when it is not. A determination that a forecast is unbiased is a necessary, 

although not sufficient, condition for concluding that a forecast is efficient.

Efficiency Efficiency examines whether a forecast has taken into account all available 

information. Establishing that a forecast is efficient means that no other 

model or readily available information would be able to improve the 

forecast, and there is no way to predict the direction or size of the errors. A 

test of efficiency makes use of the simple linear model where we regress 

the actual outcome on the forecast

If the forecast is efficient in predicting the actual outcome, then the 

intercept, , should be equal to 0 and the slope, , should be equal to 1.8

 

Using the regression model defined above, we perform a joint hypothesis 

test to check whether these conditions hold simultaneously.

t ε N −1

: 0 H0 β 0 =

: 0. HA β 0 ≠

8

A zero intercept implies that the errors are randomly distributed; they vary 

unsystematically (unbiased). However, a forecast can be unbiased and not efficient. A slope 

of 1 implies a straight line through the zero intercept, denoting efficiency. A slope of 1 

indicates that the forecast and actual value essentially coincide.

β 0 β 1
Appendix III

Assessment of IMF Forecasting

Page 43 GAO-03-734 International Financial Crises

The algebraically simplified version of the test statistic for this hypothesis 

is shown below. 9

The reference distribution for this statistic is an F-distribution with 2 and 

 degrees of freedom. If the p-value for this statistic is less than .05, 

then we reject the hypothesis at the 95 percent level of significance that 

both the intercept is zero and the slope is one. This means that there is only 

a 5 percent chance that we are making a false rejection, that is saying the 

forecast is not efficient when it is. 

WEO Forecasts 

Demonstrate Some 

Accuracy, but Also 

Optimistic Bias

Our analysis of the WEO forecasts for 87 emerging market countries shows 

that WEO forecasts perform somewhat better than a naive model for GDP 

growth and inflation, but not for current account (see table 2). 

Table 2: Forecast Quality for 87 Emerging Market Countries, 1990–2001

Source: GAO analysis of IMF data.

a

The percentage of countries in which the WEO forecast does a better job than the naive forecast 

(based on the Theil statistic).

b

The percentage of countries in which there is statistically significant bias (at the 5 percent level).

c

When bias occurs in 15 percent or more countries and the forecast errors tend to vary predominantly 

in one direction (more than 70 percent), then we indicate the direction of this bias. 

9

RSS is the residual sum of squares. 

N − 2

Accuracya

(percent)

Biasb

(percent) Direction of biasc

Efficiencyd

(percent)

Forecast variable

YearaheadNeartermYearaheadNeartermYearaheadNeartermYearaheadNearterm

GDP growth rate 62 74 21 15 Upward Upward 76 85

Inflation 55 70 21 13 Downward Downwarde 72 67

Current account 

(billions of U.S. $)

24 56 8 8 Upwarde Indeterminate 67 79
Appendix III

Assessment of IMF Forecasting

Page 44 GAO-03-734 International Financial Crises

d

Percentage of countries where we fail to reject the joint hypothesis that the intercept is zero and the 

slope is one; i.e., the evidence is not strong enough to reject the assumption of efficiency. 

e

Italics indicate that there are too few countries to consider bias a systematic trend.

We found the year-ahead WEO forecast does a better job than the naive 

forecast in more than 60 percent of the countries with GDP, in more than 

half the countries with inflation, and in about one-quarter of the countries 

with the current account. However, even for GDP, nearly 40 percent of the 

country forecasts were no better than an assumption that next year’s value 

is the same as this year’s. For all three variables, the shorter the forecast 

period, the more accurate the forecast. When the forecast time horizon 

shortens from 1 year to 6 months, the percentage of cases in which the 

WEO does a better job forecasting than the naive model increases for all 

three variables, exceeding 50 percent for the current account.

WEO forecasts for GDP and inflation demonstrated bias in about 20 

percent of the country cases. The direction of the bias was upward for GDP 

and downward for inflation, indicating an optimistic tendency within the 

WEO forecasting process.10 Although the bias was upward for the current 

account, also consistent with optimism, it occurred in only 8 percent of 

country forecasts.11 For all three variables, we could not reject the 

hypothesis that WEO forecasts were efficient for at least two-thirds of the 

country’s forecasts. However, in about one-fourth of the country cases, the 

WEO forecast could have been improved through the use of a different 

model or the addition of new information.

Forecasts for Major 

Industrialized Countries Are 

Better than Emerging 

Market Forecasts and 

Program Forecasts Are 

Better than Nonprogram 

Forecasts

WEO forecasts of the most developed countries are superior to its 

forecasts of emerging market countries when compared to the naive model 

forecasts. The improved forecast quality is likely due to better available 

data and greater stability of the wealthiest economies. Similarly, WEO 

forecasts for countries that borrow from the IMF are superior to those that 

do not. The increased scrutiny of borrowing countries by IMF staff likely 

contributes to the improved forecasts.

10Our finding of an optimistic (upward) bias for GDP forecasts is consistent with, and helps 

explain, our finding in the main report of the WEO’s difficulty in forecasting recessions.

11An optimistic bias in the current account means that the forecast was for a smaller current 

deficit or a larger surplus than occurred. 
Appendix III

Assessment of IMF Forecasting

Page 45 GAO-03-734 International Financial Crises

WEO Forecasts for Major 

Industrialized Countries Are 

Better than Those for Emerging 

Markets

WEO forecasts of GDP and inflation for the G-7 countries are considerably 

better than its forecasts for the emerging market countries when compared 

to the naive model forecasts. (see table 3). 

Table 3: Forecast Quality for G-7 Countries, 1990-2001

Legend

N/A = not applicable

Source: GAO analysis of IMF data.

Note: Number of countries, and not percentages, are reported given the small number of cases.

a

The number of countries for which the WEO forecast does a better job than the naive forecast (based 

on the Theil statistic).

b

The number of countries in which there is statistically significant bias (at the 5 percent level).

c

The number of countries where we fail to reject the joint hypothesis that the intercept is zero and the 

slope is one, i.e., the evidence is not strong enough to reject the assumption of efficiency. 

d

Italics indicate that there are too few countries to consider bias a systematic trend.

This improvement is evident across the full range of analyses. For example, 

in six out of seven countries, the year-ahead WEO forecasts of GDP and 

inflation for the G-7 countries were found to be accurate, and the near-term 

forecasts for GDP and inflation were accurate for all of the G-7 countries. 

These results are considerably better than WEO forecasts for emerging 

markets. In the year-ahead forecasts, bias and efficiency were a concern in 

two forecasts of GDP, and one current account forecast, but they were not 

a concern in the inflation forecasts. Although the year-ahead forecast for 

current account was inaccurate for six of seven countries, the near-term 

forecast was accurate for five of the G-7 country cases. The improved 

quality of WEO forecasts for the G- 7 countries is likely due to better 

available data and greater stability of these economies, compared to 

emerging market countries. 

Accuracya

(number of 

countries)

Biasb

(number of 

countries)

Direction of bias

(number of countries)

Efficiencyc

(number of countries)

Forecast variable

YearaheadNeartermYearaheadNeartermYearaheadNeartermYearaheadNearterm

GDP growth rate 6 7 2 0 Upwardd N/A 5 7

Inflation 6 7 0 0 N/A N/A 7 7

Current account 

(billions of U.S. $)

1 5 10 Downwardd N/A 5 7
Appendix III

Assessment of IMF Forecasting

Page 46 GAO-03-734 International Financial Crises

WEO Forecasts of Countries on 

an IMF Program Are Superior to 

Nonprogram Country Forecasts 

We found that WEO forecasts for 57 countries that were on an IMF 

program12 (or that borrowed Fund resources under the CCFF) for any part 

of the forecast period tend to be more accurate than WEO forecasts for the 

30 countries that were never on an IMF program during this period (see 

table 4). 

Table 4: Comparison of Year-Ahead WEO Forecasts for Program and Nonprogram Countries, 1990–2001

Legend

N/A = not applicable

Source: GAO analysis of IMF data.

a

For pooled countries, if the U statistic is greater than 1, the naive forecast does a better job than the 

forecast under evaluation.

b

Bias at the 5 percent level of significance means that there is less than a 5 percent chance that we are 

making a false rejection—i.e., saying the forecast is biased when it is not. 

Countries that borrow from the IMF are likely to be under greater scrutiny 

from Fund staff than those that do not borrow, which could contribute to 

an improved forecast. For this analysis, we compared the Theil statistics 

for GDP growth, inflation, and current account for the two pooled 

forecasts. We found that the WEO forecasts of GDP and inflation for 

program countries are more accurate than those for nonprogram countries. 

That is, when compared to the naive model, the program countries have a 

lower Theil statistic than nonprogram countries. For both groups, the 

forecast of current account is inferior to the naive forecast (a Theil statistic 

greater than 1). The WEO program countries forecasts for GDP and 

inflation are biased, whereas the forecasts for the nonprogram countries 

were not. This indicates that by assuming implementation of the policies 

12We include countries that were on the following General Resources Account-funded IMF 

programs and facilities: SBA, EFF, STF, CCFF. Some countries were on more than one 

program during the forecast period.

Accuracy

Theil U-statistica

Bias

5-percent significance levelb Direction of bias

Forecast variable

57 program

countries

30 Nonprogram

countries

57 program

countries

30 Nonprogram

countries

57 program

countries

30 Nonprogram

countries

GDP growth rate .827 .941 Yes No Upward N/A

Inflation .776 .918 Yes No Downward N/A

Current account

(billions of U.S. $)

 2.399 1.129 No Yes N/A Downward
Appendix III

Assessment of IMF Forecasting

Page 47 GAO-03-734 International Financial Crises

contained within the program, the Fund expects that GDP and inflation will 

perform better than they actually do. 

In addition to the publicly available WEO forecasts for all countries, the 

IMF also produces a set of program forecasts for countries in the years they 

borrow from the Fund.13 According to the Fund, these two forecasts should 

be very similar since they are prepared by the same staff in the same 

manner. Our comparison of program14 and WEO forecasts for the initial 

year that each country was on program confirmed that the accuracy of the 

two forecasts for GDP and inflation were nearly identical (see table 5). 

Table 5: Comparison of Program and WEO Year-Ahead Forecasts, for 52 Countries in the Initial Years on Program, 1990–2001a

Legend

N/A = not applicable

Source: GAO analysis of IMF data.

Note: This analysis is based on 96 country/year observations, which is an average across three 

variables.

a

We were able to obtain program projections for only 52 of the 57 program countries.

b

If the U statistic is greater than 1, the naive forecast does a better job than the forecast under 

evaluation.

c

Bias at the 5 percent level of significance means that there is less than a 5 percent chance that we are 

making a false rejection—i.e., saying the forecast is biased when it is not.

d

The forecast is biased at the 10 percent level of significance means that there is less than a 10 

percent chance that we are making a false rejection—i.e., saying the forecast is biased when it is not.

13For most of the years studied, these forecasts were not made public. In recent years, when 

countries agree, these forecasts are made public.

14We used the original program numbers, that is, the projections established at the outset 

when the IMF’s Executive Board first approves an arrangement.

Accuracy

Theil U-statisticb

Bias

5 percent significance levelc Direction of bias

Forecast variable

Program

forecasts

WEO

forecasts

Program 

forecasts 

WEO

forecasts 

Program

forecasts 

WEO 

forecasts 

GDP growth rate .689 .676 Yes Yes Upward Upward

Inflation .683 .694 Nod Nod N/A N/A

Current account

(billions of U.S. $)

.524 2.292 No No N/A N/A
Appendix III

Assessment of IMF Forecasting

Page 48 GAO-03-734 International Financial Crises

However, program forecasts of current account are substantially better 

than those reported in the WEO. In addition, in all cases the program 

forecasts were substantially more accurate than the naive model. This is 

further evidence that the greater scrutiny experienced by these countries 

while under a program probably contributes to an improved forecast. 

Other Forecasting Efforts 

Have Difficulties Similar to 

the Fund

Our review of other forecast evaluations found that the shortcomings we 

observed in WEO forecasting are similar to difficulties encountered by 

other forecasters.15 These studies examined forecasts16 produced by the 

private sector (for example, consensus forecasts), governments, and 

multinational agencies including the IMF and Organization for Economic 

Cooperation and Development. These studies, similar to our observation in 

this report, found a general inability to predict recessions. In addition, 

consistent with our results, these studies found that (1) the shorter the time 

horizon, the more accurate the forecasts; (2) current account forecasts are 

15Harjit K. Arora & David J. Smyth, “Forecasting the Developing World: An Accuracy 

Analysis of the IMF’s Forecasts,” International Journal of Forecasting 6 (1990): 393-400; 

Michael J. Artis, “How Accurate are the IMF’s Short-Term Forecasts? Another Examination 

of the World Economic Outlook,” WP/96/89 (Washington, D.C.: International Monetary Fund 

[IMF], August 1996); José M. Barrionuevo, “How Accurate Are the World Economic Outlook 

Projections?” in World Economic Outlook, ch. 2, Staff Studies for World Economic Outlook

(Washington, D.C.: IMF, 1993); Roy Batchelor, “The IMF and OECD versus Consensus 

Forecasts” (London, England: City University Business School, August 2000); William W. 

Beach, Aaron B. Schavey, & Isabel M. Isidro, “How Reliable Are IMF Economic Forecasts?” 

CDA 99-05 (Washington, D.C.: The Heritage Foundation, The Heritage Center for Data 

Analysis, August 27, 1999); David Fintzen & H.O. Stekler, “Why Did Forecasters Fail to 

Predict the 1990 Recession?” International Journal of Forecasting 15 (1999): 309-323; IMF, 

“The Accuracy of World Economic Outlook Growth Forecasts: 1991–2000,” in World 

Economic Outlook, Box 3.1 (Washington, D.C.: IMF, December 2001): 37-39, and IMF, “How 

Well Do Forecasters Predict Turning Points?” in World Economic Outlook, Box 1.1 

(Washington, D.C.: IMF, May 2002): 6-8; Grace Juhn & Prakash Loungani, “Further CrossCountry Evidence on the Accuracy of the Private Sector’s Output Forecasts,” IMF Staff 

Papers 49, no. 1 (2002); Prakash Loungani, “How Accurate are Private Sector Forecasts? 

Cross-Country Evidence from Consensus Forecasts of Output Growth,” International 

Journal of Forecasting 17 (2001): 419-432; Albert Musso & Steven Phillips, “Comparing 

Projections and Outcomes of IMF-Supported Programs,” IMF Staff Papers 49, no. 1 (2002); 

Scott Shuh, “An Evaluation of Recent Macroeconomic Forecast Errors,” New England 

Economic Review (January/February 2001); Marten Blix et al., “How Good is the 

Forecasting Performance of Major Institutions?” Economic Review (Stockholm, Sweden: 

Sveriges Riksbank Monetary Policy Department, March 2001): 38-68; and Victor Zarnowitz, 

“The Record and Improvability of Economic Forecasting,” NBER Working Paper No. 2099

(December 1986).

16Forecasters evaluated the accuracy of many variables, including output (GDP), inflation, 

current account, exports, and imports, among others.
Appendix III

Assessment of IMF Forecasting

Page 49 GAO-03-734 International Financial Crises

markedly weaker than the forecasts for GDP and inflation; (3) when bias is 

found, forecasts tend to overestimate GDP and underestimate inflation; 

and (4) GDP and inflation forecasts for the industrial countries tend to be 

more accurate and less biased than forecasts for developing countries. 

While several studies found that WEO forecasts for developing countries 

were inferior to those generated by the naive model, one study found that 

WEO forecasts for developing countries did notably better than a naive 

forecast. A number of studies compared the quality of WEO forecasts with 

those produced by the private sector. Although some studies found the 

relative quality of the forecasts to be generally the same, a few studies 

found WEO forecasts to be less accurate than those of the private sector. 
Page 50 GAO-03-734 International Financial Crises

Appendix IV

Eighty-seven Emerging Market Countries ApenVIdxi

Country Regiona Income groupb

1 Albania Countries in transition Lower middle income

2 Algeria Africa Lower middle income

3 Antigua and Barbuda Western Hemisphere Upper middle income

4 Argentina Western Hemisphere Upper middle income

5 Bahrain Middle East and Turkey Upper middle income

6 Belarus Countries in transition Lower middle income

7 Belize Western Hemisphere Lower middle income

8 Bolivia Western Hemisphere Lower middle income

9 Botswana Africa Upper middle income

10 Brazil Western Hemisphere Upper middle income

11 Bulgaria Countries in transition Lower middle income

12 Cape Verde Africa Lower middle income

13 Chile Western Hemisphere Lower middle income

14 China Developing Asia Lower middle income

15 Colombia Western Hemisphere Lower middle income

16 Costa Rica Western Hemisphere Upper middle income

17 Cote d’Ivoire Africa Low income

18 Croatia Countries in transition Upper middle income

19 Czech Republic Countries in transition Upper middle income

20 Djibouti Africa Lower middle income

21 Dominica Western Hemisphere Upper middle income

22 Dominican Republic Western Hemisphere Lower middle income

23 Ecuador Western Hemisphere Lower middle income

24 Egypt Middle East and Turkey Lower middle income

25 El Salvador Western Hemisphere Lower middle income

26 Equatorial Guinea Africa Lower middle income

27 Estonia Countries in transition Upper middle income

28 Fiji Developing Asia Lower middle income

29 Gabon Africa Upper middle income

30 Grenada Western Hemisphere Upper middle income

31 Guatemala Western Hemisphere Lower middle income

32 Guyana Western Hemisphere Lower middle income

33 Honduras Western Hemisphere Lower middle income

34 Hungary Countries in transition Upper middle income

35 India Developing Asia Low income
Appendix IV

Eighty-seven Emerging Market Countries

Page 51 GAO-03-734 International Financial Crises

36 Indonesia Developing Asia Low income

37 Iran, Islamic Rep. of Middle East and Turkey Lower middle income

38 Iraq Middle East and Turkey Lower middle income

39 Jamaica Western Hemisphere Lower middle income

40 Jordan Middle East and Turkey Lower middle income

41 Kazakhstan Countries in transition Lower middle income

42 Kiribati Developing Asia Lower middle income

43 Korea Developing Asia Upper middle income

44 Latvia Countries in transition Lower middle income

45 Lebanon Middle East and Turkey Upper middle income

46 Libya Middle East and Turkey Upper middle income

47 Lithuania Countries in transition Lower middle income

48 Macedonia, former Yugoslav Rep. of Countries in transition Lower middle income

49 Malaysia Developing Asia Upper middle income

50 Maldives Developing Asia Lower middle income

51 Mauritius Africa Upper middle income

52 Mexico Western Hemisphere Upper middle income

53 Morocco Africa Lower middle income

54 Namibia Africa Lower middle income

55 Nigeria Africa Low income

56 Oman Middle East and Turkey Upper middle income

57 Pakistan Developing Asia Low income

58 Panama Western Hemisphere Upper middle income

59 Papua New Guinea Developing Asia Lower middle income

60 Paraguay Western Hemisphere Lower middle income

61 Peru Western Hemisphere Lower middle income

62 Philippines Developing Asia Lower middle income

63 Poland Countries in transition Upper middle income

64 Romania Countries in transition Lower middle income

65 Russia Countries in transition Lower middle income

66 Samoa Developing Asia Lower middle income

67 Saudi Arabia Middle East and Turkey Upper middle income

68 Seychelles Africa Upper middle income

69 Slovak Republic Countries in transition Upper middle income

70 South Africa Africa Upper middle income

71 Sri Lanka Developing Asia Lower middle income

72 St. Kitts and Nevis Western Hemisphere Upper middle income

(Continued From Previous Page)

Country Regiona Income groupb
Appendix IV

Eighty-seven Emerging Market Countries

Page 52 GAO-03-734 International Financial Crises

Sources: IMF, World Bank, and J.P. Morgan.

a

Based on WEO regional groupings, IMF 2002. 

b

World Bank analytical classification by income, July 2001.

73 St. Lucia Western Hemisphere Upper middle income

74 St. Vincent and the Grenadines Western Hemisphere Lower middle income

75 Suriname Western Hemisphere Lower middle income

76 Swaziland Africa Lower middle income

77 Syrian Arab Rep. Middle East and Turkey Lower middle income

78 Thailand Developing Asia Lower middle income

79 Tonga Developing Asia Lower middle income

80 Trinidad and Tobago Western Hemisphere Upper middle income

81 Tunisia Africa Lower middle income

82 Turkey Middle East and Turkey Upper middle income

83 Turkmenistan Countries in transition Lower middle income

84 Ukraine Countries in transition Low income

85 Uruguay Western Hemisphere Upper middle income

86 Vanuatu Developing Asia Lower middle income

87 Venezuela Western Hemisphere Upper middle income

(Continued From Previous Page)

Country Regiona Income groupb
Page 53 GAO-03-734 International Financial Crises

Appendix V

Standards, Codes, and Principles Assessed 

under IMF and World Bank Reports on the 

Observance of Standards and Codes ApenVdxi

Assessment start 

date Responsible institution Standard or code and rationale for adoption

Transparency standards: The standards on transparency in government operations and policy making are considered within the Fund’s 

direct operational focus.

1996–97 IMF IMF Special Data Dissemination Standard (SDDS) and General 

Data Dissemination Standard (GDDS).

The purpose of the IMF’s SDDS is to guide member country 

governments that have, or might seek, access to international 

capital markets in publishing comprehensive, timely, accessible, 

and reliable economic and financial statistics. The purpose of the 

GDDS is to help any member country government provide more 

reliable economic data. The SDDS and GDDS were created in 

1996 and 1997, respectively. 

1998 

(modified 2001)

IMF IMF Code of Good Practices on Fiscal Transparency.

This Code is intended to help member country governments 

improve the disclosure of information about the design and results 

of fiscal policy, making governments more accountable for policy 

implementation and strengthening credibility and public 

understanding of macroeconomic policies and choices. 

1999 IMF IMF Code of Good Practices on Transparency in Monetary and 

Financial Policies.

This Code is designed to increase the effectiveness of monetary 

and financial policies by raising public awareness of the 

government’s policy goals and instruments and making 

governments (especially independent central banks and financial 

agencies) more accountable. 

Financial sector standards: The financial sector standards are considered within the direct operational focus of both the Fund and the 

World Bank and are generally assessed under the joint Fund-Bank FSAP. 

1999 Joint IMF-Bank Basel Committee’s Core Principles for Effective Banking 

Supervision (BCP).

The BCP is intended to guide the development of an effective 

system for supervising banks, a large sector of most economies. 

The IMF and World Bank began assessments of countries’ 

compliance with the BCP standard in conjunction with the Financial 

Sector Assessment Program (FSAP) launched in 1999. 

1999 Joint IMF-Bank International Organization of Securities Commissions’ (IOSCO) 

Objectives and Principles for Securities Regulation. 

The IOSCO Objectives and Principles are designed to help 

governments establish effective systems to regulate securities 

which contribute strongly to investor confidence. The IMF and 

World Bank began using them to assess securities regulation in 

conjunction with the FSAP, launched in 1999. 
Appendix V

Standards, Codes, and Principles Assessed 

under IMF and World Bank Reports on the 

Observance of Standards and Codes

Page 54 GAO-03-734 International Financial Crises

1999

(modified 2000)

Joint IMF-Bank International Association of Insurance Supervisors’ (IAIS) 

Insurance Core Principles.

The IAIS Core Principles are designed to contribute to effective 

insurance supervision that supports financial stability. The IMF and 

World Bank began assessing member countries’ regulatory 

practices in this area in conjunction with the FSAP, launched in 

1999. 

1999 Joint IMF-Bank Committee on Payments and Settlements Systems’ (CPSS) Core 

Principles for Systemically Important Payment Systems.

The CPSS Core Principles are intended to strengthen payments 

systems, which provide the channels through which funds are 

transferred among banks and other institutions. The IMF and the 

World Bank began assessing member countries’ observance of 

this standard in conjunction with the FSAP, launched in 1999. 

2002 Joint IMF-Bank Financial Action Task Force (FATF) 40 Recommendations on AntiMoney Laundering and 8 Special Recommendations on Terrorism 

Financing.

The FATF’s 40 Recommendations and 8 Special 

Recommendations are intended to promote policies that combat 

money laundering and terrorist financing, which threaten financial 

system integrity and may undermine the sound functioning of 

financial systems. In 2002, the IMF and World Bank agreed to 

perform anti-money laundering and terrorist financing 

assessments as a 12-month pilot program, generally in conjunction 

with the FSAP. 

Corporate sector standards: The corporate sector standards are considered important for the effective operation of domestic and 

international financial systems and are assessed by the World Bank.

2000 World Bank Organization for Economic Cooperation and Development’s 

(OECD) Principles of Corporate Governance.

The OECD developed its corporate governance principles to help 

governments evaluate and improve their legal, institutional, and 

regulatory frameworks for corporate governance. The World Bank 

developed a template for assessing adherence to corporate 

governance principles based on the OECD’s Principles established 

in 1999. 

(Continued From Previous Page)

Assessment start 

date Responsible institution Standard or code and rationale for adoption
Appendix V

Standards, Codes, and Principles Assessed 

under IMF and World Bank Reports on the 

Observance of Standards and Codes

Page 55 GAO-03-734 International Financial Crises

Sources: IMF and World Bank.

2000 World Bank International Accounting Standards Board’s (IASB) International 

Accounting Standards.

The ROSC’s accounting module is intended to compare member 

countries’ corporate accounting practices with international 

accounting standards and to analyze actual accounting practice to 

determine the extent of compliance with applicable standards. 

There is special focus on the strengths and weaknesses of the 

institutional framework for supporting high quality accounting and 

financial reporting. In 2000, the World Bank developed a template 

for assessing adherence to accounting standards based on the 

IASB Standards. 

2000 World Bank International Federation of Accountants’ (IFAC) International 

Standards on Auditing.

The ROSC’s auditing module compares member countries’ 

auditing standards and auditors’ professional code of ethics with 

the standards and codes issued by IFAC. Also, the quality of actual 

auditing practices is evaluated. There is special focus on the 

strengths and weaknesses of the institutional framework for 

supporting high quality audit. In 2000, the World Bank developed a 

template for assessing adherence to auditing standards based on 

the IFAC’s Standards. 

2001 (draft) World Bank World Bank Principles and Guidelines for Effective Insolvency and 

Creditor Rights Systems.

In 2001, the World Bank developed draft Principles and Guidelines

intended to help countries develop effective insolvency and creditor 

rights’ systems, two important components of financial system 

stability. The World Bank has conducted several assessments 

based on its draft Principles and Guidelines. The United Nations 

Commission on International Trade Law (UNCITRAL) is completing 

a draft Legislative Guide on Insolvency Law. UNCITRAL, Bank, 

and IMF staff are working toward a single standard. 

(Continued From Previous Page)

Assessment start 

date Responsible institution Standard or code and rationale for adoption
Page 56 GAO-03-734 International Financial Crises

Appendix VI

Update on the International Monetary Fund’s 

Safeguards Assessments ApenVdxiI

In response to allegations of misreporting and misuse of International 

Monetary Fund (IMF or the Fund) disbursements in the late 1990s, the 

Fund increased its efforts to protect its resources by introducing 

safeguards assessments, a process for evaluating the controls employed by 

the central banks of borrowing member countries and for recommending 

measures to address inadequacies. Safeguards assessments have detected 

numerous inadequacies that could lead to misuse of Fund resources and 

have recommended measures to remedy them.

The Fund Introduces 

Safeguards 

Assessments for 

Member Countries 

That Currently Borrow

In 2000, the Fund introduced safeguards assessments, a process for 

identifying inadequacies in central banks’ ability to ensure the integrity of 

their operations, especially the use of Fund resources. Safeguards 

assessments evaluate central banks’ internal and external audit 

mechanisms, legal structure and independence, financial reporting 

procedures, and systems of internal controls. 

In April 2002, the Fund’s Executive Board made safeguards assessments a 

permanent policy. Safeguards assessments apply to all member countries 

with current or anticipated borrowing arrangements with the Fund. 

Countries with borrowing arrangements approved after June 30, 2000, are 

subject to a full safeguards assessment covering the five areas listed above. 

Countries with arrangements in effect before June 30, 2000, that have not 

yet repaid all Fund resources, were subject to a partial assessment 

covering only the external audit mechanism. Countries that do not have 

borrowing arrangements or have already repaid all Fund resources are not 

subject to safeguards assessments. According to Fund officials, since 2000 

the IMF has not provided financial resources to countries that did not meet 

its safeguards requirements. 

As of December 2002, the Fund had performed 37 full safeguards 

assessments and 27 partial assessments, with 23 assessments under way.1

 

The completed assessments detected a number of serious vulnerabilities 

that could lead to misuse of central bank resources, including those 

borrowed from the Fund. Of the full safeguards assessments, the Fund 

found the following:

1

International Monetary Fund, Safeguards Assessments – Semi-Annual Update, SM/03/88 

(Washington, D.C.: Mar. 6, 2003).
Appendix VI

Update on the International Monetary Fund’s 

Safeguards Assessments

Page 57 GAO-03-734 International Financial Crises

• Inadequate accounting standards in 82 percent of the central banks, 

which interfere with the accurate recording of central bank operations. 

For example, some central banks did not adhere to a financial reporting 

framework such as the International Accounting Standards (IAS), which 

would help prevent misreporting of transactions. 

• Deficient internal audit in 79 percent of central banks, which reduces 

their ability to address risks of misuse and misreporting of Fund 

resources. For example, some internal audit departments did not audit 

high-risk areas such as foreign reserves management.

• Poor controls over foreign reserves and data reporting to the Fund in 49

percent of the central banks, increasing the possibility of misreporting 

and misuse of Fund resources. For example, safeguards assessments 

identified improper techniques for valuing foreign reserves and failure 

to reconcile data reported to the Fund for program monitoring purposes 

with underlying accounting records. 

According to Fund officials, when IMF staff detect significant weaknesses 

in the controls of assessed central banks, they recommend that the 

government take corrective actions. For actions that IMF staff consider 

essential, they may incorporate the recommendations into the list of 

preconditions that the Fund requires borrower countries meet before 

receiving IMF resources, or they may suggest that the government include 

the recommended actions in its official economic program. The Fund 

reports that of the 275 recommendations that were expected to be 

implemented on or before December 31, 2002, 23 percent were 

incorporated as conditions for IMF resources or included in official 

economic program statements.

Implementation of the 

Fund’s Assessment 

Recommendations

Fund staff monitor central banks’ implementation of recommendations by 

performing in-depth reviews of their audited annual financial statements 

and other documents every 12 to 18 months until the borrower country 

government has repaid all Fund resources. The Fund monitors on a 

continuous basis, central banks’ implementation of all other safeguards 

measures and of developments within the central banks that may lead to 

new vulnerabilities. 
Appendix VI

Update on the International Monetary Fund’s 

Safeguards Assessments

Page 58 GAO-03-734 International Financial Crises

Recently, the Fund reported that central banks have implemented 90 

percent of recommendations that IMF staff included as a precondition for 

receiving IMF resources. According to Fund officials, the IMF stopped 

disbursing resources in the few cases where governments failed to 

implement these essential recommendations. Similarly, the Fund reported 

that central banks have implemented 84 percent of measures included in 

governments’ official economic program statements. 

On the other hand, the Fund reported that some recommendations made by 

the safeguards assessments have not been implemented as intended, 

although Fund officials state that these delays typically occurred in 

nonpriority areas. When central bank authorities fail to implement the 

recommendations, Fund staff increase pressure to comply, often proposing 

the measures’ inclusion as a precondition for the next disbursement. 

However, the Fund reports that staff can only adopt this approach in 

countries where the Fund is actively disbursing funds. For countries that 

are not currently receiving Fund disbursements, implementation of 

recommendations from the safeguards assessments tends to be more 

problematic because the Fund cannot exert pressure through a borrowing 

arrangement. 
Page 59 GAO-03-734 International Financial Crises

Appendix VII

Fund/World Bank FSAP and ROSC Process ApenVdxiI
Page 60 GAO-03-734 International Financial Crises

Appendix VIII

Country Participation in and Publication of 

FSAPs and ROSCs ApenVdxiI

Figure 4 lists all countries that have participated in Financial Sector 

Assessment Program (FSAP) or Reports on the Observance of Standards 

and Codes (ROSC) assessments and whether or not these assessments 

were published. The figure describes participation and publication by the 

33 major emerging market countries. Countries highlighted in bold have 

not participated in any assessments. Figure 5 describes participation and 

publication by other countries (industrial, developing, and smaller 

emerging markets).
Appendix VIII

Country Participation in and Publication of 

FSAPs and ROSCs

Page 61 GAO-03-734 International Financial Crises

Figure 4: FSAP and ROSC Participation and Publication by Major Emerging Market Countries 

a

These countries participated in the FSAP under a pilot program. The reports were not intended for 

publication.
Appendix VIII

Country Participation in and Publication of 

FSAPs and ROSCs

Page 62 GAO-03-734 International Financial Crises

Figure 5: FSAP and ROSC Participation and Publication by Other Countries
Appendix VIII

Country Participation in and Publication of 

FSAPs and ROSCs

Page 63 GAO-03-734 International Financial Crises

a

These countries participated in the FSAP under a pilot program. The reports were not intended for 

publication.
Page 64 GAO-03-734 International Financial Crises

Appendix IX

Fund Resources Provided under the 

Supplemental Reserve Facility ApenXIdxi

In recent financial crises, the International Monetary Fund (IMF or the 

Fund) provided large short-term loans under the Supplemental Reserve 

Facility (SRF) with high interest rates to member countries experiencing 

exceptional balance of payments problems. These problems resulted from 

a sharp decline of investor confidence and significant outflows of capital. 

These loans generally were provided when the countries had exceeded 

their financing limit under other loan mechanisms, including the Stand-By 

Arrangement (SBA). In some circumstances, such as Argentina and 

Uruguay, the Fund provided a mix of SRF and SBA loans. Table 6 lists Fund 

members receiving SRF loans from 1997 to 2002.

Table 6: IMF Supplemental Reserve Facility Loans, 1997–2002

Source: IMF.

Country Loan approval date Loan mix

SRF repaid as of May 

2003

Korea 12/18/97 SRF Yes

Russia 7/20/98 SRF Yes

Brazil 12/2/98 SRF Yes

Turkey 12/21/00 SRF Yes

Argentina 1/12/01

SRF (40%) 

SBA (60%) No

Argentina 9/7/01

SRF (63%) 

SBA (37%) No

Brazil 9/14/01 SRF No

Uruguay 6/25/02

SRF (33%)

SBA (67%) No

Brazil 9/6/02

SRF (33%)

SBA (67%) No
Page 65 GAO-03-734 International Financial Crises

Appendix X

Comments from the Department of the 

Treasury ApenXdxi
Appendix X

Comments from the Department of the 

Treasury

Page 66 GAO-03-734 International Financial Crises
Page 67 GAO-03-734 International Financial Crises

Appendix XI

Comments from the International Monetary 

Fund ApenXdxiI

Note: GAO comments 

supplementing those in 

the report text appear 

at the end of this 

appendix.

See comment 1.

See comment 2.
Appendix XI

Comments from the International Monetary 

Fund

Page 68 GAO-03-734 International Financial Crises

See comment 3.

See comment 4.

See comment 5.

See comment 6.
Appendix XI

Comments from the International Monetary 

Fund

Page 69 GAO-03-734 International Financial Crises
Appendix XI

Comments from the International Monetary 

Fund

Page 70 GAO-03-734 International Financial Crises

The following are GAO’s comments on the letter from the International 

Monetary Fund, dated June 2, 2003. 

GAO Comments 1. Our assessment of IMF’s efforts to anticipate financial crises did not 

focus exclusively on the World Economic Outlook (WEO) and the Early 

Warning System (EWS) models. We examined all six components of 

the IMF’s vulnerability assessment framework. However, as the only 

mature and quantifiable elements of the framework, our analysis 

focused more heavily on the track records of the WEO and EWS. As we 

reported, these elements have not performed well at anticipating prior 

crises. While we acknowledge that the new framework is more 

comprehensive than past efforts at anticipating crises, it is too early to 

assess whether this framework will be successful in anticipating future 

crises. 

2. The IMF’s comment supports our finding. The IMF states that its 

responsibility to maintain financial stability could make its predictions 

appear less accurate since an accurate prediction of crises within WEO 

forecasts would be irresponsible and could lead to a crisis. In effect, 

the IMF acknowledges that their forecasts are overly optimistic and 

validates our finding on the weakness of the WEO component of the 

vulnerability assessment framework. This raises questions regarding 

the purpose and credibility of the WEO forecasts. 

3. We state in the report that the IMF’s new vulnerability assessment 

framework, which includes the examination of external reserve 

adequacy and the strengths and weaknesses of banking systems, is 

more comprehensive than its previous efforts to identify countries at 

risk of crisis. However, it is too early to assess whether this framework 

will be successful in anticipating future crises.

4. We recognize that the EWS models are just one of six components of 

the IMF’s vulnerability assessment framework. However, the IMF’s 

own internal review of the EWS models concluded that the false alarm 

rate was too high.

5. The report recognizes that the FSAP and ROSC assessments constitute 

a valuable source of information about vulnerabilities in member 

countries and states that IMF staff use these assessments to formulate 

policy recommendations. The report also recognizes that the IMF often 

provides technical assistance to help member countries build their 
Appendix XI

Comments from the International Monetary 

Fund

Page 71 GAO-03-734 International Financial Crises

capacity to implement FSAP and ROSC recommendations. However, 

the report points out several factors that limit the usefulness of FSAP 

and ROSC assessments. Our recommendation, with which the IMF 

agrees, is designed to improve the timeliness and coverage of these 

assessments. 

6. We based our description of IMF safeguards assessments on the IMF’s 

reviews of this program. We consider this topic to be within the scope 

of this evaluation because the framework for conducting safeguards 

assessments is derived from the IMF’s Code of Good Practices on 

Transparency in Monetary and Financial Policies. Safeguards 

assessments are thus related to the standards initiative, which 

constitutes a central element of this report.
Page 72 GAO-03-734 International Financial Crises

Appendix XII

Comments from The World Bank ApenXdxiI

Note: GAO comments 

supplementing those in 

the report text appear 

at the end of this 

appendix.

See comment 1.

See comment 2.
Appendix XII

Comments from The World Bank

Page 73 GAO-03-734 International Financial Crises
Appendix XII

Comments from The World Bank

Page 74 GAO-03-734 International Financial Crises

The following are GAO’s comments on the letter from the World Bank, 

dated June 2, 2003. 

GAO Comments 1. The report states unambiguously that crises can stem from a number of 

factors, some of which are outside the scope of the FSAP and ROSC 

assessments. However, there is broad agreement that the roots of the 

Mexican and Asian financial crises lay in weaknesses in financial 

systems and other institutions. The IMF and the World Bank based 

their decision to launch the FSAP and ROSC initiatives on the premise 

that timely identification of financial sector and institutional 

vulnerabilities can contribute to crisis prevention. The IMF and the 

World Bank have also acknowledged that FSAP and ROSC assessments 

can contribute to crisis prevention efforts by helping private sector 

participants make better informed investment decisions. 

2. Our recommendation to pursue strategies to increase participation in 

the FSAP and ROSC assessments, including the possibility of making 

these assessments mandatory, stems from the IMF’s and the World 

Bank’s recognition of the need to prioritize participation by important 

emerging market countries. Although many of these countries have 

volunteered to participate in these assessments, others have not. While 

we are not suggesting that the assessments should be made mandatory, 

it is evident that the voluntary nature of the FSAP and ROSC has posed 

an obstacle to full participation by important emerging market 

countries. 
Page 75 GAO-03-734 International Financial Crises

Appendix XIII

GAO Contacts and Staff Acknowledgments ApenXdxiI

GAO Contacts Tom Melito (202) 512-9601

Zina Merritt (202) 512-5257

Staff 

Acknowledgments

In addition to those individuals named above, Eric Clemons, Suzanne Dove, 

Bruce Kutnick, Jonathan Rose, R.G. Steinman, Ian Ferguson, Mary 

Moutsos, Lynn Cothern, Carl Barden, David Dornisch, and Martin De 

Alteriis made key contributions to this report. 

(320114)
GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal government 

for the American people. GAO examines the use of public funds; evaluates federal 

programs and policies; and provides analyses, recommendations, and other 

assistance to help Congress make informed oversight, policy, and funding 

decisions. GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail this 

list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO 

Mailing Lists” under “Order GAO Products” heading.

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Service Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00