Millennium Challenge Corporation: Projected Impact of Vanuatu Compact Is Overstated
Published: 2007-07-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In January 2004, Congress established the Millennium Challenge Corporation (MCC) for foreign assistance. Congress has appropriated almost $6 billion to MCC. As of March 2007, MCC had signed almost $3 billion in compacts with 11 countries, including a 5-year, $65.7 million compact with Vanuatu. MCC states that the Vanuatu compact will have a transformational effect on the country's economy, increasing per capita income and GDP and benefiting 65,000 poor, rural people. This testimony summarizes a July 2007 report (GAO-07-909) examining (1) MCC's methods of projecting economic benefits, (2) MCC's portrayal and analysis of the projected benefits, and (3) risks that may affect the compact's impact. To address these objectives, GAO reviewed MCC's analyses and met with officials and business owners in Vanuatu as well as with other donors. In its July 2007 report, GAO recommended that the Chief Executive Officer of MCC revise the public reporting of the Vanuatu compact's projected impact; assess whether similar reporting in other compacts accurately reflects underlying analyses; and improve its economic analyses by more fully accounting for risks to project benefits. MCC did not directly address GAO's recommendations but commented that it had not intended to make misleading statements and that its portrayal of projected results was factual and consistent with underlying data.
section_title: What GAO Found
paragraphs: MCC projects that the Vanuatu compact's transportation infrastructure projects will provide direct benefits such as reduced transportation costs and induced benefits from growth in tourism and agriculture. MCC estimated the costs and benefits over 20 years, with benefits beginning in full in 2008 or 2009 and growing each year, and it counted poor, rural beneficiaries by defining the area where benefits were likely to accrue. Using projected benefits and costs, MCC calculated the compact's economic rate of return (ERR) and its effects on Vanuatu's gross domestic product (GDP) and per capita income. MCC's portrayal of the projected impact does not reflect its underlying data. MCC states that per capita income will increase by approximately $200, or 15 percent, by 2010 and by $488, or 37 percent, by 2015. However, MCC's underlying data show that these figures represent the sum of individual years' gains in per capita income relative to 2005 and that actual gains will be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent, in 2015. MCC also states that GDP will increase by an additional 3 percent a year, but its data show that after GDP growth of 6 percent in 2007, the economy's growth will continue at about 3 percent, as it would without the compact. MCC states that the compact will benefit approximately 65,000 poor, rural inhabitants, but this statement does not identify the financial benefits that accrue to the rural poor or reflect its own analysis that 57 percent of benefits go to others.We identified five key risks that could affect the compact's projected impacts. (1) Cost estimate contingencies may not be sufficient to cover project overruns. (2) Compact benefits will likely accrue more slowly than MCC projected. (3) Benefit estimates assume continued maintenance, but MCC's ability to ensure maintenance will end in 2011, and Vanuatu's maintenance record is poor. (4) Induced benefits depend on businesses' and residents' response to new opportunities. (5) Efficiency gains, such as time saved in transit, may not increase per capita income. Our analysis of these areas of risk illustrates the extent that MCC's projections are dependent on assumptions of immediate realization of benefits, long-term maintenance, realization of induced benefits, and benefits from efficiency gains.

section_title: Letter
paragraphs: Thank you for the opportunity to discuss our recent work regarding the  Millennium Challenge Corporation’s (MCC) compact with Vanuatu.
In January 2004, Congress established MCC to administer the Millennium  Challenge Account for foreign assistance. MCC’s mission is to reduce  poverty by supporting sustainable, transformative economic growth in  developing countries that have demonstrated a commitment to ruling  justly and democratically, encouraging economic freedom, and investing  in people. Congress appropriated almost $6 billion to MCC for fiscal years  2004 through 2007, and the President has requested an additional $3 billion  in MCC funding for fiscal year 2008. As of March 2007, MCC had signed 11  compacts totaling approximately $3 billion. MCC’s 5-year, $65.7 million  compact with Vanuatu focuses on increasing economic activity and  incomes in rural areas through investments in transportation  infrastructure. Although MCC’s Vanuatu compact is its smallest compact  monetarily, it provides by far the largest amount relative to the country’s  population and gross domestic product (GDP).
Publicly available documents show that MCC expects its compacts to  significantly benefit the countries’ economies. In its Vanuatu compact and  its March 2006 congressional notification, MCC states that it expects the  compact to have a “transformational” impact—that is, as MCC defines it,  “a dramatic and long-lasting impact on poverty reduction through  sustainable economic growth.” Using its projected benefit and cost data,  MCC calculated of the compact’s expected economic rate of return (ERR)  and impact on poverty reduction and economic growth. MCC states that  its compacts will provide or contribute to a transformational impact in 5 of  its 11 compacts.
In my testimony today, I will address (1) MCC’s methods of projecting and  calculating the Vanuatu compact’s impact on poverty reduction and  economic growth, (2) MCC’s portrayal and analysis of the Vanuatu  compact’s projected impact, and (3) risks that could affect the Vanuatu  compact’s actual impact. This statement summarizes the findings in our  report released today.
In our report, we addressed our first and second objectives by evaluating  MCC’s economic analysis of the Vanuatu compact proposal and MCC’s  public statements about the compact’s impacts. We could not validate  most of MCC’s underlying data and assumptions, because the data were  not available or could not be checked within the time frames of our  engagement. To address our third objective, we identified risks to MCC’s  compact results, based on our review of MCC’s internal documentation,  donor reporting, and academic literature. To illustrate the impact of these  risks on MCC’s economic analyses of ERR, GDP, and per capita income,  we modeled the risks using the data from MCC’s economic analyses;  however, we did not validate these data. We focused our analysis and field  work on MCC’s three transportation infrastructure projects on Vanuatu’s  two most populous islands, Santo and Efate, which represent 56 percent of  compact cost. We interviewed Vanuatu and MCC officials and interested  parties such as tourism and agriculture business owners and contacted  MCC’s contractor. We conducted this work from August 2006 through May  2007 in accordance with generally accepted government auditing  standards.
subsections: section_title: Background
paragraphs: Vanuatu consists of 83 islands spread over hundreds of miles of ocean in  the South Pacific, 1,300 miles northeast of Sydney, Australia. About 39  percent of the population is concentrated on the islands of Santo and  Efate. Vanuatu’s capital, Port Vila, is on Efate, and Vanuatu’s only other  urban center, Luganville, is on Santo.
In the past decade, Vanuatu’s real GDP growth averaged 2 percent,  although more rapid population growth led to a decline in per capita GDP  over the same period. Average growth of real GDP per capita was negative  from 1993 to 2005. An estimated 40 percent of Vanuatu’s population of  about 207,000 has an income below the international poverty line of $1 per  day. Agriculture and tourism are the principal productive sectors of  Vanuatu’s economy, contributing approximately 15 percent and 19 percent  to GDP, respectively. Although agriculture represents a relatively small  share of Vanuatu’s overall economy, approximately 80 percent of  Vanuatu’s residents live in rural areas and depend on subsistence  agriculture for food and shelter. The tourism sector is dominated by  expatriates of foreign countries living in Vanuatu, who also predominate in  other formal sectors of the economy such as plantation agriculture and  retail trade.
On May 6, 2004, MCC determined that Vanuatu was eligible to submit a  compact proposal for Millennium Challenge Account funding. Vanuatu’s  proposal identified transportation infrastructure as a key constraint to  private-sector development. The timeline in figure 1 shows the  development and implementation of the Vanuatu proposal and compact.
The $65.7 million Vanuatu compact includes $54.5 million for the  rehabilitation or construction of 11 transportation infrastructure assets on  8 of Vanuatu’s 83 islands, including roads, wharves, an airstrip, and  warehouses (see fig. 2). The compact also includes $6.2 million for an  institutional strengthening program to increase the capacity of the  Vanuatu Public Works Department (PWD) to maintain transportation  infrastructure. The remaining $5 million is for program management and  monitoring and evaluation. More than half of the compact, $37 million, is  budgeted for three road projects on Santo and Efate islands. The compact  provides for upgrading existing roads on both islands; the compact also  includes five new bridges for an existing road on Santo.
MCC’s compact with Vanuatu and congressional notification state that the  compact will have a transformational impact on Vanuatu’s economic  development, increasing average per capita income by approximately  $200—15 percent—by 2010 and increasing total GDP by “an additional 3  percent a year.” MCC’s investment memo further quantifies the per capita  income increase as $488—37 percent—by 2015. The compact and the  congressional notification also state that the compact will provide benefits  to approximately 65,000 poor, rural inhabitants (see fig. 3).
section_title: MCC Projected Compact’s Impact Using Estimates of Benefits, Costs, and Catchment Area
paragraphs: In projecting the impact of the Vanuatu compact, MCC estimated the  benefits and costs of the proposed infrastructure improvements. MCC also  estimated the number of beneficiaries within a defined catchment area— that is, the geographic area in which benefits may be expected to accrue.  MCC used the estimated benefits and costs to calculate the compact’s ERR  and impact on Vanuatu’s GDP and per capita income.
MCC’s analysis determined that the compact will reduce transportation  costs and improve the reliability of access to transportation services for  poor, rural agricultural producers and providers of tourism-related goods  and services and that these benefits will, in turn, lead to increases in per  capita income and GDP and reduction in poverty. MCC projects several  direct and induced benefits from the compact’s infrastructure  improvement projects over a 20-year period, beginning in full in 2008 or  2009 and increasing by at least 3 percent every year.
Direct benefits. MCC projects that direct benefits will include, for  example, construction spending, reduced transportation costs, and time  saved in transit on the improved roads.
Induced benefits. MCC projects that induced benefits from tourism and  agriculture will include, for example, increased growth in Vanuatu  tourism, tourist spending, and hotel occupancy and increased crop,  livestock, and fisheries production.
Figure 4 illustrates MCC’s logic in projecting the compact’s impact.
MCC expects compact benefits to flow from different sources, depending  on the project and its location. In Efate, the Ring Road is expected to  provide direct benefits from decreased road user costs and induced  benefits through tourism and foreign resident spending. In Santo, MCC  anticipates similar benefits as well as the induced benefit of increased  agricultural production. On other islands, where tourism is not as  developed, MCC expects benefits to derive primarily from user cost  savings and increased agriculture.
To calculate construction and maintenance costs for the transportation  infrastructure projects, MCC used existing cost estimates prepared for the  government of Vanuatu and for another donor as well as data from the  Vanuatu PWD.
To estimate the number of poor, rural beneficiaries, MCC used Vanuatu  maps to identify villages in the catchment area and used the 1999 Vanuatu  National Population and Housing Census to determine the number of  persons living in those villages. In all, MCC calculated that approximately  65,000 poor, rural people on the eight islands would benefit from MCC  projects.
On the basis of the costs and benefits projected over a 20-year period,  MCC calculated three summaries of the compact’s impact: its ERR, effect  on per capita income, and effect on GDP. MCC projected an overall  compact ERR of 24.7 percent over 20 years. In projecting the compact’s  impact on Vanuatu’s per capita income, MCC used a baseline per capita  income of $1,326 for 2005.
MCC also prepared a sensitivity analysis to assess how a range of possible  outcomes would affect compact results. MCC’s tests included a 1-year  delay of the start date for accrued benefits; a 20 percent increase of all  costs; a 20 percent decrease of all benefits; and a “stress test,” with a 20  percent increase of all costs and a 20 percent decrease of all benefits. MCC  calculated a best-case compact ERR of 30.2 percent and a worst-case  compact ERR of 13.9 percent.
section_title: MCC’s Data Do Not Support Its Portrayal of Compact Benefits
paragraphs: MCC’s public portrayal of the Vanuatu compact’s projected effects on per  capita income and on GDP suggest greater impact than its analysis  supports. In addition, MCC’s portrayal of the compact’s projected impact  on poverty does not identify the proportion of benefits that will accrue to  the rural poor.
Impact on per capita income. In the compact and the congressional  notification, MCC states that the transportation infrastructure project is  expected to increase “average income per capita (in real terms) by  approximately $200, or 15 percent of current income per capita, by 2010.”  MCC’s investment memo states that the compact will cause per capita  income to increase by $488, or 37 percent, by 2015. These statements  suggest that as a result of the program, average incomes in Vanuatu will be  15 percent higher in 2010 and 37 percent higher in 2015 than they would be  without the compact. However, MCC’s underlying data show that these  percentages represent the sum of increases from per capita income in 2005  that MCC projects for each year. For example, according to MCC’s data,  Vanuatu’s per capita income in a given year between 2006 and 2010 will  range from about 2 percent to almost 4 percent higher than in 2005; in its  statements, MCC sums these percentages as 15 percent without stating  that this percentage is a cumulative increase from 2005. Our analysis of  MCC’s data shows that actual gains in per capita income, relative to  income in 2005, would be $51, or 3.9 percent, in 2010 and $61, or 4.6  percent, in 2015 (see fig. 5).
Figure 6 further illustrates MCC’s methodology in projecting the compact’s  impact on per capita income levels for 2010 and 2015.
Impact on GDP. Like its portrayal of the projected impact on per capita  income, MCC’s portrayal of the projected impact on GDP is not supported  by the underlying data. In the compact and the 2006 congressional  notification, MCC states that the compact will have a transformational  effect on Vanuatu’s economy, causing GDP to “increase by an additional 3  percent a year.” Given the GDP growth rate of about 3 percent that MCC  expects in Vanuatu without the compact, MCC’s statement of a  transformational effect suggests that the GDP growth rate will rise to  about 6 percent. However, MCC’s underlying data show that although  Vanuatu’s GDP growth rate will rise to about 6 percent in 2007, in  subsequent years the GDP growth rate will revert to roughly the rate MCC  assumes would occur without the compact, approximately 3 percent (see  fig. 7). Although MCC’s data show that the compact will result in a higher  level (i.e., dollar value) of GDP, the data do not show a transformational  increase to the GDP growth rate.
Impact on poverty. MCC’s portrayal of the compact’s projected impact on  poverty does not identify the proportion of the financial benefits that will  accrue to the rural poor. In the compact and the congressional  notification, MCC states that the program is expected to benefit  “approximately 65,000 poor, rural inhabitants living nearby and using the  roads to access markets and social services.” In its underlying  documentation, MCC expects 57 percent of the monetary benefits to  accrue to other beneficiaries, including expatriate tourism services  providers, transport providers, government, and local businesses; 43  percent is expected to go to the local population, which MCC defines as  “local producers, local consumers and inhabitants of remote communities”  (see fig. 8). However, MCC does not establish the proportion of local- population benefits that will go to the 65,000 poor, rural beneficiaries.
section_title: Several Risks May Lead to Reduced Project Benefits
paragraphs: Our analysis shows that risks related to construction costs, timing of  benefits, project maintenance, induced benefits, and efficiency gains may  lessen the Vanuatu compact’s projected impact on poverty reduction and  economic growth. Accounting for these risks could reduce the overall  compact ERR.
Construction costs. Although MCC considered the risk of construction  cost increases, the contingencies used in its calculations may not be  sufficient to cover actual construction costs. Cost estimate documentation  for 5 of MCC’s 11 construction projects shows that these estimates include  design contingencies of 20 percent. However, cost overruns of more than  20 percent occur in many transportation projects, and as MCC’s analysis  notes, the risk of excessive cost overruns is significant in a small country  such as Vanuatu. Any construction cost overrun must be made up within  the Vanuatu compact budget by reducing the scope, and therefore the  benefits, of the compact projects; reduced project benefits would in turn  reduce the compact’s ERR and effects on per capita income and GDP.
Timing of benefits. Although MCC’s analysis assumes compact benefits  from 2008 or 2009—shortly after the end of project construction—we  found that benefits are likely to accrue more slowly. Our document review  and discussions with tourism services providers and agricultural and  timber producers suggest that these businesses will likely react gradually  to any increased market opportunities resulting from MCC’s projects, in  part because of constraints to expanding economic activity. In addition,  MCC assumes that all construction spending will occur in the first year,  instead of phasing the benefits from this spending over the multiyear  construction schedule.
Project maintenance. Uncertainty about the maintenance of completed  transportation infrastructure projects after 2011 may affect the compact’s  projected benefits. Vanuatu’s record of road maintenance is poor.  According to World Bank and Asian Development Bank officials,  continuing donor involvement is needed to ensure the maintenance and  sustainability of completed projects. However, although MCC has  budgeted $6.2 million for institutional strengthening of the Vanuatu PWD,  MCC has no means of ensuring the maintenance of completed projects  after the compact expires in 2011; the Millennium Challenge Act limits  compacts to 5 years. Poor maintenance performance will reduce the  benefits projected in the MCC compact.
Induced benefits. The compact’s induced benefits depend on the response  of Vanuatu tourism providers and agricultural producers. However,  constraints affecting these economic sectors may prevent the sector from  expanding as MCC projects. Limited response to the compact by tourism  providers and agricultural producers would have a significant impact on  compact benefits.
Efficiency gains. MCC counts efficiency gains—such as time saved  because of better roads—as compact benefits. However, although  efficiency gains could improve social welfare, they may not lead to  changes in per capita income or GDP or be directly measurable as net  additions to the economy.
Accounting for these risks could reduce the overall compact ERR from  24.2 percent, as projected by MCC, to between 5.5 percent and 16.5  percent (see table 1).
section_title: Conclusions
paragraphs: MCC’s public portrayal of the Vanuatu compact’s projected benefits— particularly the effect on per capita income—suggests a greater impact  than MCC’s underlying data and analysis support and can be understood  only by reviewing source documents and spreadsheets that are not  publicly available. As a result, MCC’s statements may foster unrealistic  expectations of the compact’s impact in Vanuatu. For example, by  suggesting that per capita incomes will increase so quickly, MCC suggests  that its compact will produce sustainable growth that other donors to  Vanuatu have not been able to achieve. The gaps between MCC’s  statements about, and underlying analysis of, the Vanuatu compact also  raise questions about other MCC compacts’ projections of a  transformational impact on country economies or economic sectors.  Without accurate portrayals of its compacts’ projected benefits, the extent  to which MCC’s compacts are likely to further its goals of poverty  reduction and economic growth cannot be accurately evaluated. In  addition, the economic analysis underlying MCC’s statements does not  reflect the time required to improve Vanuatu’s transportation  infrastructure and for the economy to respond and does not fully account  for other risks that could substantially reduce compact benefits.
section_title: Recommendations
paragraphs: In our report, we recommend that the CEO of MCC take the following  actions:  revise the public reporting of the Vanuatu compact’s projected impact to  clearly represent the underlying data and analysis;   assess whether similar statements in other compacts accurately reflect the  underlying data and analysis; and improve its economic analysis by phasing the costs and benefits in  compact ERR calculations and by more fully accounting for risks such as  those related to continuing maintenance, induced benefits, and monetized  efficiency gains as part of sensitivity analysis.
In comments on a draft of our report, MCC did not directly acknowledge  our recommendations. MCC acknowledged that its use of projected  cumulative compact impact on income and growth was misleading but  asserted that it had no intention to mislead and that its portrayal of  projected compact benefits was factually correct. MCC questioned our  finding that its underlying data and analysis do not support its portrayal of  compact benefits and our characterization of the program’s risks. (See  app. VI of our report for MCC comments and our response.)
Mr. Chairman, this completes my prepared statement. I would be happy to  respond to any questions you or other Members of the Subcommittee may  have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For further information about this testimony, please contact me at (202)  512-3149 or gootnickd@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. In addition to the person named above, Emil Friberg, Jr.  (Assistant Director), Gergana Danailova-Trainor, Reid Lowe, Angie  Nichols-Friedman, Michael Simon, and Seyda Wentworth made key   contributions to this statement. Also, David Dornisch, Etana Finkler, Ernie  Jackson, and Tom McCool provided technical assistance.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.