Millennium Challenge Corporation: Projected Impact of Vanuatu Compact Is Overstated
Released: 2007-07-26T00:00:00

Letter

Background

Vanuatu consists of 83 islands spread over hundreds of miles of ocean in the South Pacific, 1,300 miles northeast of Sydney, Australia. About 39 percent of the population is concentrated on the islands of Santo and Efate. Vanuatu’s capital, Port Vila, is on Efate, and Vanuatu’s only other urban center, Luganville, is on Santo.

In the past decade, Vanuatu’s real GDP growth averaged 2 percent, although more rapid population growth led to a decline in per capita GDP over the same period. Average growth of real GDP per capita was negative from 1993 to 2005. An estimated 40 percent of Vanuatu’s population of about 207,000 has an income below the international poverty line of $1 per day. Agriculture and tourism are the principal productive sectors of Vanuatu’s economy, contributing approximately 15 percent and 19 percent to GDP, respectively. Although agriculture represents a relatively small share of Vanuatu’s overall economy, approximately 80 percent of Vanuatu’s residents live in rural areas and depend on subsistence agriculture for food and shelter. The tourism sector is dominated by expatriates of foreign countries living in Vanuatu, who also predominate in other formal sectors of the economy such as plantation agriculture and retail trade.

On May 6, 2004, MCC determined that Vanuatu was eligible to submit a compact proposal for Millennium Challenge Account funding. Vanuatu’s proposal identified transportation infrastructure as a key constraint to private-sector development. The timeline in figure 1 shows the development and implementation of the Vanuatu proposal and compact.

The $65.7 million Vanuatu compact includes $54.5 million for the rehabilitation or construction of 11 transportation infrastructure assets on 8 of Vanuatu’s 83 islands, including roads, wharves, an airstrip, and warehouses (see fig. 2). The compact also includes $6.2 million for an institutional strengthening program to increase the capacity of the Vanuatu Public Works Department (PWD) to maintain transportation infrastructure. The remaining $5 million is for program management and monitoring and evaluation. More than half of the compact, $37 million, is budgeted for three road projects on Santo and Efate islands. The compact provides for upgrading existing roads on both islands; the compact also includes five new bridges for an existing road on Santo.

MCC’s compact with Vanuatu and congressional notification state that the compact will have a transformational impact on Vanuatu’s economic development, increasing average per capita income by approximately $200—15 percent—by 2010 and increasing total GDP by “an additional 3 percent a year.” MCC’s investment memo further quantifies the per capita income increase as $488—37 percent—by 2015. The compact and the congressional notification also state that the compact will provide benefits to approximately 65,000 poor, rural inhabitants (see fig. 3).

MCC Projected Compact’s Impact Using Estimates of Benefits, Costs, and Catchment Area

In projecting the impact of the Vanuatu compact, MCC estimated the benefits and costs of the proposed infrastructure improvements. MCC also estimated the number of beneficiaries within a defined catchment area— that is, the geographic area in which benefits may be expected to accrue. MCC used the estimated benefits and costs to calculate the compact’s ERR and impact on Vanuatu’s GDP and per capita income.

MCC’s analysis determined that the compact will reduce transportation costs and improve the reliability of access to transportation services for poor, rural agricultural producers and providers of tourism-related goods and services and that these benefits will, in turn, lead to increases in per capita income and GDP and reduction in poverty. MCC projects several direct and induced benefits from the compact’s infrastructure improvement projects over a 20-year period, beginning in full in 2008 or 2009 and increasing by at least 3 percent every year.

Direct benefits. MCC projects that direct benefits will include, for example, construction spending, reduced transportation costs, and time saved in transit on the improved roads.

Induced benefits. MCC projects that induced benefits from tourism and agriculture will include, for example, increased growth in Vanuatu tourism, tourist spending, and hotel occupancy and increased crop, livestock, and fisheries production.

Figure 4 illustrates MCC’s logic in projecting the compact’s impact.

MCC expects compact benefits to flow from different sources, depending on the project and its location. In Efate, the Ring Road is expected to provide direct benefits from decreased road user costs and induced benefits through tourism and foreign resident spending. In Santo, MCC anticipates similar benefits as well as the induced benefit of increased agricultural production. On other islands, where tourism is not as developed, MCC expects benefits to derive primarily from user cost savings and increased agriculture.

To calculate construction and maintenance costs for the transportation infrastructure projects, MCC used existing cost estimates prepared for the government of Vanuatu and for another donor as well as data from the Vanuatu PWD.

To estimate the number of poor, rural beneficiaries, MCC used Vanuatu maps to identify villages in the catchment area and used the 1999 Vanuatu National Population and Housing Census to determine the number of persons living in those villages. In all, MCC calculated that approximately 65,000 poor, rural people on the eight islands would benefit from MCC projects.

On the basis of the costs and benefits projected over a 20-year period, MCC calculated three summaries of the compact’s impact: its ERR, effect on per capita income, and effect on GDP. MCC projected an overall compact ERR of 24.7 percent over 20 years. In projecting the compact’s impact on Vanuatu’s per capita income, MCC used a baseline per capita income of $1,326 for 2005.

MCC also prepared a sensitivity analysis to assess how a range of possible outcomes would affect compact results. MCC’s tests included a 1-year delay of the start date for accrued benefits; a 20 percent increase of all costs; a 20 percent decrease of all benefits; and a “stress test,” with a 20 percent increase of all costs and a 20 percent decrease of all benefits. MCC calculated a best-case compact ERR of 30.2 percent and a worst-case compact ERR of 13.9 percent.

MCC’s Data Do Not Support Its Portrayal of Compact Benefits

MCC’s public portrayal of the Vanuatu compact’s projected effects on per capita income and on GDP suggest greater impact than its analysis supports. In addition, MCC’s portrayal of the compact’s projected impact on poverty does not identify the proportion of benefits that will accrue to the rural poor.

Impact on per capita income. In the compact and the congressional notification, MCC states that the transportation infrastructure project is expected to increase “average income per capita (in real terms) by approximately $200, or 15 percent of current income per capita, by 2010.” MCC’s investment memo states that the compact will cause per capita income to increase by $488, or 37 percent, by 2015. These statements suggest that as a result of the program, average incomes in Vanuatu will be 15 percent higher in 2010 and 37 percent higher in 2015 than they would be without the compact. However, MCC’s underlying data show that these percentages represent the sum of increases from per capita income in 2005 that MCC projects for each year. For example, according to MCC’s data, Vanuatu’s per capita income in a given year between 2006 and 2010 will range from about 2 percent to almost 4 percent higher than in 2005; in its statements, MCC sums these percentages as 15 percent without stating that this percentage is a cumulative increase from 2005. Our analysis of MCC’s data shows that actual gains in per capita income, relative to income in 2005, would be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent, in 2015 (see fig. 5).

Figure 6 further illustrates MCC’s methodology in projecting the compact’s impact on per capita income levels for 2010 and 2015.

Impact on GDP. Like its portrayal of the projected impact on per capita income, MCC’s portrayal of the projected impact on GDP is not supported by the underlying data. In the compact and the 2006 congressional notification, MCC states that the compact will have a transformational effect on Vanuatu’s economy, causing GDP to “increase by an additional 3 percent a year.” Given the GDP growth rate of about 3 percent that MCC expects in Vanuatu without the compact, MCC’s statement of a transformational effect suggests that the GDP growth rate will rise to about 6 percent. However, MCC’s underlying data show that although Vanuatu’s GDP growth rate will rise to about 6 percent in 2007, in subsequent years the GDP growth rate will revert to roughly the rate MCC assumes would occur without the compact, approximately 3 percent (see fig. 7). Although MCC’s data show that the compact will result in a higher level (i.e., dollar value) of GDP, the data do not show a transformational increase to the GDP growth rate.

Impact on poverty. MCC’s portrayal of the compact’s projected impact on poverty does not identify the proportion of the financial benefits that will accrue to the rural poor. In the compact and the congressional notification, MCC states that the program is expected to benefit “approximately 65,000 poor, rural inhabitants living nearby and using the roads to access markets and social services.” In its underlying documentation, MCC expects 57 percent of the monetary benefits to accrue to other beneficiaries, including expatriate tourism services providers, transport providers, government, and local businesses; 43 percent is expected to go to the local population, which MCC defines as “local producers, local consumers and inhabitants of remote communities” (see fig. 8). However, MCC does not establish the proportion of local- population benefits that will go to the 65,000 poor, rural beneficiaries.

Several Risks May Lead to Reduced Project Benefits

Our analysis shows that risks related to construction costs, timing of benefits, project maintenance, induced benefits, and efficiency gains may lessen the Vanuatu compact’s projected impact on poverty reduction and economic growth. Accounting for these risks could reduce the overall compact ERR.

Construction costs. Although MCC considered the risk of construction cost increases, the contingencies used in its calculations may not be sufficient to cover actual construction costs. Cost estimate documentation for 5 of MCC’s 11 construction projects shows that these estimates include design contingencies of 20 percent. However, cost overruns of more than 20 percent occur in many transportation projects, and as MCC’s analysis notes, the risk of excessive cost overruns is significant in a small country such as Vanuatu. Any construction cost overrun must be made up within the Vanuatu compact budget by reducing the scope, and therefore the benefits, of the compact projects; reduced project benefits would in turn reduce the compact’s ERR and effects on per capita income and GDP.

Timing of benefits. Although MCC’s analysis assumes compact benefits from 2008 or 2009—shortly after the end of project construction—we found that benefits are likely to accrue more slowly. Our document review and discussions with tourism services providers and agricultural and timber producers suggest that these businesses will likely react gradually to any increased market opportunities resulting from MCC’s projects, in part because of constraints to expanding economic activity. In addition, MCC assumes that all construction spending will occur in the first year, instead of phasing the benefits from this spending over the multiyear construction schedule.

Project maintenance. Uncertainty about the maintenance of completed transportation infrastructure projects after 2011 may affect the compact’s projected benefits. Vanuatu’s record of road maintenance is poor. According to World Bank and Asian Development Bank officials, continuing donor involvement is needed to ensure the maintenance and sustainability of completed projects. However, although MCC has budgeted $6.2 million for institutional strengthening of the Vanuatu PWD, MCC has no means of ensuring the maintenance of completed projects after the compact expires in 2011; the Millennium Challenge Act limits compacts to 5 years. Poor maintenance performance will reduce the benefits projected in the MCC compact.

Induced benefits. The compact’s induced benefits depend on the response of Vanuatu tourism providers and agricultural producers. However, constraints affecting these economic sectors may prevent the sector from expanding as MCC projects. Limited response to the compact by tourism providers and agricultural producers would have a significant impact on compact benefits.

Efficiency gains. MCC counts efficiency gains—such as time saved because of better roads—as compact benefits. However, although efficiency gains could improve social welfare, they may not lead to changes in per capita income or GDP or be directly measurable as net additions to the economy.

Accounting for these risks could reduce the overall compact ERR from 24.2 percent, as projected by MCC, to between 5.5 percent and 16.5 percent (see table 1).

Conclusions

MCC’s public portrayal of the Vanuatu compact’s projected benefits— particularly the effect on per capita income—suggests a greater impact than MCC’s underlying data and analysis support and can be understood only by reviewing source documents and spreadsheets that are not publicly available. As a result, MCC’s statements may foster unrealistic expectations of the compact’s impact in Vanuatu. For example, by suggesting that per capita incomes will increase so quickly, MCC suggests that its compact will produce sustainable growth that other donors to Vanuatu have not been able to achieve. The gaps between MCC’s statements about, and underlying analysis of, the Vanuatu compact also raise questions about other MCC compacts’ projections of a transformational impact on country economies or economic sectors. Without accurate portrayals of its compacts’ projected benefits, the extent to which MCC’s compacts are likely to further its goals of poverty reduction and economic growth cannot be accurately evaluated. In addition, the economic analysis underlying MCC’s statements does not reflect the time required to improve Vanuatu’s transportation infrastructure and for the economy to respond and does not fully account for other risks that could substantially reduce compact benefits.

Recommendations

In our report, we recommend that the CEO of MCC take the following actions: revise the public reporting of the Vanuatu compact’s projected impact to clearly represent the underlying data and analysis; assess whether similar statements in other compacts accurately reflect the underlying data and analysis; and improve its economic analysis by phasing the costs and benefits in compact ERR calculations and by more fully accounting for risks such as those related to continuing maintenance, induced benefits, and monetized efficiency gains as part of sensitivity analysis.

In comments on a draft of our report, MCC did not directly acknowledge our recommendations. MCC acknowledged that its use of projected cumulative compact impact on income and growth was misleading but asserted that it had no intention to mislead and that its portrayal of projected compact benefits was factually correct. MCC questioned our finding that its underlying data and analysis do not support its portrayal of compact benefits and our characterization of the program’s risks. (See app. VI of our report for MCC comments and our response.)

Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions you or other Members of the Subcommittee may have at this time.

GAO Contact and Staff Acknowledgments

For further information about this testimony, please contact me at (202) 512-3149 or gootnickd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. In addition to the person named above, Emil Friberg, Jr. (Assistant Director), Gergana Danailova-Trainor, Reid Lowe, Angie Nichols-Friedman, Michael Simon, and Seyda Wentworth made key contributions to this statement. Also, David Dornisch, Etana Finkler, Ernie Jackson, and Tom McCool provided technical assistance.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.