Afghanistan Reconstruction: Progress Made in Constructing Roads, but Assessments for Determining Impact and a Sustainable Maintenance Program Are Needed
Released: 2008-07-08T00:00:00

Letter

Background

In 2001, when the United States and its allies removed the Taliban regime, Afghanistan was in economic crisis. The nation’s roads had deteriorated, limiting economic growth and access to resources and services for its citizens. The Afghan government has recognized that improving the country’s roads is an important element in lifting the population from poverty. It considers a viable major road network essential to becoming a transport and trading hub in Central Asia. Additionally, road access for rural areas would help to promote licit crops and broaden access to health and education. In 2003, the Afghan government and international donors proposed a road network for Afghanistan, which they hoped to complete by 2015. The proposed network (as shown in fig. 1) identifies the country’s major roads, including the ring road, which links Afghanistan’s major cities of Kabul, Kandahar, Herat, and Mazar-e Sharif.

Afghanistan National Development Strategy

In February 2006, the Afghan government introduced the interim Afghanistan National Development Strategy and its accompanying implementation plan, the Afghanistan Compact. The compact included two benchmarks for roads: (1) a fully upgraded ring road as well as roads connecting the ring road to neighboring countries by the end of 2008, and (2) a fiscally sustainable system for road maintenance by the end of 2007.

The Afghan government and international donors established the Joint Coordination and Monitoring Board to oversee the implementation of compact benchmarks. In addition, representatives of relevant government ministries, agencies, and the international community have formed groups to address implementation issues. For example, the group working on roads is commonly referred to as the Consultative Group for the Transport Sector.

Afghan Ministries

Several Afghan ministries have transportation sector-related functions. The two ministries with road construction and maintenance responsibilities are the Ministry of Public Works, which has jurisdiction over highways and provincial roads, and the Ministry of Rural Rehabilitation and Development, which oversees the rural roads. The Ministries of Transportation and Civil Aviation, Finance, Commerce, Interior, and Foreign Affairs collect fees or tolls associated with the transportation sector. Additionally, the Ministry of Finance disburses funds to other ministries for road maintenance and operations. The Ministry of Economy is responsible for all government procurement, including for road maintenance.

Road Types

Since 2005, the Ministry of Public Works has attempted to standardize the nomenclature of roads in its Interim Road and Highway Standards, and estimated the length of roads in the Road Sector Master Plan. The four broad categories of Afghan roads are regional highways, national highways, provincial roads, and rural roads (see fig. 2).

U.S.-Funded Afghan Road Projects

The United States, through USAID and Defense, has committed about $1.7 billion for road reconstruction in Afghanistan between fiscal years 2002 and 2007. USAID, whose commitments represent 67 percent of the total U.S. assistance, has funded highways and provincial roads through its infrastructure program. In addition, USAID has funded rural roads through its Agriculture, Alternative Development, and Quick Impact Project programs, as well as the World Bank-led National Emergency Rural Access Project. Defense, whose funding accounts for the remaining 33 percent of the total U.S. assistance, has committed CERP funds for civilian road projects, and Military Construction and Military Engineering funds for roads for military purposes (see fig. 3).

Lrge-le infrastrctre development such as regionnd ntionl highw nd provincil roSll-le project ndertken dring erly phas of recontrction to win locsupport National Emerency Rural Access Project: A mlitilterl effort led y the World Bnk to Alternative Development and Ariculture: Incldel rod recontrction to provide ltern- tive to poppy prodction nd increasgricltprodctivity, rl employment, nd food ecrity 0 Fund committed (doll in million)

Commander’ Emerency Repone Proram: Sll-le, rgent, hnitrin relief nd recontrction project to immeditely asst the Military Contruction: Contrction of pved ro for militry prpoContrction for rgent militry prpo such as combat tril for opertion0 Fund committed (doll in million)

Since 2002, USAID and Defense have implemented road projects using contracts and grant awards. For its major infrastructure projects, USAID awarded contracts to the Louis Berger Group (LBG) in 2002 and to LBG- Black & Veatch Joint Venture (LBG/B&V) in 2006, and a grant to the United Nations Office for Project Services (UNOPS) in 2004. Defense has awarded contracts for CERP-funded road projects to USACE or directly to local contractors through the U.S.-led Provincial Reconstruction Teams between 2004 and 2007. Table 1 lists U.S. government awards for road reconstruction in Afghanistan.

International Donor Funding for Afghan Highways

The United States and other international donors have committed over $2.2 billion for reconstruction of the regional and national highways identified in the Afghan government’s Road Sector Master Plan. Of the total donor commitment for these roads, the United States (through USAID) has committed about 42 percent (over $960 million), followed by the Asian Development Bank (24 percent), Japan, and the World Bank (6 percent each) (see fig. 4). The government of Afghanistan has also committed approximately $47 million for the construction of highways.

Donors Have Made Progress in Building Highways, but Status of Other Roads Is Uncertain and Challenges Have Delayed Road Construction

The United States and other donors have completed construction of about 2,700 kilometers of regional and national highways in Afghanistan since 2002, but limited information is available on the status of provincial and rural roads and various challenges have delayed road construction. The Afghan government and international donors established the goal to complete Afghanistan’s regional highway network by the end of 2008, and as of February 2008, about 60 percent of the network was complete. The United States, through USAID, has completed its commitment of 715 kilometers, but the entire network is not expected to be completed by the end of 2008, as planned. The United States has also completed construction of 726 kilometers of national highways and construction of an additional 917 kilometers of these highways, for which the United States and other donors have committed funds, is ongoing or not yet started. Limited information is available on the status of provincial and rural roads, as donors have not reported these data to the joint Afghan government and international donor group that collects data on all Afghan road projects. Although Defense reported completing construction of 155 out of about 1,600 kilometers of CERP-funded civilian roads, project data are incomplete, and according to officials, they cannot identify the location of some CERP-funded roads. Moreover, Defense has not reported project data on CERP-funded roads for inclusion in a USAID database, as required by guidance. Donors have made progress, but road construction is behind schedule and, in some cases, has cost more than expected. For example, USAID has not met its annual completion targets for roads for 2006 and 2007. Challenges including poor security conditions, limited capacity of project implementers, and implementation without adequate planning have contributed to project delays and cost increases.

The United States and International Donors Have Built or Committed Funds for Afghan Highways, but Data on Other Roads Are Limited

The United States and other donors have completed portions of the regional and national highways (see fig. 5). While the donors have also committed funds for the construction of provincial and rural roads, the overall status of these roads is uncertain because of the lack of available data.

Although more than half of the regional highway network in Afghanistan has been completed, construction is behind schedule. In its Afghanistan National Development Strategy, the Afghan government, in conjunction with international donors, established the goal of completing a fully upgraded regional highway network by the end of 2008. The construction of the regional highways is a top road reconstruction priority of the Afghan government and international donors, as these roads are expected to foster regional trade and contribute to Afghanistan’s economic development. The United States and other donors have committed more than $1.5 billion for the over-3,200-kilometers-long regional highway network and had completed about 60 percent (1,954 kilometers) of these highways as of February 2008 (see figs. 6 and 7). The United States, through USAID, has completed the construction of the 715 kilometers of the regional highways it funded. In addition, it also managed the construction for the 115-kilometer-long Saudi-funded section, which is complete. However, construction of an additional 29 percent (932 kilometers) of the regional highway network is ongoing and donors have committed funding but have yet to start construction of the remaining 11 percent (354 kilometers). Completion of at least 300 kilometers of the regional highway network is not expected until December 2009, in part because funding was not committed until September 2006.

International donors have completed 44 percent (726 kilometers) of the 1,600 kilometers of national highways for which they have committed approximately $716 million. The national highways are expected to connect the capitals of all 34 Afghan provinces to regional highways.

USAID has completed construction of 726 kilometers of national highways and has committed funds for an additional 300 kilometers, which is under construction. Other donors, consisting of the Asian Development Bank, the World Bank, and Italy, have committed funds to construct about 600 kilometers of national highways, but none of these roads is complete (see app. II for the status of regional and national highway projects).

Limited Information Is Available on Construction of Provincial and Rural Roads; Defense Has Not Reported Data on Its Civilian Roads, as Required

The status of provincial and rural road construction is uncertain because donors have not reported this information to the group collecting road project data for all Afghan roads. Additionally, Defense has not reported project details on its CERP-funded roads for inclusion in a USAID database, as required by CERP guidance.

Limited Information on the Status of Provincial and Rural Roads Afghanistan’s Road Sector Master Plan identifies about 9,700 kilometers of provincial roads and an estimated 17,000 kilometers of rural roads, and the United States and international donors have funded and built some of these roads. However, limited information is available on donors’ funding commitments and completion status for these roads. The Consultative Group for the Transport Sector, which includes officials from donor agencies and Afghan ministries involved in the transportation sector, is responsible for collecting information on all road projects. However, information on some provincial and all rural roads, which are under the jurisdiction of the Ministry of Rural Rehabilitation and Development, is currently not reported to the group, according to the official who collects road project data. The official noted that international donors have reported only road project information for roads that are under the jurisdiction of the Ministry of Public Works.

According to USAID, it has completed construction of 237 kilometers of provincial roads, and construction of over 185 additional kilometers is under way. In addition, it has constructed over 1,000 kilometers of rural roads, such as labor-intensive cobblestone roads and farm-to-market roads, as part of its Agriculture and Alternative Development programs. However, the agency could not provide the kilometers of roads constructed through its Quick Impact Project, which included construction of small roads during the early phase of reconstruction in Afghanistan.

Defense Has Not Reported Data on Its Civilian Roads Although Defense has provided CERP funding for road reconstruction since 2004, its project status data are incomplete. According to Defense officials in Afghanistan, Defense has committed CERP funds to construct approximately 1,600 kilometers of provincial and rural roads for civilian use, of which about 155 kilometers are reported as complete. In addition to civilian roads, Defense officials reported that about 540 kilometers of roads are under construction for military purposes, including 400 kilometers of roads that have been paved to prevent insurgents from planting improvised explosive devices. However, Defense’s information on CERP-funded roads may not accurately represent kilometers of roads under construction or completed, as the project data provided to us were incomplete. For example, according to Defense officials in Washington, about $6 million of CERP funds was provided for roads in the provinces of Farah and Helmand (see app. III for CERP funding by province). However, as shown in figure 8, project location information provided by Defense officials in Afghanistan does not show any CERP roads in these provinces.

Although Defense is required by CERP guidance to report on the status of road projects to a USAID-maintained database, it has not done so. The 2006 CERP guidance requires data for all construction projects to be reported for inclusion in a USAID database. However, according to USAID, no CERP project data have been reported into the database. Furthermore, according to Defense officials in Afghanistan, Defense’s data collection systems often change when a new military command takes charge, and project information is lost during the transition. They said that because of missing documentation and frequent staff rotation, they do not know where some CERP-funded roads were built prior to fiscal year 2006. As a result, a comprehensive database of all U.S.-funded road projects does not exist.

Several Challenges Have Contributed to Delays and Cost Increases

Despite progress in building roads, construction is behind schedule. For instance, USAID did not meet its annual targets for kilometers of roads constructed as established in its Afghanistan Performance Management Plan (2006-2008). Specifically, for fiscal year 2006, the agency set a target to complete 550 kilometers of roads, but completed only 393 kilometers. In 2007, its initial target of 580 kilometers was later decreased to 423 kilometers, but the agency completed only 228 kilometers. It has also decreased the target for the current fiscal year from 550 kilometers to 314 kilometers.

U.S. agencies and other donors have faced several challenges in implementing road projects, such as poor security conditions, limitations of project implementers, and starting construction with limited planning. These factors have contributed to project delays and cost increases.

Poor Security Conditions

According to U.S. government and other donor officials, poor security is the most difficult challenge they face in implementing road projects. The security situation in some parts of the country has delayed road construction projects and increased costs. For example, USAID signed a contract for about $8 million to construct the Kajaki road in May 2007. According to USAID, this road is important as it connects the Kajaki dam, a source of water and power in southern Afghanistan, to the ring road. However, because attacks prevented contractors from working on the project, it was terminated in 2008 after USAID had spent about $5 million. USAID’s first Afghan infrastructure program also experienced cost increases due to deteriorating security. According to a USAID Office of the Inspector General report, almost $9 million of a proposed ceiling increase to the contract was directly attributable to increased security costs. Additionally, according to USACE road project data and officials, security issues resulted in frequent schedule delays for roads. For example, a contractor in Zabul province had six employees kidnapped, and as of May 2008, work on the project had not progressed for almost 60 days.

According to donors we spoke with in Afghanistan, security is a major concern in the south and the east, and is also becoming a concern in the north. Due to deteriorating security conditions, as of February 2008, Japan has been able to complete construction of only 12 percent of a 114- kilometer-long section of the Kandahar-Herat road in southern Afghanistan. This road section was originally estimated to be completed by July 2006. According to a Japanese official, the construction costs for this section have increased to more than three times the initial estimate. Additionally, because the Japanese contractor responsible for this section left due to increasingly dangerous conditions, Japan has re-awarded a contract for 50 kilometers of this road section to a local contractor.

In addition to monetary costs, attacks in areas where roads are being reconstructed have resulted in loss of life and injuries among workers. As of March 2008, 162 contractors associated with USAID roads programs have been killed and 202 have been injured or disabled in attacks since 2003. There are more casualties associated with road programs than with any other USAID program in Afghanistan, according to USAID data. Defense contractors have also suffered casualties. For example, one project was delayed by at least 30 days while the contractor organized a new team after 18 workers were killed in an attack, according to a USACE official.

Project Implementer Limitations

Limited capacity of local project implementers has impeded donors’ efforts to complete road construction projects on time and within budget. According to donor officials we spoke to, the list of capable local contractors is growing, but it still remains small. According to USACE officials, one of their objectives is to build local capacity by using Afghan contractors. However, the agency has had difficulties with poor subcontractor performance, which, according to USACE officials, has resulted in time delays. For example, although USACE planned to complete its first road project in June 2007, it was not until March 2008 that USACE officials signed a letter documenting project completion. When we visited the project in October 2007, the contract cost had increased by about 45 percent and the completion date had been extended by over a year. USACE officials said they rejected work done on the project because of “poor contractor methods” and required the contractor to redo work.

Additionally, according to USAID officials and a report on USAID’s first major road program, LBG terminated multiple subcontracts during construction of the Jalalabad-Asmar road because of poor performance, which resulted in an estimated 3- to 4-month delay for each termination. Moreover, USAID paid over $7 million for the construction of several segments of this road, all of which required rework. In an end-of-year progress report to USAID, UNOPS stated that it had terminated 10 contracts in 2005 due to poor contractor performance that had resulted in construction delays. According to USAID officials, one of the reasons they opted to award funds to UNOPS was to support their capacity building of local contractors, even though USAID officials knew it would likely result in delays and additional cost. Each terminated contract results in cost overruns due to the recurrent cost of getting the contractor’s equipment and materials to the construction site and building a new construction camp, according to a USAID official.

USAID’s concerns about UNOPS’s performance led to the termination of the grant agreement after USAID had awarded over $365 million (see app. IV for more details). USAID officials expressed concern about UNOPS’s financial management of grant funds and, in July 2007, required UNOPS to have an independent financial audit done as a condition to approving funds to cover cost overruns. In December 2007, USAID terminated the grant due to “serious shortcomings in financial and technical management” of the grant, according to USAID’s technical officer for the grant. At the time of the grant’s termination, construction of one major road project was only 60 percent complete, although grant funds were depleted.

Starting Construction with Limited Planning

Limited planning by donors prior to starting construction has resulted in increased costs. According to a senior Ministry of Public Works official, there have been several instances of cost increases because donors have started construction without adequate planning, such as completing engineering studies prior to construction. For example, according to the ministry official, the cost of a regional highway section that the Asian Development Bank funded increased from $28 million to $56 million because detailed design work had not been completed prior to project implementation. As a result of this cost increase, the project had to be put on hold for up to 3 years. The official stated that other donors, including Italy and the United States, have also faced similar challenges.

According to a USAID official, detailed planning, including engineering and feasibility studies, can take 1 year or more and such studies may not be feasible due to deteriorating security conditions. Additionally, long time frames for projects have not been acceptable in Afghanistan due to pressure from donors and the Afghan government to show faster results, according to USAID officials. As a result, USAID has not completed project designs before beginning construction of its road projects. For example, in a memo approving an expansion of the UNOPS grant, USAID officials stated that the political context necessitated having some roads under contract within weeks and under construction “as soon as possible.” In some cases, UNOPS was unable to develop complete project descriptions and corresponding cost estimates before it awarded contracts. In another case, according to a USAID Office of Inspector General report, a detailed assessment of bridges and an old road had not been done when USAID awarded the construction contract for the Kandahar-Herat road. As a result, work plans were modified during construction, contributing to a 43 percent cost increase. Our review of contract file documentation for one of the four U.S.-built sections of the Kandahar-Herat road showed a net increase of over $10 million, due in part to revisions of construction requirements of bridges.

Little Is Known about the Impact of Road Projects in Afghanistan

U.S. agencies involved in road reconstruction efforts in Afghanistan know little about the impact of road projects, since they have not conducted assessments to determine the degree to which the projects have achieved economic development and humanitarian assistance goals. While agencies reported some examples of positive impact, these are based on limited data, a qualitative evaluation, or anecdotal information, and therefore cannot be generalized. Several factors have limited agencies’ abilities to conduct impact assessments. For USAID, these include limited attention to the design of impact evaluations prior to project implementation, lack of timely data collection for its two major road projects, and deficiencies in the frameworks for assessing results of Afghanistan road projects. For Defense, they include failure to assess the results of its road projects and lack of clear guidance on the evaluation of its CERP-funded projects. Moreover, since no other donor has conducted impact evaluation of its road projects in Afghanistan, little is known about the impact of other road reconstruction as well. Completing sound impact evaluations in coordination with other donors involved in road reconstruction is vital to ensure that the assistance provided by the United States and other international donors is achieving the intended results and desired outcomes.

USAID Has Not Performed Sound Impact Assessments; Examples of Impact Are Mostly Anecdotal

USAID has not performed a sound impact assessment of its completed road projects in Afghanistan. Because of this, there is limited evidence of the positive impact of U.S.-funded road projects. In some cases the limited evidence comes from a USAID-funded qualitative study, while in other cases it is based on rough estimates, anecdotes, or impressions. Neither the qualitative study nor the anecdotes shared by U.S. officials are representative and therefore the results are not generalizable. Examples of positive results reported in the qualitative study or via anecdotes and impressions include the following: According to USAID, the ring road is being heavily used and travel times have decreased. Additionally, USAID estimates that 3,500 labor days are produced for each kilometer of road constructed. However, it could not provide us data to support this estimate.

Defense officials told us that they observed that traffic increased significantly when a new road is built. According to these officials, new roads have led to an increase in the number of gas stations, which is evidence of an increase in commerce. Moreover, according to these officials, roads have opened lines of communication among districts, provinces, and the central government, thereby improving local governance. Because they did not quantify these changes, they cannot demonstrate the extent to which these projects led to observed impacts.

According to some local elders we spoke with in a village close to Jalalabad, Nangarhar province, the cobblestone road constructed with USAID funds had enabled access to markets for agricultural products, as well as to clinics and hospitals for the local population; generated employment for daily laborers during road construction; increased commerce as new shops have been constructed along the road; and promoted community participation as locals contributed their land to widen the road.

UNOPS performed a qualitative study, funded by USAID, of the impact of rehabilitated district roads and concluded that improvements to these roads led to increased employment opportunities, reduced transport costs, and increased traffic flows. In addition, the road improvements helped to raise land values and encourage the development of a marketing network. The results of this study were based on discussions with a nonrepresentative group of respondents and did not include a comparison, for example, of how employment opportunities changed for a comparable location that did not benefit from a new road.

LBG provided some estimates of the reductions in travel time and cost savings from USAID’s completed road projects. The road reconstruction led to a reduction in travel time between 30 and 50 percent, and in vehicle operating cost savings between 25 and 62 percent, according to the estimates. These estimates are based on surveys by LBG’s engineers. However, LBG officials could not verify the accuracy of these numbers, and to date, USAID has not financed a follow-up study of its completed projects to support them.

Limitations in USAID’s Planning Efforts Impede Its Ability to Assess Impact

Limitations in planning for evaluation prior to project implementation, problems with data collection, and weaknesses associated with USAID’s Afghanistan-specific results frameworks have impeded impact assessments of its road projects.

Limitations in Funding and Data Collection

USAID’s plans show that the agency focused little attention and funding on the design of impact assessment for its first road reconstruction program prior to project implementation. Sound evaluation design includes plans to collect data before, during, and after project implementation. Although its first large road reconstruction program was launched in 2002, USAID did not issue a task order to perform a socioeconomic impact assessment for this program until October 2006. Although an assessment was planned for spring 2008, no assessments for transport infrastructure projects have been completed under this task order.

Additionally, USAID officials did not collect data for the two major completed road projects, the Kabul-Kandahar and Kandahar-Herat highways, in a timely manner to allow them to accurately measure impact over time. Baseline data on project beneficiaries and a control group collected before project implementation are used for comparison with data collected after the project is completed to assess the project’s net results. Two baseline studies were conducted, one each for the USAID- funded sections of the Kabul-Kandahar and Kandahar-Herat roads. However, the baseline study for the Kabul-Kandahar road was undertaken after considerable work had already been done, so an evaluation comparing conditions before and after this project is likely to understate its impact. While USAID conducted the baseline study for the Kandahar- Herat road before starting construction, it has not collected follow-up data since the project’s December 2006 completion. Although USAID officials have not completed a baseline study for any projects in its latest roads program, they intend to collect baseline data for six road projects and have begun efforts for one of them.

Limitations in Frameworks Linking Reconstruction Activities to Impact

While USAID recently designed a framework that links reconstruction activities in Afghanistan to expected impacts, weaknesses in Afghanistan- wide and program-specific frameworks also impede assessment of the efficient use of resources and evaluation of project impact.

Although USAID has developed an Afghanistan-specific framework to assess results, it does not clearly explain how road reconstruction will lead to the agency’s strategic objective. In its Afghanistan Performance Management Plan (2006-2008), developed following a 2005 GAO recommendation, USAID lays out a logical framework to link road reconstruction activities to its strategic objective of a licit thriving economy led by the private sector. The framework includes objectives, indicators to identify how progress will be measured, and annual targets as numeric goals that help to assess progress toward objectives (see fig. 9). However, the plan does not explain how the kilometers of roads constructed (output) would lead to impacts such as an increase in foreign direct investment and income from the private sector. Specifically, the plan does not include outcome indicators (such as cost and time savings) that would link the outputs to impact indicators.

USAID also established a program-specific framework to link projects that are part of USAID’s most recent road program, IRP, to USAID’s strategic objectives, though the framework does not include output indicators and associated targets to assess progress. Unlike the countrywide plan, the IRP framework contains outcome indicators such as vehicle operating costs and time savings (see fig. 10). These indicators measure progress toward intermediate objectives, which link to the strategic objectives of a licit economy led by the private sector and improving health and education of Afghans. However, the framework does not include output indicators and associated targets that would allow the agency to assess progress annually.

Because the IRP framework for assessing impact was not fully based on the estimated benefits in the cost-benefit analysis used to select road projects, USAID cannot verify that IRP projects will lead to the estimated benefits and expected impact. A cost-benefit analysis should be the basis for setting indicators and their target values in a framework linking the project to expected impact. USAID’s cost-benefit analysis identified the indicators used in the framework and the target values that could be used to measure annual progress toward those indicators. However, USAID did not include those measurable targets in its results framework for IRP. In its monitoring and evaluation plan for IRP, USAID stated that targets would be set based on baseline studies for individual projects by the end of 2007. However, baseline studies have not been completed and no targets have been set to date. Therefore, the estimated benefits from USAID’s cost-benefit analysis used to select IRP road projects may differ from the expected impact identified by the IRP framework.

Defense Has Not Assessed the Impact of and Lacks Clear Guidance for CERP Projects

Defense has not conducted any impact assessment of its civilian road projects. Defense’s objective for CERP is to enable local U.S. military commanders to respond to small-scale, urgent humanitarian relief and reconstruction requirements. Because CERP is used for urgent local needs such as small roads, the impact of such projects may be difficult to quantify. Although CERP-funded projects contain measures of desired impact—such as improved public sentiment toward local and national governments, increased jobs and economic growth, and increased local support to deter terrorist recruitment—it is not stated how these indicators will be measured and what data collection efforts are planned. Additionally, while CERP guidance for Afghanistan requires project proposals to have an “adequate” plan to measure success in achieving the desired impact, it does not contain criteria for developing such a plan for performance measurement and evaluation.

No Other Donors Have Performed Impact Evaluations

No other donors that we spoke to have performed impact evaluations of their road projects, and therefore little is known about the impact of road reconstruction in Afghanistan. Representatives from Japan, the European Commission, the Asian Development Bank, and the World Bank said they had not performed any evaluations to assess the impact of roads they funded. Because various donors are constructing different parts of an interconnected road network, attributing impact to a specific donor’s road project is difficult. Therefore, conducting impact evaluations in coordination with other donors is vital to ensure that the assistance provided by the United States and other international donors is achieving the desired outcomes.

The World Bank has, however, taken some steps to allow for future impact evaluations. For example, the bank developed a rough cost-benefit analysis for its emergency infrastructure project in Afghanistan, which allows for a framework to assess the outcomes of its project. For its National Emergency Rural Access Program, which includes the construction of rural roads, the bank plans to undertake a rigorous impact evaluation.

A Fiscally Sustainable Road Maintenance Program for Afghan Roads Has Not Been Established

A fiscally sustainable program for road maintenance has not been established, although the Afghan government and international donors set a goal to do so by the end of 2007. Several factors have contributed to the Afghan government’s maintenance shortfall, including (1) its lack of human and financial resources, (2) a fragmented institutional organization, and (3) the lack of a comprehensive legal framework and means to enforce fee collection for the transportation sector in Afghanistan. Since 2005, various plans have estimated that road maintenance would cost between about $70 million and $90 million annually. Because the Afghan government has not been able to maintain newly reconstructed roads, the European Commission and USAID have agreed to fund maintenance on a temporary basis for the roads they built. However, USAID did not meet its 2007 target to maintain 100 kilometers of reconstructed roads.

The Target Date for Achieving a Fiscally Sustainable Road Maintenance Program Has Not Been Met due to Various Limitations

The target to establish a fiscally sustainable road maintenance program for Afghanistan by the end of 2007 was not met. The Afghan government, in conjunction with international donors, established this target in the interim Afghanistan National Development Strategy. Although the target date was subsequently extended to March 2008, according to an official from the Consultative Group for Transport Sector, a maintenance program has not been established. Moreover, while the final Afghanistan National Development Strategy, issued in April 2008, still includes the goal of establishing a maintenance program, it does not include a date by which the program should be established.

Several factors have limited the Afghan government’s support for road maintenance activities. These include (1) resource constraints such as a lack of human and financial resources and weak private sector capacity, (2) a fragmented institutional organization, and (3) the lack of a comprehensive legal framework and means to enforce road-related fee collection.

Lack of Resources

According to the Transport Sector Strategy (2008-2013), ministries and institutions involved in the transportation sector lack human capacity and organization to carry out tasks such as budgeting, procurement, and contract administration, which are necessary to implement a road maintenance program. Donor officials we spoke to in Afghanistan stated that the Afghan government is not able to attract young workers due to low wages. According to a March 2008 United Nations report, low salaries place many working Afghans, including government employees, at risk of falling below the poverty line. A senior official at the Ministry of Public Works with whom we spoke noted that his ministry cannot pay adequate salaries to its engineers and technical staff. For example, the general director for maintenance is paid only about $200 per month, while a person with similar qualifications could earn five times that salary in the private sector, according to the official.

According to a USAID official and a senior Afghan government official, the Afghan government also lacks financial resources to cover road maintenance costs. The Ministry of Public Works’ Road Sector Master Plan estimated in 2005 that the total maintenance cost for regional and national highways and provincial roads would be $38 million for 2007. In October 2007, a senior ministry official told us that while the Ministry of Public Works estimated that it needed $30 million to maintain roads in 2007, the Ministry of Finance had allocated only $8 million. Thus, the Ministry of Public Works planned only to undertake routine maintenance, and postpone more expensive periodic maintenance work.

Weak private sector capacity has also hindered the Afghan government’s ability to carry out road maintenance. Many donors have recommended that the Ministry of Public Works involve more private sector entities in its road maintenance efforts. However, according to a senior official at the ministry, the private sector still lacks the capacity to undertake and complete road maintenance projects. For example, in 2006, the Ministry of Public Works requested bids for maintenance of the Kabul-Kandahar road three times without receiving a single bid. In its 2005 assessment of road maintenance needs in Afghanistan, the World Bank recommended that Ministry of Public Works staff perform urgent maintenance until private sector contractors have the capacity to do so. According to a USAID needs assessment of the transportation sector, the Afghan private sector’s participation is also limited due to a lack of available cash and debt financing. This lack of initial funding hinders local companies from bidding for costly road projects, as road contractors usually get paid later in the life of the contract, thus requiring them to have the resources to finance project start-up costs.

Fragmented Institutional Organization

The Afghan government’s participation in road maintenance has also been limited by a fragmented organization of the transportation sector and the lack of a comprehensive legal framework for this sector. Numerous ministries are involved in road operations and maintenance (see table 2). According to officials from USAID and the European Commission, donors affiliated with the Consultative Group for the Transport Sector consider this fragmented governance a major impediment to effective management, and they recommended consolidating the transportation sector under a single ministry in the draft Transport Sector Strategy. However, USAID officials and a Ministry of Finance official said that ministries with a stake in the transportation sector are expected to oppose such a recommendation. The Transport Sector Strategy finalized in February 2008 includes a recommendation to establish an interministry working group for the transportation sector, but falls short of recommending consolidation of transportation sector functions under one ministry. Additionally, donors have recommended that, in accordance with international best practices, a road fund be established to ensure that revenues from road user charges are used only for road operations and maintenance. However, the Ministry of Finance, which currently receives revenues from road-related charges, has opposed this.

The government also lacks a comprehensive legal framework for the transportation sector. According to the Transport Sector Strategy and USAID needs assessment of transportation sector capacity, the existing decrees and laws do not clearly delineate the responsibilities of various Afghan ministries involved in the transportation sector. For example, as shown in table 2, more than one ministry collects fees from and issues permits to commercial vehicles. Consequently, there are overlapping jurisdictions and functions. Additionally, according to these documents, transportation sector laws are incomplete. For example, while there is a traffic regulation law that defines restrictions on drivers and regulates the weight of vehicles, it does not set the penalties for vehicle overloading, according to the strategy. Overloaded vehicles cause the most serious damage to roads in Afghanistan, according to USAID’s needs assessment.

Additionally, the Afghan government has not been able to generate sufficient revenues for road maintenance due, in part, to ineffective enforcement of tolls and fee collection. For instance, according to a USAID needs assessment, the Ministry of Finance started a new system of issuing stickers for a fee based on vehicle size to drivers who use toll roads. However, the ministry had no mechanism to ensure that vehicles passing through road checkpoints had the stickers that allowed them to use the toll road. According to the assessment, the Ministry of Finance did not appear to have coordinated its efforts with the Ministry of Interior, which controls the Afghan National Police who man the checkpoints and have enforcement authority. Consequently, the effort failed. A senior Ministry of Public Works official confirmed that the sticker system had failed to generate expected toll revenues. Moreover, the Afghan Parliament as well as the Ministry of Public Works have opposed a fuel tax as a revenue generation option on the grounds that it is difficult to implement and may adversely affect the poor or those who use fuel for electricity generators.

Plans Estimate Costs and Funding Options for Road Maintenance

A variety of donor-funded plans have estimated costs and funding options for road maintenance. Some of the funding options are already being used by Afghan ministries, but with limited success. Additionally, the estimates of maintenance costs do not include rural roads.

In May 2005, the World Bank issued the Afghanistan Road Maintenance Note, which identified maintenance costs as about $72 million annually for major roads by the year 2011, when the regional and priority national highways were expected to be completed. The document recommended that the Afghan government develop clear policies for road maintenance and prepare a maintenance program for the World Bank-funded emergency road project that was implemented during the early stage of reconstruction in Afghanistan.

In April 2006, the Asian Development Bank-funded Road Sector Master Plan estimated the cost of road maintenance to be $85 million to $90 million annually between 2011 and 2015 once construction of the currently funded road network is completed. This cost estimate does not include maintenance costs for rural roads. The plan presented several options to fund road maintenance, such as a fuel tax, vehicle registration fees, and road tolls. Various Afghan ministries were already using some of these mechanisms, such as road tolls and vehicle registration fees; however, these methods have been plagued by problems such as lack of enforcement and corruption. As a result, revenues from these sources were less than $4 million in 2004.

In April 2007, the Ministry of Public Works published the final draft of its strategic plan, with funding and technical assistance from USAID. The plan estimated road maintenance costs for the next 5 years at about $400 million (or an average of $80 million annually) for 7,000 kilometers of highways and provincial roads. It articulated goals and measures as well as proposed activities such as developing public and private sector capacity, standards for road maintenance, and a revenue generation system (see table 3). However, this plan only covers roads and activities under the Ministry of Public Works’ jurisdiction, and does not include rural roads that are under the jurisdiction of the Ministry of Rural Rehabilitation and Development.

Donors Have Recently Started Maintenance Programs for Sections of the Major Roads; USAID Did Not Meet its 2007 Maintenance Target

Since 2006, the European Commission and USAID started separate 3-year road maintenance programs that would cover over 1,600 kilometers of the regional highways they built. However, USAID did not meet its 2007 target to maintain 100 kilometers of reconstructed roads.

In June 2006, the European Commission initiated a road maintenance program for the 150-kilometer section of the regional highway from Kabul to Jalalabad, the construction of which it had previously funded. USAID signed a task order under the IRP contract effective in November 2007 for an estimated cost of $33 million for a 3-year road maintenance program covering about 1,500 kilometers of roads it has completed. Additionally, Defense has provided ad hoc assistance for road maintenance using CERP funding. For example, CERP funds have been used to provide road maintenance equipment to local governments and to undertake some urgent winter maintenance.

USAID’s road maintenance program includes establishing a Road Maintenance Unit housed in the Ministry of Public Works to implement maintenance projects with the assistance of a contractor. Additionally, the contractor will establish new management procedures and develop manuals on maintenance-related topics such as planning, budgeting, contracting, and performance monitoring.

In its Performance Management Plan (2006-2008), USAID identified kilometers of reconstructed road maintained as the indicator to measure progress toward road maintenance and established yearly targets for fiscal years 2007 and 2008. However, because USAID had not started any maintenance, it did not meet its target of 100 kilometers of reconstructed roads maintained for fiscal year 2007. Additionally, while USAID proposed measuring Afghan government contributions toward road maintenance by calculating the increase in the Ministry of Public Works’ budget for this purpose, our review of the Performance Management Plan found that it did not include annual targets or report on actual contributions of the ministry. Therefore, it is not clear whether or to what extent the Afghan government has increased its support for road maintenance.

Conclusions

The United States and other international donors have committed billions of dollars toward road reconstruction in Afghanistan to promote economic and social development as well as security and stability. While some have noted that reconstructed roads contribute positively to economic and social conditions in Afghanistan, there is currently little evidence based on sound impact assessments that these projects have resulted in expected benefits. Because donors are building an interconnected road network, coordination in planning and conducting impact assessments would complement individual donors’ efforts to assess the impact of their own projects and could result in more efficient use of resources. Moreover, without a sustainable road maintenance program, newly constructed roads will ultimately deteriorate, expected benefits will not materialize, and the billions of dollars spent on road reconstruction would be wasted. There is a lack of comprehensive information on all donor-funded road projects in Afghanistan, as donors are not reporting road project data systematically and regularly to the Consultative Group for the Transport Sector, a joint Afghan government and donor group responsible for collecting these data. Furthermore, there is a lack of comprehensive data on U.S.-funded road projects, as agencies are not reporting information for all their civilian road projects to a USAID database. In the absence of complete information on all road projects in Afghanistan, the Afghan government and international donors will not be able to efficiently coordinate their efforts and better plan future road projects. Additionally, due to lack of coordination among Afghan ministries involved in the transportation sector, it is not clear whether or when the Afghan government will be able to assume greater responsibility for its road construction and maintenance activities. Improvements in efforts to assess, maintain, and document road reconstruction projects are necessary for USAID and Defense to provide better assurance that their efforts will achieve U.S. and Afghan government road reconstruction goals.

Recommendations for Executive Action

To improve evaluation as well as efficiency and effectiveness of USAID- funded road reconstruction projects, we recommend that the Administrator of USAID take the following actions: improve the results framework to ensure that it is based on expected benefits from a cost-benefit analysis with clearly stated goals, indicators, and targets and in coordination with other donors, consider building impact evaluations into project design and perform such evaluations after project implementation.

To ensure the evaluation of CERP-funded roads, we recommend that the Secretary of Defense require impact evaluations of these projects where applicable.

Until the Afghan government is able to establish a road maintenance program, we recommend that the Administrator of USAID work with the Afghan government to take steps to address urgent maintenance needs, such as formulating and enforcing vehicle weight standards, while continuing to take steps to address long-term maintenance needs, such as by developing a maintenance implementation plan and require that future agreements for road reconstruction projects include plans detailing options for funding road maintenance.

To ensure that Defense and USAID officials have adequate information to make effective future project management decisions, we recommend that the Secretary of Defense require that data for Defense’s CERP-funded road projects be reported for inclusion in USAID’s database, as required by CERP guidance.

Agency Comments and Our Evaluation

USAID and Defense provided written comments on a draft of this report, which are reproduced in appendixes V and VI, respectively. USAID and Defense agreed with our findings, conclusions, and recommendations. In response to our recommendations, USAID stated that it is conducting an assessment of the Kabul-Kandahar-Herat road, which will be completed by early fall 2008, and will revise its results framework for road reconstruction projects to include goals, indicators, and targets, and to link it to a cost-benefit analysis. USAID also indicated it will introduce the issue of impact evaluations to the joint government-donor forum for road sector policy coordination. USAID also stated that it will offer a technical assistance package to the Afghan government to help draft road regulations needed to maintain Afghanistan’s roads, and highlighted plans to follow through with the $33 million it committed in fiscal year 2008 for road maintenance while longer-term solutions for road maintenance are being developed. Defense indicated that its CERP guidance has been revised to include a requirement that proposals for CERP projects include performance metrics and indicators for evaluating projects. The guidance also requires Defense to transfer project data to all required databases, including USAID’s, as part of CERP project closeouts. USAID and Defense also reviewed a draft of this report for technical accuracy. Defense provided technical comments, which we incorporated into the report as appropriate.

We are sending copies of this report to interested congressional committees. We will also make copies available to others on request. In addition, this report is available on GAO’s Web site at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII.

Appendix I: Scope and Methodology

To review the status of donor-funded road projects, we obtained road project data on international donors from the Consultative Group for the Transport Sector, a coordination body that includes donor representatives and Afghan officials from relevant ministries. For U.S.-funded roads, we obtained relevant data from the U.S. Agency for International Development (USAID), the U.S. Army Corps of Engineers (USACE), and the Department of Defense’s (Defense) Army Budget Office as well as the Combined Joint Task Force in Afghanistan.

To assess the reliability of the data on the commitments made by international donors, we (1) interviewed the USAID-funded contractor at the Consultative Group for the Transportation Sector who is responsible for compiling these data on a monthly basis based on information provided by all the donors and the government of Afghanistan, and (2) performed some basic reasonableness checks of the data against other sources of information. We determined that the data are sufficiently reliable for the purposes of (1) assessing the status of road projects, and (2) making a broad comparison of the United States’ contributions to those of other major donors. However, the data have various limitations, including the fact that they are largely self-reported by donor nations to the Afghan government, are affected by differences in exchange rates, and may not represent all road projects. Due to these limitations and the fact that we could not contact each of the donors, we were unable to determine the reliability of the most current status of projects and the precise dollar amounts committed by every donor. Nevertheless, because these are the best available data on donor-funded road reconstruction projects, we present the status and funding amounts reported by each donor in appendix II.

For USAID road projects, we primarily used data from its database called Geobase. These data were for two USAID departments involved in road construction: the Office of Infrastructure, Engineering and Energy and the office of the Provincial Reconstruction Team. While we received comprehensive information for Office of Infrastructure, Engineering and Energy roads, we obtained Provincial Reconstruction Team road data only for completed Quick Impact Projects and not for current road projects. The amount of funding for the current road projects is very small, according to USAID officials.

For Defense road projects, we obtained data from several departments and agencies. For USACE-implemented projects, which include a number of Commander’s Emergency Response Program (CERP)-funded road projects and all Military Construction-funded roads, we obtained detailed data directly from the agency. Defense officials in Afghanistan provided data for projects that used CERP, Military Construction, and Military Engineering funds. CERP and Military Construction project data included kilometers of roads for completed and ongoing projects. However, funding information for a large number of CERP projects was missing. Consequently, we obtained funding data for CERP from the Army Budget Office at the Pentagon. Last, due to sensitivity of Military Engineering roads, Defense officials provided only the aggregate funds committed for these projects.

To assess challenges in implementing road reconstruction projects, we interviewed agency officials from USAID and Defense in Washington, D.C., and in Afghanistan. In Afghanistan, we met with officials in Kabul and Bagram. We also met with U.S. Provincial Reconstruction Team officials during site visits to road projects in Badakhshan, Nangarhar, and Khost provinces. We reviewed USAID’s major contract and grant agreements and select Defense CERP contracts and analyzed contract file documents, such as agency memorandums, modifications to contracts, and contract summary reports where appropriate and available. We also reviewed construction industry best practices and USAID and Defense Inspector General reports relating to road construction.

To examine USAID and Defense’s assessment of impact of road reconstruction projects in Afghanistan, we reviewed documentation including agencywide guidance for monitoring and evaluation, performance monitoring plan, and documents for USAID’s Rehabilitation of Economic Facilities and Services (REFS) program and Infrastructure and Rehabilitation Program (IRP) projects and Defense’s CERP projects. We limited our review of results frameworks to USAID’s overall Performance Management Plan, and its REFS and IRP programs. We did not assess the framework for the projects implemented under the United Nations Office for Project Services (UNOPS) grant. We assessed the documents for the following elements: a logical framework to assess results; statement of strategic and intermediate goals; output, outcome, and impact indicators; and targets for these indicators. Additionally, we reviewed the explanation of how targets were selected and if they were met. We also reviewed articles and books on monitoring and evaluation that provided us tools to assess impact evaluation of the completed road projects and helped identify methodological challenges in the design and implementation of such evaluations.

To assess the extent of road maintenance for newly constructed roads, we reviewed pertinent strategic and planning documents, and interviewed Afghan and donor officials during our fieldwork. To identify the national goal for road maintenance and progress on this goal, we reviewed the Afghan Compact, the interim and final Afghan National Development Strategy, and the periodic Joint Coordination and Monitoring Board reports. To identify USAID’s maintenance goals and progress on these, we reviewed USAID’s Afghanistan Performance Management Plan (2006-2008) and the agency’s reporting on these goals. To identify the extent to which donors and the Afghan government have discussed and planned for road maintenance, we reviewed key documents including the World Bank’s Road Maintenance Note, Road Sector Master Plan for Afghanistan, Ministry of Public Works Strategic Plan, draft and final Transport Sector Strategy, and Needs Assessment for Transport Sector for USAID-funded road projects. To assess USAID’s project for road maintenance, we reviewed the relevant contract to identify details of the plan.

Appendix II: USAID and Other Donor Road Project Data

Appendix II: USAID and Other Donor Road Project Data Donor (Implementer) in millions)

Kabul-Kandahar (Sections B to F)

USAID (LBG)

USAID (implementer)

USAID (LBG)

USAID (LBG and UNOPS)

USAID (LBG/B&V)

USAID (UNOPS)

USAID (UNOPS)

USAID (UNOPS)

Donor (Implementer) in millions)

USAID (UNOPS)

USAID (UNOPS)

USAID (LBG/B&V)

USAID (UNOPS)

USAID (LBG/B&V)

USAID (UNOPS)

USAID (UNOPS)

USAID (LBG/B&V)

USAID (UNOPS)

USAID (USACE)

USAID (UNOPS)

USAID (UNOPS)

USAID (LBG and UNOPS)

The government of Afghanistan is in negotiations with the Asian Development Bank for an additional $10 million for the reconstruction of this road section.

The government of Afghanistan will contribute an additional $4 million to this project.

Funding data have been rounded and therefore may not precisely match the total.

These road projects have been terminated due to poor security conditions.

Appendix III: CERP Funding Data for Road Projects by Province in Afghanistan

Appendix III: CERP Funding Data for Road Projects by Province in Afghanistan No.

No.

Appendix IV: USAID’s Grant Agreement with UNOPS for Road Reconstruction in Afghanistan

Throughout the life of its road reconstruction grant with UNOPS, USAID increased its size over 10 times, from $35 million to over $365 million, adding nearly 700 kilometers of roads to the original 300, and extending the period of performance by nearly 3 years (see fig. 11). USAID transferred several roads from REFS to UNOPS because REFS hit its contract ceiling. According to USAID officials, they did not want to use LBG as the prime contractor for all of their major infrastructure projects in Afghanistan, especially in light of “congressional and private sector” criticism of LBG’s management of other Afghan infrastructure projects. However, USAID required UNOPS to subcontract engineering responsibilities for the transferred roads to LBG, adding an extra layer of management. After the termination of the grant due to concerns about UNOPS’s financial management, five senior UNOPS officials associated with the USAID program resigned or were released, according to a USAID official.

Appendix V: Comments from the U.S. Agency for International Development

Appendix VI: Comments from the Department of Defense

Appendix VII: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the person named above, Hynek Kalkus (Assistant Director), Jaime Allentuck, Jeffrey Baldwin-Bott, Gergana Danailova-Trainor, Karen Deans, Etana Finkler, and Mona Sehgal made key contributions to this report. Michael Armes, Timothy DiNapoli, Mark Dowling, Cindy Gilbert, John Hutton, and Jena Sinkfield also provided technical assistance.

Related GAO Products

Afghanistan Security: Further Congressional Action May Be Needed to Ensure Completion of a Detailed Plan to Develop and Sustain Capable Afghan National Security Forces. GAO-08-661. Washington, D.C.: June 18, 2008.

Afghanistan Security: U.S. Efforts to Develop Capable Afghan Police Forces Face Challenges and Need a Coordinated, Detailed Plan to Help Ensure Accountability. GAO-08-883T. Washington, D.C.: June 18, 2008.

Securing, Stabilizing, and Reconstructing Afghanistan: Key Issues for Congressional Oversight. GAO-07-801SP. Washington, D.C.: May 24, 2007.

Afghanistan Drug Control: Despite Improved Efforts, Deteriorating Security Threatens Success of U.S. Goals. GAO-07-78. Washington, D.C.: Nov. 15, 2006.

Afghanistan Reconstruction: Despite Some Progress, Deteriorating Security and Other Obstacles Continue to Threaten Achievement of U.S. Goals. GAO-05-742. Washington, D.C.: July 28, 2005.

Afghanistan Security: Efforts to Establish Army and Police Have Made Progress, but Future Plans Need to Be Better Defined. GAO-05-575. Washington, D.C.: June 30, 2005.

Afghanistan Reconstruction: Deteriorating Security and Limited Resources Have Impeded Progress; Improvements in U.S. Strategy Needed. GAO-04-403. Washington, D.C.: June 2, 2004.

Foreign Assistance: Lack of Strategic Focus and Obstacles to Agricultural Recovery Threaten Afghanistan’s Stability. GAO-03-607. Washington, D.C.: June 30, 2003.