Corps Of Engineers: Improved Analysis of Costs and Benefits Needed for Sacramento Flood Protection Project
Released: 2003-11-26T00:00:00

Letter

Background

Sacramento, California, was established at the confluence of the American and Sacramento Rivers shortly after gold was discovered upstream at Sutter’s Mill in 1848. Frequent flooding has been a problem in Sacramento since its founding. To help reduce flooding, over time a complex system of levees, dams, and other related facilities were built. Levees line both sides of the American River from where it meets the Sacramento River upstream for a distance of about 17 miles, and the Natomas Basin is completely surrounded by levees. In addition, the Folsom Dam, completed in 1956 and located upstream from Sacramento on the American River, uses a portion of its storage capacity for flood protection.

The Sacramento area flood protection system was designed on the basis of records of rainfall during the first half of the 20th century. However, since 1950, the American River watershed has experienced five floods that were larger than any recorded in the pre-1950 period, although downtown Sacramento was not flooded during any of these events. Nonetheless, the Sacramento area has less protection than the designers of the original flood protection system realized. In fact, much of urbanized Sacramento is located in areas where a flood has a 1 percent chance of occurring every year—known as the 100-year floodplain. Because of this limited level of protection, the Corps estimates that a very large flood—one with a 0.25 percent chance of occurring every year—would flood the 400-year floodplain, resulting in residential, commercial, industrial, and public property damage of about $15.5 billion as well as loss of lives. According to the Corps, about 305,000 people live in more than 100,000 residential properties located within the American River floodplain. A major flood also would cause toxic and hazardous waste contamination; disrupt the city’s downtown business and government areas, including the state capitol; and interfere with the transportation system, including two interstate highways.

A major flood in 1986, the largest one ever recorded on the American and Sacramento Rivers, severely strained the levee system protecting Sacramento. Although the levees held and downtown Sacramento was not flooded, the event spurred efforts by federal, state, and local entities to identify measures to increase Sacramento’s level of flood protection. In 1987, the Corps began work on a comprehensive study of flood protection alternatives for Sacramento. In its 1991 report, the Corps’ Sacramento district office considered six flood protection options and recommended building a new dam on the American River at Auburn, California, but Congress did not approve the dam’s construction. Subsequently, in response to the Department of Defense Appropriations Act of 1993, the Corps reevaluated three alternatives for increasing flood protection. In its 1996 report, the Corps examined (1) building a new dam near Auburn, California; (2) modifying the existing Folsom Dam; and (3) increasing the amount of water released from Folsom Dam during a flood, coupled with other flood protection measures. The Corps again recommended building a dam at Auburn, but, again, Congress did not approve its construction.

Recognizing the magnitude of the opposition to the proposed Auburn Dam, in June 1996, the Corps recommended the Common Features Project, which included improving sections of the American and Sacramento Rivers’ levees, primarily by constructing cut-off walls, to provide small- scale improvements to flood protection for the Sacramento area while the options for more extensive improvements continued to be considered. The WRDA of 1996 authorized $57 million for the Common Features Project, which included 24 miles of levee improvements on the American River and 12 miles on the Sacramento River along the western border of the Natomas Basin. Subsequently, the Corps concluded that it could provide the same level of flood protection on the American River by modifying only about 21 miles of levees. Figure 2 shows how a cut-off wall, which is composed primarily of a soil, cement, and clay mixture that forms an impermeable barrier when it hardens, can prevent water from seeping under or through a levee.

In January 1997, numerous rivers in northern California flooded causing extensive damages, although not in the Natomas Basin or downtown Sacramento. This flood, which was nearly as large as the 1986 flood, highlighted the continuing vulnerabilities of the existing flood protection system. In response, the WRDA of 1999 (1) modified the Common Features Project by adding about 3.8 miles of additional levee modifications along the American River and 10 miles on the Natomas Cross Canal, located on the northern border of the Natomas Basin, and (2) increased the project’s authorization from $57 million to $92 million.

When Congress approves a flood protection project, it authorizes a specific amount of money for the project, which provides the basis for the maximum project cost. According to section 902 of the Water Resources Development Act of 1986, as amended, the maximum project cost is the sum of (1) the original authorized amount, with the costs of unconstructed project features adjusted for inflation; (2) the costs of modifications that do not materially alter the scope of the project, up to 20 percent of the original authorized amount (without adjustment for inflation); and (3) the cost of additional studies, modifications, and actions authorized by the 1986 Act or any later law. As a result of these provisions, the $92 million that Congress authorized for the Common Features Project in 1999 translates to an allowable maximum project cost of about $120 million in 2003.

When Congress authorized the Common Features Project in 1996, federal law required that nonfederal partners pay 25 percent of the cost of flood protection projects. For the Common Features Project, these partners are the State of California Reclamation Board and the Sacramento Area Flood Control Agency. In this report, when we refer to project costs, including the maximum allowable project cost, we are referring to the combined federal and nonfederal expenditures.

Design Modifications Have Greatly Raised Costs for the Common Features Project

Estimated costs for the Common Features Project grew from $57 million in 1996, when the project was first authorized, to between $270 million and $370 million in 2002, primarily because the Corps changed the design of the levee improvements. For the American River levee improvements authorized in 1996, estimated costs more than tripled, due largely to changes in the design of the cut-off walls. New work authorized in 1999 added another $15 million to the cost increase. The Corps has completed much of the American River work authorized in 1996, but it has not begun construction on the work authorized in 1999. Regarding the Natomas Basin component, estimated costs increased from $13 million to between $112 million and $212 million. Costs rose primarily because the Corps changed the design of the levee improvements and proposed adding other improvements to this component. The Natomas Basin work is in the early planning stages, and the Corps has not begun construction. As of July 2003, the Corps had spent or made plans to spend nearly all of the money authorized for the Common Features Project. It therefore will not be able to finish constructing the American River work authorized in 1996, begin constructing the American River work authorized in 1999, or complete planning for the Natomas Basin work unless Congress increases the project’s authorized funding.

Estimated Costs for the American River Levee Improvements Authorized in 1996 Have More Than Tripled Because of Design Changes

The Corps’ cost estimate for the American River levee improvements authorized in 1996 has more than tripled, from $44 million in 1996 to about $143 million in July 2002, as shown in table 1.

As table 1 shows, costs rose primarily because of the increased costs of the cut-off walls. The Corps’ original design called for building cut-off walls to a depth of between 20 and 30 feet to prevent water from seeping through the levees and for allowing gaps in the cut-off walls at bridge and utility crossings. However, after the 1997 flood, the Corps realized it also needed to address the problem of water seeping under levees. It therefore increased the depth of the cut-off walls to between 60 and 80 feet and closed the gaps in the cut-off walls at bridge and utility crossings. For some sections of the levees, the Corps could not close the gaps using its standard approach for cut-off walls because of problems accessing the sites. As a result, the Corps employed a new and more expensive approach—known as jet grouting—to build cut-off walls by drilling and injecting concrete material into areas that were difficult to access. Closing the gaps in the cut- off walls by jet grouting raised estimated costs by $52 million and increasing their depth raised costs by $24 million, according to the Corps’ July 2002 cost estimate. However, in September 2002, the Corps determined that fewer gaps needed to be closed using jet grouting, which should reduce costs to some extent. As of June 2003, however, the Corps had not incorporated these potential cost reductions into an official project cost update.

As table 1 also shows, the Corps’ response to accidents that occurred during construction of the 1996 authorized work added $11 million to project costs. On three occasions, liquid material from the cut-off walls accidentally leaked into either the American River or the backyards of homes that are built against the levees. As a result, the Corps incurred costs cleaning up these spills and responding to new work requirements mandated by the Environmental Protection Agency to help prevent future leaks.

Lastly, in addition to the cost increases related to the 1996 authorized work, new flood protection measures authorized in 1999 added about $15 million in costs to the American River component of the project. These measures include raising levee banks at two locations, installing gates and pumps at an existing drain, and installing cut-off walls in two additional levee segments.

Of the American River work authorized in 1996, the Corps has completed about 90 percent and must still close gaps in the cut-off walls at some remaining bridge and utility crossings to complete this work. For the levee improvements authorized in 1999, the Corps has done some planning but has not begun any construction. However, as of July 2003, the Corps had spent or had plans to spend $116 million of the $120 million authorized for the entire Common Features Project. The Corps could not give an exact accounting of how much of the $116 million it had spent on the 1996 American River work. However, on the basis of the information that the Corps provided, we estimate the Corps has spent, or made plans to spend, at least $103 million for planning and constructing the 1996 American River work. Because the Corps has spent or made plans to spend most of the project’s authorized funds, it will not be able to complete the 1996 and 1999 work on the American River unless Congress increases the project’s authorized funding.

Natomas Basin Costs Are Expected to Increase Significantly, and Lack of Funds Has Halted Planning and Cost Estimating Efforts

The Corps’ preliminary cost estimates for the Natomas Basin component of the project increased from $13 million in 1996 to between $112 million and $212 million in 2002, as shown in table 2.

As table 2 shows, the Corps estimates that the costs for the original levee improvements will increase by between $47 million and $88 million due to design changes to add cut-off walls or provide other methods of flood protection to control seepage under levees. The Corps proposed new work in 2002 that will increase costs by between $37 million and $84 million. This work is located in an area of the levee where the Corps previously had constructed a cut-off wall to stop water from seeping through the levee. However, the Corps later determined that the cut-off wall was not deep enough to prevent water from seeping under the levee, and the proposed new work will address this problem. Finally, the Corps estimates that the additional work authorized in 1999 to modify levees along the Natomas Cross Canal, which empties into the Sacramento River at the north end of the Natomas Basin, will add between $14 million and $26 million to the cost of this component of the project.

The Natomas Basin work—authorized in 1996 and 1999 and the additional work the Corps identified—is in the planning stages and no construction has yet begun. The Corps has been updating information on the extent of the levee problems and the costs of the improvements identified in the original plan and intends to submit a more precise cost estimate to Congress when it completes its planning. However, the Corps halted its Natomas Basin planning work in June 2003 because it had spent or made plans to spend nearly all of the money authorized for the entire Common Features Project.

Given that the Natomas Basin levee improvements will cost significantly more than originally estimated and no construction has yet begun, identifying and evaluating alternative flood protection measures could result in cost savings. For example, one possible alternative method for flood protection identified by the Sacramento Area Flood Control Agency, as well as the Corps, involves lowering the water level in the Sacramento River during floods by diverting water through the Fremont Weir and into the Yolo Bypass, which is located at a point just before where the Sacramento River flows past the Natomas Basin. The Fremont Weir is a low dam that controls the movement of large volumes of floodwater from the Sacramento River by diverting it into the Yolo Bypass. The Yolo Bypass is a continuous, 40-mile open space corridor that is protected from urban development pressure by flood easements. (See fig. 3.)

Lowering the water level in the Sacramento River as it passes the Natomas Basin could, among other things, improve the reliability of the Natomas Basin levees and may provide more cost-effective flood protection than the current Natomas Basin levee improvement plan. However, as of June 2003, the Corps had not yet analyzed the costs and benefits of modifying the weir and the bypass or any other alternative method for Natomas Basin flood protection.

The Corps Did Not Adequately Analyze Likely Cost Increases for the Common Features Project or Communicate Them to Congress in a Timely Manner

After the 1997 storm demonstrated vulnerabilities in the American River levees, the Corps significantly changed the design of the levee improvements but did not analyze the likelihood of cost increases for the Common Features Project. The Corps then began constructing the American River levee improvements without informing Congress that the changes could greatly increase the overall costs of the project. By the time that the Corps reported the significant cost increases in 2002, it had already spent or made plans to spend more than double its original estimate for the American River levee improvements authorized in 1996. Furthermore, as previously discussed, the Corps estimates that it will spend more than three times its original estimate by the time it completes this work. The Corps has been able to pay for these levee improvements by spending funds originally planned for the Natomas Basin and the additional American River improvements authorized in 1999.

The Corps Did Not Adequately Analyze Likely Cost Increases for the American River Levee Improvements

The Corps did not analyze the risk of cost increases after changing the design of the American River levee improvements in 1997 and, therefore, did not provide Congress with information on the project’s exposure to significant cost increases. A storm in January 1997 demonstrated that the American River levees were vulnerable to floodwaters seeping under them, which could cause them to fail. On the basis of this information, the Corps significantly changed the design of the levee improvements but did not conduct a cost risk analysis, or any other type of analysis, to determine the extent to which these changes would increase the costs for the Common Features Project.

According to the Corps’ policy, project management teams should consider conducting a cost risk analysis when developing cost estimates for projects with considerable uncertainties. A cost risk analysis identifies the areas of a project that are subject to significant uncertainty about costs and provides decision makers with a range of potential costs for a project and the probability that these costs will be exceeded. For example, a cost risk analysis might determine that there is a 50 percent chance that costs for a particular project will exceed $5 million but only a 20 percent chance that costs will exceed $8 million. According to a report from the Corps’ Institute for Water Resources, this type of estimate is more accurate than a single point cost estimate and provides decision makers with better and more complete information.

However, the Corps did not analyze the risk of cost increases after changing the design of the American River levee improvements even though it had identified several factors that could lead to significant cost increases. For example, by July 1997 the Corps recognized that it had to close the gaps in the cut-off walls at bridges and other areas and extend the depth of some walls from about 20 to about 60 feet, although the Corps had not developed a final design for these improvements. By identifying a project element with significant cost uncertainty—the design and depth of the cut-off walls—the Corps essentially performed the first step of cost risk analysis. However, the Corps did not follow through by quantifying this uncertainty and determining a range of potential costs for the cut-off walls or the likelihood that the potential costs within that range would be exceeded—the second and third steps of the cost risk analysis. Given that the Corps’ original cost estimate for the American River work was nearly equal to its estimates of the benefits, if the Corps had conducted a cost risk analysis, it would have shown whether there was a significant likelihood that project costs would be greater than the economic benefits.

Furthermore, despite experiencing significant cost increases for the 1996 work, the Corps did not conduct a cost risk analysis to determine its exposure to potentially significant cost increases for the 1999 work. In addition, the Corps is not planning to conduct a cost risk analysis for the Natomas Basin improvements. According to Sacramento district officials, the Corps did not conduct a cost risk analysis because it did not believe such an analysis was necessary to account for uncertainties in the project.

The Corps Did Not Provide Congress with Timely Information about Significant Potential Cost Increases for the American River Levee Improvements

The Corps’ planning guidance generally directs the Corps to seek new spending authority from Congress if it determines that a project’s estimated costs exceed the maximum project cost before it has awarded a project’s initial contract. However, after making significant changes to the project’s design in 1997, the Corps did not reevaluate its cost estimate to determine if it could still implement the project without exceeding the maximum project cost. For example, the Corps did not estimate the potential for cost increases due to tripling the depth of some cut-off walls, which eventually added $24 million in estimated costs to the project. In addition, the Corps did not estimate the potential for cost increases due to closing the gaps in the cut-off walls at bridges and other areas. This expense was not considered in the Corps’ original 1996 cost estimate and potentially involved the use of jet grouting—a technology the Corps had not previously used to construct cut-off walls. Closing the gaps in the cut-off walls eventually added $52 million in estimated costs to the project.

In spite of significantly changing the project’s design, the Corps awarded the project’s first contract without updating its cost estimate to determine whether it would need additional spending authority to complete the project. In June 1998, the Corps issued the first Common Features Project solicitation for bids to construct about 1.6 miles of the redesigned cut-off wall on the north bank of the American River. These levee improvements represented only about 8 percent of the total miles of planned American River levee improvements, but the bid that the Corps selected amounted to 24 percent of the estimated cost for all of the American River levee work. We believe that this difference should have (1) alerted the Corps to the possibility that costs were likely to be much higher than it had originally estimated and (2) warranted an update of the Corps’ cost estimate before it awarded the initial contract.

According to a headquarters official, the Corps issued the first contract without updating its total project cost estimate because it would have been impractical to delay the project while the agency revisited cost estimates. Furthermore, according to the Corps, the first contract was expected to be more costly than future contracts because, among other reasons, it involved work on only a small stretch of the levee, which limited possible cost efficiencies. However, because the Corps did not analyze the potential for cost increases for the remainder of the American River levee improvements, it did not determine the likelihood that it would need additional spending authority to complete the project before it awarded the first contract.

The Corps has paid for the significantly increased costs of the American River levee improvements by using funds planned for the Natomas Basin and for the additional American River work authorized in 1999. Although the Common Features Project has two separate components, and Congress approved parts of the project in 2 different years, the project is subject to a single maximum project cost. The Corps has the flexibility to spend Common Features Project funds as it sees fit and is not required to allocate funds in proportion to its original cost estimates for each component.

Following project authorization in 1996, the Corps began to construct the American River levee improvements before the Natomas Basin improvements. Although the Corps exhausted the funds it had originally estimated that it would need to construct the American River levee improvements, it was able to continue implementing the American River work by spending funds it had originally planned to use for the Natomas Basin work. With the authorization of additional work in 1999, effectively raising the project’s maximum cost to about $120 million, the Corps also was able to use funds planned for this work to pay for the increased costs of the American River work authorized in 1996.

After it awarded the first Common Features Project contract, the Corps was not required to inform Congress of project cost increases until it could not contract for additional work without exceeding the maximum project cost. According to the Corps, in March 2001 it briefed a number of Members of Congress on its intention to prepare a report that would evaluate the potential for the cost of the Common Features Project to exceed the project’s maximum cost. However, it was not until February 2002, more than 4 years after it significantly modified the design of the American River levee improvements, that the Corps reported to Congress for the first time that due to significant cost increases, it could not complete the project without exceeding the maximum project cost. By this time, the Corps had spent or awarded contracts for more than twice the amount it originally planned to spend on the American River levee improvements authorized in 1996 and had completed about 90 percent of the work. Furthermore, the Corps estimates that it will spend more than three times its original estimate by the time it completes this work. Because the Corps did not update its cost estimate or report the significant cost increases to Congress until most of the 1996 American River work was complete, Congress did not have the opportunity to determine whether the significantly more expensive levee improvements were still the most appropriate means of providing flood protection for Sacramento.

Corps’ Benefit Estimates for the American River Levee Improvements Are Incorrect

The Corps made mistakes in estimating the benefits for the American River levee improvements because it incorrectly counted and valued the properties that the levee improvements would protect and used an inappropriate methodology to determine the amount of flood damages they would prevent. Seven years after Congress authorized the project, the Corps has not yet prepared an accurate assessment of the benefits of the American River levee improvements. In addition, contrary to its guidance, the benefit estimate the Corps prepared in 2002 did not describe the range of possible benefits and the likelihood that the values in this range would be realized. This additional information, describing the uncertainty of the benefit estimate, would have provided decision makers with information on the likelihood that the project’s benefits would be greater than its costs. Furthermore, the Corps’ three-tiered quality control process did not identify the mistakes that we found during the course of our review.

The Corps Made Mistakes in Counting and Valuing Properties and Determining Flood Damages When Estimating Project Benefits

In its original 1996 analysis of the benefits and costs of the American River levee improvements, the Corps incorrectly counted the residential properties that the proposed levee improvements would protect. As a result, the Corps incorrectly calculated the benefits that these improvements would provide. According to the Corps, the methodology it used to count the number of residential properties in 1996 was “accepted practice and consistent with Corps guidance and technology applicable at the time.” In 2002, the Corps used a different methodology that incorporated new technologies and provided a more precise estimate of the number of properties protected. Using this new approach, the Corps determined that the actual number of residential properties protected by the levee improvements is about 20 percent less than its original estimate. The Corps did not calculate the amount that benefits would decrease due to this change. However, given the small difference between the original estimated annual benefits ($5.6 million) and the annual costs ($5.5 million) of the American River levee improvements, if the Corps had incorporated a more accurate estimate of the property inventory in its 1996 analysis, the benefits of these improvements may have been less than the costs.

For flood protection projects, such as the Common Features Project, the Corps calculates benefits as the dollar value of the physical damages to residential, commercial, industrial, and public properties and infrastructure that the levee improvements prevent. To calculate the reduction in flood damage to properties, the Corps counts the number of properties located in the potential flood area—known as the floodplain— and then assesses the monetary value of the structures and their contents. The Corps uses this information to determine the property damage that would result from floods of various depths and to estimate the impact that the levee improvements would have in preventing this damage.

It is important to remember that, in addition to the economic benefits from preventing property damage, levee improvements may reduce the risk of loss of human lives, which is a benefit that is not included in the Corps’ calculations. According to the Corps, about 305,000 people live within the American River floodplain and the number of lives lost because of levee failure would depend on a variety of factors, such as the size of the flood, warning time, time of day, and availability of evacuation routes. Because of the many factors involved and the lack of historical data, the Corps was not able to estimate the number of lives that would be lost as a result of levee failure and flooding in the Sacramento area.

Although the Corps updated its benefit estimate in 2002 to incorporate the benefits from the new levee improvements authorized in 1999, a Sacramento district official acknowledged that the Corps again made mistakes in estimating the number of properties the levee improvements would protect. For the American River levee improvements authorized in 1999, the Corps identified an area that was larger than the area the levee improvements would actually protect. As a result, the Corps overestimated the number of properties protected and the benefits provided by the work authorized in 1999. According to a Sacramento district official, the Corps currently does not have the information it needs to determine the correct area the levee improvements would protect and therefore is unable, at this time, to provide a reliable estimate of the benefits from the 1999 work.

In addition, the Corps made mistakes in its 2002 analysis in estimating the value of the residential properties the American River levee improvements would protect. The Corps’ policy calls for calculating a property’s value as the cost of replacing the structure less any depreciation, which accounts for a reduction in a structure’s value due to deterioration prior to flooding. Because the Corps had more than 100,000 residential properties to assess and a limited amount of time and resources, it determined depreciated replacement values for a small sample of 365 properties and then used the results to estimate the depreciated replacement values for all properties. However, the Corps did not correctly select the sample of properties. According to members of both the Appraisal Institute and The Appraisal Foundation, to accurately appraise a large number of properties by sampling requires a separate sample for each residential property type, such as single-family homes, condominiums, and apartment buildings. Instead of conducting a separate sample for each type of property, the Corps sampled all property types together and calculated an average depreciated replacement value for all property types. As a result, it is unclear whether the Corps accurately calculated depreciation, which in turn raises questions about its estimates of the value of the residential properties the American River levee improvements would protect.

Moreover, the Corps did not use a consistent, objective appraisal methodology to calculate depreciation for the properties in the sample. Instead, the Corps subjectively determined depreciation. For example, if the Corps determined a structure was in “very good” condition it was assigned a zero percent, 5 percent, or 10 percent level of depreciation. However, the Corps could not provide us with its criteria for assigning the level of depreciation. Furthermore, the Corps’ economists who made these subjective decisions did not consult with the professional appraisers in the Corps’ Sacramento district office to identify alternative appraisal methodologies that may have been more appropriate. According to the Corps, the methods it used to determine depreciation are “standard practice at the Corps and are consistent with prior and existing guidance.” Nonetheless, we believe that the shortcomings identified above raise questions about the accuracy of the Corps’ property value estimates and, in turn, the project benefit estimates that are, in part, based on them. The Corps said it recognizes the need to strengthen its methodologies and is currently developing a new tool to estimate property values.

Finally, the Corps’ 2002 analysis did not use the methodology described in Corps guidance to determine the number of properties that are located in the 100-year floodplain and the damages they would sustain in a 100-year flood. The 100-year floodplain is the land area that may be affected during a flood that has a 1 percent chance of occurring every year. Instead of following Corps guidance by directly counting the properties located in the 100-year floodplain and calculating the damages they would sustain in a 100-year flood, the Corps estimated the damages using a methodology that relied on the results from its incorrect 1996 count of properties. The Corps’ use of this alternative methodology further calls into question the accuracy of its benefit estimate for the American River levee improvements authorized in both 1996 and 1999, which is based in part on this flood damage assessment. The Corps told us that it could have directly counted the properties in the 100-year floodplain but the necessary information was not available in a “user friendly” format, and that the additional effort needed to collect more accurate information was not expected to change the results. As a result, the Corps did not believe this was an effective use of resources. However, the Corps did not provide us with any evidence to support the validity of calculating the 100-year flood damages as it did or to validate its contention that the results would not change if it had used the methodology prescribed in its guidance.

The Corps Has Not Provided Congress with Information on the Range of Possible Benefits from the Levee Improvement Work or the Likelihood They Will Be Realized

The Corps has not followed its policy to provide Congress with an estimate of the range of possible benefits from the American River and Natomas Basin levee improvements and the likelihood that these benefits will actually be realized. In 1996, the Corps established a policy calling for benefit estimates and benefit-cost comparisons for flood protection projects to be reported with their associated probabilities. For example, rather than reporting that the benefits for a particular project are exactly $1.5 million, the Corps could report that it is 80 percent confident that project benefits will be at least $1 million but it is only 30 percent confident that benefits will reach $2 million. The Corps recognizes that this information can assist Congress in understanding the uncertainty involved in achieving various levels of benefits and in determining whether those risks justify funding the project. According to the Corps, it did not estimate a range of benefits for the Common Features Project in 1996 because the computer software used to assess the project’s benefits and costs was developed prior to the 1996 guidance and did not have the capability to calculate a range of values.

However, in its 2002 reanalysis of project benefits, when a new version of the software capable of calculating benefit ranges and probabilities was available and costs for the American River work had significantly increased, the Corps chose not to calculate a range of benefits and instead continued to report a single estimate. Because the Corps’ 2002 estimates of benefits and costs for the American River work were so close in value (1.1 to 1), an analysis of the potential range of benefits would have revealed whether there was a significant probability that project benefits could be lower than the single estimate the Corps reported and perhaps lower than project costs. According to a Sacramento district official, the Corps did not use the new version of its software that could have calculated the range of benefits to maintain consistency with information on flood protection it had previously released to the public. For example, the Corps has reported to the public that the American River levees have about a 1 percent chance of being breached by floodwaters in any given year. This estimate of flood protection could be different if calculated using the newer version of the software. The Corps was concerned that using the newer software would require it to report a different, and perhaps slightly lower, level of flood protection, which would confuse the public. However, by taking this approach, the Corps did not provide Congress with important information about the uncertainty surrounding the amount of benefits the project would provide.

The Corps’ Quality Control Process Did Not Identify Flaws in Its Benefit Analyses

Three organizational levels within the Corps—district, division, and headquarters—reviewed and approved the 1996 and 2002 benefit analyses for the American River component of the Common Features Project, but these reviews did not identify the mistakes that we found. This issue raises questions about the adequacy and effectiveness of the Corps’ review process. We raised similar concerns about the Corps’ review process in our report on the Delaware River Deepening Project, which found significant miscalculations and invalid assumptions in the project’s economic analysis that the Corps did not find during its reviews.

For the Common Features Project, the Corps’ Sacramento district office conducted the 1996 study that analyzed the technical and economic aspects of the proposed project and the 2002 report updating that information. The Corps’ Los Angeles district office reviewed the 2002 economic analysis for technical accuracy. Next, the Corps’ South Pacific division reviewed the analysis; although, following the Corps’ policy, it did not review the district’s work for technical accuracy or verify the underlying analysis. Rather, the division checked that the district’s reports had undergone a technical review, and that the district had issued a quality control certification report with the necessary district office-level approvals. The division then forwarded the project to headquarters. Corps headquarters also did not conduct a technical review of the analysis. Rather, headquarters checked that the district’s report adhered to Corps policies for conducting a benefit-cost analysis and addressed any concerns headquarters had raised.

These review processes, however, were ineffective in detecting and correcting the mistakes in the benefit analyses we identified. For example, for the 2002 study, we found no indication that the mistakes made in calculating the number and the value of residential properties or the mistake made in calculating flood damages were detected during the Corps’ review process. For the 2002 analysis of the American River levee improvements, a Corps economist from another district independently reviewed the benefits analysis. However, the review was not comprehensive enough to sufficiently identify methodological problems. The review primarily focused on process-oriented issues, such as assessing whether the Sacramento district conducted certain analyses, rather than examining the technical aspects of how the analyses should have been and were conducted.

Conclusions

It is critical that decision making and priority setting be informed by accurate information and credible analysis. Reliable information from the Corps about the costs and benefits for the American River component of the Common Features Project has not been present to this point. The analysis on which Congress has relied contained significant mistakes. And of most relevance today, the analyses for the remaining work do not provide a reliable economic basis upon which to make decisions concerning the American River levee improvements authorized in the WRDA of 1999. To provide a reliable economic basis for determining whether these improvements are a sound investment, the Corps’ analysis needs to adequately account for the risk that project costs could increase substantially, correctly count and value the properties the project would protect, and include information on the range of potential project costs and benefits.

Moreover, because the Corps has not made some critical decisions regarding the Natomas Basin work, it is not yet known whether the Corps will be able to identify cost-effective flood protection options for this area. Specifically, the Corps has not determined whether it will (1) conduct a cost risk analysis of its current plan to identify its exposure to potentially significant cost increases or (2) evaluate the costs and benefits of alternatives to the current levee improvement plan to identify the most cost-effective flood protection option. In addition, identifying cost-effective flood protection involves reporting the range of potential project benefits and the probability of achieving them, which the Corps has not done for the Natomas Basin work. If the Corps begins implementing the authorized Natomas Basin work before it completes a comprehensive, accurate cost- benefit analysis, significant unanticipated cost increases could materialize, as they did with the American River work. Finally, for Congress to have confidence that the Corps’ economic analyses have been prepared accurately, the Corps’ quality control process would need to be sufficiently independent and detailed to identify the types of mistakes that our review revealed.

Recommendations for Executive Action

For the American River levee improvements authorized in 1999 and for the planned Natomas Basin work, we recommend that the Secretary of the Army direct the Corps of Engineers to determine whether it is appropriate to conduct risk analyses of project costs and document the basis for that decision in its project files; report information to Congress on the range of potential project benefits and the probability of achieving those benefits, as called for in the Corps’ guidance, in future benefit-cost analyses; and arrange for a credible, independent review of the completeness and accuracy of the revised benefit-cost analyses.

For the American River project component, we also recommend that the Secretary of the Army direct the Corps of Engineers to reanalyze the benefits of the improvements authorized in the WRDA of 1999, correcting for the mistakes made in counting and valuing properties and the inappropriate methodology used to calculate flood damages.

Additionally, for the Natomas Basin project component, we recommend that the Secretary of the Army direct the Corps of Engineers to analyze the costs and benefits of alternatives to the current levee improvement plan and identify the flood protection plan that provides the greatest net benefits and submit a report to Congress that includes a cost estimate for all of the planned Natomas Basin work, and wait until Congress authorizes funding that is based on the report before beginning construction of any Natomas Basin levee improvements.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of the Army for review and comment. In commenting on the draft report, the Army concurred with all of our recommendations. Perhaps most significantly, the Army acknowledged that on the basis of the Corps’ experience in constructing the American River levee improvements, there is a potential for substantial cost increases for the Natomas Basin levee improvements, and therefore the Corps needs to investigate a wider array of alternatives for providing flood protection for the Natomas Basin. In addition, although the Army concurred with our recommendation to reanalyze the benefits of the improvements added to the American River component of the project in 1999, it contended that the Corps has already completed the reanalysis. We disagree. In 2002, the Corps prepared an analysis of the economic benefits for the work added to the project in 1999. However, our review found several mistakes in this analysis, including mistakes in counting and valuing properties and using an inappropriate methodology to calculate flood damages. We continue to believe that before the Corps begins construction of the work added to the American River project component in 1999, it should reanalyze this work to ensure it is cost beneficial.

The Army stated that the report does not recognize the significant role Congress played in 1999 by adding additional work to the project and providing funds for construction before the Corps had developed reliable cost estimates, which created the situation of which our report is critical. By focusing its comment on the relatively small amount of work added in 1999, the Army avoided the main issues regarding the American River levee improvements discussed in our report. Specifically, that (1) the costs for the American River component of the project approved in 1996 are more than triple the original estimate; (2) the Corps had information, before construction began, that should have alerted it that costs would likely increase greatly; and (3) the Corps should have communicated this information to Congress at that time, but it did not. Furthermore, the additional funding provided by Congress for the work authorized in 1999 has not been used for that purpose, but rather has been used to fund the cost overruns for the work authorized in 1996. The full text of the Army’s comments, and our responses to them, are presented in appendix III.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the date of this letter. At that time, we will send copies of this report to the appropriate congressional committees, other interested Members of Congress, and the Secretary of the Army. We also will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you, or your staff, have any questions about this report, please contact me at (202) 512-3841. Key contributors to this report are listed in appendix IV.

Scope and Methodology

To determine the reasons for the cost increases for the Common Features Project, we obtained the key cost estimation documents prepared by the U.S. Army Corps of Engineers’ (the Corps) Sacramento district office. Specifically, we obtained the Corps’ 1996 Supplemental Information Report, American River Watershed Project; the 1997 Addendum to the Supplemental Information Report; the 2002 Second Addendum to the Supplemental Information Report; and other related documents. We reviewed the Supplemental Information Report, which examined a number of different flood protection alternatives, because it provided the foundation, including cost estimates, for the project elements that the Corps later grouped together as the Common Features Project. The Addendum to this report documented the Corps’ first cost estimate that specifically and exclusively addressed the Common Features Project and included separate costs for both the American River component and the Natomas Basin component of the project. We reviewed the Second Addendum, the Corps’ most current official cost estimate, to establish the amount of and the reasons for the increased costs. We also analyzed construction contracts to determine the cost of responding to accidents constructing the levee improvements authorized in 1996. We calculated the extent of inflation for both components of the project, using the Corps’ Civil Works Construction Cost Index System and a cost index from the Office of Management and Budget. Finally, we discussed the reasons for the cost increases with economists, cost estimators, project managers, engineers, and other staff from the engineering, constructions operations, and planning divisions of the Corps’ Sacramento district office.

To determine whether the Corps analyzed the likelihood of significant cost increases for the project and reported them to Congress in a timely manner, we reviewed the Corps’ (1) policy regarding the use of cost risk analysis in estimating costs for civil works projects (Engineer Regulation 1110-2-1302) and (2) requirements for updating project cost estimates and informing Congress of cost increases (Engineer Regulation 1105-2-100). We also reviewed a document from the Corps’ Institute for Water Resources on incorporating risk and uncertainty into cost estimation. We examined the American River levee improvement construction contracts to determine when the Corps became aware of cost increases for this component of the project. In addition, we reviewed the Corps’ annual budget documents related to the Common Features Project, which contained information on the project’s status and any changes or cost increases. We examined the Corps’ cost estimates from 1996, 1997, and 2002 for compliance with relevant Corps cost estimating guidance and to determine if the Corps provided Congress with accurate information about significant expected cost increases. Finally, we discussed the Corps’ cost estimating procedures and awareness of likely cost increases with cost estimators, project managers, and other staff from Corps headquarters and the Sacramento district office.

To determine whether the Corps correctly estimated the economic benefits of the American River levee improvements, we reviewed the extent to which the Corps followed accepted economics practices and whether the major assumptions used in the analysis were reasonable and well supported. We obtained the Corps’ 1996, 1997, and 2002 economic analyses for the Common Features Project and discussed the sources of these data and conduct of the analyses with the Corps economists responsible for preparing them. We also discussed the basis for the hydrologic and engineering assumptions used in the economic analysis with the Corps specialists who provided this information. In addition, we obtained the Corps’ guidance (Engineer Regulations 1105-2-100 and 1105-2-101 and Engineer Manual 1110-2-1619) on the accepted economic and engineering methodologies for incorporating risk and uncertainty into benefit estimation. To verify and supplement the information we received from officials in the Corps’ Sacramento district office, we spoke with, among others, Corps officials at the Hydrologic Engineering Center and the Institute for Water Resources and experts in real estate appraisal from the Appraisal Foundation and the The Appraisal Institute. Where we identified problems that affected the accuracy of the benefit analysis, we discussed them with the responsible Corps staff and considered any new data or revisions that they provided. Finally, we identified the roles and responsibilities of the Sacramento district office, South Pacific division, and headquarters in the Corps’ internal quality control process for the Common Features Project. We also obtained copies of the quality control reviews and the reviewers’ comments on the economic analysis and discussed the comments and their resolution with Corps officials.

We conducted our review from September 2002 through September 2003 in accordance with generally accepted government auditing standards.

Conversion of Costs to Constant Dollars

In this report, unless otherwise noted, we present costs in the dollar values for the years in which they were estimated, not in constant dollars. For example, the Corps estimated the original cost of the project as $57 million in 1996, and that is how we present it in this report. We did not adjust the costs to constant dollars to account for inflation to maintain consistency with the figures in published Corps reports on the Common Features Project. However, table 3 shows the Corps’ 1996 cost estimates for key components of the Common Features Project and also shows the same estimates adjusted to 2002 constant dollars to account for inflation.

Comments from the Department of the Army

GAO Comments

The following are GAO’s comments on the Department of the Army’s letter dated September 22, 2003. 1. Although the Army asserted that we made some factual errors, its subsequent comments failed to identify any specific factual errors. 2. The Army believes that the report does not recognize the significant role Congress played in 1999 when it added additional work to the project and authorized funds for construction before the Corps had developed reliable cost estimates. While the Congress did add work to the Common Features Project in the Water Resources Development Act (WRDA) of 1999 without a Corps report, the cost of this work is relatively small in comparison to the work authorized in 1996. We believe the Army’s comment is not relevant to the main focus of our report, which is the significant cost increases for the work the Corps recommended and the Congress authorized in 1996. For example, the costs for the work the Corps recommended on the American River more than tripled from $44 million in 1996 to $143 million in 2002. In contrast, the estimated cost for the work on the American River levees the Congress added in 1999 is about $15 million. We believe our report accurately reflects the limited impact the addition of work in 1999 had on the American River component of the project’s overall cost. Furthermore, the additional funding provided by Congress for the work authorized in 1999 has not been used for that purpose, but rather has been used to fund the cost overruns for the work authorized in 1996. 3. The Army stated that the consistent provision of funds to the Corps by Congress, at or exceeding the Corps’ budget request, created the situation of which our report is critical. We do not agree. Two of the main issues in our report are that the costs of the American River component of the project nearly tripled due to design changes, and that the Corps began construction of the American River levee improvements without analyzing the likelihood of these cost increases or reporting the potential cost increases to Congress. The fact that Congress provided funding for the project does not absolve the Corps of its responsibility to communicate project cost increases in a timely manner. 4. The Army implied that Congress was informed of potential cost increases for the Common Features Project during the yearly appropriations process. This is not the case on the basis of our review of all of the Corps’ submissions for the annual appropriations process from 1997 through 2001. As our report states, it was not until February 2002, more than 4 years after it had significantly modified the design of the American River levee improvements, that the Corps informed Congress for the first time of the significant cost increases for the American River component of the project. 5. The Army stated that the levee improvements were not originally designed to withstand the destructive effect of seepage and that this design was not an error. Rather, an unknown condition (i.e., the potential for destructive seepage under the levees) resulted in design changes and increased costs. Our report does not criticize the Corps for not anticipating the need for a levee improvement design that would stop seepage under the levees. We acknowledge that the flood of January 1997 caused the Corps to change the design of its levee improvements. However, as our report notes, the Corps did not develop new cost estimates after making these design changes and did not communicate the resulting significant cost increases to Congress in a timely manner. 6. We do not consider the separable elements of the Common Features Project as separate projects. This report makes clear that there is one Common Features Project comprised of an American River component and a Natomas Basin component. 7. We agree with the Army that, in 1996, the Corps was not aware of any significant areas of cost uncertainty for the proposed American River levee improvements. However, as the Army recognizes, the flood of January 1997 showed that the Corps’ design for the levee improvements should be significantly modified. After making these design changes, though, the Corps did not estimate the potential for cost increases due to tripling the depth of some cut-off walls or closing the gaps in cut-off walls at bridges and other areas. These design changes eventually added $76 million to the cost of the project. 8. The Army stated that the Corps believes that its review process results in decision documents that form the basis for sound recommendations. However, in two recent cases, we found that the process did not serve its intended purpose. As this report documents, the Corps’ review process was ineffective in detecting and correcting the mistakes in the benefit analyses we identified. We raised similar concerns about the review process in our June 2002 report on the Delaware River Deepening Project. 9. We did not recommend that the Corps reanalyze the costs and benefits of the work authorized in 1996. We agree that a reanalysis of this work, which is nearly complete, would be of little value. However, we continue to believe that a reanalysis of the economic benefits from the work authorized in 1999 is necessary because the Corps’ initial analysis contained significant mistakes and construction of the work has not yet begun. Before beginning construction of this work, the Corps should verify that the work is in fact cost beneficial. In addition, the Corps should arrange for a credible, independent review of the completeness and accuracy of its reanalysis. 10. The Army contends that the Corps has already completed the reanalysis we recommended of the work added to the American River component of the project in 1999. We disagree. The Corps analyzed the economic benefits for the 1999 work added to the project for the first and only time in 2002. Our review found several problems with the Corps’ 2002 analysis of the benefits from this work. For example, we found that the Corps had made mistakes in how it counted and valued properties and had used an inappropriate methodology to calculate flood damages. As a result, the Corps has not yet prepared an accurate assessment of the benefits resulting from the 1999 work. The Corps has not begun any construction for the work authorized in 1999, and it is not currently known if the benefits provided by this work are greater than the costs. Consequently, we recommend a reanalysis of these benefits in order to correct the mistakes that we identified. 11. The Army stated that the Corps does not conduct individual real estate appraisals to determine the value of each property that could be damaged in a flood. Our report does not suggest that the Corps should conduct such appraisals. Rather, we identified weaknesses in the sample the Corps used to estimate property values and its methodology for calculating depreciation for the properties in the sample. For example, to accurately appraise a large number of properties by sampling requires a separate sample for each residential property type, such as single-family homes and apartment buildings. However, the Corps sampled all property types together. In addition, the Corps did not use a consistent objective appraisal methodology to calculate depreciation for the properties in the sample. These weaknesses raise questions about the accuracy of the Corps’ property value estimates and the project benefit estimates that are, in part, based on them. 12. The Army claims that there are approximately 163,000 residential structures in the 400-year floodplain. This is not correct on the basis of the Corps’ most current analysis. The estimate of 163,000 residential structures comes from the Corps’ 1996 economic analysis. However, in 2002, the Corps updated its analysis and found that it had overestimated the number of residential structures in 1996. The Corps’ 2002 analysis estimated that there were 115,347 residential structures in the 400-year floodplain.

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual above, Jeff Arkin, Chuck Barchok, Judy Hoovler, Richard Johnson, Mark Metcalfe, Ryan Petitte, and Stephen Secrist made key contributions to this report.

GAO’s Mission

The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs