California High-Speed Passenger Rail: Project Estimates Could Be Improved to Better Inform Future Decisions
Published: 2013-03-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The planned 520-mile California high-speed rail project, which would link San Francisco to Los Angeles, would be designed to operate at speeds up to 220 miles per hour. At an estimated cost of $68.4 billion (in year-of-expenditure dollars), it is expected to be one of the most expensive transportation projects undertaken in the United States. The Authority is responsible for implementing the project and federal funding is being provided from the FRAs High-Speed Intercity Passenger Rail program. GAO reviewed (1) the reliability of project cost estimates, (2) the reasonableness of revenue and passenger rail ridership forecasts, (3) the risks attendant with the projects funding plan, and (4) the comprehensiveness with which the projects economic impacts were identified.
GAO obtained documents from and conducted interviews with federal officials and officials from the Authority related to cost, financing, ridership and revenue modeling and estimation, and business plans and analyses related to potential economic impacts. GAO also interviewed state and local officials as well as the projects peer review group members.
section_title: What GAO Found
paragraphs: The California High-Speed Rail Authority (Authority) met some, but not all of the best practices in GAO's Cost Estimating and Assessment Guide (Cost Guide) for producing cost estimates that are accurate , comprehensive , well documented , and credible . By not following all best practices, there is increased risk of such things as cost overruns, missed deadlines, and unmet performance targets. The Authority substantially met the criteria for the accurate characteristic by, for example, the cost estimate's reflecting the current scope of the project. However, the Authority partially met the criteria for the other three characteristics since the operating costs were not sufficiently detailed ( comprehensive ), the development of some cost elements were not sufficiently explained ( well documented ), and because no systematic assessment of risk was performed ( credible ). The Federal Railroad Administration (FRA) issued limited guidance for preparing cost estimates, and this guidance did not reflect best practices in the Cost Guide . The Authority plans to improve its cost estimates.
GAO found the Authority's ridership and revenue forecasts to be reasonable; however, additional updates are necessary to refine the ridership and revenue model for the 2014 business plan. GAO also found the travel-demand-modeling process used to generate these forecasts followed generally accepted travel- demand-modeling practices. For example, the Authority revised several assumptions, such as gasoline price forecasts, to reflect changes in current and anticipated future conditions. However, additional updates, such as the development of a new travel survey, will be necessary to further refine these forecasts and improve the model's utility to make future decisions. External peer review groups have also recommended additional updates.
The project's funding, which relies on both public and private sources, faces uncertainty, especially in a tight federal and state budget environment. Obtaining $38.7 billion in federal funding over the construction period is one of the biggest challenges to completing this project. In the latter stages, the Authority will also rely on $13.1 billion in private-sector financing, but will require more reliable operating cost estimates and revenue forecasts to determine whether, or the extent to which, the system will be profitable. The Authority's plan recognizes the uncertainty of the current funding environment and is building the project in phases. The Authority has also identified an alternative funding source. However, that funding source is also uncertain.
The Authority did a comprehensive job in identifying the potential economic impacts of the high-speed rail project. This includes identification of user impacts, such as effects on travel time reliability, and non-user impacts, such as effects on highway congestion. However, the nature of specific economic impacts will depend on a number of factors, including future project decisions. GAO also found limitations in the Authority's benefit-cost analysis of the project that could limit its usefulness to decision makers. Finally, GAO found that construction of the high-speed rail project will not eliminate the need for additional improvements to meet future statewide-travel demand, but current statewide- transportation assessments and planning have given little consideration to this issue.
section_title: What GAO Recommends
paragraphs: To produce reliable cost estimates, FRA should improve its guidance so it is in line with the best practices in GAO's Cost Guide . The Department of Transportation did not agree or disagree with the recommendation but said, with further analysis, applying the Cost Guide would be feasible. The Authority said it will incorporate many of the report's findings into future cost and ridership estimates.

section_title: Letter
paragraphs: California’s high-speed rail project is poised to be the first rail line in the  United States designed to operate at speeds up to 220 miles per hour. At  an estimated cost of $68.4 billion (in year-of-expenditure dollars), it is  expected to be one of most expensive transportation projects undertaken  in the United States. The Federal Railroad Administration (FRA) has  obligated nearly $3.5 billion in grants to begin construction later this year.  To complete the project, the state of California expects to rely on  significant public-sector funding, in addition to private funding, through the  project’s anticipated completion date in 2028.
Over the past two decades, there has been interest in developing high- speed rail between San Francisco and Los Angeles. Rail proponents  argue that California faces significant transportation congestion and that  high-speed rail can help alleviate the need to expand highway and air  infrastructure. Proponents also argue that the corridor has characteristics  that make high-speed rail viable: it connects highly populated  metropolitan centers along a corridor that is currently highly traveled by  air, rail, and automobile passengers. Opponents of the project argue that  it is too expensive, especially with tight federal and state budgets, and  has not sufficiently demonstrated that the project will be financially viable.  As we reported in 2009, completing a high-speed rail project requires  significant and sustained political, public, and financial support given the  high costs and multiple stakeholders involved. In recent years, legislation  has been enacted at the state and federal level that has allowed  California’s project to move forward by appropriating funding for high- speed rail. In 2008, California’s voters approved a $9.95-billion bond  measure, and in 2012, the state legislature appropriated over $4 billion  from this bond to the project and connecting transit upgrades. At the  federal level, the Passenger Rail Investment and Improvement Act of  2008 (PRIIA) authorized development of high-speed intercity passenger- rail corridors and the American Recovery and Reinvestment Act of 2009  (Recovery Act) appropriated $8 billion to fund development of these  corridors and intercity passenger-rail projects. In June 2009, the FRA  established the High Speed Intercity Passenger Rail (HSIPR) program  that provides discretionary grants for high-speed or intercity passenger  rail projects. In December 2009, the Fiscal Year 2010 Department of  Transportation (DOT) appropriations act appropriated $2.5 billion for the  HSIPR program ($400 million of which was later rescinded).
The California High-Speed Rail Authority (Authority)—a state agency  formed in 1996—has been tasked with implementing the project and has  published four successive business plans detailing the project’s cost  estimates, financing plan, ridership projections, and anticipated economic  impacts. Since 2009, the scope of the project has changed along with the  project’s estimated costs. In addition, questions have been raised about  how the project will be funded beyond the current $11.5 billion in state  and federal funding commitments, as well as the project’s ability to attract  sufficient ridership and bring economic benefits to the cities and  communities along the corridor. This report assesses: (1) the reliability of  the Authority’s estimates of the project’s costs, (2) the reasonableness of  the Authority’s passenger rail ridership and revenue forecasts, (3) the  risks attendant with the Authority’s funding plan for the project, and (4) the  comprehensiveness with which the Authority identified potential economic  impacts of the project. As such, we are assessing the quality of the  available information used by policymakers. We are not evaluating the  merits of the project itself, which should be considered in light of whether  this project best meets the transportation needs of the estimated 51  million Californians in 2050.
We obtained information from numerous sources to address our  objectives, including the models underlying the various estimates and  interviews with Authority officials and contractors that prepared them, the  peer review group members that reviewed them, outside critics and  supporters, high-speed rail experts and FRA officials. We based our  analysis on estimates underlying the Authority’s April 2012 revised  business plan.
To assess the reliability of the project’s cost estimates, we compared the  Authority’s cost estimating approach to GAO’s best practices found in the  2009 GAO Cost Estimating and Assessment Guide (Cost Guide). We  used the Cost Guide to conduct an in-depth evaluation of the methods  and assumptions used by the Authority to develop cost estimates of  Phase 1 of the project; our goal was to determine whether best practices  were employed that would help ensure that the cost estimates are well- documented, comprehensive, accurate, and credible. The Cost Guide  was developed for cost estimators to assist them in preparing reliable  estimates of capital program costs. While GAO’s Cost Guide provides  criteria to evaluate the Authority’s methods for developing its cost  estimates, we cannot use it to determine whether or not the Authority’s  cost estimate is precise. Nor can we use the Cost Guide to evaluate the  benefits of this project in comparison to other transportation  improvements. Such an evaluation would require a cost-benefit and other  analyses, which we did not conduct as part of this work. To assess the  reasonableness of the ridership and revenue estimates, we analyzed the  extent to which the Authority’s ridership model methodology adhered to  FRA guidance and generally accepted practices and reviewed peer  review reports assessing the model’s methodology. We also interviewed  Authority officials, their contractor, and peer review panel members. To  assess the Authority’s financing plan, we reviewed the plan, conducted  interviews with Authority and other state and federal officials, and  reviewed literature and other information on financing for high-speed rail  projects in other countries as well as large transportation projects in the  United States. To assess the comprehensiveness with which the  Authority identified potential economic impacts of the project, we  compared the impacts identified in the April 2012 revised business plan  and related economic analyses prepared by the Authority with criteria for  such assessments contained in DOT’s program-funding notices,  applicable legislation, and best practices found in academic literature and  federal guidelines. For purposes of this report, potential economic  impacts include those impacts on users of a transportation system such  as travel time savings and those impacts to non-users of a system such  as highway congestion or noise levels. (For more detailed information on  our objectives, scope, and methodology, see appendix I.)
We conducted this performance audit from February 2012 to March 2013  in accordance with generally accepted government auditing standards.  These standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: In 2008, California voters approved Proposition 1A, which authorized  $9.95 billion in state bond funding for construction of the California high- speed rail system and connection improvements to existing passenger  rail systems. Proposition 1A established several requirements for this  high-speed rail system, such as that the rail system must be capable of  sustained operating speeds of no less than 200 miles per hour, and once  built, must operate without a public subsidy. The planned 520-mile high- speed rail system will operate between San Francisco and Los Angeles at  speeds up to 220 miles per hour (see fig.1). The Authority is the state  entity charged with planning, designing, and constructing the California  high-speed rail system. The Authority has a nine-member policy board  appointed by the California legislature and Governor, and a staff of  approximately 55 state employees who oversee, among other things,  contracts for environmental review, preliminary engineering design,  preliminary right-of-way acquisition tasks, contractor oversight and other  activities.
Construction of the California high-speed rail project is expected to occur  in phases beginning with a 130-mile section from just north of Fresno,  California, to just north of Bakersfield, California. Construction will begin  in the Central Valley and proceed to other portions of the corridor as  funding is available. The Central Valley is the furthest advanced in terms  of design and engineering work, as well as environmental reviews. For  example, FRA approved a preferred route alignment for the Merced to  Fresno, California, portion of the corridor in September 2012. According  to FRA, the federally funded portion of the project in the Central Valley  has more complete cost estimates than subsequent segments given that  has more complete cost estimates than subsequent segments given that  preliminary engineering and environmental reviews are complete or  nearly complete. Other project segments, however, are in different stages  of development and have different levels of information from which to  develop cost estimates. In July 2012, the California legislature  appropriated $4.7 billion of the $9.95 billion in state bond funds, including  $2.6 billion for construction of the high-speed rail project and $1.1 billion  for upgrades in the San Francisco peninsula and in the Los Angeles basin  (commonly referred to as the “bookends”). The process of acquiring  property for the right-of-way and construction has begun. Requests for  proposals to select construction contractors and right-of-way acquisition  were issued in March and September 2012, respectively. In addition, in  January 2013, the Authority awarded a project and construction  management contract for the initial phase of California’s high-speed rail  project. According to the Authority, a design-build contract for the first  construction (covering approximately 30 miles) is expected to be awarded  in June 2013 with construction planned to commence in summer 2013.  (See fig. 2).
The federal government has committed funding to the project. The FRA  awarded the state approximately $3.3 billion in capital construction funds  and $231 million for environmental review and preliminary engineering  work under the HSIPR program for a total of approximately $3.5 billion.The California high-speed rail project is the largest recipient of HSIPR  funds, with about 35 percent of program funds obligated. Most of the  HSIPR money awarded to the project was appropriated by the Recovery  Act and, in accordance with governing grant agreements, must be  expended by September 30, 2017.million in fiscal year 2010 funding was awarded to the project by FRA and  is to remain available until expended. While the funds remain available  until expended under FRA’s fiscal year 2010 appropriation, the governing  grant agreements specify the schedule for expenditure of funds.
In addition, approximately $945  Even though some funding has been committed, additional funding will be  needed to complete the project. For example, according to the Authority’s  finance plan, over $38 billion in federal funds and over $4 billion from  Proposition 1A proceeds will be needed to complete Phase 1 of the  project. In addition, the Authority is also planning to obtain another $13.1  billion in private-sector capital to help defray the cost of construction after  the initial operating segment is completed in 2028.
As the federal agency responsible for awarding and overseeing grants to  HSIPR applicants, the FRA established guidance outlining requirements  and procedures and developed an oversight program to ensure that the  project’s goals, objectives, performance requirements, budgets, and other  related program criteria are being met. Thus far, FRA’s guidance to  HSIPR grant applicants has been limited with respect to developing cost  estimates and ridership and revenue forecasts. The Department of  Transportation’s Office of Inspector General (DOT OIG) noted that the  lack of clear, detailed guidance allows for analyses of widely varying  quality, making it difficult to accurately assess whether projects will be  viable or require substantial financial support and has recommended FRA  improve its guidance. In addition, we have previously reported that a  clear definition of the federal role, goals, and objectives in conjunction  with a robust grant oversight program, is critical to FRA making sound  federal investments in high-speed rail projects.
The Authority is required to prepare and periodically submit to the state  legislature a business plan, which must identify, among other things,  ridership estimates, operating and maintenance costs, and the source of   The Authority’s next business plan is expected to be  project funding. released in 2014. Several groups have been established to review and  comment on the estimates presented in the Authority’s business plans. For  example, an independent peer review group (PRG) was established in  accordance with California law to evaluate the Authority’s funding plans and  report its judgment as to the feasibility and the reasonableness of the plans,  appropriateness of assumptions, analyses and estimates, and any  observations or evaluations it deems necessary. In addition, the Authority  convened a Ridership and Revenue Peer Review Panel (Panel) to review  the Authority’s ridership and revenue-forecasting process and outcomes and  conduct an in-depth review of the models used to estimate ridership and  revenue and the forecasts derived from them. The California state auditor is  required to periodically audit the Authority’s use of bond proceeds.
In response to the initial high estimated cost of building the San Francisco  to Los Angeles route—about $98 billion—and other criticisms of the  Authority’s November 2011 draft business plan, the project underwent  substantial revision for the April 2012 revised business plan. Most  significantly, the Authority scaled back its plans to build dedicated high- speed rail lines over the project’s entire length. Instead, the April 2012  revised business plan adopted a “blended” system in which high-speed rail  service would be provided over a mix of dedicated high-speed lines and  existing and upgraded local rail infrastructure (entirely at the bookends of  the system on the San Francisco peninsula and in the Los Angeles basin).  The estimated cost in the April 2012 revised business plan is $68.4 billion.
The ridership and revenue forecasts in the April 2012 revised business  plan also changed from the November 2011 plan. For example, in the  November 2011 draft business plan, the Authority provided low and high  estimates of ridership in 2030 of 14.4 million and 21.3 million passengers.  In the April 2012 revised business plan, these estimates increased by 12  and 26 percent, respectively, to 16.1 million and 26.8 million. Revenues  similarly increased between the two plans from a low and high estimate of  $1.05 billion and $1.56 billion in the November 2011 plan to $1.06 billion  and $1.81 billion in April 2012. The ridership and revenue estimates  increased because among other things, a “one-seat” service from San  Francisco to Los Angeles would begin sooner under the blended  approach than the original solely dedicated lines approach. However, by  2040 ridership forecasts under the blended approach are less than the  original full build approach. The range between the high and low  estimates also increased between the November and April plans  reflecting a greater degree of uncertainty in the estimates. We have  previously reported that forecasting ridership and revenue is a complex  and iterative process and that early stage estimates should be based on  the best available data and what is initially known about the proposed  project. As additional information becomes available, the Authority’s  model used to produce the forecasts is intended to be updated.
Development of the high-speed rail system has been controversial with  many strongly held beliefs among the numerous supporters and opponents  of the project. Supporters have cited the need for high-speed rail to  address growing congestion concerns, particularly in the metropolitan  areas, and to address future transportation demands. Supporters have  noted that California’s expected population growth—which is expected to  increase from 38 million in 2012 to an estimated 51 million Californians in  2050—and economic growth will continue to place more demands on  California’s transportation infrastructure requiring that significant new  capacity be added to its transportation network. Further, supporters argue  that the cost of expanding the current network of highways and airports to  meet current and future transportation needs is cost prohibitive and would  be detrimental to air quality and that high-speed rail will increase economic  development in local communities to be served by high-speed rail and  generate new jobs. In addition, supporters also note that several critical  airport and highway expansions are infeasible due to land constraints,  particularly at key airports and urban segments of highways. Opponents of  the plan have argued, among other things, that the cost of the high-speed  rail system is too great and that future funding for the system is too  uncertain given the current fiscal environment. Opponents have also raised  concerns about the credibility of the ridership and revenue forecasts  presented in the Authority’s business plans and specifically, the system’s  ability to attract the ridership levels needed to avoid public operating  subsidies. Local communities and property owners in California’s Central  Valley have also raised concerns about the project and its potential impact  to the agriculture sector in the region. As we reported in our December  2012 testimony, the Authority will face several challenges with acquiring  rights-of-way in a timely manner, including potential construction delays as  well as additional project costs. Timely right-of-way acquisition will be  critical since some properties are in priority construction zones. Property to  be acquired will include homes, businesses, and farmland. Not having the  needed right-of-way could cause delays and add to project costs. There  are a total of approximately 1,100 parcels to be acquired for the first  construction segment; all of which are in California’s Central Valley.  According to Authority officials, although the Authority may face challenges  in acquiring right-of-way, they have built contingencies for time and cost  into their acquisition plan.
section_title: Authority Substantially or Partially Met GAO’s Best Practices for Producing Reliable Cost Estimates, but Can Make Improvements
subsections: section_title: Reliable Cost Estimates Are Critical to Successful Project Planning, Funding, and Oversight, but Little FRA Guidance Is Available
paragraphs: The Authority estimates that Phase 1 of the high-speed rail project in  California will cost $68.4 billion to construct and hundreds of millions of  dollars to operate and maintain annually. Since the project’s financing  plan, as articulated in the April 2012 revised business plan, will depend  on an additional estimated nearly $38.7 billion in federal funds, it is vital  that the Authority, FRA, and Congress be able to rely on these cost  estimates for project planning, funding, and oversight. In addition,  because the value of potential private investment depends on the cost of  operating the system, it is vital that the Authority and the private sector be  able to rely on the operating cost estimate. Given that our past work on  high-speed rail projects around the world has shown that projects’ cost  estimates tend to be underestimated, ensuring the reliability of the  estimates is critical to the success of this project.
FRA provided limited guidance to grant applicants, including the  Authority, about preparing cost estimates. FRA grant applicants were  required to submit detailed capital cost estimates and high-level operating  cost estimates, but FRA did not provide guidance on how applicants  should produce these cost estimates to help ensure reliability. Moreover,  the limited guidance that was provided did not reflect best practices  included in our Cost Guide. FRA officials acknowledged that they  specified the categories and types of costs to be estimated, not how  applicants should prepare these cost estimates. FRA officials told us that  they did not provide prescriptive guidance to grantees in preparing cost  estimates because of the Recovery Act requirement to begin funding  activities quickly following the enactment of the act in February 2009. In  addition, FRA noted that the first two rounds of the HSIPR program were  open to a wide range of project types and the level of detail necessary for  an individual station is different from the level of detail necessary for a  large, long-term corridor program like California.
According to FRA officials, the Authority’s application complied with the  HSIPR grant application requirements. FRA found the cost estimates to be  reasonable based on their comparison of the Authority’s cost estimates (on  a unit cost basis) to other rail projects in the United States and abroad. The  Authority and its contractor told us that, in the absence of specific guidance  on preparing cost estimates, they relied on their professional experience  supplemented by available cost-estimating guidance from the Federal  Transit Administration (FTA) which they thought was the most applicable  guidance available. For example, since FRA’s guidance did not require the  Authority to perform an independent cost estimate (this involves a  comparison of the Authority’s original cost estimates to those performed by  an independent entity), the Authority turned to FTA guidance to provide  direction on how and when to conduct an independent cost estimate.
We evaluated the Authority’s cost estimates against GAO’s Cost Guide,  which details best practices for generating high-quality cost estimates at  all levels of government. While not required by FRA, the best practices  identified in our Cost Guide help estimators develop reliable cost  estimates, which have the four following characteristics:
An accurate cost estimate is unbiased, not overly conservative or  overly optimistic, and based on an assessment of most likely costs.
A credible cost estimate discusses any limitations of the analysis from  uncertainty or biases surrounding data or assumptions.
A comprehensive cost estimate ensures that costs are neither omitted  nor double counted.
A well-documented cost estimate is thoroughly documented, including  source data and significance, clearly detailed calculations and results,  and explanations for choosing a particular method or reference.
Ensuring that cost estimates reflect these four characteristics helps  minimize the risk of cost overruns, missed deadlines, and unmet  performance targets. The Cost Guide also provides criteria for evaluating  cost estimates to determine whether they exhibit these characteristics.  We have previously applied the Cost Guide in reviewing several  transportation and infrastructure projects and we applied it in our review  of the Authority’s cost estimates.
section_title: Authority Substantially or Partially Met Best Practices in our Cost Guide for Producing Reliable Cost Estimates
paragraphs: The Authority substantially met best practices in our Cost Guide for  producing accurate cost estimates, but only partially met our best  practices for producing comprehensive, well documented, and credible  estimates. By not following all best practices, there is increased risk of  such things as cost overruns, missed deadlines, and unmet performance  targets.
Our assessment of the Authority’s $68.4 billion construction and operating  cost estimates for the high-speed rail project is summarized in table 1.  Our assessment is discussed in more detail in appendix 2.
We found that the Authority substantially met best practices for  developing accurate cost estimates. Consistent with best practices, the  estimates reflect the new “blended” system, which will rely, in part, on  existing rail infrastructure; they contained few, if any, mathematical errors;  and they have been adjusted for inflation. Furthermore, the Authority’s  contractor used a construction industry database of project costs  supplemented with actual bid-price data from other transportation  infrastructure projects. While the Authority generally complied with best  practices for producing accurate cost estimates, we could not determine  whether the estimates were unbiased. This was the only best practice  related to accuracy where the Authority fell short. To help ensure an  unbiased estimate, the Cost Guide recommends conducting a systematic  analysis of the potential risks to the project and their likelihood of  occurring—called a risk and uncertainty analysis. A risk and uncertainty  analysis is also a best practice for developing a credible cost estimate as  discussed below.
We found that the Authority partially met best practices for producing  comprehensive cost estimates. For example, the Authority met the best  practice for including in the cost estimates the major components of the  project’s construction and operating costs. The construction cost estimate  is based on detailed construction unit costs that are, in certain cases,  more detailed than the cost categories required by FRA in its HSIPR  grant application. However, the operating cost estimate was not as  detailed as its construction cost estimate, as over half of the operating  costs are captured in a single category called Train Operations and  Maintenance. Authority officials told us that they developed their cost  estimates consistent with FRA’s guidance which emphasized greater  detail on the construction cost estimate and less detail on the operating  cost estimate. FRA officials confirmed that they emphasize construction  cost estimates because HSIPR grants are required by federal law to only  fund the capital costs of a project, not its operating costs. However,  sufficiently comprehensive operating cost estimates are necessary to  determine the potential profitability of California’s project, a key  consideration to attracting private-sector investment in the project that the  Authority is counting on to help complete construction of the project. In  addition, the Authority did not follow the best practice that calls for clearly  describing certain assumptions underlying the construction and operating  cost estimates. For example, Authority officials told us that the California  project will rely on proven high-speed rail technology from systems in  foreign countries, but it is not clear how the cost estimates were adjusted  for applying the foreign technology in California and how these  adjustments are reflected in the complete project cost estimates.  California requirements on the technology may differ in terms of speed or  safety requirements from that of other systems. Authority officials said  that they produced a thorough description of the technical requirements  for high-speed rail, but we were unable to see how this document was  linked to the cost estimates, that is, a description of how these technical  requirements will impact the project cost estimates. Without  comprehensive cost estimates, it is not possible to independently assure  that all cost-influencing factors and assumptions were considered.
We found that the Authority partially met best practices for producing well- documented cost estimates. In many cases, the methodologies used to  derive the construction cost estimate were well documented, but in other  cases, were more limited. For example, while track infrastructure costs  ($23.6 billion in 2011 dollars) were thoroughly documented, costs for  other elements, such as building new stations and acquiring trains ($3.2  billion in 2011 dollars), were not supported with sufficient documentation  to identify how these costs were developed and what costs were included  or excluded. Authority officials told us that since station locations and train  technology are not yet finalized, they used a higher-level cost estimate.  Additionally, we were unable to trace the estimates back to their source  data and recreate the estimates using the stated methodology. For  example, we were unable to identify the basis for how the operating costs  from analogous foreign high-speed rail projects were adjusted for use in  California. Authority officials said that the operating cost estimate was  used at a high level to determine whether or not the California system will  operate with an operating surplus. Authority officials plan to refine the  operating cost estimate as the project progresses. However, without more  detailed documentation, the Authority’s cost estimates are more difficult to  support and it may be harder to make changes to the estimates as they  are revised since the basis of the original estimate may not be  documented. In addition, without more thorough documentation, FRA and  other oversight officials cannot replicate and evaluate what the Authority  did to prepare its estimates and potentially exposes the project to  possible cost overruns because the basis for costs are not known.
Estimates that lack documentation are not useful for updates or  information sharing and can hinder understanding and proper use.
We found that the Authority partially met best practices to help ensure the  credibility of its cost estimates. Those practices include:  testing such estimates with a sensitivity analysis, such as assessing  the effect of changes in key cost inputs; obtaining an independent cost estimate conducted by an unaffiliated  party to see how outside estimates compare to the original estimates;  and  conducting risk and uncertainty analysis.
In regard to the construction cost estimates, the Authority performed a  sensitivity analysis for the approximately first 30 miles of construction and  obtained an independent cost estimate for the first 185 miles of  construction in the Central Valley but neither covered the entire Los  Angeles to San Francisco project. And, as noted under the accuracy  discussion, the Authority did not conduct a risk and uncertainty analysis  on the cost estimates for any construction segment. Authority officials told  us that in the absence of relevant FRA guidance, they followed FTA  guidance for these types of evaluations. They noted that FTA guidance  recommends conducting sensitivity tests once route alignments are  selected and that, thus far, only 30 miles of the project meet those  criteria. Similarly, Authority officials told us that they interpreted FTA  guidance to require an independent cost estimate for those segments that  have passed the 15 percent design milestone, which includes the first two  construction segments from Merced to Fresno and Fresno to Bakersfield.  Because the Authority’s cost estimates cover both construction of the full  system—from San Francisco to Los Angeles—and include operating  costs, sensitivity analyses and independent cost estimates are more  beneficial when they cover the entire project to help ensure greater  credibility. The methodology of these tests can be altered to reflect the  level of design, so a segment that has met a certain level of design can  still be evaluated for credibility. Finally, as noted above, the Authority did  not perform a risk and uncertainty analysis, which would improve the  estimates’ credibility by identifying a range of potential costs and  indicating the degree of confidence decision-makers can place on the  cost estimates. For example, the Authority faces the potential challenge  of acquiring rights-of-way in a timely manner. Authority officials told us  there about 400 parcels in the first construction package, about 100  parcels of which are considered to be potentially at-risk for timely delivery  for construction. However, without a risk and uncertainty analysis it is not  possible to determine how the cost estimates might be affected by such  things as delays in acquiring necessary rights-of-way or having to pay  more for property to keep the project on schedule.
Authority officials said that they did not conduct a risk and uncertainty  analysis yet because FRA did not require one, and the FTA guidance to  which they turned, recommends such an analysis after a final route  alignment is selected. In addition, Authority officials told us that they  added contingencies to the cost estimates to account for the risk of cost  overruns. However, according to our Cost Guide, risk and uncertainty  analysis—as with sensitivity analysis and independent cost estimates— should cover the entire cost estimate and can be performed at varying  levels of detail commensurate to the level of design. And, while  contingencies are designed to cover potential cost overruns, based on  our review, the Authority’s contingencies, which range from 10 to 25  percent for various cost elements, were not calculated based on the  results of a risk and uncertainty analysis (since this was not performed)  but rather were based on professional judgment. Without a risk and  uncertainty analysis, we cannot be assured that the contingencies are  accurately calculated, and more importantly, what level of confidence we  can have in the cost estimates.
For the operating cost estimate, the Authority conducted sensitivity tests  under various ridership scenarios as recommended by the Panel;  however, these tests were designed to measure the ability of the initial  operating section to cover operating costs with ticket revenues and not to  determine the potential risk factors that may affect the operating cost  estimate itself. The Authority also did not compare its operating cost  estimate to an independent cost estimate or conduct a risk and  uncertainty analysis. The Authority told us that it views the sensitivity test  already conducted as well as a forthcoming evaluation of operating costs  by the International Union of Railways (UIC) as sufficient to meet these  requirements. To make its operating cost estimate more comprehensive  and better documented, the Authority has contracted with the UIC to  evaluate the existing methodology and data and help refine the  Authority’s estimates.expert review of the Authority’s operating cost estimates, it may not  address some of the key practices that ensure credibility. For example,  the UIC’s evaluation is not expected to result in new, independently- produced cost estimates that can be compared to the Authority’s original  estimates.
section_title: Authority Can Make Improvements to Its Cost Estimates as the Project Progresses
paragraphs: The quality of any cost estimate can be improved as more information  becomes available. And, based in part on evaluations from the PRG, the  Authority is taking some steps to improve the cost estimates that will be  provided in the 2014 business plan. As noted above, the Authority has  contracted with the UIC to evaluate and provide recommendations on the  Authority’s operating cost estimates. While the study will provide  additional analysis from a reputable source, it may not address all best  practices from the Cost Guide that would help ensure that the operating  cost estimate is comprehensive, accurate and credible.
Cost estimates should also be updated with actual costs so that they are  always relevant and current. Continual updating of a cost estimate as a  program matures not only results in a higher-quality estimate but also  provides an opportunity to incorporate lessons learned. While the  Authority was not able to incorporate actual costs because construction  had not yet begun, it will have the opportunity once contracts are awarded  and actual costs begin to incur for the initial construction in the Central  Valley, which is expected to begin in 2013. The bids for the first 30-mile  construction package have been submitted to the Authority and will  provide a check on how well the Authority has estimated the costs for this  work, as well as provide more information on its cost estimates for other  segments of the project.
section_title: Ridership and Revenue Forecasts Are Reasonable for Current Purposes, but Will Require Updates
paragraphs: The Authority’s ridership and revenue forecasts to date are reasonable  and the methods used to develop them followed generally accepted  travel-demand-modeling practices. In addition, the Authority completed  several updates to its ridership-and-revenue forecasting model after the  release of the April 2012 revised business plan and also completed  several sensitivity analyses to test the reasonableness of its model.  However, the Authority will need to complete several additional updates  to improve its model and the resulting forecasts for the 2014 business  plan. Authority officials stated that they have plans in place to complete  several critical updates, including completing a new travel preference  survey and developing a second generation travel demand model, but will  not be able to complete these improvements in time for the 2014  business plan.
subsections: section_title: Forecasts Are Reasonable Based on Use of Accepted Travel-Demand-Modeling Practices, but Ridership and Revenue Model Will Require Additional Updates
paragraphs: Based on our review, we found that the Authority’s methods and model  used to produce its ridership and revenue forecasts adhere to generally  accepted travel-demand-modeling practices. However, the Authority will  need to complete several updates to improve these forecasts for the 2014  business plan. In its April 2012 revised business plan, the Authority  forecasts between 16.1 million and 26.8 million passengers per year and  annual revenues of $1.06 billion to $1.81 billion in 2030. These forecasts  were derived from a statewide ridership and revenue model, developed  under contract to the Metropolitan Transportation Commission (MTC).  Developing travel demand and revenue forecasts is difficult in almost  every circumstance, across every mode and for a variety of reasons. As  we have previously reported, limited data and information, especially  early in a project before specific service characteristics are known, make  developing reliable ridership and revenue forecasts difficult. To the  extent early stage data and information are available they need to be  updated to reflect changes in the economy, project scope, and consumer  preferences. In addition, risks of inaccurate forecasts are a recurring  challenge for project sponsors. Research on ridership and revenue  forecasts for rail infrastructure projects around the world have shown that  ridership forecasts are often overestimated. Although forecasting is  inherently risky, reliable ridership and revenue forecasts are critical to  accurately estimate the financial viability of a high-speed rail project and  determine what project modifications, if any, may be needed. Such  forecasts enable policymakers and private entities to make informed  decisions about the proposed project and to determine the associated  risks when making investment decisions. In addition, ridership forecasts  are critical because they serve as the basis for revenue forecasts. If the  California high-speed rail project is unable to generate the necessary  ridership and revenue to cover the system’s operating costs, the project  may not be able to operate without a subsidy—as required by Proposition  1A. Conversely, if forecasts are overly conservative, it could lead to the  state capturing less value from private investment than warranted. As  such it is critical that the Authority’s process for developing these  forecasts is reliable and provides some assurance that the resulting  forecasts provide a reasonable estimate of future demand for the system.
Unlike our cost estimating criteria discussed earlier, there is no industry  standard or established criteria for developing or evaluating intercity  passenger high-speed rail ridership forecasts. FRA has not established  guidance on acceptable approaches to the development of reliable  ridership and revenue forecasts and has established only minimal  requirements and guidance related to information HSIPR grant applicants  must provide regarding forecasts. We previously reported that developing  guidelines, methods, and analytical tools to develop credible and reliable  ridership forecasts is necessary to ensure equitable consideration of high- speed rail as a potential option to address demands on the nation’s  transportation system. We recommended that the Secretary of  Transportation develop guidance and methods for ensuring the reliability  of ridership and other forecasts used to determine the viability of high- speed rail projects and support the need for federal grant assistance.  The DOT OIG has also recommended that FRA develop specific and  detailed guidance for the preparation of HSIPR ridership and revenue  forecasts. According to FRA officials, they are in the process of  developing an oversight plan that will include criteria for evaluating  ridership forecasts. FRA officials indicated that they intend to use the  DOT OIG’s HSIPR Best Practices: Ridership and Revenue Forecasting  guide as a starting point for developing this guidance.
For the purposes of our assessment, we identified generally accepted  travel-demand-modeling practices for high-speed rail projects from a  variety of sources and developed criteria based on these practices to  assess the reasonableness of the approach used to create the ridership  and revenue models for the California high-speed rail project. In  developing our criteria, we relied primarily on a 2011 report, prepared for  the DOT OIG’s office by the firm Steer Davies Gleave, on best practices  related to travel demand modeling. In addition, we also examined other  literature on developing rail ridership and revenue forecasts to  supplement information in the Steer Davies Gleave report. Specifically,  we reviewed, among other sources, our prior GAO reports, Federal  Highway Administration (FHWA) and FRA guidance, and academic  literature. (See app. I for a list of guidance and reports we reviewed.)  From our review of these reports and other sources, we identified  common approaches to developing ridership and revenue forecast  models and elements affecting the validity of those models. We identified  seven key steps of the ridership-and-revenue forecasting process and  then compared the Authority’s process for completing these tasks to  generally accepted travel-demand-modeling practices. We found that the  Authority followed generally accepted travel-demand-modeling practices  for each of the seven key steps: (1) developing trip tables, (2) determining  and applying service characteristics, (3) developing mode choice models,  (4) estimating induced travel, (5) estimating expected fare revenue,   (6) conducting sensitivity analysis, and (7) conducting validation testing.  (For a detailed description of generally accepted travel-demand-modeling  practices and the Authority’s process for completing each of these steps,  see app.III).
The Authority’s process for developing trip tables and collecting and  compiling data on current travel patterns followed generally accepted  practices. A central task of the ridership forecasting process involves  collecting and compiling data on current travel patterns along the  proposed high-speed rail route into trip tables. We found that the  Authority followed generally accepted practices for developing trip tables.  For example, trips were distinguished by mode of travel (auto, air, rail),  trip purpose (commute, business, recreation, and other), and trip length  (over and under 100 miles) as is general practice. Various data sources  were used to develop the base year trip tables, including, among other  sources, 2000 Census Bureau data, survey data from a 2005 travel  survey and a 2011 long-distance travel survey conducted by Harris  Interactive, and existing regional models used by metropolitan planning  organizations (MPOs). Overall, the data sources used for developing  the trip tables were consistent with generally accepted standards. One  potential limitation, however, is that the 2011 survey sample was not  selected at random from among California residents but rather was  limited to individuals who had opted to join an online survey panel. The  American Association for Public Opinion Research (AAPOR)recommended against the use of such panels, often called “opt-in” survey  panels, when accurate population estimates are needed, due to concerns  about data quality and the possibility that a panel may differ from the  intended target population in unknown ways. Data were weighted to  adjust for differences between the survey sample and the California  population on four characteristics—geographic location, age, wealth, and  employment status—but, the data may still not be representative of the  California population in other characteristics related to travel behavior.
The Authority’s process for determining and applying service  characteristics followed generally accepted practices; however, as  detailed service plans are finalized or scenarios are changed, the model  will need to be updated to reflect revised service characteristics.  Ridership forecasts require information on the service characteristics  (such as travel time and fares) of competing modes of travel, such as  automobile and air travel along the proposed route. We found that the  Authority collected and considered relevant service characteristics and  used appropriate data sources. These included information on service  characteristics of the interregional transportation modes in the area  including information on time, cost, and other service characteristics for  each mode—auto, conventional rail, and air—based on the most currently  available published or observed data. High-speed rail service  characteristics were defined based on the initial service plans and fare  structure because published or observed data do not exist. Some or all of  the high-speed rail characteristics will likely change as service plans are  finalized and engineering decisions are made, and those changes can  significantly affect ridership and revenue forecasts. This was illustrated in  a 2012 sensitivity analysis completed by the Authority in which service  characteristics for the proposed high-speed rail system were adjusted to  reflect reduced service on the San Francisco Peninsula, which is likely  under the current “blended” approach whereby the high-speed rail system  will share tracks with Caltrain.ridership and revenue forecasts by 11 percent and 13 percent,  respectively, compared with forecasts in the April 2012 revised business  plan. Updated representations of the base and forecast year level of  service characteristics will be important for producing realistic ridership  forecasts in the future.
Academic experts from the University of California Berkeley’s ITS conducted a review of  ridership and revenue forecast models used to develop forecasts in June 2010 and  produced a report summarizing their findings. See D. Brownstone, M. Hansen, and S.  Madanat, Review of Bay Area/California High-Speed Rail Ridership and Revenue  Forecasting Study, University of California Berkeley’s Institute of Transportation Studies,  UCB-ITS-RR-2010-1, June 2010. survey (revealed preference data), or on hypothetical situations presented  to travelers in a survey (stated preference), or both.
There are advantages and disadvantages to using revealed preference  and stated preference data. Revealed preference data provides  information on travelers’ actual choices made in a specific market.  However, according to the Steer Davies Gleave report, when collected  from travel surveys, biases may exist in the respondents’ responses due  to a desire of respondents to justify their chosen mode. In addition, since  true high-speed rail does not exist in the U.S., it is not possible to use  revealed preference alone to determine how American travelers would  actually use high-speed rail. To address this problem, stated preference  data have been used in high-speed rail studies to asses likely traveler  responses to a new service. According to the Steer Davies Gleave report,  while stated preference data can provide detailed information about a  traveler’s likely responses to different modes or services that do not  currently exist, this type of data may also exhibit bias; specifically, survey  respondents may respond favorably to a hypothetical new mode, when in  reality it may be more difficult to change habitual behavior. According to  the Steer Davies Gleave report, limitations with each of these types of  data can be mitigated by taking various steps such as by combining both  revealed preference and stated preference data. The primary source of  data for the Authority’s mode choice model was a revealed preference  and stated preference survey, of air, rail, and auto trip passengers,  conducted at airports, rail stations, and by telephone from August to  November 2005. However, in its July 2011 and May 2012 reports to the  Authority, the Panel reported that the Authority’s main mode choice model  was based solely on stated preference responses, and recommended in  its May 2012 report that the Authority collect new survey data and use  both revealed and stated preference data in developing a new mode  choice model. Authority officials stated that they are currently developing  a new revealed preference and stated preference survey, which they plan  to begin administering in early 2013. We discuss this survey in further  detail in the next section.
In addition, academic experts from the University of California Berkeley’s  ITS have previously reviewed the ridership and revenue model and  have also identified limitations in the Authority’s method of applying a  statistical model to the survey data. These data came from a choice- based sample, meaning that survey respondents were selected for having  already chosen a mode of transport (air, rail, or auto). The Authority used  a conventional method of applying a statistical model to such data, but a  newer method has been identified in a recent research paper. According  to some academic experts, the use of the latest method could be an  improvement. The Panel also reviewed this issue in its May 2012 report  and stated that while they do not see the non-use of this new method in  the Version 1 model as an important defect, it is worth investigating this  issue as the Authority continues to refine the travel demand model.
The Authority followed generally accepted practices when producing  induced travel estimates. Induced travel refers to trips that occur as a  result of the high-speed rail project and that might not otherwise have  been made using existing modes. In general, these are new trips that are  generated because a new travel mode exists. The Authority estimated  induced travel for the California high-speed rail project to be on average 2  percent of total high-speed rail trips. According to the Steer Davies  Gleave report, based on its review of forecasts of proposed U.S. high- speed rail systems, an upper limit on induced travel of approximately 10  percent of total high-speed rail trips is widely accepted for proposed U.S.  high-speed rail systems. Steer Davies Gleave also reviewed actual  induced travel for high-speed rail systems outside the United States and  found that it ranged from 6 to 27 percent. The Authority forecasted  induced travel to be on average 2 percent. This estimate appears to be  conservative and reasonable when compared to proposed U.S. high- speed rail systems and actual induced travel for high-speed rail systems  outside the United States.
The Authority followed generally accepted practices when estimating  expected fare revenue for the California high-speed rail project. Expected  fare revenue is a product of forecast ridership and average fares. High- speed rail fares are based on a boarding fare and a per mile fare for  interregional trips. For example, travel from Los Angeles to San Francisco  would mean an average, one-way, high-speed rail fare of $81 in 2010  dollars or 83 percent of average 2009 airfares from Los Angeles to San  Francisco. Forecast ridership is impacted by high-speed rail fares and  revenue is an output of the ridership and revenue model. According to the  Authority, the annual number of riders needed to breakeven when the  Phase 1 blended system is opened in 2029 is 6.1 million or 23 percent of  the high forecast. Authority officials stated that they did not produce a  revenue optimization forecast—that is, a ridership forecast that would  maximize revenue—to produce these high-speed-rail fare estimates and  acknowledged that they will need to do so in the future when meeting with  potential private high-speed rail operators, which will establish their own  revenue-maximizing fares for the system. According to the  Authority, the  Authority will not operate the high-speed rail system and a private  operator is expected to serve as a contract operator of the system. As  such, the private operator will be expected to assume all revenue risks of  the project (including setting fares). The Authority’s process for  calculating expected fare revenue adheres to generally accepted  practices; however, as other factors and inputs change, total expected  fare revenue will likely also change. For example, over time, it will be  important for the Authority to monitor changes in airfares, gasoline prices,  and other key assumptions and incorporate these changes, as necessary,  into future revenue forecasts.
The Authority followed generally accepted practices when conducting  sensitivity analysis of key model assumptions for the California high- speed rail project. A sensitivity analysis is typically conducted by varying  key model assumptions such as socioeconomic data, type of trips taken,  gasoline and auto fleet efficiency, and airfares, or parameter values to  determine how the model behaves in response to changes to these  assumptions. The Authority conducted several sensitivity analyses on its  ridership and revenue model, most in 2011. For example, the Authority  conducted sensitivity analyses that tested key factors, such as changes in  fuel economy, air and auto travel time, air and auto travel costs, and high- speed rail travel time assumptions. In one analysis, the Authority tested  the overall effect of a higher auto fuel economy and found that this  change reduced ridership and revenue forecasts by 16 and 19 percent,  respectively, from the Phase 1 high ridership and revenue forecasts  presented in the November 2011 draft business plan. According to the  Panel that reviewed these analyses, the results of the various sensitivity  analyses that the Authority has conducted show that the model is  appropriately sensitive across the range of variables tested.
In addition, the Authority performed a sensitivity analysis of an extreme  downside scenario to test the ridership and revenue implications of a  series of downside events, such as increased average rail travel time  from Merced to the San Fernando Valley (140 minutes instead of 126  minutes), decreased train frequency (3 trains per hour instead of 4 trains  per hour during peak times), lower auto-operating costs, and lower air  fares (10 percent below actual 2009 average air fares). Based on this  analysis, the Authority determined that an extreme downside scenario  would be expected to reduce ridership and revenue forecasts by 27  percent and 28 percent, respectively, below that shown for the low IOS  forecasts in the April 2012 revised business plan. According to the  Authority, they tested these events using the IOS phase because the  financial viability will be the most fragile during the early stages of  ridership. According to the Authority, these forecasts are still sufficient to  cover the Authority’s estimated operating costs and not require a public  operating subsidy. Authority officials stated that they intend to conduct  additional sensitivity analyses going forward.
The Authority mostly followed generally accepted practices to validate the  ridership and revenue model. Model validation generally involves verifying  that the model reflects observed traveler behavior including total travel,  region-to-region travel flows, and observed market shares by mode. In  the United States, without another high-speed rail system to use for  comparison, model validation is a difficult task. Furthermore, validating  the proposed California project with foreign high-speed rail systems is  difficult because some of the travel market characteristics in other  countries with high-speed rail, such as the cost of driving, may not be  comparable. The Authority has taken some steps to validate the model  through tests performed using data on Amtrak’s premium Acela service in  the Northeast Corridor (NEC) as input to the California high-speed rail  model and compared the output with 2008 actual ridership and 2030 NEC  forecasts. The results of the Authority’s comparison of the California  high-speed rail model with NEC input to actual 2008 ridership data  showed that ridership in the California high-speed rail model with NEC- like conditions in 2008 is 79 percent of actual 2008 NEC ridership.  Similarly, the Authority’s comparison of the California high-speed rail  system ridership forecasts from the model run with “NEC-like” service is  about the same as projected 2030 ridership on the Acela service.  Authority officials told us they believe the results from these tests  demonstrate that the ridership and revenue model is reasonably sensitive  to speed, frequency, and fares. While the NEC corridor has some  comparable characteristics to the proposed California high-speed rail  corridor—such that it is capable of reaching top speeds up to 150 miles  per hour and covers a distance of over 400 miles—the proposed  California high-speed rail corridor and the NEC differ in important ways.  For example, population density, congestion, and travel behavior in the  two corridors differ, and as such, forecast comparisons should be  interpreted with caution.
In developing the forecasts for the April 2012 revised business plan, the  Authority also revised several model assumptions used in the initial  ridership and revenue forecasts presented in the November 2011 draft  business plan. Specifically, the Authority revised model assumptions to  reflect changes in current and anticipated future conditions for airfares  and airline service frequencies, decreases in gasoline price forecasts,  and anticipated declines in the growth rates for population, number of  households, and employment. Some of the initial assumptions were  largely based on pre-2007 data and did not reflect potential effects of the  2007 to 2009 recession. (See table 2 for a summary of updates that have  thus far been completed.) According to Authority officials, this was done  to build in additional conservatism in the ridership forecasts.
Updating model assumptions can help mitigate of the risks of  overestimating ridership and revenue forecasts—referred to as optimism  bias. Biased ridership forecasts are a recurring problem with rail  infrastructure projects and we have previously reported that forecasting  ridership and revenue is a complex and uncertain process and ridership  forecasts of high-speed rail projects are often overestimated. Other  research on ridership and revenue forecasts for rail infrastructure projects  have confirmed that actual ridership is likely to be lower than forecasted  unless steps are taken to incorporate more conservative assumptions into  the model. For example, a recent study examined a sample of 62 rail  projects and found the ridership forecasts of 53 of them were overstated;  actual ridership was, on average, 41 percent lower than forecasted.  Updates to model inputs, such as fuel prices and other projections, are  important for updating ridership forecasts for any project; in this instance,  updates to model inputs resulted in more conservative ridership forecasts.
section_title: Next Business Plan Will Not Reflect All Planned Improvements to Ridership Forecasts
paragraphs: The Authority has plans to complete future improvements to its ridership  and revenue forecasts, including completing a new travel preference  survey and developing a second generation travel demand model.  However, the Authority will not be able to complete these critical  improvements in time for the 2014 business plan. According to Authority  officials, the 2014 plan is expected to include, among other updates,  updated ridership and revenue forecasts. Two critical updates to the  ridership forecasts that peer reviewers and academic experts have  recommended are the development of a new 2013 revealed and stated  preference travel survey and a second generation travel demand model— which will make use of the new survey. Although the Authority has begun  taking steps to complete both of these tasks, neither the survey nor the  second generation model will be completed in time for the 2014 business  plan.
The Authority began work developing a new 2013 revealed and stated  preference survey in late 2012. According to Authority officials, a survey  sample and survey questionnaire design was initiated in December 2012.
Data collection for the 2013 revealed preference and stated preference  survey will not begin until 2013. Full data collection, cleaning, and  preliminary analysis of results are expected to be completed by mid-April  2013. The new survey will include a larger sample of 4,500 respondents  compared with the 3,172 respondents to the 2005 survey. In addition, the  revealed preference portion of the data set will be designed and coded to  facilitate estimation using revealed and stated preference responses  simultaneously, which was not done in the first version of the model.  According to the Panel, development of a new survey is critical as it will  address several long-term issues that can only be overcome with the  collection and analysis of new survey data.
The Authority also has plans to develop a second-generation travel  demand model; however, the second generation model will not be  completed in time for the 2014 business plan. According to Authority  officials, work on the second-generation model will not begin until the  ridership and revenue analysis for the 2014 business plan is completed.  According to the Authority, the second generation model will use data  from the new 2013 revealed and stated preference survey to supplement  data from the 2005 survey. In addition, the Authority plans to replace the  2011 Harris Interactive long-distance travel survey data with data from the  2012 California Household Travel Survey being conducted by the  California Department of Transportation. According to Authority officials,  both surveys will be needed for developing its second-generation travel  demand model. The Panel, which has released five reports assessing the  Authority’s ridership and revenue forecasts, has reported that the  Authority’s ridership and forecasts to date are reasonable for planning  purposes but has also stated that additional updates and enhancements,  particularly the development of a new model, will be critical for future  project decision making. For example, in its most recent October 2012  report, the Panel stated that a second-generation model will be required  to meet the Authority’s long-term goals of completing detailed planning  studies and make key planning and operational decisions on issues such  as specific rail alignments, station design requirements, and pricing  strategies.
While the Authority will not be able to complete a second-generation  travel demand model in time for the 2014 business plan, it has begun  work on developing an enhanced model that will be used to produce the  ridership and revenue forecasts for the 2014 business plan. The  enhanced model will retain the same structure of the original model, but  some of the individual model components will be updated. For example,  the main mode choice model will use both revealed preference and stated  preference results from the 2005 travel survey. According to Authority  officials, the enhanced travel demand model will be completed by   May 31, 2013.
Even if the Authority is not able to complete the major ridership and  revenue forecast improvements in time for the 2014 business plan,  ongoing disclosure of interim results from model improvements both  before and after the business plan are published will be important to  outside reviewers and the public. Peer reviewers and other groups that  have examined the Authority’s ridership and revenue forecasts have  reported the need for greater transparency in the Authority’s analyses.  For example, in its January 2012 report, the California State Auditor  reported that the Authority’s November 2011 draft business plan lacked  detail in its presentation of some of the revenue forecasts. Similarly, in its  October 2012 report, the Panel advised the Authority to provide  summaries on the Authority’s website, describing, among other things,  recent forecasts, key input assumptions used to develop the forecasts  (e.g., fuel price trends, socioeconomic growth rates, and changes in  household size and structure), and updated service characteristic  information. According to Authority officials, documentation supporting  analysis in its 2014 draft business plan will be available on the Authority’s  website when the plan is made public. In addition, Authority officials  stated that the Authority has generally posted key technical documents as  they are made public.
The Authority also has developed a work plan for other travel demand  model improvements. This includes, among other things, plans to  complete additional validation testing of model results using data from the  NEC. For the 2014 business plan, the Authority is planning to conduct  further testing and sensitivity analysis of the ridership and revenue  forecasts to examine the sensitivity of the forecasts to reduced  frequencies of services; changes in alternative fare structures (for  example, premium fares for intra-regional trips in the San Francisco Bay  Area and Los Angeles Basin); changes to service plans (destinations and  schedules); and other sensitivity analyses aimed at quantifying risks. In its  October 2012 report, the Panel also recommended additional sensitivity  analyses to be completed on the second generation travel demand  model, including analyses examining the impact of pricing strategies on  revenue, impact of local transit feeder systems on station choice, and  impact of major changes in the roadway network on highway congestion  and subsequent mode choice decisions. In addition, the Authority is  planning to conduct Monte Carlo simulations to test numerous potential  combinations of assumptions on the forecasts that will be part of the 2014  Business Plan during fiscal years 2013 through 2014, provided the  foundational information to construct, test, and analyze the simulation and  its results is sufficiently developed at that stage of the program. All of  these will be important as the ridership and revenue forecasts continue to  evolve with development of the high-speed rail project.
section_title: The Authority’s Funding Plan Faces Uncertainty
paragraphs: The project’s funding, which relies on both public and private sources of  financing, faces uncertainty about whether those funds can be obtained in  a tight federal and state budget environment. The Secretary of  Transportation and the Governor of California have committed to funding  this project, but obtaining sustained congressional and public support for  appropriating additional funds is one of the biggest challenges to  completing this project. In the latter stages, the Authority will also rely on  private-sector financing, but will require more reliable operating cost  estimates and revenue forecasts to determine whether, and the extent to  which, the system will be profitable, as well as the value of any private  investment. The Authority’s financing plan recognizes the uncertainty of  the current funding environment so the Authority is building the project in  phases and has identified an alternative funding source that is also  uncertain. However, delays in obtaining funds as planned will likely lead  to project delays and higher costs for construction. A summary of funding  already committed to date can be found in table 3.
The Authority is relying primarily on public-sector funding to complete  construction of Phase 1 of the project with $55 billion, or 81 percent of the  total construction cost, expected to come from state and federal sources.  Heavy reliance on public-sector funding is not unusual for a project of this  size. For example, France built its high-speed train system primarily with  public-sector funding. In the United States, federal-aid highway system  continues to be funded by the federal government through the gas tax  and, more recently, with transfers from the general fund. The Authority  expects to obtain public sector funds over the life of the project as  individual segments are ready for construction. This type of “phased”  funding is typical for major transportation infrastructure projects. Table 4  provides a summary of the Authority’s funding plan for Phase 1 the high- speed rail project.
The Authority’s April 2012 revised business plan relies on approximately  $42 billion in federal funding for the project’s construction, which includes  the $3.3 billion that has already been obligated. The remaining $38.7  billion in federal funds have not been identified in federal budgets or  appropriations but would amount to an average of more than $2.5 billion  annually over the life of the project’s construction. This exceeds the  average annual funding made available under DOT’s New Starts transit- funding grant program since 2008 (about $1.6 billion per year). Moreover,  it exceeds the federal government’s average annual appropriations to  Amtrak since 2008 (about $1.5 billion per year). Largely as a result of the  funding challenge, the Authority is taking a phased approach—planning to  build segments as funding is available. Thus, according to the Authority’s  2012 revised business plan, no additional funding will be needed until  2015 when it hopes to begin construction beyond the first construction  segment.
Based on our past work on high-speed rail, successful projects require  significant and sustained financial commitments from the public sector  before private investors will participate, and the Authority’s plan reflects  this funding model. For example, in Japan, private investment is  contingent on substantial government investment. Other federally- supported transportation programs—like those for highway and certain  transit infrastructure—rely on a dedicated revenue source for their funding  and allow for multi-year funding agreements for eligible projects. In  contrast, the HSIPR program has not been funded with a dedicated  revenue stream, but from the general fund, a process that means that the  program has to compete for appropriations with other discretionary  programs. In addition, the HSIPR program has provided one-time grants  and, as currently structured has not awarded multi-year agreements for  grantees. Our 2013 High-Risk Series report identified the use of general  funds for high-speed rail projects as a challenge to project completion.
Given that the HSIPR grant program has not received funding since 2010  and that future funding proposals will likely be met with continued concern  about the general level of federal spending, the largest block of expected  funding for the California project is uncertain.
The Authority is also relying on a total of about $8.2 billion in state high- speed rail bond proceeds (which includes the $3.7 billion that has been  appropriated to date) and another $5 billion in locally-generated funds for   The proceeds from the high-speed rail bonds  the project’s construction. are dedicated, in that they can only be used for this project and do not  have to compete with other budgetary priorities. However, the remaining  $4.5 billion will have to be appropriated to the project. The $5 billion in  local funds—most of which are expected at the end of the project’s  construction timeline—have not been committed by local entities yet.  Authority officials told us that these local funds could include revenues  derived from property development in and around high-speed rail stations  or improved service on existing transit corridors. For example, planned  improvements on the Caltrain corridor may result in increased ridership; if  so, a portion of increased revenues could be earmarked for high-speed  rail construction.
According to their April 2012 revised business plan, the Authority is planning on applying  $8.2 billion of the $9.95 billion in state high-speed rail bond proceeds to project  construction. Some $950 million of the remaining $1.75 billion will be used for  transportation projects that will connect to high-speed rail. And, the other $800 million will  be used for environmental, planning, and support costs.
subsections: section_title: Securing Private-Sector Investment Will Depend on Reliable Operating Cost and Ridership Forecasts
paragraphs: In addition to the challenges of obtaining public-sector funding, the  Authority may face challenges in attracting private-sector funding if its’  operating cost estimate and ridership forecasts prove to be optimistic.  The Authority expects that once the initial operating segment is  operational in 2023, it will generate a profit that would be attractive to  private investors. The Authority is planning to raise approximately $13.1  billion by selling an operating concession to a private firm or consortium  of firms.complete construction of the system.
The Authority plans to use the proceeds of this sale to help  Our past work on high-speed rail systems has shown that private sector  investment is easier to attract only after the public sector has made a  substantial capital investment in the system. The Authority’s plan is  consistent with this funding approach; however, to successfully attract  private investment in this project, the Authority will have to meet two  significant milestones:  complete construction of the IOS (this will require at least an  additional $25 billion of investment from public sources), and demonstrate that the IOS can operate at a profit.
Public-private partnerships for intercity passenger rail, such as what the  Authority is planning for, have been proposed but not implemented in the  United States. However, in other transportation sectors, public-sector  infrastructure owners have used public-private partnerships to incorporate  private-sector operating expertise and encourage private investment. For  example, the state of Indiana raised $3.8 billion in 2006 by arranging a  private operating concession for its existing Indiana Toll Road. In addition,  according to DOT, the city of Denver, Colorado, raised nearly $100 million  from private sources to help finance a $2 billion expansion to its transit  and commuter rail project. While the private-sector can provide needed  funding and management expertise to a transportation project, this  approach is not risk-free. As we have previously reported for highway  projects, public-private partnership can also present trade-offs, such as  the risk that the private operator will demand more revenue from users  (e.g., tolls) than initially expected.
Other nations’ experiences with high-speed rail indicate that under certain  circumstances the private sector can operate these systems and that they  could potentially be profitable on an operating basis. For example, Japan  sold certain high-speed rail lines to private operators and does not  provide operating subsidies to these firms. And, in 2010, Britain sold an  operating concession for its High-Speed 1 line to a consortium of private  investors for approximately £2 billion (approximately $3.2 billion).  Private firms have also expressed interest in operating high-speed rail  projects in the United States. For example, according to FRA, several  private consortia were preparing to submit bids on a HSIPR-funded  project between Tampa and Orlando, Florida. However, public-sector  support was withdrawn when the governor canceled the project which  precluded private-sector investment. And, in Texas, a privately-financed  high-speed rail project failed when the investors encountered financial  difficulties. Authority officials told us that they have met with a number of  private firms and high-speed rail operators that have expressed interest in  California’s project but have not entered into any agreements since the  project has not yet been built. Attracting private investment may not only  require up-front public investment but may also require the use of  revenue guarantees, or public guarantees of a minimal level of income to  the project regardless of ridership levels. Such guarantees will reduce the  risk for private operators, and therefore their cost of raising capital, but  according to the Authority, a revenue guarantee is considered to be a  type of operating subsidy that is barred by the legislation that authorized  the state high-speed rail bonds (Proposition 1A bonds).
Private-sector investment in the California high-speed rail project, if any,  will ultimately be determined by the profitability of the system—that is, the  extent to which operating revenues exceed operating costs. The Authority  currently estimates an operating profit in the first and all subsequent  years of operation. However, this estimate is only as reliable as the  underlying operating cost and revenue forecasts. As discussed earlier,  the Authority’s current ridership and revenue forecasts are reasonable for  planning purposes, however, further refinements will be required as the  project continues to evolve. The Authority’s current operating cost  estimates will also need to be improved in the future. Accordingly, both  cost and ridership forecasts will change before the initial operating  segment is completed in 2022, making the future value of potential private  funding uncertain at this time.
section_title: The Authority Comprehensively Identified Potential Economic Impacts, but It Is Too Early to Determine Specific Impacts
subsections: section_title: Potential Economic Impacts Were Comprehensively Identified, but Specific Impacts Will Depend on Project Decisions, Economic Conditions, and Other Factors
paragraphs: The Authority comprehensively identified key economic impacts that  could result from the high-speed rail project, including user and non-user  impacts, as required by FRA and other federal requirements. FRA  guidance, as contained in the program Notices of Funding Availability  (NOFA), requires HSIPR applicants to identify the potential benefits and  costs of proposed projects with a focus on a public return on  investment. A public return on investment includes a project’s potential  to deliver transportation, economic recovery, and other benefits. To assist  in project evaluation, FRA encouraged HSIPR applicants to provide an  economic analysis that quantified the monetary value of user benefits  and, if available, public benefits. However, according to FRA, program  NOFAs did not explicitly require either a formal benefit-cost analysis  (BCA) or preparation of an economic impact analysis (EIA). FRA  officials said their review of economic impacts was based on a  reasonableness test—that is, were economic impacts identified and were  the assumptions behind the impacts reasonable. The officials said that  FRA did not have the time or resources to conduct an in-depth analysis  and that a reasonableness test provided increased assurance as to  potential economic impacts of project proposals. Projects awarded  funding must also comply with the National Environmental Policy Act  (NEPA) and its implementing regulations. NEPA requires that government  agencies undertaking a major federal action (such as providing grant  funding) with significant effects on the environment prepare an analysis of  the environmental impacts of the proposed action, including a discussion  of alternatives to the proposed action. Under FRA’s guidelines for  considering environmental impacts, among the impacts to be considered  in a NEPA environmental assessment are such things as land use and  potential economic effects on existing business districts and metropolitan  areas. The program NOFAs required HSIPR applicants seeking funds to  develop new high-speed rail corridors and intercity passenger rail  services to complete a NEPA review and the June 2009 program NOFA  required HSIPR applicants to present information that provided a  business and investment justification that contained project cost and  benefit estimates.
DOT OIG, FRA Needs to Expand Its Guidance on High-Speed Rail Project Viability  Assessments, CR-2012-083 (March 28, 2012). In particular, we focused on best practices  contained in the report HSIPR Best Practices: Public Benefits Assessment, Steer Davies  Gleave, June 2011 that was prepared for the Office of Inspector General in conjunction  with the March 2012 report. The Steer Davies Gleave report included a list of components  that would be included in a public benefits assessment of high-speed rail projects. The  components of public benefit assessments were similar to economic impacts identified  through a review of the HSIPR NOFAs. transportation connectivity may have on allowing firms to access larger  labor or product markets or increasing the labor supply because people  can more easily access jobs. The latter are included in what is termed  “wider economic impacts.” FRA’s requirement for a public return on  investment from HSIPR projects included aspects of both user and non- user impacts. The best practices report noted the importance of both  ridership and revenue forecasts and cost estimates in determining public  benefits. In particular, it stated that public benefits assessment depends  heavily on ridership and revenue forecasts and the implications these  have on project impacts on travelers and the general population.  Similarly, operating, maintenance, and capital cost estimates were also  identified as important elements of public benefits assessments.
The Authority’s April 2012 revised business plan identified the primary  user and non-user economic impacts of the California high-speed rail  project (see table 5). For example, the plan identified potential user  impacts such as travel time reliability for high-speed rail users and non- user impacts such as the effects on highway congestion and economic  development around stations. In addition to the business plan, the  Authority prepared a BCA that provided a more detailed analysis of both  user and non-user impacts included in the revised business plan. The  Authority also prepared an EIA that focused on those other economic  impacts of the system that do not fall into the BCA framework. According  to the Authority, the EIA presented longer-term impacts on California’s  economy from building the high-speed rail system.
Although the Authority comprehensively identified the primary potential  economic impacts, it is too early to determine specific economic impacts  since these will depend on a number of factors. These include the  following:
Future project decisions. The California high-speed rail project is in its  early stages of development, and a number of project decisions have  yet to be made, including final alignment of train routes, some station  locations, and the type and frequency of service. Decisions such as  these can be expected to have a bearing on potential economic  impacts. For example, route alignments and station locations can  affect economic development. While acknowledging that the extent to  which high-speed rail would change California’s economic landscape  was not fully understood, the EIA suggested that based on studies in  other countries, the main economic impacts from high-speed rail in  California will likely occur in areas within 2 hours of major economic  centers, such as the San Francisco Bay Area and Los Angeles.  However, the EIA also concluded that the greatest volume of  redevelopment attributable to high-speed rail will likely occur in major  metropolitan areas and that the Central Valley could see moderate  clustering of development around stations. Although economic  development around stations offers the potential for economic  impacts, achieving such development may be subject to a number of  factors, and certain impacts may not be easy to identify. In July 2012,  we reported on the potential for economic development associated  with bus rapid transit projects. We found that in the five case study  locations we examined, although the bus rapid-transit project was  having some positive effect on economic development, individuals  associated with these projects were unsure about how much  economic activity could be attributed to the presence of bus rapid  transit compared with other factors or circumstances. In addition, the  project sponsors and experts we spoke with told us that transit- supportive policies and development incentives can play a crucial role  in helping to attract and spur economic development associated with  bus rapid transit. In September 2009 we reported the characteristics  of transit-oriented development around stations can increase nearby  land and housing values, but we also found that determining transit- oriented effects on the availability of affordable housing in these  developments are complicated by lack of direct research and data.
Economic conditions over the life of the project. As we reported in  March 2012, the closer the economy is to full employment, the smaller  net effect a project will have on total economic activity. The speed  with which the nation and California recover from the 2007 to 2009  recession, which cannot be known in advance, will affect the net  employment from any new infrastructure projects. If the economy  achieves full employment, such projects would affect the composition  of employment but not its level or rate of growth. The high-speed rail  project will be constructed and operated over a period of many  decades and likely over many economic cycles. The Authority’s April  2012 BCA used a 67-year period (from 2013 to 2080) to estimate  potential economic benefits and costs of the project, and, for purposes  of analyzing potential operating and maintenance costs, the April  2012 revised business plan assumed a 38-year operating period  (2022 to 2060). Over such an extended period, economic conditions  can be expected to change, as will potential economic impacts. The  Authority’s April 2012 EIA recognizes that the project’s economic  impacts will be affected by California’s economy and unemployment  rates. According to the Authority’s EIA, as of February 2012  California’s 10.9 percent unemployment rate was the nation’s third  highest. It goes on to estimate that the high-speed rail project has  the potential to create about 1 million direct and indirect job-years  through construction of the Phase 1 blended system, based on the  assumption that 20,000 job-years would be created for each $1 billion  in capital investment.the economic conditions and unemployment rates at the time the jobs  are created. The EIA acknowledged this uncertainty when it stated  that multipliers used to estimate indirect and induced jobs are  snapshots in time of an economy and represent only current or recent  economic relationships and technologies. They do not capture  structural changes in the economy, new technologies, or changes in  wages that occurred since the multiplier data were produced or that  might occur in the future.
The accuracy of this estimate will depend on
Uncertainty about some impacts. Uncertainty may particularly affect  non-user impacts, some of which are difficult to measure or quantify.  The Authority’s April 2012 EIA discusses the potential for wider  economic impacts from high-speed rail such as the benefits of  bringing California’s economic activities and markets closer together  by reducing travel times. The EIA states, by improving transportation  connectivity and reducing congestion, the high-speed rail system  could make California’s economy more efficient, productive, and  competitive by such things as bringing businesses closer to labor and  other markets and providing workers with greater access to jobs.  However, the best practices report prepared for the DOT OIG found  that additional data on things like the relationship between economic  density and productivity, labor supply elasticity, and price-cost  margins were needed to assess wider economic impacts of high  speed rail projects. The study went on to note that no research was currently available regarding wider economic impacts of U.S. high- speed rail projects since such projects do not yet exist. The  Authority’s April 2012 EIA recognized this difficulty and states that the  extent to which the high-speed rail project will affect the economic  landscape of California is not well understood, though transportation  infrastructure investments have historically created fundamental shifts  in the spatial relationship between places. These limitations do not  imply deficiencies in Authority or DOT performance but rather the  inherent analytical complexities of large infrastructure investments.
Uncertainty regarding local or regional impacts. The April 2012 EIA  discusses the potential for economic impacts along the high-speed rail  line, including direct and indirect employment opportunities, increased  efficiencies and productivity from bringing labor and other markets  closer together, and transit-oriented development around stations.  The specific impacts to regions or localities will depend on a number  of factors, including project-related factors and factors associated with  local policies and decisions. Among the project-related factors are  both the rate of project spending over time as well as where project  funds are spent. The high-speed rail project is expected to be  constructed over a long period of time and in phases when funding  becomes available. The rate of spending and its timing will influence  when and to what extent regions and localities may experience  economic impacts associated with the project. Which specific regions  or localities may experience economic impacts will be influenced by  such project decisions as route alignments and station locations.
Local policies and decisions will also affect regional and local  economic impacts. Studies conducted for the Authority by the  University of California at Berkeley suggested there are opportunities  for economic development from the high-speed rail system in a variety  of locations, including Fresno and Bakersfield in California’s Central  Valley. However, the studies cautioned that the extent of economic  development will depend on cities establishing a framework of  planning and development policies that encourage development.  Some cities have begun taking actions to promote economic  development related to the high-speed rail system. For example, in  June 2012 Fresno issued a solicitation for consultants to prepare a  master plan for its planned downtown rail station that would enable  the city to maximize local economic benefits from the high-speed rail  system. Other cities have not yet acted. The Authority’s April 2012  EIA found that of the 13 potential stations on the Phase I blended  corridor, 7 (53.8 percent) did not have station-specific development  plans.
We also found some limitations in the specific economic analyses  prepared by the Authority. In particular, the April 2012 BCA has  shortcomings that could limit its usefulness to decision makers.
Identification of negative impacts of the project and their effects on the  BCA analysis. The BCA lacked detail regarding the handling of  negative impacts associated with the high-speed rail project. DOT  guidance on benefit-cost analysis suggests that negative (or adverse)  impacts, such as non-user (or highway) delays associated with rail  construction, be included in BCA analyses to facilitate consistent  project comparisons. The guidance recommends that negative  impacts, such as highway delays associated with rail construction, be  shown as a negative benefit and not included in project investment  costs to better facilitate comparisons between projects. The April 2012  BCA recognized that during the period of project construction there  would be roadway delays in urban areas that would offset some travel  time savings. However, the BCA excluded such impacts from the  analysis since the impacts were expected to be (1) localized, (2)  minimal since the high-speed rail project minimizes urban grade  crossings, and (3) negligible in proportion to overall travel time  savings once the project is complete.
Aside from roadway delays, the Authority acknowledged there were  additional categories of economic impacts that may be negative. For  example, the April 2012 revised business plan discusses how the  high-speed rail system could limit access to parts of farmland in the  Central Valley potentially reducing the output of affected farmlands.  The BCA contained little discussion of such impacts and states that  the BCA did not incorporate or monetize land use and land value  impacts the high-speed rail project may cause (positive or negative).  The Authority said negative impacts were assumed to be part of the  mitigation measures that would be conducted as part of the  environmental review process and right-of-way acquisition and that  the costs of these measures were included in the cost side of the  benefit-cost calculation. According to the Authority, including negative  impacts as a negative benefit would lead to double counting them as  both a negative benefit as well as a cost in the benefit-cost  calculation. We agree that negative impacts of the high-speed rail  project should not be double counted, but the BCA should include  discussion of potential negative impacts and how they are treated in  the analysis. Such information would better inform decision makers  about the existence of negative impacts and their potential effect on  project benefits or costs.
Identification of the risks and uncertainties associated with the BCA  analysis. The BCA did not discuss the potential risks and uncertainties  associated with either the benefits or costs used in the analysis.  Forecasts are inherently uncertain, including those for ridership and  economic projections. Recognition and analysis of risks is an  important part of project evaluation. As we reported in February 2011,  Executive Order 12893 and Office of Management and Budget (OMB)  Circulars Nos. A-94 and A-4 indicate that benefit and cost information  shall be used in agency decision making and that the level of  uncertainty in estimates of benefits and costs shall be disclosed. In  particular, Executive Order 12893 requires that uncertainties about  the amount and timing of important benefits and costs associated with  an infrastructure investment be recognized and addressed through  appropriate quantitative and qualitative assessments.
Similarly, DOT’s TIGER guidance, which the Authority used to  prepare its BCA, requires applicants to assess the reliability of any  forecasts used to generate benefit estimates but does not specifically  require a discussion of risks and uncertainties in a BCA. We reported  in February 2011 that the majority of the applications to the TIGER  and HSIPR programs we reviewed did not provide information related  to uncertainties in projections, data limitations, or the assumptions  underlying their models. Even when such information was provided,  we found that it was not always comprehensive. We recommended  that DOT require, among other things, grant applicants to clearly  communicate the level of uncertainty in estimates of project benefits  and costs. The Authority did not conduct a risk analysis beyond  examining the potential effects of high and low cost scenarios. While  the Authority may have used credible sources for variables in its  analysis (e.g., fuel prices), this does not eliminate all forecasting risks,  and those risks should be identified. In addition, as we have noted,  decisions on route alignments and other aspects of the project have  yet to be made; decisions that could add to project risk.
GAO-11-290.
section_title: High-Speed Rail’s Impact on Meeting State Travel Demand Has Not Been Fully Assessed
paragraphs: Although the Authority comprehensively identified the project’s potential  economic impacts, additional analysis is needed at the state level of how  high-speed rail will affect other transportation modes and their ability to  meet future travel demand. This includes the potential cost of additional  improvements that may be needed or conversely planned projects that  may not be needed. An important aspect of non-user impacts is a  project’s potential effect on other transportation modes, including  highways, aviation, and local transit systems. This is an important issue  since the Authority has estimated that, as a result of population growth  and other factors, overall interregional trips in California will increase from  about 500 million in 2000 to about 900 million in 2030. In addition, under  the blended approach adopted by the Authority, success of the high- speed rail system will depend in part on local transit improvements.
As part of the planning process, the Authority considered high-speed rail’s  impact on the capacity of other transport modes. For example, in April  2012 the Authority issued an analysis of the highway and airport  improvements that would be needed to provide an equivalent capacity to  the high-speed rail system envisioned in the April 2012 revised business  plan. The analysis found the total cost (in 2011 dollars) of equivalent  capacity investment in highways and airports would range between $123  billion and $138 billion to build up to 4,600 highway lane-miles, 115  airport gates, and 4 airport runways. However, the analysis did not  focus on potential additional highway or other transportation  improvements that may be required even with construction of the high- speed rail system. Rather, the analysis identified the potential highway  and airport improvements that would be required to provide an equivalent  capacity to that of the high-speed rail, not an assessment of additional  improvements required to meet future intercity travel demand. Identifying  such improvements was not the task of the Authority since its task is to  develop a high-speed rail system. Rather, this task would fall to the state  as part of its overall planning responsibilities under federal transportation- planning requirements.
Constructing a high-speed rail system is not expected to meet all of  California’s future intercity travel demand. Among other things,  development of the phase I blended approach will affect the need for  additional improvements to local transportation systems to support high- speed rail. For example, officials with the Orange County Transportation  Authority told us they have bus projects and a street car project in the  planning phases that will be linked to both local commuter rail and the  high-speed rail system. Similarly, officials from the Southern California  Association of Governments told us the cost of integrating high-speed rail  with local transit was still being developed and that the cost of an initial  list of prioritized projects to facilitate this integration exceeded $3 billion.  Of this $3 billion in projects, about $1 billion in projects were categorized  as high priority. The officials said that although they have got agreement  from the Authority to help fund some of the high priority projects and  some funding is expected to come from Proposition 1A, a funding source  for the list of potential projects had not been determined.
Even though high-speed rail is not expected to meet all of California’s  future intercity travel demand, statewide transportation planning has not  yet fully assessed the impact of the high-speed rail system in meeting this  demand. In November 2011, the California Transportation Commission  (CTC) issued a Statewide Transportation System Needs Assessmentthat identified the preservation, management, and expansion projects  required over the 2011 to 2020 period. The assessment identified a total  cost for the projects of about $540 billion and a nearly $300 billion funding  gap in meeting the project needs identified. According to CTC officials,  the needs assessment included information about high-speed rail  that  was readily available from Authority documents. However, high-speed rail  was not assessed in terms of project needs, costs, or the funding gap  because, according to CTC officials, Regional Transportation Plans that  formed the basis of the assessment did not include high-speed rail in  identifying project needs and costs. Similarly, California Department of  Transportation officials told us current highway transportation planning  has not looked at this issue, and the department’s long range  transportation plans have not included consideration of high-speed rail.  According to the officials, the department did not see an immediate need  to do an assessment since the high-speed rail system is not expected to  be operational for another 10 years or more. The officials agreed that the  high-speed rail system will affect highways, and its impact will need to be  considered in future transportation plans.
section_title: Conclusion
paragraphs: As currently conceived, California’s high-speed rail project is expected to  be among the most expensive infrastructure projects that has been  undertaken in this country. Therefore, concerns about potential cost  escalations and optimistic ridership forecasts, as well as the potential  burden this could place on public budgets are well placed. Cost and  revenue estimates for large projects are, by their nature, imprecise; these  estimates endeavor to predict many years into the future within the  confines of what is known today. According to our past work reviewing  high-speed rail projects in other countries, cost and ridership estimates  tend to be overly optimistic. However, experts agree that taking steps to  anticipate project risks and improve the credibility of such estimates will  lower the risk of cost overruns and missed revenue forecasts. Improving  the reliability of cost and revenue forecasts is critical to providing project  sponsors, FRA, the Congress, and ultimately, the public with greater  confidence that this project can be viable. This confidence is of particular  importance as the Authority will seek significant and sustained funding  from federal, state, and private sources.
We found that the Authority did not fully employ best practices for  producing reliable cost estimates as expressed in GAO’s Cost Guide,  which are recommended practices but not required. The cost estimates  can be improved as the project progresses from design to construction,  and ultimately, to operation. The Authority was not required to follow the  Cost Guide; instead, it was required to follow FRA’s guidance which we  found to be limited. That guidance identified the cost categories that  applicants should include in its cost analyses, but did not specify how cost  estimates should be generated. The Authority told us that it looked to  FTA’s cost-estimating guidance to help inform the Authority’s cost- estimating methodology. The Authority can be commended for  supplementing its analyses using FTA’s guidance, but this does not  necessarily ensure a fully reliable cost estimate. Our past work, as well as  that of the DOT OIG, has shown that FRA has yet to develop sufficient  program guidance for project evaluation and oversight under its additional  grant-making responsibilities under the HSIPR program. Developing  guidance for HSIPR applicants and grantees that incorporates best  practices from the Cost Guide would allow cost estimators to improve the  reliability of cost estimates for expensive projects like high-speed rail.  Such guidance would help ensure project costs reflect the four  characteristics required for developing reliable cost estimates and  minimize the risk of cost overruns, missed deadlines, and unmet  performance targets.
The Authority is in the process of updating its ridership and revenue  model in response to recommendations provided by experts and peer  review groups, such as the Ridership and Revenue Peer Review Panel.  We believe that these steps will have the potential to improve the  Authority’s ridership forecasts, and we encourage continued refinement,  as more information becomes available and continued review by peer  review groups. Improved forecasts will be particularly important as the  Authority seeks to secure private investment in the project. The potential  project revenues—which are primarily dependent on ridership—will help  determine how much the Authority may be able to obtain from private  sources. Similar to the cost estimates, FRA has developed minimal  guidance for applicants to develop reliable ridership and revenue  forecasts. We, along with the DOT OIG, have previously made  recommendations that FRA improve this guidance to ensure reliability of  ridership and revenue forecasts that are used to determine the viability of  high-speed rail projects. According to FRA officials, the agency is  currently in the process of implementing these recommendations. Since  we have already made recommendations to FRA on this issue in our prior  work and FRA is taking actions on these recommendations, we are not at  this time making additional recommendations related to improving FRA’s  ridership and revenue forecasting guidance.
section_title: Recommendation
paragraphs: We recommend that the Secretary of Transportation direct the  Administrator of FRA to improve its guidance for high-speed rail project  sponsors to better ensure that cost estimates that are submitted by  applicants seeking federal funding are accurate, comprehensive, well- documented, and credible according to the best practices detailed in  GAO’s Cost Guide.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOT and the Authority for review and  comment. DOT neither agreed nor disagreed with our recommendation.  In an e-mailed response, DOT said it was pleased that the Authority met  many of the criteria in the Cost Guide for producing accurate,  comprehensive, well-documented, and credible cost estimates; that  ridership and revenue forecasts were reasonable; and that the Authority  did a comprehensive job in identifying potential economic impacts.  However, DOT noted (1) that the currently funded project has sound cost  estimates while future, currently unfunded phases will continue to be  refined as the project progresses and data improves, (2) that FRA’s cost- estimating guidance was the best available at the time, and (3) that  GAO’s Cost Guide focuses on federally managed acquisitions. DOT’s  response noted that the project is a multi-decade effort consisting of  many segments and phases, each in a different stage of development.   DOT’s response also noted that the Cost Guide was issued in March  2009, one month after passage of the Recovery Act and a few months  before deadlines for the HSIPR guidance, and, therefore, it was not  feasible for FRA to incorporate Cost Guide best practices into guidance.  Finally, DOT said applying the Cost Guide principles to future FRA capital  cost guidance, while feasible, would require analysis and adaptation to  accommodate unique aspects of long-term grantee-managed  transportation projects. DOT noted that the Cost Guide is focused  primarily on federally managed acquisitions and programs, not  infrastructure projects that non-federal parties will develop and build. DOT  also provided technical comments that we incorporated as appropriate.
We recognize DOT’ concerns; however, our charge was to assess the  reliability of the cost estimates and not just whether they complied with  FRA’s guidance, which we found to be less than best practices.  While  the Cost Guide was released in 2009, it is a culmination of cost- estimating best practices that have previously been published and have  been available to federal agencies for many years. Therefore, these  practices could have been considered when preparing HSIPR program  guidance. Finally, the best practices contained in the guide are applicable  to developing cost estimates for a wide variety of programs and projects,  whether federally managed or not.
The Authority provided a letter summarizing its comments about the  report (see app. IV). In general, the Authority believes that the report  highlights its efforts to produce cost estimates that reflect the scope of the  project, that methods and models used to develop ridership and revenue  forecasts adhered to applicable best practices, and that comprehensively  identifying potential economic impacts demonstrates a strong economic  case for the project. However, the Authority noted that different  components of its program (such as implementation phases and  construction packages) are at different stages of development and that it  would not be practicable to apply the full complement of tools in the Cost  Guide at the program level at this time. This is because the environmental  review is under way on a number of sections and alignments and other  choices are still to be made. The Authority also stated it plans to improve  its cost estimates and ridership forecasts. For example, the Authority  stated that the updated ridership model being developed for the 2014  business plan will incorporate many of the changes we suggested and  that the Authority will improve the quantification of project risks.
We commend the Authority for planning to improve its cost estimates and  forecasts. Regarding not applying the Cost Guide at the program level,  we note that a program does not need to be in an advanced stage of  planning in order to complete a sensitivity or cost risk and uncertainty  analysis. In fact, such analyses are most valuable when performed early  in a program’s life cycle. Single point estimates are more uncertain at the  beginning of a program because less is known about its detailed  requirements and the opportunity for change is greater. For example,  undefined or unknown technical information, uncertain economic  conditions, and political issues are often encountered during a program’s  acquisition. For management to make good decisions, the program  estimate must reflect the degree of uncertainty, so that a level of  confidence can be given about the estimate. Therefore, it is important to  conduct a risk and uncertainty analysis at all stages of a project so cost  estimates reflect the risk and uncertainty that exist.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution of this report until 30  days from the report date. At that time, we will send copies of this report  to the appropriate congressional committees, the Secretary of  Transportation, the Administrator of FRA, and the Director of OMB. The  report will also be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-2834 or flemings@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report assesses (1) the reliability of the California High-Speed Rail  Authority’s (Authority) estimates of the project’s costs; (2) the  reasonableness of the Authority’s passenger rail ridership and revenue  forecasts; (3) the risks to the Authority’s plan to fund the project; and   (4) the comprehensiveness with which the Authority identified potential  economic impacts of the project. Our analysis focused on the Authority’s  cost estimates, ridership and revenue forecasts, and economic estimates  presented in the April 2012 revised business plan.
To address these objectives, we reviewed numerous documents,  including Federal Railroad Administration (FRA) guidance, Department of  Transportation Office of Inspector General (DOT OIG) reports, prior GAO  reports, and pertinent legislation. In addition, we obtained documents  from and conducted interviews with Authority officials to obtain  information about the Authority’s process for developing its various cost  estimates, ridership and revenue forecasts, and economic impact  estimates. Specifically, we reviewed the November 2011 draft business  plan and April 2012 revised business plans and documentation used to  develop the analysis presented in those plans. We conducted interviews  with the Authority’s contractors—Parsons Brinckerhoff and Cambridge  Systematics—to obtain additional information about the Authority’s  processes for developing these estimates and to clarify information in  their written documentation. In addition, we also conducted interviews  with officials from various federal and state agencies, peer review groups,  academic experts, advocacy groups, and transit and local government  groups to obtain information on, among other things, their role with the  California high-speed rail project and their views on the Authority’s cost  estimates, financing plans, ridership and revenue forecasts, and potential  economic impacts. (See table 6 for a list of organizations and individuals  we interviewed for this study.)
To assess the reliability of the project cost estimates, we analyzed the  Authority’s cost estimating approach against GAO’s best practices found in  the 2009 GAO Cost Estimating and Assessment Guide (Cost Guide).  GAO designed the Cost Guide to be used by federal agencies to assist  them in developing reliable cost estimates and also as an evaluation tool  for existing cost estimates. To develop the Cost Guide, GAO cost experts  assessed measures applied by cost-estimating organizations throughout  the federal government and industry and considered best-practices for the  development of reliable cost-estimates. We analyzed the cost estimating  practices used by the Authority against these best practices. For our  reporting needs, we collapsed these best practices into four general  categories representing practices that help ensure that a cost estimate is  (1) accurate, (2) well documented, (3) comprehensive, and (4) credible.
After a review of all source data, including but not limited to electronic  cost models for both capital investment and operating and maintenance  phases, all supporting documentation, personal interviews, and  independent research, we assessed the extent to which the Authority met  these best practices on a five-point scale:
Not Met—Authority provided no evidence that satisfies any of the  criteria.
Minimally Met—Authority provided evidence that satisfies a small  portion of the criterion.
Partially Met—Authority provided evidence that satisfies about half of  the criterion.
Substantially Met—Authority provided evidence that satisfies a large  portion of the criterion.
Fully Met—Authority provided complete evidence that satisfies the  entire criterion.
We determined the overall assessment rating by assigning each  individual rating a number: Not Met = 1; Minimally Met = 2; Partially   Met = 3; Substantially Met = 4; and Fully Met = 5. For the purposes of this  assessment we have also included a Not Applicable (N/A) assessment  category. Then, we took the average of the individual assessment ratings  to determine the overall rating for each of the four characteristics. The  resulting average becomes the Overall Assessment as follows: Not Met =  0 to 1.4; Minimally Met = 1.5 to 2.4; Partially Met = 2.5 to 3.4;  Substantially Met = 3.5 to 4.4; and Fully Met = 4.5 to 5.0.
To assess the reasonableness of the Authority’s ridership and revenue  forecasts, we analyzed the extent to which the Authority’s methods for  developing the ridership model and resulting ridership and revenue  forecast adhered to federal guidance and generally accepted travel  demand modeling practices for high-speed rail projects. Unlike with  GAO’s cost-estimating criteria discussed earlier, there is no single  industry standard for developing or evaluating intercity passenger high- speed rail ridership forecasts. As such, for the purposes of our  assessment, we reviewed a variety of sources that identify generally  accepted travel demand modeling practices and developed criteria based  on these practices to assess the reasonableness of the approach used to  create the ridership and revenue models for the California high-speed rail  project. In developing our criteria, we relied primarily on a 2011 report  prepared for the DOT OIG’s office by the firm Steer Davies Gleave, on  best practices related to developing high-speed rail ridership and revenue  forecasts. The report provides a description of current standard practices  in high-speed rail ridership and revenue forecasting, key steps typically  involved in completing these forecasts, and a description on the range of  data and methods used in the forecasting process. The intent of this  guidance is to provide information that will assist reviewers to understand  and evaluate forecasting studies. In addition, we also examined other  literature on developing rail ridership and revenue forecasts to  corroborate information in the Steer Davies and Gleave report.  Specifically, we reviewed, among other sources, forecasting guidance  from the FRA, Federal Highway Administration (FHWA), prior GAO  reports and other ridership and revenue guidance in academic research.  (See table 7 for a list of sources used to develop criteria).
From these sources we identified recommended practices related to  seven key steps relating to: (1) developing trip tables, (2) determining and  applying service characteristics, (3) developing mode choice models,   (4) estimating induced travel, (5) estimating expected fare revenue,   (6) conducting sensitivity analysis, and (7) conducting validation testing.  We compared generally accepted practices for each of these steps to the  Authority’s process for developing the ridership and revenue forecast as  outlined in the April 2012 revised business plan and in supporting  technical documentation. We could not evaluate each of the many  detailed design decisions, assumptions, and model inputs used by the  Authority, but rather focused on the seven key steps and whether they  were implemented in accordance with generally accepted practices. We  reviewed documents from and conducted interviews with Authority  officials and their contractor—Cambridge Systematics—to obtain  information about the Authority’s process for developing the ridership and  revenue forecasts. Specifically, we examined the Authority’s process for  developing the models used to produce the various forecasts, the  assumptions and data sources used to develop the models, the survey  instruments used to collect data, and the Authority’s process for model  estimation, calibration, and validation.
We focused our analysis on identifying key steps in developing ridership  forecast models for high-speed rail projects, elements affecting validity  and reliability of models, common limitations of models and pitfalls, and  recommended approaches for external review. In addition, we interviewed  organizations that had conducted reviews of the Authority’s ridership and  revenue forecasts, such as academic experts from University of California  Berkeley’s Institute of Transportation Studies (ITS) and members of the  Ridership and Revenue Peer Review Panel and the California High- Speed Rail Peer Review Group. From these interviews, we obtained  additional information about (1) generally accepted methods used for  project ridership and revenue for high-speed projects and elements of  these approaches that have the greatest potential risk, (2) general  assumptions underlying demand forecast models, elements impacting  validity and reliability of models, and existing data limitations, and (3) the  extent to which the Authority’s approach follows generally accepted  practices for developing valid and reliable ridership and revenue  estimates.
To assess the Authority’s financing plan, we reviewed the plan,  conducted interviews with Authority officials and other state and federal  officials, and reviewed literature and other information on financing for  high-speed rail projects in other countries as well as large transportation  projects in the United States.
To assess how well the Authority identified economic impacts associated  with the high-speed rail project, we reviewed the April 2012 revised  business plan as well as the April 2012 benefit-cost analysis and April  2012 economic impact analysis. In addition, to establish criteria for the  various components of economic impact analysis, including user and non- user impacts, we reviewed pertinent legislation, such as the Passenger  Rail Investment and Improvement Act of 2008 and the National  Environmental Policy Act, and the NOFAs associated with the HSIPR and  Transportation Investment Generating Economic Recovery (TIGER) grant  programs. FRA officials told us the NOFA’s outlined the type of  information HSIPR grant applicants were to provide regarding project  benefits and costs and how this information would be reviewed by FRA in  reviewing grant applications. We also reviewed reports from the DOT OIG  regarding HSIPR project viability assessments. In particular, we reviewed  the June 2011 report prepared for the DOT OIG’s office on best practices  related to public benefit assessments of high-speed rail projects produced  by the firm Steer Davies Gleave. The Steer Davies Gleave report  identified important components of user and non-user impacts associated  with public benefits assessments.
To gain a better understanding of economic impact analysis, we reviewed  the Economic Impact Analysis Primer prepared by the FHWA’s Office of  Asset Management. The Economic Analysis Primer identified the basic  process of identifying and analyzing economic impacts, including benefit- cost analyses. The document also identified the similarities and  differences between benefit-cost analyses and economic impact  analyses. We also interviewed officials from FRA, the Authority, FHWA,  the Federal Transit Administration, DOT OIG, and Steer Davies Gleave  about economic impact issues. To assess issues related to high-speed  rail and future travel demand, we reviewed the Authority’s revised  business plan and April 2012 equivalent capacity study as well as the  November 2011 Statewide Transportation Systems Needs Assessment  prepared by the California Transportation Commission (CTC). We also  discussed these issues with officials from CTC and the California  Department of Transportation as well as officials with local transportation  agencies in California about potential improvement projects they had  planned that were associated with the high-speed rail project.
The proposed high-speed rail project is a very large public works project  with costs expected to be spread over more than a decade. Depending  on how cost figures are presented, different impressions of the magnitude  and funding requirements of the program could be given. Whether or not  the effects of inflation are included in the estimate is a source of  significant differences. Year of expenditure (YOE) dollars include inflation  in out-year costs, a convention adopted to facilitate budgeting over time  but not necessarily a good representation of the true economic costs of  the project. Removing the increase in cost attributable solely to inflation in  the price level provides a better picture of burden on taxpayers and other  funders because the tax base, including incomes, property values, and  retail sales, would have increased with inflation as well. In the case of the  high-speed rail project, the YOE cost total is 25 percent greater than that  when inflation effects are removed. An estimate of the cost in present  value terms, which accounts for inflation and the time value of money,  would be smaller still.
We conducted this performance audit from February 2012 to March 2013  in accordance with generally accepted government auditing standards.  These standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: GAO Analysis of California High- Speed Rail Authority’s Cost Estimates
subsections: section_title: Characteristic Accurate
paragraphs: The cost estimate results are unbiased,  not overly conservative or optimistic,  and based on an assessment of most  likely costs.
Individual assessment  Partially Met  While the Authority has attempted to ensure accuracy  and eliminate bias in their estimate by conducting  sensitivity analysis, parametric checks, and the use of  peer review, these have all been on subsets of the total  program. No risk or sensitivity analysis has been  developed at the program level or between the low and  high estimates. Alternative high and low estimates do not  create a range of estimates, but rather point estimates  evolving around potential options. In the absence of cost  risk and uncertainty analysis it is not possible to  determine if the estimate is unbiased. Unless the  estimate is based on an assessment of the most likely  costs and reflects the degree of uncertainty given all of  the risks considered, management will not be able to  make informed decisions.
The estimate has been adjusted  properly for inflation.
Substantially Met  Both capital investment and operations and maintenance  (O&M) costs are inflated to YOE dollars using sound  data and methodologies. Source data used for cost  estimating are normalized to appropriate base years,  although in some instances the normalizing processes  were not clear.
The estimate contains few, if any, minor  mistakes.
The cost estimate is regularly updated to  reflect significant changes in the  program so that it is always reflecting  current status.
Variances between planned and actual  costs are documented, explained, and  reviewed.
The estimate is based on a historical  record of cost estimating and actual  experiences from other comparable  programs.
Substantially Met  The estimate relies on construction cost data from  commercial databases heavily supplemented with local  construction bids from analogous construction projects.  The Authority collects technical and summary-level cost  data on existing and future high-speed trainsets, but  there is no documentation that explains how these data  were adjusted for use in the cost estimate. The O&M  estimate relies on applicable historical data. However,  the extent of applicability is unknown because  adjustments are not thoroughly documented.
section_title: Characteristic
paragraphs: The estimating technique for each cost  element was used appropriately.
Individual assessment  Substantially Met  The estimating techniques are reasonable for those  Standard Cost Categories (SCC) elements discretely  estimated, where a unit price estimating methodology  was employed. The 2012 O&M model is a simplified  version of the 2009 model, appropriately suited to the  cost model’s stated purpose of establishing program  viability. However, the simplification results in an  unnecessary loss of fidelity in some cost elements.
Substantially Met  The Authority has included all relevant costs with the  relatively minor exclusion of disposition costs in the  capital investment estimate.
Partially Met  The technical baseline description for the capital  investment cost estimate resides in multiple documents  that collectively comprise the technical baseline of the  program. However, there is no distinct technical baseline  description for the O&M estimate. Officials stated that  later versions of the O&M estimate will align with the  Concept of Operations plan, which was approved in  February 2012.
The cost estimate work breakdown  structure (WBS) is product-oriented,  traceable to the statement of  work/objective, and at an appropriate  level of detail to ensure that cost  elements are neither omitted nor double- counted.
Partially Met  The program utilized the FRA (SCC) and associated  definitions for the capital investment costs. The cost  estimate expands upon this structure to provide detailed  identification of infrastructure work, but has reduced  insight into common support costs. The standardized  O&M FRA SCC elements were not used for capturing  O&M costs because the O&M estimate was not required  to comply with the SCC elements. While the O&M  estimate includes common elements for administration  and support costs, the O&M WBS is greatly simplified.  As a consequence, up to two-thirds of O&M costs are  collected in a single cost element.
section_title: Characteristic
paragraphs: The estimate documents all cost- influencing ground rules and  assumptions.
Individual assessment  Partially Met  Ground rules and assumptions are imbedded in much of  the documentation for both the capital investment and  O&M estimates as well as in the cost models, but not all  assumptions have supporting rationale or sources. As  the design for a specific section advances, risks are  quantified and assigned to specific WBS elements. At  the program level, contingency factors are used to  capture less-defined risks. However, O&M risks are not  specifically related to O&M WBS elements, and the  impact of budget constraints on specific WBS elements  has not been clearly defined. In addition, the impacts of  technology maturity on cost are not entirely defendable.  Unless ground rules and assumptions are clearly  documented, the cost estimate will not have a basis for  areas of potential risk to be resolved.
The documentation should capture the  source data used, the reliability of the  data, and how the data were  normalized.
Partially Met  The documentation provides some insight into the  development of the cost estimates; however, much of  our analysis was based on information derived from  interviews and discussions with Authority  representatives, not from readily available information in  the documentation. The O&M model includes relevant  data, but sources and variables can only be described as  somewhat documented. For the most part,  documentation relates how inputs are adjusted from past  O&M models but fails to account for how earlier values  were derived. Without sufficient background knowledge  about the source and reliability of the data, the cost  estimator cannot know with any confidence whether the  data collected can be used directly or need to be  modified.
The documentation describes in  sufficient detail the calculations  performed and the estimating  methodology used to derive each  element’s cost.
Substantially Met  The documentation provided varying degrees of insight  into the estimating methodology. The majority of costs— that is, infrastructure and site work—are described at a  detailed level by unit cost, quantities, labor rates,  equipment, and material costs, and the like. However,  some cost elements had little or no supporting  documentation.
section_title: Characteristic
paragraphs: The documentation describes step-by- step how the estimate was developed so  that a cost analyst unfamiliar with the  program could understand what was  done and replicate it.
Individual assessment  Partially Met  Details of the estimating process and methodology were  provided for the track structure and track and site work  elements of the capital investment model, but supporting  data and details of how other elements are estimated  were not available. No comprehensive document exists  that explains the O&M model element by element.  Without good documentation, management and  oversight will not be convinced that the estimate is  credible. In addition, analysts unfamiliar with the program  will not be able to replicate the estimate because they  will not understand the logic behind it.
The documentation discusses the  technical baseline description and that  the data in the baseline are consistent  with the estimate.
Partially Met  The documentation of the capital investment cost model  and the technical baseline are consistent with one  another. The primary emphasis and underlying data  sources are for the infrastructure and site work, but little  definition or supporting data are provided for the  remaining cost elements. In addition, the O&M cost  estimate is not based on an approved technical baseline  document, although officials state that later versions will  be aligned to the Concept of Operations plan. Because  the technical baseline is intended to serve as the basis  for developing a cost estimate, it should be discussed in  the cost estimate documentation.
The documentation provides evidence  that the cost estimate was reviewed and  accepted by management.
Partially Met  Documents indicate that the Authority’s management  team was engaged in reviewing the cost estimates and  there are multiple indications that management reviewed  pieces of the cost estimate. However, many of these  reviews appear to be for subsets of the total program,  either by construction package or phase, and focusing  more on the financing rather than the detailed estimating  methodology or underlying assumptions. While specific  subsets of the estimate appeared to be reviewed by or  discussed with management, we found no specific  instance where the total program estimate, including  supporting source data and estimating methodologies,  was provided to senior management for review,  discussion, and subsequent approval.  Because a cost estimate should form the basis for  establishing the budget, it is imperative that  management understands how the estimate was  developed, including the risks associated with source  data and estimating methodologies.
section_title: Characteristic Credible
paragraphs: The cost estimate includes a sensitivity  analysis that identifies a range of  possible costs based on varying major  assumptions, parameters, and data  inputs.
Individual assessment  Partially Met  A formal sensitivity analysis has been performed for  Contract Package 1, design and construction of the first  26-33 miles of trackway infrastructure between the  counties of Madera and Fresno. In addition, the Authority  conducted limited sensitivity analysis on summary-level  variables in the O&M model. However, sensitivity  analysis of the entire program estimate has not been  done. The capital investment estimate includes low and  high cost alternative alignments, and the O&M estimate  provides three alternative scenarios driven by ridership  options. However, without a complete sensitivity analysis  that reveals how the cost estimate is affected by a  change in a single assumption, the cost estimator will not  fully understand which variable most affects the cost  estimate.
A risk and uncertainty analysis was  conducted that quantified the imperfectly  understood risks and identified the  effects of changing key cost driver  assumptions and factors.
Partially Met  The Authority utilized FRA guidance in developing its  estimates, guidance that does not require risk or  uncertainty analysis at the program level this early in the  design stage. Authority officials stated that more  advanced engineering designs are being developed to  support the process and that risk and uncertainty  analysis has been undertaken on Contract Package 1.  Authority officials acknowledge the existence of risk and  have tried to accommodate expected risk through the  application of contingency factors. While the capital  investment and O&M models include a contingency  element, the factors used do not appear to be based on  historical data or analogous sources.  Lacking risk and uncertainty analysis, management  cannot determine a defensible level of contingency  reserves that are necessary to cover increased costs  resulting from unexpected design complexity, incomplete  requirements, technology uncertainty, and other  uncertainties.
section_title: Characteristic
paragraphs: Major cost elements were crossed  checked to see whether results were  similar.
Individual assessment  Partially Met  The Authority recognizes the importance of crosschecks  and identified a series of crosschecks to verify and  validate the results of the data. Authority officials stated  there are several stages of crosschecks and quality  control which are described in their cost estimating  procedures. Yet little documentation has been provided  that would allow us to verify that crosschecks and  alternative methodologies have been developed. For  example, estimators have crosschecked major cost  factors in the O&M model with cost data from foreign  systems, but there is no evidence that costs have been  estimated using different methodologies.  The main purpose of cross-checking is to determine  whether alternative methods produce similar results. If  so, then confidence in the estimate increases, leading to  greater credibility. The Authority has contracted with the  International Union of Railways (UIC) for a study that  intends to verify and validate the capital investment cost  model. Authority officials stated that the UIC panel of  experts will also provide a set of international cost  comparisons for infrastructure maintenance and rolling  stock maintenance.
An independent cost estimate was  conducted by a group outside the  acquiring organization to determine  whether other estimating methods  produce similar results.
Minimally Met  An independent cost estimate (ICE) was performed on  the Merced-Fresno and Fresno-Bakersfield segments for  infrastructure costs. However, while the segments cover  35 percent of the planned system rail length, they make  up less than 10 percent of the overall estimated program  cost. An ICE should be performed on the entire program,  including O&M costs. ICEs can provide decision makers  with additional insights into a program’s potential costs  because they frequently use different methods and are  less burdened with organizational bias.
section_title: Appendix III: Description of Generally Accepted Travel- Demand-Modeling Practices and Authority’s Methods for Developing Ridership and Revenue Forecasts
subsections: section_title: Travel-demand-modeling tasks Developing trip tables
paragraphs: The Authority collected data from a variety of  sources including, among others,  socioeconomic data from local agencies, U.S.  Census Bureau, and the California Department  of Finance; travel data from various travel  surveys; and highway, air, conventional rail,  and urban transit network data from local  agencies.  The high-speed ridership and revenue model  for inter-regional travel was developed utilizing  surveys and other statewide travel information.  Intra-regional travel models from Metropolitan  Planning Organizations (MPOs) in the San  Francisco and Los Angeles regions were  adapted for use in the high-speed rail ridership  and revenue model from the models maintained  by the MPOs for those regions. A factoring  process was used to estimate ridership in the  San Diego region.  Base year trip tables were developed from  existing California regional models used by  local authorities including the Metropolitan  Transportation Commission (MTC) and the  Southern California Association of  Governments as well as interregional trip tables  developed from travel survey data. Forecast  year trip tables were developed by projecting  base year forecast data to forecast year 2030,  and then the models were run on 2030  projections.  Trip were segmented by long versus short trips  (over and under 100 miles), and trip purposes  (commute, business, recreation, other). development: The base and forecast year  input trip tables are the basis for a study’s  ridership estimates and revenue forecasts.  Any overestimate or underestimate of the trip  tables will translate to high or low forecasts of  ridership. Base trip tables generally  summarize the current total number of trips  by mode for each city pair along the route and  are generally prepared by using a variety of  sources of data on actual trip making  patterns. Growth factors—which determine  the rate of increase over time—can then be  applied to the base trip tables to develop  forecast year trip tables, which contain  estimates of future travel on various modes in  the absence of a proposed high-speed rail  alternative. Forecast year trip tables may also  be prepared by estimating future-year trips  directly.  Trip segmentation: Trip tables are generally  segmented by mode of travel, trip purpose,  and other traveler characteristics. Criteria  frequently used in defining market segments  include trip purpose, trip length, traveler  income, travel party size, and others.
section_title: Travel-demand-modeling tasks Determining and applying service characteristics
paragraphs: Authority’s methods for developing ridership  and revenue forecasts  The Authority developed a detailed network  representation for the entire state to  forecast travel between regions. Data   were obtained from the existing statewide  highway network and details were added  using data from local regional models, from  the MTC, the Southern California  Association of Governments, the San  Diego Association of Governments, and  data from the Kern County region.  LOS characteristics were defined for the  four inter-regional travel modes: auto,  conventional rail, high-speed rail, and air.  LOS characteristics covered three broad  categories: costs, times, and reliability,  which were summarized in travel skim  tables. Several of these characteristics  were varied during model application to  see how ridership and revenue would be  impacted. Characteristics were collected  from published or observed data from  various sources including, the MTC and the  Federal Aviation Administration. The high- speed rail characteristics were based on  the initial service plan and fare structure.
Network representation can be detailed (i.e.,  with detailed representations of street and  transit networks that include location,  alignment, connections, and service  characteristics) or can be less explicit and  instead focus directly on zone to zone level of  service data. A less explicit network  representation can be used if the structure of  the network is very simple).  Preparing skim tables: Skim tables contain  data on the time, cost, and other service  characteristics of the various modes that are  available for a trip.
Accurate and realistic representation of the base  and forecast year LOS characteristics is of  paramount importance for realistic high-speed  ridership forecasting. Rail LOS information may be  approximately derived from the service plan but  may not represent it in complete detail.
section_title: Travel-demand-modeling tasks Developing a mode choice model
paragraphs: Authority’s methods for developing ridership  and revenue forecasts  The Authority developed two choice models: intra- regional urban model (models behavior associated  with shorter distance and more frequent trip making)  and an inter-regional model (models traveler  behavior associated with longer-distance travel). predict the decisions of travelers considering  alternative transportation modes. Multinomial  logit models and nested logit model are types  of choice model that can be used.
Diversion choice model: A diversion choice  model considers only two modes–the one in  use in the base situation and the high-speed  rail alternative.
Intra-regional (Urban) Models: For both the  San Francisco Bay Area and the greater Los  Angeles regions, mode choice models were  adapted from existing models to include the  high-speed rail mode. The updated mode  choice models were applied using the MPO trip  tables for each region as input. San Diego is  the only other region that contains the  possibility of intra-regional high-speed rail trips,  but the estimate of these riders was very low  relative to the other regions. Because the level  of effort to develop, calibrate, and apply the  regional mode choice model was very high,  intra-regional ridership for San Diego was  developed using a population-based estimate  rather than a traditional mode choice model.  Inter-regional models: The Authority  developed four sets of models which included  trip frequency, destination choice, primary  mode choice, and access/egress mode choice.  The destination choice component predicts the  destinations of the trips generated in the trip  frequency component based on zonal  characteristics and travel impedances. The  mode choice components (main mode choice,  access mode choice and egress mode choice)  predict the modes that the travelers would  choose based upon the modal service levels as  well as characteristics of the travelers and trips  being made.
Data were derived from, among other sources,  the California Department of Transportation  Statewide Model, existing regional mode choice  models, and revealed preference and stated  preference survey data. The economic and  household characteristics were forecast for  each zone in the year 2030 based on data and  forecasts from state, regional, and local  government agencies. The primary main mode  choice model relied primarily on stated  preference data.
section_title: Conduct model validation
paragraphs: Generally accepted travel-demand-modeling  practices  A high-speed rail project will improve the overall  level of service for intercity travel within a given  corridor. This improvement will make conditions  more favorable for travel. Trips will therefore be  taken on high-speed rail that might not otherwise  have been made using any of the current modes.  The new trips are commonly referred to as  induced travel.  An upper limit on induced travel of approximately  10 percent of total high-speed rail trips is widely  accepted for proposed high-speed rail systems in  the U.S.  Expected fare revenue is determined by a  calculation using the ridership estimates  generated by the model and the average fares.  The total ridership for the system is generally  calculated by adding the diverted trips calculated  from the mode choice models and the induced  trips to produce the total ridership for the system.  All ridership and revenue forecasting studies  should incorporate an analysis of the sensitivity of  forecast results to key inputs and modeling  assumptions including fare, running time, service  frequency, station locations, and assumptions  about socio-economic and travel growth in  forecast years. Sensitivity analysis typically is  conducted by varying, more or less systematically,  selected forecasting model inputs, parameters, or  assumptions (e.g., inflation rate or fuel cost)  around their “standard” value, running the model,  and examining the variation in outputs. Sensitivity  analysis can help determine the reliability  associated with the model output forecasts and  can help identify the factors that have greatest  impact on project ridership and revenue.  Model validation is a key component of ridership  and revenue forecasting and generally consists of  testing the validity of the model using data other  than (and usually newer than) the data from which  it was estimated, to assess how well the model  predicts actual ridership.  There are two superior (but not often performed)  ways of checking model performance: (1) the  historical method, in which a prior-year model is  used to forecast current travel, which is then  compared with actual current travel; and (2)  “backcasting,” in which a current year model is  used to estimate travel for a prior year, which is  then compared with actual travel in the prior year.  Backcasting is used by 5 percent of all and 13  percent of large MPOs.
Authority’s methods for developing ridership  and revenue forecasts  The Authority forecasted 2.05 percent induced travel  for the blended Phase 1 low scenario.
The Authority calculated fare revenue by multiplying  the ridership estimates generated from the ridership  model by the average high-speed rail fares  forecasted for each region-to-region pair.
Several sensitivity tests were done to determine  how the model reacted to different sets of  assumptions such as changes to fuel costs, travel  times, and fares. In addition, the Authority  developed an extreme case scenario to test the  sensitivity of the model to a series of downside  events, such as increased average rail travel time  from Merced to the San Fernando Valley (140 min.  instead of 126 min), decreased train frequency (3  trains per hour instead of 4 trains per hour during  peak times), lower auto-operating costs and lower  air fares (10 percent below actual 2009 average air  fares).
The Authority validated the model through tests  performed using Amtrak’s Acela service in the  Northeast Corridor (NEC) as input to the California  high-speed rail model and compared the output with  2008 actual ridership and 2030 NEC forecasts.  Efforts to validate the model by comparing to the  NEC appear reasonable. The NEC is not an ideal  test for the model but it is the only one available in  the U.S. Use of foreign systems would raise difficult  issues of comparability.
section_title: Appendix IV: Comments from the California High-Speed Rail Authority
section_title: Appendix V: GAO Contact and Staff Acknowledgment
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Paul Aussendorf, Assistant  Director; Russell Burnett; Jason Lee; Delwen Jones; Richard Jorgenson;  James Manzo (Technomics, Inc.); Maria Mercado; Susan Offutt; Paul  Revesz; Max Sawicky; Maria Wallace; and Crystal Wesco made key  contributions to this report.