Public Transportation: Use of Contractors is Generally Enhancing Transit Project Oversight, and FTA is Taking Actions to Address Some Stakeholder Concerns
Published: 2010-09-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Many states, cities, and localities are building or planning mass transit projects to meet the nation's transportation needs. The New Starts program--administered by the U.S. Department of Transportation's (DOT) Federal Transit Administration (FTA)--is an important source of new capital investment in mass transportation, providing grants to project sponsors (e.g., state and local government authorities), for the construction of major transit facilities. FTA uses contractors--known as project management oversight contractors (PMOC) and financial management oversight contractors (FMOC)--to help oversee the planning, construction, and financing of major capital projects, including those funded under the New Starts program. This report, as mandated by law, discusses (1) how FTA uses PMOCs and FMOCs to oversee New Starts projects and how the agency procures, monitors, and evaluates the contractors' services; and (2) the benefits of FTA's oversight approach and the challenges FTA faces in conducting its oversight. GAO reviewed applicable statutes, FTA guidance, regulations, and budget data, and interviewed DOT officials, project sponsors, contractors, and industry stakeholders. GAO is not making any recommendations in this report. DOT officials generally agreed with GAO's findings and provided technical comments, which we incorporated as appropriate.
section_title: What GAO Found
paragraphs: FTA procures PMOC and FMOC services to provide critical input into FTA's decisions regarding New Starts projects. Specifically, the reviews that PMOCs conduct keep FTA informed of a project's status and support the agency's decision on whether to advance or fund the project. Separately, financial assessments conducted by FMOCs help the agency ensure that project sponsors--which can be state or local government authorities that implement New Starts projects--have sufficient financial capacity to build and operate their projects. Although PMOCs and FMOCs have different oversight roles, services for both are procured in accordance with the Federal Acquisition Regulation (FAR). FTA recently changed how it procures PMOC services. Prior to 2009, FTA awarded PMOC contracts only to architectural and engineering firms for their services using specialized procedures in the FAR; however, in part to expand the pool of available PMOCs, FTA revised its procurement approach and now uses the competitive negotiation procedures in the FAR which permit cost and noncost tradeoffs. Using competitive negotiations has increased the pool of contractors available for FTA projects, and while some PMOC officials expressed concerns that the changes have affected the quality of staff provided for FTA work and put more emphasis on cost, FTA officials said that they have not observed any negative effects. FTA monitors PMOCs and FMOCs by setting performance expectations and using a multilayered system to evaluate their performance. Recently, FTA has taken steps to improve its contractor performance evaluation system to help ensure that its regional offices conduct evaluations consistently. FTA officials, contractors, and project sponsors identified benefits of FTA's oversight approach. For example, FTA officials and project sponsors said that FTA's oversight approach has improved project management, supplemented existing FTA staff, and provided insights through technical assistance and expertise from PMOCs and FMOCs. However, FTA's oversight program faces some challenges, including balancing project management oversight and advancing projects, managing the larger, more complex projects entering the New Starts portfolio, and communicating with project sponsors. FTA has taken some actions to address these challenges. To more effectively balance oversight and project advancement, FTA developed procedures to assist PMOCs with their oversight responsibilities and issued an Advance Notice of Proposed Rulemaking on, among other things, the extent to which the level of its oversight should be based on risk. As part of FTA's effort to oversee larger, more complex projects, FTA has directed its PMOCs and FMOCs to conduct oversight activities earlier in project development, helping identify potential problems earlier. To improve communications, FTA developed checklists which project sponsors found helpful in understanding FTA's oversight requirements. However, some project sponsors we spoke with said that FTA's communications were not consistently timely or clearly documented, thus delaying sponsors' responses and, sometimes, project time frames. FTA recognizes that the New Starts process can be lengthy, but officials indicate that project sponsors do not always provide FTA needed information.

section_title: Letter
paragraphs: Many states, cities, and localities are building or planning mass transit  projects to replace aging infrastructure or add new capacity to meet the  nation’s transportation needs. The federal government contributes funding  to a number of major mass transit projects each year, primarily by  covering a share of the capital costs through the Federal Transit  Administration’s (FTA) New Starts grant program.  Under the New Starts  program, FTA identifies and recommends new fixed guideway transit  projects or extensions to existing projects to Congress for grants, and  provides such grants to project sponsors, which can be state or local  government authorities, including transit authorities. Many New Starts  projects require large federal investments, take years to construct, and can  be complex because of unique design or construction elements.
To strengthen the management and monitoring of major capital transit  projects, in 1987, the Surface Transportation and Uniform Relocation  Assistance Act of 1987 (STURAA) authorized FTA’s project management  oversight program. Furthermore, in 1990, Congress authorized, and in  1991 FTA established, the financial management oversight program to help  mitigate the financial risks associated with transit projects. FTA’s project  and financial management programs use contractors—known as project  management oversight contractors (PMOC) and financial management  oversight contractors (FMOC)—to help FTA oversee the planning,  construction, and financing of major capital projects, including those  projects funded under the New Starts program.
In response to a requirement in the Safe, Accountable, Flexible, Efficient  Transportation Equity Act: A Legacy for Users (SAFETEA-LU) that we  review FTA’s New Starts program annually, this report discusses (1) how  FTA uses contractors to oversee New Starts projects and how the agency  procures, monitors, and evaluates the contractors’ services; and (2) the  benefits of FTA’s oversight approach and the challenges FTA faces in  conducting its oversight.
To address how FTA uses and monitors PMOCs and FMOCs to oversee  New Starts projects and the benefits and challenges to oversight, we  reviewed relevant legislation establishing project management oversight  responsibilities and the funding available for these oversight activities.  Additionally, we reviewed applicable federal statutes and regulations for  the procurement of oversight contractors, and FTA procedures and  guidance for PMOCs and FMOCs. We focused our review on the oversight  activities that informed FTA’s recommendation for federal funding for 10  New Start projects. We judgmentally selected these 10 projects to include  a range in (1) transit mode (i.e., heavy rail, light rail, commuter rail, or  bus), (2) the total project cost, (3) the amount of federal funding, and (4)  geographic areas. Our sample also includes project sponsors who have  been recommended for multiple full-funding grant agreements (FFGA) by  FTA. For each of these 10 projects, we interviewed New Starts project  sponsors, PMOCs, FMOCs, and FTA headquarters and regional officials.  The information from the interviews is intended to provide views on  oversight from stakeholders involved in a range of New Starts projects and  is not generalizable to all New Starts projects. Appendix I contains  additional information on our scope and methodology.
We conducted this performance audit from January to September 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: Project sponsors can begin constructing and operating New Starts projects  upon the successful completion of specific project development  milestones and upon obtaining an FFGA from FTA. To obtain an FFGA,  project sponsors are required by law to go through a planning and project  development process, which is divided into three phases: alternatives  analysis, preliminary engineering, and final design, followed by  construction. In the alternatives analysis phase of the New Starts process,  project sponsors identify the transportation needs in the corridor and  evaluate a range of modal and alignment alternatives to address the locally  identified problems in a specific corridor. Project sponsors complete the  alternatives analysis phase by selecting a locally preferred alternative  (LPA), which is the New Starts project that FTA evaluates for funding.  During the preliminary engineering phase, project sponsors refine the  design of the LPA, taking into consideration all reasonable design options  and estimating each option’s costs, benefits, and impacts (e.g., financial or  environmental). When the preliminary engineering phase is completed and  federal environmental requirements are satisfied, FTA may approve the  project’s advancement into final design, after which FTA may recommend  the project for an FFGA. To help inform administration and congressional  decisions about which projects should receive federal funding, FTA  distinguishes among proposed projects by evaluating and assigning ratings  to various statutorily prescribed evaluation criteria—including both  project justification and local financial commitment criteria—and then  assigning an overall project rating. Once a project obtains an FFGA, the  project sponsor can begin constructing and, subsequently, operating it.
PMOCs oversee project sponsors’ management of New Starts projects,  starting when a project prepares to enter the preliminary engineering  phase and continuing through the beginning of operations. The main  oversight reviews that a PMOC conducts before FTA recommends a  project for an FFGA include an evaluation of the project’s risk, scope,  cost, schedule, and project management plan, as well as the project  sponsor’s technical capacity and capability. (See table 1.) In addition,  PMOCs conduct monthly and quarterly oversight reviews. For some  reviews that PMOCs conduct, such as real estate reviews, PMOCs retain  the services of subcontractors with specific industry experience.
PMOCs perform many of these reviews repeatedly throughout the New  Starts process, tailoring them to the stage of a project’s development. For  example, FTA directs PMOCs to review a project sponsor’s technical  capacity and capability three times—before the project enters the  preliminary engineering phase, enters the final design phase, and again  when the sponsor requests an FFGA. According to FTA guidance, some  reviews, such as the technical capacity and capability review, require more  detail during the early stages of a project (i.e., during preliminary  engineering); however, during later stages of a project (i.e., during final  design or FFGA) these same reviews may only require an update of earlier  findings. See appendix II for a full list and description of the reviews  PMOCs conduct.
In contrast to PMOCs, who focus on project management oversight,  FMOCs oversee how project sponsors will fund the capital and operating  costs of the proposed project as well as the existing systems. FMOCs  conduct two types of financial assessments:    Local financial commitment assessments are short, less detailed reviews  which evaluate whether a project is supported by a stable and reliable  capital financial plan and whether the project sponsor can fund the  operations and maintenance of an existing transit system once the new  transit project is built. These assessments also review the amount of  funding that comes from sources other than New Starts, such as federal  formula grants and state and local funding.
Financial capacity assessments, which are more detailed reviews,  analyze a project sponsor’s financial condition and capability to fulfill its  current and future financial obligations.
While PMOCs provide continuous oversight of projects, FMOCs have  oversight responsibilities at defined points in the project development  process. Specifically, FMOCs conduct financial assessments before a  project enters preliminary engineering and final design stages and before  an FFGA is awarded. Figure 1 illustrates the New Starts project  development process and shows when PMOCs and FMOCs conduct their  oversight.
section_title: FTA Procures Contractor Services to Provide Input into Its Decisions and Monitors These Contractors through Various Methods
subsections: section_title: PMOCs and FMOCs Provide Critical Input to FTA on Decisions to Advance and Fund New Starts Projects
paragraphs: FTA officials rely on PMOCs and FMOCs to provide critical input into  FTA’s decisions on project advancement and funding. This input— including the PMOCs’ and FMOCs’ findings, recommendations, and  professional opinions—comes through reports that PMOCs and FMOCs  submit to FTA on the results of their reviews, as well as through  discussions that these contractors have with FTA officials on these results.
PMOC oversight efforts help FTA ensure that a federally funded transit  project’s scope, schedule, and cost are well developed and that the  project’s design and construction conform to applicable statutes,  regulations, and guidance. Furthermore, these efforts keep FTA informed  of a project’s status and support FTA’s decision on whether to advance the  project to the next phase of development or recommend the project for an  FFGA. For example, FTA officials said that based on a PMOC’s technical  capacity and capability review, they determined that a project sponsor did  not have sufficient staff—such as engineers and senior project managers— with the requisite qualifications and experience to advance the project  from the preliminary engineering to the final design phase.
Separately, FMOCs help ensure that project sponsors have sufficient  financial capacity to build and operate the proposed project as well as the  existing system. Specifically, FTA uses the analyses and reports that  FMOCs prepare, such as local financial commitment assessments, to  develop each project’s local financial commitment rating. FTA also uses  FMOCs’ financial assessments to anticipate financing problems and to  foster dialog with project sponsors about their ability to carry out planned  projects. For example, one FMOC we interviewed said its cash flow  analysis of a project found that the sponsor’s forecast of the project’s  schedule and cost would result in a cash flow shortfall. As a result, the  relevant FTA regional office had an early indication of possible problems,  and both FTA and the sponsor were able to proactively address the  project’s financing issues. In another instance, the FMOC determined  that the project sponsor, as part of the review of one of the 10 projects in  our sample, did not have the financial capacity to move forward on a  project, and because of this FMOC’s recommendation to FTA, FTA did not  advance the project to the next phase of development.
section_title: FTA Procures PMOC and FMOC Services Using Federal Acquisition Regulation Procurement Procedures, and Recently Revised Its Procurement Approach
paragraphs: FTA procures project management and financial management oversight  services in accordance with the Federal Acquisition Regulation (FAR),  which prescribes uniform policies and procedures for all executive  agencies to acquire goods and services. Under these contracts, PMOCs and  FMOCs provide oversight services, such as those described earlier, to  support FTA’s oversight of a number of grant programs, including New  Starts. FTA awards contracts for PMOC oversight services for 5 years and  for FMOC oversight services for a base year, with four 1-year options to  extend, for a maximum of 5 years. Funding for these contracts is  provided through statutory set-asides made available annually for specific  FTA grant programs. FTA is currently authorized to use up to 1 percent  of section 5309 capital investment program funds for oversight, whic includes PMOC and FMOC activities.
Federal agencies have discretion to determine what services they need to  procure and, based on the services being procured, what procurement  procedures are appropriate. One of the guiding principles of the FAR is to  deliver products or services that represent the “best value”—that is, the  acquisition outcome that provides the greatest overall benefit in response  to the agency’s requirements. An agency can obtain “best value” by using,  for example, the General Services Administration’s (GSA) Multiple Award  Schedule program or using competitive negotiations.
To acquire the services of FMOCs, FTA follows procedures prescribed in  the FAR. For local financial commitment assessments, it uses GSA’s  Multiple Award Schedule, as described in the FAR, and for financial  capacity assessments, FTA uses eligible firms certified under the Small  Business Administration’s (SBA) 8(a) Business Development Program.
FTA recently revised its approach to procuring PMOC services for two  primary reasons, the first of which was to expand the pool of available  PMOCs. Prior to 2009, FTA awarded contracts for PMOC services to  architectural and engineering firms in accordance with the Brooks Act and  its implementing regulations in the FAR, which prescribe specialized  policies and procedures for procuring “architect-engineer services.”  However, during the August 2004 through September 2009 contract period,  a growing number of mergers and acquisitions reduced the pool of  available architectural and engineering firms from 16 to 13. Partly in  response to this decrease in the number of contractors, FTA officials  reviewed the services its PMOCs were providing and determined that they  did not constitute “architectural and engineering services” within the  meaning of the Brooks Act and the FAR. Accordingly, FTA stopped using  Brooks Act procedures and, in 2009, began using the competitive  negotiation procedures described in the FAR. Negotiated acquisitions  allow trade-offs between cost or price and noncost factors and allow the  government to accept other than the lowest priced proposal, provided that  each factor’s relative importance is clearly stated in the contract  solicitation. As part of the 2009 procurement process, FTA officials said,  they separately evaluated the technical and cost proposals of potential  contractors and considered both qualifications and cost before starting  negotiations, whereas under the Brooks Act and its implementing  regulations, they had ranked competing firms on the basis of their  qualifications and then started negotiations on cost.
FTA’s second primary reason for changing its procurement procedures  was to reduce the potential for conflicts of interest. Many of the  contractors who FTA could potentially use as PMOCs have the knowledge  and skills not only to oversee transit projects for FTA, but also to support  project sponsors directly through an independent contract between the  contractor and a project sponsor. When a project sponsor contracts  directly with a contractor for work, this contractor becomes ineligible to  oversee that sponsor’s project for FTA as a PMOC, since this dual role  working for the project sponsor and providing oversight of the project for  FTA would constitute a conflict of interest. As a result, FTA identifies  conflicts of interest among its PMOCs before assigning a PMOC to a New  Starts project by analyzing whether a PMOC has contracted to work for a  project sponsor. According to FTA officials, FTA’s flexibility in assigning  PMOCs to New Starts projects was increasingly limited as mergers and  acquisitions reduced the pool of available firms during the 2004 through  2009 contract period. However, FTA officials indicated that their flexibility  with assigning contractors has increased with this new group of  contractors.
The change in FTA’s PMOC procurement procedures has increased the  number of available contractors. In 2009, when FTA first used the FAR’s  competitive negotiation procedures to procure PMOCs, the pool of  available contractors increased from 13 to 19, an increase of over 46  percent. According to FTA officials, this increase in the size of the PMOC  pool has helped increase competition when the contractors in the pool  compete for assignment to a New Starts project. FTA officials noted  further that the change in procurement procedures did not exclude  professional architectural and engineering firms from competing for the  contracts, but rather opened the competition to a wider group of  contractors. According to FTA officials, of the 19 contractors in the 2009  pool, 11 are contractors who worked as PMOCs during the 2004 through  2009 contract period, and 8 are project management contractors who are  participating for the first time as prime contractors.
Additionally, the change in FTA’s PMOC procurement procedures has had  the benefit of expanding small businesses’ participation in the PMO  program, according to FTA officials. They noted that during the 2004  through 2009 contract period, one PMOC was a small business and one  additional small business participated in the PMO program as part of a  mentor/protégé program. But as part of the change in its procedures, FTA  reduced the minimum number of hours required of PMOCs over the 5-year  contract period, which encouraged participation from small businesses.  Under the most recent procurement, conducted using competitive  negotiations, 3 PMOCs were small businesses.
However, the change in FTA’s procurement procedures may have some  negative effects, according to PMOCs. In their view, the competitive  negotiation process has affected their staffing for FTA work and puts more  emphasis on cost than did the Brooks Act process. For example, two  contractors we spoke with said PMOCs may have to use less experienced  personnel to provide oversight, and spend more time and money training  them, because they perceive that FTA now places increased emphasis on  cost as an evaluation factor during its procurement process. According to  these contractors, the most recent contract awards resulted in ceiling  prices for hourly compensation that were both lower than expected and  lower than the rates under the previous contracts. With the lower ceiling  prices, they said, some contractors may have to provide less experienced  staff to FTA to avoid exceeding the ceiling price per hour. For example,  officials from one PMOC noted that under the previous contract, the firm  regularly used a leading industry expert to perform New Starts risk  management reviews. However, under the new contract, the expert’s  hourly labor rate exceeded FTA’s ceiling price and the PMOC had to use a  different consultant from the same firm who was less qualified and could  not provide the same level of quality. In addition, PMOC officials stated  that more documentation is required under the new procedures but the  additional requirements have not added value to the process. For example,  they said that FTA asked the PMOCs to propose prices and identify staff  for 44 labor categories, but noted that 95 percent of PMOC activities are  completed using only a handful of the 44 labor categories. Finally, two of  the contractors we spoke with speculated that FTA’s use of competitive  negotiations could discourage some contractors from offering proposals  for future PMOC solicitations.
Despite these concerns, FTA officials maintain that the use of competitive  negotiations has not affected staffing or put greater emphasis on a  contractor’s price. FTA officials stated that there should be no decline in  the quality of the personnel provided by PMOCs because prior to any  change in the personnel assigned to a contract, FTA officials review the  résumés of the incoming and outgoing staff to ensure any personnel  changes will meet FTA’s qualification requirements. For example, an FTA  official noted that, for one PMOC contract, FTA recently rejected 20  percent of the proposed changes to contractor staff because the proposed  personnel did not meet the qualifications specified in the contract that was  awarded. Additionally, although FTA officials acknowledged that cost was  an evaluation factor reviewed in the award of the 2009 PMOC contracts,  they noted that it was weighed fifth among the eight factors and subfactors  considered during the evaluation of proposals. Furthermore, despite  contractors’ concerns about ceiling prices, FTA officials explained that the  ceiling prices were negotiated and agreed to by the PMOCs and FTA, and  were based on rates that had been audited by the Defense Contract Audit  Agency (DCAA). FTA officials also noted that if a PMOC’s overhead rates  increase, the contractor may have an opportunity to negotiate new  contract rates, subject to a review by DCAA.
section_title: FTA Sets Contractor Expectations, Uses a Team Approach to Monitor Contractor Activities, and Evaluates Contractor Performance through an Improved System
paragraphs: FTA sets expectations for contractors through contracts, task orders, and  written guidance. After signing a contract with a contractor, FTA issues  task orders which it expects the contractor to complete over the term of  the contract. FTA provides guidance for PMOCs—known as Project  Management Oversight Procedures—and guidance for FMOCs to assist the  contractors in the oversight reviews that they are required to conduct.  FTA further defines its expectations for PMOCs through work orders,  which identify shorter-term products and deadlines, along with expected  labor hours. For example, one task order covering 5 years directed the  PMOC to deliver up to three project management plan (PMP) reviews,  among other reviews, in accordance with the Project Management  Oversight Procedures. The related work order refines this expectation by  directing the PMOC to complete one of these reviews within the first 12  months and estimates the number of labor hours needed to do so. The  scope of the work defined in the work order depends on the phase of the  sponsor’s project development, prior oversight work done, and other  factors.
Both FTA regional and headquarters project management officials are  involved in monitoring PMOCs, while the FTA Office of Planning and  Environment at headquarters monitor FMOCs. The regional Task Order  Manager has the day-to-day responsibilities for overseeing PMOC products  and activities. For example, the Task Order Manager regularly reviews  contractor vouchers and invoices and monitors the status of PMOC  activities. Furthermore, both headquarters and regional officials, including  planners and engineers, review the reports for accuracy and approve the  reports that PMOCs develop to document the results of their oversight  activities. FTA program management officials at headquarters also ask  specialists within FTA, such as officials from FTA’s Office of Civil Rights,  the Office of the Chief Counsel, and other offices as needed, to review  PMOC reports. Separately, for FMOCs, FTA’s Office of Planning and  Environment officials are responsible for monitoring FMOC work and use  monthly FMOC status reports to do so. These reports include information  on work completed and scheduled, problems encountered, FMOC labor  hours, and items requiring FTA action. FTA officials provide feedback to  PMOCs and FMOCs on these work products, including edits, questions,  and corrections to reports provided to FTA.
In addition to reviewing reports, FTA officials meet with PMOCs and  FMOCs to monitor their performance. According to FTA officials, these  meetings keep FTA informed of PMOCs’ findings, issues of concern, and  recommendations for FTA or project sponsor action. For example, FTA  meets quarterly with the PMOCs as a group for a briefing on major issues  of concern and provides an opportunity for PMOCs to ask questions. FTA  regional officials and PMOCs indicate that in addition to these meetings,  FTA regional staff talk with PMOCs as needed. Several FTA regional  managers we spoke with hold scheduled biweekly teleconferences and  monthly meetings with the PMOCs, and according to FTA officials, many  PMOCs participate in the biweekly calls between FTA and the project  sponsors. FTA officials discuss a project’s progress and items to keep it on  track, and review PMOC reports with PMOCs prior to project sponsor  meetings. For example, FTA officials noted that PMOCs may contact the  FTA task order manager for clarification of an oversight requirement. In  addition, according to FTA officials, FTA provides training to PMOCs on  FTA’s expectations and the administration of the oversight contracts  usually through annual PMOC conferences, quarterly PMOC conference  calls, webinars, and the annual engineers’ meeting. Furthermore,  according to FTA officials, some regions provide additional training during  meetings with PMOCs. Because of the periodic nature of FMOC oversight,  FTA holds meetings with FMOCs, as needed, when the FMOC is  performing a specific task. FTA indicated that if FMOCs have questions  they can e-mail or phone FTA.
FTA officials have recently taken steps to improve their multilayered  performance evaluation system for PMOCs. FTA annually evaluates its  PMOCs on four criteria—cost, quality, timeliness, and business relations.  In the past, FTA found that the performance levels for these four criteria  were not defined with enough specificity and were not applied  consistently to PMOCs. To improve the application of these criteria, FTA  headquarters officials organized a team from the regional offices and  developed a worksheet in 2008 that defined these four criteria and related  levels of performance to help improve consistency in the application of the  criteria. For example, prior to this improvement, a “good rating” for  timeliness of performance was defined simply as “there are no, or minimal,  delays that impact achievement of contract requirements.” With the  enhancements due to the worksheet, FTA could evaluate, among other  things, the extent to which the PMOC identified problems early, the PMOC  provided the project sponsor with timely technical assistance that added  value to the project, and the PMOC met the original completion date for  task assignments or deliverables. In addition, for the 2009 contract, FTA  developed Acceptable Quality Levels (AQL) to improve the objectiveness  of the quality criterion. FTA applies AQLs to evaluate the quality of PMOC  reports against FTA’s Oversight Procedures. According to FTA officials,  the AQLs are a means of documenting FTA’s quality expectations for  reports and oversight activities. FTA incorporates the results of the AQLs  into its worksheet and uses the results of the worksheet and the AQLs to  assign a numerical rating for each of the four evaluation criteria. FTA  contracting officials have indicated that if there is a performance issue  with a contractor, the contractor is notified, either by phone or by letter,  and that such action has resulted in rectification of the performance issue.
FTA also informally obtains feedback on contractor performance from  project sponsors. While some project sponsors provided both solicited and  unsolicited feedback, others indicated they would like to have a more  formal mechanism for evaluating contractor performance.
Once FTA officials have gathered performance information and evaluated  a contractor, using the AQLs and the internally developed worksheet, they  annually place the evaluation and supporting information into the  Contractor Performance System (CPS)—a multiagency system used to  collect, maintain, and disseminate contractor performance evaluations for  federal departments and agencies. According to FTA officials, contractors  are forwarded a copy of the completed evaluation form. A contractor has  30 days to respond to or appeal an evaluation, whereupon FTA again  reviews the evaluation. After 30 days, the evaluation is considered official  and CPS delivers the finalized evaluation into the Past Performance  Information Retrieval System (PPIRS)—a federal system that collects  information on contractor performance across all federal agencies.  See  figure 2 for FTA’s current performance evaluation approach for PMOCs.
In general, PMOCs considered the reviews, evaluations, and feedback  provided by the FTA as positive and helpful in improving their ability to  carry out their oversight responsibilities. The evaluation and feedback  process is still new and implementation is still underway and, as such, two  regional managers noted they had not yet used AQLs to evaluate PMOCs.  Similarly, one PMOC indicated that FTA had not gone over AQLs with  them.
FTA’s Office of Planning and Environment directs and evaluates FMOC  work. FTA indicated that because there are fewer reports and the FMOC  work is periodic and not continuous, the Office of Planning and  Environment handles all contractor assignments and performance  evaluations. According to the FTA’s Office of Planning and Environment, it  has determined that the use of an AQL worksheet for consistency in  evaluations is not necessary since one office reviews all of the FMOCs’  work. The Office of Planning and Environment provides the contractor  evaluation form to the Office of Procurement, which enters the data into  CPS.
section_title: Stakeholders Identified Benefits of and Challenges to FTA’s Oversight, and FTA Is Taking Actions to Address Some Stakeholder Concerns
subsections: section_title: Stakeholders Cited Benefits of FTA’s Use of PMOCs and FMOCs, Including Improved Management and Access to Additional Expertise and Technical Assistance
paragraphs: FTA officials, contractors, and project sponsors cited numerous examples  of how FTA’s use of oversight contractors has improved project and  financial management. For example, one project sponsor noted that  feedback from the PMOC on the project management plan helped ensure  that the plan documents were complete and covered project costs  adequately. On a different project, officials from the project sponsor noted  that the PMOC’s risk management review helped identify program risks  that might have affected the project’s budget. These stakeholders also  noted that the FMOC’s financial oversight has helped improve projects.  For example, an FMOC’s independent assessment of a New Starts project  confirmed that the project sponsor had the resources to construct and  operate the project, despite a complex financing approach. In another  case, an FMOC determined that the New Starts project did not have the  financial capability to continue and recommended to FTA that the project  not advance to the next project development stage. Furthermore, we have  previously reported on how FTA’s use of PMOCs and FMOCs has  benefited both project sponsors and FTA. Specifically, we observed that  FTA’s oversight program has helped improve project sponsors’ controls  over project costs, schedules, quality, and safety. We noted that the PMO  program has provided FTA with a better understanding of the issues  surrounding complex construction projects and an increased awareness of  potential problems that could lead to schedule delays or cost increases.
PMOCs and FMOCs also provide FTA with additional expertise. Since the  inception of its Project Management Oversight program, FTA has  supplemented its own staff as well as used the expertise of PMOC staff,  and later, FMOC staff, who have specialized areas of expertise beyond that  of most FTA staff, to help oversee New Starts projects. Since the New  Starts program has grown significantly while FTA’s staff size has stayed  the same, FTA officials have used these oversight contractors to help  compensate for this human resource gap. For example, FTA officials noted  that while PMOCs are not FTA employees, the PMOCs help serve as FTA’s  “eyes and ears” at project sites.
Finally, in addition to oversight services, PMOCs occasionally provide  technical assistance on behalf of FTA to project sponsors. Such assistance  may include providing information and instruction in project management  and project analysis practices, or sharing technical expertise in transit  project design and construction. In the course of providing oversight,  PMOCs are to notify FTA about any opportunities for project sponsors to  benefit from technical assistance. FTA officials stated that New Starts  projects have benefited from the technical assistance that PMOCs and  FMOCs provided to project sponsors. However, while PMOCs and FMOCs  can provide technical assistance in the form of examples or best practices,  project sponsors are responsible for deciding how to proceed. For  example, FTA assigned a PMOC to provide guidance to a project sponsor  to help clarify safety requirements. As part of this process, the PMOC  provided the project sponsor with examples of best practices used to meet  safety requirements. The project sponsor used these best practices to  resolve its safety issues and noted that having access to the PMOC and its  specialized skills provided clarity and helped resolve the safety issue  quickly.
section_title: Finding the Appropriate Balance between Oversight and Advancing Projects Is Challenging; FTA Is Taking Actions to Help Address This Challenge
paragraphs: Finding the right balance between protecting federal investments through  project management oversight and advancing projects through the project  development process is challenging. As noted above, FTA’s project  management oversight program is intended to strengthen the management  and monitoring of major transit projects. However, determining the  appropriate level of oversight that is needed for a project is a delicate task,  since in certain cases, too much or too detailed oversight could slow a  project’s progress. We have previously reported that one option FTA could  consider to reduce the burden placed on project sponsors and move  projects more quickly through the New Starts project development  process is to tailor the level of oversight to the risks of a New Starts  project (indicated, for example, by the project’s cost or complexity).
FTA has developed procedures to assist PMOCs in ensuring that a  project’s scope, schedule, and cost are in balance and that the project  follows applicable federal requirements, but FTA has indicated that it also  tailors PMOC oversight to each project. Specifically, FTA’s Oversight  Procedures for PMOCs serve as guidance on how PMOCs should conduct  their work; however, according to FTA officials, the Oversight Procedures  are written to allow flexibility in the assignment of task and work orders  based on the specific needs of a project. According to FTA officials, while  some reviews in the Oversight Procedures are conducted for all projects  that undergo PMOC oversight, such as project management plan reviews,  FTA officials may decide not to have a PMOC conduct certain reviews or  may ask the PMOC to conduct a more detailed review than is described in  the Oversight Procedures. For example, FTA officials in an FTA region  told us that they did not require the PMOC working on a project in their  region to conduct a technical capacity and capability review because the  PMOC was familiar with the project sponsor’s technical capacity and  capability, the project sponsor had recently completed three other New  Starts projects on time and within budget, and there was no change in  project sponsor staff. In another example, an FTA official in another  region asked the PMOC overseeing one of the projects in that region to  conduct a real estate review that was not completely specified at that time  in the Oversight Procedures because of concerns about the project  sponsor’s real estate management approach. FTA officials noted that it is  impossible to anticipate all requirements and cover them all in the  Oversight Procedures. Therefore, FTA officials stated, it is possible to  deviate from the Oversight Procedures if the deviations are clearly  documented in the work order.
Despite FTA’s efforts to tailor its oversight, four project sponsors we  spoke with said that the level of oversight and PMOC involvement in a  project sometimes seem excessive or detailed, making it difficult for them  to spend time developing the project. For example, one project sponsor  we interviewed said that FTA has layered additional oversight on projects  in response to problems that FTA faced in other projects in other regions.  In this case, project sponsor officials said that some of the additional  oversight was not applicable to its project and that the project ended up  spending time and resources addressing additional oversight requirements  rather than developing the project. In addition, this project sponsor said  that the level of PMOC involvement on their projects is also excessive  because PMOC oversight is detailed. For example, rather than providing  broad oversight over the project’s schedule, the project sponsor officials  said that the PMOC requested very detailed information regarding the  schedule. According to the project sponsor, as a result of this level of  oversight, they had to hire more staff, which increases project costs.
FTA currently oversees project sponsors in accordance with the Project  Management Oversight Rule (PMO rule). In September 2009, FTA issued  an Advance Notice of Proposed Rulemaking (ANPRM) to obtain public  comment on proposed modifications to the rule intended to increase the  effectiveness of its oversight approach. As part of the ANPRM, FTA is  considering changes to the PMO rule that could potentially base the level  of oversight on risk. According to FTA officials, amending the rule to  emphasize its risk-informed process would formalize and standardize  FTA’s current practice of tailoring its PMOC oversight to each project. A  few project sponsors who provided comments to FTA on the ANPRM said  that the level of oversight could be tailored to the experience of the  project sponsor, with sponsors with more experience in the New Starts  process undergoing less detailed oversight. Two of these project sponsors  also suggested that oversight could be tailored to the level of federal  investment. FTA is currently evaluating comments it received from  stakeholders on this ANPRM, and expects to issue the Notice of Proposed  Rulemaking in late 2010.
section_title: FTA Is Taking Actions to Address Challenges Related to Projects’ Growing Complexity and Is Seeking Stakeholder Input on Other Potential Changes to the PMO Program
paragraphs: FTA faces challenges in ensuring that it has specialized expertise and staff  to oversee the growing number of complex capital projects, including New  Starts projects. FTA, in its 2009 ANPRM, noted that the agency is  participating in a larger number of megaprojects. These megaprojects  often involve technology, design, or construction elements, such as  tunnels, that make a project more complex. For example, one megaproject  that is currently seeking New Starts funding from FTA requires the  construction of two new tunnels under a river. To ensure that  technologies, design, or construction elements, such as tunnels, do not  increase the costs and schedules of the complex projects, FTA requires  oversight by individuals who have specialized knowledge and experience  in these areas. Furthermore, FTA officials said that the number of projects  in FTA’s Small Starts program is growing and some Small Starts are fairly  large, complex projects, particularly those nearing the $250 million total  capital cost threshold of the program. Thus, demand for FTA oversight  resources is growing. The challenges associated with the growing number  of large projects and the additional demands they place on FTA’s oversight  are particularly significant given that FTA’s staffing levels have remained  about the same since 1982.
The financing of New Starts projects has also become more challenging.  To cover the large dollar amounts of some New Starts projects,  particularly the megaprojects, project sponsors are borrowing more  money to finance their projects and are increasingly relying on innovative  financing methods, such as public-private partnerships and value capture  strategies. In addition, because of the current economic climate, state  finances are stretched, forcing state project sponsors to finance these  projects through multiple sources or less traditional means. For example,  another New Starts project we reviewed is using an innovative approach  to finance an extension of an existing system. This project is funded  through a variety of new revenue sources, including bonds backed by  future revenue collections from a toll road. As states use such new funding  methods for projects, the financial capacity oversight work that FMOCs  conduct for FTA also becomes more involved, requiring more effort and  time to understand the funding approaches and ensure that project  sponsors can sufficiently fund their proposed projects, and also maintain  and preserve their current transit systems.
As noted earlier, PMOCs and FMOCs provide FTA with access to  specialized experience and knowledge and additional staff support, and  FTA has taken actions to use these contractors to a greater extent to  address the challenges related to the growing size and complexity of New  Starts projects. For example, FTA is involving PMOCs and FMOCs earlier  in the project development process. FTA, beginning in 2006, began to  assign PMOCs to oversee New Starts projects when the sponsor applies to  enter into preliminary engineering, rather than waiting until the project is  already in that stage. FTA officials said that identifying and addressing  problems at an earlier stage has helped them ensure that the projects are  more likely to stay within budget and on schedule. According to project  sponsors we spoke with, ensuring that a project is on schedule is  important because delays can significantly increase project costs.  Minimizing the potential for increased project costs and delays is  particularly relevant for large projects that present greater risks of project  cost increases or schedule delays because they use unique technology,  design, or construction elements. Additionally, in 2008, FTA began to ask  its FMOCs to conduct financial capacity assessments before projects  entered into final design (with an update performed at the FFGA stage),  rather than waiting until the projects were at the FFGA stage. FTA officials  told us that this change helps them identify financing problems earlier,  allowing project sponsors more time for corrective actions. Specifically,  FTA officials stated that a project sponsor may need additional time to  identify additional sources of funding if FTA’s review found that there was  insufficient funding or financial capacity to undertake the project.
In addition, FTA noted in its 2009 ANPRM that it is considering a potential  revision to the PMO rule that would better reflect the current environment  of a larger federal transit program and an increased number of larger,  more complex projects. Specifically, through this ANPRM, FTA is seeking  comments from stakeholders on how FTA should best use its PMOCs to  provide specialized expertise when needed; how the agency should use  PMOCs to supplement its limited staff in overseeing increasingly complex,  major capital projects; whether the use and role of PMOCs should be  expanded to overseeing projects other than major capital projects; and at  what stage FTA should assign PMOCs to New Starts projects, among other  things. FTA is also considering whether it should expand the PMO rule to  include a focus on project management, in addition to its current focus on  project oversight. FTA officials explained that the expanded focus would  establish performance expectations for project sponsors on certain  project management areas. For example, the PMO rule could be expanded  to include a requirement that project sponsors have sufficient staff in  place to demonstrate that they have the capacity and capability to develop  their project. FTA officials explained that by ensuring that project  sponsors have the appropriate project management skills, the sponsors  can better control their project costs and schedules.
section_title: FTA and PMOCs Have Multiple Ways to Communicate with Project Sponsors during the Oversight Process, but Project Sponsors Say Communication Challenges Still Exist
paragraphs: Effective oversight management of FTA’s New Starts program and project  sponsors’ advancement of transit projects through the New Starts program  depend, in part, on effective communication. We have noted that an  effective communication strategy should facilitate an honest two-way  exchange with, and allow for feedback from, stakeholders. Such  communication is central to forming the effective internal and external  partnerships that are vital to a program’s success. GAO has also noted  that relevant and timely information is needed for an agency to achieve its  objectives.
FTA has multiple avenues through which it communicates with project  sponsors about its oversight activities. Among its efforts, FTA informs  project sponsors of the steps they need to take in the New Starts funding  process; addresses New Starts issues through training workshops,  guidance letters, circulars, and quarterly meetings with the project  sponsor and the PMOC in attendance; and provides feedback on PMOCs’  reports on sponsors’ projects. Furthermore, in 2008, FTA developed  checklists to provide detailed information to project sponsors on the  submittals required for each project development phase. Sponsors have  indicated they found this tool and the meetings to be very helpful in  understanding FTA’s and PMOC’s oversight requirements.
However, three project sponsors we spoke with stated that  communications with FTA and PMOCs were not consistently timely,  which delayed their response to FTA’s concerns. For example, one project  sponsor we interviewed noted that during final design the sponsor was  informed of a level boarding issue pertaining to the Americans with  Disabilities Act, which negatively affected the project’s schedule and cost.  The project sponsor said that these effects might have been avoided if FTA  had raised the issue with the project sponsor earlier in the preliminary  engineering stage. In addition, another project sponsor we interviewed  said that waiting for FTA to approve its readiness reviews led to additional  costs, since the project consultants were idle during this time. Some of the  consultants cost the project up to $50,000 to $200,000 per month and the  project sponsor risked losing these consultants to other projects. A third  sponsor indicated that it gets final reports months later, by which time the  reports can be outdated. This practice can raise issues that may have  already been addressed, can affect relations with other funding sources,  and can result in late project delivery.
These apparent delays may occur for a number of reasons. FTA officials  noted that project sponsors sometimes provide information that is  incomplete or inaccurate, resulting in additional review time and delays.  An FTA headquarters official explained that they may delay providing  sponsors with final oversight reports until they make a decision, positive  or negative, to move to the next phase of project development. Similarly,  an FTA regional official we spoke with stated that they have delayed  sharing reports with project sponsors when the reports included sensitive  information, such as the impact of risk assessment on cost estimates used  as the basis for FTA funding decisions, which affect ongoing funding  negotiations between the project sponsor and FTA. In addition, an FTA  regional official and a PMOC noted that because FTA uses a team  approach to oversee the New Starts program, there are many people  involved in reviewing these final reports, which might increase the time  needed for reviews. For example, after the FTA regional office initially  reviews a report focused on a capital cost estimate review, the report is  reviewed by FTA headquarters officials, including those from the program  management office who supervise the regional officials and PMOC  technical work. Furthermore, FTA’s Office of Planning and Environment  as well as other officials, including representatives from the Office of Chief  Counsel and Office of Civil Rights, also review PMOC reports as needed.  FTA has acknowledged that its current New Starts process can be lengthy  and frustrating. In an October 2009 speech, the FTA administrator stated  that FTA currently has a decision-making process that is frustrating,  lengthy, and incomprehensible, and that FTA is working on a streamlined  decision-making process; he cautioned however, that the decision may not  always be to approve a project. Finally, as noted previously, FTA officials  said that they are involving PMOCs earlier in the project development  process to avoid problems.
In addition to communication delays, three project sponsors we spoke  with said that they provide a considerable amount of documentation to the  PMOCs and FTA and do not get clear written reactions to this information.
Five out of the six FTA regions we interviewed indicated that they do not  consistently provide project sponsors with draft PMOC reports. FTA’s  oversight procedures for PMOCs direct the PMOCs to discuss draft report  findings with project sponsors before finalizing reports. FTA officials  stated that PMOCs are expected to confirm the facts of the draft report  with project sponsors. An FTA official indicated that the PMOCs are  expected to share the facts with the project sponsor, and that project  sponsors are “generally aware” of the issues identified in the oversight  reports because of its communications with PMOC and FTA officials.  Project sponsors believe getting draft reports could help them (1) ensure  that PMOCs and FTA have the proper, most up-to-date information, and  (2) understand the context of concerns or requests for more information.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOT for its review and comment.  DOT officials generally agreed with our findings and provided technical  comments, which we incorporated as appropriate.
We are sending copies of this report to interested congressional  committees and the Secretary of Transportation. In addition, this report  will be available at no charge on GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-2834 or wised@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Under the Safe, Accountable, Flexible, Efficient Transportation Equity Act  - A Legacy for Users (SAFETEA-LU), we are required to report annually on  the Federal Transit Administration’s (FTA) New Starts process.  Accordingly, the objectives of this report were to review (1) how the FTA  uses contractors to oversee New Starts projects and how the agency  procures, monitors, and evaluates the contractors’ services, and (2) the  benefits of FTA’s oversight approach and the challenges FTA faces in  conducting its oversight. Our review focused on oversight activities that  project management oversight contractors (PMOC) and financial  management oversight contractors (FMOC) conduct between the  preliminary engineering stage of the New Starts project development  process and the full-funding grant agreement (FFGA) stage that inform  FTA’s recommendations for New Starts funding. Furthermore, we focused  on New Starts projects, and not on Small Starts or Very Small Starts  projects, since FTA does not generally assign PMOCs to Small or Very  Small Starts projects.
To address our two objectives, we reviewed relevant legislation and FTA  documents. Specifically, we reviewed SAFETEA-LU and other relevant  legislation, such as the Surface Transportation and Uniform Relocation  Assistance Act (STURAA), which authorized the project management  oversight program and provided funding for these oversight activities. In  addition, we reviewed and analyzed FTA documents, including  procedures, guidance, and performance evaluation criteria for PMOCs and  FMOCs. We also reviewed FTA’s 1989 project management oversight rule;  FTA’s Advanced Notice of Proposed Rulemaking (ANPRM), which seeks  public comment on a proposed amendment to this rule, and FTA  information on the agency’s budget for its project and financial  management oversight programs.
In addition to these efforts, to determine how FTA procures oversight  contractors, we reviewed FTA’s procurement solicitations for PMOCs and  FMOCs, reviewed GAO bid protest decisions, and interviewed FTA  procurement officials in headquarters. Furthermore, we reviewed  procedures for acquisition under the Brooks Act and the Federal  Acquisition Regulation (FAR), including procedures for competitive  negotiations.
We also conducted interviews on topics related to our two objectives. In  particular, we interviewed FTA officials in headquarters who are  responsible for managing and planning New Starts projects, as well as  officials familiar with the budget for FTA’s project and financial  management oversight programs. Furthermore, we interviewed  representatives from associations familiar with the New Starts program,  such as the American Public Transportation Association and the New  Starts Working Group.
Finally, we focused our review on 10 New Starts projects that FTA  recommended for FFGAs between fiscal year 2006 to fiscal year 2011. For  each of these 10 projects, we conducted semistructured interviews with  the project sponsors responsible for implementing the projects and the  PMOCs, FMOCs, and FTA regional officials responsible for overseeing the  projects. We judgmentally selected the 10 projects to include a range in (1)  transit mode (i.e., heavy rail, light rail, commuter rail, or bus); (2) total  project cost (from $117 million to $8.7 billion); (3) amount of federal  funding (from 26.7 percent to 80 percent of total project costs); and (4)  geographic area. Our sample also includes project sponsors who have  been recommended for multiple FFGAs by FTA. We interviewed all of the  project sponsors by telephone except the sponsor for the Dulles Corridor  Metrorail project, which is located near enough to Washington, D.C., for us  to interview the sponsor in person. Because the projects were selected as  a nonprobability sample, the results cannot be generalized to all projects  or the New Starts oversight process as a whole. Table 2 lists the 10 New  Starts projects we focused on for our review.
We conducted this performance audit from January to September 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: List and Description of Project Management Oversight Contractor Reviews
paragraphs: Table 3 describes the oversight reviews conducted by PMOCs and  described in the Federal Transit Administration’s (FTA) Project  Management Oversight Procedures, which were most recently revised by  FTA in May 2010. According to FTA officials, the procedures described in  this guidance were new to the project management oversight contracts  awarded in 2009 contracts, but are based on the program guidance  instructions for project management oversight contractors (PMOC) used  in prior contracts.
section_title: Appendix III: Funding Obligated for Project Management Oversight and Financial Management Oversight, Fiscal Years 2000-2009
paragraphs: Funding for PMOC and FMOC activities is provided through statutory set- asides made available annually for specific FTA grant programs. Federal  law currently authorizes FTA to use up to 1 percent of amounts made  available for its major capital investments, including New Starts, for  project management oversight, to fund the oversight activities of both  PMOCs and FMOCs. According to FTA, this funding is pooled with  funding that is authorized for FTA’s other oversight activities and then  allocated across FTA’s grant programs.
FTA obligates funding upon award of a contract to a PMOC or FMOC.  Table 4 describes obligations made for project management oversight and  financial management oversight activities for the New Starts program, as  well as other FTA grant programs. According to FTA officials, the funding  amounts obligated for oversight activities varies annually. FTA officials  stated that this funding is allocated for program management oversight  and financial management oversight activities based on where each capital  project, including New Starts projects, is in the project development  process and the oversight activities anticipated for each project in the  coming budget year.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Catherine Colwell, Assistant  Director; Eli Albagli; Lauren Calhoun; Roshni Davé; Elizabeth Eisenstadt;  Brandon Haller; Kristine Hassinger; Matthew Voit; and William Woods  made significant contributions to this report.