Public Transportation: Use of Contractors is Generally Enhancing Transit Project Oversight, and FTA is Taking Actions to Address Some Stakeholder Concerns
Released: 2010-09-14T00:00:00

Letter

Background

Project sponsors can begin constructing and operating New Starts projects upon the successful completion of specific project development milestones and upon obtaining an FFGA from FTA. To obtain an FFGA, project sponsors are required by law to go through a planning and project development process, which is divided into three phases: alternatives analysis, preliminary engineering, and final design, followed by construction. In the alternatives analysis phase of the New Starts process, project sponsors identify the transportation needs in the corridor and evaluate a range of modal and alignment alternatives to address the locally identified problems in a specific corridor. Project sponsors complete the alternatives analysis phase by selecting a locally preferred alternative (LPA), which is the New Starts project that FTA evaluates for funding. During the preliminary engineering phase, project sponsors refine the design of the LPA, taking into consideration all reasonable design options and estimating each option’s costs, benefits, and impacts (e.g., financial or environmental). When the preliminary engineering phase is completed and federal environmental requirements are satisfied, FTA may approve the project’s advancement into final design, after which FTA may recommend the project for an FFGA. To help inform administration and congressional decisions about which projects should receive federal funding, FTA distinguishes among proposed projects by evaluating and assigning ratings to various statutorily prescribed evaluation criteria—including both project justification and local financial commitment criteria—and then assigning an overall project rating. Once a project obtains an FFGA, the project sponsor can begin constructing and, subsequently, operating it.

PMOCs oversee project sponsors’ management of New Starts projects, starting when a project prepares to enter the preliminary engineering phase and continuing through the beginning of operations. The main oversight reviews that a PMOC conducts before FTA recommends a project for an FFGA include an evaluation of the project’s risk, scope, cost, schedule, and project management plan, as well as the project sponsor’s technical capacity and capability. (See table 1.) In addition, PMOCs conduct monthly and quarterly oversight reviews. For some reviews that PMOCs conduct, such as real estate reviews, PMOCs retain the services of subcontractors with specific industry experience.

PMOCs perform many of these reviews repeatedly throughout the New Starts process, tailoring them to the stage of a project’s development. For example, FTA directs PMOCs to review a project sponsor’s technical capacity and capability three times—before the project enters the preliminary engineering phase, enters the final design phase, and again when the sponsor requests an FFGA. According to FTA guidance, some reviews, such as the technical capacity and capability review, require more detail during the early stages of a project (i.e., during preliminary engineering); however, during later stages of a project (i.e., during final design or FFGA) these same reviews may only require an update of earlier findings. See appendix II for a full list and description of the reviews PMOCs conduct.

In contrast to PMOCs, who focus on project management oversight, FMOCs oversee how project sponsors will fund the capital and operating costs of the proposed project as well as the existing systems. FMOCs conduct two types of financial assessments: Local financial commitment assessments are short, less detailed reviews which evaluate whether a project is supported by a stable and reliable capital financial plan and whether the project sponsor can fund the operations and maintenance of an existing transit system once the new transit project is built. These assessments also review the amount of funding that comes from sources other than New Starts, such as federal formula grants and state and local funding.

Financial capacity assessments, which are more detailed reviews, analyze a project sponsor’s financial condition and capability to fulfill its current and future financial obligations.

While PMOCs provide continuous oversight of projects, FMOCs have oversight responsibilities at defined points in the project development process. Specifically, FMOCs conduct financial assessments before a project enters preliminary engineering and final design stages and before an FFGA is awarded. Figure 1 illustrates the New Starts project development process and shows when PMOCs and FMOCs conduct their oversight.

FTA Procures Contractor Services to Provide Input into Its Decisions and Monitors These Contractors through Various Methods

PMOCs and FMOCs Provide Critical Input to FTA on Decisions to Advance and Fund New Starts Projects

FTA officials rely on PMOCs and FMOCs to provide critical input into FTA’s decisions on project advancement and funding. This input— including the PMOCs’ and FMOCs’ findings, recommendations, and professional opinions—comes through reports that PMOCs and FMOCs submit to FTA on the results of their reviews, as well as through discussions that these contractors have with FTA officials on these results.

PMOC oversight efforts help FTA ensure that a federally funded transit project’s scope, schedule, and cost are well developed and that the project’s design and construction conform to applicable statutes, regulations, and guidance. Furthermore, these efforts keep FTA informed of a project’s status and support FTA’s decision on whether to advance the project to the next phase of development or recommend the project for an FFGA. For example, FTA officials said that based on a PMOC’s technical capacity and capability review, they determined that a project sponsor did not have sufficient staff—such as engineers and senior project managers— with the requisite qualifications and experience to advance the project from the preliminary engineering to the final design phase.

Separately, FMOCs help ensure that project sponsors have sufficient financial capacity to build and operate the proposed project as well as the existing system. Specifically, FTA uses the analyses and reports that FMOCs prepare, such as local financial commitment assessments, to develop each project’s local financial commitment rating. FTA also uses FMOCs’ financial assessments to anticipate financing problems and to foster dialog with project sponsors about their ability to carry out planned projects. For example, one FMOC we interviewed said its cash flow analysis of a project found that the sponsor’s forecast of the project’s schedule and cost would result in a cash flow shortfall. As a result, the relevant FTA regional office had an early indication of possible problems, and both FTA and the sponsor were able to proactively address the project’s financing issues. In another instance, the FMOC determined that the project sponsor, as part of the review of one of the 10 projects in our sample, did not have the financial capacity to move forward on a project, and because of this FMOC’s recommendation to FTA, FTA did not advance the project to the next phase of development.

FTA Procures PMOC and FMOC Services Using Federal Acquisition Regulation Procurement Procedures, and Recently Revised Its Procurement Approach

FTA procures project management and financial management oversight services in accordance with the Federal Acquisition Regulation (FAR), which prescribes uniform policies and procedures for all executive agencies to acquire goods and services. Under these contracts, PMOCs and FMOCs provide oversight services, such as those described earlier, to support FTA’s oversight of a number of grant programs, including New Starts. FTA awards contracts for PMOC oversight services for 5 years and for FMOC oversight services for a base year, with four 1-year options to extend, for a maximum of 5 years. Funding for these contracts is provided through statutory set-asides made available annually for specific FTA grant programs. FTA is currently authorized to use up to 1 percent of section 5309 capital investment program funds for oversight, whic includes PMOC and FMOC activities.

Federal agencies have discretion to determine what services they need to procure and, based on the services being procured, what procurement procedures are appropriate. One of the guiding principles of the FAR is to deliver products or services that represent the “best value”—that is, the acquisition outcome that provides the greatest overall benefit in response to the agency’s requirements. An agency can obtain “best value” by using, for example, the General Services Administration’s (GSA) Multiple Award Schedule program or using competitive negotiations.

To acquire the services of FMOCs, FTA follows procedures prescribed in the FAR. For local financial commitment assessments, it uses GSA’s Multiple Award Schedule, as described in the FAR, and for financial capacity assessments, FTA uses eligible firms certified under the Small Business Administration’s (SBA) 8(a) Business Development Program.

FTA recently revised its approach to procuring PMOC services for two primary reasons, the first of which was to expand the pool of available PMOCs. Prior to 2009, FTA awarded contracts for PMOC services to architectural and engineering firms in accordance with the Brooks Act and its implementing regulations in the FAR, which prescribe specialized policies and procedures for procuring “architect-engineer services.” However, during the August 2004 through September 2009 contract period, a growing number of mergers and acquisitions reduced the pool of available architectural and engineering firms from 16 to 13. Partly in response to this decrease in the number of contractors, FTA officials reviewed the services its PMOCs were providing and determined that they did not constitute “architectural and engineering services” within the meaning of the Brooks Act and the FAR. Accordingly, FTA stopped using Brooks Act procedures and, in 2009, began using the competitive negotiation procedures described in the FAR. Negotiated acquisitions allow trade-offs between cost or price and noncost factors and allow the government to accept other than the lowest priced proposal, provided that each factor’s relative importance is clearly stated in the contract solicitation. As part of the 2009 procurement process, FTA officials said, they separately evaluated the technical and cost proposals of potential contractors and considered both qualifications and cost before starting negotiations, whereas under the Brooks Act and its implementing regulations, they had ranked competing firms on the basis of their qualifications and then started negotiations on cost.

FTA’s second primary reason for changing its procurement procedures was to reduce the potential for conflicts of interest. Many of the contractors who FTA could potentially use as PMOCs have the knowledge and skills not only to oversee transit projects for FTA, but also to support project sponsors directly through an independent contract between the contractor and a project sponsor. When a project sponsor contracts directly with a contractor for work, this contractor becomes ineligible to oversee that sponsor’s project for FTA as a PMOC, since this dual role working for the project sponsor and providing oversight of the project for FTA would constitute a conflict of interest. As a result, FTA identifies conflicts of interest among its PMOCs before assigning a PMOC to a New Starts project by analyzing whether a PMOC has contracted to work for a project sponsor. According to FTA officials, FTA’s flexibility in assigning PMOCs to New Starts projects was increasingly limited as mergers and acquisitions reduced the pool of available firms during the 2004 through 2009 contract period. However, FTA officials indicated that their flexibility with assigning contractors has increased with this new group of contractors.

The change in FTA’s PMOC procurement procedures has increased the number of available contractors. In 2009, when FTA first used the FAR’s competitive negotiation procedures to procure PMOCs, the pool of available contractors increased from 13 to 19, an increase of over 46 percent. According to FTA officials, this increase in the size of the PMOC pool has helped increase competition when the contractors in the pool compete for assignment to a New Starts project. FTA officials noted further that the change in procurement procedures did not exclude professional architectural and engineering firms from competing for the contracts, but rather opened the competition to a wider group of contractors. According to FTA officials, of the 19 contractors in the 2009 pool, 11 are contractors who worked as PMOCs during the 2004 through 2009 contract period, and 8 are project management contractors who are participating for the first time as prime contractors.

Additionally, the change in FTA’s PMOC procurement procedures has had the benefit of expanding small businesses’ participation in the PMO program, according to FTA officials. They noted that during the 2004 through 2009 contract period, one PMOC was a small business and one additional small business participated in the PMO program as part of a mentor/protégé program. But as part of the change in its procedures, FTA reduced the minimum number of hours required of PMOCs over the 5-year contract period, which encouraged participation from small businesses. Under the most recent procurement, conducted using competitive negotiations, 3 PMOCs were small businesses.

However, the change in FTA’s procurement procedures may have some negative effects, according to PMOCs. In their view, the competitive negotiation process has affected their staffing for FTA work and puts more emphasis on cost than did the Brooks Act process. For example, two contractors we spoke with said PMOCs may have to use less experienced personnel to provide oversight, and spend more time and money training them, because they perceive that FTA now places increased emphasis on cost as an evaluation factor during its procurement process. According to these contractors, the most recent contract awards resulted in ceiling prices for hourly compensation that were both lower than expected and lower than the rates under the previous contracts. With the lower ceiling prices, they said, some contractors may have to provide less experienced staff to FTA to avoid exceeding the ceiling price per hour. For example, officials from one PMOC noted that under the previous contract, the firm regularly used a leading industry expert to perform New Starts risk management reviews. However, under the new contract, the expert’s hourly labor rate exceeded FTA’s ceiling price and the PMOC had to use a different consultant from the same firm who was less qualified and could not provide the same level of quality. In addition, PMOC officials stated that more documentation is required under the new procedures but the additional requirements have not added value to the process. For example, they said that FTA asked the PMOCs to propose prices and identify staff for 44 labor categories, but noted that 95 percent of PMOC activities are completed using only a handful of the 44 labor categories. Finally, two of the contractors we spoke with speculated that FTA’s use of competitive negotiations could discourage some contractors from offering proposals for future PMOC solicitations.

Despite these concerns, FTA officials maintain that the use of competitive negotiations has not affected staffing or put greater emphasis on a contractor’s price. FTA officials stated that there should be no decline in the quality of the personnel provided by PMOCs because prior to any change in the personnel assigned to a contract, FTA officials review the résumés of the incoming and outgoing staff to ensure any personnel changes will meet FTA’s qualification requirements. For example, an FTA official noted that, for one PMOC contract, FTA recently rejected 20 percent of the proposed changes to contractor staff because the proposed personnel did not meet the qualifications specified in the contract that was awarded. Additionally, although FTA officials acknowledged that cost was an evaluation factor reviewed in the award of the 2009 PMOC contracts, they noted that it was weighed fifth among the eight factors and subfactors considered during the evaluation of proposals. Furthermore, despite contractors’ concerns about ceiling prices, FTA officials explained that the ceiling prices were negotiated and agreed to by the PMOCs and FTA, and were based on rates that had been audited by the Defense Contract Audit Agency (DCAA). FTA officials also noted that if a PMOC’s overhead rates increase, the contractor may have an opportunity to negotiate new contract rates, subject to a review by DCAA.

FTA Sets Contractor Expectations, Uses a Team Approach to Monitor Contractor Activities, and Evaluates Contractor Performance through an Improved System

FTA sets expectations for contractors through contracts, task orders, and written guidance. After signing a contract with a contractor, FTA issues task orders which it expects the contractor to complete over the term of the contract. FTA provides guidance for PMOCs—known as Project Management Oversight Procedures—and guidance for FMOCs to assist the contractors in the oversight reviews that they are required to conduct. FTA further defines its expectations for PMOCs through work orders, which identify shorter-term products and deadlines, along with expected labor hours. For example, one task order covering 5 years directed the PMOC to deliver up to three project management plan (PMP) reviews, among other reviews, in accordance with the Project Management Oversight Procedures. The related work order refines this expectation by directing the PMOC to complete one of these reviews within the first 12 months and estimates the number of labor hours needed to do so. The scope of the work defined in the work order depends on the phase of the sponsor’s project development, prior oversight work done, and other factors.

Both FTA regional and headquarters project management officials are involved in monitoring PMOCs, while the FTA Office of Planning and Environment at headquarters monitor FMOCs. The regional Task Order Manager has the day-to-day responsibilities for overseeing PMOC products and activities. For example, the Task Order Manager regularly reviews contractor vouchers and invoices and monitors the status of PMOC activities. Furthermore, both headquarters and regional officials, including planners and engineers, review the reports for accuracy and approve the reports that PMOCs develop to document the results of their oversight activities. FTA program management officials at headquarters also ask specialists within FTA, such as officials from FTA’s Office of Civil Rights, the Office of the Chief Counsel, and other offices as needed, to review PMOC reports. Separately, for FMOCs, FTA’s Office of Planning and Environment officials are responsible for monitoring FMOC work and use monthly FMOC status reports to do so. These reports include information on work completed and scheduled, problems encountered, FMOC labor hours, and items requiring FTA action. FTA officials provide feedback to PMOCs and FMOCs on these work products, including edits, questions, and corrections to reports provided to FTA.

In addition to reviewing reports, FTA officials meet with PMOCs and FMOCs to monitor their performance. According to FTA officials, these meetings keep FTA informed of PMOCs’ findings, issues of concern, and recommendations for FTA or project sponsor action. For example, FTA meets quarterly with the PMOCs as a group for a briefing on major issues of concern and provides an opportunity for PMOCs to ask questions. FTA regional officials and PMOCs indicate that in addition to these meetings, FTA regional staff talk with PMOCs as needed. Several FTA regional managers we spoke with hold scheduled biweekly teleconferences and monthly meetings with the PMOCs, and according to FTA officials, many PMOCs participate in the biweekly calls between FTA and the project sponsors. FTA officials discuss a project’s progress and items to keep it on track, and review PMOC reports with PMOCs prior to project sponsor meetings. For example, FTA officials noted that PMOCs may contact the FTA task order manager for clarification of an oversight requirement. In addition, according to FTA officials, FTA provides training to PMOCs on FTA’s expectations and the administration of the oversight contracts usually through annual PMOC conferences, quarterly PMOC conference calls, webinars, and the annual engineers’ meeting. Furthermore, according to FTA officials, some regions provide additional training during meetings with PMOCs. Because of the periodic nature of FMOC oversight, FTA holds meetings with FMOCs, as needed, when the FMOC is performing a specific task. FTA indicated that if FMOCs have questions they can e-mail or phone FTA.

FTA officials have recently taken steps to improve their multilayered performance evaluation system for PMOCs. FTA annually evaluates its PMOCs on four criteria—cost, quality, timeliness, and business relations. In the past, FTA found that the performance levels for these four criteria were not defined with enough specificity and were not applied consistently to PMOCs. To improve the application of these criteria, FTA headquarters officials organized a team from the regional offices and developed a worksheet in 2008 that defined these four criteria and related levels of performance to help improve consistency in the application of the criteria. For example, prior to this improvement, a “good rating” for timeliness of performance was defined simply as “there are no, or minimal, delays that impact achievement of contract requirements.” With the enhancements due to the worksheet, FTA could evaluate, among other things, the extent to which the PMOC identified problems early, the PMOC provided the project sponsor with timely technical assistance that added value to the project, and the PMOC met the original completion date for task assignments or deliverables. In addition, for the 2009 contract, FTA developed Acceptable Quality Levels (AQL) to improve the objectiveness of the quality criterion. FTA applies AQLs to evaluate the quality of PMOC reports against FTA’s Oversight Procedures. According to FTA officials, the AQLs are a means of documenting FTA’s quality expectations for reports and oversight activities. FTA incorporates the results of the AQLs into its worksheet and uses the results of the worksheet and the AQLs to assign a numerical rating for each of the four evaluation criteria. FTA contracting officials have indicated that if there is a performance issue with a contractor, the contractor is notified, either by phone or by letter, and that such action has resulted in rectification of the performance issue.

FTA also informally obtains feedback on contractor performance from project sponsors. While some project sponsors provided both solicited and unsolicited feedback, others indicated they would like to have a more formal mechanism for evaluating contractor performance.

Once FTA officials have gathered performance information and evaluated a contractor, using the AQLs and the internally developed worksheet, they annually place the evaluation and supporting information into the Contractor Performance System (CPS)—a multiagency system used to collect, maintain, and disseminate contractor performance evaluations for federal departments and agencies. According to FTA officials, contractors are forwarded a copy of the completed evaluation form. A contractor has 30 days to respond to or appeal an evaluation, whereupon FTA again reviews the evaluation. After 30 days, the evaluation is considered official and CPS delivers the finalized evaluation into the Past Performance Information Retrieval System (PPIRS)—a federal system that collects information on contractor performance across all federal agencies. See figure 2 for FTA’s current performance evaluation approach for PMOCs.

In general, PMOCs considered the reviews, evaluations, and feedback provided by the FTA as positive and helpful in improving their ability to carry out their oversight responsibilities. The evaluation and feedback process is still new and implementation is still underway and, as such, two regional managers noted they had not yet used AQLs to evaluate PMOCs. Similarly, one PMOC indicated that FTA had not gone over AQLs with them.

FTA’s Office of Planning and Environment directs and evaluates FMOC work. FTA indicated that because there are fewer reports and the FMOC work is periodic and not continuous, the Office of Planning and Environment handles all contractor assignments and performance evaluations. According to the FTA’s Office of Planning and Environment, it has determined that the use of an AQL worksheet for consistency in evaluations is not necessary since one office reviews all of the FMOCs’ work. The Office of Planning and Environment provides the contractor evaluation form to the Office of Procurement, which enters the data into CPS.

Stakeholders Identified Benefits of and Challenges to FTA’s Oversight, and FTA Is Taking Actions to Address Some Stakeholder Concerns

Stakeholders Cited Benefits of FTA’s Use of PMOCs and FMOCs, Including Improved Management and Access to Additional Expertise and Technical Assistance

FTA officials, contractors, and project sponsors cited numerous examples of how FTA’s use of oversight contractors has improved project and financial management. For example, one project sponsor noted that feedback from the PMOC on the project management plan helped ensure that the plan documents were complete and covered project costs adequately. On a different project, officials from the project sponsor noted that the PMOC’s risk management review helped identify program risks that might have affected the project’s budget. These stakeholders also noted that the FMOC’s financial oversight has helped improve projects. For example, an FMOC’s independent assessment of a New Starts project confirmed that the project sponsor had the resources to construct and operate the project, despite a complex financing approach. In another case, an FMOC determined that the New Starts project did not have the financial capability to continue and recommended to FTA that the project not advance to the next project development stage. Furthermore, we have previously reported on how FTA’s use of PMOCs and FMOCs has benefited both project sponsors and FTA. Specifically, we observed that FTA’s oversight program has helped improve project sponsors’ controls over project costs, schedules, quality, and safety. We noted that the PMO program has provided FTA with a better understanding of the issues surrounding complex construction projects and an increased awareness of potential problems that could lead to schedule delays or cost increases.

PMOCs and FMOCs also provide FTA with additional expertise. Since the inception of its Project Management Oversight program, FTA has supplemented its own staff as well as used the expertise of PMOC staff, and later, FMOC staff, who have specialized areas of expertise beyond that of most FTA staff, to help oversee New Starts projects. Since the New Starts program has grown significantly while FTA’s staff size has stayed the same, FTA officials have used these oversight contractors to help compensate for this human resource gap. For example, FTA officials noted that while PMOCs are not FTA employees, the PMOCs help serve as FTA’s “eyes and ears” at project sites.

Finally, in addition to oversight services, PMOCs occasionally provide technical assistance on behalf of FTA to project sponsors. Such assistance may include providing information and instruction in project management and project analysis practices, or sharing technical expertise in transit project design and construction. In the course of providing oversight, PMOCs are to notify FTA about any opportunities for project sponsors to benefit from technical assistance. FTA officials stated that New Starts projects have benefited from the technical assistance that PMOCs and FMOCs provided to project sponsors. However, while PMOCs and FMOCs can provide technical assistance in the form of examples or best practices, project sponsors are responsible for deciding how to proceed. For example, FTA assigned a PMOC to provide guidance to a project sponsor to help clarify safety requirements. As part of this process, the PMOC provided the project sponsor with examples of best practices used to meet safety requirements. The project sponsor used these best practices to resolve its safety issues and noted that having access to the PMOC and its specialized skills provided clarity and helped resolve the safety issue quickly.

Finding the Appropriate Balance between Oversight and Advancing Projects Is Challenging; FTA Is Taking Actions to Help Address This Challenge

Finding the right balance between protecting federal investments through project management oversight and advancing projects through the project development process is challenging. As noted above, FTA’s project management oversight program is intended to strengthen the management and monitoring of major transit projects. However, determining the appropriate level of oversight that is needed for a project is a delicate task, since in certain cases, too much or too detailed oversight could slow a project’s progress. We have previously reported that one option FTA could consider to reduce the burden placed on project sponsors and move projects more quickly through the New Starts project development process is to tailor the level of oversight to the risks of a New Starts project (indicated, for example, by the project’s cost or complexity).

FTA has developed procedures to assist PMOCs in ensuring that a project’s scope, schedule, and cost are in balance and that the project follows applicable federal requirements, but FTA has indicated that it also tailors PMOC oversight to each project. Specifically, FTA’s Oversight Procedures for PMOCs serve as guidance on how PMOCs should conduct their work; however, according to FTA officials, the Oversight Procedures are written to allow flexibility in the assignment of task and work orders based on the specific needs of a project. According to FTA officials, while some reviews in the Oversight Procedures are conducted for all projects that undergo PMOC oversight, such as project management plan reviews, FTA officials may decide not to have a PMOC conduct certain reviews or may ask the PMOC to conduct a more detailed review than is described in the Oversight Procedures. For example, FTA officials in an FTA region told us that they did not require the PMOC working on a project in their region to conduct a technical capacity and capability review because the PMOC was familiar with the project sponsor’s technical capacity and capability, the project sponsor had recently completed three other New Starts projects on time and within budget, and there was no change in project sponsor staff. In another example, an FTA official in another region asked the PMOC overseeing one of the projects in that region to conduct a real estate review that was not completely specified at that time in the Oversight Procedures because of concerns about the project sponsor’s real estate management approach. FTA officials noted that it is impossible to anticipate all requirements and cover them all in the Oversight Procedures. Therefore, FTA officials stated, it is possible to deviate from the Oversight Procedures if the deviations are clearly documented in the work order.

Despite FTA’s efforts to tailor its oversight, four project sponsors we spoke with said that the level of oversight and PMOC involvement in a project sometimes seem excessive or detailed, making it difficult for them to spend time developing the project. For example, one project sponsor we interviewed said that FTA has layered additional oversight on projects in response to problems that FTA faced in other projects in other regions. In this case, project sponsor officials said that some of the additional oversight was not applicable to its project and that the project ended up spending time and resources addressing additional oversight requirements rather than developing the project. In addition, this project sponsor said that the level of PMOC involvement on their projects is also excessive because PMOC oversight is detailed. For example, rather than providing broad oversight over the project’s schedule, the project sponsor officials said that the PMOC requested very detailed information regarding the schedule. According to the project sponsor, as a result of this level of oversight, they had to hire more staff, which increases project costs.

FTA currently oversees project sponsors in accordance with the Project Management Oversight Rule (PMO rule). In September 2009, FTA issued an Advance Notice of Proposed Rulemaking (ANPRM) to obtain public comment on proposed modifications to the rule intended to increase the effectiveness of its oversight approach. As part of the ANPRM, FTA is considering changes to the PMO rule that could potentially base the level of oversight on risk. According to FTA officials, amending the rule to emphasize its risk-informed process would formalize and standardize FTA’s current practice of tailoring its PMOC oversight to each project. A few project sponsors who provided comments to FTA on the ANPRM said that the level of oversight could be tailored to the experience of the project sponsor, with sponsors with more experience in the New Starts process undergoing less detailed oversight. Two of these project sponsors also suggested that oversight could be tailored to the level of federal investment. FTA is currently evaluating comments it received from stakeholders on this ANPRM, and expects to issue the Notice of Proposed Rulemaking in late 2010.

FTA Is Taking Actions to Address Challenges Related to Projects’ Growing Complexity and Is Seeking Stakeholder Input on Other Potential Changes to the PMO Program

FTA faces challenges in ensuring that it has specialized expertise and staff to oversee the growing number of complex capital projects, including New Starts projects. FTA, in its 2009 ANPRM, noted that the agency is participating in a larger number of megaprojects. These megaprojects often involve technology, design, or construction elements, such as tunnels, that make a project more complex. For example, one megaproject that is currently seeking New Starts funding from FTA requires the construction of two new tunnels under a river. To ensure that technologies, design, or construction elements, such as tunnels, do not increase the costs and schedules of the complex projects, FTA requires oversight by individuals who have specialized knowledge and experience in these areas. Furthermore, FTA officials said that the number of projects in FTA’s Small Starts program is growing and some Small Starts are fairly large, complex projects, particularly those nearing the $250 million total capital cost threshold of the program. Thus, demand for FTA oversight resources is growing. The challenges associated with the growing number of large projects and the additional demands they place on FTA’s oversight are particularly significant given that FTA’s staffing levels have remained about the same since 1982.

The financing of New Starts projects has also become more challenging. To cover the large dollar amounts of some New Starts projects, particularly the megaprojects, project sponsors are borrowing more money to finance their projects and are increasingly relying on innovative financing methods, such as public-private partnerships and value capture strategies. In addition, because of the current economic climate, state finances are stretched, forcing state project sponsors to finance these projects through multiple sources or less traditional means. For example, another New Starts project we reviewed is using an innovative approach to finance an extension of an existing system. This project is funded through a variety of new revenue sources, including bonds backed by future revenue collections from a toll road. As states use such new funding methods for projects, the financial capacity oversight work that FMOCs conduct for FTA also becomes more involved, requiring more effort and time to understand the funding approaches and ensure that project sponsors can sufficiently fund their proposed projects, and also maintain and preserve their current transit systems.

As noted earlier, PMOCs and FMOCs provide FTA with access to specialized experience and knowledge and additional staff support, and FTA has taken actions to use these contractors to a greater extent to address the challenges related to the growing size and complexity of New Starts projects. For example, FTA is involving PMOCs and FMOCs earlier in the project development process. FTA, beginning in 2006, began to assign PMOCs to oversee New Starts projects when the sponsor applies to enter into preliminary engineering, rather than waiting until the project is already in that stage. FTA officials said that identifying and addressing problems at an earlier stage has helped them ensure that the projects are more likely to stay within budget and on schedule. According to project sponsors we spoke with, ensuring that a project is on schedule is important because delays can significantly increase project costs. Minimizing the potential for increased project costs and delays is particularly relevant for large projects that present greater risks of project cost increases or schedule delays because they use unique technology, design, or construction elements. Additionally, in 2008, FTA began to ask its FMOCs to conduct financial capacity assessments before projects entered into final design (with an update performed at the FFGA stage), rather than waiting until the projects were at the FFGA stage. FTA officials told us that this change helps them identify financing problems earlier, allowing project sponsors more time for corrective actions. Specifically, FTA officials stated that a project sponsor may need additional time to identify additional sources of funding if FTA’s review found that there was insufficient funding or financial capacity to undertake the project.

In addition, FTA noted in its 2009 ANPRM that it is considering a potential revision to the PMO rule that would better reflect the current environment of a larger federal transit program and an increased number of larger, more complex projects. Specifically, through this ANPRM, FTA is seeking comments from stakeholders on how FTA should best use its PMOCs to provide specialized expertise when needed; how the agency should use PMOCs to supplement its limited staff in overseeing increasingly complex, major capital projects; whether the use and role of PMOCs should be expanded to overseeing projects other than major capital projects; and at what stage FTA should assign PMOCs to New Starts projects, among other things. FTA is also considering whether it should expand the PMO rule to include a focus on project management, in addition to its current focus on project oversight. FTA officials explained that the expanded focus would establish performance expectations for project sponsors on certain project management areas. For example, the PMO rule could be expanded to include a requirement that project sponsors have sufficient staff in place to demonstrate that they have the capacity and capability to develop their project. FTA officials explained that by ensuring that project sponsors have the appropriate project management skills, the sponsors can better control their project costs and schedules.

FTA and PMOCs Have Multiple Ways to Communicate with Project Sponsors during the Oversight Process, but Project Sponsors Say Communication Challenges Still Exist

Effective oversight management of FTA’s New Starts program and project sponsors’ advancement of transit projects through the New Starts program depend, in part, on effective communication. We have noted that an effective communication strategy should facilitate an honest two-way exchange with, and allow for feedback from, stakeholders. Such communication is central to forming the effective internal and external partnerships that are vital to a program’s success. GAO has also noted that relevant and timely information is needed for an agency to achieve its objectives.

FTA has multiple avenues through which it communicates with project sponsors about its oversight activities. Among its efforts, FTA informs project sponsors of the steps they need to take in the New Starts funding process; addresses New Starts issues through training workshops, guidance letters, circulars, and quarterly meetings with the project sponsor and the PMOC in attendance; and provides feedback on PMOCs’ reports on sponsors’ projects. Furthermore, in 2008, FTA developed checklists to provide detailed information to project sponsors on the submittals required for each project development phase. Sponsors have indicated they found this tool and the meetings to be very helpful in understanding FTA’s and PMOC’s oversight requirements.

However, three project sponsors we spoke with stated that communications with FTA and PMOCs were not consistently timely, which delayed their response to FTA’s concerns. For example, one project sponsor we interviewed noted that during final design the sponsor was informed of a level boarding issue pertaining to the Americans with Disabilities Act, which negatively affected the project’s schedule and cost. The project sponsor said that these effects might have been avoided if FTA had raised the issue with the project sponsor earlier in the preliminary engineering stage. In addition, another project sponsor we interviewed said that waiting for FTA to approve its readiness reviews led to additional costs, since the project consultants were idle during this time. Some of the consultants cost the project up to $50,000 to $200,000 per month and the project sponsor risked losing these consultants to other projects. A third sponsor indicated that it gets final reports months later, by which time the reports can be outdated. This practice can raise issues that may have already been addressed, can affect relations with other funding sources, and can result in late project delivery.

These apparent delays may occur for a number of reasons. FTA officials noted that project sponsors sometimes provide information that is incomplete or inaccurate, resulting in additional review time and delays. An FTA headquarters official explained that they may delay providing sponsors with final oversight reports until they make a decision, positive or negative, to move to the next phase of project development. Similarly, an FTA regional official we spoke with stated that they have delayed sharing reports with project sponsors when the reports included sensitive information, such as the impact of risk assessment on cost estimates used as the basis for FTA funding decisions, which affect ongoing funding negotiations between the project sponsor and FTA. In addition, an FTA regional official and a PMOC noted that because FTA uses a team approach to oversee the New Starts program, there are many people involved in reviewing these final reports, which might increase the time needed for reviews. For example, after the FTA regional office initially reviews a report focused on a capital cost estimate review, the report is reviewed by FTA headquarters officials, including those from the program management office who supervise the regional officials and PMOC technical work. Furthermore, FTA’s Office of Planning and Environment as well as other officials, including representatives from the Office of Chief Counsel and Office of Civil Rights, also review PMOC reports as needed. FTA has acknowledged that its current New Starts process can be lengthy and frustrating. In an October 2009 speech, the FTA administrator stated that FTA currently has a decision-making process that is frustrating, lengthy, and incomprehensible, and that FTA is working on a streamlined decision-making process; he cautioned however, that the decision may not always be to approve a project. Finally, as noted previously, FTA officials said that they are involving PMOCs earlier in the project development process to avoid problems.

In addition to communication delays, three project sponsors we spoke with said that they provide a considerable amount of documentation to the PMOCs and FTA and do not get clear written reactions to this information.

Five out of the six FTA regions we interviewed indicated that they do not consistently provide project sponsors with draft PMOC reports. FTA’s oversight procedures for PMOCs direct the PMOCs to discuss draft report findings with project sponsors before finalizing reports. FTA officials stated that PMOCs are expected to confirm the facts of the draft report with project sponsors. An FTA official indicated that the PMOCs are expected to share the facts with the project sponsor, and that project sponsors are “generally aware” of the issues identified in the oversight reports because of its communications with PMOC and FTA officials. Project sponsors believe getting draft reports could help them (1) ensure that PMOCs and FTA have the proper, most up-to-date information, and (2) understand the context of concerns or requests for more information.

Agency Comments and Our Evaluation

We provided a draft of this report to DOT for its review and comment. DOT officials generally agreed with our findings and provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to interested congressional committees and the Secretary of Transportation. In addition, this report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2834 or wised@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

Under the Safe, Accountable, Flexible, Efficient Transportation Equity Act - A Legacy for Users (SAFETEA-LU), we are required to report annually on the Federal Transit Administration’s (FTA) New Starts process. Accordingly, the objectives of this report were to review (1) how the FTA uses contractors to oversee New Starts projects and how the agency procures, monitors, and evaluates the contractors’ services, and (2) the benefits of FTA’s oversight approach and the challenges FTA faces in conducting its oversight. Our review focused on oversight activities that project management oversight contractors (PMOC) and financial management oversight contractors (FMOC) conduct between the preliminary engineering stage of the New Starts project development process and the full-funding grant agreement (FFGA) stage that inform FTA’s recommendations for New Starts funding. Furthermore, we focused on New Starts projects, and not on Small Starts or Very Small Starts projects, since FTA does not generally assign PMOCs to Small or Very Small Starts projects.

To address our two objectives, we reviewed relevant legislation and FTA documents. Specifically, we reviewed SAFETEA-LU and other relevant legislation, such as the Surface Transportation and Uniform Relocation Assistance Act (STURAA), which authorized the project management oversight program and provided funding for these oversight activities. In addition, we reviewed and analyzed FTA documents, including procedures, guidance, and performance evaluation criteria for PMOCs and FMOCs. We also reviewed FTA’s 1989 project management oversight rule; FTA’s Advanced Notice of Proposed Rulemaking (ANPRM), which seeks public comment on a proposed amendment to this rule, and FTA information on the agency’s budget for its project and financial management oversight programs.

In addition to these efforts, to determine how FTA procures oversight contractors, we reviewed FTA’s procurement solicitations for PMOCs and FMOCs, reviewed GAO bid protest decisions, and interviewed FTA procurement officials in headquarters. Furthermore, we reviewed procedures for acquisition under the Brooks Act and the Federal Acquisition Regulation (FAR), including procedures for competitive negotiations.

We also conducted interviews on topics related to our two objectives. In particular, we interviewed FTA officials in headquarters who are responsible for managing and planning New Starts projects, as well as officials familiar with the budget for FTA’s project and financial management oversight programs. Furthermore, we interviewed representatives from associations familiar with the New Starts program, such as the American Public Transportation Association and the New Starts Working Group.

Finally, we focused our review on 10 New Starts projects that FTA recommended for FFGAs between fiscal year 2006 to fiscal year 2011. For each of these 10 projects, we conducted semistructured interviews with the project sponsors responsible for implementing the projects and the PMOCs, FMOCs, and FTA regional officials responsible for overseeing the projects. We judgmentally selected the 10 projects to include a range in (1) transit mode (i.e., heavy rail, light rail, commuter rail, or bus); (2) total project cost (from $117 million to $8.7 billion); (3) amount of federal funding (from 26.7 percent to 80 percent of total project costs); and (4) geographic area. Our sample also includes project sponsors who have been recommended for multiple FFGAs by FTA. We interviewed all of the project sponsors by telephone except the sponsor for the Dulles Corridor Metrorail project, which is located near enough to Washington, D.C., for us to interview the sponsor in person. Because the projects were selected as a nonprobability sample, the results cannot be generalized to all projects or the New Starts oversight process as a whole. Table 2 lists the 10 New Starts projects we focused on for our review.

We conducted this performance audit from January to September 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: List and Description of Project Management Oversight Contractor Reviews

Table 3 describes the oversight reviews conducted by PMOCs and described in the Federal Transit Administration’s (FTA) Project Management Oversight Procedures, which were most recently revised by FTA in May 2010. According to FTA officials, the procedures described in this guidance were new to the project management oversight contracts awarded in 2009 contracts, but are based on the program guidance instructions for project management oversight contractors (PMOC) used in prior contracts.

Appendix III: Funding Obligated for Project Management Oversight and Financial Management Oversight, Fiscal Years 2000-2009

Funding for PMOC and FMOC activities is provided through statutory set- asides made available annually for specific FTA grant programs. Federal law currently authorizes FTA to use up to 1 percent of amounts made available for its major capital investments, including New Starts, for project management oversight, to fund the oversight activities of both PMOCs and FMOCs. According to FTA, this funding is pooled with funding that is authorized for FTA’s other oversight activities and then allocated across FTA’s grant programs.

FTA obligates funding upon award of a contract to a PMOC or FMOC. Table 4 describes obligations made for project management oversight and financial management oversight activities for the New Starts program, as well as other FTA grant programs. According to FTA officials, the funding amounts obligated for oversight activities varies annually. FTA officials stated that this funding is allocated for program management oversight and financial management oversight activities based on where each capital project, including New Starts projects, is in the project development process and the oversight activities anticipated for each project in the coming budget year.

Appendix IV: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact named above, Catherine Colwell, Assistant Director; Eli Albagli; Lauren Calhoun; Roshni Davé; Elizabeth Eisenstadt; Brandon Haller; Kristine Hassinger; Matthew Voit; and William Woods made significant contributions to this report.