Rail Grant Oversight: Greater Adherence to Leading Practices Needed to Improve Grants Management
Released: 2016-05-26T00:00:00

Letter

Background

FRA is the primary federal agency responsible for issuing and enforcing railroad safety regulations and for distributing federal funds for intercity passenger rail service. PRIIA mandated new responsibilities for FRA to plan, award, and oversee the use of federal funds for intercity passenger rail. The American Recovery and Reinvestment Act of 2009 (Recovery Act) appropriated funding for high-speed rail projects, which resulted in a dramatic increase in federal funding for intercity passenger rail projects. Prior to 2009, FRA had a very limited grant portfolio, receiving appropriations for approximately $30 million in grant funding in fiscal year 2008, for example, primarily for intercity-passenger rail grants to states. With expanded responsibilities, the agency had to quickly award approximately $8 billion in Recovery Act funds while simultaneously developing policies and procedures for grants management. In 2015, FRA managed a portfolio of approximately 200 grants and managed $17.7 billion in obligated grants. In December 2015, Congress passed the Passenger Rail Reform and Investment Act of 2015 as a title in the Fixing America’s Surface Transportation Act (FAST Act), which authorized a new consolidated rail infrastructure and safety-improvements grant program to assist grantees in financing the cost of improving passenger and freight rail transportation systems.

In 2010, when FRA was in the early stages of developing its grant oversight program, GAO identified principles that would become important as FRA transitioned from awarding grants to overseeing their performance. For example, a well-designed and implemented grant oversight program is critical to ensuring effective use of federal grant funds. FRA began drafting a Grants Manual to communicate the agency’s overall approach to grants management in April 2010 when it was in the process of building up its grants management program. This Grants Manual includes procedures developed to address the agency’s expanded responsibilities following the enactment of the Recovery Act. FRA’s 2013 Program Management Plan outlined a “matrixed” organization structure intended to facilitate the flow of skills and information across functions within FRA (see fig. 1 below). FRA’s grant and loan portfolio is further organized into geographically based regional portfolios. Each regional portfolio is managed by a Regional Team comprised of a team lead—the Regional Manager—and subject matter experts from other FRA offices. Teams are responsible for funding, project delivery, and monitoring and oversight activities. For example, for the section 305 procurement projects, the “project” entails the design, manufacturing, and delivery of the bi-level and locomotive equipment, and FRA is responsible for evaluating and monitoring—from start to finish—all the steps required to ensure successful project delivery. The Regional Manager is responsible for coordinating the entire regional team, and may share some advisory and oversight responsibilities with the designated grant managers—those directly responsible for overseeing and maintaining specific portions of the grants management process. For example, for the two section 305 equipment procurements—the bi-level cars and the locomotives—grant oversight includes collaboration between Regional Managers and Grant Managers within the Office of Railroad Policy and Development, subject matter experts within the Office of Safety and the Office of Chief Counsel, and contractors.

Launched in January 2010, the NGEC has developed, adopted, and promulgated six specifications for next-generation corridor equipment. For example, one of the specifications was to develop and build vehicles for the future or next generation, such as a locomotive capable of speeds up to 125 mph. The NGEC developed standardized specifications intended to make it possible for a group of states to buy equipment faster, at a lower cost, with reduced operating and maintenance costs going forward. Using the NGEC-developed specifications as a foundation for two equipment procurement projects, Caltrans took the lead on the bi- level car project, and IDOT leads the locomotive project. For example, IDOT—representing the Midwest coalition of Missouri, Michigan, and Iowa—serves as the lead state on behalf of itself, Caltrans, and WSDOT in conducting the joint locomotive procurement. Though Caltrans and IDOT are the lead states for the two projects, they have established memorandums of understanding for the equipment with their partner states, including Washington, Missouri, Michigan, Iowa, and Wisconsin.

The NGEC is comprised of FRA, Amtrak, state, and industry participants. The NGEC structure includes an Executive Board responsible for writing the technical requirements document and approving the final specifications. The requirements document outlines the design objectives and specific performance requirements that need to be met for each type of equipment—in the case of the section 305 equipment-procurement projects, diesel electric locomotives and bi-level passenger cars (see fig. 2 below). The specification is a detailed technical document developed by the NGEC Technical Subcommittee intended to address the range of operational considerations needed to procure, design, and manufacture a fleet of bi-level cars or locomotives for use in intercity corridor service. The Technical Subcommittee includes working groups in several functional areas—e.g., structural, electrical, and mechanical—responsible for drafting relevant elements of each technical specification. The NGEC Review Panel evaluates specifications for compliance with the requirements document as well as compliance with regulations regarding safety, accessibility, and operations. The NGEC also has a detailed system for approved specifications and documents to be revised, edited, and updated through a formal process.

Six grant agreements between FRA and Caltrans, IDOT, and WSDOT, respectively, are funding the two locomotive and bi-level equipment procurement projects (see table 1 below). FRA entered into two grant agreements with Caltrans, three grant agreements with IDOT, and one grant agreement with WSDOT. WSDOT’s grant agreement was developed prior to the decision to designate one lead state for each of the two equipment projects. After all 6 grant agreements were executed, the efforts were split into two projects, as Caltrans contracted with Sumitomo Corporation of America and Nippon Sharyo in November 2012 to purchase 130 bi-level cars and IDOT contracted with Siemens in March 2014 to manufacture 47 locomotives. These grants are funded by a mix of Recovery Act and DOT fiscal year 2010 appropriations. For the six grants, approximately 75 percent of the funding is Recovery Act money and four of the six grants have an expenditure deadline of September 30, 2017.

FRA’s Grants Management Approach Has Evolved to Administer the Section 305 Equipment Procurement Projects

FRA’s management of the grants funding the section 305 equipment procurement projects has evolved from a general grants management approach to include additional project-level oversight. This framework is outlined in the Grants Manual—including routine and scheduled monitoring reviews—and in the specific activities outlined in the terms of the grant agreements funding the section 305 equipment procurements. FRA’s approach to manage the section 305 equipment projects evolved as the projects progressed, to include additional involvement in project- level activities. Specifically, in 2015, FRA transitioned to a project-based oversight structure to carry out its grants management framework for the section 305 equipment procurement projects and, in 2014, used independent contractors to support project oversight. For example, FRA used additional contractor resources to support the agency’s grants management oversight after delays were identified in the bi-level project schedule.

FRA Implemented General Grants Management Procedures from the Start of the Equipment Procurement Projects

Beginning in 2010, FRA implemented its draft grants management framework to manage the post-award phase of the grants funding the section 305 equipment procurement projects. The general framework used to manage FRA’s entire grant portfolio—including the six grants funding the procurements—is described in the agency’s Grants Manual, articulating its oversight and monitoring procedures to track grantees’ performance. After grants are awarded, the general oversight and monitoring procedures described in the Grants Manual are carried out in accordance with the specific terms of the individual grant agreements. According to the Grants Manual and interviews with FRA and state grantee officials, FRA’s monitoring procedures include what is referred to as “routine” and “scheduled” monitoring:

Routine monitoring: the periodic review of progress for all active FRA grants to ensure grantees are in compliance with the terms of their respective grant agreements. For example, FRA’s grant managers review the quarterly progress reports (QPR) grantees typically are required to submit under their grant agreements, describing the status of grant projects, including any issues encountered affecting the scope, schedule, and budget described in the grant agreement. For the section 305 equipment procurement projects, FRA reviews QPRs from Caltrans, IDOT, and WSDOT for their respective grant agreements. While FRA provides grantees with a QPR template that includes fields for narrative descriptions of significant accomplishments and any technical, cost, or schedule problems experienced during the review period, our review of the QPRs FRA provided shows that the information included in these reports varied by grantee. For example, IDOT first reported delays with the bi-level delivery schedule in the first quarter of fiscal year 2013, while the bi-level project lead, Caltrans, did not report schedule problems until the first quarter of fiscal year 2014. In addition, the level of detail regarding identified issues within the QPRs varied. For example, in the first quarter of fiscal year 2014, IDOT and Caltrans each discussed the bi-level delivery schedule; however, IDOT provided additional information raising concerns that the projected delivery schedule would not meet the September 30, 2017, deadline for the expenditure of funds. FRA officials told us QPRs do not always identify project issues encountered in a given quarter because the agency works informally with grantees to address problems as they arise. According to FRA officials, the agency has developed a new QPR form for grantees that will enable more detailed data on status and progress to be collected. FRA officials told us the agency plans to begin using this form in 2016, but the new QPR forms were not in use at the time of this review.

Scheduled monitoring: a detailed annual review of select grant projects’ progress, which FRA officials carry out using desk and/or on- site review sessions. FRA selects grant projects from its full grant portfolio to receive scheduled monitoring using factors such as project type, funding level, the time elapsed since previous monitoring activities, and knowledge of existing project issues. For the section 305 equipment procurement projects, a series of checklists help guide FRA’s scheduled monitoring reviews. According to the Grants Manual, the results of scheduled monitoring activities are recorded in a monitoring review report and should identify any significant findings (i.e., issues that jeopardize project completion or compliance) and areas of interest (i.e., issues that have the potential to become significant). FRA’s monitoring and oversight procedures require grantees to develop and submit corrective action plans when significant findings—such as issues that put the entire project at risk— are identified, describing the practices that grantees and other stakeholders, as appropriate, will follow to address the identified issues. To date, FRA has conducted 13 scheduled monitoring reviews for the six grants providing funding for the section 305 procurement projects, identifying significant findings associated with the bi-level car project in 2014 and 2015. For example, one monitoring report identified the bi-level design and delivery schedule as a significant finding because the manufacturing timeline did not meet the expenditure deadline. The bi-level schedule delay was reported by FRA in 2013, but at that time it was identified as an area of interest rather than as a significant finding. While no significant findings have been identified with the locomotive project to date, FRA has identified areas of interest associated with the locomotive project. For example, in a 2015 monitoring report, FRA reported one area of interest for the locomotive project—that the states receiving the locomotive equipment do not yet have an ownership, operation, or maintenance plan in place to manage the equipment upon delivery.

As FRA began implementing its general grants management framework, it also began to award the six grants providing funding for the section 305 equipment procurement projects. See figure 3 below for a timeline of FRA’s grants management and section 305 equipment project milestones.

FRA’s grant responsibilities are also defined by the six individual grant agreements funding the equipment purchases. As noted above, the grant agreements require that the equipment purchased with federal funds comply with specifications developed by the NGEC. The grant agreements also generally incorporate the requirement that the equipment be procured in a manner consistent with Buy America requirements. In our analysis of the six agreements, we identified areas where FRA has specific responsibilities for the equipment grants as the projects progressed. Depending on the terms of the particular grants funding the section 305 procurement projects, FRA’s involvement in the locomotive and bi-level project may include reviewing the grantees’ draft requests for proposal (RFP) seeking interest from potential equipment manufacturers, participation in the design review meetings with the grantees and equipment manufacturers, and participation during section 305 equipment testing.

Reviewing and approving RFP packages: Three of the grant agreements explicitly require FRA’s review and approval of the grantees’ draft RFPs; the other three grants are either silent or require approval by NGEC. The FRA officials we spoke with explained that the agency’s participation in the proposal review process included reviewing the RFP package to ensure the grantees’ bid solicitations included applicable federal provisions (e.g., Buy America and Davis- Bacon requirements). For example, FRA officials reviewed the RFP for the contract that Caltrans eventually awarded to Nippon-Sharyo. In addition, FRA officials told us the agency had worked to reconcile differences between the various state laws of equipment grantees. According to FRA, the difficulties in reconciling each state’s requirements were a source of delay in awarding the initial procurements. The grant agreements do not provide for FRA’s participation in the grantee’s bid evaluation or selection process, nor is this process required by applicable federal law. FRA officials told us agency officials were not involved in these processes and did not provide concurrence with IDOT or Caltrans’ bid selections.

Reviewing and approving design specifications: Four of the grant agreements explicitly require FRA to approve the design of the bi- level rail car and/or locomotive, while the other grant agreements provide for FRA to review the design specifications as a stakeholder. FRA officials told us agency officials participated in multiple reviews of the manufacturers’ design documents for both the bi-level cars and locomotive equipment —from the initial design concept to a final design plan—to ensure the designs comply with federal safety standards. For example, according to the IDOT officials we met with, FRA officials attend all locomotive design-review meetings and the state submits design documentation to FRA’s safety team for approval.

Participation in section 305 equipment testing: Some of the grant agreements provide a role for FRA in the testing of the equipment. For example, two of the grant agreements specify that FRA will be given the opportunity to witness and approve production tests. Another grant agreement authorizes FRA to verify test results. The FRA officials we spoke with said agency officials participated in equipment testing to ensure the bi-level cars and locomotive equipment meet federal safety standards and that the testing procedures are appropriate. According to Caltrans and IDOT officials, FRA is invited to all inspections and testing for the bi-level equipment.

Because the agreements between FRA and the states are cooperative agreements, FRA provides substantial programmatic involvement in the section 305 equipment procurements. As provided in the grant agreements, substantial programmatic involvement means that FRA will assist and coordinate with grantees and participate in grant project activities after the contract is awarded. For example, FRA provides grantees with administrative, programmatic, and technical assistance as needed. The FRA officials we met with said the agency has provided training on the Buy America requirements and the agency’s financial reporting requirement in response to the needs of the section 305 equipment grantees. At the start of the bi-level and locomotive procurements, the extent of FRA’s involvement varied based on the state’s previous project management experience. For example, a former FRA official involved at the start of the bi-level car project told us the Caltrans staff managing the project at the beginning did not seek FRA’s involvement beyond the areas described in the grant agreements funding the equipment procurements. These staff had previous procurement experience but lacked experience in industry and manufacturing issues, affecting the state’s ability to effectively manage the project. In addition, Caltrans had prior experience purchasing rail cars and the 305 specification that the NGEC developed for the bi-level cars was based on the “California Car” that Caltrans had purchased between 1995 and 1997. In contrast, the IDOT officials we met with said that since IDOT did not have previous procurement experience, the state proactively sought FRA’s involvement from the start of the locomotive project and hired an industry expert with locomotive experience to help manage the project. For example, FRA reviewed IDOT’s ordering agreement with Siemens (the locomotive manufacturer) and the IDOT officials we spoke with said FRA has participated in monthly meetings with the Midwest Partner states that will be receiving the locomotive equipment from the start of the project.

As the Bi-level Project Encountered Challenges, FRA’s Approach for the Section 305 Equipment Procurement Projects Evolved to Include Additional Project-Level Oversight

As the section 305 procurement projects have progressed, the bi-level car project has encountered challenges that jeopardize project completion by the expenditure deadline. While FRA reported issues with the bi-level project’s schedule in a 2013 monitoring review report, the issues facing the bi-level project were not identified as a significant finding—thus, did not require a corrective action plan—until 2014. In 2014, the bi-level project schedule was also reported as a significant issue in an independent review of the project by an FRA consultant. For example, the independent review raised concerns that the level of detail in the bi-level schedule did not meet the project management requirement and that design activities were up to 7 months behind schedule. In FRA’s 2014- scheduled monitoring review report, the bi-level car project’s compressed design and delivery schedule was identified as a significant finding because the schedule posed a risk to the project leading up to the expenditure deadline. Grantees are responsible for developing corrective action plans to resolve significant issues, and FRA officials told us that the agency supported the grantee’s efforts to develop corrective actions in response to the bi-level project issues identified. For example, in April 2015, FRA convened a meeting with Caltrans, IDOT, and Nippon Sharyo to work through the bi-level project issues. The meeting resulted in a series of corrective actions including replacing Caltrans and contractor program managers and establishing a designated bi-level program manager to oversee activities related to the scope, schedule, and budget of the project. Following that meeting, in May 2015, documentation of the corrective action plan to address bi-level issues revealed that the bi-level project was 14 months behind schedule.

In August 2015, the bi-level car schedule encountered an additional setback after the bi-level car shell suffered a structural failure during testing and production stopped for car shell redesign. In a December 2015 monitoring review report, FRA stated that the manufacturers’ project schedule showed a delay of 24 months and that the bi-level project would not be completed by the expenditure deadline. Furthermore, FRA’s monitoring report revealed the agency’s concern with Nippon Sharyo’s quality standards and project management. For example, FRA reported that Nippon Sharyo was increasing risk to the bi-level project by manufacturing bi-level car parts without an accepted car shell design. As of April 2016, the bi-level production schedule is on hold and the final project equipment delivery date is unknown. To date, FRA has not identified significant findings with the locomotive project through its monitoring reviews, and as of April 2016 the project was on schedule, with planned equipment delivery in June 2017.

In addition to FRA’s efforts to address significant issues related to the bi- level car project, FRA’s grants management approach for both of the section 305 equipment procurements has evolved to include additional project-level oversight. In practice, FRA is involved in the procurements at two levels—(1) as grants manager overseeing the state grantees, and (2) increasingly, as support for the underlying activity between the states and their selected contractors in which FRA has elected to participate. As discussed below, this approach includes a revised management structure, informal interaction with section 305 equipment procurement stakeholders, and the use of outside contractors.

Revised management structure: FRA initially carried out its grants management framework under the regional oversight structure used to manage its entire grant portfolio. However, FRA officials told us this regional approach—under which FRA’s management for the six grant agreements funding the section 305 equipment procurement projects was carried out by three different FRA regional teams—was not working for the section 305 equipment procurements. According to the FRA officials we spoke with, it became clear that there should be a single manager responsible for overseeing both projects due in part to the number of grants managers and regional managers involved and due to project management challenges. In 2015, FRA transitioned to a project-based oversight structure for the section 305 equipment projects—referred to as the National Vehicle Program—making the Midwest Regional Manager the primary FRA official responsible for managing and coordinating grantees’ efforts for both the bi-level and locomotive equipment projects, instead of managing the six individual grants. In addition, FRA hired an individual with prior passenger-rail industry experience into the role of the National Vehicle Program Manager, to participate in routine meetings with grantees and report relevant information to the Midwest Regional Manager. According to one of the grantees we met with, FRA did not initially have sufficient experience to manage the needs of the delivery program, and each of the grantees said turnover in FRA’s grants management staff was a challenge. For example, Caltrans officials worked with three different FRA grants managers and four different FRA regional managers in 5 years, and IDOT officials worked with two regional managers in the same time. According to one grantee, many of the FRA staff involved in the project early on had policy experience but lacked experience in project delivery. According to FRA officials, the agency has continued to experience turnover in key roles associated with the equipment procurements.

Informal Interactions with project stakeholders: While FRA continues to apply the monitoring and oversight procedures in its Grants Manual as described above, FRA officials told us frequent and informal interactions—for example, ongoing participation in meetings between project stakeholders such as grantees and equipment manufacturers—help the agency monitor grantees’ performance. Caltrans and IDOT officials told us that FRA officials and consultants currently participate in project-level meetings several times a week. For example, Caltrans officials told us that FRA officials attend meetings with project stakeholders four times a week, including a quality assurance meeting with Nippon Sharyo and a weekly coordination meeting with IDOT. According to FRA officials we spoke with, these informal interactions are the primary way information about the status of the locomotive and bi-level projects is communicated. In addition, according to FRA, the agency and its contractors have held a series of meetings with Nippon Sharyo to address bi-level rail car project challenges—including discussions related to specific project deficiencies and the manufacturer’s responses to the areas of concern identified by FRA.

Additional contractor involvement: In May 2014, FRA awarded a new contract for Monitoring and Technical Assistance Contractors (MTAC) to help manage the equipment procurements, and more recently increased contractor support to meet the growing needs of the bi-level equipment project. The overall role and responsibilities of MTAC contractors are detailed in FRA’s Monitoring Procedures, published in August 2014, and include supporting FRA’s project-level oversight with technical expertise and delivering training and technical assistance to grantees. The MTAC contract enables contractors with subject-matter expertise to be deployed in response to a range of FRA programs or projects as needed. The FRA officials we met with told us MTAC is not included in the grants management framework described in the Grants Manual because the use of MTAC contractors is dependent on the needs of the grant project, allowing the agency to obtain MTAC support and expertise as needed. For example, FRA has significantly increased MTAC support based on the needs of the section 305 equipment procurement projects and, in particular, the needs of the bi-level project. FRA officials told us they executed a new MTAC contract because the contractor support necessary to oversee the bi-level project was more than the agency anticipated, exceeding the resources provided under the agency’s original contract. MTAC contractors participate in meetings between bi-level and locomotive grantees and vendors and provide project updates to FRA through summary reports of these interactions. In addition, the National Vehicle Program Manager and MTAC contractors meet weekly to discuss the procurements and report relevant project information, including any issues, to FRA’s Midwest Regional Manager who is responsible for overseeing the National Vehicle Program. FRA officials told us that MTAC’s direct involvement with the grantees receiving funding for the section 305 equipment procurement projects provides FRA with a “boots on the ground perspective” of the project’s status. In addition, MTAC provides Caltrans with technical support and oversees Caltrans project management by participating in weekly meetings with the states and manufacturer.

FRA’s Grants Management Approach Partially Follows Leading Practices

We found that FRA partially follows the grants management leading practices we identified in the areas of performance monitoring, written documentation, training, and communication. Practices in these four areas are closely related, and improvements or shortfalls in one practice may contribute to improvements or shortfalls in another practice. Establishing, documenting, and following practices—and their supporting characteristics—in these four areas can contribute to a more effective grants management framework. Since 2010, FRA has developed policies and procedures in all four areas; however, it has not fully implemented those policies and procedures. Table 2 shows our overall assessment of FRA’s grants management activities compared to leading practices with examples of selected characteristics. Appendix II provides greater detail on our comparison of FRA’s approach with each supporting characteristic for each leading practice.

FRA Does Not Have Performance Measures Directly Linked to Project Goals and Does Not Fully Evaluate Results of Monitoring Activities

An effective grants management framework includes establishing a process that ensures project goals are identified, tracked, and fulfilled and deliverables received. FRA partially follows the performance monitoring leading practice area because while FRA developed a strategic vision for the HSIPR program and outlined a monitoring process in its Grants Manual, it does not have project goals or performance measures linked to the grants funding the section 305 equipment procurement projects and it does not fully evaluate the results of monitoring activities.

While FRA has developed a strategic vision for the HSIPR program— including promoting energy efficiency, environmental quality, and economic competitiveness—the agency has not formalized project goals for the two section 305 equipment procurement projects or performance measures that clearly link to project goals. FRA officials we met with stated that there is no stand-alone document outlining goals or performance measures for the section 305 equipment procurement projects, and that the officials measure project progress by tracking the scope, schedule, and budget outlined in each grant agreement. However, the agency has not formalized this approach to performance monitoring, for example in its Grants Manual, or documented specific goals or associated performance measures for the section 305 equipment procurement projects in a way that would demonstrate the agency has a process for comparing actual results against planned performance. Without explicit project goals and associated performance measures, it may be challenging for decision-makers to track and assess a project’s progress, make decisions about future efforts, and keep grantees accountable for outcomes.

While FRA outlines its monitoring procedures in the Grants Manual, we found it does not fully evaluate and document results of monitoring activities, including plans for corrective action. As discussed above, FRA outlines a detailed process for routine and scheduled monitoring in its Grants Manual. According to the Federal Standards for Internal Control, management should evaluate the results of monitoring activities and remediate identified internal control deficiencies on a timely basis. In other words, management should determine the appropriate corrective actions in a timely manner based on the identified deficiency. FRA noted in a 2013 monitoring report for the bi-level car project that the schedule showed that not all rail cars would be delivered by the grant agreement’s expenditure deadline for FRA funding. While this was an issue that directly affected the project’s scope and schedule, FRA did not identify it as a significant finding that would require a corrective action plan. FRA officials ultimately published a corrective action letter to lay out expectations and discuss how to get the project back on schedule in 2015—2 years after the issue was identified in a monitoring report. The DOT Inspector General reported in 2015 that corrective action plan deadlines for several other HSIPR grants had passed without any documentation that grantees took the necessary actions or staff extended deadlines to complete the plans. According to FRA officials, they use internal reports—separate from the routine and scheduled monitoring—to inform agency management about any issues related to a project’s scope, schedule, and budget. However, the procedures for and use of these reports are not formally documented in the Grants Manual as a mechanism for evaluating progress. Furthermore, it is unclear how the internal reports relate to the monitoring and oversight approach FRA has outlined in its Grants Manual or how they are used to hold grantees accountable for potential risks to their grants. Leading practices emphasize processes used to evaluate and monitor efforts should be formalized and documented. Without evaluating and documenting the results of monitoring activities in a timely manner grant managers may not be able to address risks to a project’s completion, a situation that could have a potential negative impact on the grant. This situation is particularly important for the grants funding the section 305 equipment procurement projects because most of the funding has an expenditure deadline of September 30, 2017, and equipment purchases such as these represent a new type of project for FRA.

FRA Developed Internal Documentation for Grants Management Policies and Procedures, but Has Not Developed Written Guidance For Grantees

An effective grants management framework includes developing and maintaining written documentation as a means to obtain and retain organizational knowledge and to ensure accountability for achieving agreed-upon results. FRA partially follows the written documentation leading practice area because while the agency has developed and maintained written policies and procedures to communicate grants management knowledge among its staff, it has not provided documentation outlining grantee expectations or developed guidance specific to the section 305 equipment procurement projects.

FRA has developed and maintained internal grants-management policies and procedures to communicate knowledge among agency staff. For example, FRA’s Grants Manual, the main policy document for agency staff, describes monitoring and oversight activities that staff are expected to conduct. The manual is periodically updated to reflect new federal regulations, such as regulations issued under the Office of Management and Budget’s new uniform guidance for federal awards, and changes in process, such as a more detailed issue escalation process. FRA also developed detailed Monitoring Procedures outlining FRA’s expectations of MTAC contractors and provided guidance to MTAC on its oversight responsibilities. For example, the monitoring procedure for oversight reports outlines the purpose of recurring oversight, documents that MTAC contractors must review, and the format and contents of the reports MTAC contractors must submit. Finally, in late 2015, FRA began drafting a Portfolio Management Guide with centralized guidance for regional managers—intended to complement the Grants Manual—recognizing that the Grants Manual was primarily written for grants managers.

Although FRA has developed and maintained internal grants management policies and procedures, it has not outlined expectations or developed guidance specific to the grants that fund the section 305 equipment procurement projects for grantees. For example, one grantee stated that while the grant agreements include high-level expectations, such as complying with federal regulations and meeting the NGEC technical specifications, expectations specific to the section 305 equipment procurement projects were unclear. According to this grantee, because FRA did not assign a central point of contact for the multi-state projects, each state grantee had to coordinate separately through their own grant manager, and expectations changed with each new state grant manager. All three grantees we spoke with stated that written guidance or procedures to manage the grants funding the section 305 procurement projects would be helpful. In a 2011 monitoring report, one grantee requested that FRA provide a public, written policy on receiving, reviewing, and accepting deliverables because grantees did not know if or when they would receive feedback from FRA and whether the deliverables being submitted were in an acceptable format or of sufficient quality. Those grantee officials told us that their project manager informally uses the Federal Transit Administration’s (FTA) guidance in the absence of guidance from FRA. In a 2014 monitoring report, one grantee noted that circulars specific to the grant program would be helpful. According to FRA officials we met with, the agency informs grantees of expectations at a “kick-off meeting” when the grant is first awarded and through routine monitoring and targeted training and technical assistance related to items such as Buy America provisions or safety matters. However, according to the U.S. Domestic Working Group’s Grant Accountability Project, written policies serve as guidelines to ensure new grant programs include provisions for holding grantees accountable for properly using funds and achieving agreed-upon results. To date, FRA has not developed written guidance on grants or project management procedures for grants funding the section 305 equipment procurement projects. According to the Federal Standards for Internal Control, management should implement control activities through policies. For example, management can document responsibilities through policies and communicate policies and procedures so that personnel can implement control activities for their assigned responsibilities. Other agencies have developed detailed written guidance for grantees. For example, FTA developed grants-management circulars on general requirements as well as ones specific to each of its programs, which include the documentation that FTA needs to review and requirements associated with each program. In addition, FTA has developed guidance specific to joint procurements of rail equipment. A lack of guidance could result in FRA’s not receiving sufficient and necessary information from its grantees to carry out its grant oversight activities.

Though FRA Identifies Training Needs, It Has Not Developed Training for Grantees and Agency Staff on Procedures Governing Grants That Fund the Section 305 Procurement Projects

An effective grants management framework includes a mechanism that allows grant recipients and agency staff to establish and maintain a level of subject-matter expertise and competence so that they can fulfill their responsibilities. FRA partially follows the training leading practice area because FRA identifies training needs for grantees and agency staff, but it has not developed training on procedures governing the grants funding the section 305 equipment procurement projects.

FRA identifies training needs of grantees and agency staff. FRA officials stated that they identify grantee training needs formally through scheduled monitoring reports, which specifically ask whether the grantee needs any training or technical assistance. For example, we found that grantees asked for training in 8 of 13 monitoring reports we reviewed, but one grantee asked for training on the terms and conditions of its grant in two consecutive reports. FRA officials stated that agency staff has the opportunity to develop Individual Development Plans to identify training needs for their current positions and development goals.

FRA has not developed training on procedures governing the grants funding the section 305 equipment procurement projects for either grantees or agency staff. According to the Federal Standards for Internal Control, management should demonstrate a commitment to develop competent individuals. For instance agencies can enable individuals to develop competencies appropriate for key roles, reinforce standards of conduct, and tailor training based on the needs of the role. FRA provided grantees with a webinar called “Railroads 101,” which covered several broad industry topics, such as train types and basic rail operations. While the webinar provided to grantees gives a basic overview of the industry, it does not include particular information related to the policies and procedures governing HSIPR grant funds. For example, one grantee said that webinar and in-person trainings on the procedures to manage the grants would be helpful. According to FRA officials, all regional and grants managers can receive project management training; however, new employees often shadow current staff and use electronic systems to learn about their assigned grants. In a 2014 monitoring report, one grantee specifically stated that a training program similar to FTA’s would be very beneficial for grantees. While agency staff can learn on the job, inconsistent understanding of grant administration policies and procedures across the agency could result in risks—such as those related to scope, schedule or budget—not being identified in a timely manner with potential negative impact on a project’s completion or the management of grant funds. For example, the bi-level rail car project’s schedule challenges were not formally reported as significant findings requiring a corrective action plan until 2014, despite schedule concerns noted in a 2013 monitoring report. DOT operating administrations, such as FRA, have the option to determine grants training requirements though we recognize that resources needed for training, such as budgets and staff time, compete with other agency priorities. FRA officials stated that in 2015 it began to develop e-training modules specific to roles within the regional team structure, such as regional managers. While some recent training efforts have been targeted to meet staff and grantee’s needs, FRA officials acknowledged it is reasonable to formalize training efforts going forward.

FRA Has Established Some Communication Mechanisms, but Lacks a Centralized System to Monitor Grants

An effective grants management framework includes establishing an organizational structure, including well-defined reporting lines, to permit the flow of quality information and to assist agency staff and grantees in fulfilling their responsibilities. FRA partially follows the communication leading practice area because, while FRA has developed mechanisms to obtain relevant data based on project information, it does not have a centralized system to monitor grant awards.

Since 2012 FRA has established mechanisms to obtain relevant data based on project information requirements. For example, FRA’s most recent Grants Manual outlines a specific process for grantees’ reporting requirements using a table to outline each step of the process for collecting, reviewing, and approving quarterly progress reports, including the individual responsible for each step and the database or form that should be used to complete the step. The previous version used a narrative format that was less detailed.

We found FRA does not use a centralized system to monitor grant awards. FRA uses multiple systems for grants management, including GrantSolutions, Program Management Tracker, and Sharepoint. GrantSolutions and Program Management Tracker are organized by grant agreement number and are used to track grant administration materials—such as financial reports–and program management materials—such as final copies of monitoring and quarterly progress reports—respectively. Since these systems organize information by the grant agreement’s number and information is divided between different systems, there is no central location for materials related to projects funded through multiple grants, such as the section 305 equipment procurement projects. According to the U.S. Domestic Working Group’s Grant Accountability Project, a centralized information system can allow grant management staff to track a grant’s status, tell how well a grantee is performing, and keep track of problems. An FRA official stated that the agency considered integrating the two systems to provide a central system that combined grants and project management, but it was ultimately deemed too expensive and technologically challenging.

Conclusions

It has been over 5 years since PRIIA significantly expanded FRA’s grant- making role. In that time, FRA has developed a new grants management framework—a large undertaking encompassing all projects funded through its grants—that involved concurrently hiring new staff, developing grants oversight policies and procedures, and awarding grants. In addition, the agency’s grants management role has included overseeing the section 305 equipment procurement projects—a fundamentally new type of project for states to lead, involving technically complex new equipment that FRA had limited experience overseeing. In 2010, we noted that a confluence of factors, including simultaneously carrying out multiple new responsibilities, could pose risks for the use of federal funds for high speed rail projects and that a robust grant-oversight program would be a critical element to making sound federal investments in high speed rail.

FRA’s grants management experience with the grants that fund the section 305 equipment procurement projects demonstrates that additional improvements to performance monitoring, written documentation, and training could enable more effective grants management. A lack of specific goals and measures can make it difficult to keep grantees accountable and ensure that grantees are making progress toward project deliverables. Further, evaluating and documenting the results of monitoring activities in a timely manner would help FRA to more proactively address risks to a project’s completion as they arise. Providing written guidance to grantees on procedures and agency expectations would help ensure that agency officials obtain the information they need to fulfill their oversight responsibilities. Training can ensure consistent understanding of policies to ensure project risks are identified in a timely manner; however, we recognize that training requires decision makers to make trade-offs based on funding and staff availability, as these resources must be used to meet multiple agency priorities. While using a centralized electronic-grant system could help agency staff better track grant progress, FRA has explored this option and determined that it would be too costly and technologically challenging.

The section 305 equipment procurement projects can be used to identify lessons learned to strengthen FRA’s overall grants management framework and project oversight for future grants. While no new section 305 equipment procurement projects are currently planned, FRA manages and oversees approximately 200 ongoing grant projects and the Passenger Rail Investment and Reform Act of 2015, passed as a title in the FAST Act, authorized a new infrastructure and safety grant program to assist grantees in financing the cost of improving passenger and freight rail transportation systems. While FRA officials began efforts to further formalize grants management procedures during the course of our review—such as drafting a portfolio management guide to centralize guidance related to the regional manager role for project oversight— additional steps could further enhance the agency’s approach to grants management. Improving the agency’s processes to be more proactive in overseeing and monitoring grant performance would help minimize project risk, increase grantee accountability, and improve the efficient and effective use of federal funds for FRA’s grants portfolio.

Recommendations for Executive Action

To strengthen FRA’s grants management practices, we recommend the Secretary of Transportation direct the FRA Administrator to take the following actions: 1. Enhance the process outlined in the Grants Manual to monitor project performance for future grants to include: (1) performance measures directly linked to project goals, and (2) fully incorporating timely and actionable information on grantee performance into FRA’s review process to help determine whether current efforts are in line with the overall project goals. 2. Develop and provide written guidance to grantees to include FRA’s expectations on the type of information grantees should provide, such as guidance specific to deliverables and milestones for each grant project. 3. Analyze training needs and formalize a training plan for grantees and agency staff, which could include training on grant-specific procedures and policies.

Agency Comments

We provided a draft of this report to Amtrak and the Department of Transportation for review and comment. In written comments, reproduced in appendix III, DOT concurred with GAO’s recommendations. FRA also provided technical comments that were incorporated, as appropriate. Amtrak did not provide comments.

We are sending copies of this report to the Secretary of Transportation, the Administrator of the Federal Railroad Administration, and Amtrak. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

This report addresses the following questions: 1) How has FRA carried out its grants management roles and responsibilities regarding the PRIIA section 305 equipment procurement projects? 2) To what extent has FRA’s approach to grants management for the PRIIA section 305 equipment procurements met leading practices and whether FRA’s grants management practices could be improved?

To determine how FRA has carried out its grants management roles and responsibilities for the grants funding the section 305 equipment procurement projects, we reviewed FRA policies and other guidance established to outline the agency’s responsibilities, such as the August 2015 Grants Management Manual and the 2013 Program Management Plan. We reviewed the terms and conditions of the six grant agreements FRA executed with the California Department of Transportation (Caltrans), the Illinois Department of Transportation (IDOT), and the Washington State Department of Transportation (WSDOT) that fund the section 305 equipment procurements to identify project milestones and deliverables, as well as specific activities or points in the administration of the grants where FRA reviews, approves, or concurs with the state grantees on specific project activities. While three of these grants covered broader corridor improvements beyond the equipment projects (Chicago-St. Louis Corridor Improvement, the Chicago-Quad Cities Expansion Program, and the Pacific Northwest Rail Corridor Program), we focused on grant administration and oversight activities related to the tasks associated with the equipment procurements. We did not assess FRA’s grant management activities beyond those applicable to these six grant awards.

We reviewed additional documentation related to FRA’s oversight of these six selected grants, including 96 quarterly progress reports provided by FRA for fiscal years 2013 through 2015 and 13 monitoring reports completed by FRA for calendar years 2011 through 2015 to better understand how FRA carries out its grants management roles and responsibilities. We focused our review on FRA’s grants management activities post-award, including monitoring of grant awards, project oversight, and technical assistance activities. We did not examine the agency’s award issuance or grant closeout processes, as the equipment projects are ongoing at the time of this report’s issuance. We reviewed additional policy and guidance documentation related to the oversight of the section 305 equipment procurement projects, including the relevant Notices of Funding Availability issued by FRA, FRA’s Monitoring Procedures describing the oversight conducted by the Monitoring and Technical Assistance Contractors (MTAC), and the MTAC task order outlining specific tasks and technical assistance contractors may perform for the equipment projects. We also reviewed reports independent consultants provided to FRA, such as reports on the bi-level car project’s status and routine summaries of project-level activities and meetings between state grantees, equipment vendors, and subject matter experts. To collect additional information about how FRA carried out grants management activities for the locomotive and bi-level car equipment procurements, we interviewed FRA Regional Managers and Grant Managers within the Office of Railroad Policy and Development, subject matter experts within other FRA offices such as the Office of Research, Development and Technology, and the Office of Chief Counsel, and MTAC contractors supporting the agency’s oversight activities. We also interviewed Amtrak officials, California, Illinois, and Washington State departments of transportation, as well as Next Generation Equipment Committee (NGEC) participants and independent consultants involved with the section 305 equipment procurement projects.

To assess the extent to which FRA’s approach to grants management for the equipment procurements met leading practices, we identified relevant and applicable leading practices and supporting characteristics that contribute to those practices, using generally accepted grants management practices from a variety of sources (see table 3).

Using these sources, we identified four leading grants management practices—communication, written documentation, training, and performance monitoring—as well as specific characteristics of grants management that support these practices. To assess FRA’s grants management, we reviewed grant documentation and FRA’s grants management plans, policies, and procedures to determine the extent to which FRA’s practices aligned with the supporting characteristics of our leading practices. Each leading practice is aligned with a few supporting characteristics. For example, supporting characteristics for the written documentation practice include: (1) develop and maintain written policies and procedures that communicate knowledge among agency staff, and (2) develop guidance specific to each grant program, including documentation outlining agency and grantee expectations, among others. For additional information on the leading practices and supporting characteristics, see appendix II. We used that information in aggregate to determine the extent to which the leading practice was followed. For example, we reviewed grant documentation, such as quarterly progress reports submitted by grantees and monitoring reports completed by FRA officials for the section 305 equipment procurements from calendar year 2011 through 2015. We also reviewed FRA’s grants management policies and procedures, such as those outlined in the Grants Management Manual and Monitoring Procedures to determine the extent to which the activities and processes described met the supporting characteristics. We also interviewed officials at FRA and the California, Illinois, and Washington State departments of transportation as well as FRA’s independent contractors. Our assessment of the alignment of FRA’s practices with the supporting characteristics served as the basis for our overall assessment as to whether each leading practice was followed or substantially followed; partially followed; or minimally or not followed. For example, if we found supporting evidence that two of the three characteristics of a practice were substantially followed but the third characteristic was not followed, we determined that the leading practice was partially followed. The criteria to determine if practices and supporting characteristics were: followed or substantially followed—plans, policies, or processes have been developed and implemented properly for all or nearly all supporting characteristics partially followed—plans, policies, or processes have been developed and implemented properly for some supporting characteristics minimally or not followed—plans, policies, or processes are lacking for all or nearly all supporting characteristics To further determine whether FRA’s grants management practices could be improved, we interviewed FRA and Caltrans, IDOT, and WSDOT officials, NGEC participants, and independent contractors to obtain perspectives on lessons learned from the projects to date, as well as examples of what worked well with the FRA’s management and oversight of the section 305 equipment procurements.

We conducted this performance audit from June 2015 to May 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Grants Management Leading Practices with GAO Assessments

We assessed the extent to which the Federal Railroad Administration’s (FRA) approach to grants management for the grants funding the section 305 equipment procurement projects met leading practices and supporting characteristics for grants management that we identified in the areas of performance monitoring, written documentation, training, and communication. We reviewed grant documentation and FRA’s plans, policies, and procedures and analyzed interviews with FRA and grantee officials as well as Monitoring and Technical Assistance Contractors (MTAC). Based on our analysis, we determined the extent to which each characteristic was followed or substantially followed; partially followed; or minimally or not followed. Our assessment of the characteristics served as the basis for our overall assessment on the extent to which each leading practice was followed. We categorized the assessments using the scale below:

Followed or substantially followed the leading practice—plans, policies, or processes have been developed and implemented properly for all or nearly all supporting characteristics.

Partially followed the leading practice—plans, policies, or processes have been developed and implemented properly for some supporting characteristics.

Minimally or did not follow the leading practice—plans, policies, or processes are lacking for all or nearly all supporting characteristics.

Table 4 provides greater detail, including examples, of our comparison of FRA’s grants management approach with the supporting characteristics that are aligned with leading practices.

Appendix III: Comments from the Department of Transportation

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, the following individuals made important contributions to this report: Melissa Bodeau, Steve Cohen, Swati Deo, Derry Henrick, SaraAnn Moessbauer, Malika Rice, Maria Wallace, and Crystal Wesco.