Rail Grant Oversight: Greater Adherence to Leading Practices Needed to Improve Grants Management
Published: 2016-05-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Passenger Rail Investment and Improvement Act of 2008 (PRIIA) expanded FRA's role by, among other things, authorizing grant programs for intercity passenger rail. Section 305 of PRIIA established a Next Generation Equipment Committee to design, develop specifications, and procure standardized rail equipment. FRA awarded approximately $800 million in grant funding for two locomotive and bi-level passenger car procurement projects.
GAO was asked to review issues related to FRA's oversight of the grants funding the PRIIA section 305 equipment procurements. This report examines: (1) how FRA has carried out its grants management roles and responsibilities for the PRIIA section 305 equipment procurement projects, and (2) the extent to which FRA's approach has met leading practices and whether FRA's grants management practices could be improved. GAO reviewed grants management policies and practices and identified relevant and applicable leading practices to be used as criteria in assessing FRA's grants management.
section_title: What GAO Found
paragraphs: The Federal Railroad Administration (FRA) initially used a regional oversight and monitoring approach outlined in its Grants Management Manual (Grants Manual) to manage the grants funding the section 305 equipment procurement projects, but in the face of challenges that approach evolved to include additional project-level oversight. When FRA began awarding the grants for the section 305 equipment procurement projects in 2010, its grants management approach was defined by the Grants Manual —including routine and scheduled monitoring—and by the terms of the grant agreements funding the locomotive and bi-level passenger car projects (see figure). As the bi-level car project encountered significant challenges—including major schedule delays—FRA moved to a project structure including increased contractor support to better oversee the locomotive and bi-level car projects in response to problems identified in the bi-level car project's schedule. In August 2015, the bi-level car suffered a structural testing failure, and as of April 2016, production of the bi-level car was on hold, and the final equipment delivery date was unknown.
FRA's grants management approach partially follows GAO-identified leading practices for performance monitoring, communication, training, and written documentation, but FRA's approach could be improved by better alignment with those practices. For example, while FRA stated that project progress is measured by tracking scope, schedule, and budget, it has not documented a process to identify project-specific goals and associated performance measures. Establishing a process that ensures project goals are identified, tracked, and fulfilled is a leading practice of effective grants management. Without explicit project goals and associated performance measures, it may be challenging for decision makers to track and assess a project's progress, make decisions about future efforts, and keep grantees accountable for outcomes. In addition, FRA has not provided documentation outlining grantees' expectations or developed written guidance specific to the section 305 equipment procurement projects. An effective grants management framework includes developing and maintaining written documentation as a means to obtain and retain organizational knowledge and to ensure accountability. According to FRA officials, the agency informs grantees of expectations through routine monitoring and technical assistance. However, the lack of written guidance, goals, and performance measures could result in FRA's not receiving sufficient and necessary information from its grantees to carry out its grant oversight activities.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that the Secretary of Transportation direct FRA to enhance its process for monitoring project performance to include project goals and performance measures that directly link to those goals, and develop and provide written guidance to grantees outlining agency expectations and deliverables. The department concurred with the recommendations.

section_title: Letter
paragraphs: The Passenger Rail Investment and Improvement Act of 2008 (PRIIA)  expanded the Federal Railroad Administration’s (FRA) role in developing  and managing the nation’s rail system by, among other things, authorizing  grant programs for high-speed and intercity passenger rail. Section 305 of  PRIIA required the National Railroad Passenger Corporation (Amtrak) to  establish a Next Generation Corridor Equipment Pool Committee (NGEC)  consisting of Amtrak, the FRA, interested states, and others to design,  develop specifications for, and procure standardized rail-corridor  equipment. PRIIA authorized FRA to award capital rail project grants for  these section 305 procurements. FRA created the High Speed Intercity  Passenger Rail (HSIPR) program to administer several PRIIA grant  programs and competitively award grants, including those providing  funding for the section 305 rail equipment. Through this program, FRA  has awarded approximately $800 million in grant funding for two  equipment-procurement projects for diesel-electric locomotives  (locomotive) and bi-level passenger cars. The grant agreements that fund  these two procurements require that rail corridor equipment purchased  with federal funds comply with technical specifications developed by the  NGEC.
You asked us to review FRA’s oversight of the equipment procurement  projects funded under HSIPR (referred to in this report as the PRIIA  section 305 equipment procurement projects). This report examines (1)  how FRA has carried out its grants management roles and responsibilities  regarding the PRIIA section 305 equipment procurement projects, and (2)  the extent to which FRA’s approach to grants management for the PRIIA  section 305 equipment procurements has met leading practices and  whether FRA’s grants management practices could be improved.
To determine how FRA has carried out its grants management roles and  responsibilities for the grants funding the section 305 equipment  procurement projects, we reviewed FRA policies and other guidance  established to outline the agency’s responsibilities, such as FRA’s Grants  Management Manual (Grants Manual). We also reviewed the terms and  conditions of the six grant agreements FRA executed with the California  Department of Transportation (Caltrans); the Illinois Department of  Transportation (IDOT); and the Washington State Department of  Transportation (WSDOT) funding the section 305 equipment procurement  projects. We focused specifically on FRA’s post-award oversight and  monitoring efforts related to the grants that fund the section 305  procurement projects, and we reviewed additional documentation related  to FRA’s oversight and monitoring of these six grants, including grantees’  quarterly progress reports and FRA’s monitoring reports completed from  2011 through 2015. We interviewed FRA Regional Managers and Grant  Managers within the Office of Railroad Policy and Development, as well  as subject matter experts within other FRA offices. We also interviewed  Amtrak officials, and officials at the California, Illinois, and Washington  State departments of transportation as well as NGEC participants and  independent consultants involved with the section 305 equipment  procurement projects about how FRA carried out grants management  activities for them.
To assess the extent to which FRA’s approach to grants management for  the equipment procurements met leading practices, we reviewed grants  management practices and identified relevant and applicable leading  practices and supporting characteristics that contribute to those practices  to be used as criteria. To do this step, we identified generally accepted  grants management practices from a variety of sources, including  Standards for Internal Control in the Federal Government, prior GAO  reports, the Department of Justice, Office of the Inspector General’s  Improving the Grant Management Process, and the U.S. Domestic  Working Group’s Grant Accountability Project. Using sources such as  these, we identified four leading grants-management practices— communication, written documentation, training, and performance  monitoring—as well as specific characteristics of grants management that  support these practices. To assess FRA’s grants management, we  reviewed grant documentation and FRA’s grants management plans,  policies, and procedures to determine the extent to which FRA’s practices  aligned with the supporting characteristics of our leading practices. We  used that information in aggregate to determine the extent to which the  leading practice was followed. We also interviewed FRA and Caltrans,  IDOT, and WSDOT officials and FRA’s independent contractors. Our  assessment of the alignment of FRA’s practices with the supporting  characteristics served as the basis for our overall assessment as to  whether each leading practice was followed or substantially followed,  partially followed, or minimally or not followed. For example, if we found  supporting evidence that two of the three characteristics of a practice  were substantially followed but the third characteristic was not followed,  we would determine that the leading practice was partially followed. To  further determine whether FRA’s grants management practices could be  improved, we interviewed FRA and Caltrans, IDOT, and WSDOT officials  as well as NGEC participants and independent consultants to obtain  perspectives on FRA’s involvement in the management and oversight of  the section 305 equipment procurement projects.
We conducted this performance audit from June 2015 to May 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: FRA is the primary federal agency responsible for issuing and enforcing  railroad safety regulations and for distributing federal funds for intercity  passenger rail service. PRIIA mandated new responsibilities for FRA to  plan, award, and oversee the use of federal funds for intercity passenger  rail. The American Recovery and Reinvestment Act of 2009 (Recovery  Act) appropriated funding for high-speed rail projects, which resulted in a  dramatic increase in federal funding for intercity passenger rail projects.  Prior to 2009, FRA had a very limited grant portfolio, receiving  appropriations for approximately $30 million in grant funding in fiscal year  2008, for example, primarily for intercity-passenger rail grants to states.  With expanded responsibilities, the agency had to quickly award  approximately $8 billion in Recovery Act funds while simultaneously  developing policies and procedures for grants management. In 2015,  FRA managed a portfolio of approximately 200 grants and managed  $17.7 billion in obligated grants. In December 2015, Congress passed the  Passenger Rail Reform and Investment Act of 2015 as a title in the Fixing  America’s Surface Transportation Act (FAST Act), which authorized a  new consolidated rail infrastructure and safety-improvements grant  program to assist grantees in financing the cost of improving passenger  and freight rail transportation systems.
In 2010, when FRA was in the early stages of developing its grant  oversight program, GAO identified principles that would become  important as FRA transitioned from awarding grants to overseeing their  performance. For example, a well-designed and implemented grant  oversight program is critical to ensuring effective use of federal grant  funds. FRA began drafting a Grants Manual to communicate the agency’s  overall approach to grants management in April 2010 when it was in the  process of building up its grants management program. This Grants  Manual includes procedures developed to address the agency’s  expanded responsibilities following the enactment of the Recovery Act.  FRA’s 2013 Program Management Plan outlined a “matrixed”  organization structure intended to facilitate the flow of skills and  information across functions within FRA (see fig. 1 below). FRA’s grant  and loan portfolio is further organized into geographically based regional  portfolios. Each regional portfolio is managed by a Regional Team  comprised of a team lead—the Regional Manager—and subject matter  experts from other FRA offices. Teams are responsible for funding,  project delivery, and monitoring and oversight activities. For example, for  the section 305 procurement projects, the “project” entails the design,  manufacturing, and delivery of the bi-level and locomotive equipment, and  FRA is responsible for evaluating and monitoring—from start to finish—all  the steps required to ensure successful project delivery. The Regional  Manager is responsible for coordinating the entire regional team, and may  share some advisory and oversight responsibilities with the designated  grant managers—those directly responsible for overseeing and  maintaining specific portions of the grants management process. For  example, for the two section 305 equipment procurements—the bi-level  cars and the locomotives—grant oversight includes collaboration between  Regional Managers and Grant Managers within the Office of Railroad  Policy and Development, subject matter experts within the Office of  Safety and the Office of Chief Counsel, and contractors.
Launched in January 2010, the NGEC has developed, adopted, and  promulgated six specifications for next-generation corridor equipment.  For example, one of the specifications was to develop and build vehicles  for the future or next generation, such as a locomotive capable of speeds  up to 125 mph. The NGEC developed standardized specifications  intended to make it possible for a group of states to buy equipment faster,  at a lower cost, with reduced operating and maintenance costs going  forward. Using the NGEC-developed specifications as a foundation for  two equipment procurement projects, Caltrans took the lead on the bi- level car project, and IDOT leads the locomotive project. For example,  IDOT—representing the Midwest coalition of Missouri, Michigan, and  Iowa—serves as the lead state on behalf of itself, Caltrans, and WSDOT  in conducting the joint locomotive procurement. Though Caltrans and  IDOT are the lead states for the two projects, they have established  memorandums of understanding for the equipment with their partner  states, including Washington, Missouri, Michigan, Iowa, and Wisconsin.
The NGEC is comprised of FRA, Amtrak, state, and industry  participants. The NGEC structure includes an Executive Board  responsible for writing the technical requirements document and  approving the final specifications. The requirements document outlines  the design objectives and specific performance requirements that need to  be met for each type of equipment—in the case of the section 305  equipment-procurement projects, diesel electric locomotives and bi-level  passenger cars (see fig. 2 below). The specification is a detailed  technical document developed by the NGEC Technical Subcommittee  intended to address the range of operational considerations needed to  procure, design, and manufacture a fleet of bi-level cars or locomotives  for use in intercity corridor service. The Technical Subcommittee includes  working groups in several functional areas—e.g., structural, electrical,  and mechanical—responsible for drafting relevant elements of each  technical specification. The NGEC Review Panel evaluates specifications  for compliance with the requirements document as well as compliance  with regulations regarding safety, accessibility, and operations. The  NGEC also has a detailed system for approved specifications and  documents to be revised, edited, and updated through a formal process.
Six grant agreements between FRA and Caltrans, IDOT, and WSDOT,  respectively, are funding the two locomotive and bi-level equipment  procurement projects (see table 1 below). FRA entered into two grant  agreements with Caltrans, three grant agreements with IDOT, and one  grant agreement with WSDOT. WSDOT’s grant agreement was  developed prior to the decision to designate one lead state for each of the  two equipment projects. After all 6 grant agreements were executed, the  efforts were split into two projects, as Caltrans contracted with Sumitomo  Corporation of America and Nippon Sharyo in November 2012 to  purchase 130 bi-level cars and IDOT contracted with Siemens in March  2014 to manufacture 47 locomotives. These grants are funded by a mix of  Recovery Act and DOT fiscal year 2010 appropriations. For the six  grants, approximately 75 percent of the funding is Recovery Act money  and four of the six grants have an expenditure deadline of September 30,  2017.
section_title: FRA’s Grants Management Approach Has Evolved to Administer the Section 305 Equipment Procurement Projects
paragraphs: FRA’s management of the grants funding the section 305 equipment  procurement projects has evolved from a general grants management  approach to include additional project-level oversight. This framework is  outlined in the Grants Manual—including routine and scheduled  monitoring reviews—and in the specific activities outlined in the terms of  the grant agreements funding the section 305 equipment procurements.  FRA’s approach to manage the section 305 equipment projects evolved  as the projects progressed, to include additional involvement in project- level activities. Specifically, in 2015, FRA transitioned to a project-based  oversight structure to carry out its grants management framework for the  section 305 equipment procurement projects and, in 2014, used  independent contractors to support project oversight. For example, FRA  used additional contractor resources to support the agency’s grants  management oversight after delays were identified in the bi-level project  schedule.
subsections: section_title: FRA Implemented General Grants Management Procedures from the Start of the Equipment Procurement Projects
paragraphs: Beginning in 2010, FRA implemented its draft grants management  framework to manage the post-award phase of the grants funding the  section 305 equipment procurement projects. The general framework  used to manage FRA’s entire grant portfolio—including the six grants  funding the procurements—is described in the agency’s Grants Manual,  articulating its oversight and monitoring procedures to track grantees’  performance. After grants are awarded, the general oversight and  monitoring procedures described in the Grants Manual are carried out in  accordance with the specific terms of the individual grant agreements.  According to the Grants Manual and interviews with FRA and state  grantee officials, FRA’s monitoring procedures include what is referred to  as “routine” and “scheduled” monitoring:
Routine monitoring: the periodic review of progress for all active  FRA grants to ensure grantees are in compliance with the terms of  their respective grant agreements. For example, FRA’s grant  managers review the quarterly progress reports (QPR) grantees  typically are required to submit under their grant agreements,  describing the status of grant projects, including any issues  encountered affecting the scope, schedule, and budget described in  the grant agreement. For the section 305 equipment procurement  projects, FRA reviews QPRs from Caltrans, IDOT, and WSDOT for  their respective grant agreements. While FRA provides grantees with  a QPR template that includes fields for narrative descriptions of  significant accomplishments and any technical, cost, or schedule  problems experienced during the review period, our review of the  QPRs FRA provided shows that the information included in these  reports varied by grantee. For example, IDOT first reported delays  with the bi-level delivery schedule in the first quarter of fiscal year  2013, while the bi-level project lead, Caltrans, did not report schedule  problems until the first quarter of fiscal year 2014. In addition, the level  of detail regarding identified issues within the QPRs varied. For  example, in the first quarter of fiscal year 2014, IDOT and Caltrans  each discussed the bi-level delivery schedule; however, IDOT  provided additional information raising concerns that the projected  delivery schedule would not meet the September 30, 2017, deadline  for the expenditure of funds. FRA officials told us QPRs do not always  identify project issues encountered in a given quarter because the  agency works informally with grantees to address problems as they  arise. According to FRA officials, the agency has developed a new  QPR form for grantees that will enable more detailed data on status  and progress to be collected. FRA officials told us the agency plans to  begin using this form in 2016, but the new QPR forms were not in use  at the time of this review.
Scheduled monitoring: a detailed annual review of select grant  projects’ progress, which FRA officials carry out using desk and/or on- site review sessions. FRA selects grant projects from its full grant  portfolio to receive scheduled monitoring using factors such as project  type, funding level, the time elapsed since previous monitoring  activities, and knowledge of existing project issues. For the section  305 equipment procurement projects, a series of checklists help guide  FRA’s scheduled monitoring reviews. According to the Grants  Manual, the results of scheduled monitoring activities are recorded in  a monitoring review report and should identify any significant findings  (i.e., issues that jeopardize project completion or compliance) and  areas of interest (i.e., issues that have the potential to become  significant). FRA’s monitoring and oversight procedures require  grantees to develop and submit corrective action plans when  significant findings—such as issues that put the entire project at risk— are identified, describing the practices that grantees and other  stakeholders, as appropriate, will follow to address the identified  issues. To date, FRA has conducted 13 scheduled monitoring reviews  for the six grants providing funding for the section 305 procurement  projects, identifying significant findings associated with the bi-level car  project in 2014 and 2015. For example, one monitoring report  identified the bi-level design and delivery schedule as a significant  finding because the manufacturing timeline did not meet the  expenditure deadline. The bi-level schedule delay was reported by  FRA in 2013, but at that time it was identified as an area of interest  rather than as a significant finding. While no significant findings have  been identified with the locomotive project to date, FRA has identified  areas of interest associated with the locomotive project. For example,  in a 2015 monitoring report, FRA reported one area of interest for the  locomotive project—that the states receiving the locomotive  equipment do not yet have an ownership, operation, or maintenance  plan in place to manage the equipment upon delivery.
As FRA began implementing its general grants management framework,  it also began to award the six grants providing funding for the section 305  equipment procurement projects. See figure 3 below for a timeline of  FRA’s grants management and section 305 equipment project  milestones.
FRA’s grant responsibilities are also defined by the six individual grant  agreements funding the equipment purchases. As noted above, the  grant agreements require that the equipment purchased with federal  funds comply with specifications developed by the NGEC. The grant  agreements also generally incorporate the requirement that the  equipment be procured in a manner consistent with Buy America  requirements. In our analysis of the six agreements, we identified areas  where FRA has specific responsibilities for the equipment grants as the  projects progressed. Depending on the terms of the particular grants  funding the section 305 procurement projects, FRA’s involvement in the  locomotive and bi-level project may include reviewing the grantees’ draft  requests for proposal (RFP) seeking interest from potential equipment  manufacturers, participation in the design review meetings with the  grantees and equipment manufacturers, and participation during section  305 equipment testing.
Reviewing and approving RFP packages: Three of the grant  agreements explicitly require FRA’s review and approval of the  grantees’ draft RFPs; the other three grants are either silent or require  approval by NGEC. The FRA officials we spoke with explained that  the agency’s participation in the proposal review process included  reviewing the RFP package to ensure the grantees’ bid solicitations  included applicable federal provisions (e.g., Buy America and Davis- Bacon requirements). For example, FRA officials reviewed the RFP  for the contract that Caltrans eventually awarded to Nippon-Sharyo. In  addition, FRA officials told us the agency had worked to reconcile  differences between the various state laws of equipment grantees.  According to FRA, the difficulties in reconciling each state’s  requirements were a source of delay in awarding the initial  procurements. The grant agreements do not provide for FRA’s  participation in the grantee’s bid evaluation or selection process, nor  is this process required by applicable federal law. FRA officials told us  agency officials were not involved in these processes and did not  provide concurrence with IDOT or Caltrans’ bid selections.
Reviewing and approving design specifications: Four of the grant  agreements explicitly require FRA to approve the design of the bi- level rail car and/or locomotive, while the other grant agreements  provide for FRA to review the design specifications as a stakeholder.  FRA officials told us agency officials participated in multiple reviews of  the manufacturers’ design documents for both the bi-level cars and  locomotive equipment —from the initial design concept to a final  design plan—to ensure the designs comply with federal safety  standards. For example, according to the IDOT officials we met with,  FRA officials attend all locomotive design-review meetings and the  state submits design documentation to FRA’s safety team for  approval.
Participation in section 305 equipment testing: Some of the grant  agreements provide a role for FRA in the testing of the equipment. For  example, two of the grant agreements specify that FRA will be given  the opportunity to witness and approve production tests. Another  grant agreement authorizes FRA to verify test results. The FRA  officials we spoke with said agency officials participated in equipment  testing to ensure the bi-level cars and locomotive equipment meet  federal safety standards and that the testing procedures are  appropriate. According to Caltrans and IDOT officials, FRA is invited  to all inspections and testing for the bi-level equipment.
Because the agreements between FRA and the states are cooperative  agreements, FRA provides substantial programmatic involvement in the  section 305 equipment procurements. As provided in the grant  agreements, substantial programmatic involvement means that FRA will  assist and coordinate with grantees and participate in grant project  activities after the contract is awarded. For example, FRA provides  grantees with administrative, programmatic, and technical assistance as  needed. The FRA officials we met with said the agency has provided  training on the Buy America requirements and the agency’s financial  reporting requirement in response to the needs of the section 305  equipment grantees. At the start of the bi-level and locomotive  procurements, the extent of FRA’s involvement varied based on the  state’s previous project management experience. For example, a former  FRA official involved at the start of the bi-level car project told us the  Caltrans staff managing the project at the beginning did not seek FRA’s  involvement beyond the areas described in the grant agreements funding  the equipment procurements. These staff had previous procurement  experience but lacked experience in industry and manufacturing issues,  affecting the state’s ability to effectively manage the project. In addition,  Caltrans had prior experience purchasing rail cars and the 305  specification that the NGEC developed for the bi-level cars was based on  the “California Car” that Caltrans had purchased between 1995 and 1997.  In contrast, the IDOT officials we met with said that since IDOT did not  have previous procurement experience, the state proactively sought  FRA’s involvement from the start of the locomotive project and hired an  industry expert with locomotive experience to help manage the project.  For example, FRA reviewed IDOT’s ordering agreement with Siemens  (the locomotive manufacturer) and the IDOT officials we spoke with said  FRA has participated in monthly meetings with the Midwest Partner states  that will be receiving the locomotive equipment from the start of the  project.
section_title: As the Bi-level Project Encountered Challenges, FRA’s Approach for the Section 305 Equipment Procurement Projects Evolved to Include Additional Project-Level Oversight
paragraphs: As the section 305 procurement projects have progressed, the bi-level car  project has encountered challenges that jeopardize project completion by  the expenditure deadline. While FRA reported issues with the bi-level  project’s schedule in a 2013 monitoring review report, the issues facing  the bi-level project were not identified as a significant finding—thus, did  not require a corrective action plan—until 2014. In 2014, the bi-level  project schedule was also reported as a significant issue in an  independent review of the project by an FRA consultant. For example, the  independent review raised concerns that the level of detail in the bi-level  schedule did not meet the project management requirement and that  design activities were up to 7 months behind schedule. In FRA’s 2014- scheduled monitoring review report, the bi-level car project’s compressed  design and delivery schedule was identified as a significant finding  because the schedule posed a risk to the project leading up to the  expenditure deadline. Grantees are responsible for developing corrective  action plans to resolve significant issues, and FRA officials told us that  the agency supported the grantee’s efforts to develop corrective actions  in response to the bi-level project issues identified. For example, in April  2015, FRA convened a meeting with Caltrans, IDOT, and Nippon Sharyo  to work through the bi-level project issues. The meeting resulted in a  series of corrective actions including replacing Caltrans and contractor  program managers and establishing a designated bi-level program  manager to oversee activities related to the scope, schedule, and budget  of the project. Following that meeting, in May 2015, documentation of the  corrective action plan to address bi-level issues revealed that the bi-level  project was 14 months behind schedule.
In August 2015, the bi-level car schedule encountered an additional  setback after the bi-level car shell suffered a structural failure during  testing and production stopped for car shell redesign. In a December  2015 monitoring review report, FRA stated that the manufacturers’ project  schedule showed a delay of 24 months and that the bi-level project would  not be completed by the expenditure deadline. Furthermore, FRA’s  monitoring report revealed the agency’s concern with Nippon Sharyo’s  quality standards and project management. For example, FRA reported  that Nippon Sharyo was increasing risk to the bi-level project by  manufacturing bi-level car parts without an accepted car shell design. As  of April 2016, the bi-level production schedule is on hold and the final  project equipment delivery date is unknown. To date, FRA has not  identified significant findings with the locomotive project through its  monitoring reviews, and as of April 2016 the project was on schedule,  with planned equipment delivery in June 2017.
In addition to FRA’s efforts to address significant issues related to the bi- level car project, FRA’s grants management approach for both of the  section 305 equipment procurements has evolved to include additional  project-level oversight. In practice, FRA is involved in the procurements at  two levels—(1) as grants manager overseeing the state grantees, and (2)  increasingly, as support for the underlying activity between the states and  their selected contractors in which FRA has elected to participate. As  discussed below, this approach includes a revised management  structure, informal interaction with section 305 equipment procurement  stakeholders, and the use of outside contractors.
Revised management structure: FRA initially carried out its grants  management framework under the regional oversight structure used  to manage its entire grant portfolio. However, FRA officials told us this  regional approach—under which FRA’s management for the six grant  agreements funding the section 305 equipment procurement projects  was carried out by three different FRA regional teams—was not  working for the section 305 equipment procurements. According to the  FRA officials we spoke with, it became clear that there should be a  single manager responsible for overseeing both projects due in part to  the number of grants managers and regional managers involved and  due to project management challenges. In 2015, FRA transitioned to a  project-based oversight structure for the section 305 equipment  projects—referred to as the National Vehicle Program—making the  Midwest Regional Manager the primary FRA official responsible for  managing and coordinating grantees’ efforts for both the bi-level and  locomotive equipment projects, instead of managing the six individual  grants. In addition, FRA hired an individual with prior passenger-rail  industry experience into the role of the National Vehicle Program  Manager, to participate in routine meetings with grantees and report  relevant information to the Midwest Regional Manager. According to  one of the grantees we met with, FRA did not initially have sufficient  experience to manage the needs of the delivery program, and each of  the grantees said turnover in FRA’s grants management staff was a  challenge. For example, Caltrans officials worked with three different  FRA grants managers and four different FRA regional managers in 5  years, and IDOT officials worked with two regional managers in the  same time. According to one grantee, many of the FRA staff involved  in the project early on had policy experience but lacked experience in  project delivery. According to FRA officials, the agency has continued  to experience turnover in key roles associated with the equipment  procurements.
Informal Interactions with project stakeholders: While FRA  continues to apply the monitoring and oversight procedures in its  Grants Manual as described above, FRA officials told us frequent and  informal interactions—for example, ongoing participation in meetings  between project stakeholders such as grantees and equipment  manufacturers—help the agency monitor grantees’ performance.  Caltrans and IDOT officials told us that FRA officials and consultants  currently participate in project-level meetings several times a week.  For example, Caltrans officials told us that FRA officials attend  meetings with project stakeholders four times a week, including a  quality assurance meeting with Nippon Sharyo and a weekly  coordination meeting with IDOT. According to FRA officials we spoke  with, these informal interactions are the primary way information about  the status of the locomotive and bi-level projects is communicated. In  addition, according to FRA, the agency and its contractors have held  a series of meetings with Nippon Sharyo to address bi-level rail car  project challenges—including discussions related to specific project  deficiencies and the manufacturer’s responses to the areas of  concern identified by FRA.
Additional contractor involvement: In May 2014, FRA awarded a  new contract for Monitoring and Technical Assistance Contractors  (MTAC) to help manage the equipment procurements, and more  recently increased contractor support to meet the growing needs of  the bi-level equipment project. The overall role and responsibilities of  MTAC contractors are detailed in FRA’s Monitoring Procedures,  published in August 2014, and include supporting FRA’s project-level  oversight with technical expertise and delivering training and technical  assistance to grantees. The MTAC contract enables contractors with  subject-matter expertise to be deployed in response to a range of  FRA programs or projects as needed. The FRA officials we met with  told us MTAC is not included in the grants management framework  described in the Grants Manual because the use of MTAC contractors  is dependent on the needs of the grant project, allowing the agency to  obtain MTAC support and expertise as needed. For example, FRA  has significantly increased MTAC support based on the needs of the  section 305 equipment procurement projects and, in particular, the  needs of the bi-level project. FRA officials told us they executed a new  MTAC contract because the contractor support necessary to oversee  the bi-level project was more than the agency anticipated, exceeding  the resources provided under the agency’s original contract. MTAC  contractors participate in meetings between bi-level and locomotive  grantees and vendors and provide project updates to FRA through  summary reports of these interactions. In addition, the National  Vehicle Program Manager and MTAC contractors meet weekly to  discuss the procurements and report relevant project information,  including any issues, to FRA’s Midwest Regional Manager who is  responsible for overseeing the National Vehicle Program. FRA  officials told us that MTAC’s direct involvement with the grantees  receiving funding for the section 305 equipment procurement projects  provides FRA with a “boots on the ground perspective” of the project’s  status. In addition, MTAC provides Caltrans with technical support  and oversees Caltrans project management by participating in weekly  meetings with the states and manufacturer.
section_title: FRA’s Grants Management Approach Partially Follows Leading Practices
paragraphs: We found that FRA partially follows the grants management leading  practices we identified in the areas of performance monitoring, written  documentation, training, and communication. Practices in these four  areas are closely related, and improvements or shortfalls in one practice  may contribute to improvements or shortfalls in another practice.  Establishing, documenting, and following practices—and their supporting  characteristics—in these four areas can contribute to a more effective  grants management framework. Since 2010, FRA has developed policies  and procedures in all four areas; however, it has not fully implemented  those policies and procedures. Table 2 shows our overall assessment of  FRA’s grants management activities compared to leading practices with  examples of selected characteristics. Appendix II provides greater detail  on our comparison of FRA’s approach with each supporting characteristic  for each leading practice.
subsections: section_title: FRA Does Not Have Performance Measures Directly Linked to Project Goals and Does Not Fully Evaluate Results of Monitoring Activities
paragraphs: An effective grants management framework includes establishing a  process that ensures project goals are identified, tracked, and fulfilled and  deliverables received. FRA partially follows the performance monitoring  leading practice area because while FRA developed a strategic vision for  the HSIPR program and outlined a monitoring process in its Grants  Manual, it does not have project goals or performance measures linked to  the grants funding the section 305 equipment procurement projects and it  does not fully evaluate the results of monitoring activities.
While FRA has developed a strategic vision for the HSIPR program— including promoting energy efficiency, environmental quality, and  economic competitiveness—the agency has not formalized project goals  for the two section 305 equipment procurement projects or performance  measures that clearly link to project goals. FRA officials we met with  stated that there is no stand-alone document outlining goals or  performance measures for the section 305 equipment procurement  projects, and that the officials measure project progress by tracking the  scope, schedule, and budget outlined in each grant agreement. However,  the agency has not formalized this approach to performance monitoring,  for example in its Grants Manual, or documented specific goals or  associated performance measures for the section 305 equipment  procurement projects in a way that would demonstrate the agency has a  process for comparing actual results against planned performance.  Without explicit project goals and associated performance measures, it  may be challenging for decision-makers to track and assess a project’s  progress, make decisions about future efforts, and keep grantees  accountable for outcomes.
While FRA outlines its monitoring procedures in the Grants Manual, we  found it does not fully evaluate and document results of monitoring  activities, including plans for corrective action. As discussed above, FRA  outlines a detailed process for routine and scheduled monitoring in its  Grants Manual. According to the Federal Standards for Internal Control,  management should evaluate the results of monitoring activities and  remediate identified internal control deficiencies on a timely basis. In  other words, management should determine the appropriate corrective  actions in a timely manner based on the identified deficiency. FRA noted  in a 2013 monitoring report for the bi-level car project that the schedule  showed that not all rail cars would be delivered by the grant agreement’s  expenditure deadline for FRA funding. While this was an issue that  directly affected the project’s scope and schedule, FRA did not identify it  as a significant finding that would require a corrective action plan. FRA  officials ultimately published a corrective action letter to lay out  expectations and discuss how to get the project back on schedule in  2015—2 years after the issue was identified in a monitoring report. The  DOT Inspector General reported in 2015 that corrective action plan  deadlines for several other HSIPR grants had passed without any  documentation that grantees took the necessary actions or staff extended  deadlines to complete the plans. According to FRA officials, they use  internal reports—separate from the routine and scheduled monitoring—to  inform agency management about any issues related to a project’s scope,  schedule, and budget. However, the procedures for and use of these  reports are not formally documented in the Grants Manual as a  mechanism for evaluating progress. Furthermore, it is unclear how the  internal reports relate to the monitoring and oversight approach FRA has  outlined in its Grants Manual or how they are used to hold grantees  accountable for potential risks to their grants. Leading practices  emphasize processes used to evaluate and monitor efforts should be  formalized and documented. Without evaluating and documenting the  results of monitoring activities in a timely manner grant managers may not  be able to address risks to a project’s completion, a situation that could  have a potential negative impact on the grant. This situation is particularly  important for the grants funding the section 305 equipment procurement  projects because most of the funding has an expenditure deadline of  September 30, 2017, and equipment purchases such as these represent  a new type of project for FRA.
section_title: FRA Developed Internal Documentation for Grants Management Policies and Procedures, but Has Not Developed Written Guidance For Grantees
paragraphs: An effective grants management framework includes developing and  maintaining written documentation as a means to obtain and retain  organizational knowledge and to ensure accountability for achieving  agreed-upon results. FRA partially follows the written documentation  leading practice area because while the agency has developed and  maintained written policies and procedures to communicate grants  management knowledge among its staff, it has not provided  documentation outlining grantee expectations or developed guidance  specific to the section 305 equipment procurement projects.
FRA has developed and maintained internal grants-management policies  and procedures to communicate knowledge among agency staff. For  example, FRA’s Grants Manual, the main policy document for agency  staff, describes monitoring and oversight activities that staff are expected  to conduct. The manual is periodically updated to reflect new federal  regulations, such as regulations issued under the Office of Management  and Budget’s new uniform guidance for federal awards, and changes in  process, such as a more detailed issue escalation process. FRA also  developed detailed Monitoring Procedures outlining FRA’s expectations  of MTAC contractors and provided guidance to MTAC on its oversight  responsibilities. For example, the monitoring procedure for oversight  reports outlines the purpose of recurring oversight, documents that MTAC  contractors must review, and the format and contents of the reports  MTAC contractors must submit. Finally, in late 2015, FRA began drafting  a Portfolio Management Guide with centralized guidance for regional  managers—intended to complement the Grants Manual—recognizing that  the Grants Manual was primarily written for grants managers.
Although FRA has developed and maintained internal grants  management policies and procedures, it has not outlined expectations or  developed guidance specific to the grants that fund the section 305  equipment procurement projects for grantees. For example, one grantee  stated that while the grant agreements include high-level expectations,  such as complying with federal regulations and meeting the NGEC  technical specifications, expectations specific to the section 305  equipment procurement projects were unclear. According to this grantee,  because FRA did not assign a central point of contact for the multi-state  projects, each state grantee had to coordinate separately through their  own grant manager, and expectations changed with each new state grant  manager. All three grantees we spoke with stated that written guidance or  procedures to manage the grants funding the section 305 procurement  projects would be helpful. In a 2011 monitoring report, one grantee  requested that FRA provide a public, written policy on receiving,  reviewing, and accepting deliverables because grantees did not know if or  when they would receive feedback from FRA and whether the  deliverables being submitted were in an acceptable format or of sufficient  quality. Those grantee officials told us that their project manager  informally uses the Federal Transit Administration’s (FTA) guidance in the  absence of guidance from FRA. In a 2014 monitoring report, one grantee  noted that circulars specific to the grant program would be helpful.  According to FRA officials we met with, the agency informs grantees of  expectations at a “kick-off meeting” when the grant is first awarded and  through routine monitoring and targeted training and technical assistance  related to items such as Buy America provisions or safety matters.  However, according to the U.S. Domestic Working Group’s Grant  Accountability Project, written policies serve as guidelines to ensure new  grant programs include provisions for holding grantees accountable for  properly using funds and achieving agreed-upon results. To date, FRA  has not developed written guidance on grants or project management  procedures for grants funding the section 305 equipment procurement  projects. According to the Federal Standards for Internal Control,  management should implement control activities through policies. For  example, management can document responsibilities through policies  and communicate policies and procedures so that personnel can  implement control activities for their assigned responsibilities. Other  agencies have developed detailed written guidance for grantees. For  example, FTA developed grants-management circulars on general  requirements as well as ones specific to each of its programs, which  include the documentation that FTA needs to review and requirements  associated with each program. In addition, FTA has developed guidance  specific to joint procurements of rail equipment. A lack of guidance  could result in FRA’s not receiving sufficient and necessary information  from its grantees to carry out its grant oversight activities.
section_title: Though FRA Identifies Training Needs, It Has Not Developed Training for Grantees and Agency Staff on Procedures Governing Grants That Fund the Section 305 Procurement Projects
paragraphs: An effective grants management framework includes a mechanism that  allows grant recipients and agency staff to establish and maintain a level  of subject-matter expertise and competence so that they can fulfill their  responsibilities. FRA partially follows the training leading practice area  because FRA identifies training needs for grantees and agency staff, but  it has not developed training on procedures governing the grants funding  the section 305 equipment procurement projects.
FRA identifies training needs of grantees and agency staff. FRA officials  stated that they identify grantee training needs formally through  scheduled monitoring reports, which specifically ask whether the grantee  needs any training or technical assistance. For example, we found that  grantees asked for training in 8 of 13 monitoring reports we reviewed, but  one grantee asked for training on the terms and conditions of its grant in  two consecutive reports. FRA officials stated that agency staff has the  opportunity to develop Individual Development Plans to identify training  needs for their current positions and development goals.
FRA has not developed training on procedures governing the grants  funding the section 305 equipment procurement projects for either  grantees or agency staff. According to the Federal Standards for Internal  Control, management should demonstrate a commitment to develop  competent individuals. For instance agencies can enable individuals to  develop competencies appropriate for key roles, reinforce standards of  conduct, and tailor training based on the needs of the role. FRA provided  grantees with a webinar called “Railroads 101,” which covered several  broad industry topics, such as train types and basic rail operations. While  the webinar provided to grantees gives a basic overview of the industry, it  does not include particular information related to the policies and  procedures governing HSIPR grant funds. For example, one grantee said  that webinar and in-person trainings on the procedures to manage the  grants would be helpful. According to FRA officials, all regional and grants  managers can receive project management training; however, new  employees often shadow current staff and use electronic systems to learn  about their assigned grants. In a 2014 monitoring report, one grantee  specifically stated that a training program similar to FTA’s would be very  beneficial for grantees. While agency staff can learn on the job,  inconsistent understanding of grant administration policies and  procedures across the agency could result in risks—such as those related  to scope, schedule or budget—not being identified in a timely manner  with potential negative impact on a project’s completion or the  management of grant funds. For example, the bi-level rail car project’s  schedule challenges were not formally reported as significant findings  requiring a corrective action plan until 2014, despite schedule concerns  noted in a 2013 monitoring report. DOT operating administrations, such  as FRA, have the option to determine grants training requirements though  we recognize that resources needed for training, such as budgets and  staff time, compete with other agency priorities. FRA officials stated that  in 2015 it began to develop e-training modules specific to roles within the  regional team structure, such as regional managers. While some recent  training efforts have been targeted to meet staff and grantee’s needs,  FRA officials acknowledged it is reasonable to formalize training efforts  going forward.
section_title: FRA Has Established Some Communication Mechanisms, but Lacks a Centralized System to Monitor Grants
paragraphs: An effective grants management framework includes establishing an  organizational structure, including well-defined reporting lines, to permit  the flow of quality information and to assist agency staff and grantees in  fulfilling their responsibilities. FRA partially follows the communication  leading practice area because, while FRA has developed mechanisms to  obtain relevant data based on project information, it does not have a  centralized system to monitor grant awards.
Since 2012 FRA has established mechanisms to obtain relevant data  based on project information requirements. For example, FRA’s most  recent Grants Manual outlines a specific process for grantees’ reporting  requirements using a table to outline each step of the process for  collecting, reviewing, and approving quarterly progress reports, including  the individual responsible for each step and the database or form that  should be used to complete the step. The previous version used a  narrative format that was less detailed.
We found FRA does not use a centralized system to monitor grant  awards. FRA uses multiple systems for grants management, including  GrantSolutions, Program Management Tracker, and Sharepoint.  GrantSolutions and Program Management Tracker are organized by  grant agreement number and are used to track grant administration  materials—such as financial reports–and program management  materials—such as final copies of monitoring and quarterly progress  reports—respectively. Since these systems organize information by the  grant agreement’s number and information is divided between different  systems, there is no central location for materials related to projects  funded through multiple grants, such as the section 305 equipment  procurement projects. According to the U.S. Domestic Working Group’s  Grant Accountability Project, a centralized information system can allow  grant management staff to track a grant’s status, tell how well a grantee is  performing, and keep track of problems. An FRA official stated that the  agency considered integrating the two systems to provide a central  system that combined grants and project management, but it was  ultimately deemed too expensive and technologically challenging.
section_title: Conclusions
paragraphs: It has been over 5 years since PRIIA significantly expanded FRA’s grant- making role. In that time, FRA has developed a new grants management  framework—a large undertaking encompassing all projects funded  through its grants—that involved concurrently hiring new staff, developing  grants oversight policies and procedures, and awarding grants. In  addition, the agency’s grants management role has included overseeing  the section 305 equipment procurement projects—a fundamentally new  type of project for states to lead, involving technically complex new  equipment that FRA had limited experience overseeing. In 2010, we  noted that a confluence of factors, including simultaneously carrying out  multiple new responsibilities, could pose risks for the use of federal funds  for high speed rail projects and that a robust grant-oversight program  would be a critical element to making sound federal investments in high  speed rail.
FRA’s grants management experience with the grants that fund the  section 305 equipment procurement projects demonstrates that additional  improvements to performance monitoring, written documentation, and  training could enable more effective grants management. A lack of  specific goals and measures can make it difficult to keep grantees  accountable and ensure that grantees are making progress toward  project deliverables. Further, evaluating and documenting the results of  monitoring activities in a timely manner would help FRA to more  proactively address risks to a project’s completion as they arise. Providing  written guidance to grantees on procedures and agency expectations  would help ensure that agency officials obtain the information they need  to fulfill their oversight responsibilities. Training can ensure consistent  understanding of policies to ensure project risks are identified in a timely  manner; however, we recognize that training requires decision makers to  make trade-offs based on funding and staff availability, as these  resources must be used to meet multiple agency priorities. While using a  centralized electronic-grant system could help agency staff better track  grant progress, FRA has explored this option and determined that it would  be too costly and technologically challenging.
The section 305 equipment procurement projects can be used to identify  lessons learned to strengthen FRA’s overall grants management  framework and project oversight for future grants. While no new section  305 equipment procurement projects are currently planned, FRA  manages and oversees approximately 200 ongoing grant projects and the  Passenger Rail Investment and Reform Act of 2015, passed as a title in  the FAST Act, authorized a new infrastructure and safety grant program  to assist grantees in financing the cost of improving passenger and freight  rail transportation systems. While FRA officials began efforts to further  formalize grants management procedures during the course of our  review—such as drafting a portfolio management guide to centralize  guidance related to the regional manager role for project oversight— additional steps could further enhance the agency’s approach to grants  management. Improving the agency’s processes to be more proactive in  overseeing and monitoring grant performance would help minimize  project risk, increase grantee accountability, and improve the efficient and  effective use of federal funds for FRA’s grants portfolio.
section_title: Recommendations for Executive Action
paragraphs: To strengthen FRA’s grants management practices, we recommend the  Secretary of Transportation direct the FRA Administrator to take the  following actions:  1.  Enhance the process outlined in the Grants Manual to monitor project  performance for future grants to include: (1) performance measures  directly linked to project goals, and (2) fully incorporating timely and  actionable information on grantee performance into FRA’s review  process to help determine whether current efforts are in line with the  overall project goals.  2.  Develop and provide written guidance to grantees to include FRA’s  expectations on the type of information grantees should provide, such  as guidance specific to deliverables and milestones for each grant  project.  3.  Analyze training needs and formalize a training plan for grantees and  agency staff, which could include training on grant-specific procedures  and policies.
section_title: Agency Comments
paragraphs: We provided a draft of this report to Amtrak and the Department of  Transportation for review and comment. In written comments, reproduced  in appendix III, DOT concurred with GAO’s recommendations. FRA also  provided technical comments that were incorporated, as appropriate.  Amtrak did not provide comments.
We are sending copies of this report to the Secretary of Transportation,  the Administrator of the Federal Railroad Administration, and Amtrak. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-2834 or flemings@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report addresses the following questions:  1)  How has FRA carried out its grants management roles and  responsibilities regarding the PRIIA section 305 equipment  procurement projects?  2)  To what extent has FRA’s approach to grants management for the  PRIIA section 305 equipment procurements met leading practices and  whether FRA’s grants management practices could be improved?
To determine how FRA has carried out its grants management roles and  responsibilities for the grants funding the section 305 equipment  procurement projects, we reviewed FRA policies and other guidance  established to outline the agency’s responsibilities, such as the August  2015 Grants Management Manual and the 2013 Program Management  Plan. We reviewed the terms and conditions of the six grant agreements  FRA executed with the California Department of Transportation  (Caltrans), the Illinois Department of Transportation (IDOT), and the  Washington State Department of Transportation (WSDOT) that fund the  section 305 equipment procurements to identify project milestones and  deliverables, as well as specific activities or points in the administration of  the grants where FRA reviews, approves, or concurs with the state  grantees on specific project activities. While three of these grants  covered broader corridor improvements beyond the equipment projects  (Chicago-St. Louis Corridor Improvement, the Chicago-Quad Cities  Expansion Program, and the Pacific Northwest Rail Corridor Program),  we focused on grant administration and oversight activities related to the  tasks associated with the equipment procurements. We did not assess  FRA’s grant management activities beyond those applicable to these six  grant awards.
We reviewed additional documentation related to FRA’s oversight of  these six selected grants, including 96 quarterly progress reports  provided by FRA for fiscal years 2013 through 2015 and 13 monitoring  reports completed by FRA for calendar years 2011 through 2015 to better  understand how FRA carries out its grants management roles and  responsibilities. We focused our review on FRA’s grants management  activities post-award, including monitoring of grant awards, project  oversight, and technical assistance activities. We did not examine the  agency’s award issuance or grant closeout processes, as the equipment  projects are ongoing at the time of this report’s issuance. We reviewed  additional policy and guidance documentation related to the oversight of  the section 305 equipment procurement projects, including the relevant  Notices of Funding Availability issued by FRA, FRA’s Monitoring  Procedures describing the oversight conducted by the Monitoring and  Technical Assistance Contractors (MTAC), and the MTAC task order  outlining specific tasks and technical assistance contractors may perform  for the equipment projects. We also reviewed reports independent  consultants provided to FRA, such as reports on the bi-level car project’s  status and routine summaries of project-level activities and meetings  between state grantees, equipment vendors, and subject matter experts.  To collect additional information about how FRA carried out grants  management activities for the locomotive and bi-level car equipment  procurements, we interviewed FRA Regional Managers and Grant  Managers within the Office of Railroad Policy and Development, subject  matter experts within other FRA offices such as the Office of Research,  Development and Technology, and the Office of Chief Counsel, and  MTAC contractors supporting the agency’s oversight activities. We also  interviewed Amtrak officials, California, Illinois, and Washington State  departments of transportation, as well as Next Generation Equipment  Committee (NGEC) participants and independent consultants involved  with the section 305 equipment procurement projects.
To assess the extent to which FRA’s approach to grants management for  the equipment procurements met leading practices, we identified relevant  and applicable leading practices and supporting characteristics that  contribute to those practices, using generally accepted grants  management practices from a variety of sources (see table 3).
Using these sources, we identified four leading grants management  practices—communication, written documentation, training, and  performance monitoring—as well as specific characteristics of grants  management that support these practices. To assess FRA’s grants  management, we reviewed grant documentation and FRA’s grants  management plans, policies, and procedures to determine the extent to  which FRA’s practices aligned with the supporting characteristics of our  leading practices. Each leading practice is aligned with a few supporting  characteristics. For example, supporting characteristics for the written  documentation practice include: (1) develop and maintain written policies  and procedures that communicate knowledge among agency staff, and  (2) develop guidance specific to each grant program, including  documentation outlining agency and grantee expectations, among others.  For additional information on the leading practices and supporting  characteristics, see appendix II. We used that information in aggregate to  determine the extent to which the leading practice was followed. For  example, we reviewed grant documentation, such as quarterly progress  reports submitted by grantees and monitoring reports completed by FRA  officials for the section 305 equipment procurements from calendar year  2011 through 2015. We also reviewed FRA’s grants management policies  and procedures, such as those outlined in the Grants Management  Manual and Monitoring Procedures to determine the extent to which the  activities and processes described met the supporting characteristics. We  also interviewed officials at FRA and the California, Illinois, and  Washington State departments of transportation as well as FRA’s  independent contractors. Our assessment of the alignment of FRA’s  practices with the supporting characteristics served as the basis for our  overall assessment as to whether each leading practice was followed or  substantially followed; partially followed; or minimally or not followed. For  example, if we found supporting evidence that two of the three  characteristics of a practice were substantially followed but the third  characteristic was not followed, we determined that the leading practice  was partially followed. The criteria to determine if practices and  supporting characteristics were:  followed or substantially followed—plans, policies, or processes have  been developed and implemented properly for all or nearly all  supporting characteristics partially followed—plans, policies, or processes have been developed  and implemented properly for some supporting characteristics minimally or not followed—plans, policies, or processes are lacking for  all or nearly all supporting characteristics  To further determine whether FRA’s grants management practices could  be improved, we interviewed FRA and Caltrans, IDOT, and WSDOT  officials, NGEC participants, and independent contractors to obtain  perspectives on lessons learned from the projects to date, as well as  examples of what worked well with the FRA’s management and oversight  of the section 305 equipment procurements.
We conducted this performance audit from June 2015 to May 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Grants Management Leading Practices with GAO Assessments
paragraphs: We assessed the extent to which the Federal Railroad Administration’s  (FRA) approach to grants management for the grants funding the section  305 equipment procurement projects met leading practices and  supporting characteristics for grants management that we identified in the  areas of performance monitoring, written documentation, training, and  communication. We reviewed grant documentation and FRA’s plans,  policies, and procedures and analyzed interviews with FRA and grantee  officials as well as Monitoring and Technical Assistance Contractors  (MTAC). Based on our analysis, we determined the extent to which each  characteristic was followed or substantially followed; partially followed; or  minimally or not followed. Our assessment of the characteristics served  as the basis for our overall assessment on the extent to which each  leading practice was followed. We categorized the assessments using the  scale below:
Followed or substantially followed the leading practice—plans,  policies, or processes have been developed and implemented  properly for all or nearly all supporting characteristics.
Partially followed the leading practice—plans, policies, or processes  have been developed and implemented properly for some supporting  characteristics.
Minimally or did not follow the leading practice—plans, policies, or  processes are lacking for all or nearly all supporting characteristics.
Table 4 provides greater detail, including examples, of our comparison of  FRA’s grants management approach with the supporting characteristics  that are aligned with leading practices.
section_title: Appendix III: Comments from the Department of Transportation
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, the following individuals made  important contributions to this report: Melissa Bodeau, Steve Cohen,  Swati Deo, Derry Henrick, SaraAnn Moessbauer, Malika Rice, Maria  Wallace, and Crystal Wesco.