Public Transportation: FTA's Triennial Review Program Has Improved, But Assessments of Grantees' Performance Could Be Enhanced
Published: 2009-06-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Federal Transit Administration (FTA) oversees about $5.5 billion in federal funds each year to transit agencies serving urban areas (grantee), in part through its triennial review program, which evaluates grantee adherence to federal requirements at least once every 3 years. GAO recommended in a 1998 oversight report that FTA improve the program. The subcommittee requested that GAO review this program. GAO identified (1) the extent to which triennial reviews indicate that grantees met applicable federal requirements from fiscal years 2000 through 2008; (2) the strengths and weaknesses of the triennial review process; and (3) FTA's performance measures for the triennial review and the extent to which they meet key attributes of successful performance measures. GAO addressed these objectives by analyzing oversight data on 424 grantees that had three triennial reviews, reviewing triennial review reports and guidance, assessing FTA's performance measures; and interviewing FTA headquarters and regional officials, contractors who conduct the reviews, and grantees.
section_title: What GAO Found
paragraphs: GAO's analysis of FTA's triennial review oversight data found that over two-thirds of the 424 grantees analyzed have not consistently improved overall performance in terms of meeting more federal requirements from fiscal years 2000 through 2008. Fifty-one percent of grantees had mixed results in meeting requirements and 17 percent consistently met fewer requirements; while 31 percent consistently met more requirements--one of the goals of the triennial review program. Executives from three grantees that met most requirements attributed their performance to, among other things, having job descriptions that link employee responsibilities to the triennial review--a practice they said contributed to a culture of accountability. During the same time, grantees had the greatest number of findings in 5 of 23 triennial review areas, including the procurement and drug and alcohol testing areas. While FTA helps grantees address findings, additional efforts to identify the underlying causes and the severity of findings could further benefit grantees. FTA's triennial review program uses some strong management practices--having a well-defined process, using an information system to monitor grantees, and issuing reports timely. Still, two areas could be strengthened. First, while FTA is legislatively required to conduct a complete review of grantees' adherence to federal requirements at least once every 3 years, GAO identified a few instances where documentation does not clearly show that FTA reviewed all requirement areas. For example, 10 triennial review reports for 2008 showed that the drug and alcohol program area was "not reviewed." FTA's practice is to review all areas, regardless of documentation, but because FTA's guidance is not clear about how to document the review of areas where FTA has conducted a related special review in the prior two years, a few grantees may not be reviewed for 5 years. FTA plans to revise its guidance to avoid ambiguity. Second, FTA is aware of the burden oversight reviews place on grantees and works to limit this burden. However, in a limited number of cases, FTA did not coordinate its special oversight reviews with the triennial review schedule, which may place undue burden on a few grantees receiving multiple oversight reviews in the same fiscal year. FTA's two timeliness performance measures for assessing the triennial review program--(1) closing 80 percent of grantees' deficient findings within 30 days of their due date and (2) issuing 95 percent of the final triennial review reports within 30 days of completing a review--meet some, but not all key attributes of successful measures. Although both measures link throughout the organization, have measurable targets, are clearly stated, and do not overlap, the "close findings" measure does not meet the objectivity and reliability attribute. For example, data inaccuracies in past "close findings" data raised questions about the reliability of the measure. Also, both measures do not assess the core program activity to evaluate grantees' performance or governmentwide priorities, such as the quality of the triennial review program, and thus, as a whole, are not balanced, making it difficult for managers to not overemphasize one priority at the expense of others.

section_title: Letter
paragraphs: Public transportation systems are playing an increasingly important role in  the nation’s overall transportation system. Public transit increases  mobility for millions of Americans in communities both large and small,  provides congestion relief, promotes safe travel, and helps reduce our  country’s dependence on foreign oil. The Department of Transportation’s  Federal Transit Administration (FTA) fosters the development and  maintenance of bus, subway, trolley, passenger ferry boat, and other  public transportation systems through several grant programs to transit  agencies serving both rural (population less than 50,000) and urban  communities (population 50,000 or more). A large portion of FTA’s grants  are formula grants to transit agencies in urban areas across the country.  This Urbanized Area Formula Program received 42 percent of FTA grant  funds—about $22 billion from fiscal years 2004 through 2009. In addition,  Congress recently appropriated an additional $6 billion for the Urbanized  Area Formula Program in the American Recovery and Reinvestment Act of  2009.
Federal law requires the Secretary of Transportation to review and  evaluate completely the performance of each recipient of Urbanized Area  Formula Program grants (grantee) that receives urbanized area formula  funds under this section in complying with statutory and administrative  requirements, at least once every 3 years. This evaluation—commonly  known as the “triennial review”—is one of the primary means FTA uses to  evaluate to what extent Urbanized Area Formula Program grantees are  meeting federal requirements. Grantees must comply with a variety of  requirements to safeguard federal funds and help ensure safe  transportation, among other things.
As the subcommittee prepares for the 2009 surface transportation  reauthorization, we assessed, at your request, FTA’s triennial review  program. Specifically, we identified (1) to what extent FTA’s triennial  reviews indicate that grantees met applicable federal requirements from  fiscal years 2000 through 2008, (2) the strengths and weaknesses of the  triennial review process, and (3) FTA’s performance measures for the  triennial review and to what extent FTA’s measures meet key attributes of  successful performance measures.
To identify the extent to which triennial reviews indicate that grantees are  meeting applicable federal requirements from fiscal years 2000 through  2008, we obtained and analyzed data from FTA’s Oversight Tracking  System (OTRAK)—the official electronic information system used for  tracking and monitoring oversight activities. Specifically, we conducted  various analyses of oversight data on 424 of the approximately 600  urbanized area formula fund grantees. These 424 grantees received exactly  3 triennial reviews from fiscal years 2000 through 2008—the period of time  for which OTRAK data was available. We found most OTRAK data  sufficiently reliable for our purposes, but have reliability concerns about  closed findings data used for performance measurements, which we  discuss later in this report. To identify the strengths and weaknesses of the  triennial review process, we reviewed federal guidance for improving  grant accountability, previous GAO reports on oversight, FTA’s oversight  review order, the Triennial Review Contractors’ Guide, oversight review  schedules, and the contents of over 200 triennial review reports for fiscal  years 2007 and 2008. We also attended FTA’s new grantee triennial review  workshop and observed one of FTA’s 2008 triennial reviews. To identify  performance measures for the triennial review and the extent to which  they meet key attributes of successful performance measures, we  reviewed the key attributes of successful performance measures, as  described in our past work on the Government Performance and Results  Act of 1993 and FTA’s strategic and annual performance plans.  Specifically, we compared each of FTA’s performance measures against  each of the key attributes to determine the extent to which they fully met,  partially met, or did not meet all key attributes. Finally, we interviewed  FTA headquarters officials, staff in all 10 FTA regional offices,  representatives of the four contracting firms that conduct the triennial  reviews, and professional transportation associations. We also selected 10  grantees and interviewed executives from these grantees to discuss their  performance, as assessed by the triennial review. We selected these  grantees on several factors, including their past performance, as assessed  by triennial reviews; having both a triennial review and another FTA  oversight review in the same fiscal year; and demographic features, such  as size and geographic region.
We conducted this performance audit from June 2008 to June 2009 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. Further details about our objectives, scope,  and methodology appear in appendix I.
subsections: section_title: Background
paragraphs: As one of 10 administrations within the Department of Transportation,  FTA provides financial assistance to transit agencies to develop new  systems and improve, maintain, and operate existing systems in all 50  states, the District of Columbia, and some U.S. territories. FTA maintains  its headquarters in Washington, D.C., with 10 regional offices throughout  the continental United States, and employs about 500 of the approximately  60,000 employees of the department. Figure 1 summarizes key information  about FTA’s 10 regions, including the total amount and funds for the  Urbanized Area Formula Program.
In 1994, FTA laid out a common framework for regional and headquarters  staff to administer oversight reviews in FTA Order 5400.1. That order,  which remains in effect today, describes the process for conducting  various oversight reviews and taking corrective actions. In addition, the  order established an internal working group called the Oversight Review  Council to develop oversight review schedules, coordinate review  schedules to avoid unnecessary duplication and overlap, and achieve a  comprehensive, uniform, and cohesive approach, among other things.
The triennial review provides FTA with a valuable opportunity to routinely  evaluate the use of federal funds for about 600 urban area grantees at least  once every three years—or about 200 grantees each year. It covers many  areas, such as procurement, financial management, and drug and alcohol  programs. Appendix II contains FTA’s descriptions of all 23 triennial  review areas. FTA also uses a similar, general review to oversee grantees  that receive rural area formula funds—a state management review.
In addition to the more general triennial and state management reviews,  FTA conducts special oversight reviews with a narrowed subject matter  focus, such as procurement system reviews, which help ensure that  grantees (especially 30 grantees that receive large amounts of federal  transit funds) are purchasing buses and trains using competitive bidding  practices, as well as special civil rights reviews, which help ensure that  grantees are, among other things, providing public transportation to the  disabled and adhering to equal employment opportunity practices.  Grantees also experience non-FTA reviews, such as single audits that  examine grantees’ financial statements, inspector general audits, and  various state audits. Table 1 describes FTA’s primary oversight reviews.
The triennial review covers 23 triennial review areas, but given the need to  review all of these areas, the time constraints (estimated at about 130  hours for each review), and in order to reduce the oversight burden upon  grantees, FTA does not conduct an in-depth document review or  inspection. For example, FTA normally samples no more than 15 financial  records during site visits, conducts a spot check of facility and  maintenance records, and may only inspect a few buses and subway cars  during the review. However, according to FTA officials, when an area of  concern or deficiency is noted, FTA will expand the review to examine  these areas more in-depth. Figure 2 summarizes the major steps in the  triennial review process.
The triennial review also gives FTA an opportunity to provide technical  assistance to individual grantees—assistance specifically aimed to help  individual grantees resolve specific deficient findings resulting from  triennial reviews. FTA also helps grantees prepare for the triennial review  by holding annual triennial review workshops throughout the country and  offering formal training courses focused on specialized subject areas  through the National Transit Institute—a training institute funded by an  FTA grant.
While officials from both FTA headquarters and regional offices are  ultimately responsible for implementing the triennial reviews, they have  increasingly relied on contractors since the program was created in the  early 1980s. After we reported in 1992 that FTA had not fully utilized  contractors for oversight activities, it has, since 1996, used contractors to  conduct all triennial reviews. FTA, through its Office of Program  Management, awarded multiple-year, firm, fixed-price contracts totaling  about $4 million each year to four contractors to conduct the fiscal years’  2008 through 2012 triennial reviews.
To assist contractors with the reviews, FTA developed the Triennial  Review Contractors’ Guide. FTA has expanded the contractors’ guide  from its original 29 pages to over 200 pages in 2008. The contractors’ guide  provides the questions to ask grantees for each of the 23 triennial review  areas, a list of grantees’ documents that contractors are to examine, and a  method that contractors are to use for determining whether grantees are  meeting federal requirements.
The contractors’ guide assists contractors with the decision-making  process about what constitutes a deficient finding by providing condition  statements for the 23 triennial review areas. For example, according to the  contractors’ guide, if the grantee has included the lobbying prohibitions  clause (that is, the prohibition from using federal funds to lobby for  additional federal funds) in its procurement solicitations, then the grantee  is not deficient; otherwise, the grantee is deficient. The contractors’ guide  also contains suggested corrective actions if a grantee is found deficient,  (e.g., the grantee must certify to FTA that it will have subgrantees,  contractors, and subcontractors comply with the lobbying prohibitions  requirement.) In another example, if FTA conducted a special  procurement system review (see table 1) in the prior two fiscal years or if  one is scheduled for the year of the grantee’s triennial review, then a  review of the procurement area is not necessary and a notation is made in  the triennial review report as “not reviewed.” Subsequently, contractors  enter predetermined, deficiency codes into OTRAK that correspond to  explanations for any deficient findings, herein called findings. Once  entered, FTA regional offices track and monitor the finding until the  grantee addresses it. Table 2 shows the four categories FTA uses to  document triennial review results.
section_title: FTA’s Triennial Review Data Show Many Grantees Are Not Making Progress Toward Meeting Federal Requirements, and FTA Could Do More to Improve Grantees’ Performance
paragraphs: From fiscal years 2000 through 2008, triennial review data show many  grantees are not making progress toward meeting federal requirements, as  indicated by not consistently reducing grantees’ findings with federal  requirements. While FTA uses several tools to help grantees meet more  requirements, especially in areas with the greatest number of findings, it  could do more to improve grantees’ performance by identifying the  underlying causes and by objectively noting the severity of the finding.
subsections: section_title: Over Two-Thirds of the 424 Grantees We Analyzed Did Not Consistently Meet More Federal Requirements From Fiscal Years 2000 through 2008, But Practices Used by Some Grantees Could Improve Other Grantees’ Performance
paragraphs: Our analysis of OTRAK data shows that 51 percent of 424 grantees had  mixed results in terms of meeting more triennial review requirements from  fiscal years 2000 through 2008, as demonstrated by both increases and  decreases in their numbers of findings over three consecutive reviews.  Over the same time period, 17 percent of grantees consistently met fewer  requirements from one review to the next, as indicated by a consistent  increase in the numbers of findings in their triennial reviews. Thirty-one  percent of grantees consistently met more federal requirements over  time—one of the goals of the triennial review—as demonstrated by a  consistent decrease in their numbers of findings over three consecutive  reviews. Finally, 1 percent of grantees met the same number of  requirements in all three reviews. Figure 3 summarizes the performance in  terms of meeting federal requirements for 424 of the approximately 600  current urban area formula fund grantees, as measured by the change in  the number of findings in three triennial reviews from fiscal years 2000   fiscal years 2000  through 2008.  through 2008.
While comparing the change in the numbers of findings over time does not  show improvement across all grantees, a different analysis of OTRAK data  shows that some grantees averaged fewer than two findings per triennial  review, which we identify for this analysis as high-performing grantees.  Figure 4 illustrates the range in the average number of findings after three  triennial reviews for each of the 424 grantees from fiscal years 2000  s 2000  through 2008.  through 2008.
This analysis of OTRAK data shows that about 31 of 424 grantees averaged  fewer than 2 of 229 possible findings per triennial review. Twenty-six of  these grantees provide transportation services to small or medium-sized  urban communities, located in various parts of the country.
Three of the 10 grantees we interviewed were high-performing grantees.  Executives representing these three grantees implemented some  management practices that they believe contributed to their high  performances. One management practice included linking the job  descriptions of pertinent individuals to the triennial review—a practice  that grantee executives said contributed to a culture of accountability.  Specifically, the controller’s and grants administrator’s job descriptions for  the large grantee explicitly required these individuals to be responsible for  the triennial review. An assistant director of public works, responsible  for managing the transit system of a small grantee, was required to help  implement FTA requirements, among other things. In addition, according  to the chief executive officer of the medium grantee, the officer set verbal  job expectations with managers of zero findings, which the officer  believed contributed to the high performance in the triennial review.
Another practice implemented by 2 of the 3 high-performing grantees was  to initiate external reviews by independent consultants to assess their  operations, which contributed to, or is expected to sustain, their high  performance. Executives from a large grantee we interviewed attributed  their high performance to an external review conducted several years ago  that focused on the grantee’s readiness for the triennial review. Despite  having zero findings in their last two consecutive triennial reviews,  executives from the medium grantee recently completed an external  maintenance review that they believe should help them achieve their goal  of operating an efficient transit system and that is necessary to remain  good stewards of public funding.
FTA officials recognize the importance of sharing best practices among  grantees. As such, it advocates a grantee mentoring system where regional  offices, within resource and budget restraints, initiate, encourage and  direct grantees to team with recognized grantee leaders in order to  improve grantee performance. FTA officials also indicated that they  address mentoring during their triennial review workshops.
section_title: FTA Helps Grantees Meet More Requirements, Especially in Five Review Areas with Most Findings; However, FTA Does Not Identify the Underlying Causes of Findings
paragraphs: When we analyzed OTRAK data from fiscal years 2000 through 2008 by  area, we found that the 424 grantees had the greatest numbers of findings  in five review areas—procurement, drug and alcohol program,  maintenance, technical, and satisfactory continuing control areas. These  five areas accounted for about 54 percent of all findings over the last nine  years. This may not be unexpected since we also found a positive  relationship between the number of questions asked for certain areas and  the number of findings. Specifically, grantees tended to have higher  numbers of findings in those areas where the contractors’ guide requires  contractors to ask more questions when compared to other areas. For  example, for fiscal year 2008, contractors asked grantees 20 questions in  the procurement area, as compared to 2 or 3 questions in other areas, such  as school bus and half-fare, which had far fewer findings. Figure 5 shows  the percentage of triennial review findings for the top 5 areas, compared to  the other 18 areas for the same 424 grantees over the nine-year period.
FTA has also analyzed OTRAK data on past triennial reviews, identified  similar areas with the greatest number of findings, and provided resources  at a national level to help grantees address the findings in these areas.  According to several end of year reports, FTA analysis of deficiency codes  related to the procurement area showed grantees lacking justification for  non-competitive awards, lacking open competition, having no contract  administration system, and having no written record of procurement  history. For the drug and alcohol area, deficiency codes pointed to  grantees having policies that were out-of-date or lacked required elements,  and conducting random drug and alcohol tests below the federally  required level. To help grantees improve their performance in areas with  the greatest number of findings, FTA has provided an increasing amount  of technical assistance to individual grantees; workshops that focus on,  among other things, procurement related matters; and training. In its 2003  end of year report, FTA noted that a decline in findings from fiscal year  2000 to 2003 indicated the effectiveness of its efforts to improve grantees’  performance.
We could not definitively determine whether, or to what degree, FTA’s  technical assistance, workshops, and training improved grantees’  performance because while workshops increased from 2005 to 2008, so  did the total number of findings. Other factors could have contributed to a  reduction of grantees’ findings over the last nine years. The reduction in  findings could have been caused by contractors reporting that they did not  review certain areas for some grantees that had recently experienced a  special oversight review or by FTA conducting a pilot study in 2005 to  streamline the triennial review process, which also resulted in some areas  not being completely reviewed. Further, grantees, as well as FTA  headquarters and regional officials, cite a shift over the years in how FTA  conducts the triennial reviews—from a “gotcha” auditing approach to a  customer-service approach, helping grantees quickly resolve findings  during the reviews and lessening the need to report the findings.
Although FTA analyzes deficiency codes in OTRAK data to determine  problem areas, these codes alone do not describe the underlying causes  for the findings. Generally accepted government auditing standards  recommend that federal agencies, such as FTA, identify, to the extent  possible, the underlying cause in developing findings as this provides  persuasive evidence on the factor(s) responsible for the problem.  Deficiency codes pointing to grantees lacking open competition or not  having a written record of their procurement history, for example, is not  an underlying cause, as grantees’ ability to meet procurement  requirements may be caused by grantees not having sufficient or  knowledgeable staff to initiate a competitive contracting process. If FTA  would identify these as common underlying causes, it could prescribe, as  part of the necessary corrective action, that grantees attend specific  procurement training courses. Knowing the common underlying causes of  grantees’ findings could provide FTA with valuable information to help  many grantees find solutions to correct their findings and improve  grantees’ long-term performance. FTA officials stated that during the  workshops, they plan to emphasize ways to identify systemic deficiencies.
Officials expressed concern about their authority to use oversight funds  for the purpose of undertaking additional efforts to help the grantees meet  or exceed requirements by identifying the common underlying causes for  less than satisfactory levels of performance. Officials told us that they  have the authority to use oversight funds for providing individual grantees  with technical assistance—assistance necessary to satisfactorily address  specific findings resulting from the grantee’s triennial review. However,  they acknowledged that they can help grantees develop capabilities to  improve their long-term performance in other ways, such as National  Transit Institute courses, that do not require using oversight funds.
section_title: FTA Plans to Improve the Data It Collects on the Severity of Findings
paragraphs: While FTA collects data on the numbers, the areas, and the reasons for the  findings by the deficiency codes in OTRAK, the information it collects on  severity of findings is not objective. OTRAK requires that contractors and  FTA regional staff enter a level of severity—”high,” “medium,” and “low”— for each finding. However, individuals entering severity data into OTRAK  labeled almost 98 percent of findings as medium, which makes it difficult  for FTA to prioritize technical assistance in problem areas. According to  FTA officials, contractors or regional staff made judgments about the level  of severity because FTA has not developed definitions from which to make  this judgment. This could mean the information on severity is not  consistent across the country. FTA officials said that they plan to improve  their data collection efforts for the fiscal year 2010 triennial reviews by  establishing definitions for the three levels of severity, which could  provide more meaningful findings data and add significant value to the  information FTA collects to target its technical assistance for the purposes  of improving grantees’ performance.
section_title: The Triennial Review Program Uses Some Strong Management Practices, but Can Be Improved in a Few Areas
paragraphs: Strengths of the triennial review include a well-defined process, the  increased use of an electronic tracking system to monitor grantees’  performance, increased timeliness in issuing triennial review reports, and  the willingness of FTA staff to seek out opportunities for continuous  improvement. However, we found a few areas where FTA could improve  the triennial review process by ensuring and documenting grantees receive  a complete review at least once every three years and by coordinating the  schedules of multiple oversight reviews in the same fiscal year.
subsections: section_title: The Triennial Review Program Has A Well- Defined Process and Continues to Improve
paragraphs: One strength of the triennial review program is that the process is well  defined. FTA Order 5400.1 and the Triennial Review Contractors’ Guide  detail the roles and responsibilities for those involved in the triennial  review, including FTA headquarters, regional offices, contractors, and  grantees. These documents outline the process, identify the questions that  contractors are to ask grantees, specify the types of documents that  grantees are to provide to FTA, and describe any follow up required of  regional staff. Preparing policies and procedures that outline individual  responsibilities meets an important element of strong federal grant  accountability best practices.
Another strength is an increased use of an information system to track and  monitor grantees. In 1998, we criticized FTA for not making full use of its  information system because its regional staff had not been uniformly  updating the system and its headquarters staff was not enforcing its use as  a management tool. FTA upgraded OTRAK in 2003 from a local-area  computer network to a Web-based system, which allows headquarters and  regions to share information and track oversight activities more efficiently.  FTA has continued to incrementally update this system. In addition, the  contractors’ guide directs contractors to input the findings from each  review into OTRAK and all regional staff and contractors we spoke with  reported using OTRAK. Performance information, such as that recorded  in OTRAK, helps agencies identify, prioritize, and manage potential at-risk  grantees.
FTA has strengthened its triennial review reporting process by issuing  more timely triennial review reports than in previous years. In 1998, we  criticized FTA for taking an average of 182 days after the site visit to  process its 1995 and 1996 triennial review reports and recommended that  FTA follow its established 90-day time frame. FTA subsequently  established a performance measure in 2001 of issuing 95 percent of  triennial review reports within 30 days of completing a review. FTA has  significantly improved its performance in issuing reports in recent years.  FTA reported it issued between 82 to 94 percent of triennial review reports  within 30 days of completing the review from fiscal year 2004 through  2008. During that same time period, only 2 of 902 reports were issued more  than 60 days after the review.
Finally, FTA has looked for ways to continuously improve by contracting  with Booz Allen Hamilton to examine its oversight review processes and  recommend ways to improve them. In January 2008, Booz Allen Hamilton  issued its final report, outlining four areas of improvement: program  planning, program execution, program evaluation, and change  management. We found that FTA is taking steps to respond to  recommendations in these areas, including developing a strategic  approach for prioritizing each of the Booz Allen Hamilton  recommendations and developing a Web-based, master calendar feature in  OTRAK that is expected to improve the visibility of future oversight review  schedules to assist both FTA headquarters and regional staff in  coordinating the oversight review schedules.
section_title: FTA Could Better Ensure and Document that Grantees Receive a Complete Review at Least Once Every Three Years
paragraphs: FTA is required to conduct a complete review and evaluation of grantees’  adherence with statutory and administrative requirements at least once  every three years. In reviewing 198 fiscal year 2008 triennial review  reports, we identified, but could not confirm, 33 instances where FTA may  not have completely reviewed and evaluated grantees in all areas.  Specifically, contractors recorded in the 2008 triennial review reports that  certain areas were “not reviewed” because FTA had conducted special  oversight reviews on these areas in the prior two years—fiscal years 2006  or 2007. For example, contractors recorded in the triennial review reports  of ten grantees that the drug and alcohol program area was “not reviewed”  during the 2008 triennial review because these grantees had received  special drug and alcohol program reviews in fiscal year 2006. Because the  area was not reviewed during the 2008 triennial review, these grantees  may not be reviewed in this area for 5 years (this area was last reviewed  during the 2006 special oversight review and will not be reviewed again  until the 2011 triennial review). While FTA’s contractors’ guide describes  the flexibility to not review certain areas, FTA regional office staff  provides triennial review contractors with the approval to not review  certain areas when a special oversight review has been conducted in the  prior two years or is scheduled in the fiscal year of the triennial review.
FTA officials indicate that their practice is to update its review and  evaluation of an area having an oversight review in the prior two years.  However, the guidance does not always instruct contractors to document  this update in the triennial review reports to help FTA ensure it meets the  statutory requirement of a complete review at least once every three years.  As an example, the contractors’ guide describes such a process for  reviewing and evaluating the planning area, telling contractors to review  previous planning certification review reports during the triennial review  desk review, verify the status of any findings, and determine if corrective  actions have been implemented on schedule. The contractors’ guide does  not provide similar guidance for other areas, including the drug and  alcohol program area, and we did not see—nor could FTA provide us  with— any such documentation in the 2008 triennial review reports. In  commenting on our draft report, FTA said it would revise applicable  guidance, including the contractors’ guide for fiscal year 2010, to avoid  ambiguity and ensure consistency within the triennial review program.
FTA headquarters officials indicated that regional officials and contractors  discuss all 23 areas during the triennial review process and identify  concerns about the grantees. Specifically, headquarters officials explained  that regional staff and contractors discuss the results of special oversight  reviews conducted in the prior two years, such as any outstanding findings  from the prior reviews, and reach consensus on the extent to which a  grantee would be reviewed in the related area during the triennial review,  ranging from an area not being reviewed at all to a more detailed review in  the area, depending on the situation. However, officials acknowledged that  the results of those discussions are not always documented in the triennial  review reports, especially in regards to the extent to which areas are to be  reviewed or not reviewed during the triennial review. Because of this lack  of documentation, we could not confirm in the 33 instances noted above  whether or to what extent all 23 areas were reviewed and evaluated.
By allowing flexibility in the contractors’ guide to not review certain areas,  FTA may not be ensuring that all areas are completely reviewed and  evaluated at least once every three years, as required by statute. In  addition, because grantees can and do experience findings during triennial  reviews, despite having special oversight reviews in the prior two years,  updating previously conducted special oversight reviews is important. We  identified seven instances where grantees had received special oversight  reviews in the financial management and civil rights areas in fiscal years  2006 and 2007, yet during the 2008 triennial review, findings in these areas  resurfaced. Finally, proper evaluation and documentation provides future  reviewers with a more transparent audit trail of findings and gives FTA  managers the confidence that grantees were completely reviewed and  evaluated in all 23 areas at least once every three years. Officials plan to  refine and use OTRAK to document, among other things, the results of  special oversight reviews that occurred in the years between triennial  reviews, which would help ensure grantees adhere to requirements.
section_title: Uncoordinated Scheduling of Multiple Oversight Reviews May Unduly Burden Some Grantees
paragraphs: We found a limited number of instances in which grantees may have been  unduly burdened by uncoordinated scheduling of multiple oversight  reviews in the same fiscal year. We found that 23 of 198 grantees (12  percent) receiving a triennial review in fiscal year 2008 also received one  or more special oversight reviews in the same year. Twenty-four of 183  grantees (13 percent) received multiple reviews in fiscal year 2007,  including one medium sized grantee that underwent a special procurement  system review in January 2007, a drug and alcohol program review in  February 2007, and a triennial review in June 2007.  The same grantee  also underwent two special civil rights reviews—an Americans with  Disabilities review in October 2007 and a Disadvantaged Business  Enterprises review in November of 2007.
One grantee with whom we spoke said that although the triennial review is  beneficial, the multiple oversight reviews it received in the same fiscal  year were burdensome, redundant and delayed project schedules and cost  the grantee about $18,000 to $20,000. Four other grantees provided  information showing they spent about 100 to 300 hours preparing for and  hosting 2- to 3-day triennial reviews involving sometimes over 20  individuals and many departments. Some grantees also said that special  oversight reviews continue for two or more weeks and require even more  grantee time and attention.
Representatives from the American Public Transportation Association—an  association that represents many grantees—also said that two or more  FTA oversight reviews in the same year could be burdensome. According  to the association representatives and grantees with whom we spoke,  multiple reviews in the same year are especially burdensome for smaller  grantees, whose employees tend to be responsible for multiple review  areas. For example, a small grantee told us that the triennial review alone  taxed its two employees who were responsible for addressing many of the  areas in the triennial review. Large grantees, on the other hand, typically  have several employees in a division responsible for one triennial review  area, so the burden of multiple reviews may be less taxing for them than  for small grantees.
The decision-making body responsible for developing and coordinating  FTA’s oversight review schedules is the Oversight Review Council; but,  according to FTA officials, the Oversight Review Council is not fulfilling  these responsibilities, as it was inactive on and off in recent years because  of management changes and priorities. FTA Order 5400.1 requires the  Oversight Review Council to develop and coordinate various oversight  review schedules to avoid unnecessary duplication and overlap, among  other things. The Council was reinstituted in 2007 after management  changes, and continues to redefine its roles and responsibilities. Further,  FTA is aware of the burden oversight reviews place on grantees and works  to limit this burden.
Without the Oversight Review Council taking responsibility for developing  and coordinating oversight review schedules, FTA offices developed their  own oversight processes, which, in some instances, could lead to grantees  receiving a triennial review and a special review in the same fiscal year.  For example:    FTA’s goal is to conduct special procurement reviews every three years on  the 30 grantees that receive large amounts of federal transit funds.
The Offices of Civil Rights and Safety and Security conduct their own  special oversight reviews, including reviews of grantees that have had civil  rights complaints filed against them. According to officials from the Civil  Rights office, after notifying the regions and others of the schedules, the  regions are to notify headquarters about any scheduling conflicts.
The drug and alcohol review schedules are sometimes made without  informing regional staff because this information is sensitive.
Other FTA officials who conduct special oversight reviews enter their  scheduled review date of grantees in OTRAK. However, regional staff said  they are sometimes not aware of special review scheduling.
In receiving urban area formula funds, grantees agree to the  congressionally-mandated triennial review at least once every three years.  Some grantees are subject to other FTA oversight reviews. Having an  effective body to coordinate the strategic planning of special oversight  reviews, in coordination with the triennial review, could help increase  FTA’s oversight coverage of grantees, while at the same time, minimize the  undue burden of uncoordinated multiple oversight reviews on some  grantees. FTA’s new Web-based, master calendar feature in OTRAK is  expected to improve the visibility of future oversight review schedules.
section_title: FTA’s Two Triennial Review Performance Measures Meet Some, But Not All Key Attributes of Successful Performance Measures
paragraphs: FTA established two performance measures to assess timeliness of steps  in the triennial review program: (1) 80 percent of triennial review findings  are to be closed within 30 days of their due date (“Close Findings Timely”);  and (2) 95 percent of the final triennial review reports are to be issued  within 30 days of the completion of the review (“Issue Reports Timely”).  While these measures meet some, they do not meet all key attributes of  successful measures, as described in our past work on the Government  Performance and Results Act of 1993. Further, FTA lacks performance  measures that assess other core elements of FTA’s triennial review  program, including the outcome and quality of the reviews. FTA officials  recognize the limits of their current performance measures and have  begun efforts to enhance them. Table 3 summarizes the 9 key attributes we  used to analyze FTA’s performance measures.
Figure 6 compares FTA’s triennial review performance measures to these  key attributes, showing that they meet some but not all attributes. Both  measures meet 4 of 9 attributes; that is, they link to agency goals, provide  a measurable target, are clear, and have limited overlap with other  measures. The Issue Reports Timely measure meets 2 additional attributes  by demonstrating objectivity and reliability.
Linkage. We found FTA’s two triennial review performance measures link  throughout the organization—from FTA regional offices, through  headquarters, and to the department level. Both performance measures  aligned with the Department of Transportation’s strategic goal of  organizational excellence in the FTA Annual Performance Plan, FTA  regional business and strategic plans, and individual performance  appraisals. In addition, FTA officials—including officials in the  Performance Management Division, the Office of Program Management,  and all 10 regional offices—confirmed in interviews that performance  goals were communicated across the agency. Such communication is  important to ensure FTA officials understand what the organization is  trying to achieve and the goals it seeks to reach.
Measurable target. Both performance measures also have measurable  numeric targets—given in percentages—that allow FTA officials to easily  assess whether overall goals and objectives were achieved. Numeric  targets facilitate assessments of whether objectives were achieved  because comparisons can be easily made between projected performance  and actual results. FTA reports meeting the Close Findings Timely  measure in fiscal years 2004 through 2007; and in fiscal year 2008, closed  74 percent of findings within 30 days. Although FTA reports never meeting  the Issue Reports Timely measure in fiscal years 2004 through 2008, it has  greatly improved the timeliness with which it issues triennial review  reports. From fiscal years 2004 through 2008, FTA nearly met its goals in 4  of the 5 years.
Clarity. We found the performance measures are clearly stated and each  has a definition that is consistent with the methodology used to calculate  it. The Issue Reports Timely formula is the percentage of reports issued  within 30 days of the final conduct of the review, and the Close Findings  Timely formula is the number of findings closed within 30 days of the due  date, divided by the sum of the number of findings closed plus the number  of findings open that are 30 days past the due date. Clear measures provide  transparency and allow managers to accurately assess FTA’s performance  in issuing triennial review reports and closing findings in a timely manner.
Limited overlap. Both program performance measures have unique  names, definitions, and do not overlap. One measure evaluates the  timeliness of issuing reports, while the other assesses the timeliness of  closing findings. The presence of distinct performance measures prevent  managers from sifting through unnecessary or redundant information.
Objectivity. We found the Issue Reports Timely measure is objective  because the data elements used to calculate it are free from significant  bias. However, our analysis of the Close Findings Timely measure found  that its objectivity is limited because decisions to close findings, establish  due dates for when grantees must address findings, and extend, if  necessary, any due dates, is done largely at the discretion of regional staff,  in consultation with grantees, and, when appropriate, headquarters  officials. This presents opportunities to influence this measure because  regional staff may use different criteria for when to close or extend  deadlines. While FTA officials recently met to discuss a mandatory  process to uniformly close findings across all FTA regions, the current  lack of criteria reduces the transparency of FTA’s performance in closing  findings. For example, an FTA official stated that the Closing Findings  Timely performance measure could result in the unintended consequence  of prematurely closing findings. The annual performance appraisals and  resulting compensation for regional staff, including the Regional  Administrators, are based, in part, on meeting the Close Finding Timely  performance measure. While we did not find widespread instances when  findings were prematurely closed, an official from the civil rights office  stated there have been cases when regional staff closed triennial review  findings in OTRAK and requested that the civil rights office monitor the  grantees’ progress in resolving them.
Reliability. We found the Issue Reports Timely measure is reliable, as FTA  has standard procedures for collecting data, and based on our examination  of individual triennial review reports, accurately stated the number of  reports issued in the most recent end of year report for fiscal year 2008.  However, our analysis of data used to calculate the Close Finding Timely  performance measure found data inaccuracies and, therefore, the measure  does not meet the key attribute of reliability. After examining closed  findings data we found several instances where OTRAK reported findings  open more than seven years after they were due for closure. FTA officials  familiar with the database said that these inaccuracies resulted from FTA  regional staff not entering data into OTRAK after the transition to the new  Web-based information system in 2003 and said, but could not document,  that the findings were likely closed. However, we also found OTRAK data  entry errors from fiscal years 2005 through 2007—after FTA had  transitioned to the Web-based system—where the database showed almost  200 findings were open more then one year after they were due for  closure. Headquarters officials said, but could not document, that these  findings are inaccurately listed as open in OTRAK and were likely closed  at the regional level. Officials further stated they plan to re-examine the  regional process for closing findings and issue guidance to improve this  process. Finally, the 2008 Booz Allen Hamilton report noted instances of  missing data in OTRAK because some regions were using stand alone  spreadsheets instead of fully using the database to report closing  findings. Officials acknowledged that entering closed findings data into  the database can be problematic, especially for inexperienced regional  staff not familiar with OTRAK. Officials believe that a 2007 decision to tie  regional staff performance appraisals to data entered into OTRAK should  provide an incentive to ensure future data are reliable. In addition, officials  said they are emphasizing the importance of recording data into OTRAK to  regional offices and contractors. Without reliable data, FTA managers  cannot be confident in the extent to which performance goals have been  achieved, including the timeliness in closing findings.
Core program activities. Both triennial review performance measures do  not meet the triennial review program’s core program activity, evaluating  whether grantees meet federal requirements over time. While FTA’s  existing performance measure of Close Findings Timely may increase  adherence to federal requirements by encouraging grantees to address  findings, it does not provide an overall program level assessment of  grantees’ performance over time. In addition, FTA measures annual  grantee compliance information in Triennial Review Program end-of-year  reports, but has not developed performance measures that set targets to  evaluate progress. Therefore, FTA does not assess whether the triennial  review program is increasing grantees’ ability to continuously meet federal  requirements. FTA could measure the core program activity of the  triennial review program in various ways, such as assessing grantees’  performance at the end of each review year or assessing grantees’ long  term performance, as our previous analysis of findings data shows. See  figure 3.
Governmentwide priorities. FTA’s current performance measures  partially address governmentwide priorities by ensuring that FTA reports  on its timeliness in conducting reviews and in addressing corrective  actions. However, the measures do not evaluate other governmentwide  priorities, including the outcome of the reviews—the extent to which  grantees are meeting federal requirements, or the quality of the triennial  reviews.
Outcome. The performance measures assess the output of the triennial  review program—the closing of findings and issuing of final reports— but, not the outcome—grantees’ ability to meet federal requirements.  As previously stated in the discussion of core program activities,  outcome performance measures—such as grantees’ performance over  time, or annual progress against a measurable target—can help  evaluate the results of the triennial review programs. Without such a  measure, FTA is limited in the extent to which it can effectively  ascertain and document the extent to which the triennial review  program is improving grantees’ performance in meeting federal  requirements and FTA managers have limited information on program  performance, making the possibility of achieving program goals less  likely.
Quality. FTA has no performance measures for assessing the quality of  the triennial review program. FTA officials help ensure quality by  checking triennial review reports to determine whether the  contractors made reasonable findings for grantees, determining what  actions grantees must take to resolve findings, and attending some site  visits to oversee contractors. However, FTA officials are unsure how  to develop a measure for evaluating the quality of the triennial review  program. In addition, Booz Allen Hamilton reported that FTA  headquarters and field staff do not have a common understanding of  what constitutes a quality review or how it could be measured.  Without common quality standards, FTA cannot evaluate the overall  quality of the program nor ensure the results and objectives of the  triennial review program are being met. Performance measures related  to quality could assess the implementation of triennial reviews,  including a way to oversee contractors and test the overall quality of  reviews. Many federal agencies periodically review their programs to  ensure they meet quality standards. FTA officials recognize the  benefits of such performance measures and plan to enhance its current  measures and standards to assess the quality and value added of the  triennial review program to help ensure that the results and objectives  of the triennial review program are being met.
Balance. FTA’s triennial review program performance measures address  only one of three governmentwide priorities and, therefore, are not  balanced—an attribute that applies to the overall suite of measures, rather  than the measures individually. Balanced performance measures allow  managers to understand what other triennial review aspects are working  well and why. Adding outcome and quality performance measures will  help balance the existing timeliness measures and could allow managers  to better set triennial review priorities by not overemphasizing one or two  priorities at the expense of others. FTA officials recognize current  performance measures do not assess some aspects of the triennial review  program and are undertaking efforts to enhance them.
section_title: Conclusions
paragraphs: Strong FTA oversight of federal surface transportation grant programs  helps ensure that grantees meet federal requirements, provide safe public  transportation, and safeguard from misuse the billions of dollars that  Congress appropriated for surface transportation programs to help  America recover from the economic downturn and the billions more that  Congress plans to reauthorize for surface transportation. FTA’s efforts  over the past decade to improve the congressionally-mandated triennial  review—the primary means for overseeing hundreds of grantees—have  produced concrete results. For example, FTA now regularly collects  oversight data on grantees, issues the vast majority of triennial review  reports within 30 days of the review (a significant improvement since our  last report in 1998), and provides grantees with technical assistance to  address problem areas. Nonetheless, FTA should undertake additional  efforts to further improve the triennial review program.
Although one of the goals of the triennial review program is to assist  grantees in meeting more federal requirements over time, our analysis of  OTRAK data shows many grantees with mixed success in reducing their  numbers of findings over three triennial reviews from fiscal years 2000  through 2008; and 17 percent of the grantees we reviewed met fewer  federal requirements in their latest triennial review than they did in their  two previous reviews. While FTA analyzes apparent problem areas, its  analysis does not provide information on the underlying causes of findings  and information on the level of severity for each finding. Without  conducting further analyses of its OTRAK data and better understanding  the underlying causes of grantees’ performance in triennial reviews, FTA  may be missing valuable opportunities to increase its effectiveness in  helping grantees improve their performance.
The triennial review process could be improved in two respects. First,  when a grantee has recently received or is scheduled to receive a special  oversight review, the flexibility currently allowed in the contractors’ guide  could lead to confusion about how to document that all 23 triennial review  areas are completely reviewed at least once every three years. Until FTA  revises its guidance to remove the “not reviewed” category, require an  updated evaluation of areas reviewed in the prior two years of the triennial  review, and describe how to document triennial review reports with the  updates, FTA does not provide assurance that grantees are being assessed  equally and are receiving a complete review and evaluation at least once  every three years, as required. Second, until the Oversight Review Council  performs its role as oversight review coordinator or delegates this role to  another FTA body, uncoordinated, multiple oversight reviews may not  effectively increase FTA’s coverage of grantees and may place undue  burden on a few grantees.
Finally, although FTA’s two triennial review performance measures meet  some of the key attributes we have previously reported are necessary for a  successful program, the Close Findings Timely measure could be  improved to minimize the possibility for manipulation and increase the  reliability of closed finding data. Without an internal control process to  verify whether grantees have satisfactorily resolved findings, FTA cannot  determine the validity of reported results for the Close Findings Timely  performance measure. Without reliable data, FTA may be unable to  determine the validity of reported results and make appropriate decisions.  FTA also lacks performance measures for evaluating the outcome and  quality of the triennial review program. Such performance measures are  essential because they can help decision makers determine areas for  improvement and increase the likelihood of achieving program goals, such  as increasing the number of grantees that meet federal requirements.
section_title: Recommendations for Executive Action
paragraphs: To support the triennial review program’s goal of having grantees  consistently meet more federal requirements, the Secretary of  Transportation should direct the FTA Administrator:  1.  To further analyze and use OTRAK data on the triennial review to help  grantees improve and sustain their performance. As part of this  analysis, FTA should consider and identify, when necessary: (1) high  performing grantees and their management practices, among other  things; (2) the areas in which problem findings occur, and the  underlying causes of these findings; and (3) the level of severity for  each finding. 2.  To help ensure that a complete performance review and evaluation of  each grantee is conducted at least once every three years, in  accordance with the statutory requirement, revise the contractors’  guide to remove the “not reviewed” category, require an updated  evaluation of areas reviewed in the prior two years of the triennial  review, and describe how contractors should document the triennial  review reports with updates to other reviews, such as special oversight  reviews conducted in the prior two years. 3.  To strengthen the triennial review process by ensuring that the  Oversight Review Council, or another body, carries out the  responsibilities specified in FTA Order 5400.1 to be actively involved in  coordinating oversight review schedules in order to increase FTA’s  oversight coverage of grantees while, at the same time, minimizing the  undue burden of multiple oversight reviews on grantees. 4.  To improve the objectivity and reliability of the “Close Findings  Timely” performance measure by:   developing an internal controls process to verify grantees have  satisfactorily resolved findings before closing them; continuing to emphasize the use of OTRAK across all regions and  developing a process to close out triennial reviews in OTRAK upon  grantees’ completion of corrective actions.  5.  To meet standards set forth in previous GAO work, based on the  Government Performance and Results Act of 1993, and develop  performance measures to assess: the outcomes of the triennial review program, such as a method for  evaluating improvements in grantee performance in meeting more  federal requirements over time; and the quality of the triennial review through improved contractor  oversight, testing, or inspection.
In addition, FTA should ensure additional outcome and quality  performance measures are balanced with the existing measures for  assessing the timeliness of the triennial review program so that one  priority is not emphasized at the expense of others.
section_title: Agency Comments
paragraphs: The Department of Transportation reviewed and commented on a draft of  this report. The department generally agreed with the report’s findings and  recommendations. It also suggested that we provide more information on  FTA’s efforts to review grantees at least once every three years and  coordinate its oversight reviews, which we added, as appropriate. The  Department also provided a technical correction, which we have  incorporated.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to appropriate congressional  committees, to the Secretary of Transportation, and to appropriate FTA  officials. The report will also be available at no charge on the GAO Web  site at www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 2834 or herrp@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To determine the extent to which the triennial reviews indicate grantees  met federal requirements from fiscal years 2000 through 2008, we  reviewed documentation on the Federal Transit Administration’s (FTA)  Oversight Tracking System (OTRAK)—the official electronic information  system for tracking and monitoring oversight activities. Documents  included a system description, requirements, and background; reference  guides; data entry controls and processes; data dictionary; deficiency  codes; and the users’ manual. We requested and obtained OTRAK findings  data by grantee and by area from fiscal years 2000 through 2008. Data prior  to 2000 was not available. We sorted the data to, among other things,  determine the extent to which grantees’ performance in meeting federal  requirements increased, decreased, both increased and decreased, or  remained constant in three triennial reviews; which grantees averaged  fewer than two findings in their triennial reviews; and which areas  accounted for the greatest numbers of findings. We did not have direct  access to the OTRAK database, but took several steps to assess the  reliability of the data. We performed electronic testing to identify missing  data, obtained and reviewed the queries that FTA used to generate the  data that we requested, and discussed with FTA and its OTRAK  contractors the results of the information. Based on these tests, we  excluded 150 grantees because they did not meet our criteria, having  exactly three triennial reviews in the last nine years and excluded 27 other  grantees because, according to FTA officials, they received triennial  reviews for multiple triennial review cycles, more than 3 fiscal years, or  both as a result of triennial reviews that were either canceled or closed  without findings.
To address the strengths and weaknesses of the triennial review process,  we reviewed federal guidance for improving grant accountability, federal  internal control standards, GAO reports on performance measures, FTA  Order 5400.1, triennial review reports of numerous grantees, and a 2008  Booz Allen Hamilton report of FTA’s oversight programs. We also  reviewed and compared schedules for the triennial and other special  oversight reviews by grantee from fiscal years 2006 through 2008 to  determine which grantees experienced both a triennial review and at least  one special oversight review in fiscal year 2008. In addition, we examined  specific areas reviewed or not reviewed during fiscal years 2007 and 2008  triennial review reports and compared the areas to completed or planned  special oversight reviews. We attended the triennial review of the Los  Angeles County Metropolitan Transit Authority in September 2008. We  also attended a FTA triennial workshop for new grantees in Kansas City,  Missouri in July 2008.
We selected 10 grantees with a range of findings, from various FTA  regions, having special oversight reviews in fiscal years 2006, 2007, or  2008, and from different population sizes—small (50,000-199,999), medium  (200,000 to 999,999), or large (1 million or more) urban centers. We  interviewed these grantees in order to understand grantees’ practices that  contributed to them averaging fewer than 2 findings per review,  understand the burden multiple reviews had on grantees’ operations, and  understand if the triennial review unduly burdened grantees with different  size operations. The following lists the 10 grantees interviewed:    Southwest Ohio Regional Transportation Agency, Cincinnati, Ohio    Spokane Transit Authority, Spokane, Washington    St. Joseph Transit, St. Joseph, Missouri    Utah Transit Agency, Salt Lake City, Utah    Southeastern Pennsylvania Transportation Authority, Philadelphia,    Santa Monica Blue Bus, Santa Monica, California    Wichita Transit, Wichita, Kansas    Hampton Roads Transit, Hampton, Virginia    City of Hattiesburg, Mississippi    Santa Clara Valley Transportation Authority, San Jose, California  With respect to this and our other reporting objectives, we interviewed  officials from the following organizations:    FTA’s Associate Administrator for Program Management    FTA’s Office of Civil Rights    FTA’s Office of Safety and Security    FTA’s Office of Program Management    FTA’s Office of Performance Management    All FTA regional offices    All four triennial review contractors (Reid Consulting LLC, CDI/DCI Joint  Venture, Advanced Systems Technical and Management Inc., and  Interactive Elements LLC)
Booz Allen Hamilton    The American Public Transportation Association    Community Transportation Association of America.
To determine FTA’s performance measures for the triennial review and to  what extent FTA’s performance measures meet nine key attributes, we  obtained, reviewed, and analyzed, among other documents: GAO reports  describing the importance of establishing effective performance measures,  GAO guidance on assessing the strengths and weaknesses of federal  agencies’ performance measures and identifying the key attributes of  successful performance measures, FTA triennial review end-of-year  reports, and FTA performance appraisal templates. We also spoke with  FTA management to identify the current performance measures for  evaluating the success of the triennial review program, the reasons for  them, and when they were established. We then compared FTA’s two  performance measures to each of the nine key attributes in order to  determine the extent to which and why they either fully met, partially met,  or did not meet all attributes.
section_title: Appendix II: Description of FTA’s 23 Triennial Review Areas
paragraphs: Appendix II: Description of FTA’s 23  Triennial Review Areas FTA’s Description of the Requirements in Each Area  The grantee must be eligible and authorized under state and local law to request, receive,  and dispense FTA funds and to execute and administer FTA-funded projects. The  authority to take all necessary action and responsibility on behalf of the grantee must be  properly delegated and executed.
The grantee must demonstrate the ability to match and manage FTA grant funds,  maintain and operate federally-funded facilities and equipment, and conduct an annual  independent organization-wide audit in accordance with the provisions of OMB Circular A- 133.
The grantee must be able to implement the Urbanized Area Formula Grant Program of  Projects in accordance with the grant application, Master Agreement, and all applicable  laws and regulations, using sound management practices.
The grantee must maintain control over real property, facilities, and equipment and ensure  that they are used in transit service.
The grantee must keep federally-funded equipment and facilities in good operating order.
FTA grantees will use their own procurement procedures that reflect applicable state and  local laws and regulations, provided that the process ensures competitive procurement  and that the procedures conform to applicable federal law.
The grantee must comply with the policy of the Department of Transportation that DBEs  are ensured nondiscrimination in the award and administration of Department of  Transportation-assisted contracts. Grantees also must create a level playing field on  which DBEs can compete fairly for Department of Transportation-assisted contracts;  ensure that only firms that fully meet eligibility standards are permitted to participate as  DBEs; help remove barriers to the participation of DBEs; and assist the development of  firms that can compete successfully in the marketplace outside the DBE program.
Per “Buy America” law, federal funds may not be obligated unless steel, iron, and  manufactured products used in FTA-funded projects are produced in the United States,  unless FTA has granted a waiver, or the product is subject to a general waiver. Rolling  stock must have sixty percent domestic content and final assembly must take place in the  United States.
To protect the public interest and prevent fraud, waste, and abuse in federal transactions,  persons or entities, which by defined events or behavior, potentially threaten the integrity  of federally administered programs, are excluded from participating in FTA-assisted  programs.
Recipients of federal grants and contracts exceeding $100,000 must certify compliance  with restrictions on lobbying before they can receive funds. In addition, grantees are  required to impose the lobbying restriction provisions on their contractors.
The grantee must participate in the transportation planning process in accordance with  FTA requirements; Safe, Accountable, Flexible, Efficient Transportation Equity Act: A  Legacy for Users requirements; and the Metropolitan and Statewide Planning  Regulations.
The grantee must ensure that no person in the United States shall, on the grounds of  race, color, or national origin, be excluded from participating in, or denied the benefits of,  or be subject to discrimination under any program, or activity receiving federal financial  assistance.
FTA’s Description of the Requirements in Each Area  The grantee is expected to have a written copy of a locally developed process to solicit  and consider public comment before raising a fare or carrying out a major reduction of  transportation services.
Grantees must ensure that elderly persons and persons with disabilities, or an individual  presenting a Medicare card will be charged, during non-peak hours for transportation  using or involving a facility or equipment of a project financed under Section 5307, not  more than 50 percent of the peak hour fare.
Titles II and III of the Americans with Disabilities Act of 1990 provide that no entity shall  discriminate against an individual with a disability in connection with the provision of  transportation service. The law sets forth specific requirements for vehicle and facility  accessibility and the provision of service, including complementary paratransit service.
FTA grantees are prohibited from using federally-funded equipment and facilities to  provide charter service except on an incidental basis and when one or more of applicable  exceptions set forth in the charter service regulation applies.
FTA grantees are prohibited from providing exclusive school bus service unless it qualifies  under specified exceptions. In no case can federally-funded equipment or facilities be  used.
The grantee must collect, record, and report financial and non-financial data in  accordance with the Uniform System of Accounts and updated with the National Transit  Database Reporting Manual.
Any recipient of Urbanized Area Formula Grant Program funds must annually certify that it  is spending at least one percent of such funds for transit security projects or that such  expenditures for security systems are not necessary.
FTA grantees are required to maintain a drug-free workplace for all employees and to  have an ongoing drug-free awareness program.
Grantees receiving FTA funds under Capital Grant (Section 5309), Urbanized Area  Formula Grant (Section 5307), or Non-Urbanized Area Formula Grant (Section 5311).  Programs must have a drug and alcohol testing program in place for all safety-sensitive  employees.
The grantee must ensure that no person in the United States shall on the grounds of race,  color, creed, national origin, sex, age, or disability be excluded from participating in, or  denied the benefits of, or be subject to discrimination in employment under any project,  program, or activity receiving federal financial assistance from the federal transit laws.
ITS projects funded by the Highway Trust Fund and the Mass Transit Account must  conform to the National ITS Architecture, as well as to Department of Transportation  adopted ITS Standards.
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact:
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Catherine A. Colwell (Assistant  Director), Matthew A. Cook, Elizabeth H. Curda, Bert Japikse, Delwen A.  Jones, Grant M. Mallie, Steve Martinez, Amanda K. Miller, Joshua H.  Ormond, Kelly L. Rubin, and Sandra E. Sokol made key contributions to  this report.