On-the-Job Training: Federal Highway Administration Needs to Strengthen Program Assessment
Published: 2011-09-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Federal Highway Administration's (FHWA) on-the-job training program-- a relatively small part of the federal aid highway program--requires states to implement job training programs to provide traditionally underrepresented groups with opportunities in highway construction. To increase the effectiveness of state job training programs, FHWA grants up to $10 million annually for supportive services, such as job placement assistance. This report examines the extent to which (1) FHWA's job training program enhances training and career opportunities for these groups, (2) FHWA oversees the job training programs, and (3) supportive services provide assistance to these programs. To address these topics GAO reviewed federal legislation, good management practices identified in prior GAO reports, FHWA documents, and proposals and reports submitted by states. GAO conducted an in-depth examination of these efforts in four states, and interviewed a cross-section of FHWA staff, state officials, and industry groups.
section_title: What GAO Found
paragraphs: It is unclear the extent to which FHWA's on-the-job training program enables women, minorities, and economically disadvantaged individuals to reach journeylevel status in the highway construction trades, although stakeholders believe it can create some opportunities. FHWA's decentralized management of the program--in which state transportation agencies and FHWA's division offices are generally responsible for program implementation--has led to a wide range of practices. As a result, the types of training opportunities created by the program vary from state to state in terms of, for example, the length of training and the entities involved in providing training. In addition, the extent that state programs focus on creating training opportunities for traditionally underrepresented groups differs. The limited amount of useable information available on program results varies among states. As a result, FHWA does not know how well the program is doing, and GAO could not accurately determine how many trainees participate in the program or the demographics of those trainees; however, GAO estimates that several thousand likely participate in any one year. FHWA's oversight approach does little to assess program results. FHWA lacks clear criteria that articulate what states are supposed to accomplish through their job training programs. While some broad program expectations are stipulated in guidance and regulations, FHWA acknowledges some of these are outdated. Furthermore, FHWA's oversight approach does not determine the overall effectiveness of state programs or measure state progress. For example, although state transportation agencies are required to submit achievement information on an annual basis to FHWA division offices, states submitted this information using a wide range of different output terms and different demographic and trade classification categories. GAO has reported that program criteria are key aspects of results-oriented performance management. Through a separate program, FHWA provides funding for a variety of activities intended to increase the overall effectiveness of the on-the-job training program, but its overall stewardship of the program is limited. FHWA's supportive services program provides grants for locally tailored initiatives, such as skills training, child care, and career awareness events, that directly and indirectly link to job training programs. However, there is insufficient data to determine how effective these efforts have been in enhancing job training opportunities. Although FHWA has articulated the types of data states should collect and report, the agency does not know, and GAO could not determine, the number of participants in the supportive services program or its effect, in part because grantees do not always provide information about their program results. However, GAO estimated that there are about 10,000 people participating in any one year. Furthermore, past performance information is not required of applicants or scored during funding reviews. Given that many grantees are funded repeatedly, good management practices suggest that using past performance information can inform and improve recipient selection approaches. Program results are important for making budgetary and programmatic decisions. Without insight into program activities, FHWA cannot ensure that funding is used effectively.
section_title: What GAO Recommends
paragraphs: GAO recommends that FHWA (1) strengthen on-the-job training program criteria, (2) create and implement an oversight approach for its job training program, and (3) evaluate the extent to which supportive services programs have met their goals and use this information to inform future funding decisions. The agency generally agreed with these recommendations and provided technical comments, which were incorporated as appropriate.

section_title: Letter
paragraphs: By providing states with about $35 billion to $40 billion per year, the  federal-aid highway program supports tens of thousands of high-skill and  high-wage jobs in the construction industry. Ensuring that these jobs are  equally available to all Americans has long been an objective of the  federal government. Specifically, following the enactment of the Federal  Aid Highway Act of 1968, the federal government created an on-the-job  training program to address the traditional underrepresentation of specific  groups in the highway construction trades. In implementing this program,  states may include an on-the-job training special provision in some  federal-aid highway construction contracts that requires contractors to  provide training and skill enhancement opportunities targeted to women,  minorities, and economically disadvantaged individuals. Although the  program typically has just a few trainees per project—and is a relatively  small part of the federal-aid highway program overall—the cumulative  scope of the program has covered decades of federal-aid highway  projects. Additionally, through a separate supportive services program,  the U.S. Department of Transportation is authorized to grant up to $10  million per year to fund a range of grants and other services, such as job  placement assistance, aimed at increasing the effectiveness of the on- the-job training program. Both of these programs are administered by the  Federal Highway Administration (FHWA) within the U.S. Department of  Transportation.
With the recent economic recession and the subsequent persistence of  high unemployment rates, the effectiveness of these decades-old job  training programs became increasingly important. During the recession of  2007–2009, unemployment in the construction industry exceeded 10  percent and unemployment for most of the groups targeted by the  programs was above the national rate. For example, in 2009,  unemployment among African Americans was nearly 15 percent  compared to 9.3 percent for the population as a whole. As Congress  began oversight of the American Recovery and Reinvestment Act of 2009  (Recovery Act), concerns were raised in Congress that these programs  may not be effectively helping vulnerable populations find training and  jobs. In light of these concerns and given the sparse information already  available on the results of FHWA’s on-the-job training and supportive  services programs, you asked us to review how well FHWA’s two job  training programs have been working.
This report focuses on the extent to which (1) states’ implementation of  the FHWA on-the-job training program provides training and career  enhancement opportunities for traditionally underrepresented groups to  work on federal-aid highway projects, (2) FHWA has overseen the  program and demonstrated results, and (3) the supportive services  program established by FHWA has assisted state job training and  apprenticeship programs. Additionally, in response to your interest, we  gathered information on financial incentives and penalties states use to  encourage contractors to provide job training and we present this  information in appendix II. To address these topics we reviewed laws and  regulations pertaining to these programs, reviewed FHWA guidance and  good program practices identified in our prior reports, and analyzed  proposals and reports from states on the activities planned and  completed as a result of these programs. We also interviewed a cross- section of state transportation officials; representatives and members of  industry groups, such as general contractors and unions; and FHWA  program officials. Specifically, we interviewed officials in 11 states and  examined how they implemented the job training programs and  conducted oversight. We selected these states based on a variety of  factors, including unionization, size and location, and the state’s use of  supportive services funds. For four of these states (Missouri,  Pennsylvania, Texas, and Washington), we conducted a more in-depth  examination of how they implemented and conducted oversight for these  programs, taking into account the effect of variations in demographics;  prevailing labor market conditions; how job training is structured within the  state, including any unique practices; and the extent to which the state  participates in the on-the-job training program and the supportive services  program. The information gathered through these four in-depth reviews is  intended to describe these unique programs and is not generalizable to  all states or to other states with similar characteristics. We focused our  review on program activities conducted from fiscal years 2008 through  2010. We also estimated the number of participants in these programs  using available data in proposals and reports submitted to FHWA. The  quality of the data reported by states to FHWA about the results of the  programs’ activities cannot be independently verified, but we determined  that the data were of sufficient quality for our purposes, which were to  assess the types of information collected by states and reported to FHWA  and to provide illustrations of states’ reported achievements.
We conducted this performance audit from August 2010 through  September 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives. (Additional  information on our scope and methodology is contained in app. I.)
subsections: section_title: Background
paragraphs: Building highways requires a diverse collection of skill sets and expertise.  The highway construction industry is composed of a variety of different  trades such as equipment operators, mechanics, truck drivers,  ironworkers, carpenters, and cement masons. These trades are not  specific to highway construction but also apply to other areas such as  general building construction. Historically, many of these trades have  been dominated by white males. Over the last 25 years, participation of  women in the construction industry has been relatively stable, averaging  around 8 to 10 percent. Among minority groups, while the percentage of  African Americans participating in the construction workforce has been  relatively constant (5 to 7 percent of the workforce), the proportion of  Latinos in the industry has increased significantly and is estimated to  comprise about a quarter of the construction workforce.
In 1968, Congress authorized a job training program, which required that  states receiving federal aid for highways implement apprenticeship and  training programs that ensure equal employment opportunities without  regard to race, color, creed, national origin, or sex. FHWA has targeted  this job training program to women, minorities, and economically  disadvantaged individuals to provide these groups with highway  construction-related skills to reach journey-level status. As a  nondiscriminatory program, it cannot exclude any individual from  participation. The Secretary of Transportation must certify that states  have such a program in place. While the program does not have a distinct  funding source, job training expenses on federal-aid highway projects are  eligible for federal dollars, just like other project costs. According to  FHWA officials, each state has discretion in how much or little to spend  on this program, as well as how a contractor will be reimbursed, if at all,  for training activities on its highway projects.
State transportation offices are responsible for providing their state  program’s framework. Individual states establish overall training goals for  the program such as articulating the number of training slots on selected  federal-aid highway construction contracts they expect to award during  the coming year. These state agencies also determine which federal-aid  highway construction contracts will require that training is provided to a  minimum number of trainees and thus which contracts include a training  special provision. State transportation agencies—using on-site personnel  such as contractor compliance specialists, engineers, and project  inspectors—monitor and oversee contractors’ adherence to job training  requirements.
Multiple FHWA offices are involved in the on-the-job training program.  FHWA’s Office of Civil Rights, located in the agency’s headquarters  office, provides leadership in setting policy for the program, and in this  capacity has assigned oversight responsibilities to the FHWA division  offices in each state and has established regulations for state  transportation agencies to administer the program. This decentralized  management structure—in which certain oversight responsibilities are  assigned to division offices—is consistent with FHWA’s approach for the  federal-aid highway program in general. Among the primary  responsibilities of the division offices are to review the annual goals  submitted by the states and oversee whether states are meeting their  annual job training goals by reviewing accomplishment reports and  annual contractor training reports. FHWA division offices also play a role  in assessing the efforts of both the individual contractors participating in  this program and the states to meet their respective job training goals and  requirements. FHWA’s Resource Center also provides assistance to  state transportation agencies and FHWA offices in implementing FHWA  policy and programs by providing training on program requirements and  other activities (see table 1).
While a state approves a contractor’s training approach, the contractor is  responsible for meeting a training requirement. If a contractor fails to  meet the training requirement stipulated in the contract, the state  transportation agency can withhold payments or limit the contractor’s  ability to bid on future contract opportunities. The amount of the sanctions  or penalties will depend on the state transportation agency’s contract  administration procedures. For example, in Missouri, the state  transportation agency may levy fines against contractors that do not fulfill  their training requirements.
Congress authorized the supportive services program in 1970 to  supplement the on-the-job training program and support state  transportation agency training programs by providing services to highway  construction contractors and assistance to highway construction  apprentices and trainees. According to FHWA, the main objectives of  the program are to increase the overall effectiveness of each state job  training program in connection with federal-aid highway projects and seek  other ways to increase the training opportunities for women, minorities,  and economically disadvantaged individuals. Depending on the needs  and preferences in a specific state, services funded under this program  may include basic training in the construction industry, stipends to support  childcare or defray transportation costs for highway construction workers,  transportation career awareness programs, and internships. FHWA is  authorized to grant up to $10 million per year in this program.
FHWA funds some supportive services through a discretionary grant  program. From 2008 through 2010, FHWA awarded 153 grants totaling  about $37 million to 37 states, the District of Columbia, Guam, and Puerto  Rico for supportive services through the discretionary grant program.  According to FHWA guidance, supportive services activities must be  connected to a federal-aid highway construction project in order to be  eligible for funding. State transportation agencies must submit a  statement of work to FHWA that details how the grantee proposes to use  the requested funding. The proposals are required to establish  measurable goals for results, although FHWA permits wide variation in  defining these goals. According to FHWA, proposed activities must target  their efforts towards women, minorities, or economically disadvantaged  individuals to receive funding. According to FHWA guidance, applications  are reviewed by panels consisting of FHWA officials and scored using  criteria to evaluate the content of the statement of work, including whether  the statement of work has defined end dates, how the needs of the  supportive services program would be addressed through the proposed  activities, and a monitoring plan. FHWA requires each grantee to submit  regular progress reports to the respective FHWA division offices as well  as a final accomplishment report within 60 days of the program’s  completion, which provides information on previously established  performance goals. According to FHWA officials, an interim  accomplishment report is also requested from on-going activities when  additional funding is requested during the annual grant solicitation  process.
FHWA also uses the supportive services funding for transportation- related education and internship opportunities. FHWA distributes some  funding to summer transportation institutes (located at universities and  colleges), which provide a curriculum designed to introduce secondary  school students to careers in all modes of transportation and encourage  them to pursue transportation-related courses of study at the university  level. There is also a summer internship program, wherein FHWA selects  and places students from various colleges and universities in internships  at one of the U.S. Department of Transportation’s modal agencies, such  as FHWA, or at a state transportation agency. These programs are  targeted at women, minorities, and economically disadvantaged  individuals to provide hands-on experience and on-the-job training while  working on current transportation-related topics and issues.
section_title: State Programs Provide a Range of Different Opportunities for Enhancing Skills and Vary in Their Results
paragraphs: The overall extent that FHWA’s on-the-job program has helped women,  minorities, and the economically disadvantaged reach journey-level status  in the highway construction trades is unclear, primarily because a limited  amount of usable information is available on program results. However,  based on the limited information that is available, state programs have  created some new opportunities tailored to individual state circumstances.  The skill enhancement opportunities provided by state programs vary  across states by requiring different lengths of training, relying on a  diversity of entities and training structures to deliver the training, and  giving contractors various flexibilities in how and what skills they provide  to individuals. Additionally, state programs have varying degrees of focus  on reducing underrepresentation. While we cannot definitively report on  how well women, minorities, and economically disadvantaged individuals  are moving to journey-level status in highway construction, we can  describe some of the approaches states have chosen to implement the  program and make some observations about how the program has  contributed to enhancing the career prospects of individuals from the  targeted groups.
subsections: section_title: Wide-ranging State Programs Provide Different Opportunities for Enhancing Skills
paragraphs: One way state programs differ in providing opportunities for enhancing  skills is by providing different length training periods to participants.  Contracts that road builders enter into with state transportation agencies  require varying degrees of training that must be provided to trainees,  yielding differing degrees of trade proficiency for participants, from a brief  to extensive exposure to the trades. For example, in an effort to give as  many individuals as possible some construction experience, Florida has  adopted a program in which a contractor must provide each trainee with  approximately 300 hours of training. State officials explained that this  program is intended to give trainees a brief exposure to a trade, not  mastery of a craft. Florida reported that 288 people completed training  through the program in 2009. Conversely, some states require  contractors to have trainees substantively complete or graduate from  training programs that may take multiple years—and thousands of  training hours—to complete and provide trainees with a high level of work  experience and a marketable credential, but that reach fewer people. For  example, Texas, which employs this approach, graduated 57 trainees in  2009. Other states, such as Missouri and Washington, fall in between  these examples by requiring contractors to provide individual trainees  with 400 to 1,000 hours on any given project, believing this is the  minimum needed for meaningful skill acquisition. As a result of these  different state approaches to the contract training requirement, trainees in  different programs acquire different levels of job experience and trade  proficiency to take with them when seeking subsequent employment.
A second way state programs differ in providing opportunities for  enhancing skills is in using different entities to deliver training. Depending  on what is available in the state, some trainees participate in the on-the- job training program as part of preexisting training programs—such as  apprenticeship programs—or training that was designed and managed by  the contractor or an industry group (see fig. 1). For largely unionized  construction workforces, training is typically provided through  apprenticeship programs. Apprenticeship programs vary in length  depending on the trade, but are commonly about 4 years long and follow  specified training steps, including meeting job site training time  requirements and classroom instruction. As long as an individual is an  apprentice, he or she could be hired to meet a contract training  requirement.
Contractors that are not unionized may use industry-sponsored  apprenticeship programs, other training programs established by industry,  or, in the absence of such programs, design their own training plans. For  example, in Texas, trainees generally participate in a program established  by the state chapter of the Associated General Contractors and the Texas  Department of Transportation. Contractor-designed plans may be  approved by the state transportation agency or FHWA division office  officials, such as civil rights officials. These training plans are tailored to  the individual project and the specific contract requirement; unlike an  apprenticeship, these plans could include training in more than one trade.
A third way state programs differ is in program structure; specifically, how  much flexibility states give contractors in determining how to fulfill training  requirements. For example, some states have implemented contractor- based programs that provide contractors with the flexibility to have  trainees work on multiple projects. Typically, job training requirements are  specific to individual federal-aid projects and, to successfully meet the  terms of the contract, the contractor must meet this training requirement  by the time the project is completed. In states with contractor-based  programs, the state transportation agencies assign some contractors with  a target number of workers that they are expected to train and the  contractors have flexibility to decide on which federal-aid projects trainees  will work to meet the requirement. Under this approach, trainees may  work on more than one project, especially if one project does not have  enough work for an individual to complete the required training. For  example, Texas has adopted this contractor-based approach. In 2009,  the Texas Department of Transportation assigned training goals to 19 of  the largest contractors in the state based on the amount of work awarded  to each contractor in the preceding year. Through our review of state  reports and discussions with FHWA officials, we found that four additional  states—Indiana, Iowa, North Carolina, and North Dakota—have also  adopted this approach.
Other state practices give contractors added flexibility. For example,  Florida has established a system in which contractors can train  individuals on state-funded projects that do not receive any federal funds  and earn training credits that the contractor can later use to meet training  requirements on federal-aid projects. Transportation officials in Florida  indicated that trainees working on state projects must meet the same  training and demographic standards as on federal-aid projects. South  Carolina also allows contractors to temporarily transfer trainees between  projects and accumulate up to 25 percent of their required training hours  from other projects. Contractors do not receive payment for any of the  training hours received on other projects but these hours can count  towards completion of the training requirement. State officials explained  that contractors frequently take advantage of this option.
The effect of these differences in training length, delivery, and program  structure on the ultimate career enhancement of trainees is unclear. For  example, some union, state, and apprenticeship program officials noted  that apprenticeship programs provide workers with more structured  training and connection to a specific trade than training delivered  independently by contractors. One apprenticeship program official  explained that program staff can help ensure that a trainee gets job  opportunities. Similarly, according to U.S. Department of Labor officials,  without the structure of an apprenticeship program, trainees may not be  well positioned for long-term careers. Additionally, apprenticeship  programs, by definition, provide a clear demarcation of when an individual  moves from apprentice to journey-level. However, stakeholders also  reported that a contractor who is solely responsible for a training plan  could provide a wide variety of training cutting across multiple trades and  directly related to the skills needed by the employer, better positioning a  trainee for future opportunities with that contractor. In general, union and  nonunion officials agreed that effective training is more likely to lead to  employment if the training is directly linked to a specific job.
Views also differ about what type of training program structure— contractor-based or project-specific—is most effective. Proponents of  contractor-based approaches, including officials with FHWA’s Office of  Civil Rights and Resource Center, stated that this structure creates a  more solid relationship between the contractor and the trainee and helps  ensure the trainee gets a sufficient number of hours and the most useful  training, both of which improve ultimate employment outcomes.  Conversely, some state transportation officials indicated that the project- specific approach has advantages, such as giving trainees more freedom  to find employment with other contractors and giving contractors flexibility.
Ultimately, stakeholders agreed that the effectiveness of these training  activities in leading to jobs for participants depends heavily on the  condition of the broader economy. As states face budget shortfalls,  transportation funding is constrained, thus diminishing opportunities for  training. Moreover, during economic slowdowns, contractors may be  reluctant to bring in new employees as trainees. Given that  unemployment in the construction industry was roughly 20 percent in May  2010, potential trainees have faced strong competition in the labor market  for scarce construction employment opportunities, according to officials  from industry and labor groups. (See app. III for information on the on- the-job training opportunities created by the Recovery Act.)
section_title: State Programs Can Create Opportunities but Results Vary
paragraphs: Many states compile annual reports on their achievements. These reports  generally show that state programs have led to a wide range of  opportunities. However, these state accomplishment reports by  themselves do not provide a clear and reasonably comprehensive picture  of program achievement. Due to FHWA’s oversight approach and the  limitations of available information, the data collected and available from  FHWA do not provide a clear picture of the program’s results, limiting our  assessment to state-specific illustrations and anecdotes. As with other  federally-funded job training programs, such as those administered by the  U.S. Department of Labor, tracking the career outcomes of individuals  that receive training may be difficult because of the amount of resources  and effort required to measure long-term outcomes. The implications of  this difficulty are discussed later in this report. However, using available  data, we nevertheless estimate that several thousand people are trainees  in any given year nationally. Further, to provide more detailed illustrations  of the training provided through the program, in those states we reviewed  more closely, we found that most workers enrolled in the program were  minorities and that there was a range of female participation (see table 2).  For example, in Pennsylvania in 2009, all participants were either women  (64 participants) or males from a minority group (90 participants).
Additionally, program stakeholders told us that the job training program— by addressing ongoing barriers to employment for traditionally  underrepresented groups—can create opportunities that would not have  otherwise existed. For example, state officials in two states noted that  without the program fewer minorities would work in highway construction,  explaining that the construction industry continues to be dominated by  informal personal networks and a handful of firms, making it difficult for  traditionally underrepresented individuals to find career opportunities.  Many program stakeholders see the program as a useful counter to these  barriers. For example, one female journey-level equipment operator we  spoke with in Washington state attributes her success in the trade to an  opportunity the federal program provided in the 1980s. However, others  noted that the program could unintentionally disadvantage some workers.  For example, officials familiar with two states’ training programs noted  that contractor requests for women and minorities leave other potential  workers with fewer job opportunities.
Numerous officials noted that the success of the program depends largely  on whether the contractor; the state project oversight personnel, such as  project engineers and inspectors; and state transportation leadership are  committed to it. According to some of these officials, if a contractor wants  to create new training opportunities and sees value in diversifying the  firm’s workforce, then the program can have a positive impact. Similarly,  according to FHWA officials, the visible commitment and support of top  transportation leaders, a strong state civil rights office, and an  understanding of the program’s goals by construction engineers and field  staff help the program succeed. Conversely, contractors that are not  committed to the program may be able to find ways to fulfill a contractual  training requirement without actually hiring trainees. For example, a  contractor might be able to meet the requirement through the  demonstration of a good faith effort—in which the contractor describes its  unsuccessful attempts to find a suitable trainee—rather than through  actual hiring. Though good faith efforts may be legitimate and thoroughly  reviewed by oversight officials, in some cases, state officials responsible  for approving such good faith efforts may accept them without scrutiny  because they may not take the training requirement seriously or want to  avoid potential legal issues with the contractor, according to a senior state  official and a labor official. Additionally, some contractors may meet  training requirements by upgrading the skills of employees that are  already a part of their company’s workforce. In these situations, the  training requirement does little to create opportunities for  underrepresented groups because contractor hiring practices are not  changed.
Contractors may have different agreements with unions that dictate their  ability to specifically request—and for unions to provide—apprentices that  are women, minorities, or disadvantaged individuals in order to meet a  training requirement. Likewise, a union may be limited in its ability to meet  such requests if women and minorities are not available within their  membership. To address these barriers, some contractors and unions  have arrangements that help contractors to recruit new members to meet  demographic goals for trainees. For example, one union and contractors  in Pennsylvania allow for year-round enrollment of women and minorities  into one apprenticeship program instead of during shorter annual  enrollment periods.
State programs have varying levels of focus on women, minorities, and  disadvantaged individuals in accepting trainees on projects with job  training requirements. We generally found that the state transportation  agencies we reviewed intended for trainees to be women or minorities;  however, state programs varied in how rigorously they pushed  contractors to hire trainees from these targeted groups. For example,  states like Pennsylvania and Washington placed a strong emphasis on  accepting trainees that were either women or minorities and rarely  approved trainees not in these groups. Other states, however, enroll more  nontargeted individuals. For example, in 2009, white men were 83  percent of trainees in Ohio.
State programs target training opportunities toward either the highway  construction labor force in general or try to target underrepresentation of  women and minorities within specific trades. Some state programs  focused on increasing opportunities for women and minorities more  generally. For example, officials in southwestern states noted that some  construction trades in the region are now predominantly Latino and many  contract training requirements are fulfilled by Latino workers. When a  contractor in those states hires a Latino trainee to work within a trade that  is predominantly Latino, it meets the broad intent of the program by  creating an opportunity for a minority group that has been historically  underrepresented, but may not address current underrepresentation of  other groups. In contrast, other states may encourage contractors to  target training opportunities toward underrepresentation in specific trades,  although officials acknowledged that this approach can be difficult. For  example, an FHWA official explained that minorities and women were not  underrepresented as laborers in King County, Washington, so the state  tried to limit when a minority or woman could be hired as a laborer trainee  and focus training positions in other trades in that geographic area.  According to state officials in Florida and Washington, it can be difficult to  address underrepresentation of demographic groups in specific trades  because identifying such individuals can be a challenge for contractors.  Additionally, the training requirement cannot exclude individuals, such as  white men, from participation.
Some states have implemented other approaches to give contractors  flexibility in hiring trainees, although it is unclear if these flexibilities have  led to increased opportunities for women and minorities. For example, in  Ohio, contractors are not required to meet specific contract training  requirements or an overall annual goal (as in contractor-based programs).  Instead, state transportation officials explained that contractors are  required to provide training on all federal-aid projects unless they can  demonstrate that doing so is not feasible. In this approach, the  contractor—not the state—makes the initial determination of whether a  project is conducive to training. Although officials believe this approach  has increased opportunities for longer term employment for trainees, only  17 percent of Ohio’s 2009 trainees were women or minorities, including  trainees involved in Recovery Act projects. However, Florida also gives  contractors flexibility in hiring trainees. In 2009, Florida reported that 8  percent of participants were women and 88 percent were minorities (both  men and women). It is unclear if differences in how the states have  structured their job training programs, state demographics, or other  factors explain this difference.
section_title: FHWA’s Oversight Approach Does Little to Assess Program Results
paragraphs: FHWA has done little to assess its on-the-job training program nationally  and has not determined whether states’ efforts have led to increased  opportunities for workers traditionally underrepresented in the highway  construction workforce. Moreover, FHWA does not attempt to determine  how effective division offices are in their oversight of states efforts. FHWA  officials cited insufficient resources and higher priorities within the agency  as reasons for the lack of national oversight.
subsections: section_title: FHWA Has Not Established Clear Criteria For What States Should Accomplish
paragraphs: While FHWA has established regulations and broad guidelines for the  program, the guidelines are vague and it is unclear what states should  accomplish. The regulations and guidelines provide some assistance to  state transportation agencies in implementing their job training programs.  For example, FHWA regulations provide state transportation agencies  with a list of general factors—such as local demographics and project  size—to consider in selecting projects and determining the number of  workers to be trained to fulfill contract requirements. Similarly, as required  by statute, FHWA collects general types of information from states that  enable the agency to judge compliance with the requirements of the on- the-job training program. States provide information to FHWA about the  composition of the highway construction workforce and the results of their  training program, indicating progress towards state goals. However,  FHWA’s regulations for the program were written in the 1970s and,  according to FHWA officials and an industry official familiar with the  program, have not been updated to reflect changes in program  implementation. For example, the regulations provide that FHWA  headquarters shall establish suggested annual minimum training goals for  each state, but, in practice today, state goals are proposed by state  transportation agencies and approved by FHWA division offices, or not  set at all. FHWA Civil Rights officials told us that they have taken some  early steps towards considering possible revisions of the regulations,  including a change that would require state transportation agencies— rather than FHWA headquarters—to propose annual training goals.  Finally, the FHWA Resource Center, which provides a wide range of civil  rights training and support to state transportation agencies and FHWA  division offices, has developed materials that provide state transportation  agencies with some program definitions and requirements. For example,  the FHWA Resource Center has created general overview and frequently  asked questions documents aimed at helping state transportation officials  familiarize themselves with the program.
While FHWA’s current regulations and guidelines establish general  requirements for states, they do not establish clear criteria for what states  must do, what is optional, and the degree to which states can adapt their  programs to their own circumstances. Instead, FHWA has left the  interpretation of these regulations and guidelines, and subsequent  program implementation, largely to state transportation agencies and  FHWA division offices. For example, FHWA’s regulations and guidelines  list what factors should be considered in selecting projects and  determining the number of trainees, but they do not explain how a state  ought to consider factors like local demographics or project size when  setting a contract requirement or how a state’s goal should be  determined. The lack of clear criteria can make program implementation  more difficult for states. For example, Washington state transportation  agency officials told us that the lack of criteria within the existing  guidelines has made it difficult to structure their program and direct their  efforts.
Because FHWA has not established clear criteria, state transportation  agencies rely on their own judgment in implementing their programs and  key attributes of different state programs vary considerably, as previously  discussed. A number of federal and state officials familiar with the  program noted that some variation in how the program is implemented (1)  allows for states to adapt the program to their own circumstances and  preferences and (2) provides an opportunity for states and contractors to  innovate. For example, as previously mentioned, some states have  implemented contractor-based programs that provide contractors with  flexibility to move trainees across multiple projects, while other states  directly select which projects will provide on-the-job training.
Although providing states with flexibility may have some benefits, such as  allowing for state innovations, a lack of clear criteria for state programs  prevents FHWA from having a basis to judge the effectiveness of the  varying state programs, demonstrate program results, and identify and  share best practices. We have reported that federal program standards  that successfully address important and varied aspects of program  performance are key aspects of a results oriented performance  management. Thus, agencies should create a set of federal program  goals and standards that address important dimensions of program  performance that allow for the variation in individual state programs.  Without federal program standards to unify a federal agency’s efforts,  division offices and states are left to interpret the regulations, structure  their programs, and measure program achievements in whatever manner  they deem reasonable. Consequently, there is no basis to judge whether  varied state efforts are meeting federal standards for performance  management. For example, without set criteria for how states may or may  not structure their programs, it may be difficult to determine what program  structures are appropriate or what innovations are successful and should  be expanded. A lack of clear criteria about what the programs should  accomplish could hinder the program’s effectiveness in moving women,  minorities, and economically disadvantaged individuals into journey-level  positions.
section_title: FHWA Does Not Assess States’ Results
paragraphs: Although FHWA division offices receive results information from state  transportation agencies annually, the type of information that states  submit does not help determine the overall effectiveness of state  programs. As previously mentioned, states are required to provide FHWA  division offices with annual progress reports, also known as  accomplishment reports, that describe states’ and contractors’ efforts to  hire underrepresented individuals into the highway construction work  force. These accomplishment reports should contain information about  the previous year’s accomplishments, such as the number, demography,  and trades of trainees that contractors hired as well as the proposed goal  for the following year. These reports may also contain information about  the other actions the state has taken as part of its program. For example,  South Carolina’s 2008 accomplishment report included information on the  state’s update to its on-the-job training procedures manual. In addition to  annual accomplishment reports, states also provide FHWA with highway  construction workforce composition data, which summarize the number of  individuals contractors have hired in specific demographic groups in  specific job categories. This data provides a snapshot of the highway  workforce for a period in July—typically a month with a high volume of  highway construction activity—by trade and various demographic groups.
However, FHWA lacks an understanding of the effectiveness of individual  state programs and their results nationally for several reasons. First, it is  difficult to determine the overall effectiveness of state programs because  the accomplishment information that FHWA receives does not include all  states and is of limited usefulness in determining how well a state is  addressing underrepresentation. Based on our review, states that  provided annual accomplishment reports on their training program’s  achievements almost all included some form of quantified results  information. For example, of the 30 reports that describe 2009 program  results, 29 included some quantified information on the state’s  accomplishment. These reports frequently included data on trainee  enrollment, completion, and demographics. Of the 30 state reports, 18  included annual statewide goals; 17 of these 18 reports also included  achievement data regarding whether the state met that goal or not. While  a little more than half (17 of the 30) of the state reports we reviewed  included the state’s accomplishments and the annual statewide goal they  were striving towards, FHWA could not assess whether 13 of those state  transportation agencies successfully accomplished their goal. Very few of  the annual reports we reviewed included information on the extent to  which underrepresented groups participated in the program; the reports  that did (2 of the 30) included cursory information. As FHWA has not  provided states with clear criteria on how to complete these reports,  states included the information they chose to or that was requested by  their FHWA division offices.
Second, states submit accomplishment information with a wide range of  different output terms and different demographic and trade classification  categories (see fig. 2). For example, states may have listed their  accomplishments in different output terms such as hours, enrollees, or  graduates. Furthermore, even when states used similar terms—such as  training “spots” assigned to highway construction projects—it was not  clear that states were defining the term in the same way. This lack of  consistency prevents comparisons between states or aggregation of  accomplishment data across states. Besides different output terms, states  also reported information using different demographic and trade  classification categories. The 19 reports (out of 30) that provided  demographic data for trainees in 2009 used different demographic  categories. For example, Montana reported trainee demographics using  the general category of minority whereas some other states reported  more specific categories such as Latino, African American, Native  American, Asian, or Pacific Islander. Additionally, some states included  data on the trades of trainees, but these trades were not defined in the  reports and may not necessarily be comparable. Twenty of 30 state  reports contained trade data in 2009; however, the reports used different  trade categories. For example, Kansas reported trainees in multiple  different equipment operator positions—scraper, crane, and tractor  operator, for example—whereas other states, such as Idaho, listed  “operator” as one category. Consequently, though no fault of individual  states, it is difficult to consolidate or compare the trade data in these  reports.
Third, states also submit FHWA highway construction workforce  composition data, which provide a snapshot of the highway workforce for  a period in July by trade and various demographic groups. However, the  data that FHWA receives from states do not include useful information on  which groups are underrepresented in which trades. According to a state  official familiar with the program, the highway construction workforce  composition data that states report do not separate the number of  trainees and apprentices hired as a result of the training requirement  versus those hired independently of the training requirement. In other  words, the highway workforce composition data that states submit merge  the number of trainees and apprentices who work on projects with training  requirements as well as projects that do not have training requirements.  State officials told us that some trainees and apprentices reported by  states and contractors cannot be attributed to training requirements, but  might have been on the job independently of any training requirements.  As a result, the workforce composition data that states and contractors  submit are not effective in capturing information on how well states and  contractors are addressing underrepresentation within highway  construction work trades. FHWA civil rights officials acknowledged that  these data are insufficient and created a working group made up of  FHWA division and state transportation agency officials to examine how  this information could be improved. In April 2010, the working group  implemented several changes to improve the highway construction  workforce composition data collected from states, including revising the  ethnic and racial categories and making the reporting forms easier to  read.
As a result of the limitations described previously, FHWA is not meeting  governmentwide guidance, which states that management should track  major agency achievements and compare these achievements to plans,  goals, and objectives established under the Government Performance  and Results Act of 1993. Also, managers need to compare actual  performance to planned or expected results through the organization and  analyze significant differences. These control activities are an integral  part of an entity’s planning, implementing, reviewing, and accountability  for stewardship of government resources and achieving effective  results.
section_title: FHWA Does Not Assess How Well Its Division Offices Oversee States’ Efforts and Results
paragraphs: Although FHWA division offices fulfill a number of important oversight  responsibilities, FHWA does not have a clear overall understanding of  how division offices are carrying out their oversight of the on-the-job  training program. As previously mentioned, FHWA headquarters relies  heavily on division offices to provide oversight of state programs and to  ensure that states establish apprenticeship and training programs that are  targeted to move women, minorities, and disadvantaged individuals into  journey-level positions. We found many instances in which civil rights staff  in FHWA division offices provided oversight of state and contractor efforts  to meet training requirements on federal-aid projects. Various FHWA  division officials we spoke to, for example, reviewed contractor training  and enrollment plans, approved new training classifications, provided  direction to states in establishing appropriate annual goals, monitored and  assessed state achievements by reviewing state reports, assisted states  in assessing contractor compliance with training requirements, and  recommended use of administrative tools and penalties against  contractors not in compliance.
However, it is unclear to what extent division offices are monitoring states  primarily because FHWA has neither provided clear guidance on how  divisions should carry out their oversight responsibilities nor has FHWA  held division offices accountable in meeting performance criteria such as  ensuring that state transportation agencies establish active training  programs. In our review of various state programs, FHWA division offices  executed their oversight responsibilities with different levels of rigor. For  example, the FHWA division in Missouri told us that they were actively  involved in reviewing contractor’s good faith effort proposals. Conversely,  some FHWA division offices had been lax in their oversight of state  transportation agencies. For example, we found that FHWA division  offices had allowed two states to go through extended periods without  active training programs. These states have re-established their  programs, in part, due to the insistence of the FHWA division offices.  Previously, FHWA had allowed them to lapse without any objection.  According to an FHWA official familiar with the program, New Mexico  went through a period starting in the 1990s without a program and only  reinstated training requirements in federal-aid highway contracts at the  insistence of the FHWA division in 2009. The program may have been  discontinued because a change in state leadership may not have  considered the program a priority, according to an official. Similarly,  Texas department of transportation officials told us the training program in  Texas was formally established in 2006 and implemented in 2007 when a  new civil rights program specialist came on board and said that Texas  needed to develop it. They also stated that prior to that Texas had an  optional program because the Texas FHWA division civil rights specialist  did not require the state transportation agency to develop a training  program. FHWA civil rights officials acknowledged that there may be  instances of states lacking active training programs due to limited  resources within the Office of Civil Rights to manage state programs and  the low priority states may assign to these programs relative to other civil  rights programs.
In addition, some division offices did not adequately oversee states’  annual training goal setting, a central component of existing program  regulations and guidance, but one that some program stakeholders see  as not valuable. Some division offices played key roles in helping states  determine appropriate state goals while others were less rigorous in  ensuring that the state transportation agency established appropriate  goals. For example, an FHWA official familiar with Alabama’s state  program told us that the division office plays an active role in determining  whether the training goal is reasonable by examining past reports and  historical data and looking to see if the state transportation agency  achieved its training goal the previous year. However, other states like  Arizona, California, New Mexico, and Ohio, were exempted by their  FHWA division offices from having to establish an annual statewide goal.  The lack of consistent adherence to this requirement can be explained in  part by the lack of clear criteria on how division offices should oversee  state programs and differing views among program stakeholders about  the usefulness of having state goals. While some state officials told us  that setting state goals helps state transportation agencies track their  progress, others—including officials from the Office of Civil Rights— stated that individual contract goals were sufficient and statewide goal  setting did not add value. Currently, headquarters cannot and does not  assess how well division offices oversee states’ adherence in establishing  a state goal because headquarters has not established clear criteria that  all states must establish training goals. If FHWA headquarters were to  strengthen criteria by requiring states to establish annual goals, then it  would provide headquarters a basis—specifically by determining how  adequately divisions oversee states goal setting—upon which to assess  whether division offices were consistently holding states to federal  criteria.
Although FHWA division offices execute a number of important oversight  responsibilities to ensure that states and contractors meet training  requirements on federal-aid projects, division office performance should  be monitored by headquarters. We have reported that it is a useful  practice to assess the oversight activities of division offices in order to  monitor states’ progress in meeting overall program objectives. In the  absence of clear criteria on how division offices should oversee state  programs and how to hold state transportation agencies accountable,  FHWA will be unable to fully assess whether division offices are effective  in ensuring that states meet program requirements.
section_title: FHWA Cites Insufficient Resources and Higher Priorities as Reasons for Lack of National Oversight
paragraphs: The Office of Civil Rights focuses on policy while paying minimal attention  to national program oversight in part because the office lacks the  resources to provide adequate oversight to state transportation agencies,  according to FHWA officials. The FHWA Office of Civil Rights only has  one full-time equivalent employee that oversees both the on-the-job  training and supportive services programs. Also, FHWA officials told us  that the agency has provided little national oversight to state  transportation agencies because it has placed a higher priority on other  civil rights programs that are at higher risk—based on FHWA’s corporate  risk assessment—than the on-the-job training program. FHWA identified  higher risk programs as including the Disadvantaged Business  Enterprise, Title VI of the Civil Rights Act of 1964, and the Americans with  Disabilities Act programs. Because headquarters is responsible for  overseeing these other civil right programs, FHWA has not been able to  provide sufficient national oversight to the on-the-job training program,  according to FHWA officials. Nevertheless, FHWA has recently  undertaken several measures that may strengthen its oversight and  assess program results. For example, FHWA surveyed all 52 division  offices in 2011 and conducted baseline assessments of state civil rights  offices in 2008 to gather basic information on the states. According to  FHWA civil rights officials, the survey—which focused on the supportive  services program—could identify training and workforce development  issues that may need to be addressed by the division office. Additionally,  FHWA officials have recently established what they describe as an  informal partnership with the Department of Labor to explore how local  career centers might assist tracking the results of the on-the-job training  program and the supportive services program, among other things.
Although limited resources and competing priorities are valid concerns,  especially for a smaller program, FHWA’s oversight approach needs to  provide the agency with a basic assessment of how well the program has  functioned on a national basis. In the absence of national oversight from  headquarters, it is unclear what impact the program has beyond the  individual state-level.
section_title: FHWA Provides Funding for Activities Supporting Job Training Programs, but Overall Stewardship is Limited
paragraphs: FHWA grants about $10 million each year for supportive services that  directly and indirectly support job training activities. However, even after  considering the small size of the supportive services program, we found  that FHWA’s stewardship is limited. The agency collects some  information on services provided, but does not always determine results  achieved and does not routinely consider grantees’ performance in  making funding decisions. FHWA officials said they have few resources  devoted to the program and results are sometimes difficult to aggregate.
subsections: section_title: Supportive Services Provide Direct and Indirect Support to Training Programs, but It Is Unclear How Many People Receive Such Services
paragraphs: Currently, FHWA provides about $5 million to $6 million each year for  services that directly support job training programs (through the  discretionary grant program) and about $4 million each year for programs  that indirectly support job training programs (through the discretionary  grant program, summer institutes, and internships). Regarding the  discretionary grant program, the agency typically provides funds to about  20 states each year, averaging about $240,000 for each state. FHWA  does not attempt to determine, and we could not establish, how many  people are participating in the program since FHWA does not collect this  information. From reviewing applications and considering the degree to  which FHWA provided requested grant amounts, we estimate that roughly  2,000 people may have received some direct supportive services in any  one year. FHWA officials noted that while they do not know how many  individuals participate in the supportive services program, they estimated  that several thousand individuals may receive services in a given year,  depending upon the number of projects funded, as well as other factors.  FHWA officials stated because these activities can vary considerably from  state to state and they lack the resources needed to determine such a  count, they have been unable to determine the total number of  participants.
We could not determine with any confidence how many people received  supportive services through the discretionary grant program because  FHWA has not collected this information. Also, we could not analyze the  demographics of individuals served by this program. However, a rough  indication of the services provided can be gleaned from states’  applications for funding (see table 3). About three-quarters of the states  proposed to provide job placement and other outreach activities (e.g.,  referral services or employment counseling) and preapprenticeship type  training (e.g., safety training, overviews of highway construction  equipment, or training certifications). To a lesser extent, states proposed  to provide life skills training, physical construction tools, transportation,  remedial math or language training, and child care. For example,  Arkansas proposed a variety of services to support a 20-person class for  heavy equipment training and proposed providing tools, boots,  transportation, and child care support to all participants employed on a  federal-aid highway construction project.
FHWA’s discretionary grants also fund some activities that support job  training indirectly. For example, a number of states have used supportive  services funds for construction career days, which expose high school  students to various areas of construction, including heavy highway  construction, with the intent of increasing interest among women,  minorities, or economically disadvantaged individuals in the transportation  industry and highlighting potential career opportunities. Applying the  same approach used to estimate the number of individuals receiving  direct supportive services, we estimate that roughly 8,000 individuals,  primarily through the construction career days, are exposed indirectly to  highway construction jobs.
Similarly, FHWA provides more than $2 million each year to universities  through state transportation agencies for summer transportation institutes  that expose students to the transportation industry broadly, not just  highway construction. Activities include exposure to engineering and  construction concepts, presentations by industry groups, and site visits to  local transportation facilities, such as train stations and transportation  simulators. We estimate that roughly 1,200 people participated in the  activities in 2010 across 32 states, the District of Columbia, and Puerto  Rico. Finally, FHWA funds about 100 internships each year at agencies  within the U.S. Department of Transportation (e.g., Federal Transit  Administration and Federal Railroad Administration) and at state  transportation agencies, costing about $1.5 million annually. These  internships expose students to transportation careers and encourage the  pursuit of transportation-related studies in college. In the activities  supported by the fiscal year 2010 summer institute and internship  programs, we estimate that female and male participation in each  program was relatively equal, and that a significant number of the  participants served by these two programs were members of minority  groups (see fig. 3). While these activities can increase awareness of  highway construction job opportunities and lead to future work force  entries, the effect of these efforts, if any, may not be apparent for years  since students typically return to school upon completion. They may not  enter the workforce for a year or more, whether in construction or other  transportation fields or in a nontransportation field.
FHWA has not used any of the information available to it to determine the  overall effectiveness of its supportive services activities. FHWA does not  track how many of the state grantees have achieved their goals in directly  supporting on-the-job training. Although states are required to provide  reports on their accomplishments to FHWA division offices, this  information is not typically relayed to the FHWA officials making grant  decisions. FHWA has only received about one-third of the final fiscal year  2008 accomplishment reports for the discretionary grant program, and  FHWA officials noted that a low response rate is typical. Officials  responsible for making the grant funding decisions explained that they do  not routinely receive monthly or quarterly progress reports from the  division offices but can request them on an as-needed basis. They also  noted that occasionally, they request these reports to identify promising  practices. However, these officials noted that they have limited tools to  induce states to meet their reporting requirements. Furthermore, they  explained that some states do not provide reports because of delays in  awarding contracts, noting that often contracts are not in place by the  time the reporting requirements have begun, and thus the states have no  results to report. FHWA officials recognized that they did not have  sufficient information on the results and the accomplishments of these  programs, and surveyed the division offices in 2011 to obtain information  on what grantees have accomplished and what approaches have worked  well. As of August 2011, FHWA officials were still collecting and analyzing  responses from the states, in part, due to additional time needed by the  division offices to complete the survey as well as comply with additional  reporting requirements for other FHWA programs.
Moreover, FHWA has not established how state-reported data can be  used to evaluate whether grantees have met FHWA’s overall program  objectives to increase the overall effectiveness of each state’s job training  program and enhance the training opportunities for women, minorities,  and economically disadvantaged individuals. FHWA officials stated that  they have not analyzed the extent to which grantees are achieving their  performance goals, citing a variety of reasons, including a lack of  resources and other competing programmatic priorities. FHWA officials  anticipate that the survey, which they expect to complete in the summer  of 2011, will provide much of the information needed to analyze the  effectiveness of the program.
Regarding indirect supportive services, FHWA officials noted that they  have collected some data on the number and demographics of the  individual attendees for the summer institute and student internship  program, and further heard anecdotally that the indirect services provided  (e.g., awareness programs) are effective in persuading students to  consider a career in the transportation field. However, for reasons noted  previously, FHWA lacks this information for the indirect activities  supported by the discretionary grant program, even though the grant  program receives a sizable portion of FHWA’s supportive services  funding. Moreover, FHWA officials do not track and do not know to what  extent indirect supportive services are leading to jobs in the highway  construction industry. Officials noted that they do not track the outcomes  for a number of reasons, including privacy concerns with collecting data  on minors.
FHWA’s past efforts to determine the effects that its supportive service  funding provided have been limited, but we recognize that doing so is  complicated. For programs designed to support diverse state or local  activities, such as the supportive services program, finding common  performance metrics to describe the overall program’s effectiveness can  be challenging. We have reported on options for building accountability  provisions into grants that help balance states’ flexibility to select a mix of  activities and services that will best allow them to achieve a particular  national outcome with accountability for achieving that outcome. These  options include emphasizing results-based evaluation (e.g., workforce  retention) in addition to examining output activities (e.g., the number of  participants receiving a service). For example, under a results-oriented  evaluation, FHWA could specify national goals and objectives, set up a  process for establishing them, or adopt some combination of the two.  FHWA officials requested results information through its survey of  supportive services providers in the discretionary grant program. A  results-orientation requires ongoing information. FHWA officials indicated  that they plan to annually gather some of the basic information that was  solicited from states in the current survey. In addition, FHWA states that it  requires that grantees establish performance goals and measurable  objectives for their proposed activities, and report on progress towards  meeting these goals. However, this approach, to the extent it is  employed, focuses on outputs rather than end results.
Program results are important for making budgetary and programmatic  decisions. High-level decision makers, such as Members of Congress  and agency heads, need to know which programs are achieving their  goals and objectives to make informed decisions about whether to  expand, reduce, or maintain funded activities. Furthermore, agencies  are responsible for ensuring that grantees use federal funds for eligible  activities. In the absence of reported program results, FHWA cannot know  whether funds have been used effectively and appropriately.
section_title: FHWA Has Not Sufficiently Used Results to Inform Funding Decisions
paragraphs: FHWA often funds the same projects for supportive services year after  year. For example, 17 of the 26 projects (65 percent) that received grants  in fiscal year 2010 also received them in 2008, 2009, or both years.  FHWA reviewers of discretionary grant applications are instructed to  score the scope of the work, performance goals, objectives and  measurements, budget summary, monitoring and evaluation plan, and  reporting plan. However, past performance information is not required of  applicants in the application nor is past performance one of the criteria  scored during funding reviews. Furthermore, the solicitation issued to  states does not request information from grantees about how they have  used prior FHWA grants, or the resulting effectiveness of their activities.
FHWA officials told us that they informally consider performance  information when screening discretionary grant applications. For example,  they stated that they may use data from the accomplishment reports to  look for potential problems. In addition, officials noted that states whose  accomplishment reports show positive results provide FHWA assurance  that the state may be able to successfully implement additional programs,  which can influence the state’s chance of receiving funding in a  subsequent year.
However, as noted previously, FHWA decision makers have received a  limited number of progress and final reports, which are the primary source  of performance information for discretionary grants. Furthermore, FHWA  decided to require only one final oversight report for four funding cycles  occurring in fiscal years 2009 and 2010, which included two rounds of  Recovery Act awards. FHWA officials stated that they made this decision  in order to streamline the reporting requirements for grantees.  Accordingly, FHWA has only received a few accomplishment reports for  this period. As a result, FHWA will not be able to determine the effects of  the individual award decisions until after all rounds of funding are  complete, which, in some cases, will be 2 years after the decisions were  made, following subsequent funding decisions by FHWA.
Similarly, for the summer transportation institute program, FHWA reviews  the annual reports submitted by states and informally assesses  performance, and it considers past performance during its funding review  to some degree. Like the discretionary grant program, many of these  applicants are funded year after year. For example, 34 of the 53  applicants (64 percent) funded in fiscal year 2010 were funded in each of  the previous two fiscal years. According to FHWA officials, the division  offices look at earlier annual reports to see if the same host site is  soliciting funding for the subsequent year, and try to determine whether  the host achieved its program objectives. However, like the grant  program, past performance information is not required of applicants in the  application and performance is not one of the criteria scored during  funding reviews. Furthermore, the solicitation issued to states does not  request information from grantees about how they have used prior FHWA  grants and the resulting effectiveness of their activities.
For the summer internship program, FHWA evaluates a prospective  intern’s performance through the student’s academic and professional  work history, which, among other things, entails a review of the  applicant’s resumé, references, and academic transcript. FHWA also  surveys students at the end of the program to gather information on how  well the program worked, and to suggest recommendations for how to  address these issues in the future.
Given that many of the activities funded through the program are funded  repetitively, good practices suggest that the use of performance  information can inform and improve recipient selection approaches, even  for a smaller program. While the $10 million provided annually through  this program is much smaller than many other federal grant programs,  integrating results information can help FHWA identify approaches that  work or leverage good practices to improve the program, and decrease  funding to poor or marginal performers (or encourage them to improve  their performance).
FHWA is considering whether to change the grant funding approach from  discretionary to formula-based funding. However, using performance data  could still help ensure accountability for results. Under the formula  approach that FHWA is considering, each state would receive some  supportive services funds, in contrast to the 20 or so states that receive  funding each year under the current approach. Using performance data,  FHWA could help states that have not participated in the program to more  easily identify previously successful approaches to effectively  implementing their own supportive services programs. The performance  data could also be used to determine whether funds should be withheld  from or provided conditionally to any currently low-performing states.  FHWA hopes to implement the new funding approach in fiscal year 2012.
section_title: Conclusions
paragraphs: FHWA’s on-the-job training program and its counterpart, the supportive  services program, represent decades-long federal initiatives to increase  participation of women, minorities, and economically disadvantaged  individuals in the highway construction industry. FHWA allows each state  to tailor its job training activities to fit its unique demographic, geographic,  and social characteristics, which can spur innovation and help states  meet their needs. However, FHWA has not established clear criteria for  what states must do, what is optional, the degree of flexibility states have  in implementing their programs, and what they are intended to  accomplish. Within state transportation agencies, FHWA, and state  partners involved in the program, we found officials working toward the  programs’ success. However, FHWA does not know what these programs  have accomplished or how effectively these activities have advanced the  broader goal of bringing underrepresented individuals into the highway  construction workforce, because FHWA has performed only limited  assessments of these programs over their nearly 40-year history.  We  recognize that doing so may be difficult.  FHWA’s current activities—such  as its survey regarding the supportive services program—are positive  steps towards making such an assessment.
While it is appropriate for FHWA headquarters to rely extensively on its  division offices to oversee state on-the-job training programs, such a  decentralized management approach does not relieve FHWA  headquarters from the responsibility of providing leadership and a  national perspective for its programs. Even though the on-the-job training  and supportive services programs are relatively small compared to  FHWA’s other programs, we would still expect FHWA to conduct  oversight to ensure program goals are met and at a scale appropriate for  the size of the programs. A necessary first step is for the agency to  establish clear criteria—existing program criteria are not clear—for states  on what they are to accomplish and for its division offices on how they are  to oversee state activities. These criteria could continue to allow for  flexibility in state programs, as FHWA deems appropriate.  In doing so,  FHWA needs to provide guidance on state reporting that will lay the  groundwork for national assessments of results. Likewise, evaluation of  supportive services results could inform funding decisions. Such actions  will result in FHWA being able to better ensure that program goals are  being met and would allow them to improve program accountability.
section_title: Recommendations for Executive Action
paragraphs: To establish accountability for meeting the programs’ goal of increasing  the participation of traditionally underrepresented groups in the highway  construction workforce, we recommend that the Secretary of  Transportation direct the FHWA Administrator to take the following three  actions:     Strengthen criteria—through regulations, guidelines, or other  mechanisms—so that states have a clear understanding of how the  on-the-job training program should be implemented and the results  state programs are intended to accomplish.    Create and implement an oversight mechanism that (1) holds states  accountable for meeting federal training criteria and (2) clearly  stipulates how FHWA will assess state program effectiveness,  including what type of program achievement data states are to submit  and how such data will be used. This oversight mechanism should  include assessing the effectiveness of its division offices in overseeing  state activities.   For the supportive services program, develop an approach to (1)  evaluate the extent to which grantees have met their proposed annual  goals and (2) integrate the results of this evaluation into FHWA’s  funding decisions for supportive services programs.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Department of Transportation for  review and comment prior to finalizing the report. The Department of  Transportation generally agreed with the recommendations and provided  technical comments, which we incorporated as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date.  At that time, we will send copies of this report to  congressional committees with responsibilities for surface transportation  issues, the Director of the Office of Management and Budget, the  Secretary of Transportation, and the Administrator of the Federal  Highway Administration. In addition, this report will be available at no  charge on GAO’s Web site at http://www.gao.gov.
If you have any questions about this report, please contact me at (202)  512-2834 or wised@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: In this report, we assessed the Federal Highway Administration’s (FHWA)  on-the-job training program and the associated supportive services  program. To do so, we focused on the extent to which (1) states’  implementation of the FHWA on-the-job training program provides  training and career enhancement opportunities for traditionally  underrepresented groups to work on federal-aid highway projects, (2)  FHWA has overseen the program and demonstrated results, and (3) the  supportive services program established by FHWA has assisted state job  training and apprenticeship programs. In response to your interest, we  also gathered information on financial incentives and penalties states use  to encourage contractors to provide job training.
Additionally, for background, to get an understanding of historical  participation in construction for women and minority groups between 1985  and 2010, we analyzed data provided by the Department of Labor’s  Bureau of Labor Statistics on industry employment by race, ethnicity, and  gender. While the data used are not strictly comparable over time due to  changes in industry classifications systems, we used the data to describe  a general trend in employment in the construction industry. We believe  the data are sufficiently reliable for our purposes, which were to provide  historical context on the industry.
subsections: section_title: Implementation of Job Training Program
paragraphs: To determine how FHWA’s on-the-job training program has provided  training and career enhancement opportunities for traditionally  underrepresented groups on federal-aid highway projects, we reviewed  legislation, including the Federal Aid Highway Act of 1968, which  authorized the program, and the Safe, Accountable, Flexible, Efficient  Transportation Equity Act: A Legacy for Users; the program’s  implementing regulations; FHWA guidance such as its contract  compliance manual; and our reports on performance management. We  interviewed officials from FHWA’s Office of Civil Rights and its Resource  Center to understand the agency’s overall program philosophy. In  addition, to gain a basic understanding of how the program operates and  its environment, we interviewed officials from FHWA division offices and  state transportation agencies in 11 states. We selected these states  based on a variety of factors, including unionization, the state’s size, the  extent to which a state had been awarded funding for supportive services  and used workforce development funding, and geographic location. We  further examined more closely how these programs were implemented in  four of these states: Missouri, Pennsylvania, Texas, and Washington. We  selected these states because they differed in demographics;  unionization; how job training is structured within the state, including any  unique practices; and the extent to which the state participates in the  supportive services program. We also interviewed representatives from  local chapters of industry groups, such as the Associated General  Contractors, as well as members of local unions, including laborers, craft- workers, and heavy equipment operators. The information gathered  through our review of state programs is intended to illustrate these  programs; it is not generalizable to all states or to other states with similar  characteristics.
To develop an order of magnitude estimate of the number of individuals  participating in the program in a given year we used information in annual  accomplishment reports that states submitted to FHWA. In making these  calculations, we estimated the total number of participants from the  sample of reports provided by FHWA. We also used this data, along with  our interviews with FHWA and state transportation officials, to provide  examples and illustrations of the type of data collected by the states for  these programs, including activities implemented, the financial incentives  and penalties used by states, and the extent to which the American  Recovery and Reinvestment Act of 2009 (Recovery Act) provided job  opportunities. While we cannot independently verify all of the data  collected by the states and submitted to FHWA, because the data is used  solely to provide context on the overall program, we believe the data is  sufficiently reliable for our purposes.
section_title: FHWA Oversight of Job- Training Programs
paragraphs: To assess how FHWA oversees job training programs to determine  program achievement, we reviewed the program standards and outcome  measures required in FHWA’s regulations and program guidance and  compared them to good management practices, such as the assessment  of oversight activities we have identified in other programs. We also  compared the information on training activities and program results  contained in the state accomplishment reports to requirements detailed in  FHWA regulations, guidance, and other instructions communicated to  states. Because of the differences in how states collected and reported  data, we were unable to compare the results across states. Accordingly,  in order to understand trends, similarities, and differences across  implemented programs we relied on interviews with FHWA and state  transportation officials to provide examples of the type of oversight  activities implemented by each state. The information we obtained from  these officials is not generalizable across all state programs.
section_title: Supportive Services Program
paragraphs: To determine the extent to which supportive services have assisted state  job-training programs, we reviewed how FHWA has implemented and  overseen the three components of the program: (1) the discretionary  grant program, (2) the national summer transportation institute program,  and (3) the summer transportation internship program for disadvantaged  groups. We analyzed FHWA budget information to determine the amount  of funding provided to the three programs. In order to assess the data’s  reliability, we interviewed department officials and reviewed departmental  audit reports about the system which processes the data. We determined  that this data was sufficiently reliable for our purposes.
For the discretionary grant program, we analyzed legislation, such as the  Federal Aid Highway Act of 1970, which authorized the program,  regulations, and other FHWA guidance, such as FHWA’s supportive  services handbook. We reviewed the agency’s approach to requesting  and evaluating grant proposals for the program. Specifically, we analyzed  program criteria for funding proposals and incorporating performance  information by reviewing the agency’s request for proposals, rating review  sheet, and other information communicated to prospective applicants. We  compared FHWA’s use of performance information to good management  practices that we have identified. We reviewed the 153 grant proposals  that received funding from 2008 through 2010 and categorized the  activities receiving funding. In addition, we reviewed FHWA’s guidance for  reporting the results of programs. However, because FHWA could only  provide about one-third of the annual grantee accomplishment reports for  fiscal year 2008 and only a handful of reports for the following funding  cycles, we were unable to determine the extent to which states complied  with this guidance. In addition to FHWA officials, state transportation  officials, and industry groups identified previously, we also interviewed  supportive services grantees in Missouri, Pennsylvania, and Texas.  These results are not generalizable to all states or to other states with  similar characteristics. In order to estimate the number of individuals  participating in the program in a typical year, we reviewed the goals in the  funded proposals for activities proposed from fiscal years 2008 through  2010, excluding Recovery Act projects. We classified each of the  activities as either directly supporting job-training or current trainees (such  as preapprentice training, stipends for child care and transportation, and  providing job-related equipment) or indirectly supporting job training (such  as career day events). For each year, we added the numerical goals for  each type of job training activity and averaged our results over the 3-year  period. Our calculation represents a very rough estimate of the number of  participants in this program. Finally, we reviewed FHWA’s proposal to  change how it distributes funds through the program.
For the summer transportation institute program, we reviewed FHWA  guidance and other documents on the program’s history and  implementation. We reviewed FHWA’s request for proposals and  evaluation criteria and compared the requirements for the use of  performance information to the use of such information for programs that  we have identified. In addition, we analyzed the results described in  annual reports submitted by programs in 32 states, the District of  Columbia, and Puerto Rico for fiscal year 2010. Specifically, we used  these data to estimate the total number of participants in the program,  and provide demographic information, such as ethnicity, gender, and  education level, of the participants, as well as examples of the types of  activities undertaken through this program. As the data was reported  individually by each state, we limited our use of these data to background  on the participants and believe these data are sufficiently reliable for this  purpose. Finally, we interviewed officials from FHWA and two program  sites to understand how FHWA selects, implements and oversees these  programs.
Similarly, for the student internship program, we reviewed FHWA  guidance and other documents, including the program and evaluation  criteria, and compared FHWA’s use of performance information to good  practices we have identified. We also reviewed FHWA’s analysis of the  results of the program, including demographic information about the  participants, such as gender, ethnicity, proposed major, and location of  the internship. We limited our use of these data to background on the  participants and believe these data are sufficiently reliable for this  purpose. We also interviewed FHWA officials to understand how the  agency oversees and implements the internship program.
We conducted this performance audit from August 2010 through  September 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix II: Some Financial Incentives and Penalties Are Available to Encourage On-the- Job Training Performance
paragraphs: While the on-the-job training program does not have a distinct funding  source, job training expenses on federal-aid highway projects are eligible  for federal dollars in the same way as the typical 80/20 federal/state split  of project costs. Each state has discretion in how contractors will be  reimbursed for these training activities, if at all. Some states have tried to  create financial incentives for contractors to perform job training, but  these incentives are relatively small. For example, some states reimburse  contractors for the hours of training performed. Reimbursement rates  range from 80 cents per hour, in Arizona for example, to $3.50 per hour in  Missouri, in the states we reviewed. Other states include the job training  requirement as a bid item in the request for contract proposals, allowing  contractors to name their price for meeting the requirement. Officials and  contractors that we spoke with generally agreed that reimbursement rates  were too low to be much of an incentive to contractors. Some contractors  simply do not bother submitting the paperwork required to claim a  reimbursement. Likewise, in cases in which job training is a bid item, we  heard that contractors will commonly bid a minimal amount for this item,  preferring to keep their overall price as low as possible to win the contract  award.
Financial incentives can also take a penalty form, although this approach  appears to be less common. For example, Missouri has recently  established a penalty system for nonperformance. In this system, the  contractor will be held liable to pay $7 for every hour of training that  should have been provided but was not. To date, the state has rarely  levied such a penalty. On occasion, states may threaten to withhold  contract payments if a contractor is not performing. In our interviews with  state officials, we found this practice is rare and states are reluctant to do  so. Officials noted that generally the threat to withhold funds is sufficient  to correct problems.
section_title: Appendix III: Recovery Act Provided Some Job-Training Opportunities
paragraphs: Through the American Recovery and Reinvestment Act of 2009  (Recovery Act), the federal government provided states with additional  funds for highway construction projects and the same general training  requirements applied to these projects. However, to some extent, these  projects may have been less conducive to training than projects funded  through regular annual federal-aid highway apportionments. Because the  Recovery Act prioritized projects that could be completed quickly, many  projects focused on resurfacing or other less complex and lengthy  projects. Short projects tend to require few workers overall and less time  per trade to complete, minimizing the opportunity to train workers on the  project. In the four states we reviewed more closely, officials generally  agreed that the Recovery Act-funded projects led to some job training  opportunities, but these opportunities were generally shorter than  regularly funded projects.
Several states reported to FHWA that Recovery Act-funded projects  employed trainees. Three of the 30 states we reviewed significantly  increased their annual achievement due to Recovery Act funding. For  example, more than half of the trainees in Texas worked on Recovery  Act-funded projects in 2009. Specifically, 156 individuals worked as  trainees on 142 Recovery Act-funded projects. At the end of 2009, 101 of  these trainees remained actively enrolled in the program and 36 had  graduated. In addition, some states made special efforts to ensure  training occurred on Recovery Act projects. For example, Texas created a  separate job training program for Recovery Act projects that assigned  training goals to individual projects. Texas already had a contractor-based  program in place, but wanted to directly track job training results that were  associated with Recovery Act funding.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, James Ratzenberger and Teresa  Spisak (Assistant Directors), John Stambaugh, Tina Paek, Matthew Voit,  Dianne Blank, Elizabeth DeVan, Sara Ann Moessbauer, Amy Rosewarne,  and Crystal Wesco made key contributions to this report.