On-the-Job Training: Federal Highway Administration Needs to Strengthen Program Assessment
Released: 2011-10-07T00:00:00

Letter

Background

Building highways requires a diverse collection of skill sets and expertise. The highway construction industry is composed of a variety of different trades such as equipment operators, mechanics, truck drivers, ironworkers, carpenters, and cement masons. These trades are not specific to highway construction but also apply to other areas such as general building construction. Historically, many of these trades have been dominated by white males. Over the last 25 years, participation of women in the construction industry has been relatively stable, averaging around 8 to 10 percent. Among minority groups, while the percentage of African Americans participating in the construction workforce has been relatively constant (5 to 7 percent of the workforce), the proportion of Latinos in the industry has increased significantly and is estimated to comprise about a quarter of the construction workforce.

In 1968, Congress authorized a job training program, which required that states receiving federal aid for highways implement apprenticeship and training programs that ensure equal employment opportunities without regard to race, color, creed, national origin, or sex. FHWA has targeted this job training program to women, minorities, and economically disadvantaged individuals to provide these groups with highway construction-related skills to reach journey-level status. As a nondiscriminatory program, it cannot exclude any individual from participation. The Secretary of Transportation must certify that states have such a program in place. While the program does not have a distinct funding source, job training expenses on federal-aid highway projects are eligible for federal dollars, just like other project costs. According to FHWA officials, each state has discretion in how much or little to spend on this program, as well as how a contractor will be reimbursed, if at all, for training activities on its highway projects.

State transportation offices are responsible for providing their state program’s framework. Individual states establish overall training goals for the program such as articulating the number of training slots on selected federal-aid highway construction contracts they expect to award during the coming year. These state agencies also determine which federal-aid highway construction contracts will require that training is provided to a minimum number of trainees and thus which contracts include a training special provision. State transportation agencies—using on-site personnel such as contractor compliance specialists, engineers, and project inspectors—monitor and oversee contractors’ adherence to job training requirements.

Multiple FHWA offices are involved in the on-the-job training program. FHWA’s Office of Civil Rights, located in the agency’s headquarters office, provides leadership in setting policy for the program, and in this capacity has assigned oversight responsibilities to the FHWA division offices in each state and has established regulations for state transportation agencies to administer the program. This decentralized management structure—in which certain oversight responsibilities are assigned to division offices—is consistent with FHWA’s approach for the federal-aid highway program in general. Among the primary responsibilities of the division offices are to review the annual goals submitted by the states and oversee whether states are meeting their annual job training goals by reviewing accomplishment reports and annual contractor training reports. FHWA division offices also play a role in assessing the efforts of both the individual contractors participating in this program and the states to meet their respective job training goals and requirements. FHWA’s Resource Center also provides assistance to state transportation agencies and FHWA offices in implementing FHWA policy and programs by providing training on program requirements and other activities (see table 1).

While a state approves a contractor’s training approach, the contractor is responsible for meeting a training requirement. If a contractor fails to meet the training requirement stipulated in the contract, the state transportation agency can withhold payments or limit the contractor’s ability to bid on future contract opportunities. The amount of the sanctions or penalties will depend on the state transportation agency’s contract administration procedures. For example, in Missouri, the state transportation agency may levy fines against contractors that do not fulfill their training requirements.

Congress authorized the supportive services program in 1970 to supplement the on-the-job training program and support state transportation agency training programs by providing services to highway construction contractors and assistance to highway construction apprentices and trainees. According to FHWA, the main objectives of the program are to increase the overall effectiveness of each state job training program in connection with federal-aid highway projects and seek other ways to increase the training opportunities for women, minorities, and economically disadvantaged individuals. Depending on the needs and preferences in a specific state, services funded under this program may include basic training in the construction industry, stipends to support childcare or defray transportation costs for highway construction workers, transportation career awareness programs, and internships. FHWA is authorized to grant up to $10 million per year in this program.

FHWA funds some supportive services through a discretionary grant program. From 2008 through 2010, FHWA awarded 153 grants totaling about $37 million to 37 states, the District of Columbia, Guam, and Puerto Rico for supportive services through the discretionary grant program. According to FHWA guidance, supportive services activities must be connected to a federal-aid highway construction project in order to be eligible for funding. State transportation agencies must submit a statement of work to FHWA that details how the grantee proposes to use the requested funding. The proposals are required to establish measurable goals for results, although FHWA permits wide variation in defining these goals. According to FHWA, proposed activities must target their efforts towards women, minorities, or economically disadvantaged individuals to receive funding. According to FHWA guidance, applications are reviewed by panels consisting of FHWA officials and scored using criteria to evaluate the content of the statement of work, including whether the statement of work has defined end dates, how the needs of the supportive services program would be addressed through the proposed activities, and a monitoring plan. FHWA requires each grantee to submit regular progress reports to the respective FHWA division offices as well as a final accomplishment report within 60 days of the program’s completion, which provides information on previously established performance goals. According to FHWA officials, an interim accomplishment report is also requested from on-going activities when additional funding is requested during the annual grant solicitation process.

FHWA also uses the supportive services funding for transportation- related education and internship opportunities. FHWA distributes some funding to summer transportation institutes (located at universities and colleges), which provide a curriculum designed to introduce secondary school students to careers in all modes of transportation and encourage them to pursue transportation-related courses of study at the university level. There is also a summer internship program, wherein FHWA selects and places students from various colleges and universities in internships at one of the U.S. Department of Transportation’s modal agencies, such as FHWA, or at a state transportation agency. These programs are targeted at women, minorities, and economically disadvantaged individuals to provide hands-on experience and on-the-job training while working on current transportation-related topics and issues.

State Programs Provide a Range of Different Opportunities for Enhancing Skills and Vary in Their Results

The overall extent that FHWA’s on-the-job program has helped women, minorities, and the economically disadvantaged reach journey-level status in the highway construction trades is unclear, primarily because a limited amount of usable information is available on program results. However, based on the limited information that is available, state programs have created some new opportunities tailored to individual state circumstances. The skill enhancement opportunities provided by state programs vary across states by requiring different lengths of training, relying on a diversity of entities and training structures to deliver the training, and giving contractors various flexibilities in how and what skills they provide to individuals. Additionally, state programs have varying degrees of focus on reducing underrepresentation. While we cannot definitively report on how well women, minorities, and economically disadvantaged individuals are moving to journey-level status in highway construction, we can describe some of the approaches states have chosen to implement the program and make some observations about how the program has contributed to enhancing the career prospects of individuals from the targeted groups.

Wide-ranging State Programs Provide Different Opportunities for Enhancing Skills

One way state programs differ in providing opportunities for enhancing skills is by providing different length training periods to participants. Contracts that road builders enter into with state transportation agencies require varying degrees of training that must be provided to trainees, yielding differing degrees of trade proficiency for participants, from a brief to extensive exposure to the trades. For example, in an effort to give as many individuals as possible some construction experience, Florida has adopted a program in which a contractor must provide each trainee with approximately 300 hours of training. State officials explained that this program is intended to give trainees a brief exposure to a trade, not mastery of a craft. Florida reported that 288 people completed training through the program in 2009. Conversely, some states require contractors to have trainees substantively complete or graduate from training programs that may take multiple years—and thousands of training hours—to complete and provide trainees with a high level of work experience and a marketable credential, but that reach fewer people. For example, Texas, which employs this approach, graduated 57 trainees in 2009. Other states, such as Missouri and Washington, fall in between these examples by requiring contractors to provide individual trainees with 400 to 1,000 hours on any given project, believing this is the minimum needed for meaningful skill acquisition. As a result of these different state approaches to the contract training requirement, trainees in different programs acquire different levels of job experience and trade proficiency to take with them when seeking subsequent employment.

A second way state programs differ in providing opportunities for enhancing skills is in using different entities to deliver training. Depending on what is available in the state, some trainees participate in the on-the- job training program as part of preexisting training programs—such as apprenticeship programs—or training that was designed and managed by the contractor or an industry group (see fig. 1). For largely unionized construction workforces, training is typically provided through apprenticeship programs. Apprenticeship programs vary in length depending on the trade, but are commonly about 4 years long and follow specified training steps, including meeting job site training time requirements and classroom instruction. As long as an individual is an apprentice, he or she could be hired to meet a contract training requirement.

Contractors that are not unionized may use industry-sponsored apprenticeship programs, other training programs established by industry, or, in the absence of such programs, design their own training plans. For example, in Texas, trainees generally participate in a program established by the state chapter of the Associated General Contractors and the Texas Department of Transportation. Contractor-designed plans may be approved by the state transportation agency or FHWA division office officials, such as civil rights officials. These training plans are tailored to the individual project and the specific contract requirement; unlike an apprenticeship, these plans could include training in more than one trade.

A third way state programs differ is in program structure; specifically, how much flexibility states give contractors in determining how to fulfill training requirements. For example, some states have implemented contractor- based programs that provide contractors with the flexibility to have trainees work on multiple projects. Typically, job training requirements are specific to individual federal-aid projects and, to successfully meet the terms of the contract, the contractor must meet this training requirement by the time the project is completed. In states with contractor-based programs, the state transportation agencies assign some contractors with a target number of workers that they are expected to train and the contractors have flexibility to decide on which federal-aid projects trainees will work to meet the requirement. Under this approach, trainees may work on more than one project, especially if one project does not have enough work for an individual to complete the required training. For example, Texas has adopted this contractor-based approach. In 2009, the Texas Department of Transportation assigned training goals to 19 of the largest contractors in the state based on the amount of work awarded to each contractor in the preceding year. Through our review of state reports and discussions with FHWA officials, we found that four additional states—Indiana, Iowa, North Carolina, and North Dakota—have also adopted this approach.

Other state practices give contractors added flexibility. For example, Florida has established a system in which contractors can train individuals on state-funded projects that do not receive any federal funds and earn training credits that the contractor can later use to meet training requirements on federal-aid projects. Transportation officials in Florida indicated that trainees working on state projects must meet the same training and demographic standards as on federal-aid projects. South Carolina also allows contractors to temporarily transfer trainees between projects and accumulate up to 25 percent of their required training hours from other projects. Contractors do not receive payment for any of the training hours received on other projects but these hours can count towards completion of the training requirement. State officials explained that contractors frequently take advantage of this option.

The effect of these differences in training length, delivery, and program structure on the ultimate career enhancement of trainees is unclear. For example, some union, state, and apprenticeship program officials noted that apprenticeship programs provide workers with more structured training and connection to a specific trade than training delivered independently by contractors. One apprenticeship program official explained that program staff can help ensure that a trainee gets job opportunities. Similarly, according to U.S. Department of Labor officials, without the structure of an apprenticeship program, trainees may not be well positioned for long-term careers. Additionally, apprenticeship programs, by definition, provide a clear demarcation of when an individual moves from apprentice to journey-level. However, stakeholders also reported that a contractor who is solely responsible for a training plan could provide a wide variety of training cutting across multiple trades and directly related to the skills needed by the employer, better positioning a trainee for future opportunities with that contractor. In general, union and nonunion officials agreed that effective training is more likely to lead to employment if the training is directly linked to a specific job.

Views also differ about what type of training program structure— contractor-based or project-specific—is most effective. Proponents of contractor-based approaches, including officials with FHWA’s Office of Civil Rights and Resource Center, stated that this structure creates a more solid relationship between the contractor and the trainee and helps ensure the trainee gets a sufficient number of hours and the most useful training, both of which improve ultimate employment outcomes. Conversely, some state transportation officials indicated that the project- specific approach has advantages, such as giving trainees more freedom to find employment with other contractors and giving contractors flexibility.

Ultimately, stakeholders agreed that the effectiveness of these training activities in leading to jobs for participants depends heavily on the condition of the broader economy. As states face budget shortfalls, transportation funding is constrained, thus diminishing opportunities for training. Moreover, during economic slowdowns, contractors may be reluctant to bring in new employees as trainees. Given that unemployment in the construction industry was roughly 20 percent in May 2010, potential trainees have faced strong competition in the labor market for scarce construction employment opportunities, according to officials from industry and labor groups. (See app. III for information on the on- the-job training opportunities created by the Recovery Act.)

State Programs Can Create Opportunities but Results Vary

Many states compile annual reports on their achievements. These reports generally show that state programs have led to a wide range of opportunities. However, these state accomplishment reports by themselves do not provide a clear and reasonably comprehensive picture of program achievement. Due to FHWA’s oversight approach and the limitations of available information, the data collected and available from FHWA do not provide a clear picture of the program’s results, limiting our assessment to state-specific illustrations and anecdotes. As with other federally-funded job training programs, such as those administered by the U.S. Department of Labor, tracking the career outcomes of individuals that receive training may be difficult because of the amount of resources and effort required to measure long-term outcomes. The implications of this difficulty are discussed later in this report. However, using available data, we nevertheless estimate that several thousand people are trainees in any given year nationally. Further, to provide more detailed illustrations of the training provided through the program, in those states we reviewed more closely, we found that most workers enrolled in the program were minorities and that there was a range of female participation (see table 2). For example, in Pennsylvania in 2009, all participants were either women (64 participants) or males from a minority group (90 participants).

Additionally, program stakeholders told us that the job training program— by addressing ongoing barriers to employment for traditionally underrepresented groups—can create opportunities that would not have otherwise existed. For example, state officials in two states noted that without the program fewer minorities would work in highway construction, explaining that the construction industry continues to be dominated by informal personal networks and a handful of firms, making it difficult for traditionally underrepresented individuals to find career opportunities. Many program stakeholders see the program as a useful counter to these barriers. For example, one female journey-level equipment operator we spoke with in Washington state attributes her success in the trade to an opportunity the federal program provided in the 1980s. However, others noted that the program could unintentionally disadvantage some workers. For example, officials familiar with two states’ training programs noted that contractor requests for women and minorities leave other potential workers with fewer job opportunities.

Numerous officials noted that the success of the program depends largely on whether the contractor; the state project oversight personnel, such as project engineers and inspectors; and state transportation leadership are committed to it. According to some of these officials, if a contractor wants to create new training opportunities and sees value in diversifying the firm’s workforce, then the program can have a positive impact. Similarly, according to FHWA officials, the visible commitment and support of top transportation leaders, a strong state civil rights office, and an understanding of the program’s goals by construction engineers and field staff help the program succeed. Conversely, contractors that are not committed to the program may be able to find ways to fulfill a contractual training requirement without actually hiring trainees. For example, a contractor might be able to meet the requirement through the demonstration of a good faith effort—in which the contractor describes its unsuccessful attempts to find a suitable trainee—rather than through actual hiring. Though good faith efforts may be legitimate and thoroughly reviewed by oversight officials, in some cases, state officials responsible for approving such good faith efforts may accept them without scrutiny because they may not take the training requirement seriously or want to avoid potential legal issues with the contractor, according to a senior state official and a labor official. Additionally, some contractors may meet training requirements by upgrading the skills of employees that are already a part of their company’s workforce. In these situations, the training requirement does little to create opportunities for underrepresented groups because contractor hiring practices are not changed.

Contractors may have different agreements with unions that dictate their ability to specifically request—and for unions to provide—apprentices that are women, minorities, or disadvantaged individuals in order to meet a training requirement. Likewise, a union may be limited in its ability to meet such requests if women and minorities are not available within their membership. To address these barriers, some contractors and unions have arrangements that help contractors to recruit new members to meet demographic goals for trainees. For example, one union and contractors in Pennsylvania allow for year-round enrollment of women and minorities into one apprenticeship program instead of during shorter annual enrollment periods.

State programs have varying levels of focus on women, minorities, and disadvantaged individuals in accepting trainees on projects with job training requirements. We generally found that the state transportation agencies we reviewed intended for trainees to be women or minorities; however, state programs varied in how rigorously they pushed contractors to hire trainees from these targeted groups. For example, states like Pennsylvania and Washington placed a strong emphasis on accepting trainees that were either women or minorities and rarely approved trainees not in these groups. Other states, however, enroll more nontargeted individuals. For example, in 2009, white men were 83 percent of trainees in Ohio.

State programs target training opportunities toward either the highway construction labor force in general or try to target underrepresentation of women and minorities within specific trades. Some state programs focused on increasing opportunities for women and minorities more generally. For example, officials in southwestern states noted that some construction trades in the region are now predominantly Latino and many contract training requirements are fulfilled by Latino workers. When a contractor in those states hires a Latino trainee to work within a trade that is predominantly Latino, it meets the broad intent of the program by creating an opportunity for a minority group that has been historically underrepresented, but may not address current underrepresentation of other groups. In contrast, other states may encourage contractors to target training opportunities toward underrepresentation in specific trades, although officials acknowledged that this approach can be difficult. For example, an FHWA official explained that minorities and women were not underrepresented as laborers in King County, Washington, so the state tried to limit when a minority or woman could be hired as a laborer trainee and focus training positions in other trades in that geographic area. According to state officials in Florida and Washington, it can be difficult to address underrepresentation of demographic groups in specific trades because identifying such individuals can be a challenge for contractors. Additionally, the training requirement cannot exclude individuals, such as white men, from participation.

Some states have implemented other approaches to give contractors flexibility in hiring trainees, although it is unclear if these flexibilities have led to increased opportunities for women and minorities. For example, in Ohio, contractors are not required to meet specific contract training requirements or an overall annual goal (as in contractor-based programs). Instead, state transportation officials explained that contractors are required to provide training on all federal-aid projects unless they can demonstrate that doing so is not feasible. In this approach, the contractor—not the state—makes the initial determination of whether a project is conducive to training. Although officials believe this approach has increased opportunities for longer term employment for trainees, only 17 percent of Ohio’s 2009 trainees were women or minorities, including trainees involved in Recovery Act projects. However, Florida also gives contractors flexibility in hiring trainees. In 2009, Florida reported that 8 percent of participants were women and 88 percent were minorities (both men and women). It is unclear if differences in how the states have structured their job training programs, state demographics, or other factors explain this difference.

FHWA’s Oversight Approach Does Little to Assess Program Results

FHWA has done little to assess its on-the-job training program nationally and has not determined whether states’ efforts have led to increased opportunities for workers traditionally underrepresented in the highway construction workforce. Moreover, FHWA does not attempt to determine how effective division offices are in their oversight of states efforts. FHWA officials cited insufficient resources and higher priorities within the agency as reasons for the lack of national oversight.

FHWA Has Not Established Clear Criteria For What States Should Accomplish

While FHWA has established regulations and broad guidelines for the program, the guidelines are vague and it is unclear what states should accomplish. The regulations and guidelines provide some assistance to state transportation agencies in implementing their job training programs. For example, FHWA regulations provide state transportation agencies with a list of general factors—such as local demographics and project size—to consider in selecting projects and determining the number of workers to be trained to fulfill contract requirements. Similarly, as required by statute, FHWA collects general types of information from states that enable the agency to judge compliance with the requirements of the on- the-job training program. States provide information to FHWA about the composition of the highway construction workforce and the results of their training program, indicating progress towards state goals. However, FHWA’s regulations for the program were written in the 1970s and, according to FHWA officials and an industry official familiar with the program, have not been updated to reflect changes in program implementation. For example, the regulations provide that FHWA headquarters shall establish suggested annual minimum training goals for each state, but, in practice today, state goals are proposed by state transportation agencies and approved by FHWA division offices, or not set at all. FHWA Civil Rights officials told us that they have taken some early steps towards considering possible revisions of the regulations, including a change that would require state transportation agencies— rather than FHWA headquarters—to propose annual training goals. Finally, the FHWA Resource Center, which provides a wide range of civil rights training and support to state transportation agencies and FHWA division offices, has developed materials that provide state transportation agencies with some program definitions and requirements. For example, the FHWA Resource Center has created general overview and frequently asked questions documents aimed at helping state transportation officials familiarize themselves with the program.

While FHWA’s current regulations and guidelines establish general requirements for states, they do not establish clear criteria for what states must do, what is optional, and the degree to which states can adapt their programs to their own circumstances. Instead, FHWA has left the interpretation of these regulations and guidelines, and subsequent program implementation, largely to state transportation agencies and FHWA division offices. For example, FHWA’s regulations and guidelines list what factors should be considered in selecting projects and determining the number of trainees, but they do not explain how a state ought to consider factors like local demographics or project size when setting a contract requirement or how a state’s goal should be determined. The lack of clear criteria can make program implementation more difficult for states. For example, Washington state transportation agency officials told us that the lack of criteria within the existing guidelines has made it difficult to structure their program and direct their efforts.

Because FHWA has not established clear criteria, state transportation agencies rely on their own judgment in implementing their programs and key attributes of different state programs vary considerably, as previously discussed. A number of federal and state officials familiar with the program noted that some variation in how the program is implemented (1) allows for states to adapt the program to their own circumstances and preferences and (2) provides an opportunity for states and contractors to innovate. For example, as previously mentioned, some states have implemented contractor-based programs that provide contractors with flexibility to move trainees across multiple projects, while other states directly select which projects will provide on-the-job training.

Although providing states with flexibility may have some benefits, such as allowing for state innovations, a lack of clear criteria for state programs prevents FHWA from having a basis to judge the effectiveness of the varying state programs, demonstrate program results, and identify and share best practices. We have reported that federal program standards that successfully address important and varied aspects of program performance are key aspects of a results oriented performance management. Thus, agencies should create a set of federal program goals and standards that address important dimensions of program performance that allow for the variation in individual state programs. Without federal program standards to unify a federal agency’s efforts, division offices and states are left to interpret the regulations, structure their programs, and measure program achievements in whatever manner they deem reasonable. Consequently, there is no basis to judge whether varied state efforts are meeting federal standards for performance management. For example, without set criteria for how states may or may not structure their programs, it may be difficult to determine what program structures are appropriate or what innovations are successful and should be expanded. A lack of clear criteria about what the programs should accomplish could hinder the program’s effectiveness in moving women, minorities, and economically disadvantaged individuals into journey-level positions.

FHWA Does Not Assess States’ Results

Although FHWA division offices receive results information from state transportation agencies annually, the type of information that states submit does not help determine the overall effectiveness of state programs. As previously mentioned, states are required to provide FHWA division offices with annual progress reports, also known as accomplishment reports, that describe states’ and contractors’ efforts to hire underrepresented individuals into the highway construction work force. These accomplishment reports should contain information about the previous year’s accomplishments, such as the number, demography, and trades of trainees that contractors hired as well as the proposed goal for the following year. These reports may also contain information about the other actions the state has taken as part of its program. For example, South Carolina’s 2008 accomplishment report included information on the state’s update to its on-the-job training procedures manual. In addition to annual accomplishment reports, states also provide FHWA with highway construction workforce composition data, which summarize the number of individuals contractors have hired in specific demographic groups in specific job categories. This data provides a snapshot of the highway workforce for a period in July—typically a month with a high volume of highway construction activity—by trade and various demographic groups.

However, FHWA lacks an understanding of the effectiveness of individual state programs and their results nationally for several reasons. First, it is difficult to determine the overall effectiveness of state programs because the accomplishment information that FHWA receives does not include all states and is of limited usefulness in determining how well a state is addressing underrepresentation. Based on our review, states that provided annual accomplishment reports on their training program’s achievements almost all included some form of quantified results information. For example, of the 30 reports that describe 2009 program results, 29 included some quantified information on the state’s accomplishment. These reports frequently included data on trainee enrollment, completion, and demographics. Of the 30 state reports, 18 included annual statewide goals; 17 of these 18 reports also included achievement data regarding whether the state met that goal or not. While a little more than half (17 of the 30) of the state reports we reviewed included the state’s accomplishments and the annual statewide goal they were striving towards, FHWA could not assess whether 13 of those state transportation agencies successfully accomplished their goal. Very few of the annual reports we reviewed included information on the extent to which underrepresented groups participated in the program; the reports that did (2 of the 30) included cursory information. As FHWA has not provided states with clear criteria on how to complete these reports, states included the information they chose to or that was requested by their FHWA division offices.

Second, states submit accomplishment information with a wide range of different output terms and different demographic and trade classification categories (see fig. 2). For example, states may have listed their accomplishments in different output terms such as hours, enrollees, or graduates. Furthermore, even when states used similar terms—such as training “spots” assigned to highway construction projects—it was not clear that states were defining the term in the same way. This lack of consistency prevents comparisons between states or aggregation of accomplishment data across states. Besides different output terms, states also reported information using different demographic and trade classification categories. The 19 reports (out of 30) that provided demographic data for trainees in 2009 used different demographic categories. For example, Montana reported trainee demographics using the general category of minority whereas some other states reported more specific categories such as Latino, African American, Native American, Asian, or Pacific Islander. Additionally, some states included data on the trades of trainees, but these trades were not defined in the reports and may not necessarily be comparable. Twenty of 30 state reports contained trade data in 2009; however, the reports used different trade categories. For example, Kansas reported trainees in multiple different equipment operator positions—scraper, crane, and tractor operator, for example—whereas other states, such as Idaho, listed “operator” as one category. Consequently, though no fault of individual states, it is difficult to consolidate or compare the trade data in these reports.

Third, states also submit FHWA highway construction workforce composition data, which provide a snapshot of the highway workforce for a period in July by trade and various demographic groups. However, the data that FHWA receives from states do not include useful information on which groups are underrepresented in which trades. According to a state official familiar with the program, the highway construction workforce composition data that states report do not separate the number of trainees and apprentices hired as a result of the training requirement versus those hired independently of the training requirement. In other words, the highway workforce composition data that states submit merge the number of trainees and apprentices who work on projects with training requirements as well as projects that do not have training requirements. State officials told us that some trainees and apprentices reported by states and contractors cannot be attributed to training requirements, but might have been on the job independently of any training requirements. As a result, the workforce composition data that states and contractors submit are not effective in capturing information on how well states and contractors are addressing underrepresentation within highway construction work trades. FHWA civil rights officials acknowledged that these data are insufficient and created a working group made up of FHWA division and state transportation agency officials to examine how this information could be improved. In April 2010, the working group implemented several changes to improve the highway construction workforce composition data collected from states, including revising the ethnic and racial categories and making the reporting forms easier to read.

As a result of the limitations described previously, FHWA is not meeting governmentwide guidance, which states that management should track major agency achievements and compare these achievements to plans, goals, and objectives established under the Government Performance and Results Act of 1993. Also, managers need to compare actual performance to planned or expected results through the organization and analyze significant differences. These control activities are an integral part of an entity’s planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results.

FHWA Does Not Assess How Well Its Division Offices Oversee States’ Efforts and Results

Although FHWA division offices fulfill a number of important oversight responsibilities, FHWA does not have a clear overall understanding of how division offices are carrying out their oversight of the on-the-job training program. As previously mentioned, FHWA headquarters relies heavily on division offices to provide oversight of state programs and to ensure that states establish apprenticeship and training programs that are targeted to move women, minorities, and disadvantaged individuals into journey-level positions. We found many instances in which civil rights staff in FHWA division offices provided oversight of state and contractor efforts to meet training requirements on federal-aid projects. Various FHWA division officials we spoke to, for example, reviewed contractor training and enrollment plans, approved new training classifications, provided direction to states in establishing appropriate annual goals, monitored and assessed state achievements by reviewing state reports, assisted states in assessing contractor compliance with training requirements, and recommended use of administrative tools and penalties against contractors not in compliance.

However, it is unclear to what extent division offices are monitoring states primarily because FHWA has neither provided clear guidance on how divisions should carry out their oversight responsibilities nor has FHWA held division offices accountable in meeting performance criteria such as ensuring that state transportation agencies establish active training programs. In our review of various state programs, FHWA division offices executed their oversight responsibilities with different levels of rigor. For example, the FHWA division in Missouri told us that they were actively involved in reviewing contractor’s good faith effort proposals. Conversely, some FHWA division offices had been lax in their oversight of state transportation agencies. For example, we found that FHWA division offices had allowed two states to go through extended periods without active training programs. These states have re-established their programs, in part, due to the insistence of the FHWA division offices. Previously, FHWA had allowed them to lapse without any objection. According to an FHWA official familiar with the program, New Mexico went through a period starting in the 1990s without a program and only reinstated training requirements in federal-aid highway contracts at the insistence of the FHWA division in 2009. The program may have been discontinued because a change in state leadership may not have considered the program a priority, according to an official. Similarly, Texas department of transportation officials told us the training program in Texas was formally established in 2006 and implemented in 2007 when a new civil rights program specialist came on board and said that Texas needed to develop it. They also stated that prior to that Texas had an optional program because the Texas FHWA division civil rights specialist did not require the state transportation agency to develop a training program. FHWA civil rights officials acknowledged that there may be instances of states lacking active training programs due to limited resources within the Office of Civil Rights to manage state programs and the low priority states may assign to these programs relative to other civil rights programs.

In addition, some division offices did not adequately oversee states’ annual training goal setting, a central component of existing program regulations and guidance, but one that some program stakeholders see as not valuable. Some division offices played key roles in helping states determine appropriate state goals while others were less rigorous in ensuring that the state transportation agency established appropriate goals. For example, an FHWA official familiar with Alabama’s state program told us that the division office plays an active role in determining whether the training goal is reasonable by examining past reports and historical data and looking to see if the state transportation agency achieved its training goal the previous year. However, other states like Arizona, California, New Mexico, and Ohio, were exempted by their FHWA division offices from having to establish an annual statewide goal. The lack of consistent adherence to this requirement can be explained in part by the lack of clear criteria on how division offices should oversee state programs and differing views among program stakeholders about the usefulness of having state goals. While some state officials told us that setting state goals helps state transportation agencies track their progress, others—including officials from the Office of Civil Rights— stated that individual contract goals were sufficient and statewide goal setting did not add value. Currently, headquarters cannot and does not assess how well division offices oversee states’ adherence in establishing a state goal because headquarters has not established clear criteria that all states must establish training goals. If FHWA headquarters were to strengthen criteria by requiring states to establish annual goals, then it would provide headquarters a basis—specifically by determining how adequately divisions oversee states goal setting—upon which to assess whether division offices were consistently holding states to federal criteria.

Although FHWA division offices execute a number of important oversight responsibilities to ensure that states and contractors meet training requirements on federal-aid projects, division office performance should be monitored by headquarters. We have reported that it is a useful practice to assess the oversight activities of division offices in order to monitor states’ progress in meeting overall program objectives. In the absence of clear criteria on how division offices should oversee state programs and how to hold state transportation agencies accountable, FHWA will be unable to fully assess whether division offices are effective in ensuring that states meet program requirements.

FHWA Cites Insufficient Resources and Higher Priorities as Reasons for Lack of National Oversight

The Office of Civil Rights focuses on policy while paying minimal attention to national program oversight in part because the office lacks the resources to provide adequate oversight to state transportation agencies, according to FHWA officials. The FHWA Office of Civil Rights only has one full-time equivalent employee that oversees both the on-the-job training and supportive services programs. Also, FHWA officials told us that the agency has provided little national oversight to state transportation agencies because it has placed a higher priority on other civil rights programs that are at higher risk—based on FHWA’s corporate risk assessment—than the on-the-job training program. FHWA identified higher risk programs as including the Disadvantaged Business Enterprise, Title VI of the Civil Rights Act of 1964, and the Americans with Disabilities Act programs. Because headquarters is responsible for overseeing these other civil right programs, FHWA has not been able to provide sufficient national oversight to the on-the-job training program, according to FHWA officials. Nevertheless, FHWA has recently undertaken several measures that may strengthen its oversight and assess program results. For example, FHWA surveyed all 52 division offices in 2011 and conducted baseline assessments of state civil rights offices in 2008 to gather basic information on the states. According to FHWA civil rights officials, the survey—which focused on the supportive services program—could identify training and workforce development issues that may need to be addressed by the division office. Additionally, FHWA officials have recently established what they describe as an informal partnership with the Department of Labor to explore how local career centers might assist tracking the results of the on-the-job training program and the supportive services program, among other things.

Although limited resources and competing priorities are valid concerns, especially for a smaller program, FHWA’s oversight approach needs to provide the agency with a basic assessment of how well the program has functioned on a national basis. In the absence of national oversight from headquarters, it is unclear what impact the program has beyond the individual state-level.

FHWA Provides Funding for Activities Supporting Job Training Programs, but Overall Stewardship is Limited

FHWA grants about $10 million each year for supportive services that directly and indirectly support job training activities. However, even after considering the small size of the supportive services program, we found that FHWA’s stewardship is limited. The agency collects some information on services provided, but does not always determine results achieved and does not routinely consider grantees’ performance in making funding decisions. FHWA officials said they have few resources devoted to the program and results are sometimes difficult to aggregate.

Supportive Services Provide Direct and Indirect Support to Training Programs, but It Is Unclear How Many People Receive Such Services

Currently, FHWA provides about $5 million to $6 million each year for services that directly support job training programs (through the discretionary grant program) and about $4 million each year for programs that indirectly support job training programs (through the discretionary grant program, summer institutes, and internships). Regarding the discretionary grant program, the agency typically provides funds to about 20 states each year, averaging about $240,000 for each state. FHWA does not attempt to determine, and we could not establish, how many people are participating in the program since FHWA does not collect this information. From reviewing applications and considering the degree to which FHWA provided requested grant amounts, we estimate that roughly 2,000 people may have received some direct supportive services in any one year. FHWA officials noted that while they do not know how many individuals participate in the supportive services program, they estimated that several thousand individuals may receive services in a given year, depending upon the number of projects funded, as well as other factors. FHWA officials stated because these activities can vary considerably from state to state and they lack the resources needed to determine such a count, they have been unable to determine the total number of participants.

We could not determine with any confidence how many people received supportive services through the discretionary grant program because FHWA has not collected this information. Also, we could not analyze the demographics of individuals served by this program. However, a rough indication of the services provided can be gleaned from states’ applications for funding (see table 3). About three-quarters of the states proposed to provide job placement and other outreach activities (e.g., referral services or employment counseling) and preapprenticeship type training (e.g., safety training, overviews of highway construction equipment, or training certifications). To a lesser extent, states proposed to provide life skills training, physical construction tools, transportation, remedial math or language training, and child care. For example, Arkansas proposed a variety of services to support a 20-person class for heavy equipment training and proposed providing tools, boots, transportation, and child care support to all participants employed on a federal-aid highway construction project.

FHWA’s discretionary grants also fund some activities that support job training indirectly. For example, a number of states have used supportive services funds for construction career days, which expose high school students to various areas of construction, including heavy highway construction, with the intent of increasing interest among women, minorities, or economically disadvantaged individuals in the transportation industry and highlighting potential career opportunities. Applying the same approach used to estimate the number of individuals receiving direct supportive services, we estimate that roughly 8,000 individuals, primarily through the construction career days, are exposed indirectly to highway construction jobs.

Similarly, FHWA provides more than $2 million each year to universities through state transportation agencies for summer transportation institutes that expose students to the transportation industry broadly, not just highway construction. Activities include exposure to engineering and construction concepts, presentations by industry groups, and site visits to local transportation facilities, such as train stations and transportation simulators. We estimate that roughly 1,200 people participated in the activities in 2010 across 32 states, the District of Columbia, and Puerto Rico. Finally, FHWA funds about 100 internships each year at agencies within the U.S. Department of Transportation (e.g., Federal Transit Administration and Federal Railroad Administration) and at state transportation agencies, costing about $1.5 million annually. These internships expose students to transportation careers and encourage the pursuit of transportation-related studies in college. In the activities supported by the fiscal year 2010 summer institute and internship programs, we estimate that female and male participation in each program was relatively equal, and that a significant number of the participants served by these two programs were members of minority groups (see fig. 3). While these activities can increase awareness of highway construction job opportunities and lead to future work force entries, the effect of these efforts, if any, may not be apparent for years since students typically return to school upon completion. They may not enter the workforce for a year or more, whether in construction or other transportation fields or in a nontransportation field.

FHWA has not used any of the information available to it to determine the overall effectiveness of its supportive services activities. FHWA does not track how many of the state grantees have achieved their goals in directly supporting on-the-job training. Although states are required to provide reports on their accomplishments to FHWA division offices, this information is not typically relayed to the FHWA officials making grant decisions. FHWA has only received about one-third of the final fiscal year 2008 accomplishment reports for the discretionary grant program, and FHWA officials noted that a low response rate is typical. Officials responsible for making the grant funding decisions explained that they do not routinely receive monthly or quarterly progress reports from the division offices but can request them on an as-needed basis. They also noted that occasionally, they request these reports to identify promising practices. However, these officials noted that they have limited tools to induce states to meet their reporting requirements. Furthermore, they explained that some states do not provide reports because of delays in awarding contracts, noting that often contracts are not in place by the time the reporting requirements have begun, and thus the states have no results to report. FHWA officials recognized that they did not have sufficient information on the results and the accomplishments of these programs, and surveyed the division offices in 2011 to obtain information on what grantees have accomplished and what approaches have worked well. As of August 2011, FHWA officials were still collecting and analyzing responses from the states, in part, due to additional time needed by the division offices to complete the survey as well as comply with additional reporting requirements for other FHWA programs.

Moreover, FHWA has not established how state-reported data can be used to evaluate whether grantees have met FHWA’s overall program objectives to increase the overall effectiveness of each state’s job training program and enhance the training opportunities for women, minorities, and economically disadvantaged individuals. FHWA officials stated that they have not analyzed the extent to which grantees are achieving their performance goals, citing a variety of reasons, including a lack of resources and other competing programmatic priorities. FHWA officials anticipate that the survey, which they expect to complete in the summer of 2011, will provide much of the information needed to analyze the effectiveness of the program.

Regarding indirect supportive services, FHWA officials noted that they have collected some data on the number and demographics of the individual attendees for the summer institute and student internship program, and further heard anecdotally that the indirect services provided (e.g., awareness programs) are effective in persuading students to consider a career in the transportation field. However, for reasons noted previously, FHWA lacks this information for the indirect activities supported by the discretionary grant program, even though the grant program receives a sizable portion of FHWA’s supportive services funding. Moreover, FHWA officials do not track and do not know to what extent indirect supportive services are leading to jobs in the highway construction industry. Officials noted that they do not track the outcomes for a number of reasons, including privacy concerns with collecting data on minors.

FHWA’s past efforts to determine the effects that its supportive service funding provided have been limited, but we recognize that doing so is complicated. For programs designed to support diverse state or local activities, such as the supportive services program, finding common performance metrics to describe the overall program’s effectiveness can be challenging. We have reported on options for building accountability provisions into grants that help balance states’ flexibility to select a mix of activities and services that will best allow them to achieve a particular national outcome with accountability for achieving that outcome. These options include emphasizing results-based evaluation (e.g., workforce retention) in addition to examining output activities (e.g., the number of participants receiving a service). For example, under a results-oriented evaluation, FHWA could specify national goals and objectives, set up a process for establishing them, or adopt some combination of the two. FHWA officials requested results information through its survey of supportive services providers in the discretionary grant program. A results-orientation requires ongoing information. FHWA officials indicated that they plan to annually gather some of the basic information that was solicited from states in the current survey. In addition, FHWA states that it requires that grantees establish performance goals and measurable objectives for their proposed activities, and report on progress towards meeting these goals. However, this approach, to the extent it is employed, focuses on outputs rather than end results.

Program results are important for making budgetary and programmatic decisions. High-level decision makers, such as Members of Congress and agency heads, need to know which programs are achieving their goals and objectives to make informed decisions about whether to expand, reduce, or maintain funded activities. Furthermore, agencies are responsible for ensuring that grantees use federal funds for eligible activities. In the absence of reported program results, FHWA cannot know whether funds have been used effectively and appropriately.

FHWA Has Not Sufficiently Used Results to Inform Funding Decisions

FHWA often funds the same projects for supportive services year after year. For example, 17 of the 26 projects (65 percent) that received grants in fiscal year 2010 also received them in 2008, 2009, or both years. FHWA reviewers of discretionary grant applications are instructed to score the scope of the work, performance goals, objectives and measurements, budget summary, monitoring and evaluation plan, and reporting plan. However, past performance information is not required of applicants in the application nor is past performance one of the criteria scored during funding reviews. Furthermore, the solicitation issued to states does not request information from grantees about how they have used prior FHWA grants, or the resulting effectiveness of their activities.

FHWA officials told us that they informally consider performance information when screening discretionary grant applications. For example, they stated that they may use data from the accomplishment reports to look for potential problems. In addition, officials noted that states whose accomplishment reports show positive results provide FHWA assurance that the state may be able to successfully implement additional programs, which can influence the state’s chance of receiving funding in a subsequent year.

However, as noted previously, FHWA decision makers have received a limited number of progress and final reports, which are the primary source of performance information for discretionary grants. Furthermore, FHWA decided to require only one final oversight report for four funding cycles occurring in fiscal years 2009 and 2010, which included two rounds of Recovery Act awards. FHWA officials stated that they made this decision in order to streamline the reporting requirements for grantees. Accordingly, FHWA has only received a few accomplishment reports for this period. As a result, FHWA will not be able to determine the effects of the individual award decisions until after all rounds of funding are complete, which, in some cases, will be 2 years after the decisions were made, following subsequent funding decisions by FHWA.

Similarly, for the summer transportation institute program, FHWA reviews the annual reports submitted by states and informally assesses performance, and it considers past performance during its funding review to some degree. Like the discretionary grant program, many of these applicants are funded year after year. For example, 34 of the 53 applicants (64 percent) funded in fiscal year 2010 were funded in each of the previous two fiscal years. According to FHWA officials, the division offices look at earlier annual reports to see if the same host site is soliciting funding for the subsequent year, and try to determine whether the host achieved its program objectives. However, like the grant program, past performance information is not required of applicants in the application and performance is not one of the criteria scored during funding reviews. Furthermore, the solicitation issued to states does not request information from grantees about how they have used prior FHWA grants and the resulting effectiveness of their activities.

For the summer internship program, FHWA evaluates a prospective intern’s performance through the student’s academic and professional work history, which, among other things, entails a review of the applicant’s resumé, references, and academic transcript. FHWA also surveys students at the end of the program to gather information on how well the program worked, and to suggest recommendations for how to address these issues in the future.

Given that many of the activities funded through the program are funded repetitively, good practices suggest that the use of performance information can inform and improve recipient selection approaches, even for a smaller program. While the $10 million provided annually through this program is much smaller than many other federal grant programs, integrating results information can help FHWA identify approaches that work or leverage good practices to improve the program, and decrease funding to poor or marginal performers (or encourage them to improve their performance).

FHWA is considering whether to change the grant funding approach from discretionary to formula-based funding. However, using performance data could still help ensure accountability for results. Under the formula approach that FHWA is considering, each state would receive some supportive services funds, in contrast to the 20 or so states that receive funding each year under the current approach. Using performance data, FHWA could help states that have not participated in the program to more easily identify previously successful approaches to effectively implementing their own supportive services programs. The performance data could also be used to determine whether funds should be withheld from or provided conditionally to any currently low-performing states. FHWA hopes to implement the new funding approach in fiscal year 2012.

Conclusions

FHWA’s on-the-job training program and its counterpart, the supportive services program, represent decades-long federal initiatives to increase participation of women, minorities, and economically disadvantaged individuals in the highway construction industry. FHWA allows each state to tailor its job training activities to fit its unique demographic, geographic, and social characteristics, which can spur innovation and help states meet their needs. However, FHWA has not established clear criteria for what states must do, what is optional, the degree of flexibility states have in implementing their programs, and what they are intended to accomplish. Within state transportation agencies, FHWA, and state partners involved in the program, we found officials working toward the programs’ success. However, FHWA does not know what these programs have accomplished or how effectively these activities have advanced the broader goal of bringing underrepresented individuals into the highway construction workforce, because FHWA has performed only limited assessments of these programs over their nearly 40-year history. We recognize that doing so may be difficult. FHWA’s current activities—such as its survey regarding the supportive services program—are positive steps towards making such an assessment.

While it is appropriate for FHWA headquarters to rely extensively on its division offices to oversee state on-the-job training programs, such a decentralized management approach does not relieve FHWA headquarters from the responsibility of providing leadership and a national perspective for its programs. Even though the on-the-job training and supportive services programs are relatively small compared to FHWA’s other programs, we would still expect FHWA to conduct oversight to ensure program goals are met and at a scale appropriate for the size of the programs. A necessary first step is for the agency to establish clear criteria—existing program criteria are not clear—for states on what they are to accomplish and for its division offices on how they are to oversee state activities. These criteria could continue to allow for flexibility in state programs, as FHWA deems appropriate. In doing so, FHWA needs to provide guidance on state reporting that will lay the groundwork for national assessments of results. Likewise, evaluation of supportive services results could inform funding decisions. Such actions will result in FHWA being able to better ensure that program goals are being met and would allow them to improve program accountability.

Recommendations for Executive Action

To establish accountability for meeting the programs’ goal of increasing the participation of traditionally underrepresented groups in the highway construction workforce, we recommend that the Secretary of Transportation direct the FHWA Administrator to take the following three actions:  Strengthen criteria—through regulations, guidelines, or other mechanisms—so that states have a clear understanding of how the on-the-job training program should be implemented and the results state programs are intended to accomplish.  Create and implement an oversight mechanism that (1) holds states accountable for meeting federal training criteria and (2) clearly stipulates how FHWA will assess state program effectiveness, including what type of program achievement data states are to submit and how such data will be used. This oversight mechanism should include assessing the effectiveness of its division offices in overseeing state activities.  For the supportive services program, develop an approach to (1) evaluate the extent to which grantees have met their proposed annual goals and (2) integrate the results of this evaluation into FHWA’s funding decisions for supportive services programs.

Agency Comments

We provided a draft of this report to the Department of Transportation for review and comment prior to finalizing the report. The Department of Transportation generally agreed with the recommendations and provided technical comments, which we incorporated as appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to congressional committees with responsibilities for surface transportation issues, the Director of the Office of Management and Budget, the Secretary of Transportation, and the Administrator of the Federal Highway Administration. In addition, this report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you have any questions about this report, please contact me at (202) 512-2834 or wised@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

In this report, we assessed the Federal Highway Administration’s (FHWA) on-the-job training program and the associated supportive services program. To do so, we focused on the extent to which (1) states’ implementation of the FHWA on-the-job training program provides training and career enhancement opportunities for traditionally underrepresented groups to work on federal-aid highway projects, (2) FHWA has overseen the program and demonstrated results, and (3) the supportive services program established by FHWA has assisted state job training and apprenticeship programs. In response to your interest, we also gathered information on financial incentives and penalties states use to encourage contractors to provide job training.

Additionally, for background, to get an understanding of historical participation in construction for women and minority groups between 1985 and 2010, we analyzed data provided by the Department of Labor’s Bureau of Labor Statistics on industry employment by race, ethnicity, and gender. While the data used are not strictly comparable over time due to changes in industry classifications systems, we used the data to describe a general trend in employment in the construction industry. We believe the data are sufficiently reliable for our purposes, which were to provide historical context on the industry.

Implementation of Job Training Program

To determine how FHWA’s on-the-job training program has provided training and career enhancement opportunities for traditionally underrepresented groups on federal-aid highway projects, we reviewed legislation, including the Federal Aid Highway Act of 1968, which authorized the program, and the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users; the program’s implementing regulations; FHWA guidance such as its contract compliance manual; and our reports on performance management. We interviewed officials from FHWA’s Office of Civil Rights and its Resource Center to understand the agency’s overall program philosophy. In addition, to gain a basic understanding of how the program operates and its environment, we interviewed officials from FHWA division offices and state transportation agencies in 11 states. We selected these states based on a variety of factors, including unionization, the state’s size, the extent to which a state had been awarded funding for supportive services and used workforce development funding, and geographic location. We further examined more closely how these programs were implemented in four of these states: Missouri, Pennsylvania, Texas, and Washington. We selected these states because they differed in demographics; unionization; how job training is structured within the state, including any unique practices; and the extent to which the state participates in the supportive services program. We also interviewed representatives from local chapters of industry groups, such as the Associated General Contractors, as well as members of local unions, including laborers, craft- workers, and heavy equipment operators. The information gathered through our review of state programs is intended to illustrate these programs; it is not generalizable to all states or to other states with similar characteristics.

To develop an order of magnitude estimate of the number of individuals participating in the program in a given year we used information in annual accomplishment reports that states submitted to FHWA. In making these calculations, we estimated the total number of participants from the sample of reports provided by FHWA. We also used this data, along with our interviews with FHWA and state transportation officials, to provide examples and illustrations of the type of data collected by the states for these programs, including activities implemented, the financial incentives and penalties used by states, and the extent to which the American Recovery and Reinvestment Act of 2009 (Recovery Act) provided job opportunities. While we cannot independently verify all of the data collected by the states and submitted to FHWA, because the data is used solely to provide context on the overall program, we believe the data is sufficiently reliable for our purposes.

FHWA Oversight of Job- Training Programs

To assess how FHWA oversees job training programs to determine program achievement, we reviewed the program standards and outcome measures required in FHWA’s regulations and program guidance and compared them to good management practices, such as the assessment of oversight activities we have identified in other programs. We also compared the information on training activities and program results contained in the state accomplishment reports to requirements detailed in FHWA regulations, guidance, and other instructions communicated to states. Because of the differences in how states collected and reported data, we were unable to compare the results across states. Accordingly, in order to understand trends, similarities, and differences across implemented programs we relied on interviews with FHWA and state transportation officials to provide examples of the type of oversight activities implemented by each state. The information we obtained from these officials is not generalizable across all state programs.

Supportive Services Program

To determine the extent to which supportive services have assisted state job-training programs, we reviewed how FHWA has implemented and overseen the three components of the program: (1) the discretionary grant program, (2) the national summer transportation institute program, and (3) the summer transportation internship program for disadvantaged groups. We analyzed FHWA budget information to determine the amount of funding provided to the three programs. In order to assess the data’s reliability, we interviewed department officials and reviewed departmental audit reports about the system which processes the data. We determined that this data was sufficiently reliable for our purposes.

For the discretionary grant program, we analyzed legislation, such as the Federal Aid Highway Act of 1970, which authorized the program, regulations, and other FHWA guidance, such as FHWA’s supportive services handbook. We reviewed the agency’s approach to requesting and evaluating grant proposals for the program. Specifically, we analyzed program criteria for funding proposals and incorporating performance information by reviewing the agency’s request for proposals, rating review sheet, and other information communicated to prospective applicants. We compared FHWA’s use of performance information to good management practices that we have identified. We reviewed the 153 grant proposals that received funding from 2008 through 2010 and categorized the activities receiving funding. In addition, we reviewed FHWA’s guidance for reporting the results of programs. However, because FHWA could only provide about one-third of the annual grantee accomplishment reports for fiscal year 2008 and only a handful of reports for the following funding cycles, we were unable to determine the extent to which states complied with this guidance. In addition to FHWA officials, state transportation officials, and industry groups identified previously, we also interviewed supportive services grantees in Missouri, Pennsylvania, and Texas. These results are not generalizable to all states or to other states with similar characteristics. In order to estimate the number of individuals participating in the program in a typical year, we reviewed the goals in the funded proposals for activities proposed from fiscal years 2008 through 2010, excluding Recovery Act projects. We classified each of the activities as either directly supporting job-training or current trainees (such as preapprentice training, stipends for child care and transportation, and providing job-related equipment) or indirectly supporting job training (such as career day events). For each year, we added the numerical goals for each type of job training activity and averaged our results over the 3-year period. Our calculation represents a very rough estimate of the number of participants in this program. Finally, we reviewed FHWA’s proposal to change how it distributes funds through the program.

For the summer transportation institute program, we reviewed FHWA guidance and other documents on the program’s history and implementation. We reviewed FHWA’s request for proposals and evaluation criteria and compared the requirements for the use of performance information to the use of such information for programs that we have identified. In addition, we analyzed the results described in annual reports submitted by programs in 32 states, the District of Columbia, and Puerto Rico for fiscal year 2010. Specifically, we used these data to estimate the total number of participants in the program, and provide demographic information, such as ethnicity, gender, and education level, of the participants, as well as examples of the types of activities undertaken through this program. As the data was reported individually by each state, we limited our use of these data to background on the participants and believe these data are sufficiently reliable for this purpose. Finally, we interviewed officials from FHWA and two program sites to understand how FHWA selects, implements and oversees these programs.

Similarly, for the student internship program, we reviewed FHWA guidance and other documents, including the program and evaluation criteria, and compared FHWA’s use of performance information to good practices we have identified. We also reviewed FHWA’s analysis of the results of the program, including demographic information about the participants, such as gender, ethnicity, proposed major, and location of the internship. We limited our use of these data to background on the participants and believe these data are sufficiently reliable for this purpose. We also interviewed FHWA officials to understand how the agency oversees and implements the internship program.

We conducted this performance audit from August 2010 through September 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Some Financial Incentives and Penalties Are Available to Encourage On-the- Job Training Performance

While the on-the-job training program does not have a distinct funding source, job training expenses on federal-aid highway projects are eligible for federal dollars in the same way as the typical 80/20 federal/state split of project costs. Each state has discretion in how contractors will be reimbursed for these training activities, if at all. Some states have tried to create financial incentives for contractors to perform job training, but these incentives are relatively small. For example, some states reimburse contractors for the hours of training performed. Reimbursement rates range from 80 cents per hour, in Arizona for example, to $3.50 per hour in Missouri, in the states we reviewed. Other states include the job training requirement as a bid item in the request for contract proposals, allowing contractors to name their price for meeting the requirement. Officials and contractors that we spoke with generally agreed that reimbursement rates were too low to be much of an incentive to contractors. Some contractors simply do not bother submitting the paperwork required to claim a reimbursement. Likewise, in cases in which job training is a bid item, we heard that contractors will commonly bid a minimal amount for this item, preferring to keep their overall price as low as possible to win the contract award.

Financial incentives can also take a penalty form, although this approach appears to be less common. For example, Missouri has recently established a penalty system for nonperformance. In this system, the contractor will be held liable to pay $7 for every hour of training that should have been provided but was not. To date, the state has rarely levied such a penalty. On occasion, states may threaten to withhold contract payments if a contractor is not performing. In our interviews with state officials, we found this practice is rare and states are reluctant to do so. Officials noted that generally the threat to withhold funds is sufficient to correct problems.

Appendix III: Recovery Act Provided Some Job-Training Opportunities

Through the American Recovery and Reinvestment Act of 2009 (Recovery Act), the federal government provided states with additional funds for highway construction projects and the same general training requirements applied to these projects. However, to some extent, these projects may have been less conducive to training than projects funded through regular annual federal-aid highway apportionments. Because the Recovery Act prioritized projects that could be completed quickly, many projects focused on resurfacing or other less complex and lengthy projects. Short projects tend to require few workers overall and less time per trade to complete, minimizing the opportunity to train workers on the project. In the four states we reviewed more closely, officials generally agreed that the Recovery Act-funded projects led to some job training opportunities, but these opportunities were generally shorter than regularly funded projects.

Several states reported to FHWA that Recovery Act-funded projects employed trainees. Three of the 30 states we reviewed significantly increased their annual achievement due to Recovery Act funding. For example, more than half of the trainees in Texas worked on Recovery Act-funded projects in 2009. Specifically, 156 individuals worked as trainees on 142 Recovery Act-funded projects. At the end of 2009, 101 of these trainees remained actively enrolled in the program and 36 had graduated. In addition, some states made special efforts to ensure training occurred on Recovery Act projects. For example, Texas created a separate job training program for Recovery Act projects that assigned training goals to individual projects. Texas already had a contractor-based program in place, but wanted to directly track job training results that were associated with Recovery Act funding.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, James Ratzenberger and Teresa Spisak (Assistant Directors), John Stambaugh, Tina Paek, Matthew Voit, Dianne Blank, Elizabeth DeVan, Sara Ann Moessbauer, Amy Rosewarne, and Crystal Wesco made key contributions to this report.