Registered Apprenticeship Programs: Labor Can Better Use Data to Target Oversight
Released: 2005-09-13T00:00:00

Letter

Background

Although apprenticeship programs in the United States are largely private systems that are paid for largely by program sponsors, the National Apprenticeship Act of 1937 authorizes and directs the Secretary of Labor to formulate and promote labor standards that safeguard the welfare of apprentices. The responsibility for formulating and promoting these standards resides with OATELS. OATELS had a staff of about 176 full-time equivalencies and an annual appropriation of about $21 million in 2004. In addition, because of budgetary constraints, OATELS officials do not expect resources to increase. At the national level, OATELS can register and deregister apprenticeship programs (i.e., give or take away federal recognition), issue nationally recognized, portable certificates to individuals who have completed registered programs, plan appropriate outreach activities targeted to attract women and minorities, and promote new apprenticeship programs to meet workforce needs. In addition to this national role, OATELS directly oversees individual apprenticeship programs in 23 states. In these states, the director for the state’s apprenticeship system and other program staff are federal employees who monitor individual apprenticeship programs for quality and their provision of equal opportunity.

Labor can give authority to states to oversee their own apprenticeship programs if the state meets certain requirements. Labor has given this authority to 27 states, the District of Columbia, and three territories. In these states, which we refer to as council-monitored, the federal government is not responsible for monitoring individual apprenticeship programs; instead, the state is. It does so through state apprenticeship councils. OATELS does, however, conduct two types of reviews to determine how well the state fulfills its responsibilities. Quality reviews determine, in part, conformance with prescribed federal requirements concerning state apprenticeship laws, state council composition, and program registration, cancellation and deregistration provisions. Equal Employment Opportunity (EEO) reviews assess the conformity of state EEO plans, affirmative action activities, record-keeping procedures, and other activities with federal EEO regulations. In addition to these reviews, OATELS may also provide state agencies with federal staff to assist in day- to-day operations.

The number and type of construction apprenticeship programs are distributed differently across federally- and council-monitored states. Council-monitored states not only have more programs, but these programs are more likely to be jointly sponsored by employers and unions than sponsored by employers alone. On average, a construction apprenticeship program in federally-monitored states trains about 17 apprentices and in council-monitored states trains about 20. Beyond this average, it’s important to note that there can be great variation among programs, with some having over 400 participants and others 1 or 2. Figure 1 identifies states where programs are federally- and council- monitored.

Both the federal and council-monitored states collect data on the individual programs they oversee. Labor maintains a large database called the Registered Apprenticeship Information System (RAIS) and collects information about individual programs, apprentices, and sponsors for apprenticeships in the 23 states where it has direct oversight and in 8 council-monitored states that have chosen to report into this system. The other council-monitored states, 20 in total, maintain their own data and collect various pieces of information on apprenticeship systems. Labor does collect aggregate data on apprentices and programs from these states.

In all states, individuals can enter the construction trades without completing formal apprenticeship programs, but many construction workers, particularly those working in highly skilled occupations that require extensive training, such as the electrical, carpentry, and plumbing trades, receive their training though registered apprenticeship programs. To complete their programs, apprentices must meet requirements for on- the-job training and classroom instruction that must meet the minimum standards for the trade as recognized by Labor or the state apprenticeship council. Programs in some trades, for example, commercial electricity, may take 5 years to complete but programs to train laborers may only take a year. Beginning apprentices’ wages generally start at about 40 percent of the wage of someone certified in a particular trade and rise to about 90 percent of that wage near completion. Apprentices’ contracts with their program sponsors specify a schedule of wage increases.

Labor’s Monitoring of Registered Apprenticeship Programs Is Limited

Although OATELS is responsible for overseeing thousands of apprenticeship programs in the states where it has direct oversight, it reviews few of these programs each year. Also, while its apprenticeship database collects much information about individual participants and programs, Labor hasn’t used these data to systematically generate program performance indicators such as completion rates. As a result, it lacks information that would allow it to identify poorly performing programs and adjust its oversight accordingly. Furthermore, despite many technical upgrades, Labor’s database hasn’t provided information that meets the needs of federal apprenticeship directors or the needs of other stakeholders.

Few Federal Staff Are Engaged in Monitoring the Programs That Labor Directly Oversees

OATELS has reviewed very few of the apprenticeship programs in the states where it has direct oversight. Federal apprenticeship directors in these states reported they conducted 379 quality reviews in 2004, covering only about 4 percent of the programs under their watch. These reviews are done to determine, for example, whether sponsors have provided related instruction and on-the-job training hours in accordance with the standards for the program and whether wages reflected actual time in the program. The number of reviews conducted varied across states. On average, 22 quality reviews per state were conducted, but one director reported conducting as many as 67 reviews while another reported conducting no reviews at all. In addition, programs in council-monitored states were almost twice as likely as programs in federally-monitored states to have been reviewed within 3 years. (See fig. 2.) Several federal officials said over the past several years they had placed primary emphasis on registering new programs and recruiting more apprentices, particularly in nontraditional areas such as childcare and health. In addition, they told us it was not possible to do more reviews in part because of limited staff.

In addition to having fewer reviews, apprenticeships in federally- monitored states had fewer staff dedicated to monitoring activities than council-monitored states. In 2004, each staff person in a federally monitored state was responsible, on average, for about 2,000 apprentices, according to federal program directors; to put this in context, case loads of monitors in federally-monitored states were almost twice as large as those in council-monitored states. In federally-monitored states, on average there were about 2.5 staff to monitor programs, less than one-third the average in council-monitored states. Labor’s practice of assigning federal staff to monitor programs in 18 of the council-monitored states rather than to programs in federally-monitored states compounded differences in staff resources. Directors in council-monitored states reported that at least two federal employees, on average, monitored programs in their jurisdiction. As important as the number of staff, is how they spent their time. About a half of the staff in federally-monitored states spent 40 percent or more of their time in the field performing monitoring, oversight, and providing related technical assistance, according to federal program directors whereas one-half of the staff in council-monitored states spent about 70 percent or more in the field.

While Labor Collects Much Information about Apprenticeship Programs, It Does Not Systematically Use Data to Focus Its Oversight

Although Labor collects information to compute completion rates and track participants who do not complete programs in the time expected, it does not use these data to focus its oversight efforts on programs with poor performance. During a site visit in a federally-monitored state, a monitor showed us how she computed cancellation rates by hand for apprentices in programs that she felt were not doing an adequate job of training apprentices to see if her hypotheses were correct. In the absence of performance information, directors and staff in federally-monitored states reported that a variety of factors dictated which programs to review. These included size, newness, location, date of the last review, sponsor’s cooperativeness, as well as the location of staff resources.

In addition to not using program data to target reviews, Labor has not collected and consistently entered into its database information about why apprentices cancel out of programs, although its database was designed to include such information and having it could help target reviews. Officials told us that voluntary cancellation or transfers to another program were at times associated with program quality, while other nonvoluntary reasons, such as illness or military service, were not. Currently, recording the reason for an apprentice’s cancellation in the database is an optional field. We found that no reason was recorded for 60 percent of the cancellations and the remaining 40 percent did not effectively capture the reasons for leaving. Of the 18 reasons entered, the most common reasons were “Unknown,” “Voluntarily Quit,” “Unsatisfactory Performance,” “Discharged/Released,” and “Cancelled with the Occupation,” some of which did not provide useful information to target reviews. Also, other entries were close duplicates of one another, such as “left for related employment” and “left for other employment.”

Labor also treats as optional data entries for its equal employment opportunity reviews: including the date of the last review, compliance status, status of corrective actions, and other information that would improve the efficiency of managing reviews. As a result, such data were available for about 5 percent of programs in Labor’s database in fiscal year 2004. Without this information, it is more difficult to determine when programs had their last EEO review and to readily identify programs with known problems.

Labor’s Data Base Does Not Meet the Needs of Apprenticeship Directors and Other Stakeholders

Despite many technical upgrades, Labor’s database hasn’t provided information that meets the needs of its federal directors or the needs of other stakeholders. While acknowledging that Labor’s database has been updated and improved, 22 out of the 23 directors of apprenticeship programs and their monitoring staff have expressed dissatisfaction with the present system. One complained of “daily” changes to the data fields without being informed of “why or when they will change.” Expressing the desire to select and sort data on any field and generate unique reports in context with all available data, another concluded, “In short, we need a lot of flexibility with database reports that we don’t have at this time.” Many federal apprenticeship directors made recommendations for improving the database. In general, what state directors wanted most was a system that was stable, user friendly, and that would allow them to produce customized reports to better oversee the apprenticeship programs in their states. The list below shows those potential improvements endorsed by more than half of the state apprenticeship directors: Increase the timeliness of notifications to state and regional offices for changes to RAIS (e.g., provide for more frequent communication), (22 of 23 surveyed states).

Simplify instruction and procedures for producing reports (18 of 23 surveyed states).

Allow production of customized state and regional reports by type of industry (18 of 23 surveyed states).

Allow production of customized state and regional reports by sponsor type (17 of 23) and occupational type (17 of 23 surveyed states). Improve the frequency of RAIS training (17 of 23 surveyed states). Improve the quality of RAIS training (16 of 23 surveyed states).

Simplify instructions and procedures for inputting and updating data (16 of 23 surveyed states). Increase available coding options to explain why apprentices leave the program (14 of 23 surveyed states).

Allow production of customized state and regional reports by sex of apprentice and race of apprentice (14 of 23 surveyed states).

OATELS has recently purchased software that enables users to extract data from Labor’s databases in order to produce customized reports. Purchased originally for the Secretary of Labor’s use, Labor Information Technology and OATELS officials said they foresaw the software’s utility for many programs and therefore decided to purchase licenses for apprenticeship field staff. However, OATELS has not necessarily taken steps to ensure field staff will be able to make optimal use of the software. About half the directors in federally-monitored states did not know the software was available or what it was. Although the software was demonstrated at a directors’ meeting in 2004, several couldn’t recall the demonstration and others were not in attendance. Moreover, two of the directors lacked basic hardware, such as a high-speed cable needed to support the software. In fact, one director told us he was working from his home because his office didn’t have such basics as a cable hook-up for his computer. Even if such obstacles are surmounted, the new system may not meet the staffs’ data needs. Two directors who were already attempting to use the software reported to us that it did not allow them to select information using factors that would be most useful to them, such as state- level data on apprenticeship programs. In addition, Labor could or would not supply us with formal documentation describing its plans to implement the software or its vision of how the software would be used by its staff. Labor also reported that because of budget constraints and the easy use of the new software, it had no plans to provide training. Without such plans, Labor’s commitment to the full implementation and future financing of the program is questionable.

Labor Has Reviewed Council-Monitored States Infrequently, Provided Little Feedback, and Not Collected Data That Would Allow for a National Picture of Apprenticeships

Labor has infrequently reviewed states to which it has delegated oversight responsibility. This includes both quality reviews and EEO reviews to assure that these states are in compliance with federal rules for overseeing apprenticeship programs and also adhering to equal employment opportunity requirements. Moreover, states that have been reviewed in recent years reported that they had little utility for helping them manage their programs, in part, because of the little feedback they received. In terms of providing information to Congress and others, Labor does not collect from these states information that is readily available on apprenticeships by occupation or industry, even for occupations where shortages of skilled workers are anticipated.

Labor Has Reviewed Council-Monitored States Infrequently in Recent Years

Agency records indicate that Labor conducted only three quality and EEO reviews of council-monitored states in calendar years 2002 and 2003, and none in 2004 but has scheduled seven for 2005. State apprenticeship directors confirmed that reviews are infrequent. Twelve of the 27 directors in council-monitored states reported that OATELS had conducted reviews of their programs less frequently than once every 3 years and several responded that reviews had not taken place in the last 9 to 12 years. An additional five directors reported their states had never been reviewed or that they were unaware if such reviews had taken place. The remaining 10 reported reviews took place in their states at least once every 3 years. (See fig. 3.) While neither statute nor regulation specifies the frequency with which OATELS should conduct such reviews, they constitute an important mechanism for ensuring that state laws conform to requirements necessary for Labor’s recognition of a state’s registered apprenticeship program.

Officials in Most Council- Monitored States Reported Reviews Were Not Very Useful, in Part Because of Limited Feedback

State directors reported that the Quality Reviews and the EEO Reviews had limited utility for helping them manage their programs. For example, only about half of them reported that the quality reviews were at least moderately useful for helping them determine their compliance with federal regulation. (See fig. 4.) Results were similar for the EEO reviews. (See fig. 5.) For example, slightly less than half of state directors reported that EEO reviews were at least moderately useful in helping them determine their compliance with federal EEO regulations. Some directors said reviews would be more useful if they focused on reviewing program- related activities in the state. Eight of the directors suggested that Labor focus more on state and local conditions and the performance of apprenticeship programs instead of focusing only on whether council- monitored states comply with federal standards. For example, one director reported the feedback he received on EEO activities was unrelated to the racial composition of the state. Also, some suggested reviews could provide opportunities for federal officials to provide assistance and share knowledge about strategies that other states have found useful.

While directors had a number of ideas for improving the usefulness of quality and EEO reviews, many noted that Labor provided limited or no feedback as part of the review process. For example, one said his state agency received a brief letter from Labor stating only that the state was in compliance with federal regulations. Two others said their agencies received no documentation that a review had in fact been conducted, even though in one of these cases the state had made requests for the review findings. Officials in one state said feedback from their last review was positive and indicated no problems, but a few years later, OATELS took steps to get their state apprenticeship council derecognized with no prior notice or subsequent review.

Labor Has Not Collected Data That Would Allow for a National Picture of Apprenticeships

Labor collects aggregate counts of apprentices for most council-monitored states and has not developed strategies to collect more detailed information that would allow for a description of apprenticeships at the national level, even for those where shortages of skilled workers are anticipated. Of the 28 council-monitored states, 20 have their own data system and do not report data to Labor’s apprenticeship database. These 20 states represent about 68 percent of the nation’s apprentices. Labor and council-monitored states have differing opinions about why there are separate data systems. Labor officials told us that, as they were developing their database, they conducted outreach to council-monitored states. Officials from these states say otherwise. They also said that participating in Labor’s database would be an onerous process or that Labor’s system did not meet their state’s information needs and, therefore, they had invested the time and money to develop their own systems. Because many of these systems are not compatible with Labor’s, the agency collects only total counts of apprentices and programs from these 20 states, which it uses for its official reports.

While incompatible data systems may suggest that it would be difficult or costly to obtain more than aggregate counts, in collecting data for this report, we found many of the council-monitored states—including 10 with large numbers of apprentices—were both willing and capable of providing us data on apprentices by industry and by occupation as well as information on completion rates, completion times, and some wage data for occupations that we had specified. In fact, one state reported that it had designed its apprenticeship database to collect all information required by Labor’s database and had offered to report these data to Labor electronically—but Labor had not taken steps to accept this offer. Nevertheless, as one director pointed out, having a unified data picture is central to OATELS’ oversight as well as its promotional activities and, as many agree, such a system would promote the health of the registered apprenticeship system.

Construction Apprenticeship Completion Rates and Wages Vary by Program Sponsor

Construction apprentices in programs sponsored jointly by employers and unions (joint programs) generally completed at a higher rate and in greater numbers than those enrolled in programs sponsored by employers alone (non-joint programs). More importantly, despite growth in construction program enrollment, there has been a decline over time in completion rates for both types of programs. Completion rates declined from 59 percent for apprentices enrolling in 1994 to 37 percent for apprentices enrolling in 1998. It is difficult to know what factors underlie this trend because, as noted earlier, Labor does not systematically record information about why apprentices leave programs. Apprentices who completed programs within 6 years tended to finish earlier than expected. In addition, wages for joint apprentices were generally higher at the start and upon completion of their programs. Data received from 10 council- monitored states that do not report to Labor’s database generally mirrored these findings.

Nearly Half of Apprentices in Joint Programs Completed Their Apprenticeships Compared with about a Third in Non- joint Programs

Completion rates were generally higher for apprentices in joint programs than for those in non-joint programs. Of the apprentices who entered programs between 1994 and 1998, about 47 percent of apprentices in joint programs and 30 percent of apprentices in non-joint programs completed their apprenticeships by 2004. For five consecutive classes (1994-1998) of apprentices in Labor’s database, completion rates calculated after 6 years, were higher for joint programs, as shown in figure 6. The data we received from 10 additional states that do not report into Labor’s database showed similar trends, with joint apprentices having higher completion rates. For complete data that we received from these 10 states, see appendix II.

For the programs in Labor’s database, this higher completion rate for joint apprenticeship programs was true for all but 1 of the 15 largest individual trades which collectively account for 93 percent of active apprentices in construction. (See fig. 7.) It should be noted that among the trades, themselves, there were substantial variations in completion rates, often due to the nature of work environment and other constraints, according to federal and state officials. For example, roofing programs, which have low completion rates, face unpredictable weather and seasonal work flows.

Officials said that joint programs have higher completion rates because they are more established and better funded. For some joint programs, these additional resources stem in part from union members paying a small portion of their paychecks into a general training fund that is used to help defray some of the training costs for apprentices. In addition, they suggested that, because unions tend to have a network of affiliates spread across an area, they are more likely to find work for participating apprentices in other areas when work is slow in a particular area. Local union chapters often have portability agreements with one another other, which help to facilitate such transfers. Officials also said these programs provide mentoring and other social supports.

While Enrollments Increased, Completion Rates Declined in General for the Period Examined

Enrollments in construction apprenticeship programs more than doubled from 1994 to 1998, increasing from 20,670 construction apprentices to 47,487. (See fig. 8.) Meanwhile, completion rates declined from 59 percent for the class of 1994 to 37 percent for the class of 1998. This decline for these cohorts held for both joint and non-joint programs. (See fig. 9.) Completion rates for joint apprentices dropped from nearly 63 percent to 42 percent, and from 46 percent to 26 percent for non-joint apprentices. This trend was consistent across different occupations as well, with most experiencing declines.

Because Labor does not systematically record the explanations that apprentices offer for canceling out of programs, it is difficult to determine what may lie behind this downward trend. Labor suggested that some apprentices may choose to acquire just enough training to make them marketable in the construction industry in lieu of completing a program and achieving journey status. While we cannot confirm this hypothesis, we did find that those apprentices who did cancel chose to do so after receiving over a year of training. Joint apprentices cancelled after 92 weeks on average and non-joint apprentices cancelled after 85 weeks on average. Other reasons offered included a decline in work ethic, the emphasis placed by high schools on preparing students for college and the corresponding under-emphasis on preparation for the trades, and a lack of work in the construction industry. We cannot verify the extent to which unemployment played a role influencing outcomes, but, according to the Bureau of Labor Statistics, the unemployment rate for construction increased overall from 6.2 percent to 8.4 percent between 2000 to 2004, despite the predictions of future worker shortages in construction.

Apprentices in Both Joint and Non-joint Construction Programs Tended to Complete Their Programs Early

Those apprentices who completed construction programs within 6 years tended to finish earlier than they were expected to, with apprentices in non-joint programs finishing a bit sooner than their joint counterparts. On average, joint apprentices completed their programs 12 weeks early and non-joint apprentices completed 35 weeks early. This trend was similar across the largest trades in terms of enrollment as shown in table 1 below. This may be due to the willingness of program sponsors to grant apprentices credit for previous work or classroom experience that was directly related to their apprenticeship requirements.

Starting Wages and Wages upon Completion in Joint Construction Programs Were Higher on Average than Those for Apprentices in Non-joint Construction Programs

Apprentices in joint construction programs were paid higher wages at the start of their apprenticeships and were scheduled to receive higher wages upon completion of their programs. In 2004, the first year in which Labor collected information on starting wages, apprentices in joint programs earned $12.28 per hour while non-joint apprentices earned $9.90 at the start of their apprenticeships. These differences in wages were more pronounced at the journey level, that is, upon completion, with apprentices in joint programs scheduled to earn journey-level wages of $24.19 as compared with $17.85 for those in non-joint programs. As shown in figure 10, joint apprentices generally earned higher wages across the 15 trades with the largest numbers of construction apprentices. There were three trades—carpenter, structural steel worker, and cement mason—for which starting wages were higher for non-joint apprentices. For journey-level wages there was only one trade for which wages were higher for non-joint apprentices—that of millwright. Officials we spoke with commonly attributed this distinction in wages to the bargaining process associated with joint programs. Data from the 10 additional states (outside Labor’s database) whose data we examined showed a similar pattern—with joint apprentices earning higher wages. (See app. II.)

Conclusions

As a small program with finite resources tasked with an important mission, it is incumbent on Labor’s Apprenticeship Office to leverage the tools at its disposal to carry out its oversight, all the more so during a period of tight budgets. Labor’s responsibility for assuring that registered apprenticeship programs meet appropriate standards is no small charge, given the thousands of programs in operation today. In terms of the programs it directly monitors, Labor has not made optimal use of the information it collects to target resources. The failure to do so limits the agency’s ability to target its oversight activities to address and remedy areas where there may be significant need, particularly the construction trades where completion rates are declining. Underscoring this point is the fact that apprenticeship directors in federally-monitored states cannot get easy access to the data in the form of customized reports. Irrespective of distinctions between apprentice outcomes for joint and non-joint programs, without better use of its data, Labor is still not in a position to assess programs on their individual merits. Given the relatively limited number of staff available for field visits, by not using the program data it has, Labor misses opportunities to more efficiently use its staff.

With regard to states with council-monitored apprenticeship programs, Labor’s oversight practices do not necessarily ensure that those states’ activities comply with federal standards for oversight because the Apprenticeship Office has only sporadically assessed their operations. Moreover, to the extent that the federal office does not provide useful feedback to the states when it does conduct reviews, states may lose opportunities to improve programs under their jurisdiction. Finally, because Labor does not seek much information beyond aggregate numbers from a majority of council-monitored states, policymakers lose an opportunity to gain perspective and insight for aligning workforce training with national needs, specifically for key occupations within construction that are likely to be faced with shortages of skilled workers in the near future.

Recommendations

We recommend that the Secretary of Labor take steps to (1) better utilize information in Labor’s database, such as indicators of program performance, for management oversight, particularly for apprenticeship programs in occupations with expected future labor shortages; (2) develop a cost-effective strategy for collecting data from council-monitored states; (3) conduct Labor’s reviews of apprenticeship activities in states that regulate their own programs on a regular basis to ensure that state activities are in accord with Labor’s requirements for recognition of apprenticeship programs; and (4) offer substantive feedback to states from its reviews.

Agency Comments

We provided a draft of this report to the Department of Labor for review and comment. Labor provided written comments on the draft report that are reproduced in appendix V. Labor concurred with our recommendations and has already taken steps to obtain data on apprenticeships from some council-monitored states and to regularly review activities in these states. Further, Labor stated it plans to use the data to better target the performance of the apprenticeship programs that OATELS directly registers and oversees, and to provide improved feedback to states that register and oversee their own apprenticeship programs.

Unless you publicly announce its contents earlier, we plan no further distribution of this report until 14 days after the date of this letter. At that time, we will send copies of this report to the Secretary of Labor and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov.

Please contact me on 512-7215 or nilsens@gao.gov if you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Scope and Methodology

Our objectives were to determine (1) the extent to which the U.S. Department of Labor monitors the operations and outcomes of registered apprenticeship programs in the states where it has direct oversight, (2) its oversight activities for council-monitored states, and (3) outcomes for construction apprentices in programs sponsored jointly by employers and unions in relation to those sponsored by employers alone.

To carry out these objectives, we surveyed OATELS officials in charge of apprenticeship programs in 23 federally monitored states and state apprenticeship directors in 28 states, including the District of Columbia, where state apprenticeship councils oversee programs. We used two surveys—one for federally- monitored states and one for council- monitored states—to obtain national information on OATELS’ monitoring and oversight activities. We focused only on apprentices in the civilian sector of the economy and did not include military or prison-based programs. We asked questions designed to determine the amount of resources devoted to oversight, the frequency of oversight activities, and the outcomes from these activities. The surveys were conducted using self-administered electronic questionnaires posted on the World Wide Web. We pretested our surveys with a total of five federally-monitored and council-monitored state officials to determine if the surveys were understandable and if the information was feasible to collect. We then refined the questionnaire as appropriate. We sent e-mail notifications to all federally-monitored and council-monitored state officials on January 5, 2005. We then sent each potential respondent a unique password and username by e-mail on January 13, 2005, to ensure that only members of the target population could participate in the appropriate survey. To encourage respondents to complete the surveys, we sent e-mail messages to prompt each nonrespondent approximately 1½ weeks after the initial e-mail message and a final e-mail reminder on February 7, 2005. We also called nonrespondents to encourage them to complete the survey. We closed the surveys on March 18, 2005. We received responses from all 23 federally-monitored and 27 of 28 council-monitored state officials including the District of Columbia. (See table 2.) Copies of the surveys are provided in appendices III and IV.

To examine the outcomes for apprentices in the construction industry, we analyzed data from Labor’s RAIS database. In calculating completion rates, we constructed five cohorts based on when they enrolled in their programs; we had cohorts for fiscal years 1994, 1995, 1996, 1997, and 1998. We then considered the status of these cohorts 6 years after they enrolled to determine if they had completed, cancelled, or remained in training. Our analysis of wage data focused on data collected in fiscal year 2004, the first full year that Labor began collecting such information. We assessed the reliability of the RAIS database by reviewing relevant information on the database, interviewing relevant OATELS officials, and conducting our own testing of the database. This testing included examining the completeness of the data, performing data reliability checks, and assessing the internal controls of the data. Based on this information and our analysis, we determined that these data were sufficiently reliable for the purposes of our report.

Because Labor’s RAIS database does not contain data from all states, we supplemented these data with data from 10 council-monitored states that do not report to this database. We selected these states based on the number of apprentices they had and whether their data were in an electronic format that would facilitate extracting and sending these data to us. We submitted a data request that asked for selected information on enrollment, completion, and wages for the 10 largest apprenticeship occupations to these states and received data from all of them. We determined that these data were reliable for our purposes. We did not combine these data with those from RAIS; we used them as a means of comparison.

To learn more about the oversight of apprenticeship programs and their outcomes, we conducted site visits to four states—New York, California, Texas, and Washington. These states represented both federal and council- monitored states and had large numbers (from a high of about 52,000 to a low of 6,500) of construction apprentices. On these site visits, we interviewed relevant federal and state officials along with joint and non- joint program sponsors. We also toured facilities in two states where certain apprentices are trained.

Throughout the engagement we interviewed relevant Labor officials and experts that have researched apprenticeship programs and reviewed relevant past reports and evaluations of these programs. We conducted our review from August 2004 through July 2005 in accordance with generally accepted government auditing standards.

Appendix II: Completion Rates, Time Taken to Complete, and Wages for Construction Apprentices in Council-Monitored States

Non- joint

California reported no structural steel worker non-joint programs.

New York reported no completers for pipe fitter, structural steel worker, painter, and operating engineer non-joint programs.

Oregon reported no non-joint apprenticeship programs are registered in the state.

Appendix III: Responses to Survey of Directors of Apprenticeships in Federally- Monitored States

Q4. During FFY 2004, how many full-time equivalency (FTE) apprenticeship training representative, field, and other nonadministrative staff were employed by the state to monitor and oversee apprenticeship programs in your state?

22 Q6. In your opinion, would the following updates or modifications improve Registered Apprenticeship Information System’s (RAIS) usefulness to your state? 23 Q8. Did your state use WIA Governor’s 15% State Set-Aside funds to support new and/or established apprenticeship programs in FFY 2004? 23 Q9. Were WIA State Set-Aside funds used to support new and/or established apprenticeship programs in your state in FFY 2004 to do any of the following?

Q11. For which of the following reasons did your state not use WIA Set-Aside Funds to support apprenticeship programs in FFY 2004? 17 17 Q13. Were WIA funding sources other than State Set-Aside Funds used in your state to support new and/or established apprenticeship programs in FFY 2004?

Q14. Other than State Set-Aside Funds, which of the following WIA funding sources were used to support new and/or established apprenticeship programs in FFY 2004?

Q16. Did your state establish linkages between WIA state unit and the state apprenticeship unit in FFY 2004 for any of the following purposes? 22 Q19. How often does your unit conduct formalized Quality Reviews of individual apprenticeship programs that address on-the-job training, related instruction, and/or program operations in your state?

More than twice a year Q21. Approximately how many Quality Reviews did your unit conduct in FFY 2004? ( Click in the box and then enter up to a 4-digit whole number only. )

Little or no extent ( Please specify in Question 24. )

Little or no extent ( Please specify in Question 24. )

Q26. How often does your unit conduct formalized Equal Employment Opportunity (EEO) Reviews of individual apprenticeship programs?

More than twice a year Q28. Approximately how many EEO Reviews did your unit conduct in FFY 2004? ( Click in the box and then enter up to a 4-digit whole number only. ) 23 Q29. To what extent, if at all, did your state find the FFY 2004 EEO Reviews useful for the following purposes?

Little or no extent ( Please specify in Question 31. )

Q33. Did your state have procedures or policies for recording complaints filed in FFY 2004 that were elevated to the level of the state or regional OATELS office?

Q34a2. Check if actual, estimate, or do not know or cannot estimate Q34b1. How many complaints concerned termination in FFY 2004?

Q34b2. Check if actual, estimate, or do not know or cannot estimate Q34c1. How many complaints concerned discrimination in FFY 2004?

Q34c2. Check if actual, estimate, or do not know or cannot estimate 18 Q34d1. How many complaints concerned wages in FFY 2004?

Q34d2. Check if actual, estimate, or do not know or cannot estimate Q34e1. How many complaints concerned related instruction in FFY 2004?

Q34e2. Check if actual, estimate, or do not know or cannot estimate 18 Q34f1. How many complaints concerned on-the-job training in FFY 2004?

Q34f2. Check if actual, estimate, or do not know or cannot estimate Q34g1. How many complaints concerned other issues in FFY 2004?

Appendix IV: Responses to Survey of Directors of Apprenticeships in Council- Monitored States

Q5. Do you have a BAT agency in your state?

27 Q6. During state FY 2004, how many full-time equivalency (FTE) apprenticeship training staff were employed by the BAT agency in your state to monitor and oversee apprenticeship programs in your state?

Q8. How often does your OATELS conduct SAC 29/29 Review (Review of Labor Standards for Registration of Apprenticeship Programs) in your state?

More than twice a year Q10. To what extent did your state find OATELS’ most recent SAC 29/29 Review (Review of Labor Standards for Registration of Apprenticeship Programs) useful for the following purposes in your state?

Q15. How often does OATELS conduct SAC 29/30 Review (Review of Equal Employment Opportunity in Apprenticeship and Training) in your state?

More than twice a year Q17. To what extent, if at all, did your state find OATELS’ most recent SAC 29/30 Review (Equal Employment Opportunity in Apprenticeship and Training) useful for the following purposes?

Q21. Does your state presently use OATELS’ Registered Apprenticeship Information System (RAIS) to register apprentices and to track apprentice and program information? 27 Q23. Does you state plan or intend to use RAIS to register apprentices and track apprenticeship and program information in the future ?

Q26. Did your state use the WIA Governor’s 15% State Set-Aside funds to support new and/or established apprenticeship programs in state FY 2004?

Q27. Were WIA State Set-Aside funds used to support new and/or established apprenticeship programs in your state in state FY 2004 to do any of the following? 5 Q29. For which of the following reasons did your state not use WIA Set-Aside Funds to support apprenticeship programs in state FY 2004? 22 Q31. Were WIA funding sources other than State Set-Aside Funds used in your state to support new and/or established apprenticeship programs in state FY 2004?

Q32. Other than State Set-Aside Funds, which of the following WIA funding sources were used to support new and/or established apprenticeship programs in state FY 2004? 3 Q34. Did your state establish linkages between WIA and the state apprenticeship unit in state FY 2004 for any of the following purposes?

Q37. Did your state have a mechanism for conducting formalized reviews of apprenticeship programs that address on-the-job training, related instruction, and/or program operations in state FY 2004?

Q38. Which of the following components -- on-the-job training, related instruction, and/or program operations -- were included in these reviews? 25 Q40. How often does your state conduct formalized reviews of individual apprenticeship programs that address on-the-job training, related instruction, and/or program operations?

Q42. Does your state have a mechanism for conducting formalized Equal Employment Opportunity (EEO) reviews of individual apprenticeship programs?

Q43. How often does your state conduct formalized Equal Employment Opportunity (EEO) reviews of individual apprenticeship programs?

More than twice a year Q45. Did your state have procedures or policies for recording complaints filed in state FY 2004 that were elevated to the level of state apprenticeship agencies?

Q46a1. In your state, how many total complaints were referred to state officials in state FY 2004?

Q46a2. Check if actual, estimate, or do not know or cannot estimate 22 Q46b1. How many complaints concerned termination in state FY 2004?

Q46b2. Check if actual, estimate, or do not know or cannot estimate Q46c1. How many complaints concerned discrimination in state FY 2004?

Q46c2. Check if actual, estimate, or do not know or cannot estimate Q46d1. How many complaints concerned wages in state FY 2004?

Q46d2. Check if actual, estimate, or do not know or cannot estimate Q46e1. How many complaints concerned related instruction in state FY 2004?

Q46e2. Check if actual, estimate, or do not know or cannot estimate Q46f1. How many complaints concerned on-the-job training in state FY 2004?

Q46f2. Check if actual, estimate, or do not know or cannot estimate 22 Q46g1. How many complaints concerned other issues in state FY 2004?

Appendix V: Comments from the Department of Labor

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Patrick DiBattista, Assistant Director, Scott Heacock, Linda W. Stokes, and Kathleen D. White managed all aspects of the assignment. The following individuals made significant contributions to this report: Susan Bernstein, Jessica Botsford, Richard Burkard, Cathy Hurley, and Jean McSween.

Related GAO Products

Workforce Investment Act: Substantial Funds Are Used for Training, but Little Is Known Nationally about Training Outcomes. GAO-05-650. Washington, D.C.: June 2005.

Public Community Colleges and Technical Schools: Most Schools Use Both Credit and Noncredit Programs for Workforce Development. GAO-05-4. Washington, D.C.: October 2004.

Registered Apprenticeships: Labor Could Do More to Expand to Other Occupations. GAO-01-940. Washington, D.C.: September 2001.

Youth Training. PEMD-94-32R. Washington, D.C.: September 1994.

Apprenticeship Training: Administration, Use, and Equal Opportunity. HRD-92-43. Washington, D.C.: March 1992.