Job Corps: Links With Labor Market Improved but Vocational Training Performance Overstated
Published: 1998-11-04T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed Job Corps' vocational training component to describe the program's contracting policies and to assess contractor performance, focusing on: (1) how Job Corps ensures that vocational training is appropriate and relevant to employers' needs and the extent to which participants are completing vocational training and obtaining training-related jobs; and (2) Job Corps' process for contracting with vocational training providers.
section_title: What GAO Found
paragraphs: GAO noted that: (1) the Department of Labor has several activities to foster Job Corps' employer and community linkages to ensure the appropriateness of its vocational training to local labor markets and its relevance to employers' needs; (2) Labor has industry advisory groups that regularly review vocational course curricula to ensure their relevance to today's job market; (3) Labor has also introduced a school-to-work initiative designed to link Job Corps with local employers combining center-based training with actual worksite experience at more than half the Job Corps centers; (4) complementing these national efforts, three of Labor's regional offices have developed their own initiatives to improve linkages between Job Corps and local labor markets; (5) despite Labor's efforts to increase the effectiveness of its vocational training through employer and community linkages, Job Corps data on the extent to which participants complete vocational training and obtain training-related jobs are misleading and overstate the program's results; (6) although Job Corps reported that 48 percent of its program year 1996 participants completed their vocational training, GAO found that only 14 percent of the program participants actually completed all the requirements of their vocational training curricula; (7) the rest of the participants whom Job Corps considered to be vocational completers had performed only some of the duties and tasks of a specific vocational training program; (8) Labor also reported that 62 percent of the participants nationwide who obtained employment found jobs that matched the vocational training received in Job Corps; (9) at the five centers GAO visited, however, the validity of about 41 percent of the job placements reported by Labor to be training-related was questionable; (10) in looking at how training providers are selected, GAO found that about a third of Job Corps' vocational training has been provided under sole source contracts awarded to national labor and business organizations for more than 30 years, but in GAO's opinion, Labor has not adequately justified procuring these training services noncompetitively; (11) a principal reason Labor has cited for awarding these contracts on a sole source basis is that these organizations maintain an extensive nationwide placement network and are better able than nonnational organizations to place Job Corps participants who complete their training; and (12) Labor has provided no data, however, to show the extent to which these sole source contractors actually place Job Corps participants nationwide.

section_title: Letter
paragraphs: Job Corps is an employment and training program aimed at providing severely disadvantaged youths with a comprehensive array of services, generally in a residential setting. The Department of Labor spends about $1 billion under this program each year to serve more than 68,000 youths, who, on average, spend about 7 months in the program. On average, the cost per participant is more than $15,000, making Job Corps the nation’s most expensive job training program. One of the services that is critical to the program’s success is vocational training. Each of Job Corps’ 113 centers offers training in several vocational areas, such as automotive trades, health occupations, construction trades, culinary arts, clerical occupations, and building and apartment maintenance. Vocational training services can be provided by Job Corps center staff, private providers under contract to the Job Corps center, or national labor and business organizations under contract with Job Corps’ national office.
In the past, we have assessed various aspects of the Job Corps program—outreach, admissions, and placement; costs and results; similarities to state youth training initiatives; and the extent to which the program is locally based. Reflecting your continued interest in the Job Corps program, you requested that we examine Job Corps’ vocational training component to describe the program’s contracting policies and procedures and to assess contractor performance. Specifically, you asked us to determine how Job Corps ensures that vocational training is appropriate and relevant to employers’ needs and the extent to which participants are completing vocational training and obtaining training-related jobs. In addition, you asked us to describe Job Corps’ process for contracting with vocational training providers, in particular Labor’s use of sole source contracting with national training contractors.
In carrying out our work, we met with Labor officials and reviewed Labor’s policies and procedures for contracting for vocational services. We obtained data on program outcomes for Job Corps participants during the period July 1, 1995, to June 30, 1997, the two most recently completed program years. We visited five centers that used a variety of methods to provide vocational training and that served a majority of local participants—those who resided within 100 miles of a center. At these centers, we reviewed participant data on vocational completion status and placement information. In reviewing job training match information, we used reported placement information and the Dictionary of Occupational Titles and called a number of employers to clarify reported information. We visited a sixth center—one that recently opened—to gather information on its experience in deciding which vocational training courses to offer. We performed our work between January and July 1998 in accordance with generally accepted government auditing standards. (App. I contains a more detailed discussion of our scope and methodology.)
subsections: section_title: Background
paragraphs: Job Corps was established as a national employment and training program in 1964 to address employment barriers faced by severely disadvantaged youths. Job Corps enrolls youths aged 16 to 24 who are economically disadvantaged, in need of additional education or training, and living under disorienting conditions such as a disruptive homelife. In program year 1996, nearly 80 percent of the participants were high school dropouts and almost two-thirds had never been employed full-time. Participating in Job Corps can lead to placement in a job or enrollment in further training or education. It can also lead to educational achievements such as attaining a high school diploma and improving reading or mathematics skills.
Job Corps currently operates 113 centers throughout the United States, including Alaska, Hawaii, the District of Columbia, and Puerto Rico. Major corporations and nonprofit organizations manage and operate 85 Job Corps centers under contractual agreements with Labor. Contract center operators are selected through a competitive procurement process that takes into account proposed costs, an operator’s expertise, and prior program performance. In addition, the U.S. Department of the Interior and the U.S. Department of Agriculture operate 28 Job Corps centers, called civilian conservation centers, on public lands under interagency agreements with Labor. Each center provides participants with a wide range of services, including basic education, vocational skills training, social skills instruction, counseling, health care, room and board, and recreational activities.
One feature that makes Job Corps unique is that, for the most part, it is a residential program. About 90 percent of the youths enrolled each year live at Job Corps centers and are provided services 24 hours a day, 7 days a week. The premise for boarding participants is that most come from a disruptive environment and, therefore, can benefit from receiving education and training in a different setting where a variety of support services is available around the clock. The comprehensive services Job Corps provides make it a relatively expensive program. According to Labor’s program year 1996 figures, the average cost per Job Corps participant was more than $15,000. Cost varies according to how long Job Corps participants remain in the program. Participants stay in the program for an average of about 7 months but may stay as long as 2 years. Labor estimates the cost for a participant who remains in the program for a year to be about $25,000.
Vocational training is a critical element of the Job Corps program. This training is designed to offer individualized, self-paced, and open entry-open exit instruction to allow participants to progress at their own pace. Vocational training can be provided in any combination of three ways. Most vocational training is offered by instructors who are Job Corps center staff. Other vocational courses are taught by private providers under contract to the center. These private providers typically include vocational schools and community colleges. About a third of the vocational training expenditure is provided by national labor unions and business organizations under sole source contracts with Labor. In program year 1996, Job Corps’ operating costs totaled about $986 million, of which $144 million, or about 15 percent, was for vocational training (see table 1).
Overall, Job Corps offers training in 100 different vocations. Although the number of vocations offered at any one Job Corps center varies, most centers offer training in 7 to 10 different vocations. Some centers, however, offer training in as few as 5 vocations while others offer training in as many as 31 different vocations. Some vocations are available at most centers, while others are available at only a single center. For example, more than 80 percent of the centers offer training in business clerical, culinary arts, building and apartment maintenance, and carpentry. Thirty-one vocations, including computer programmer, asphalt paving, barber, teacher aide, and cable TV installer, are offered only at a single center.
Many centers also offer off-site advanced career training at such institutions as vocational schools, community colleges, and universities for participants who have been in the program for at least 6 months. Regardless of who provides the training, Job Corps policy requires that all vocational training programs use competency-based curricula that contain a series of skills, or competencies, that participants must attain. According to Labor officials, each vocational training program’s curriculum and set of required skills are regularly reviewed and updated by industry advisory groups consisting of business, industry, and training providers.
Labor uses a series of nine measures to report on the performance of the program nationally and to assess the performance of individual Job Corps centers. The measures relate to placement—in a job, in education, or in military service—learning gains in mathematics and reading, earning a general equivalency diploma certificate, completing vocational training, placement in a job related to the training received, and placement wage. In program year 1996, Job Corps reported that 80 percent of the participants leaving the program were placed—70 percent in jobs or the military and 10 percent enrolled in education—and 62 percent of those who were placed in jobs or the military obtained a job related to their training. Job Corps also reported that 48 percent of those who left the program completed vocational training.
section_title: Labor’s Initiatives Enhance the Appropriateness and Relevancy of Vocational Training
paragraphs: Labor has several activities to improve Job Corps’ employer and community linkages to ensure that vocational training is appropriate for local labor markets and relevant to employers’ needs. These efforts include initiatives enacted by Job Corps’ national office and regional offices, as well as efforts by individual Job Corps centers.
subsections: section_title: National Initiatives Recognize Need for Employer Involvement
paragraphs: Since 1984, Labor has used industry advisory groups to review vocational course curricula to ensure that course content is relevant to the job market. Each year, Labor selects a number of vocational offerings for review by an Industry Advisory Group consisting of Job Corps instructors and academic program representatives as well as industry representatives from each vocational offering being reviewed. For example, recent industry representatives included computer operators and repair technicians, electronic assemblers, diesel and heavy equipment mechanics, health occupation workers, material handlers, tile setters, and clerical workers. The Industry Advisory Group recommends to Labor changes to Job Corps’ vocational training curricula, materials, and equipment. Vocational offerings are evaluated and updated on a 3-to-5-year cycle dictated by industry changes and the number of students participating in each vocational training program.
In program year 1995, Labor introduced a school-to-work initiative at three Job Corps centers combining center-based training with actual worksite experience related to it. Labor expanded this initiative to an additional 30 centers in program year 1996 and to 30 more centers in program year 1997. Labor provided financial incentives and supportive services to encourage centers to participate in the school-to-work initiative. According to Labor officials, the school-to-work initiatives have resulted in extensive partnerships being established between the centers, area businesses, and local school systems. Through these partnerships, employers are providing worksite learning experiences, suggesting approaches for integrating curricula, developing assessment criteria for documenting skill mastery, and participating in career exposure activities. At one school-to-work Job Corps center that we visited, 35 participants from program year 1996 were involved in this initiative and all were placed—32 had jobs, 2 returned to school, and 1 joined the military. Furthermore, 70 percent of the jobs were directly related to the vocational training received in Job Corps.
Labor also involves local business and community leaders in deciding which vocational training programs are to be offered at newly established Job Corps centers. For example, at the new center we visited, we found that 2 years prior to the awarding of the center’s contract, decisions on the vocations to be offered were made with input from local business and community leaders, including representatives of the mayor’s office, the private industry council, the school department, and local businesses. The result was that this center does not offer many of the traditional Job Corps vocational programs, such as clerical, culinary arts, landscaping, and building and apartment maintenance. Instead, it has nine vocational areas in such high-demand occupations as medical assistant, phlebotomy and EKG technician, and computer repair. At another new center, Labor officials stated that local labor market information along with input from local community and business leaders, including the local private industry council, union representatives, local school system, health groups, and chamber of commerce, ensured that the vocational training courses offered at that center would be appropriate and current given the local economy.
Labor officials also informed us that changes to vocational training offerings at existing centers result from changes in labor market demand or poor performance of a particular vocational training program. Centers obtain approval for a change by completing the appropriate paperwork for a request for change and submitting it to either the regional office (if the change involves a center-operated or center-contracted vocational offering) or the national office (if the change involves a vocational course offered by a national labor union or business organization). Labor then assesses the request to change course offerings and reviews the placement analyses, wages reported, female participation rate in the course, local labor market information, and facility requirements. In addition, Labor requires the center to obtain statements from three employers stating that the vocational change is appropriate and relevant. All five of the centers we visited had recently made changes to their vocational course offerings. For example, one center added a physical therapy course after receiving numerous requests from clinics and hospitals within the community. The center was able to add this course by dropping a cosmetology course. Another center identified a local demand for qualified workers in retail sales and tourism. The center added training in these vocations while reducing the size of its clerical training program.
section_title: Regional Job Corps Initiatives Expand Employer Relations
paragraphs: In addition to national efforts, three of Labor’s regional offices have developed their own initiatives to improve linkages between Job Corps centers and employers. In one region, business leaders representing a variety of industries met with Labor and center staff to provide observations of the program and the participants they hire. The group—a business roundtable—set up a framework for obtaining employer input into the operation of the Job Corps program for the benefit of young people, employers, community leaders, and the Job Corps system nationwide. In an effort to bridge the gap between the needs of private industry and vocational training, the roundtable recommended actions and supported the implementation of new strategies to resolve employer issues that it identified and prioritized. As a direct result of this roundtable, concrete linkages were established. For example, a bank involved as a school-to-work program participant provided equipment and instructors to incorporate bank telling into the center’s clerical program. According to Labor officials, the initiative was successful, and the regional office is currently exploring the possibility of duplicating this effort in several other Job Corps centers. At another center within the region, an electronics firm reviewed the center’s electronics curriculum and suggested additional skills allowing program participants to qualify for higher-paying jobs.
Another region has endorsed a major initiative between a Job Corps center and the Q-Lube Corporation whereby a building at the center was renovated to exactly meet the specifications of a Q-Lube facility. The renovation used student painters and carpenters from other vocational training courses and Job Corps provided additional funding for this course. Q-Lube donated the equipment to the center and also provided a trained instructor. The course offering is identical to the program curriculum Q-Lube teaches at non-Job Corps sites. According to Labor officials, since the implementation of this initiative, Q-Lube has become a major employer and training link within the region.
The same regional office contacted a shipbuilding company advertising for 500 shipbuilders and worked with the company to develop a vocational training program in welding for Job Corps students that would be appropriate and relevant to the company’s needs. The company provided the two pieces of equipment needed for training purposes. Students were trained at the Job Corps center under conditions similar to those in the shipbuilding environment, tested by the company, and then provided additional training at the shipbuilding site. In addition, the company provided low-cost housing and full salary to students who passed the test before graduating from the center. The company was pleased with the students’ qualifications, attitudes, and work ethics and requested that the Job Corps program train another 100 students. The region is currently recruiting and training students for this vocation in an attempt to further meet the needs of the shipbuilding industry.
A third regional office is involved in a project to increase the involvement of employers in all facets of Job Corps operations in their region, including curriculum development, customized training, work-based learning, mentoring, identifying workforce needs, and donating staff resources and equipment. The goal of this outreach campaign is to build substantial relationships between Job Corps and the employer community at several different but mutually supportive levels: center, state, regional, and national. Labor selected a contractor through a competitive process, assisted by several national groups, to research, test, and revise its proposed strategy for increasing employer involvement within the region. Initially, the project concentrated on three centers in different states within the region. The project will soon expand to include all states and Job Corps centers within the region. If successful, the project will be expanded throughout the Job Corps system.
section_title: Local Job Corps Centers Develop Their Own Linkages
paragraphs: Job Corps centers have also independently established linkages with employers. These linkages include negotiating with employers to provide furniture and vocational training equipment and contracting with employers to train and hire program participants. For example, at one center a national employer has donated computers, copy machines, desks, chairs, and conference tables valued at approximately $50,000. At another center, an automobile maker has donated a four-wheel-drive sport utility vehicle for students in the auto repair vocational training course in an attempt to make the training more relevant to the vehicles that students would actually be working on. The center is currently working with the automobile maker to donate a car for the same purpose. Local automobile dealers are familiar with the center’s linkages to the national automobile maker and also have donated cars needing repair. In addition, local automobile dealers have trained students through the school-to-work program and have hired many of the Job Corps program participants.
Another center holds monthly employer relations meetings in which approximately 200 local employers and community representatives attend a luncheon catered by the center’s culinary arts students. Speakers discuss local employment opportunities and donate funds to benefit Job Corps participants. The funds, which are managed by the center’s Community Relations Council, are used to provide tuition scholarships for program graduates continuing their education upon completion from the center. The scholarships range from $500 to $1,000 each and are awarded to program graduates who have pursued excellence and attained a higher measure of success than their fellow program participants. To date, about $10,000 has been raised for scholarships.
A fourth center established an effective business relationship with a computer graphics firm in California. According to center officials, 31 Job Corps students enrolled in various vocational training programs, including building and apartment maintenance, clerical, electrical, and landscaping; participated in 12-week internships at the computer firm; and attended an anger management course that had been developed for the firm’s employees. These students earned $10 per hour within a work-based environment in which the firm’s staff provided on-the-job training and mentoring. The center placement official claims that the success of the internship program is evidenced by the 28 students who obtained primarily training-related jobs after terminating from the Job Corps program.
section_title: Two Job Corps Program Measures Are Misleading and Overstate Program Success
paragraphs: Two performance indicators that Labor uses to evaluate Job Corps’ success are misleading, overstating the extent to which vocational training is completed and job placements are training-related. Labor reports that nationwide about 48 percent of all program participants complete their vocational training and that about 62 percent of the jobs obtained by program participants are related to the training they received. However, we found that nationally only about 14 percent of the program participants satisfied all their vocational training requirements and that about 41 percent of the reported training-related job placements at the five centers we visited were questionable. Having complete and accurate program performance information is important to evaluating program success and being able to identify areas needing improvement.
subsections: section_title: Vocational Completion Does Not Mean Vocational Training Was Completed
paragraphs: Nationally, Job Corps reported that in program year 1996, 48 percent of its participants completed vocational training. This information is misleading. We found that only about 14 percent of the program year 1996 participants actually completed all the required tasks of their vocational training programs. Job Corps’ national data system uses three categories to identify a participant’s level of vocational training progress: trainee, completer, and advanced completer. A trainee is a participant who has not completed any vocational training component, a completer has accomplished at least one component of a vocational program, and an advanced completer has fully satisfied all required components of a vocational training program. Labor considers participants in the last two categories to be vocational training completers. Thus, Job Corps vocational completion statistics include participants who have only partially completed the required skills of a vocational training program.
Each Job Corps vocational training program has a comprehensive list of duties and tasks that participants are expected to perform. For example, the clerical vocational training program has 140 duties and tasks that must be mastered to fully complete the program, food service has 109, building and apartment maintenance has 123, and carpentry has 75. Vocational training programs, however, can be divided into several components. For example, in food service, the first component entails making a sandwich and preparing a salad (covering 39 of the 109 tasks). The second component adds preparing breakfast dishes; heating convenience foods; preparing meats, poultry, fish, and pasta; and cooking vegetables. The final component adds preparing soups, sauces, and appetizers as well as food management skills, such as preparing a menu, setting a table, developing a food preparation schedule, and conducting safety inspections.
Vocational training instructors assess participants’ performance for each duty and task, and Job Corps policy permits participants to be classified as vocational completers if they accomplish the duties and tasks associated with any one component of the vocational training program—regardless of whether they can perform all the duties and tasks required in the entire vocational training curriculum. Depending on the vocation, the percentage of tasks that a participant must accomplish to be considered a completer range from virtually all, as in the health occupations program, to about a quarter, as in the welding program (see table 2).
Thus, Job Corps policy allows participants to be classified as vocational completers if they can perform some portion of a required curriculum. For example, in the food service vocational training program, accomplishing just the tasks associated with the salad and sandwich making component would qualify a participant as a vocational completer. At the centers that we visited that had a food service program, nearly half of the reported vocational completers had completed only this first component. Similarly, nearly 80 percent of the vocational completers in the carpentry program at five centers completed only the first of three components. In contrast, about 15 percent of the vocational completers of the centers’ health occupations program completed only the first of two components (see fig. 1). Overall at the five centers, 43 percent of the vocational completers completed only the first component of their vocational training programs.
The reported percentage of vocational completers at the five centers we visited substantially overstated the percentage of participants who fully completed their vocational training programs. At these centers, about 51 percent of the 3,500 participants were considered to be vocational completers. However, only about 18 percent completed all their vocational training requirements. As shown in figure 2, the percentage of program year participants fully completing vocational training programs ranged from about 11 percent at one center to about 27 percent at another center. Nonetheless, these two centers had reported vocational completion rates of 65 percent and 73 percent, respectively.
Closer examination of the participants who completed only the first component of their vocational training program showed that many spent a short period of time—less than 90 days—enrolled in vocational training. At the five centers that we visited, nearly 15 percent of the participants who had completed the first component of their vocational training spent fewer than 90 days in training. This ranged from about 9 percent at one center to about 20 percent at another center.
section_title: Job Training Match Statistics Are Not Meaningful
paragraphs: Labor reported that in program year 1996, 62 percent of participants placed in employment found jobs that matched the training they received in Job Corps. Our review of this information at the five centers we visited, however, suggests that this report substantially overstates the program’s accomplishments. We found that the validity of about 41 percent of the reported job training matches at these centers was questionable.
In a previous report, we expressed concern with Labor’s methodology for identifying training-related placements. We concluded that Labor gave its placement contractors wide latitude in deciding whether a job was a job training match and identified many jobs that appeared to bear little, if any, relationship to the training received. We also noted that placement contractors used some creativity when reporting job titles in order to obtain a job training match. Labor questioned the accuracy of claims made by placement contractors that job training matches could be obtained for participants trained as bank tellers, secretaries, and welders who obtained jobs in fast food restaurants.
In checking reported job training match information, we reviewed all reported training-related job placements at the five centers we visited to assess the validity of reported job training matches. We verified the results by contacting a representative sample of employers who had hired the Job Corps participants. In this process, we questioned a significant number of the claimed matches. We questioned job training matches because either a job title did not seem appropriate for the employer listed (such as bank teller at a fast food restaurant) or the job title did not seem to relate to the vocational training (such as a job as an information clerk at a car rental agency after training as a home health aide). We then interviewed a random sample of 183 employers who hired Job Corps participants whose job placement was listed as related to the vocational training they received but that we questioned. Table 3 shows additional questionable examples of jobs reported as being training-related.
At the five centers we visited, we questioned 598 of the 1,306 reported job training matches. The percentages of these questionable job training matches ranged from about 30 percent at one center to about 64 percent at another center (see fig. 3).
Our discussions with employers yielded examples of jobs that, on the surface, were related to the training received, based on the reported job title, but were actually quite unrelated to this training. For example, one participant trained in welding was reported as obtaining a job as a welding machine operator at a temporary agency, but the employer informed us that this individual was actually hired to shuttle vehicles between airports. Another participant trained in auto repair was reportedly hired as a petroleum and gas laborer but was actually hired to clean residential homes. A third participant received clerical training and was reportedly hired as a sales correspondent but actually sorted bad tomatoes from good ones on a conveyor belt. All three of these Job Corps participants, therefore, were erroneously reported as having been placed in jobs related to their training.
Labor’s monitoring of reported job training matches appears to be inadequate. Labor officials stated that Job Corps’ regional offices are responsible for monitoring all aspects of placement contractor performance but that there is no fixed schedule for such monitoring. They stated that regular desk reviews of all placement forms, for both accuracy and completeness, takes place as part of the process for paying vouchers submitted by placement contractors. Our findings suggest that there is reason to question whether this procedure is adequate to ensure that reported information is accurate.
section_title: Labor’s Justification for Sole Source Contracting Is Inadequate
paragraphs: Labor has contracted with national labor and business organizations under sole source arrangements for more than 30 years. About a third of Job Corps’ vocational training is provided by such organizations contracted under sole source arrangements. Although Labor has failed to provide adequate support to justify sole source procurement for vocational training, it has nine sole source contracts with national labor and business organizations, totaling about $46 million (see table 4).
Federal procurement regulations require several conditions to be met for an agency to award a noncompetitive contract. These include (1) establishing the need for services that can be provided by only one source, (2) documenting through a market survey or on some other basis that no other known entity can provide the required services, and (3) stating a plan of action the agency may take for removing barriers to competition in the future.
Labor has offered three broad considerations in justifying its sole source awards rather than using competitive procedures in contracting with the national training contractors. The first is the contractors’ past relationship with Job Corps—that is, experience with Labor’s Employment and Training Administration, in general, and with Job Corps specifically and thorough knowledge of Job Corps’ procedures and operations. The second is organizational structure—that is, a large nationwide membership related to a trade and their strong relationship with national and local apprenticeship programs. The third is instructional capability—that is, a sufficiency of qualified and experienced instructors, the ability to provide training specifically developed for the learning level of Job Corps students, and the ability to recognize training as credit toward meeting the requirements of becoming a journey-level worker. In addition, Labor officials stated that a main reason it contracts on a sole source basis is that the contractors maintain an extensive nationwide placement network.
With regard to Labor’s long-standing practice of awarding sole source contracts for a portion of Job Corps’ vocational training, our review of Labor’s current and proposed justification for its sole source contracts and our previous work on this issue raise questions about their use. Labor’s sole source justification essentially lists the qualities Labor expects in a contractor. It does not establish that the services contracted for can be provided by only one source. Furthermore, Labor acknowledged that its national data system has no information to indicate the extent to which national training contractors are directly responsible for placing Job Corps participants in jobs. Labor’s proposed justification for upcoming contracts has many of the weaknesses of the current justification.
section_title: Conclusions
paragraphs: Job Corps is an expensive job training program that provides comprehensive services to a severely disadvantaged population. For more than 30 years, Job Corps has been assisting young people who need and can benefit from an unusually intensive program, operated primarily in a residential setting. Labor and the Congress need meaningful and accurate information if they are to effectively manage and oversee the Job Corps program. However, our work raises serious questions regarding Labor’s claims about Job Corps’ achievements. Labor’s reporting on the percentage of participants who are vocational completers includes many who have not actually completed their training; many have completed only one component of a vocational training program. Similarly, Labor’s reported statistics on the percentage of jobs obtained by participants that were related to the training they received are inaccurate. Reported job training matches include a significant number of jobs that have no apparent relationship to the training received and whose job titles have no apparent relationship to the employers’ business.
In addition, Labor has continued its long-standing practice of awarding sole source contracts for a substantial portion of Job Corps’ vocational training—a practice we suggested it re-evaluate in 1995. To date, Labor has not provided adequate support to justify sole source procurement for vocational training services provided by the nine national labor and business organizations. Labor’s justification for sole source procurement does not explain or demonstrate the basis for Labor’s determination of need.
section_title: Recommendations to the Secretary of Labor
paragraphs: Improvements are needed to ensure that the information used to assess Job Corps program performance is accurate and meaningful. Specifically, two of the measures used to judge the success of the Job Corps program—vocational completion and job training match—provide misleading information that overstates program outcomes. Therefore, we recommend that the Secretary of Labor more accurately define and report information on the extent to which program participants complete vocational training and develop a more accurate system of reporting training-related jobs and effectively monitor its implementation.
In addition, because Labor has not presented adequate justification for its long-standing practice of contracting on a sole source basis with nine national labor and business organizations for vocational training, we recommend that the Secretary of Labor properly justify its use of noncompetitive procedures if it is to continue to award contracts for vocational training services. In so doing, the agency should assess whether vocational training could be served as well through contracts competed for locally or regionally.
section_title: Agency Comments
paragraphs: In comments on a draft of this report, Labor expressed concern about our conclusion that two performance measures—vocational training completion and job training matches—overstated Job Corps’ success and misrepresented its accomplishments. Nevertheless, Labor agreed to implement our recommendations for improving the information provided by these two measures.
Labor emphasized that it did not intend to overstate Job Corps program performance in any area. Labor further noted that it places strong emphasis on performance results and data integrity and is therefore concerned about the findings contained in the report. With regard to vocational training completion, Labor stated that it was never its intention that all students master all competencies on an occupation’s training achievement record. Instead, a set of competencies for each occupational area was developed by Labor, together with industry groups, to identify appropriate competency levels needed to qualify for particular occupations. For example, Labor noted that to qualify as a full mechanic would require completion of all competencies in the automotive area, but a participant could qualify as a mechanic’s helper or brake repair mechanic by completing a subset of the full automotive training achievement record. Labor also noted that even though vocational completion may be an imperfect measure, it is a good predictor of placement, job training match, and wages. However, Labor stated that it understood and shared our concern that the terminology used to report this information may be subject to misinterpretation. Therefore, Labor said that it would take immediate action to clarify the definition of vocational completion in all subsequent Job Corps publications. In addition, Labor noted that because of the perspective gained through the recent oversight hearings and our report, it would review the extent to which the current definition may provide insufficient incentive to some students to obtain the maximum amount of training within the vocational training program. Labor noted that in direct response to these issues, it has initiated a comprehensive and detailed analysis of vocational completion and stated that it will develop a more precise and comprehensive description of student completion levels.
We believe that the actions Labor is taking to more clearly identify what it means by a vocational training completer will avoid future confusion about what is being reported. The actions will also clarify that it is not Labor’s intent to have all Job Corps participants complete all aspects of a vocational curriculum but, rather, to complete to a level that is appropriate for each individual. Such levels, as Labor noted, would correspond to industry-agreed competencies that would qualify a participant for a specific job. In addition, as Labor clarifies and refines its measures, it is likely that more will be learned about the relationship between completing various levels of a vocational program and the degree of success a participant achieves. This is an important aspect of monitoring performance and could lead to program improvements.
Regarding job training matches, Labor stated that it shares our concern about the validity of some of the matches identified in the report. Labor noted that it is currently changing to a different system for determining job training matches that will make the determination more manageable and easier to oversee. This new system is expected to be fully implemented by the close of this calendar year. In addition, Labor stated that it is developing more stringent quality control and oversight procedures to preclude questionable matches.
We believe Labor’s proposed improvements to its assessment of whether job placements are related to the participants’ training and the monitoring of the reporting of these data will improve the validity and utility of this information.
Regarding Labor’s use of sole source contracting with nine national labor unions and business organizations, Labor disagreed that it needed to do more to properly justify its use of noncompetitive procedures and expressed its belief that Job Corps’ training programs could not be served as well through locally or regionally competed procurements. Labor asserts that participants leaving national training contractor programs consistently achieve better outcomes, such as higher wages, than other participants. Labor also points out that it has received negligible responses to the last two invitations for interested organizations to submit capability statements for the administration and operation of vocational training programs and placement activities currently operated by national organizations. Labor contends that the continued strong performance of its sole source contracts and the lack of response to its attempts to solicit other qualified providers properly justify its decision to use noncompetitive procedures. Labor identified some changes, however, including that it will require the national contractors to report monthly on the number of participants who are placed directly into jobs and apprenticeships, and it has established higher performance standards for national training contractors.
We continue to believe that Labor has not adequately justified its use of sole source contracts. Labor has been unable to determine the extent to which national training contractors are responsible for placing participants and thus for their reported better performance. However, Labor’s new requirement for these contractors to report on their placements should improve Labor’s ability to assess their performance. From our review of Labor’s last two invitations for organizations to submit capability statements for the administration and operation of vocational training programs and placement activities, we conclude that the agency did not clearly state the goods and services required and was overly restrictive with respect to contractor qualifications. Thus, we believe that the two published invitations Labor cites were inadequate to inform potentially capable entities of an opportunity to compete or to afford them a reasonable opportunity to provide credible responses. As a result, Labor has not determined the availability of other potential sources and, therefore, has not properly justified its use of noncompetitive procedures.
In addition, Labor suggested two points of technical clarification regarding the approval process that Job Corps centers use to change vocational offerings and the involvement of the national office in an employment initiative in one Job Corps region. We modified the report where appropriate. (Labor’s entire comments are printed in app. III.)
We are sending copies of this report to the Secretary of Labor, the Director of the Office of Management and Budget, relevant congressional committees, and others who are interested. Copies will be made available to others on request.
If you or your staff have any questions concerning this report, please call me at (202) 512-7014 or Sigurd R. Nilsen at (202) 512-7003. Major contributors to this report are listed in appendix IV.