Job Corps was established as a national employment and training program in 1964 to mitigate employment barriers faced by severely disadvantaged youths. Job Corps enrolls youths aged 16 to 24 who are economically disadvantaged, in need of additional education or training, and living in disorienting conditions such as a disruptive homelife. Students may enroll in training programs throughout the year and progress at their own pace. Job Corps provides participants with a wide range of services, including basic education, vocational skills training, social skill instruction, counseling, health care, room and board, and recreation. The program offers vocational skills training in areas such as business occupations, automotive repair, construction trades, and health occupations. Participation in Job Corps can lead to placement in a job or enrollment in further training or education. It can also lead to educational achievements such as attaining a high school diploma and skills in reading or mathematics. Job Corps is unique in that, for the most part, it is residential. About 90 percent of the youths enrolled each year live at Job Corps centers and are provided services 24 hours a day, 7 days a week. The premise for boarding participants is that most come from a disruptive environment and, therefore, can benefit from receiving education and training in a different setting in which a variety of support services are available around the clock. Job Corps operates in a very structured and disciplined environment. For example, established daily routines must be followed, as must specific rules and regulations governing such areas as acceptable dress and behavior. Furthermore, Job Corps participants must have permission to leave the Job Corps center grounds, and participants “earn” home leave, which must be approved before being taken and can be denied for a number of reasons such as failure to follow a center’s rules of conduct. Job Corps typically employs residential staff to oversee dormitory living and security staff for the safety and well-being of its participants. The program recently implemented a “zero tolerance” policy for violence and drugs. This policy includes a “one-strike-and-you’re-out” provision for the most serious violent or criminal offenses as well as for drug violations. Job Corps currently operates 109 centers throughout mainland United States, Alaska and Hawaii, the District of Columbia, and Puerto Rico. Most states have at least one center, and several states have four or more centers. Job Corps’ nine regional directors are responsible for the day-to-day administration of the Job Corps program at the centers within their geographic boundaries. Private corporations and nonprofit organizations, selected through competitive procurement, operate the majority of the centers. However, the departments of Agriculture and Interior directly operate 28 centers, called civilian conservation centers, under interagency agreements. The regional directors are also responsible for overseeing the recruitment of youths for program participation and the placement of participants after they leave Job Corps. Recruitment, referred to as outreach and admissions by program managers, and placement services are provided by private contractors, the centers, or state employment service agencies under contract with the regional offices. During program year 1995, Job Corps spent about $60 million on outreach and admissions as well as placement contracts. This included amounts paid contractors solely for outreach and admissions and placement services. In addition, a portion of the funding for some Job Corps center operation contracts was specifically designated for outreach and admissions and placement services. Job Corps contractors are expected to meet certain levels of achievement in order to continue to participate in the program and receive program funding. A performance standard has been established for outreach and admissions contractors with respect to “quotas” of male and female youths to be enrolled (as specified in the contract), and a second standard relates to the proportion of participants who are to remain in the program for more than 30 days (90 percent). A third standard relates to the percentage of participants who are eventually placed following termination from the program (70 percent). Similarly, placement contractors are required to meet established standards related to the percentage of participants placed in jobs, the military, schools, or other training programs (70 percent). Additional standards are applied to participants who are placed in jobs. These standards relate to the percentage obtaining full-time jobs (70 percent) and jobs directly related to the vocational training received (42 percent). A fourth placement standard relates to the average wage received at placement. Individuals enroll in Job Corps by submitting applications through outreach and admissions contractors. The length of time students stay in Job Corps can vary substantially—from 1 day to 2 years. In program year 1995, about 15 percent of the enrollees left Job Corps within 30 days of entering the program and more than one-fourth left within 60 days. On the average, however, students spend about 7 months in the program. Students leave Job Corps for a variety of reasons, including successful completion of the program objectives, voluntary resignation, disciplinary termination, and being absent without leave (AWOL) for 10 consecutive training days. With a few exceptions, participants terminating from Job Corps are assigned to a placement contractor for assistance in finding a job or enrolling in other education or training programs. Placement contractors are to give priority to finding full-time, training-related jobs for participants. We found that Job Corps’ policy guidance on two of its eligibility criteria was ambiguous and incomplete. As a result, the program’s eligibility process was not following all the requirements of the law or program regulations. The law specifies program eligibility requirements, including age, economic status, educational needs, medical condition, and behavioral condition—all defined in the legislation, implementing regulations, or Labor policy guidance. Another legislative requirement—living in an environment characterized by disorienting conditions—has not been clearly defined in the statute, regulations, or Labor’s guidance. Further, Labor has not provided adequate guidance regarding the requirement that participants have the capability and aspirations to complete and secure the full benefits of Job Corps. Contractors are required to follow Labor’s Policy and Requirements Handbook, which sets out 11 eligibility criteria for the program that all participants must satisfy: age, economically disadvantaged, requires additional education or training, environment, health history, behavioral adjustment history, capability and aspirations to participate, legal U.S. resident, child care, parental consent, and Selective Service registration (see app. II). The first seven are specified in the law. The policy handbook generally provides guidance on what is needed to meet most of these criteria. For example, to be eligible under the education or training criterion, an applicant must be a dropout or in need of additional education, training, or related support services in order to hold meaningful employment, participate in regular school work, qualify for other training, or satisfy armed forces requirements. However, guidance on two of the criteria (environment and capability and aspirations) is vague. One of Job Corps’ eligibility criteria specified in the law for participation in the program relates to environment: A participant must come from “an environment so characterized by cultural deprivation, a disruptive homelife, or other disorienting conditions as to substantially impair prospects for successful participation in other programs providing needed training, education, or assistance.” Program regulations go on to explain that the disorienting condition must be one that would impair the applicant’s chance of success in a nonresidential program rather than a residential Job Corps program. Job Corps legislation, Labor’s program regulations, and Job Corps’ policy handbook list environmental factors to be considered when assessing eligibility, but these sources of program guidance are not entirely consistent nor do they contain adequate definitions (see table 1). With the exception of the regulatory definition of disruptive homelife, program guidance does not define the factors that make up the environmental criterion. In the absence of specific definitions of the environmental criterion, admissions counselors applied their own interpretations. As shown in table 1, Labor includes “limited job opportunities” in its policy handbook as a disorienting condition that fulfills the environmental eligibility requirement. However, none of the sources of program guidance specifically defines this factor or gives any direction to assessment counselors to help them interpret it, nor do they explain how limited job opportunities affect the chance of success in a residential program compared to a nonresidential one. In prior Job Corps regulations, Labor included among “disruptive conditions” that could impair an applicant’s prospect to participate fully in nonresidential training “a neighborhood or community characterized by high crime rates, high unemployment rates, high school dropout rates, and similar handicaps.” Unlike the present regulations, the prior version made clear that applicants might be subject to more than one disruptive factor and that several factors in combination might satisfy this impairment criterion. Labor’s present guidance does not explain how “limited job opportunities” by themselves can satisfy this criterion. Nonetheless, limited job opportunities was the factor cited as fulfilling the environmental eligibility requirement for 92 percent of the 68,000 Job Corps enrollees in program year 1995. Because admissions counselors generally indicate only one environmental factor on the Job Corps application form, we have no way of knowing how many of these participants would have met the environmental criterion had limited job opportunities not been used to fulfill the requirement. Further, the admissions counselors we interviewed had varying interpretations of limited job opportunity. Some thought that it referred to the applicants’ lack of job skills or lack of education, whereas others thought that it referred to the economic condition of the geographic areas in which applicants resided or their being too young or lacking transportation. Cultural deprivation, another eligibility factor that could fulfill the environmental criterion, was not clearly defined—in fact, it is not even listed in Labor’s policy handbook—and was also interpreted differently by various admissions counselors. One contractor referred to persons who had never gone to a museum or the beach; another thought it applied to a situation such as raising a minority child in a nonminority family; a third referred to living in a housing project. Most admissions counselors we interviewed admitted that they had no idea what this term meant. Finally, Labor’s policy handbook restricts what can be considered under the environmental criterion, stating that to be eligible an applicant must be living in an environment characterized by disruptive homelife; unsafe, overcrowded dwelling; limited job opportunities; or disruptive community; high crime rates. However, the handbook excludes cultural deprivation—specified in the statute and Labor’s own regulations—from permitted environmental factors. The Job Corps law states that to enroll in Job Corps, an applicant must, after careful screening, have the present capability and aspirations to complete and secure the full benefit of the program. However, in determining whether applicants meet this requirement, Labor relied primarily on an evaluation form that assesses behavior that would be expected of any and all applicants. Without more detailed guidance on the use of this criterion, the program may not always be serving those who are most likely to benefit from it. In previous work, we found that ensuring that project participants are committed to training and getting a job is a key feature of successful employment training projects. The law does not define “capabilities and aspirations” but leaves to Labor the tasks of defining this term and providing guidance on how it is to be implemented. Labor has developed the “Capability and Aspirations Assessment Tool,” which admissions counselors must complete for each applicant (see app. III). This “tool” formulates four categories of factors—commitment, attitude, capability, and compatibility of applicant and program goals—that are used to assess capability and aspirations and to demonstrate suitability for the program. Factors under commitment include meeting scheduled appointments on time, providing requested documents such as birth certificates, and reacting favorably to program requirements such as following center rules and living away from home. Attitude includes willingly responding to questions and behaving respectfully during the interview. Capability involves obtaining documentation that supports an applicant’s ability to benefit from the program such as school, court, or medical records or a letter from a former employer. Compatibility of applicant and program goals relates to the admissions counselor’s opinion that an applicant’s expressed goals—for example, for job placement or vocational training—can be realistically achieved through Job Corps. The factors specified in Labor’s assessment tool include characteristics that if not displayed would be an appropriate basis for rejecting an application. However, the possession of these characteristics does not necessarily demonstrate that an applicant has the ability and motivation to benefit from Job Corps. Job Corps outreach and admissions contractors and regional staff whom we spoke with pointed out shortcomings in the current approach to assessing applicants’ capability and aspirations. Staff in one of Labor’s regional offices stated that admissions counselors have asked for additional guidance in making better decisions on capability and aspirations. An admissions contractor with statewide recruiting responsibility in one state said that there is a need for a valid assessment tool for this criterion because the current tool is inadequate. Another contractor stated that it filled out Labor’s assessment tool because it is a program requirement but did not use it in assessing the suitability of applicants. One of Labor’s regional offices has started to develop a more meaningful tool. A substantial number of Job Corps participants leave the program within a short time after enrollment—about one-fourth of program year 1995 participants left within 2 months. Therefore, we believed that it would be useful to identify ways contractors could target recruitment efforts and the selection of applicants to the eligible youths who are more likely to stay in the program and, thus, more likely to benefit from it. To determine the factors that might be related to program retention, we visited a number of outreach and admissions contractors to examine their practices in assessing and screening applicants for the program. We also analyzed the characteristics of the more than 68,000 program year 1995 participants to determine the characteristics that were associated with remaining in Job Corps for at least 60 days. In our visits, we identified several procedures that distinguished outreach and admissions contractors with higher retention rates from other outreach and admissions contractors. In general, these procedures were aimed at identifying applicants with the commitment and motivation to remain in and benefit from the program. Our statistical analysis provides some information about characteristics significantly related to the likelihood of remaining in the program for at least 60 days that Labor could use to design outreach efforts, establish priorities among applicants, or improve the retention rate for those who might otherwise leave the program early. Of the 11 outreach and admissions contractors that we visited, those with higher retention rates (10 percent or fewer of their enrollees dropping out within the first 30 days) tended to have better procedures for identifying applicants with the commitment and motivation to remain in and benefit from the program. That is, these contractors emphasized making sure that applicants met the programs’ statutory eligibility criterion of having the capability and aspirations to complete and secure the full benefit of the program. These more-successful contractors’ procedures included “commitment checks” and preenrollment tours and briefings, which gave applicants a more realistic basis for deciding whether they wanted to enroll. The emphasis in these programs was consistent with the finding we reported in a May 1996 report on successful training programs—that a key job-training strategy shared by successful programs was a focus on ensuring that participants are committed to training and getting a job. It was also consistent with the opinions expressed by several regional directors we interviewed. The “commitment checks” contractors’ used were designed to test Job Corps applicants’ initiative. For example, several contractors required individuals interested in Job Corps to set up application appointments. Admissions counselors at four contractors also mentioned that they required applicants to arrive for their meetings dressed in proper attire; otherwise, they had to schedule another appointment. In addition, three admissions counselors required applicants to submit written statements of why they wanted to participate in the program and what they hoped to accomplish. Several admissions counselors required applicants to call weekly between the date of application and the enrollment date to determine the status of their application and to demonstrate their continued interest in the program. Finally, one contractor also used a nine-point checklist of documents that all interested persons had to acquire before they set up their application appointment. Some outreach and admissions contractors considered preenrollment tours and briefings to be extremely useful, although they were not practical in every situation. They provided applicants with a firsthand opportunity to obtain a thorough understanding of Job Corps rules and requirements, observe the living conditions, erase false expectations, and determine whether they were suited for regimented life. In some instances, these preenrollment briefings were given prior to application while others took place afterward. For example, one contractor required that all interested individuals attend a prearranged tour and briefing. After taking the tour, attending the briefing, and participating in a question and answer session, those still interested had to set up an appointment to complete an application. Another contractor required potential enrollees to take a tour after the application process. Following the tour, applicants attended a briefing and question and answer session, followed by one-on-one interviews with center staff. The value of preenrollment tours and briefings was also confirmed by Job Corps participants at two of the centers we visited who thought the tours and briefings were definitely worthwhile and by two regional directors who agreed that the preenrollment tours and briefings were very effective in preparing applicants for Job Corps and in improving program retention. These tours and briefings would help meet the law’s requirements that applicants be given a full understanding of Job Corps as well as what is expected of them after enrollment. Several regional directors commented on the importance of identifying applicants who are ready for Job Corps and can benefit from its training. For example, one regional director stated that because the program cannot afford to squander its resources on applicants who do not really want to be in the program, admissions counselors should ensure that applicants are ready and can benefit from the investment. Another regional director noted that because so many people are eligible for Job Corps (over 6 million) it was important to provide this opportunity to those most likely to benefit and that commitment should be “first and foremost” when assessing applicants. Another regional director agreed that commitment was important but considered the program’s Capability and Aspirations Assessment Tool to be ineffective in measuring it. In our analysis, we identified several characteristics associated with program retention that Labor might consider in designing outreach efforts, establishing priorities among applicants, or improving participant retention rates. Some of these characteristics would be of limited value nationwide, however, because so few participants nationwide had those characteristics. In addition, when considering how to use the results from our analysis, Labor also needs to consider other factors. Two of the characteristics most strongly related to the likelihood of remaining in the program were need for bilingual education and years of education. Of the characteristics we examined, the need for bilingual education had the strongest relationship with the likelihood of remaining in the program. Participants needing bilingual training—Spanish as well as other languages—were much more likely than others to remain in the program for at least 60 days. Education was also an important factor—participants with 12 or more years of education were more likely to remain than participants with 8 or fewer years of schooling. Another characteristic with a strong relationship to retention was age. Our analysis indicated that older participants had a greater likelihood than younger participants of remaining in the program. Specifically, when compared to 15-17-year-old participants, those aged 18 to 20 and 21 to 25 were more likely to remain in the program for at least 60 days. This analysis supported the concern expressed by many of the admissions counselors we interviewed regarding enrollment, retention, and placement of 16- and 17-year-old youths, who make up nearly 40 percent of the program year 1995 enrollees. The concerns they expressed were that these younger youths are often victimized by older participants at the center, have a harder time adjusting to center life, are more likely to drop out, and are difficult to place. Labor program year 1995 outcome data showed that 16- and 17-year-old terminees were less likely to be placed once they left the program (see fig. 1). Because of the difficulty in placing 16- and 17-year-old participants, one regional Labor official believed that the minimum age for enrollment should be increased, while another thought that there should be separate standards for these participants. In contrast, a third regional Labor official thought that maturity, and not age, should be the deciding factor for enrollment. He acknowledged, however, that the program should probably have different expectations and performance standards for 16-year-old participants. Another Labor official told us that a work group has been established to look into the problem of serving 16- and 17-year-old participants. Appendix IV discusses our statistical analysis of characteristics related to remaining in the program at least 60 days, including limitations associated with the analysis. Table IV.3 in that appendix contains the final model and significance levels. For example, it shows that other factors that had a significant relationship to the likelihood of remaining in the program for at least 60 days included residing less than 50 miles from the assigned Job Corps center, being a nonresidential student, having no dependents, and having served in the military. Additionally, some of the factors that proved to be useful predictors of remaining in the program were characteristics of only small subsets of participants. For example, because relatively few participants had a need for bilingual education (less than 3 percent of the Job Corps population), that characteristic was limited in its value as a feature for nationwide use in screening. Because we found no large subgroups with great differences, the ability of the model we used in our analysis to predict 60-day retention for the program’s full population is limited. In deciding how to use the results of this analysis, Labor would need to consider more than the statistical results. For example, it would clearly be inappropriate to use these findings to exclude applicants who met the statutory eligibility requirements because they had characteristics associated with a low likelihood of completing the program. If Labor chose to consider these characteristics in designing outreach efforts or establishing priorities for eligible applicants, it would be faced with the complexity of integrating these results with existing eligibility requirements and program policy. For example, our results showed that participants with at least 12 years of education were more likely to remain for 60 days than those with less education. Many youths with that many years in school, however, might not meet the eligibility requirement of needing additional education or training to secure and hold meaningful employment, participate successfully in regular school work, qualify for other suitable training programs, or satisfy armed forces requirements. The most clear-cut use of this information on participant characteristics may be in designing efforts to improve the retention rate of participants with characteristics associated with leaving the program early. Labor uses performance measures in deciding whether contractors are to continue to participate in the program. However, Labor does not have the information it needs to accurately assess the performance of its placement contractors. We found that two of the four measures Labor used in assessing placement contractor performance were not meaningful. One of the measures held contractors accountable for placing participants who were realistically unemployable. A second measure, relating to the placement of terminees in training-related occupations, included terminees who received little vocational training and also gave placement contractors wide latitude in deciding whether placements were related to training. Job Corps requires placement contractors to assist all terminees with placement regardless of how long they were in the program or the reason they left, and it has established the following standards to measure contractor performance: 70 percent of all terminees assigned to a contractor are to be placed, 70 percent of all placements are to be in full-time jobs, the average wage paid to participants placed in jobs is to be equal to or greater than a specified level, and 42 percent of all job placements are to be in occupations related to the training received. In calculating a contractor’s placement performance, Labor includes participants who remained in the program for as little as 1 day, those who were AWOL, and those who were expelled from Job Corps after 30 days for using drugs or committing violent acts—all individuals a placement contractor would have difficulty recommending for employment. During program year 1995, about one-third of the participants leaving Job Corps were in these categories. If Labor’s methodology were modified to include only participants who were in the program for sufficient time to obtain at least minimal benefits (that is, stayed for at least 30 days) and were employable (that is, were not terminated for drug violations and violence and were not AWOL), the average placement rate for the 12 placement contractors we visited would be about 8 points higher—ranging from an increase of 2.6 points for one contractor to 13.6 points for another contractor—and the rank order among the 12 contractors would change somewhat. (See fig. 2.) About half of the placement contractors we visited suggested that Labor should exclude certain individuals when calculating placement rates. For example, one contractor noted that it is unreasonable to expect contractors to recommend to an employer someone who was expelled for taking drugs or committing a violent act. Another contractor believed that it was a waste of resources to try to place participants who were AWOL because they were not only difficult to locate but also undependable to an employer. A third contractor suggested that Labor’s methodology include only participants who are truly employable. Similarly, a regional director stated that it is ridiculous to require placement specialists to be responsible for placing participants who stayed in the program a very short time, were expelled for drug use or violence, or were AWOL. He said that this responsibility asks the placement specialist to lie to employers by recommending they hire these people. Another regional director agreed that placement contractors should not be responsible for participants who received no benefit from the program or who were kicked out for violating the program’s drug and violence policies. The job-training match measure is used to evaluate the effectiveness of vocational training programs and placement contractors by determining the percentage of jobs terminees obtain that matches the training they received while in Job Corps. Labor allows placement contractors wide discretion in deciding whether a job placement they obtain for a terminee is related to the training received—another measure of performance. At the same time, Labor requires that terminees who receive little vocational training be included in the calculation of this measure. As a result, the value of the current job-training match performance measure is questionable. Labor is developing a new system to determine job-training matches that, it believes, will be more accurate. Labor’s guidance gives placement contractors wide latitude in deciding whether a job placement was a job-training match. According to Labor guidance, a job-training match results when a participant is placed in a job requiring skills similar to those included in the participant’s training. Placement contractors are responsible for recording this information. Labor’s guidance for these decisions consists of 16 broad categories of training programs, and within each category are a varying number of detailed occupations in which Job Corps participants may be trained. In addition, each of the 16 broad categories contains a list of jobs that would be considered a match with the training received. To illustrate, the broad training category of construction trades includes 47 detailed training occupations and 357 placement occupations. An individual who was trained in any one of the 47 training occupations and then was placed into any one of the 357 placement occupations would be counted as having made a job-training match. Overall, Labor’s system includes nearly 300 detailed training occupations and more than 5,700 job placement occupations. In addition to the wide range of jobs that are considered to be training matches under each of the broad training categories, Labor’s guidance includes jobs that appear to bear little, if any, relationship to the training received. For example, a position as a key cutter would be considered to be a training match for any of the 51 training categories under the broad category of mechanics and repairers, which includes auto mechanic, electronics assembler, and parts clerk. A position as a general laborer would be considered to be a job-training match for any of the 30 training occupations under the precision production category, which includes mechanical drafter, sheet metal worker, and welder. Table 2 lists examples of some possible matches under Labor’s guidance. Many of the positions that are considered to be related to Job Corps training require relatively little training to perform. The job placement occupational categories contained in Labor’s guidance for job-training match come from its Dictionary of Occupational Titles. The dictionary includes, for each occupation, the average time required to learn the techniques, acquire information, and develop the facility for average performance in a specific job situation. For more than 700 of the jobs included in Labor’s guidance, the average training time is indicated as either only a short demonstration or training up to and including 1 month. Thus, Labor is allowing job-training match credit for occupations requiring relatively short training time even though participants spend an average of about 7 months in the program at an average cost of about $15,300 each.While we recognize that some of these positions provide entry into an occupational area that may lead to a better job, in our view it is questionable to consider such positions to be a job-training match until the participant advances into a job commensurate with the training received. Further, Labor guidance encourages placement contractors to search among the allowable jobs for a job-training match. Its policy handbook states that, if a job-training match is not generated when a job placement code is entered in its automated system, the placement contractor is allowed to enter a different code that may generate a job-training match, “so long as integrity of data is maintained.” We found that placement contractors’ practice of recording job-training matches does indeed raise questions about the integrity of the data. One contractor told us that if a placement specialist obtained a job for a terminee that was not a job-training match under Labor’s guidance, then the manager and placement specialist would meet to determine how to make it a match. This same contractor claimed that it is possible to get a job-training match in fast-food restaurants for participants trained as bank tellers, secretaries, and welders. For the most part, the placement contractors we visited similarly indicated that creativity is used when entering the code for the placement job in order to obtain a job-training match and raised concerns about the validity of reported job-training match statistics. The job-training match performance measure may also unfairly hold placement contractors accountable for placing certain terminees in training-related jobs. All participants placed in a job or the military are included in the calculation of job-training match, regardless of how long they received vocational training. Thus, participants for a few days or weeks who had little chance to participate to any extent in vocational skill training would be included in the calculation of the job-training match measure. Most of the placement contractors and regional staff we spoke with agreed that when calculating this measure it would be more meaningful to include only participants who completed their vocational skills training. Labor officials told us that they are revising the methodology for determining job-training matches. The proposed methodology will use an existing system used by the Bureau of Labor Statistics to collect occupational employment data by various industry classifications. This system uses about 830 five-digit codes rather than the 5,700 nine-digit codes used in the current methodology based on the Dictionary of Occupational Titles. In its comments on a draft of our report, Labor acknowledged that we made a legitimate point about the need to strengthen the job-training match process. According to Labor, the proposed system will be more accurate and easier to maintain and monitor in terms of egregious job-training matches. Labor hopes to have implemented the new methodology by July 1, 1998. In addition, Labor stated that the job-training match issue is one of the primary projects being addressed by a Job Corps committee to improve the quality of vocational outcomes. We found that a characteristic common to the contractors we visited that had higher placement rates was having staff solely responsible for providing placement services to Job Corps participants. In addition, most of these placement contractors were either Job Corps centers or had staff located at the centers they served. In contrast, Labor regional officials have been concerned with the performance of state employment service agencies and have not renewed many of their contracts during the past 2 years. We also noted that Labor and several of the Job Corps centers we visited were starting to improve links to the business community in an effort to increase placements. The placement contractors we visited had had varying success in placing Job Corps participants in program year 1995. Placement included getting a job, entering the military, or returning full-time to school. The seven contractors that had relatively high placement rates (over 73 percent) included four Job Corps centers and three private organizations. A common characteristic among these contractors was having staff who had only one responsibility—placing Job Corps participants. Other contractors that were not as successful used the same staff to perform outreach and admissions as well as placement. One contractor whose staff performed these functions noted that with the program’s emphasis on maintaining centers at full capacity, placement is often secondary to admissions. We also noted that most of the contractors with higher placement rates were either Job Corps centers or had staff at the center. Placement specialists at the Job Corps center contended that being at the center allowed them easy access to instructors, counselors, and participants. One Labor regional director also mentioned the importance of having a continuity of services from the time enrollees arrive at the center until they are placed, noting that it was no accident that every center in his region also has a placement contract. In contrast, the placement contractor we visited with the highest placement rate was not a Job Corps center and did not have staff at a center. The program manager of this private company viewed Job Corps placement as a business and ran the organization accordingly—either placement specialists produced jobs for Job Corps participants or else the program manager found someone who could. Thus, having a focus on the ultimate goal—placement in a job—is a strategy associated with a high placement rate. One type of contractor that generally has not had high placement rates is state employment service agencies. Between program years 1994 and 1996, Labor regional offices did not renew two-thirds (12 of 18) of the placement contracts they had with state employment service agencies (see table 3). Labor officials in three regional offices informed us that they cancelled the placement contracts with state employment service agencies because of poor performance. A Labor official in a fourth region stated that the agency had sent a letter of concern to the state employment service agency because it was the worst-performing placement contractor in the region. Five of the six remaining state employment service placement contractors had placement rates in program year 1995 below the national Job Corps standard of 70 percent. Officials from two of the three state employment service agencies we visited expressed reservations about continuing to contract with Job Corps for placement services. For example, one employment service official said that the agency might not seek contract renewal because of its strained relations with Labor’s regional office. An official from another employment service commented that its Job Corps contract was really “small potatoes” and insufficient to provide for adequate staffing and that the only reason it was still involved was that the employment service commissioner believed that Job Corps was worthwhile and wanted to assist disadvantaged youths. An official from the third employment service agency we visited noted that the Labor regional office threatened to cancel its placement contract 2 years ago for poor performance and gave the agency another 6 months to improve. The official noted that, under new management, performance did improve and Labor renewed the agency’s contract for another 2 years. Placement specialists at the three employment service offices we visited stated that they have no contact with Job Corps participants before their termination. It also appeared that the major placement emphasis was to register Job Corps participants in the employment service databank. While this did provide access to a major source of potential jobs, it was the same service provided to regular job seekers using the employment service and was not any kind of specialized assistance. As pointed out by the Chairman of the Senate Subcommittee on Employment and Training, Committee on Labor and Human Resources, during hearings on Job Corps in April 1997, a key to program success is the development of links to the business community. However, concerns were raised about whether Job Corps had developed such links. We noted that several of the centers we visited that had higher placement rates also had good relationships with local businesses. For example, one center had established a physical therapy program to meet the needs of local health facilities, and another center used temporary agencies as a springboard for their computer services trainees to gain access to the area’s computer industry. A third center was working on improving its work experience component to better match participants’ skills and abilities to the needs of local businesses so that more permanent hires would result. Labor regional offices are also exploring ways to improve links to the business sector. For example, one office has recently started a business roundtable of 18 employers in the region who discuss placement issues. Another regional office has begun a project to get local employers involved with training and placement. The idea is to have employers identify what they need in terms of training curriculum, equipment, and skills and then determine how the program can meet these needs. Recognizing the importance of employer links, Labor has launched a new school-to-work initiative within Job Corps to involve more employers in placing program terminees and to establish the basic framework for a school-to-work program. It started as a pilot program at three Job Corps centers and will be expanded to 30 centers this year. Further expansion will depend on the availability of funding. Labor’s program guidance to admissions counselors on two eligibility requirements was ambiguous and incomplete. One of the program’s eligibility criteria—living in an environment characterized by disorienting conditions—has not been clearly defined in the statute, regulations, or Labor’s guidance. In addition, Labor has not provided adequate guidance regarding the requirement that participants have the capability and aspirations needed to complete and secure the full benefits of Job Corps. As a result, outreach and admissions contractors may not be enrolling the applicants who are most appropriate for the program. In the absence of specific Labor guidance, we noted that outreach and admissions practices varied among contractors. Those with higher participant retention rates tended to have better procedures to identify applicants who have the capability and aspirations to remain in and benefit from the program. A particularly effective tool in preparing applicants for Job Corps appeared to be preenrollment tours and briefings. Most admissions counselors expressed concern about the enrollment of 16- and 17-year-old applicants. Labor data confirm that these youths are more likely to drop out early for disciplinary reasons and less likely to be placed once they leave the program. Although Job Corps is a performance-driven program, the measures used to assess placement performance may not be meaningful and thus may not provide Labor with the information it needs to accurately assess placement contractor performance. Labor’s system for calculating a contractor’s placement performance included program terminees who were realistically unemployable. Determining what happens to every program participant is an important indicator of how well Job Corps is performing but not necessarily an appropriate measure of a contractor’s placement performance. Guidance related to another placement measure—the extent to which terminees were placed in training-related occupations—gave contractors such wide latitude when deciding whether a job was related to the training received that the validity of the measurement was questionable. In addition, the performance measure included terminees who received little vocational skills training and, therefore, were unlikely to be placed in jobs requiring an acquired skill. Labor is redesigning the methodology for determining job-training matches, which may help address some of these problems. However, any system would still be susceptible to manipulation by placement contractors without proper oversight and monitoring. To help ensure that Job Corps’ resources serve the most appropriate participants, we recommend that the Secretary of Labor provide clear and complete guidance on program eligibility criteria, ensuring that the guidance is consistent with the law, and provide better guidance to ensure that outreach and admissions contractors assess each applicant’s capability and aspirations to complete training and attain a positive outcome. Improvements are also needed to make the measures used to assess placement contractor performance more meaningful. Therefore, we recommend that the Secretary of Labor modify certain measures for placement contractors, including eliminate from the placement pool participants whom contractors realistically could not or should not be expected to place, such as participants who were expelled for criminal or violent behavior; replace the current job-training match system with one that captures realistic information and provide guidance to regional offices to ensure that the data are accurately recorded; establish separate placement performance standards for participants with different levels of program accomplishment—for example, those who completed program requirements and those who dropped out early. In Labor’s comments on a draft of this report, the agency disagreed with our recommendation that it clarify and expand its program eligibility criteria in order to ensure that they are consistent with the law. Labor stated that our report lacked acknowledgment of the detailed specifications for eligibility requirements developed over the years in conjunction with the Office of Inspector General and that the eligibility, verification, and documentation requirements contained in its policy handbook are detailed and specifically related to guidance for Job Corps admissions counselors. Labor gave no indication of any formal action it planned to take on this recommendation. Although Labor expressed some concern with our remaining recommendations, it acknowledged that they have merit, warrant consideration, and identify actions that the agency would take in response to them. Labor’s specific concerns with our report are in three broad areas—adequacy of program eligibility guidance, potential effect of additional assessment procedures, and recommended changes to placement performance measures, including training-related placements. Labor also pointed out a number of items in the draft report that it believed should be modified or clarified, and we acted on these, where appropriate. For example, we clarified that our discussion of the ambiguity of program eligibility guidance related to only 2 of the 11 criteria. We also made a number of other technical changes to our report to respond to Labor’s comments. Following is a summary of Labor’s concerns and our responses. Labor’s full comments are printed in appendix VI. Labor stated that our report lacked acknowledgment of the detailed specifications for eligibility requirements developed over the years in conjunction with the Office of Inspector General and that the eligibility, verification, and documentation requirements contained in its policy handbook are detailed and specifically related to guidance for Job Corps admissions counselors. Labor expressed concern with our characterization of the program eligibility guidance as inadequate. For example, regarding the lack of definition in Labor’s policy handbook for “limited job opportunities,” Labor commented that training conducted in program year 1995 for all admissions counselors included technical assistance material that defined this term as follows: “scarcity of jobs, commensurate with the skill levels of Job Corps-eligible youth and which has been designated as an area of substantial unemployment.” Labor added that “In essence, any applicant who lacks the specific skills required by the local labor market to obtain meaningful employment is a legitimate candidate for Job Corps.” Labor acknowledged that another eligibility factor—cultural deprivation—is not included in the policy handbook because more-specific factors—including (1) disruptive homelife, (2) unsafe or overcrowded dwelling, (3) disruptive community with high crime rates, and (4) limited job opportunities—were more useful to admissions counselors than the general term itself. Finally, Labor expressed concern with our discussion of the tool used in assessing another eligibility requirement—capability and aspirations. According to Labor, this assessment by its very nature must rely on the judgment of admissions counselors and determining aspirations is very difficult and challenging; Labor stated that the current assessment tool will be revisited and modified according to suggestions from regional offices and admissions counselors. We disagree that sufficient policy guidance defining “limited job opportunities” was provided to admissions counselors at a training seminar. Even if all admissions counselors at that time received such guidance, contractors and staff have since turned over. And, as mentioned in our report, admissions counselors we interviewed had different interpretations of “limited job opportunities,” indicating that something more is needed to ensure the consistent interpretation of limited job opportunities. Because Labor’s policy handbook was created to be “the single document containing all policy and requirements which would be: clear and concise, and up-to-date, and consistent with legislative provisions,” any definition of “limited job opportunities” that Labor develops should be incorporated into this policy handbook. In addition, the law states that environmental factors substantially impair an individual’s ability to succeed in training, not his or her ability to find employment. But Labor fails to explain the connection between its definition and the impairment of ability to succeed in training. And there is a separate eligibility requirement in the law that the applicant must “require additional education, training, or intensive counseling and related assistance in order to secure and hold meaningful employment . . . .” Labor’s interpretation of limited job opportunities appears to duplicate or at least overlap that separate requirement. Finally, Labor fails to explain how its definition satisfies the program regulations that stipulate that the environmental criteria are to be used in the context of residential versus nonresidential programs. Nowhere in its guidance does Labor mention this distinction. We also disagree that Labor provided adequate guidance regarding the term “cultural deprivation.” On the Job Corps application form, Labor not only lists each of the four factors it says define “cultural deprivation” as separate and distinct eligibility factors (any one of which would satisfy the eligibility requirement) but also adds the term “cultural deprivation” as a fifth factor that can be used to meet program eligibility. Guidance for completing the application form does not define this term and, as noted in our report, most of the admissions counselors we spoke with admitted that they did not know what the term meant. Furthermore, cultural deprivation cannot include disruptive homelife, as Labor says it does, because the law lists these as two separate environmental conditions. Regarding the eligibility requirement that participants have the capability and aspirations to complete and benefit from Job Corps, we agree with Labor that making such a determination is very difficult and challenging and, therefore, we believe that it is important that admissions counselors have guidance adequate to assist them in making these judgments. Furthermore, we agree with one regional official’s portrayal of the current assessment tool as a beginning step in providing guidance on this criterion. Accordingly, we support Labor’s decision to revisit this assessment tool and to obtain regional office and admissions contractors’ suggestions for improving it. With respect to assessment procedures, Labor agreed that Job Corps should not enroll youths who obviously have no desire to be in the program or capability to succeed and that assessing the interest and ability to benefit are important parts of the intake procedure. Labor also noted that participants’ leaving the program within the first 2 months is a cost that Job Corps must do whatever it can to minimize. However, Labor points out the need for a balance between this goal and the goal of serving youths who truly need the program, noting that overly strict assessment procedures could be a barrier to many severely disadvantaged youths. Furthermore, Labor states that the Congress clearly intended that Job Corps serve a severely at-risk population. Labor acknowledged that our report contained a number of positive suggestions (that is, “best practices”) that will be made available to outreach and admissions as well as placement contractors. Labor cautioned that the results of our analysis of characteristics associated with program retention could be misinterpreted because the report lacks the proper context. Labor further suggested that the detailed appendix related to this discussion be removed. Finally, Labor stated that the age data relating to participants who were 15 and 25 years old was inaccurate because Job Corps serves individuals aged 16 to 24. While we do not disagree that the program is to target persons most in need, the law states that the purpose is to assist youths who both need and can benefit from an intensive program. And the law requires that enrollees have the capabilities and aspirations to complete and secure the full benefits of the program. Several Labor regional directors commented on the importance of identifying applicants who are ready for Job Corps and can benefit from its training. For example, one director stated that with more than 6 million people eligible for Job Corps, admissions counselors have to identify those most likely to benefit from the program and that commitment should be first and foremost when they assess applicants. We also note that, in a previous report, we found that a key element of successful job-training projects was ensuring that participants are committed to training and to getting a job. Accordingly, we endorse Labor’s decision to make available to admissions contractors the procedures noted in our report that help identify the applicants who have the commitment and motivation to remain in and benefit from the program. We modified the report to provide our reasons for performing our analysis of characteristics associated with program retention and to highlight the limitations associated with our approach as well as the results. However, we do not believe the detailed appendix should be eliminated. In addition to describing our analysis and results in detail, it describes the related limitations. Regarding our mention of 15- and 25-year-old program participants being inaccurate, we obtained our data from Labor’s national database, which showed that less than 1 percent of program year 1995 enrollees were either 15 or 25 years old. We have added a relevant footnote. Labor expressed concern with our recommendation with respect to placement performance measures that Job Corps eliminate from the contractors’ placement pool individuals who realistically could not or should not be expected to be placed, such as those expelled from the program for using drugs or engaging in violent behavior. Labor believes that the program has the responsibility to provide placement services to all participants and that it is not asking placement contractors to mislead or lie to employers during placement. Labor further commented that the current placement measure resulted from a recommendation by its Office of Inspector General that all participants who leave the program should be included in the placement pool, thus creating incentives to keep students as long as possible. Labor acknowledged that the points we made in this portion of the report merit serious consideration and, therefore, it will convene a workgroup to discuss our recommendations and examine the incentives and disincentives resulting from any proposed changes to the performance management system. Labor also acknowledged that our report contained “some good points” with respect to training-related placements but expressed concern about our use of hypothetical examples of questionable job-training matches and the lack of data to indicate the degree to which these occur. Labor also commented that the claim by a contractor about obtaining a job-training match for participants trained as bank tellers, secretaries, and welders and placed in fast-food restaurants is inaccurate, noting that the system does not permit such matches. Although Labor may not be asking its placement contractors to lie to or mislead employers when attempting to place individuals who realistically could not be placed, by holding contractors responsible for placing individuals expelled for criminal or violent behavior, the program may be encouraging such practices. We agree with Labor that determining what happens to every participant is an important indicator of program performance, but we do not believe that it is necessarily an appropriate measure of a contractor’s placement performance. We also acknowledge that establishing an effective performance management system is complex and agree with Labor that, before any changes are made to this system, the incentives and disincentives should be thoroughly examined, and we commend Labor for its proposed action. We used “hypothetical” examples of job-training matches to illustrate the wide latitude Job Corps permits. Labor data were not available to identify the extent of abuse, but as we mentioned in the report, most placement contractors we interviewed indicated that creativity is used when entering codes for placement jobs, and they expressed their concern about the validity of reported job-training match statistics. In response to Labor’s contention that the system does not permit job-training matches for participants trained as bank tellers, secretaries, and welders who obtain jobs in fast-food restaurants, we agree that if such jobs were reported as “fast-food workers,” the system would not permit a job-training match. But, as a contractor we spoke with pointed out, reporting such jobs in fast-food restaurants as “cashier” would be an allowable match for participants trained as bank tellers and secretaries, and reporting such placements as “machine cleaners” would be an allowable match for participants trained as welders. As arranged with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 15 days after its issue date. We will then send copies to the Secretary of Labor, the Director of the Office of Management and Budget, relevant congressional committees, and others who are interested. Copies will be made available to others on request. If you or your staff have any questions concerning this report, please call me at (202) 512-7014 or Sigurd R. Nilsen at (202) 512-7003. GAO contacts and staff acknowledgments are listed in appendix VII. We designed our study to identify whether Labor’s policy guidance on eligibility was consistent with legislation and regulations and to collect information on contractors’ practices in enrolling individuals for the program and in placing them in jobs after they leave Job Corps. We reviewed Job Corps legislation as well as Labor’s program regulations and policy guidance on program eligibility, outreach and assessment of individuals for participation in the program, and placement of participants after termination. We also interviewed national and regional Job Corps officials and conducted site visits to 14 outreach, admissions, and placement contractors. We augmented the information we collected during the site visits with data from Labor’s Student Pay, Allotment, and Management Information System (SPAMIS), a database containing nationwide Job Corps data on all Job Corps participants as well as information on the outreach, admissions, and placement contractors for each participant. We analyzed program year 1995 enrollee data, the most recent full program year for which SPAMIS data were available. While we did not verify the accuracy of Labor’s SPAMIS data, we performed internal validity checks to ensure the consistency of the database. We performed our work between October 1996 and July 1997 in accordance with generally accepted government auditing standards. We visited 14 Job Corps outreach and assessment and placement contractors. We selected the sites judgmentally to provide a mixture of contractors that were private contractors, Job Corps centers, and state agencies. We also selected contractors that provided both outreach and assessment services and placement services or that provided only one of these services. In addition, we considered past contractor performance in making our selections. We selected contractors located in 5 of Labor’s 10 regions to provide some regional management diversity and geographic dispersion and to allow us to visit multiple contractors during individual trips. In making our site selections, we identified contractors that had outreach and admissions or placement contracts with Labor during program years 1994 and 1995 and that were still under contract in program year 1996. This provided us with contractors that had multiyear program experience and were currently under contract with Job Corps. In order to select among the larger contractors, we included only contractors who enrolled or were responsible for placing at least 150 participants each year. We then ranked outreach and admissions contractors according to the percentage of program year 1995 enrollees who stayed in the program for more than 30 days and placement contractors according to the percentage of program year 1995 terminees placed in jobs, school, the military, or other training. We then selected contractors from among the top, middle, and bottom third of these rankings. Table I.1 lists the contractors we visited and their characteristics. Table I.1: Outreach, Admissions, and Placement Contractors We Visited North Carolina Department of Human Resources Dynamic Educational Systems, Inc./Hubert H. Humphrey Job Corps CenterDynamic Educational Systems, Inc. Oklahoma City, Okla. Oklahoma Sacramento, Calif. Sacramento, Calif. Did not meet selection criteria (continued) Carson City, Nev. San Francisco, Calif. Women In Del Jen, Inc. We visited 11 outreach and admissions contractors from which varying percentages of program year 1995 enrollees left the program within the first 30 days. As shown in figure I.1, the percentages ranged from about 1 percent for one contractor’s enrollees to nearly 20 percent for another’s. We also selected 12 placement contractors to visit that had varying success in placing Job Corps participants in program year 1995. As shown in figure I.2, placement rates ranged from about 54 percent to about 85 percent. To obtain information on how contractors enroll individuals in Job Corps and place them after their termination, we interviewed contractor personnel using a semistructured interview protocol. We asked outreach and admissions contractors questions related to their practices and procedures in attracting youths to Job Corps and in screening applicants. We also asked about their understanding and implementation of program eligibility criteria as specified by Labor and about their views on what affects program retention. We questioned placement contractors on their procedures in placing terminees in jobs, the military, or other training; the types of services they provided to terminees; and their practices when deciding whether a placement is a job-training match. We asked both groups of contractors about their views on current Labor performance standards related to recruitment and placements and their opinions on improvements needed in the Job Corps program. At three centers (David L. Carrasco, Kittrell, and Sacramento), we also interviewed Job Corps participants (approximately six from each center) to learn about their experiences when they were recruited for Job Corps and to obtain their views about the enrollment process. We interviewed Labor officials at national and selected regional offices to obtain an overview of Job Corps enrollment, placement, and contracting. We also obtained information on Labor’s policy guidance on eligibility and how it relates to the Job Corps legislation; outreach, admissions, and placement contractors’ performance; and the program’s performance management system. In addition, we reviewed Labor’s Policy and Requirements Handbook, which was designed to include all program policy and requirements concerning eligibility criteria and policies and standards related to program enrollment and participant placement. We analyzed Job Corps participant retention data, reasons for termination, and placement information for program year 1995. We used 30-day retention data, part of Labor’s standard for evaluating outreach and admissions contractor performance, as a basis for selecting outreach and admissions contractors to visit. We expanded our analysis of retention beyond the 30-day standard and determined how many terminees left Job Corps within 60 days of enrollment in order to look at retention beyond the realm of outreach and admissions contractor performance. We also used one of Labor’s placement standards—the extent to which terminees are placed in jobs, the military, school, or other training—as a basis for selecting placement contractors to visit. Furthermore, we used the data from our analysis to supplement information obtained in discussions with admissions counselors and placement specialists. In our analysis, we examined the relationship between the characteristics of Job Corps participants and the likelihood of their remaining in the program for at least 60 days. We used the data that were available from Labor’s Student Pay, Allotment, and Management Information System (SPAMIS) on the characteristics of the more than 68,000 participants enrolled in Job Corps during program year 1995. We performed a three-stage analysis resulting in a logistic regression model that used these characteristics to predict the odds of a participant’s remaining in the program for at least 60 days. While the information from our analysis provides some indication of whether participants with specific characteristics will remain in Job Corps for at least 60 days, we do not intend to imply that only individuals with these characteristics should be enrolled in the program or that outreach and assessment efforts should be focused on them. Rather, this information is a source of insight into early program attrition for Labor’s use in Job Corps management. We also recognize that being in the program for at least 60 days indicates only longevity, not necessarily success. For our initial exploration of the data, we selected the participant characteristics from SPAMIS that appeared to be conceptually relevant to the likelihood of remaining in the program for at least 60 days. These included age at enrollment, distance between a participant’s home and the assigned Job Corps center, and educational status. We first used crosstabulations to examine the relationship of these variables to whether the participant remained in the program for 60 days. The chi-square statistics from these analyses showed the variables that seemed to exhibit no relationship to 60-day retention and helped us eliminate certain characteristics and select a set of variables for further analysis. The set of variables we selected is shown in table IV.1. No need for bilingual education Distance from home to center (continued) With these variables, we then performed a bivariate logistic regression to estimate the effects of each individual factor on remaining in Job Corps for at least 60 days. The results from the regression models are stated as odds ratios, which tell us how much more likely participants with certain characteristics are to remain in Job Corps for at least 60 days than participants without those characteristics. We give a chi-square test of significance for each of these odds ratios. To calculate the odds of a specific group remaining in Job Corps for at least 60 days, the percentage remaining and not remaining must be determined. For example, 26,687 participants aged 15-17 enrolled in Job Corps during program year 1995. As shown in table IV.1, 19,148 of these participants remained in the program for at least 60 days, while 7,539 did not. The odds of 15-17-year-old participants remaining in the program for at least 60 days were calculated by dividing the number remaining (19,148) by the number not remaining (7,539). Therefore, the odds for this group’s remaining were 2.54, meaning that 2.54 individuals remained for every 1 who did not. Similar calculations for participants aged 18 to 20 and 21 to 25 yield higher odds of 3.04 and 3.47, respectively. The logistic regression model provides us with odds ratios that tell us how different the odds are for each group and whether the differences are statistically significant. For example, when the 15-17-year-old group is used as a benchmark for comparing the two other groups, the resultant odds ratios are 3.04/2.54 = 1.20 and 3.47/2.54 = 1.37 for participants aged 18 to 20 and 21 to 25, respectively. Thus, the odds of 18-20-year-old participants remaining in Job Corps at least 60 days are 1.20 times the odds of 15-17-year-old participants, and the odds of 21-25-year-old participants remaining are 1.37 times the odds of 15-17-year-old participants. Odds ratios that deviate from 1.0 the most, in either direction, represent the most sizable effects (for example, odds ratios of 0.5 and 2.0 represent effects that are similar in size, since 0.5 indicates that one group is half as likely as the other to remain in the program for at least 60 days, while 2.0 indicates that one group is twice as likely as the other to remain). We performed this type of bivariate analysis for each characteristic we selected. The resulting odds ratios are shown under the “bivariate results” column of table IV.2. Distance from home to center Less than 50 miles vs. 300 miles or more 50-149 miles vs. 300 miles or more 150-299 miles vs. 300 miles or more (continued) Size of participant’s home city or town 250,000 or over vs. under 2,500 * Statistical significance = .05. After performing the bivariate analysis, we used the same set of variables in a multivariate logistic regression analysis, identical to the bivariate analysis except that it provides estimates of the effects of each characteristic on the likelihood of remaining in the program for at least 60 days while holding constant, or controlling for, the effects of the other characteristics. We included all factors (and levels of factors), even if their effects were not statistically significant in the bivariate analysis because, in some cases, effects that are suppressed in bivariate analysis emerge as significant in multivariate analysis. Similarly, effects that were significant in the bivariate analysis may be insignificant in the multivariate analysis. The results of the multivariate logistic regression are shown in column 2 of table IV.2 (“multivariate result”). As this column shows, when we entered all variables into the model, some variables and levels of variables had odds ratios that were not significantly different from the reference category. We dropped these variables or, in cases in which levels of variables were not significantly different from other levels within the same variable, we combined levels. For instance, in the multivariate model, the odds of remaining in Job Corps for at least 60 days for participants having a prior conviction were not significantly different from the odds of remaining for participants not having had a conviction. As shown in table IV.2, the odds ratio of .94 is not statistically significant. Therefore, we dropped this variable from subsequent analysis. Similarly, the odds of remaining for two levels of the variable “distance from home to center” (50-149 miles and 150-300 miles) were not significantly different from the odds of the reference category (over 300 miles). Therefore, we combined these two levels with the reference category to create a two-level variable for subsequent analysis. Thus, we included in the final model only the variables, and levels of variables, that were shown to be significant in the previous multivariate analysis. The results of this final model, as well as statistics related to how well the model performs, are shown in table IV.3. Model performance can be measured by the likelihood ratio method, which evaluates the probability of the observed results, given the parameter estimates. These results are shown under the –2 Log Likelihood (–2LL) entries in the note to table IV.3. As shown, the model containing the predictor variables shows an improved (smaller) –2LL compared with the model containing only the constant (that is, the model that assumes no differential effects resulting from individual variables). The model chi-square, which tests that the coefficients for all the terms in the model (except the constant) are 0 (that is, the null hypothesis), was significant at the .0000 level. Distance from home to center Less than 50 miles vs. 50 miles or more Size of participant’s home city or town Over 250,000 vs. under 250,000 * Statistical significance = .05. The results of our multivariate analysis revealed that older participants have greater odds of remaining in the program 60 days or more. When compared to 15-17-year-old participants, those aged 18 to 20 and 21 to 25 had odds of remaining that were 15-percent and 27-percent greater, respectively. In addition, we found that participants with 12 or more years of school had about 80-percent greater odds of remaining in Job Corps for at least 60 days than participants with 8 years or less of school. (See table IV.3.) We also found a relationship between the need for bilingual education and the likelihood of remaining in the program for at least 60 days. Of the variables we examined, the need for bilingual education yielded the highest odds ratio. Spanish-speaking participants needing bilingual training had odds of remaining that were almost twice the odds of those not needing bilingual education. Other non-English-speaking participants needing bilingual assistance had odds that were more than 3 times the odds of those not needing bilingual education. Our attempt to construct a model for predicting the characteristics of participants who are more likely to remain in the program for at least 60 days was limited by the variables available to us in Labor’s SPAMIS extracts. Most of these variables were demographic characteristics. We were unable to include in the analysis measures of such things as student ability, attitude, and motivation, as well as other characteristics that could potentially affect the likelihood of participants remaining in the program for at least 60 days. Additionally, the factors that proved to be the most useful predictors of remaining in the program for at least 60 days were characteristics of small subsets of participants. For example, there is evidence that participants in need of bilingual education are more likely to remain, but this group made up less than 3 percent of the Job Corps population. Similarly, participants who had completed 12 years or more of school had odds of remaining that were more than 80-percent greater than those of participants who completed 8 or fewer grades, but almost two-thirds of the participants were in neither of these groups. Consequently, while the model is very useful in predicting whether participants with specific characteristics will remain in Job Corps for at least 60 days, the model’s ability to predict 60-day retention for the program’s full population is limited because we found no large subgroups with great differences. Finally, in this analysis, we examined only main effects for the variables we investigated. An examination of the interactions among the variables might produce useful information and improve the predictive ability of the model. In addition to the contacts named above, the following persons made important contributions to this report: Thomas N. Medvetz, Wayne Dow, Deborah Edwards, Jeremiah Donoghue, Robert Crystal, and Sylvia Shanks. Job Corps: Where Participants Are Recruited, Trained, and Placed in Jobs (GAO/HEHS-96-140, July 17, 1996). Employment Training: Successful Projects Share Common Strategy (GAO/HEHS-96-108, May 7, 1996). Job Corps: Comparison of Federal Program With State Youth Training Initiatives (GAO/HEHS-96-92, Mar. 28, 1996). Job Corps: High Costs and Mixed Results Raise Questions About Program’s Effectiveness (GAO/HEHS-95-180, June 30, 1995). The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO provided information on the Job Corps program's recruitment and placement contractors, focusing on: (1) whether Job Corps' policy guidance regarding eligibility criteria is consistent with the legislation and regulations; (2) how the use of recruiting contractors could be improved to increase participant retention in the program; and (3) how the use of placement contractors could be improved to enhance positive outcomes. GAO noted that: (1) Job Corps' policy guidance for 2 of the 11 eligibility criteria was ambiguous and incomplete, which has led to an eligibility determination process that fails to follow the requirements of the law and program regulations; (2) in GAO's visits to several outreach and admissions contractors, GAO found that those with higher retention rates follow procedures aimed at identifying applicants with the commitment and motivation to remain in and benefit from the program; (3) in GAO's analysis of participant characteristics, GAO identified certain characteristics significantly related to the likelihood of remaining in the program for at least 60 days; (4) the Department of Labor (DOL) could use some of these characteristics to design outreach efforts or to establish priorities among eligible applicants; (5) although Job Corps is a performance-driven program and DOL uses performance measures to make decisions on placement contractor renewal, two of the measures DOL used were not meaningful and, thus, DOL did not have the information it needed to accurately assess the performance of placement contractors; (6) placement measures held contractors responsible for placing individuals who may have received little or no benefit from the program or who demonstrated behavior that normally would be unacceptable to most employers; (7) the job-training match measure did not accurately portray the extent to which participants obtained jobs related to their vocational training because of the wide latitude placement contractors have in deciding whether a job is related to the training received and the creativity contractors used in recording the occupational titles of the jobs obtained; (8) one aspect of placement contractors' operations associated with better performance was having staff solely responsible for placing Job Corps participants; (9) seven contractors visited by GAO with high placement rates had staff solely responsible for placing Job Corps participants; (10) in contrast, four of the five contractors having lower placement rates had the same staff responsible for performing outreach and assessment as well as placement; and (11) as a result of its concern about performance, DOL has not renewed 12 of the 18 contracts with state agencies.