Youth Opportunity Grants: Lessons Can Be Learned from Program, but Labor Needs to Make Data Available
Released: 2005-12-09T00:00:00

Letter

Background

The Workforce Investment Act of 1998 authorized Youth Opportunity Grants (YO), a $1 billion, 5-year program aimed at increasing the educational attainment and long-term employment of youth residing in high poverty areas. The program was designed to assist at-risk, hard-to- serve youth and the communities in which they live by concentrating resources into a targeted geographic area. YO expanded on earlier demonstration programs that were based on a similar design, the most recent of which were known as Kulick grants. The Department of Labor (the Department) announced the 36 YO grantees—24 urban, 6 rural, and 6 Native American—in February 2000 and set a goal for the grantees to have their programs operational by September of that year. Although the Department originally planned to continue to add grantees, funding for the program was eliminated in the budget for fiscal year 2004. Figure 1 shows the 36 grantees by location.

The Department of Labor’s stated goals for the YO program were to increase educational attainment and promote long-term employment of youth in the target areas, as well as to improve the youth service delivery systems in these communities. Specifically, grantees were expected to effect increases in the rates of high school completion, college enrollment, and employment, both for youth participating in the program and, by extension, the overall target area. Grantees were also expected to facilitate the delivery of services by partnering with other institutions in the community, especially local schools and colleges, the juvenile justice system, and private employers. The program was designed to allow grantees to enhance the local infrastructure of youth services by filling in critical service gaps, coordinating existing services, and, in the case of some rural and tribal areas, developing an infrastructure where services were limited.

Many of the YO program components were based on practices that experts have identified as effective for working with at-risk youth, and the design of the program supported flexibility and invention at the local level. Based on a youth development framework that emphasizes a comprehensive approach to meeting a young person’s needs, the program required YO grantees to offer a full range of education, employment, and leadership development services as well as provide other supports to the youth. Advocates and experts have also stressed that youth need a place to gather where they feel comfortable and a sense of belonging. In this vein, YO grantees were required to have at least one youth center in the target community that would provide a focal point for services and activities as well as a safe place for the youth to go. In addition, grantees were expected to maintain a core staff of trained youth workers and to provide follow up services to participants for at least 2 years after they completed participation in program activities, which is in keeping with the identified effective practice of providing youth access to a caring, trusted adult for an extended period of time. Additionally, experts have suggested youth benefit from the opportunity to continue a relationship with a program for as long as they need. Similarly, the program had flexible enrollment rules that allowed participants to remain enrolled even through periods of inactivity. However, the program design allowed grantees latitude in deciding many of the particulars of the program. For example, grantees could choose how and by whom services would be delivered, the number and location of the YO centers, and the institutions with which they partnered. Figure 2 displays the key components of Youth Opportunity Grant program.

Like some prior demonstration programs that the Department of Labor has administered, eligibility for YO was not based on income, but rather on geographic residence. Youth between the ages of 14 and 21 living in the target area were eligible for the program, regardless of income. In most cases, the target areas were federally designated empowerment zones or enterprise communities, which are, by definition, areas with high rates of poverty.

The Department of Labor provided extensive, ongoing technical assistance to grantees. The Department assigned grantees coaches with considerable experience working in youth programs who helped grantees with a range of issues, such as developing services and building partnerships. The Department also collaborated with a private foundation to establish a training institute that provided courses in youth development to more than 2,000 youth workers. In addition, the Department sponsored opportunities for grantees to share best practices and strategies in regular directors’ meetings and conference calls and through peer-to-peer training sessions.

The Department laid the foundation for collecting performance data on the program by setting up a management information system (MIS). The Department collected performance data on the YO program in a centralized, electronic system. Grantees submitted information to fulfill requirements of the Workforce Investment Act, which mandated that grantees report on seven performance measures related to employment, retention, earnings, attainment of skills, and attainment of credentials. In addition to these, grantees reported on interim measures developed by the Department to gauge the progress of participants as they moved through the program. These measures were designed to document youth participation in activities and other intermediate milestones.

The Department had funded several prior programs premised on a geographic and community concept similar to that of the YO, although on a much smaller scale, but little information is available on the impact of these demonstration programs. Although evaluations were conducted of these earlier programs, the impact studies were either incomplete or not released. The first of these demonstrations, Youth Opportunities Unlimited, was evaluated and the results published, but the study did not include a systematic analysis of impact based on comparison groups. The funding for the second demonstration, Youth Fair Chance, was cut 2 years into the program, and although the evaluation study was published, it was based on only 2 of the planned 5 years of the program. As a result, the authors of this study advised that the findings should be interpreted cautiously. The most recent of the demonstration programs was known as the Kulick grants. The Department has prepared an evaluation of the Kulick grants, but has not yet released the results publicly.

For the Youth Opportunity Grants, the Department set up an extensive effort in 2000 to collect and analyze information to assess the program’s impact. The Department contracted for a $24 million evaluation study that included plans to estimate the impact of the program by comparing key characteristics in the YO communities and comparable areas that did not receive YO grants. It was to begin in 2000, and the final report was originally scheduled to be completed in July 2005. The study was designed to differentiate between those observed changes in participants and the communities that resulted from the program and those that would have occurred even without the program’s intervention.

Grantees Carried Out the Program Using a Variety of Approaches

Grantees adapted key elements of the program’s design to their particular circumstances and used a variety of approaches in providing services and conducting outreach. Grantees set up centers that varied widely in terms of number and other characteristics where youth could receive services and participate in activities. The centers housed a core staff who provided case management and helped plan individualized services for youth. Staff numbers and duties differed between grantees. Most grantees used the management information system (MIS) provided by the Department of Labor to submit program data to the Department, but some used their own systems. Most grantees used a combination of approaches to provide youth services, including collaborating with other organizations and, when they deemed it necessary, developing services of their own. To reach hard- to-serve target population, grantees used a variety of recruiting techniques, ranging from the conventional to the innovative.

Grantees Adapted Program’s Design to Their Particular Circumstances

The centers set up under the terms of the grant varied widely in number and character. Because the target communities varied in size, some grantees had 1 center while other, more geographically dispersed grantees had as many as 40. The centers were intended to serve as the hub of the local programs, and the types of activities offered in them differed considerably. One center we visited in rural Louisiana had a variety of recreational facilities, such as a basketball court and recreation room, in addition to a classroom, computer lab, and staff offices. Youth participating in this program told us nothing like these facilities had existed for them before the YO program. On the other hand, a center we visited in Houston resembled a traditional career center, with computer kiosks set up near staff offices where the youth could undertake job searches and skills assessments. Houston officials said they had formerly offered some recreational activities on site, but found them to be duplicative of other services available in the neighborhood. Some grantees provided still other facilities at their centers. For example, Baltimore had an on-site health clinic and music recording studio. Some grantees also had centers located within schools. The District of Columbia had staff sited in a local high school, where they offered training and support for youth. This in-school center was located in a large classroom, with a staff office off to one side and furnished with a computer lab with equipment purchased by the grantee. Figure 3 depicts photos of two youth opportunity grant centers we visited.

The centers housed staff who varied substantially in number and makeup. Houston divided staff duties between as many as 115 people working in 4 centers. In this program, “personal service representatives” provided case management and identified youths’ goals and training needs, and “employment counselors” assisted youth with job searches and conducted follow-up services to youth who had been placed in employment. Other grantees, however, had sites with only one person performing all of the major staff duties. At the California Indian Manpower Consortium (CIMC) Manchester Point Arena site, one youth worker handled recruitment, case management, and job placement. Because of the remoteness of the site, this youth worker also frequently drove a van, weekly transporting youth to services as far as 4 hours away.

Most grantees used the management information system (MIS) provided by the Department of Labor to record program data, but some used their own systems and then submitted the data to the Department. Those who did not use the Department MIS used a variety of systems. For example, Milwaukee developed its own system with the aim of using it in conjunction with other youth programs offered by its local Workforce Investment Board. Similarly, Hartford developed customized software that enabled it to share information with other provider agencies and the schools in the community.

Grantees Used Collaboration and Invention to Varying Degrees in Delivering Services

Grantees used a combination of approaches to establish a network of educational, occupational, and other services for youth, but varied the extent to which they relied on other providers. Grantees availed themselves of existing services, either through formal arrangements or by referring participants to other organizations in the community. Kansas City arranged for youth to attend classes at other local agencies to help prepare for the GED exam. Similarly, Baltimore referred young women who needed interview clothing to a local charity organization. Most grantees also collaborated with other institutions to provide some services. San Antonio partnered with a local community college to establish academies in which youth could spend half a day on their home campuses and half a day at the community college, allowing them to earn a certificate or associate degree in areas such as aviation or biotechnology. Houston purchased credit retrieval software for local high schools that students could use to complete their academic requirements for graduation. In addition, grantees originated programs to fill perceived gaps in services. Memphis established its own charter school with an emphasis on the arts in order to provide alternative education opportunities for the youth, one of the emphasized youth activities in the program. To provide leadership development opportunities, one of the CIMC sites sponsored a cemetery care project. In this project, participants worked with the tribal elders to make a map of the local cemetery with the names of the deceased because cemeteries on this reservation did not have headstones. Table 1 shows youth activities and examples of how grantees implemented the activities.

Grantees also used an array of strategies to help youth find long-term educational or occupational opportunities and to follow up with participants after they exited the program. Some grantees had staff specifically devoted to finding appropriate employment opportunities and connecting youth with these opportunities. For example, Milwaukee created the position of job developer, who identified potential employers and spoke to them directly about the youth in the program. Grantees also used a number of methods for helping youth enroll in post-secondary institutions, such as conducting college tours, assisting with financial aid forms, and placing support staff on college campuses. The statutory provision authorizing the YO program required grantees to provide follow- up services for 2 years after the youth completed participation in program activities. Grantees recorded on a quarterly basis the employment and education status of the youth who had completed participation in the program, for example, if they were in school or working at the time of contact. Some grantees contacted youth more frequently. Staff in Baltimore told us they varied the frequency depending on the degree to which they regarded the young person to be at some risk. One might be contacted monthly, while another would be checked on nearly every day.

Grantees Actively Recruited Youth, Including the Hard to Serve, and Used Incentives to Draw and Maintain Involvement

Grantees recruited youth, including traditionally difficult to serve populations, by creating connections with other youth-serving agencies. Milwaukee operated a juvenile justice project to help youth coming out of the corrections system to reintegrate into the community. Personnel at a local correctional facility worked with program staff to identify those due for release who were expecting to live within the geographic target area of the program. Program staff traveled to the facility to conduct an orientation session and then worked with the youth, corrections officials, and parents to develop a reintegration plan. The services offered to them were similar to those offered to all of the YO participants.

Grantees also used a variety of innovative methods to recruit participants. Since many of the youth were disconnected from both school and work, as one grantee told us, they could be not only hard to serve, but hard to find. Some grantees went beyond conventional outreach activities of mailers and radio advertisements and conducted community walking campaigns using staff to saturate shopping malls and other areas where youth congregate. Others used youth to lead recruitment. In some cases, a grantee used employment as an inducement to link youth to other activities the program offered. For example, Louisiana’s work program allowed participants to work more hours in subsidized employment the further they advanced toward their educational goals. We were told many of the youth lived in unstable economic conditions, and these needs had to be addressed before the youth would focus on other areas, such as education. In other cases, grantees used cash or non-monetary rewards to encourage participation. In Houston, youth could attend special events, such as a basketball game, if they participated in the program a minimum number of hours.

Grantees Used Program Design to Address Major Challenges, but Found Start-up Time to be Short

Grantees were able to address a variety of challenges in setting up the program and delivering services to youth using local discretion, flexible enrollment rules, and other aspects of the program’s design, but they and others said a longer start-up period would have been beneficial. Most grantees found it difficult to establish centers and retain participants in their programs. They felt that these challenges were compounded by expectations for a quick start-up, and subsequently they and Department officials felt more planning time would have been beneficial. Grantees also had difficulty establishing an information-reporting system, but once in place, found it was helpful for program management purposes. Conditions in the communities such as violence and lack of jobs presented a challenge to most grantees, but they took advantage of the local discretion built into the program to develop strategies to address them. Grantees also cited as an obstacle the vast service needs of many of the youth the program was intended to serve, but case management, individualized services, and flexible enrollment rules were useful in dealing with them.

Grantees Felt Challenged to Set Up Program in Time Allotted, and More Planning Time Would Have Been Beneficial

The majority of grantees reported that finding or renovating centers was a challenge. Many grantees did not have suitable sites in their communities for use as centers, and some put considerable effort into renovating existing structures. In the extreme, the grantee in Alaska renovated 40 centers in remote communities not accessible by road and shipped equipment and material by air. In a few cases, grantees did not have a permanent building until after the first year of the program. In Boston, the program was housed in temporary facilities for a period of time because a permanent center could not be completed for the program’s opening.

Another challenge identified by most grantees was retaining participants, which grantees linked in part to expectations from the Department to start to enroll youth quickly. The Department set a goal to recruit 3,000 participants nationwide within the first few months of the program, or a little less than 100 youth per grantee. Some grantees said they felt rushed to meet recruitment goals and told us they recruited many youth who were not committed to the program. In addition, several directors mentioned that they felt they were asked to serve participants before their programming was fully developed, a situation that as one expert commented, was akin to asking grantees to “fly the plane before they were finished building it.” We were told that, in these situations, some youth became disenchanted with the program and left because of the limited offerings the center presented early on.

Grantees, agency officials, and experts said a longer planning period would have been beneficial. Grantees had an 7 month window between the announcement of the grant recipients and the Department’s target date for having the programs operational. For example, two grantees with whom we spoke said about three additional months would have been necessary for them to meet the goals set out by the Department. Agency officials also told us that grantees were pushed to start-up too quickly and could have benefited from more time to plan. Agency officials said it would have been appropriate to give grantees more time to meet the Department’s initial goals.

Grantees Had Difficulties Establishing an Information Reporting System, but Found It Useful Once in Place

Most grantees reported that establishing an information reporting system was challenging, and grantees told us it would have been useful for the Department to have had a workable system in place at the outset of the program. The management information system (MIS) initially provided by the Department was fraught with problems. Department officials told us it took about 18 months to iron out the kinks. In addition, the Office of Inspector General identified inconsistencies in the way grantees were recording data early on. Although the Department eventually developed an improved system, grantees said it would have been better had the MIS been in place from the beginning. Not having a system in place created additional burden for the grantees. For example, Louisiana developed its own system to collect information until an updated system was available and then manually transferred 11 months of data into the new MIS.

Once in place, grantees additionally used the information system for their own program management purposes such as to improve performance and reinforce accountability. The Hartford grantee was able to integrate its system with the school district, allowing case managers to track attendance and grades to allow better monitoring of participant achievements. The Hartford system also enabled program management to see which case managers were most successful at engaging youth in the activities required for the completion of their goals. Regarding accountability, Portland used its information system to monitor the performance of its contractors on a weekly basis.

Grantees Used Their Discretion to Address Community-Based Challenges

Some of the major challenges also identified by grantees were problems external to the program, yet affected their ability to deliver services. Twenty-eight of the grantees reported on the survey that a lack of jobs in the community was a challenge. In some areas, jobs were on the decline because of shifts in the local economy or relocation of major employers. For example, the grantee in San Francisco related that due to the dot-com bust, youth in the program were competing with a skilled workforce willing to fill entry level positions that would otherwise be available to youth. In other areas, especially rural ones, there were few employers, let alone large ones. In addition, 29 of 36 grantees reported on the survey that risk factors such as violence, drugs, and gangs were challenges in implementing the program. In some cases, these factors made it difficult for participants to receive services. In one urban community, we were told there were safety concerns with youth participating in evening activities sponsored by the YO because they would have to return home after dark.

Local discretion built into the program design helped grantees respond to external challenges. Within the structure of the program, grantees were allowed some amount of latitude to develop responses to circumstances in their communities, such as a scarcity of jobs. For example, the California Indian Manpower Consortium created opportunities for youth to have work experiences, such as subsidizing summer jobs with local business including a campground and senior center or working with the tribe to place youth in clerical positions. Similarly, grantees were able to develop services to address specific risk factors in their communities. Imperial County offered a curriculum series to help address interpersonal violence among adolescents, an issue they had identified as particularly problematic in their area. The series was designed to reduce impulsive and aggressive behavior through empathy training, interpersonal problem solving, behavior skill training, and anger management. In Milwaukee, the grantee addressed safety concerns by renting vans to transport the youth to nighttime events.

Service Needs of Participants Presented Challenges, but Tailored Services, Individualized Planning, and Flexible Enrollment Rules Were Useful in Addressing Them

Grantees identified as a major challenge the obstacles faced by their clients, such as homelessness, lack of family support, mental health problems, and low levels of academic attainment. Staff in Baltimore told us homelessness was a frequent issue faced by youth in their area and finding a place to sleep can preoccupy the youth and disrupt the learning process. YO staff and others said that participants may lack support in other areas of their lives, such as from their families. For example, one grantee told us of a participant who was awarded a full scholarship to college, but the parents would not sign the paperwork to receive the money. Some youth also faced mental health issues, which prevented them from moving forward in their lives. All of these factors, we were told, could mean a slow start for youth who enrolled in the program. The director of an alternative high school in Milwaukee told us that because of the disorder in their lives, many of these youth may take a year just to become comfortable in the program before they can even begin making any forward progress.

Grantees found aspects of the program design such as case management, individualized services, and flexible enrollment rules useful in addressing the service challenges of participants. For example, grantees used assessments to help determine the academic needs of clients and provide each client with the appropriate individualized services. Philadelphia designed a program in which each participant had a personalized remediation plan based on the results of an evaluation test and interview with an education coach. This plan was intended to build on clients’ strengths and accounted for their particular learning styles. After the initial assessment, participants were reevaluated at regular intervals to monitor their academic progress and goal attainment. The curriculum was designed so that youth could increase at least one grade level after 90 hours of instruction. In addition, both youth and community members told us that YO staff were key in helping youth stay motivated and guiding them through difficult situations. Grantees also found the enrollment rules useful in working toward program goals. The program’s enrollment policy allowed most participants to continue to receive services even if they experienced periods of inactivity. Staff in Houston told us that the policy was a benefit to the youth because, unlike other employment programs, YO did not end their relationship with youth who had not been participating for a while.

Grantees and Others Reported That Participants and Their Communities Made Progress, but the Program’s Impact Is Still under Study

Grantees and others reported that the participants and their communities made advancements in education and employment; however, a formal assessment of the program’s impact is still under study. Data reported by the grantees showed that a number of youth advanced their education while they were participating in the program, such as completing high school. Grantees also reported data showing a portion of the participants entered unsubsidized employment after enrolling in the program. Similarly, grantees and others believe their communities made advancements toward the Department’s stated education and employment goals of the program. In addition, the majority of grantees reported that they made improvements in the youth service delivery systems in their communities. The Department funded an evaluation to assess the impact of the program, which was designed to shed light on the extent to which observed changes can be attributed to the program. However, the study has not yet been completed.

Grantees Reported Data Showing Some Participants Advanced Their Education and Found Employment

Data reported by the grantees showed that a number of youth advanced their education while they were participating in the program. Two of the Department’s primary goals of program were to increase the rates of high school completion and college enrollment for the youth. Data that grantees entered into the management information system (MIS) showed that of the approximately 91,000 youth who signed up for in the program nationwide, about 18,700 either completed high school or attained a GED after enrolling in the program. In addition, about 11,700 youth entered college, 37 percent of whom were reported to be out of school when they initially enrolled in the program. Youth with whom we spoke credited the program with giving them a second chance to increase their education and to better their employment opportunities. One Louisiana youth’s comments typified what we heard elsewhere from participants during our site visits, “YO is the best thing that has happened to me. YO has given me a job and put a little money in my pocket.” He added, “YO has also been instrumental in keeping me out of jail. Above all, YO has helped me realize that education is important.” Figure 4 summarizes enrollment, participation, education, and employment data for youth in the program, as of June 2005.

Grantees reported data showing that a portion of the youth entered unsubsidized employment after completing YO activities. Data from the MIS show that about 17,300 youth were placed in employment that was not subsidized by the program, 62 percent of whom were out of school when they initially enrolled in the program. Grantees and others told us the job readiness training that some youth received as part of the program was particularly important in helping them get jobs. Employers we spoke with told us they knew the youth that were sent to them by YO would be well trained and ready to work. A local human resources manager for a national chain of home improvement stores told us that he thought YO participants were better prepared for employment than other residents of the area, adding that they show up with better job skills and “soft” skills.

Grantees and Others Believe Their Communities Moved toward Program Goals

Grantees and others expressed the view that their communities had made advancements in concert with the Department’s education and employment goals for the program. Almost all of the grantees reported that high school completion, college enrollment, and youth employment rates improved in their communities as a result of the program. Several grantees also asserted that affecting the youth led to changes in the community. In addition, community members with whom we spoke said the program helped their communities make progress toward greater education and employment. For example, a tribal leader in California told us that YO motivated the youth in his tribe to stay in school. Similarly, one expert pointed to some of the rural grantees who had many youth enroll in college, which she said was especially important because of a lack of jobs in these areas.

The majority of grantees reported making improvements in the youth service delivery systems in their communities. Some of the described changes were to the service infrastructure in these communities. For example, San Diego pointed out that it had created a multiservice youth center in a neighborhood where none had existed before. Grantees cited other improvements related to the mode of service delivery. Tucson reported that prior to YO, it had funded three stand-alone youth programs, but now it has a one-stop system with multiple entry points for delivering youth services. Other grantees noted that their efforts to foster better communication and collaboration among service providers had benefited the youth. As the Portland grantee commented, by partnering with their local employment department and community college system, they were able to leverage staff to provide intensive job search and college preparation services to youth. In other cases, local leaders credited the YO as a catalyst for change in their community. For example, a school superintendent in California told us that YO staff had helped to forge an agreement between the school district and five Indian tribes in the area to develop an education curriculum that would be more sensitive to the tribes’ cultures.

Department of Labor’s Study of Program Impact Has Not Been Completed

While the Department of Labor funded an evaluation designed to assess the impact of the program, it has not yet been completed. This was planned as a 5-year, $24 million evaluation, which began in 2000. As part of the evaluation, the Department conducted baseline and follow-up surveys of the target areas, and gathered extensive descriptions of the 36 programs and communities. The evaluation also included plans for an impact study that was designed to compare YO participants and communities with other, similar youth and communities that did not participate in YO. This type of comparative analysis would be necessary to determine if the events reported by grantees and others would have occurred in the absence of the program. However, the evaluation is not finished. Moreover, agency officials are unsure if the impact study that was to be part of the evaluation will be completed. Due to departmental allocations being lower than expected, the Department spent $1.9 million less than the full amount of the original contract on the evaluation. Agency officials told us that the impact study was likely to be scaled back because of the lower allocation. The evaluation was originally scheduled to be completed in July 2005. However, agency officials told us they do not expect the study to be finished until June 2006.

Conclusions

The Youth Opportunity Grant program was designed to help at-risk youth and their communities by concentrating resources geographically and by incorporating components that experts have suggested are effective in assisting this population. To continue to improve the ability to serve these youth, researchers and practitioners must be able to learn from the most promising and innovative approaches in serving these youth, including those used in the Youth Opportunity Grant program. The Department of Labor has begun an evaluation of the program that includes many pieces of a potentially useful study. Although informative, these pieces will not by themselves answer the question of impact, in other words, whether the described events would have occurred in the absence of the program. In order to understand what effect, if any, the program had on these observed events, it is necessary to have a systematic comparison with other, similar communities that did not receive grants. The Department planned, but has not yet completed, such an analysis as part of the evaluation. The Department has an opportunity to contribute to the research on programs for serving at-risk youth, if the evaluation study is completed and the results made public. However, the Department has not taken full advantage of past opportunities to release information on other programs based on a similar model, such as the Kulick grants. Given the $1 billion investment in this program—including almost $24 million for an evaluation effort—and the need for rigorous data on these types of programs, the study should be completed and the results should be made available. Unless the Department completes the evaluation of the Youth Opportunity Grant program and releases the results, researchers and practitioners will not be able to fully realize the potential to learn from the program.

Recommendation for Executive Action

To continue to improve efforts to serve at-risk youth and in order that researchers can evaluate the quality of information and determine possible impact of the program, we recommend that the Secretary of Labor take the actions necessary to complete the impact analysis of the Youth Opportunity Grant program and release the data and all related research reports from the program’s evaluation.

Agency Comments

We provided a draft of this report to the Department of Labor for its review and comment. In its response, the Department agreed with our conclusions and recommendation and indicated that it intends to complete the impact analysis and publish all related reports from the Youth Opportunity Grant program evaluation. A copy of the Department’s response is in appendix II. The Department also provided us with technical comments, which we incorporated into the report where appropriate.

We will send copies of this report to the Secretary of Labor, relevant congressional committees, and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov.

Please contact me at (415) 904-2272 if you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

To accomplish our research objectives, we surveyed Youth Opportunity Grant Program directors on the implementation of the Youth Opportunity Grants. To augment information from our survey, we conducted five site visits of programs. We chose sites for our visits to represent proportionately the three grantee types (urban, rural, and Native American) as well as their geographic distribution. The grantees we visited were the District of Columbia, Houston, Milwaukee, rural Louisiana, and the California Indian Manpower Consortium. During our visits we met with program administrators, staff, and participants to learn their perspectives on program implementation, challenges, outcomes, impact, and lessons learned. Also, we interviewed local workforce board members, school officials, community based service providers, private employers, and local government officials to discuss their perspectives on the impact and other aspects of the program. In addition, we toured Youth Opportunity Centers, in-school facilities, and observed program activities. We briefly visited an additional two grantees— Baltimore and Philadelphia—in the course of pretesting the survey, and while at their sites, toured centers and spoke with program administrators, staff, and participants. We gathered additional information by reviewing agency documents including site assessments and grant applications. We also interviewed agency officials and other relevant experts, including researchers, representatives of advocacy organizations, YO coaches, and former Department of Labor officials. We performed our work from September 2004 to November 2005 in accordance with generally accepted government auditing standards.

We also investigated several possible sources of external data to quantitatively measure outcomes and impact, but determined none of them were feasible for our purposes. The data sets we reviewed were: Current Population Survey, Common Core of Data, National Longitudinal Survey of Youth, High School and Beyond, and American Community Survey.

Survey of Youth Opportunity Grant Program Directors

To learn about the implementation of the Youth Opportunity Grants, we conducted a Web-based survey of all Youth Opportunity Grant Program Directors. We asked directors about the usefulness of program components, challenges they faced implementing the program and strategies used to deal with them, and their opinion on the program’s impact on participants and communities. Additionally, we asked directors about the size and structure of their programs, the manner in which they delivered services, and the types of organizations with which their programs partnered. The survey also included a series of questions about how programs maintained information to help us determine the reliability of data in the Department of Labor’s management information system. We pretested the survey with several program directors and modified the survey to take their comments into account. All 36 program directors completed the survey, for a response rate of 100 percent. We administered the survey between March 17 and May 31, 2005. To view selected results of the survey, go to GAO-06-56SP.

Department of Labor Management Information System (MIS)

We used electronic data collected on the program by the Department of Labor in a management information system (MIS) to describe the number of youth in the program who achieved the Department’s goals for the program, such as high school completion, college enrollment, and long- term education and employment placements. We have determined the MIS data are sufficiently reliable for the purposes of this study through discussions with officials at the Department of Labor, in-depth interviews with MIS staff during site visits, and responses to a comprehensive array of survey items in which each grantee described their procedures for editing and auditing the data they entered into the MIS. We analyzed the MIS data using guidance provided to us by the Department.

Appendix II: Comments from the Department of Labor

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

David Lehrer, Assistant Director, and Anne Welch, Analyst-in-Charge, managed this assignment and made significant contributions to all aspects of the work. Dan Concepcion and Eleanor Johnson also made significant contributions to this report. In addition, Kevin Jackson assisted in all aspects of the survey of Program Directors and in reviewing external data sources. Jerry Sandau and Cathy Hurley assisted in the analysis of the MIS data and assessment of the MIS data reliability. Jessica Botsford provided legal support, and Susan Bernstein provided writing assistance.

Related GAO Products

Workforce Investment Act: Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services. GAO-04-308. Washington, D.C.: February 23, 2004.

Workforce Investment Act: One-Stop Centers Implemented Strategies to Strengthen Services and Partnerships, but More Research and Information Sharing Is Needed. GAO-03-725. Washington, D.C.: June 18, 2003 Workforce Investment Act: Youth Provisions Promote New Service Strategies, but Additional Guidance Would Enhance Program Development. GAO-02-413. Washington, D.C.: April 5, 2002.