According to a 1995 effectiveness study of the Big Brothers Big Sisters Program, children who participated in this mentoring program achieved higher grades in school, skipped school less frequently, developed closer relationships with parents and peers, and were less likely to initiate the use of drugs and alcohol than were similar children who were not enrolled in the program. Mentoring is often defined as a sustained relationship between a youth and an older person, typically an adult, in which the adult provides the younger person with support, guidance, and assistance. Mentoring is based on the premise that if young people have access to caring, concerned adults, they will be more likely to become successful adults themselves. Historically, mentoring has meant that one volunteer commits to mentoring one child at a time. More recently, mentoring has moved beyond this traditional relationship to encompass other formats, including group and e-mail mentoring. Mentoring programs are established in many communities. Programs like the Big Brothers Big Sisters Program, a national program operating in every state, have a long history of mentoring neighborhood youth. In community-based programs, youth are often referred for mentoring by family members. Potential mentors often submit to extensive background checks and trained mentors are often allowed to engage in unsupervised activities with the youth. School-based mentoring, as its name implies, takes place on school grounds. Given their location in the school, program staff in school-based mentoring programs can easily meet with teachers. Often teachers refer youth for mentoring whom they believe could benefit from additional attention and guidance. In school-based mentoring programs, volunteers typically meet with the youth during or after school and their interactions are typically supervised. Mentors and youth can spend time on schoolwork, but also engage in other activities, including playing sports, attending a concert, reading, eating lunch together, or just “hanging out together.” To improve the outcomes of our nation’s school age children, Congress passed NCLBA. Among other things, this act authorized 3-year grants for student mentoring programs. NCLBA required that selected programs serve children with the greatest need, that is, children most at risk of failing school, dropping out of school, or being involved in criminal or delinquent activity, as well as those lacking strong positive role models. NCLBA also authorized grants to entities to achieve one or more goals for participating children, including improved academic achievement and reduced delinquent behavior and involvement in gangs. (See table 1.) A number of types of organizations are eligible to receive funding under the program, including local educational agencies, nonprofit, community- based organizations, and partnerships between a local educational agency and a nonprofit, community-based organization. NCLBA requires that applicants demonstrate that they meet a number of criteria, which Education in turn required be detailed in their grant applications. Specifically, Education required applicants to demonstrate that mentors would receive training and support and would be screened using appropriate references and background checks. The agency also required applicants to meet criteria that are not specifically outlined in NCLBA. For example, the agency required that applicants outline how they intended to achieve performance goals, such as improved academic achievement among participating children, or reduced incidences of involvement in gangs, illegal drugs, and alcohol. Grant recipients can use the funding for activities to establish or implement a mentoring program. For example, grants may use funds to hire mentor coordinators and support staff; recruit, screen and train mentors; and disseminate recruiting materials. In fiscal year 2002, Education awarded competitive grants to 122 grantees from a pool of nearly 1,300 applicants. Education funded at least one grantee in every state, with grant amounts ranging from about $39,000 to nearly $500,000 in both fiscal years 2002 and 2003. (See app. II for a list of grantees by state.) Funding for these grantees ends in fiscal year 2004; funding for the mentoring program over the 3 years will total about $50 million. Congress has increased funding for fiscal year 2004. According to Education, these funds will be used to fund an additional 200 grantees and the last year of the current wave of mentoring grantees. Education has a number of responsibilities regarding administration and oversight of the mentoring program. The agency oversees program implementation, provides technical assistance, and disseminates information on best practices. With respect to monitoring, Education, like other federal agencies, is required as part of its monitoring responsibilities to review grantees’ single audit reports if they contain findings. The Single Audit Act requires state and local governments and nonprofit organizations that expend $300,000 or more in federal awards in a fiscal year to have either a single audit or program-specific audit conducted for that year. Audit findings from such reports can include problems such as internal control weaknesses; material noncompliance with the provisions of laws, regulations, or grant agreements; and fraud affecting a federal award. Education receives a copy of the audit report if it contains findings relevant to an Education program. According to the literature we reviewed, prior to implementation, successful mentoring programs make key decisions about which youth they will serve and expected outcomes, how they will recruit mentors, and how the program will be funded; put in place management structures, such as screening, training, and recruitment policies and procedures to ensure that the program is well-managed on a daily basis; market their programs and pursue strategies to ensure long-term program viability; and evaluate program outcomes and disseminate outcome information to key stakeholders to further garner and sustain support for their programs. (See fig. 1.) According to the literature that we reviewed, successful student mentoring programs engage in considerable planning prior to launching their efforts. Such planning enables them to assess the need for the services they plan to offer and to determine whether their organizations have the assets they need to be successful. Pre-implementation planning can also help programs determine the extent to which individuals or corporations may be willing to invest in the programs. Successful mentoring programs make many decisions pertaining to program design and operations as part of their early planning processes. For example, decisions regarding program design may include how many youth a program will serve, what kinds of services it will offer, where and when mentoring will take place, the types of individuals to be recruited as mentors, and expected outcomes. In addition, successful programs often decide whether mentors will meet with youth individually or in groups. Successful programs also determine what function mentors will serve, such as whether they will offer academic support or help to socialize youth. Research suggests that having policies and procedures in place to sustain and support mentors and youth are critical elements of successful mentoring programs. According to the literature, three elements are particularly critical to the success of a mentor program: (1) mentor screening, (2) orientation and training, and (3) support and supervision. First, screening procedures provide programs a basis for selecting those adults who are most likely to be successful as mentors. Screening enables programs to better predict how a potential mentor may interact with a mentored youth, such as whether the potential mentor understands the importance of being a caring adult. In addition, screening can help determine whether the volunteer can commit enough time to the youth to build a meaningful relationship. Screening can also help ensure the safety of participating youth and can protect the program’s reputation. When screening mentors, many programs interview the potential mentors, review personal references, and check police records. Second, research indicates that mentor orientation and training experiences are critical to program success, although research has not identified how much training is ideal or what topics such training should cover. Mentor orientation and training experiences can help student mentoring programs succeed in several ways. For example, orientation and training experiences can prepare volunteers to successfully become mentors and can help ensure that both youth and mentors understand what their roles entail. In addition, orientation and training experiences can help mentors understand what they can reasonably expect to accomplish. Moreover, given that mentors often have very different backgrounds from the youth they mentor, training can help mentors better understand the youth and more effectively work toward building relationships. Third, while training can prepare mentors for potential challenges, successful programs also provide mentors with ongoing support, either from professional staff or through mentor support groups. Such ongoing support can help mentors continue to invest in their relationships with youth so these relationships can survive and thrive. By supervising and supporting mentor and youth matches, program staff can help ensure that pairs meet regularly over a substantial period of time; such regular interaction is critical to developing positive relationships between mentors and youth. Research suggests that programs in which professional staff provide regular support to volunteers are more likely to have mentor- youth matches that meet regularly. In addition, participants of such programs are more likely to report being satisfied with their mentoring relationships. In contrast, programs in which staff do not regularly contact mentors report more “failed matches”—those that do not meet consistently and, thus, do not develop into relationships. Successful mentoring programs market themselves and establish strategies to ensure long-term program viability, according to the literature. Marketing and sustainability strategies can take several forms. For example, programs may design resource development plans. Such plans may help programs diversify their fundraising by establishing how the programs will seek in-kind gifts, solicit funding from individuals and corporate donors, and apply for government funding. In addition, programs may try to garner private-sector support for mentoring by encouraging leaders in the private sector to make it easier for their employees to mentor youth. For example, program staff may encourage company leaders to allow employees to take time off from work to mentor youth. Marketing and program sustainability also includes public relations efforts. For example, mentoring programs may develop partnerships and collaborations with other organizations that support similar efforts to improve youth outcomes. Public relations also include recognition of mentors by providing tangible tokens of appreciations such as plaques or letters to mentors’ employers. Successful mentoring programs develop plans to measure expected outcomes and systematically examine and disseminate evaluation findings. For example, successful mentoring programs develop plans to measure program outcomes, determine how to measure such outcomes appropriately, and use their planned evaluation designs to assess their successes and areas needing improvement. Successful mentoring programs also disseminate their evaluation findings to volunteers, participants, funders, and the media to garner further support for their programs. Moreover, having information on program outcomes enables these programs to refine program design and operations based on evaluation findings. Most of the mentoring grantees Education funded were similar in many respects—most grantees had considerable experience operating mentoring programs, had similar goals for youth, and matched one mentor with one youth. Mentoring programs differed in the number and characteristics of youth served and the services offered them. In addition, all of the mentoring programs Education funded listed some key elements of successful programs in their applications. However, the well-established grantees we visited experienced fewer implementation challenges than did grantees new to mentoring. Our analyses of grant applications showed that most of the mentoring grantees Education funded were well-established, with considerable mentoring experience. Specifically, 81 percent of the grantees were well- established—with 5 years or more experience operating mentoring programs. For example, one grantee in Florida had mentored youth for over 40 years. Conversely, 19 percent (23) of the grantees Education funded were relatively new, with less than 5 years of experience. (See fig. 2.) In addition, most of the grantees Education funded cited similar goals for youth, reflecting the criteria identified in the application guidance, according to our review of grant applications. Nearly all grantees had goals related to improving academic achievement of participating youth (96 percent) and reducing their involvement in harmful behaviors, such as drug use and violence (87 percent). These goals were consistent with those identified in NCBLA as goals of the mentoring program. (See table 2.) About three-quarters of all grantees paired each youth to his or her own mentor, while 3 percent of all grantees (3) mentored children exclusively in groups, with 3 or 4 youth meeting at one time with a mentor. (See fig. 3.) Around one-fifth of all grantees provided both individual and group mentoring. About 70 percent of grantees listed in their grant applications that they asked prospective mentors to commit to spending at least 1 hour per week with their youth, and over 60 percent required a commitment of at least 1 school year. Other programs asked prospective mentors for a longer commitment. For example, a Nebraska grantee we visited asked prospective mentors to continue the mentoring relationships until the youth had graduated from high school. A few grantees asked mentors to commit less time than 1 hour a week. For example, one grantee we visited in Illinois asked mentors to meet with their youth for 1 hour a month. However, the mentors told us that they wanted to increase the frequency of the meetings. Although there were many similarities among grantees, they did differ in some respects, such as the number of youth they planned to serve, how much funding was available to them, and which specific at-risk youth they planned to serve. The number of youth grantees planned to serve in total over the 3-year grant period ranged from 18 in Nebraska to a high of 3,200 youth in New Mexico, according to grantee applications. Grantee award amounts varied from about $39,000 to nearly $500,000, with the average grant amount about $140,000. Although all grantees served at-risk youth, some targeted a specific group of at-risk youth. For example, one grantee in Virginia targeted children of Vietnamese refugees, another grantee in California targeted youth in foster care and residential group homes, and a New York grantee targeted court-involved youth. Grantees also differed on the types of activities mentors and youth participated in. During our visits, we observed a range of activities, some focused on academics, such as tutoring or playing a game that promoted literacy or math skills—while other activities focused on building relationships. Such activities included playing chess, playing basketball, or just simply talking. In addition, some mentors told us that they engage youth in cultural activities, such as attending a concert. Mentors also reported participating in activities with their youths that supported the youths’ communities, such as planting bulbs at a local retirement home or decorating a Christmas tree to be auctioned off at a local charity event. (See fig. 4.) Many of these mentoring activities were carried out inside of the school, such as in classrooms, the library, the gym, or in resource centers. Less frequently, mentors met with youth in their community settings, such as in a neighborhood church, community center, or public library. According to grantee applications, all grantees had some of the key elements of successful programs: initial plans for the program design and operations, including for example the number and characteristics of youth served; policies and procedures for program management such as mentor screening and training; and program evaluation activities that include an assessment of program outcomes. However, during our site visits, we found that established grantees already had fairly well-defined programs, having generally completed most aspects of the first two elements— planning and program management. Thus, these more established grantees encountered fewer implementation challenges, such as problems recruiting mentors, than did newer grantees. However, these established grantees noted the challenges they had faced in starting up their programs initially and the benefits they derived from talking with other more experienced program staff to help them along. Many of the established grantees we visited often required little additional planning for their mentoring grants. These grantees often used plans and strategies already in place, such as what youth to serve, the types of services to provide, and how to conduct mentor recruitment and training activities. For example, staff from a well-established Florida grantee that we visited told us they used the Education grant to continue serving the same youth they had served through a mentoring program whose funding had expired. Staff from a California grantee told us they used the Education grant to expand their existing school-based mentoring program into additional schools. In contrast, some of the newer grantees we visited did not have an existing base upon which to build their mentoring efforts, particularly those that were using grant funds to start a new program. As a necessary step toward implementing successful mentoring programs, these grantees had to take time during the initial part of their grant period to engage in planning activities. This planning involved determining key program design features, such as establishing program outcomes and resolving operational issues such as how to recruit mentors. Sometimes newer grantees had to revise their original plans when they experienced unexpected implementation difficulties. For example, a Delaware grantee new to mentoring had planned to provide one-to-one mentoring at local churches, but encountered difficulties transporting the children to the various locations. Subsequently, the grantee switched to a small-group mentoring approach where mentors met the children at school. Another new grantee that we visited in Nebraska had difficulty recruiting enough mentors and retaining enough youth for their mentoring effort. Moreover, during our visit to a new grantee in Idaho, we observed that some youth did not have mentors and were being randomly assigned to an available mentor on the spot for a group activity. Our review of grantee applications showed that all grantees had some policies and procedures in place to manage their ongoing operations, such as policies pertaining to mentor recruitment, screening, and training, but during our site visits we found that established and new grantees differed in the extent to which they had been able to implement such policies and procedures. Established grantees we visited already had in-place many of the policies and procedures necessary to operate a mentoring program. For example, these grantees generally had long-standing agreements with organizations in their communities that helped them attract, screen, and retain mentors. In addition, established grantees had a structure that helped them to begin operations immediately after the grant award. For example, two well-established grantees—one in Florida and the other in Ohio had either staff dedicated to recruiting or had advisory boards made up of community leaders to help them recruit and promote their efforts. In addition, more established grantees were able to retain their mentors by providing appreciation gifts and having mentor appreciation dinners and ceremonies. Some established grantees also gave mentors small gifts such as pins and note pads with the program logo on it. (See fig. 5.) In contrast, as expected for organizations in the start-up phase of their programs, the newer grantees we visited generally did not have as well- developed policies and procedures, such as those related to mentor training, recruitment, and support, as the established grantees we visited. For example, a grantee we visited in Illinois had to borrow materials from other programs to develop its training manual. Furthermore, some of the newer grantees we visited had not completed making all of their matches or the mentors and youth had only met a few times. Established grantees we visited were generally better positioned than newer grantees to market and sustain their mentoring efforts at the end of the Education grant. In particular, because many of the established grantees we visited had secured funding from multiple sources or were part of larger organizations, they were better positioned to sustain their mentoring efforts when the grants ended. For example, an established Florida grantee received funding from multiple sources, including its national affiliate, private foundations, and the United Way. In contrast, the Education mentoring grant was the only source of funding for a new grantee in Georgia. Some established grantees also developed a wide variety of materials to promote their program, including portable presentation packages, colorful, professionally printed brochures and pamphlets, magnets, and promotional videotapes. (See fig. 6.) Finally, established grantees often had more experience collecting youth and program outcomes, our site visits showed. For example, some of the established grantees that were affiliated with a national organization, such as Big Brothers Big Sisters of America, already had a set evaluation strategy, including standardized data collection forms and analysis tools. Although new grantees’ overall evaluation plans were outlined in their grant applications, some of the newer grantees we visited did not have established data collection processes or evaluation plans. Thus, unlike some of the established grantees we visited, they had to develop such processes and plans. During our site visits, some of the established grantees reported on both the challenges of starting a new program and the benefits of learning about the strategies that other mentoring programs had used to address such challenges. These grantees reported that the start-up process required many different types of activities to establish a structure and operational framework. To facilitate their implementation, they found that discussions with staff from other established mentoring organizations helped them by providing information on program design, such as strategies for recruiting and supporting mentors and program evaluation. For example, staff from an established New York grantee we visited told us they contacted other mentoring organizations for advice on mentor screening, support, and recruitment. These established grantees noted the time and effort it took to get a program operational and that key to their successful efforts was assistance they received from other more experienced programs. During our site visits, new grantees reported facing start-up difficulties, such as recruiting and retaining potential mentors. Some of the newer grantees reported seeking assistance from more experienced mentoring programs on establishing operational procedures. For example, staff from a new grantee we visited in Georgia noted they were better able to make a realistic estimate of the number of youth they could serve after consulting with an experienced mentoring program. After the grantee awards were made, Education did not establish a formal process to facilitate information sharing among grantees, although the department acknowledged the importance of information sharing among grantees and is considering such an effort. Many of the grantees we visited acknowledged the need for information sharing on grantees’ activities that could provide valuable lessons. Three of the established grantees we visited put processes in place to facilitate information sharing or presented information about their organization at conferences. For example, one grantee in Ohio that we visited developed a regional mentoring institute to share its mentoring experiences and expertise to assist interested school districts and nonprofits throughout a tristate area. To facilitate information sharing among grantees, Education is considering designating some of its fiscal year 2004 funding to develop a technical assistance center. Education officials within the OSDFS monitored grantees using multiple methods, including calling grantees regularly, examining annual performance reports, and reviewing grantee expenditure rates. However, officials did not review findings from grantees’ single audit reports. Single audit reports provide information on weaknesses related to grantee financial management, internal control, and compliance issues. OSDFS’s monitoring process included: postaward performance calls to establish progress measures; semiannual calls to grantees to determine implementation progress and issues; reviews of annual grantee performance reports to assess implementation; monitoring of expenditure rates; and visits to a limited number of sites. Based upon grantees’ annual performance reports and other data, OSDFS officials determined whether it would continue funding. With one exception, OSDFS determined that mentoring grantees were making adequate progress and warranted continued funding. Table 3 outlines elements of OSDFS’s monitoring process, including the purpose of each monitoring tool and how OSDFS provides grantees with feedback after assessing their performance using that particular tool. First, OSDFS' staff made postaward performance telephone calls soon after awarding the mentoring grants to ensure understanding of established outcomes and to offer technical assistance. During these initial telephone contacts, OSDFS staff communicated the specific outcomes the agency expected grantees to achieve and answered grantees’ questions. They also discussed measures to assess the grantee’s implementation progress. Second, OSDFS’s monitoring process has involved semiannual telephone calls to grantees to ensure that grantees are on track and to provide technical assistance as needed. During these telephone calls, OSDFS monitoring staff asked a set of questions to determine the extent to which grantees are implementing their programs as planned. Agency officials also asked grantees questions to assess the extent to which grantees have hired staff and how much staff turnover they have encountered. Third, OSDFS examined grantees’ annual performance reports. Education requires grantees to provide information in these reports that helps the agency monitor grantees. Such information includes specific examples of grantee accomplishments as well as any objectives the grantee did not meet. For example, a Florida grantee provided information in the report on the extent to which youth were meeting the program’s outcome goals, noted where desired outcomes had not been reached, and explained why. In addition, if grantees have not implemented scheduled activities, OSDFS asks that grantees explain why. OSDFS also asks grantees to describe any corrective actions they have taken or plan to take in response to previous problems OSDFS staff may have identified. Agency officials also used performance reports to ensure that grantees reconciled their expenditures with their budgets and described significant changes to their current or future budgets. Fourth, OSDFS monitored expenditure rates on a continuous basis through the Grants Accounts Payments System, according to agency staff. Agency staff used such information to identify potential problems, such as if a grantee was not expending funds at an appropriate rate. For example, while monitoring expenditure rates, OSDFS found that one grantee had spent funds, even though it had not yet begun operations. That grantee later voluntarily relinquished its grant. Fifth, as part of its monitoring process, OSDFS staff has visited a small number of grantees each year to observe how they are implementing their programs. However, because of the limited number of grantees OSDFS visited and the method by which grantees were selected, on-site monitoring is of limited value as a monitoring tool. For example, in fiscal year 2003, OSDFS officials visited three grantees. Two of them were selected because of their proximity to another grantee funded by Education under a different grant. The third program was chosen because it had ties to the program director of the grantee that voluntarily relinquished its grant. During an OSDFS visit with this grantee, agency staff also reviewed the grantee’s budget to ensure that proposed costs were allowable. Staff also verified that the grantee was serving the target population described in its application. For all three visits, OSDFS prepared a brief description of the program and the status of program implementation. Education officials in OSDFS who were directly responsible for monitoring the mentoring grants told us that they did not review grantees’ single audit reports, even though the office’s own monitoring guidance requires them to do so. Specifically, OSDFS monitoring guidance states that to decrease the likelihood of a grantee from being labeled as high risk, OSDFS should review annual performance reports, evaluation reports, and information from single audit reports, and other information readily available to them. Education officials told us that the Office of the Chief Financial Officer (CFO) within Education receives and reviews single audit reports. According to Education officials, this office did not forward information to the OSDFS officials responsible for monitoring mentoring grants because none of the information in the single audit reports pertained to the mentoring grants. Moreover, Education officials said that CFO does not receive single audit reports in instances where Education does not directly fund the program. For example, CFO would not receive a single audit report for state-administered programs, such as Title I. Using information readily and easily accessible through the online Single Audit Act database, we reviewed the mentoring grantees’ single audit summary reports. In reviewing these summary audit reports, we did not expect to find information pertaining to grantees’ handling of mentoring grants, as these were relatively new. Rather, we wanted to determine if there were issues in these same grantees’ handling of other Education grant funds they received before or around the time Education awarded them mentoring grants. How well these grantees handled other funds they received from Education could suggest how well they would manage their mentoring grant funds. We found that 8 percent of the mentoring grantees had problems with respect to other Education grants they received that were substantial enough to be reported as audit findings. For example, we found that grantees’ audit findings covered problems with cash management, procurement and reporting on Education programs. By using the online Single Audit Act database, we were also able to access information about subgrantees’ handling of Education funds. Education is currently considering whether or not it will undertake a study of its mentoring program. Although Education’s plans for an evaluation are not defined, it has required grantees to provide an evaluation of their programs at the end of the 3-year grant period. Most grantees plan to do a descriptive evaluation by reporting information on youth outcomes, particularly those related to academic achievement, incidences of harmful behavior, attendance, and drop out rates. However, the grantees varied considerably with respect to how they plan to measure these outcomes. This limits the extent to which Education can use information from the grantees to provide a national perspective on grantee outcomes. Currently, Education does not have plans to conduct a descriptive study to report on mentoring program outcomes or an effectiveness study to establish any linkages between outcomes and youth participation in mentoring programs. Many researchers consider effectiveness studies to be the best method for isolating the program’s effect on participants, from other factors, such as schooling, that could also influence participant outcomes. Such studies, which must be carefully planned and executed, are often multiyear, complex, and costly. Education officials said that although discussions are underway on whether the department will conduct a study evaluating the mentoring program, no final decision has been made. Education has required that all grantees evaluate their programs at the end of their 3-year grants and to describe their evaluation plans in the grant applications. Our review of grantee evaluation plans showed that most grantees plan to compare outcomes of participating youth at the beginning of the programs to their outcomes at the end of the 3-year grant period. In particular, grantees report plans to examine outcomes related to academic achievement, attendance, and criminal and harmful behaviors. During our site visits, we found that established and new grantees’ evaluation plans varied both in what they measured as well as measurement strategies. Newer grantees more often planned to measure program processes, such as the duration of the mentoring relationship, the number of students matched, or the number of mentors recruited. In contrast, established grantees more often had plans to report on student outcomes, such as academic achievement. Moreover, established grantees more often reported plans to use data, such as actual school grades and attendance records to measure outcomes. Newer grantees, however, more often reported plans to survey parents or teachers to gauge the extent to which outcomes improved. For example, an established grantee in New Mexico reported plans to use data from school records as well as surveys of mentors, youth, and teachers to assess whether attendance, homework completion, relationships with adults and peers, and attitudes toward school had improved. In addition, this program established targets for improvement, such as plans to decrease discipline referrals by 20 percent. In contrast, an Illinois grantee that recently began operating a mentoring program, planned mainly to report how well it complied with its process, rather than how well youth performed on outcome measures. This grantee planned to measure the extent to which its recruitment process generated participants each year and the number of children matched with mentors. Moreover, a new grantee operating in Delaware that we visited said it would report youth outcomes through self-reported information from teachers and youth. Such self-reported information may not be as accurate as that reflected in official school records. Moreover, while these individual grantee evaluations will provide some information about youth enrolled in mentoring, because of the different measures used, Education cannot combine results to provide an overall national picture. Over the past 3 years, the Congress has invested funds in a mentoring program aimed at helping children who face a significant risk of failing at school or becoming involved in illegal drugs, gangs, or alcohol have a better chance of succeeding. In funding the mentoring program in fiscal year 2004, the Congress significantly expanded the mentoring program, providing $50 million to support the last year of the existing grants as well as about 200 additional grantees. Given the recent program expansion, it will be especially important to address issues that arose during the first 2 years of the mentoring program, for example, challenges new grantees face in starting programs; limited use of monitoring tools, and the absence of a cohesive national picture of program outcomes. New mentoring grantees are faced with making many decisions about program design and operation, whereas established grantees generally have policies and procedures in place that facilitate implementation. Established mentoring grantees have benefited from consultation with other programs and from lessons learned through years of experience; both helped them operate successful programs. Without a mechanism for new grantees to access program design and implementation information, they are more likely than established grantees to struggle with program start-up and operational issues, such as recruiting and training mentors. Through its monitoring of grantees, Education has attempted to ensure that programs are managed well. However, Office of Safe and Drug Free Schools staff responsible for monitoring the mentoring grant have not used all of the means available to help it more effectively oversee programs. Findings from audit reports represent an additional monitoring tool that could provide useful information about a grantee’s stability and fiscal capacity and that may influence ongoing funding decisions. By not using single audit reports, the office responsible for monitoring the mentoring grant may be lacking information that could help it effectively assess whether programmatic and fiscal problems could weaken a grantee’s ability to successfully implement its mentoring program. Finally, Education will have some information about outcomes of youth participating in mentoring because it has required grantees to provide evaluations of their efforts. However, because these evaluations measure different outcomes and use different methodologies, their results cannot be meaningfully combined to provide a cohesive picture of program outcomes nationally. Lacking such information, Education cannot gauge the extent to which the youth outcomes NCLBA sought to affect through the mentoring grants did indeed improve during the grant period. Furthermore, because Education does not have plans for an effectiveness study, it will not be positioned to determine whether participation in the mentoring program contributes to improved youth outcomes. We recommend that the Secretary of Education (1) explore ways to facilitate the sharing of successful practices and lessons learned to help new grantees more quickly and effectively implement their programs; (2) ensure that the Office of Safe and Drug Free Schools uses grantees’ single audit reports as part of its monitoring process to take advantage of all monitoring tools that could improve the identification of fiscal and programmatic weaknesses; and (3) undertake a national study of mentoring program outcomes and in doing so, explore the feasibility of examining the effectiveness of the mentoring program in improving youth outcomes and consider collecting limited, uniform data on the next wave of mentoring grantees that could be used as the basis for such study. We provided a draft of this report to the Department of Education for review and comment. Education’s Executive Secretariat confirmed that department officials had reviewed the draft and had technical comments. In these comments, Education officials said that there is a mechanism within Education for reviewing and resolving single audit findings. Specifically, CFO within Education receives and reviews single audit reports on those entities for which the agency makes direct grants. Thus, according to Education, CFO would not receive audit reports for programs for which it does not make direct grants. We have adjusted the report to reflect Education’s technical comments. We are sending copies of this report to the Secretary of Education, relevant congressional committees, and other interested parties. Please contact me on (202) 512-7215 if you or your staffs have any questions about this report. In addition, the report will be made available at no charge on GAO’s Web site at http://www.gao.gov. Other GAO contacts and staff acknowledgments are listed in appendix III. Mentor/National Mentoring Partnership, Elements of Effective Practice, 2nd Edition, Alexandria, VA: 2003. Michael Garringer, with Mark Fulop and Vikki Rennick, Foundations of Successful Youth Mentoring: A Guidebook for Program Development, March 2003: Portland, OR. National Mentoring Center, Northwest Regional Laboratory and the Office of Juvenile and Delinquency Prevention. Susan Jekielek, Kristin Moore, et al, Mentoring Programs and Youth Development: A Synthesis. Child Trends January 2002: Washington, D.C. Jean Baldwin Grossman, editor. Contemporary Issues in Mentoring, June 1999: Philadelphia, PA, Public/Private Ventures. David DuBois, Bruce E. Holloway, et al, Effectiveness of Mentoring Programs for Youth: A Meta-Analytic Review (April, 2002, American Journal of Community Psychology. Vol. 30. No. 2, pp. 157 –197). Cynthia L. Sipe, Mentoring: A Synthesis of P/PV Research: 1988-1995. Fall 1996, Philadelphia, PA. Public/Private Ventures. Carla Herrera, C. Sipe, and et al, Mentoring School-Age Children: Relationship Development in Community-based and School Based Programs, April 2000: Philadelphia, PA, Public/Private Ventures (Prepared for the National Mentoring Partnership and funded by the U.S. Department of Education). Tierney and Grossman, Making A Difference, An Impact Study of Big Brothers Big Sister, 1995, Philadelphia, PA. Targeted group(s) Alaska $191,540 Established One-to-one pairs meeting at school 1-2 hours a week for at least 1 school year. Youth in grades 1-8 experiencing troubled home environments and attending Title I schools. pairs meeting at school 6 hours a week and groups meeting 3 hours a week for at least 4 months. Youth in grades 4-9. Youth in grades 4-12. pairs meeting at school and in the community; community- based pairs meet 3-5 hours, 2-4 times a month; school-based pairs meet 1 hour a week. Youth in grades 4-12 from Spanish-speaking families. Calif. $164,341 Established One-to-one pairs meeting at school 1 week for at least 6 months. Youth in grades 4-8 who are Hispanic. Targeted group(s) Calif. $149,885 Established One-to-one pairs meeting in the community for at least 11 months. Youth in grades 4-12 from immigrant and refugee families, youth with disabilities, or youth in foster care. Calif. $182,250 Established One-to-one pairs meeting at school. Youth living in housing projects, many whose families recently immigrated families from South America or youth from families with drug or alcohol addictions or violence. Calif. $168,530 Established Groups meeting in the community for at least 6 months. Girls in grades 4- 8 who are involved in the juvenile justice system. Calif. $171,185 Established One-to-one pairs meeting at school 3-6 hours a week. Youth in grades 3-12 who are Asian and Pacific Islander immigrants with limited English proficiency. Calif. $191,540 Established One-to-one pairs and groups meeting at school for at least 1 hour a week, for at least 1 year. Youth in grades 4-9 who are deaf or hard of hearing, in foster care, English language learners, or have mental health problems. Grass Valley Calif. $117,448 Established One-to-one pairs, meeting weekly at school for at least 1 year. Youth in grades 3-8. Targeted group(s) Calif. $184,986 Established One-to-one pairs meeting 2 hours a week at school and in the community for at least 1 year. Youth in grades 4-8 who are non- English speaking. Calif. $187,562 Established One-to-one pairs meeting at school. Youth in grades 4-12 who are involved in criminal or delinquent activities, many of whom are Mexican or non- English speaking immigrants. Calif. $95,749 Established One-to-one pairs and groups meeting in the community 4-6 hours a month for at least a year. Youth in grades 4-12 who are highly at-risk, are in foster care, reside in a group home, or have emotional and behavioral problems due to past abuse. Calif. $178,358 Established One-to-one pairs meeting at school 1-2 hours twice a week and one- to-one pairs meeting in the community 4-6 hours a week for at least a year. Youth who are Spanish speaking, girls, disabled, or Native American. Calif. $122,888 Established One-to-one pairs meeting at school and in the community 4 hours a week for at least a school year. Primarily African- American youth. Targeted group(s) Santa Paula Calif. $90,598 Established One-to-one pairs with adult mentors making weekly contact and meeting at least twice a month and one-to-one pairs with a high school mentor meeting biweekly. Youth in grades 4-12 who are most affected by violence. Youth in grades 4-12, many with limited English proficiency, including Hispanic, Asian, and refugee populations. pairs meeting in the community for 2 hours a week for at least 1 school year. Calif. $180,466 Established One-to-one pairs meeting in the community for 6 hours a month for at least 1 year. Youth in grades 4-8, middle and high school students in alternative programs such as court and community schools, and homeless and runaway youth. Calif. $183,633 Established One-to-one pairs meeting at school for 1 hour a week for at least 8 weeks (1 school semester). Youth in grades 4-8 who are Hispanic, lack adult role models, are socioeconomically disadvantaged, or have significant physical or emotional disabilities. Targeted group(s) Calif. $187,506 Established One-to-one pairs meeting at school. Youth in grades 4-12 whose families are homeless, who live in poverty (subsidized housing), who have behavior problems, or who are victims of child abuse or domestic violence. One-to-one pairs meeting at school and in the community for 2 hours a week for at least 1 year. At-risk 6 grade students who are Latino immigrants, first generation, and are involved with human services or juvenile justice. One-to-one pairs meeting at school for 1 hour a week for at least 1 school year. Youth in grades 5-6 who are Latino or bilingual. Colo. $140,231 Established One-to-one pairs meeting for at least 2 hours a week for at least 1 year. Youth in grades 4-8, including a considerable population of Hmong and Laotian children. Conn. $139,766 Established One-to-one pairs meeting at school. Court-involved youth making the transition from juvenile justice program back to public schools. Targeted group(s) One-to-one pairs and groups meeting in the community 4 contacts a month for at least 1.5 hours, with at least one in person visit lasting at least 1 hour, for at least 1 year. Youth in grades 6-8 who attend SouthEast Academy of Scholastic Excellence and live in the Capitol Hill District. One-to-one pairs meeting at school 6 hours a month plus weekly phone contact. Youth in grades 6-9 most of whom are African American. pairs. Girls in elementary, middle, or high school who are African American or other minorities. pairs meeting in the community for 2 hours a week for at least 9 months. Youth in grades 8-12 from public & charter schools with average academic records. One-to-one pairs and groups meeting at school and in the community at least twice a month. Youth in grades 4-8. pairs meeting at school for 1 hour a week, for at least 1 school year. Youth in grades 4-8 who have a history of involvement with juvenile justice system. Targeted group(s) pairs meeting at a juvenile assessment center for at least 1 year. Youth in grades 4-8 who have significant learning or emotional problems, are in an alternate school environment, or have extreme school phobias or related disorders. pairs meeting at school and in the community 30- 45 minutes a week for mentoring and twice a week for tutoring. Youth in grades 4-8. $185,985 Established This is a “drop- in” program where youth may work with several mentors during the week on different technology projects. Mentors commit to 2.5 hours a week for 6 months. Girls and Haitian, Central American, and Puerto Rican youth. pairs meeting at school 4 hours a week and groups meeting once a week for at least 1 year. Youth in grades 6-8 who are African American and reside in the 33311 zip code area. Targeted group(s) pairs meeting at school and in the community for a half hour to an hour once week. Youth in grades 4-8. pairs meeting at school at least 1 hour once a week for at least 1 school year. Youth in grades K-5 who are at risk of not reaching graduation. One-to-one pairs and groups. Youth in grades 4-12 with performance, behavior, and attendance problems. One-to-one pairs meeting at school and in the community for at least 4 hours a month. Youth in grades 4-12. pairs meeting at school 1 hour a week. Youth in grades 3-5 who consistently exhibit unruly behavior and/or are at risk of academic failure, have special needs, including but not limited to behavioral disorders, or are minority Caucasian and Asian students. Targeted group(s) pairs meeting once a week and groups meeting twice a month in the community for at least 1 year. Youth in grades 4-8. pairs meeting in the community for 2 hours a week for at least 1 year. Youth in grades 4-8 whose parents are not at home immediately after school to assist with home work or for whom English is not their first language. One-to-one pairs meeting at school for 3 hours a month. Youth in grades 6-8 who are Hawaiian. pairs meeting at school for at least 1 year. Youth 6-14 years old in Story and Boone Counties. pairs and groups meeting at school 30 minutes once a week. Youth in grades K-12. Youth in grades 4-12 who have learning disabilities or behavioral issues, a parent in prison, a parent with an addiction, or who have been in foster care. pairs meeting in the community for 1-5 hours a week for at least 1 year. Targeted group(s) pairs and groups meeting at school and in the community for at least two contacts. Youth in grades 6-12 who are Hispanic. One-to-one pairs meeting at school. Youth in grades 4 and 8-12 who are Native Americans or Hispanic migrants. Idaho $137,086 Established One-to-one pairs meeting in school for at least 1 hour a week. Youth in grades 4-8 who teachers believe are most likely to dropout, especially girls. One-to-one pairs and groups meeting at school and in the community for at least 3 years. Youth in grades 4-8. Targeted group(s) One-to-one pairs meeting at school for at least 4 hours a month for at least 1 year. Youth in grades 4-8 who are involved in criminal or delinquent activities. pairs meeting at school for 4- 8 hours a month of mentoring, 12 hours a month of tutoring, 4-8 hours a month of character development, and 30 hours a year of case management services for at least 1 year. Youth between the ages of 5 and 14. pairs meeting at school and in the community for 1-2 hours a week for at least 1 year. Youth in grades 4-8 who are African American. pairs meeting at school for 1 hour a week for at least 1 year. Youth in grades 4-5. In one elementary school, emphasis on serving youth from single-parent households. Youth who live in domestic violence emergency shelters and transitional housing. Targeted group(s) Kans. $185,959 Established One-to-one pairs meeting at school and in the community. Youth in grades 4-8. pairs meeting at school for at least 1-2 hours a week for at least 1 year. Youth in grades K-8. pairs and groups meeting at school and in the community for at least a year. Tutor Buddies meet for 1 hour a week. Big Buddies meet for 5-6 hours a month. Enrichment Buddies meet for 1 hour a week. Youth in grades K-8. One-to-one pairs and groups meeting for at least 1 year. Youth in grades 2-8 who are adopted and out- of-home youth. Framingham Mass. $126,000 Established One-to-one pairs meeting at school for 2.5 hours a week for at least 36 weeks. Youth in grades 3-5. Mass. $143,666 Established One-to-one pairs and groups meeting at school and in the community. Youth in grade 9- 12 who are Hispanic and who are talented and gifted. Targeted group(s) Youth in 8th grade who are Haitian, African- American, Caribbean, or West Indian. pairs meeting at law firms for 2 hours every other week for at least 1 year. Youth in grades 4-12 who are African-American or Hispanic, are immigrants, low- income, or have mental health or behavior problems. pairs and groups meeting in the community for 2 hours twice a week for at least 1 year. Maine $150,510 Established One-to-one pairs meeting at school for 1 hour a week for at least 1 school year. Native American youth in 2 schools. Youth in grades K-8. pairs meeting at school and in the community for 4 hours every 2 weeks. One-to-one pairs meeting at school for 1 hour a week for at least 1 year. Youth in grades 4-8, primarily boys. Minneapolis Minn. $162,407 Established One-to-one pairs meeting in the community for 3 hours a week for at least 1 year. Youth in grades 4-9 who are frequently truant. Targeted group(s) Youth in grades 4-12 who are refugees or are immigrants from Somalia, Mexico, Ethiopia, West Africa, and Latin and Central America. pairs and groups meeting at school for 1-3 hours a week for at least 1 year. Immigrant youth of Hmong, Vietnamese, Cambodian, Northeast African, and East African (Somali) descent. pairs meeting in the community for 1 hour a week for at least 1 school year. One-to-one pairs and groups meeting for 1 hour a week for at least 1 year. Youth who are Hispanic immigrants, in out-of-home placements or children of a teenage, incarcerated or court-involved parent. pairs meeting at school 2-4 times a month for at least 1 year. Youth in grades 4-8. One-to-one pairs meeting at school and in the community for 4 times a week. Youth in grades 4-12. Mont. $133,476 Established One-to-one pairs meeting at school for 1 hour a week. Youth in grades 4-12 who are learning disabled, are emotionally disturbed, have health problems, or receive inadequate support services. Targeted group(s) pairs meeting at school for 4 hours a week for at least 1 year. Youth in grades 4-8 who are people of color or are Hispanic. pairs meeting at school and in the community twice a week for a total of 3 hours a week for at least 1 year. Youth in grades 4-8 who are Hispanic and attend English as a Second Language schools. Youth are minority females, mostly African American or Hispanic. pairs meeting at school and in the community for 2 hours a week or 8 hours a month for at least 1 year. pairs meeting in the community for 3 hours a week for at least 1 year. Girls who are involved with the juvenile court, have multiple school suspensions, have experienced school failure, child abuse, poverty, and parental substance abuse, or have mental health problems. One-to-one pairs meeting at school and in the community for at least 2 years. Youth in grades 3-12 who are Native American from rural or reservation settings. Targeted group(s) pairs meeting at school for 1 hour a week for at least 1 school year. Youth in grades 1-12 with emotional, social, mental, learning, or physical disabilities or those with juvenile offenses. One-to-one pairs meeting at school for at least 1 year. Hispanic youth in grades 4-12. pairs meeting at school and in the community for 1 hour a week. Youth in grades 4-12. One grantee program serves a residential facility for juvenile offenders and another serves a school for disabled children. One-to-one pairs meeting at school and in the community with primary mentors 3 hours twice a week and twice a week phone calls, and with secondary mentors 4 hours on weekends, 4 times a month, for at least 1 year. Youth in grades 4-12 from schools with high minority populations. pairs meeting in the community for at least 1 hour a week for at least 1 school year. Mostly Hispanic and recent Mexican immigrant youth in grades 4-8. Targeted group(s) pairs meeting for 2 hours a week for at least 1 year. Youth involved with the juvenile justice system. One-to-one pairs and groups meeting at school for 1 hour a week, for at least 1 school year. Youth in grades 5-8. One-to-one pairs meeting at school for 2 hours a week for at least 1 year. Youth in the Bronx who are bilingual and multicultural. Youth in grades 3-5. pairs meeting at school for at least 1 year. Youth in grades 4-8 who are in foster care, group homes, residential mental health programs, or are “at-risk” of removal from their home due to child abuse or neglect. pairs meeting in the community.pairs and groups meeting in the community for 2-4 hours a week, for at least 1 year. Court-involved youth from the Bronx. Targeted group(s) Boys in grades 4- 8, who are in a residential treatment center and require special education, are in foster care, or have serious mental health problems. pairs meeting at school for at least 2 hours a month and meeting in the community for at least 10 hours a month, for at least 1 year. Boys. pairs meeting once a month with weekly phone contact and groups meeting 3 times a week for at least 1 year. Youth in grades 2-12. pairs meeting at school for a half hour and 1 hour a month, for at least 1 year. Youth in grades 4-8 who live in a home environment with alcoholism and/or drug addiction. pairs meeting at school for 1 hour a week and groups meeting in the community twice a month plus 2 other contacts, for at least 1 year. pairs meeting at school and in the community for at least 1 year. Youth in grades 4-8 who are first and second generation Urban Appalachians, or are deaf and have special needs. Targeted group(s) pairs and groups meeting at school and in the community for 1 hour a week, for at least 1 year. Youth in grades 4-6. Okla. $136,602 Established One-to-one pairs meeting at school for 1 hour a week for at least 1 school year. Very high-risk youth in grades 4- 8, including those from alternative schools. pairs meeting at school and in the community for 4 hours a week for at least 1 year. Youth are Hispanic, Native American, or African American. One-to-one pairs meeting at school and in the community for at least 4 hours a week. Youth in grades 3-8. One-to-one pairs and groups meeting in the community for 1 hour a week for at least 1 year. Youth in grades 4-8 who are high- risk, nonadjudicated, are adjudicated juvenile delinquents, or are adjudicated court-dependent. pairs meeting at school once a month and groups meeting in the community bimonthly for at least 1 year. Girls in grades 6- 8. Targeted group(s) Boys in 6th grade. Adults pairs and groups of three mentors/youth meeting in the community 2 days a week.pairs and groups meeting at school and in the community for 2 hours a week for at least 1 school year. Minority youth (mostly boys) in grades 6-9 who are at risk of juvenile delinquency. pairs meeting for 1-2 hours a week for at least 1 year. Youth are grade 4-8. pairs meeting for 6 hours a month for at least 1 school year. Youth are grades 2-12 with parents having a history of incarceration, addiction, or involvement with child welfare agencies. pairs meeting at school for at least 1 hour a week for at least 1 school year. Youth in grades K-12. Also focus on 9th grade at- risk and English as a Second Language students. pairs meeting at school for 1 hour a week for at least 1 school year. Youth in grades 1-12. Targeted group(s) Youth in grades 4-8, many of whom are at-risk African American youth with an incarcerated family member or with involvment in the juvenile justice system. pairs meeting for 2 hours a week.pairs meeting for at least 1 hour a week and groups meeting monthly at school. Youth in grades 3-12 who are Hispanic or African-American. Youth in grades 4-8. pairs meeting in the community for 4 hours a week.pairs and groups meeting at school once a week. Youth in grades 4-8. pairs meeting at school for 1 hour a week for at least 1 school year. Youth in grades 4-6. pairs meeting at school for 1 hour a week for at least 2 years. Youth who live in high crime areas and/or have experienced violence at home and are having mild behavior problems in school. Over half school population is Hispanic. Targeted group(s) pairs meeting in the community for 2 hours a week. Hispanic youth in grades 4-8. pairs meeting at school and in the community for at least 4 hours a week for at least 1 year. Youth are Vietnamese immigrants and refugees. pairs meeting at school and in the community for at least 4 hours a month. Youth in grades 4-8 or 9-12. Wash. $190,121 Established One-to-one pairs meeting at school and in the community for at least 1 year. Youth in grades 4-12 who are English as a Second Language students—mostly Spanish speaking or Vietnamese, Somali, Cambodian, or Russian/Ukrainian immigrants. One-to-one pairs meeting in the community at least twice a month. Youth in grades 4-8 from 3 local schools, with significant Latino and English as a Second Language students. Targeted group(s) One-to-one pairs meeting at school and in the community for at least 1 hour a week. Youth in grades 4-8. Wisc. $117,797 Established One-to-one pairs and groups meeting in the community for 2 hours a week for at least 1 year. Youth in grades 4-12 who are exhibiting predelinquent behaviors and who are involved with child protective services. Huntington W. Va. $112,363 Established One-to-one pairs meeting at school for 1 hour a week for in school program and 90 minutes a week for the after school program. Youth in grades 4-8. Wyo. $191,540 Established One-to-one pairs. Youth in grades K-12. In addition to those named above, Karen Brown, Luann Moy, James Rebbe, Thomas Broderick, and Amy Buck made key contributions to the report. No Child Left Behind Act: More information Would Help States Determine Which Teachers Are Highly Qualified. GAO-03-631. Washington, D.C.: July 17, 2003. Flexibility Demonstration Programs: Education Needs to Better Target Program Information. GAO-03-691. Washington, D.C.: June 9, 2003. Title I: Characteristics of Tests Will Influence Expenses: Information Sharing May Help States Realize Efficiencies. GAO-03-389. Washington, D.C.: May 8, 2003.

Summary: As part of the No Child Left Behind Act (NCLBA) of 2001, the Congress authorized a 3-year, $17 million per year school-based mentoring grant program. For fiscal year 2004, Congress has increased funding to about $50 million to fund additional mentoring efforts. Congress requested that GAO provide information on the student mentoring program. To do this, GAO answered the following questions: (1) What are the basic elements, policies, and procedures of successful mentoring programs? (2) What are the key characteristics of NCLBA-funded mentoring efforts, including the extent to which they have the basic elements, policies, and procedures of successful mentoring programs? (3) How does the Department of Education monitor program implementation? (4) What are Education's and grantees' plans to assess program outcomes? According to the literature GAO reviewed, successful mentoring programs (1) plan their programs carefully prior to implementation; (2) develop policies and procedures to effectively manage their programs, including mentor screening and training; (3) ensure program sustainability through marketing; and (4) evaluate program outcomes and disseminate their evaluation findings. Most of the 121 mentoring grantees that Education funded shared many characteristics--most had 5 years or more of experience mentoring youth, had similar goals, and offered "one-to-one" mentoring. All mentoring grantees listed in their applications that they had some elements of successful programs, but established grantees GAO visited reported fewer implementation challenges, such as problems recruiting mentors, than did newer grantees. Most of the 11 grantees GAO visited said they would benefit from learning about other implementation strategies through information sharing. However, Education has not facilitated information sharing among mentoring grantees, although it is considering doing so. Education used multiple methods to monitor grantees, including expenditure tracking, but the office responsible for monitoring mentoring grants did not review single audit reports as required by its guidance. Education's Chief Financial Officer reviewed the audits but did not forward audits to the office overseeing the mentoring grants because findings did not pertain to these new grants. However, GAO found that 8 percent of the mentoring grantees had audit findings related to how well they handled other Education grants. Education is currently assessing whether it will conduct an overall evaluation of its mentoring program. Education required that all grantees have evaluation plans, and most plan to report on youth outcomes related to academic achievement and attendance. However, grantees plan to use different methodologies, making it difficult for Education to have a cohesive picture of its mentoring program as a whole.