The Runaway and Homeless Youth Act, as amended, authorizes federal funding in the form of discretionary grants for three programs to assist runaway and homeless youth. These programs are administered by the Family and Youth Services Bureau (FYSB) within HHS's Administration for Children and Families (ACF). The three programs--the Basic Center Program, Transitional Living Program, and Street Outreach Program-- enable local public and private organizations and shelters in all 50 states and the U.S. territories to compete for grants that allow them to serve runaway, homeless, and sexually exploited youth who may be on the streets and in need of shelter or longer-term support. The Basic Center Program provides temporary shelter, counseling, and other services to runaway and homeless youth under the age of 18. Basic Center grants are awarded competitively to providers and may be awarded for a period of up to 3 years. The Transitional Living Program provides homeless youth ages 16 through 21 with longer-term residential services for up to 18 months. These services include such things as counseling and education in basic life skills, interpersonal skills, educational advancement, job attainment skills, and physical and mental health care. Transitional Living grants are awarded competitively to providers and may be awarded for up to 5 years. The Street Outreach Program provides education, treatment, counseling, and referrals for runaway, homeless youth under the age of 18 who have been subjected to or are at risk of being sexually abused and exploited. Street Outreach grants may be awarded for up to 3 years. See figure 1 for key aspects of these three programs. For fiscal years 2002 through 2009, funding for these programs has been just over $100 million in total, with Basic Center funding representing the largest dollar amount authorized of the three grant programs. Funding for these programs over the past several years is shown in table 1. HHS's grant award process for Runaway and Homeless Youth grants is comprised of several major steps. HHS's Family and Youth Services Bureau, which is referred to as the Program Office, and the Grants Management Office are responsible for carrying out and overseeing this process. Some of the steps in the grant award process are performed by contractors on behalf of the agency, and one step in the process is performed by panels of peer reviewers selected by the agency to evaluate grant applications. The grant award process consists of the major steps as illustrated in figure 2. Grant Announcement: Each fiscal year, the agency develops and publishes a grant announcement for each grant program. Announcements provide the information potential applicants need to determine if they are eligible to apply and the instructions on how to complete and submit their application. In addition, they include the criteria used to evaluate applications. Technical Assistance: Each announcement lists the technical service providers responsible for providing technical assistance to potential applicants to help them understand the announcement requirements. Technical assistance can be provided through a webinar, seminar, information on the Web, or upon request. Application Submission: Applications may be submitted electronically via Grants.gov or by hard copy via mail or other delivery service. Applicants may also hand deliver their application to the agency's contractor responsible for receiving the applications. The deadline for submitting applications is usually 45 to 60 days after an announcement is published. Application Pre-Screening: Applications are prescreened to determine whether they meet two requirements. Applications are eliminated from review if they are received after the deadline or if they request more funding than the maximum amount specified in the announcement. Peer Review of Applications: Applications remaining after pre- screening are submitted to peer review panels. A panel generally consists of three peer reviewers, who apply the evaluation criteria contained in the announcement to applications and score the applications, and a panel chair responsible for facilitating consensus of the peer review panel. Peer reviewers assign points to each application, based upon specific criteria that are outlined in the announcement. The points are added up and the applicant's average score is derived. This score, ranging from 0 to 100, becomes the basis for the ranked listing of applicants which the agency uses in its award decisions. Because applicants whose score places them below the total available funding limit may be denied a grant, a single point can make a difference between awarding a grant and denying a grant. Final Grant Award Decisions: Taking into account peer review panel scores and comments for each application and, in some cases, other factors, the Program Office and the Grants Management Office make the final award decisions. These decisions are documented in the final funding award decision memos, which contain the listing of all applicants, ranked by their scores, and the final award decisions. Notification of Award Decisions: Each applicant is sent a letter that communicates the grant award decision. Successful applicants are notified before letters are sent to unsuccessful applicants. application, including those for Runaway and Homeless Youth Grants, must receive an objective, "...advisory review...by a minimum of three unbiased reviewers with expertise in the programmatic area for which applications are submitted." To meet this requirement, the agency relies on the peer review process, in which three reviewers convene to evaluate and score applications based on the criteria outlined in the announcement. The peer reviewers are defined by the agency as experts in the field of runaway and homeless youth programs. Figure 3 provides an overview of the peer review process. HHS awarded grants to about one-quarter of the applicants that applied in 2007 and 2008, as shown in table 2. Based on the grant announcements we reviewed and our observation of the peer review process, the criteria upon which grant applications would be evaluated were not clearly defined in a single location in the announcement. Rather, we found that criteria were scattered throughout various sections of the announcement, had multiple labels, and were not presented in an orderly manner in a single location. For example, for the 2009 Street Outreach Program grant competition, grant applications were evaluated and scored based on how well they addressed criteria contained in three different sections of the announcement. First, applicants must address the "Program Requirements," found in Section 1 of the announcement. Second, applicants must address the "Project Description," found in Section 4. Third, applicants must address the "Evaluation Criteria" found in Section 5. However, only the "Evaluation Criteria" section of the announcement explicitly described how their responses would be evaluated and scored. Because the applicant must address criteria contained in different sections of the announcement, if applicants focused primarily on responding to the "Evaluation Criteria" they may not have adequately addressed information in the other sections. If the applicants focused only on the "Evaluation Criteria," these applicants could have received lower scores, which would have decreased their likelihood of being awarded grants. Figure 4 represents the various locations where we found descriptions of criteria. Additionally, peer reviewers we interviewed noted that consolidation of criteria in the announcement into a single location would aid them in their evaluation of applications by reducing the time it would take to review the application because they would not need to look in multiple places in the application for information. During our observation of the 2009 Street Outreach Program grant competition, we found that the agency provided detailed guidance to peer reviewers to help them evaluate and score applications. This guidance, which was not available to applicants, consolidated information from various sections of the announcement. The federal officials instructed reviewers to focus on specific information when evaluating and scoring applications. Because applicants did not have this detailed guidance, which combined information from various parts of the announcement, applicants may not have had full knowledge of what information was critical to receiving a high score. Table 3 shows examples of the guidance provided to peer reviewers during the 2009 Street Outreach Program grant competition. ACF provides technical assistance to potential applicants for runaway and homeless youth grants, as required by statute. Technical assistance is generally defined as providing expertise or support to applicants and grantees for the purpose of strengthening their capabilities for providing shelter and support services for runaway and homeless youth. In fiscal years 2006 and 2007, the agency provided technical assistance to potential applicants through its regional network of 10 providers, and listed these providers in its announcements. However, beginning in September 2007, ACF centralized its technical assistance in order to provide more consistent technical assistance for all applicants, regardless of where they were located. At that time, the agency entered into cooperative agreements with the University of Oklahoma to provide technical assistance nationwide. Through its providers, the agency coordinates technical assistance, which generally consists of a pre-application conference (webinar) covering the application requirements such as the project description, eligibility, and the evaluation criteria, among other things. After the conference, a recording and transcript is posted on the agency Web site. Potential applicants may also ask specific questions of the contacts listed on the announcement. These contacts include the Program Office officials and technical assistance providers. If the technical assistance providers cannot answer the questions, they coordinate with agency staff to obtain responses that are then posted to the Web site. The technical assistance providers also arrange seminars on broader topics related to runaway youth, such as mental health, crisis intervention, and skills training. Most of the applicants we interviewed who received technical assistance under both systems reported that they found it helpful. For example, 17 of the 20 applicants who sought technical assistance were satisfied with the help they received. However, three of these applicants said they prefer the technical assistance provided by their regional providers because of such things as the regional assistance being more "hands-on," the regional staff being more responsive and accessible, and the regional staff being more knowledgeable of local programs. Agency officials noted that the agency has moved toward centralized approach to gain a more consistent approach to the technical assistance it provides. ACF's process for determining which grant applicants will be awarded grants is primarily based on the results of the peer review process, which has weak internal controls to ensure that applications are evaluated consistently. According to GAO standards, internal controls should provide reasonable assurance that the agency's objectives, such as providing grants to the most qualified providers, are being achieved. Ideally, internal controls should be continuous, built-in components of the agency's processes, and should provide reasonable assurance that the grant award process works as it is designed to work. Our review of ACF's grant award process found that, while the agency has a number of internal controls in place to help ensure consistent application of evaluation criteria across reviewers and across panels, some of these controls are limited in their effectiveness. For example, we found weaknesses in four out of six internal controls related to the grant award process, as shown in table 4. First, ACF does not always select peer reviewers whose qualifications comply with the standards outlined in HHS policy. The policy states that each application for runaway and homeless youth grants must receive an objective, advisory review by a minimum of three unbiased reviewers with expertise in the programmatic area for which applications are submitted. Furthermore, the announcements we reviewed stated that grant application reviewers should be experts in the field of runaway and homeless youth programs. However, we found that HHS considered students, school teachers, business consultants, and television and media workers as qualified peer reviewers. Our review of resumes of all the peer reviewers and chairs for 2009 Street Outreach Program grants found that many had professional and volunteer experiences that were not always directly related to runaway and homeless youth programs. Based on the resumes of 76 peer reviewers, we found that 26 peer reviewers had direct experience with runaway and homeless youth programs listed on their resume, and another 31 had indirect experience, such as social work, teaching, or grant reviewing. However, 19 did not appear to have any of the relevant knowledge and expertise in runaway and homeless youth programs required by HHS policy. Three of these 19 reviewers were identified as "youth reviewers" in their resumes. One agency official responsible for the grant review process during 2009 explained that HHS interprets its policies governing peer reviewer qualifications broadly and accepts all related experience. He also noted that HHS encourages the use of "youth reviewers" for its peer review panels. Second, during our observation of the 2009 Street Outreach Program grant competition, we found that the meetings for peer reviewers and chairs were not mandatory. These meetings included an orientation session, panel chair meetings, and new reviewer meetings. Meetings for panel chairs are particularly important for helping to ensure consistent evaluations across panels because in these meetings, all panel chairs agree on how to apply the evaluation criteria. However, we observed that some panel chairs did not attend these meetings, and, therefore, their panels may not have applied the evaluation criteria in the same manner as panels whose chairs had attended the meetings. Similarly, new reviewers were permitted to miss the new reviewers' meetings and still participate in the reviews, which could also increase the risk of inconsistent application of evaluation criteria. Third, we observed that the agency provided detailed guidance to peer reviewers to aid them in evaluating applications. The detailed guidance provided to reviewers explaining the evaluation criteria has led to variation in application of criteria by review panels. For example, when we observed peer review panel deliberations for the 2009 Street Outreach grants, we found that peer review panels varied in the way they applied the criterion for evaluation of emergency evacuation plans. The announcement's "Evaluation Criteria" section contained the following evaluation criterion related to emergency plans: The application "describes the emergency preparedness and management plan by addressing steps to be taken in case of a local or national situation that poses risk to the health and safety of program staff and youth." At the panel session, agency officials told panels that they should also apply all of the information in the detailed guidance they were given, which included information in the "Program Requirements" section of the announcement. Federal officials advised peer reviewers that they should score the application on the following information: "Grantees must immediately provide notification to FYSB when evacuation plans are executed." As a result, peer review panels that followed the guidance gave lower scores to applicants that did not specifically indicate that they would notify the agency when an evacuation occurred. One peer review panel we observed, however, did not give lower scores when this was not specified in an application. These peer reviewers said that they did not think it was fair to assign lower scores in these cases because the more detailed information about evacuation requirements was not listed in the "Evaluation Criteria" section of the announcement. Additionally, we interviewed peer reviewers who participated in panels for 2008 runaway and homeless youth grants. Three of the six peer reviewers we interviewed told us they observed variations in the way panels applied the criteria. Reviewers said that the 2008 Transitional Living Program announcements contained evaluation criteria requiring applicants to provide background checks for all staff members who would be working with youth. However, the peer reviewers told us that the guidance provided to peer reviewers by the agency during that review process further specified that these background checks must be conducted in accordance with local, state, and national requirements. According to the peer reviewers we interviewed, this could have led to variation in how this aspect of the application was evaluated by different panels. Given that the peer review score is the key factor in determining grant awards, inconsistent evaluation criteria across panels can have a significant impact on whether an applicant is awarded a grant or not. The fourth control weakness we observed during our review of the 2009 Street Outreach Program grant competition was that agency officials did not keep a permanent record of their comments and feedback to peer review panels during their oversight of the peer review process, which introduced further potential for inconsistent application of evaluation criteria. Agency officials review the panel's scores and narrative comments for each application during the peer review process before they are finalized. The officials visit panels as peer reviewers deliberate and respond to their questions, and provide feedback to chairpersons on their panel's evaluations. Agency officials told us their review and feedback is meant to ensure that all panels apply the evaluation criteria in the same way. However, the federal officials we observed did not record this information in a permanent record. Instead, the officials provided their feedback to the chair via comments written on post-it notes. This lack of permanent documentation of federal official feedback to peer review panels makes it difficult for the agency to ensure that it is providing consistent guidance to panels and responding to problems across panels in the same way during the peer review process. No weaknesses were apparent in two of the six internal controls--(1) the provision of standard training materials to peer reviewers prior to panel sessions, and (2) the presence of federal officials on site during panel sessions to respond to questions from, and communicate on a daily basis with panels. Final funding decision memos used to internally document grant award decisions for 2007 and 2008 did not contain supporting information regarding why applications with high scores were not funded. Final decisions regarding grant awards are determined by HHS's Program Office and Grants Management Office, taking into account the review panels' scores and narrative comments for each application. According to HHS policy and guidance, the agency has the discretion to deny a grant to an applicant who would otherwise receive one based on the results of the peer review score alone. The agency is permitted to use its discretion to deny grants based on other reasons, such as the agency's concerns about the applicant's program or about the concentration of service providers in the applicant's location, which is referred to as concerns about "geographic distribution" of services. However, the agency does not always clearly document the rationale for its decision to deny a grant based on "geographic distribution" of services. When grants were denied for geographic reasons in 2007 and 2008, we found that the final funding decision memos did not clearly describe the details surrounding such denials, such as the number of other programs that exist in the same locale, the services they provide, or the numbers of youth they serve. Such details could support or justify a denial for geographic reasons. Without fully documenting and permanently recording its rationale for exercising its discretion to deny grants to highly scored applicants, the agency decision-making process is not transparent. Grant award decisions are not always communicated in a timely manner, which may present planning challenges for some applicants. According to one ACF official, successful applicants are generally notified at the end of the federal fiscal year. Based on our review of grant documents for fiscal years 2007 and 2008, we found that for all but the 2008 Transitional Living Program grants, this was true, regardless of when the announcement closed or when the funding decisions were made. For example, applications for the 2008 Basic Center Program were due in February 2008 and were evaluated and scored in March; however, applicants were not notified of their award status until September, 6 months later. HHS policy does not indicate when notification letters should be distributed to applicants, but according to an ACF official, awards to successful applicants are made by September 30 because most new programs are expected to start on or before October 1. Given the proximity of the notification date to program start date, some successful applicants with new programs we spoke with told us that the September notification timeframe did not allow enough preparation time to hire staff and secure the resources needed to provide services. See figure 5 for the timeline of dates for key events for the fiscal year 2007 and 2008 grant award process. Notification delays also create planning issues for ongoing programs that are not awarded new grants and, as a result, need to develop contingency plans for continuing or discontinuing services. Since unsuccessful applicants are notified of their grant award status after successful applicants have been notified, an applicant whose previous grant is about to expire may experience planning problems if notifications are delayed. Delays in notifying unsuccessful applicants may not give applicants adequate time to react to not being awarded a new grant. In the event that funding is denied or discontinued, earlier notification of award decisions could help providers properly plan. According to an ACF official, there is nothing in policy that prohibits notifying an applicant as soon as award decisions have been made. The official told us that delays in sending out notification letters are linked to the timeliness of writing and issuing the announcement. According to this official, announcements must be reviewed by many departments within the agency, and, therefore, the turnaround time is not as timely as it could be. However we found that even after the announcements were published and closed, applicants were still not notified of their award status for several months. For example, for the 2008 Transitional Living grant, regardless of when the announcement was published, applicants were not notified of their award, until close to 4 months after the panels had completed evaluating the applications. Similarly, notifications of decisions related to 2008 Basic Center grants were not sent out until about 7 months after the panels. In addition to the challenges applicants experienced due to notification delays, the agency created additional planning challenges for applicants when it unexpectedly changed the timing of the funding cycle for the Transitional Living Program in fiscal year 2008 without notifying applicants of this change in a timely manner. The announcement stated that ACF anticipated making grant awards in the first quarter of fiscal year 2008, which would have been from October through December of 2007. However, the grant award start date was changed to March 2008 after this announcement was published. According to an agency official, the original start date was moved in an effort to spread out the timing of peer review panels for each of the three runaway and homeless youth programs and other activities that were scheduled to occur around the same time during the summer months. As a result of moving the cycle start date--from October to March--some successful applicants were without federal funding for several months between the end of the previous grant cycle and the new grant award start date. Runaway and homeless youth service providers have also raised concerns to their congressional representatives about the timeliness of notifications. Specifically, we reviewed nine complaint letters that were sent to congressional representatives regarding runaway and homeless youth grants applications in 2007 and 2008. One letter, representing six providers, stated that notification delays created planning problems for service providers who were not able to develop contingency planning for either the continuation or discontinuation of their programs. ACF responded to the complaint by noting that it offers funding for successful applicants to recoup some of the costs that programs incurred due to the delay. In addition, the National Network for Youth, an organization that represents providers of services to youth and families also noted that the timeliness of notifications has been an issue of concern for its membership. In particular, some service providers have raised issues about the difficulties receiving timely communications from ACF concerning grant awards. All of the successful applicants we spoke with felt that their notification letters were clear and contained sufficient information; however, unsuccessful applicants were not all satisfied with the clarity and completeness of the information presented in their letters. The standard letter to unsuccessful applicants may list several possible "other factors" for the denial, beyond their peer review panel score, without any indication of which of the reasons listed in the standard notice applied to their application. See appendix I for a standard letter. The "other factors" include: "comments of reviewers and government officials," "staff evaluation and input," "geographic distribution," and "audit reports and previous program performance." Some unsuccessful applicants told us the letter did not contain enough information for them to understand why their application was denied. In particular, some applicants told us that they did not understand what the agency meant by geographic distribution, which was the basis for denying grants to at least eight applicants during fiscal years 2007 and 2008. Officials told us that "geographic distribution" means that an applicant was denied because the geographical area their program would serve is already served by another runaway and homeless youth service provider. The agency does not keep a record to document detailed information that would support or justify a denial for geographic reasons, such as the number or names of programs that exist in the same locale, the services these programs provide, or the numbers of youth they serve. As a result, it is not possible to verify that denying a grant based on "geographic distribution" was justified. Applicants who want further explanation of their award decisions may request additional information along with their scores from ACF through a Freedom of Information Act request. An ACF official told us that it would be difficult to provide all unsuccessful applicants more information supporting the denial decision based on other factors such as "geographic distribution" in notification letters because of limited resources. However, the resources needed to provide such information may be small, given that "geographic distribution" was the basis for denying grants to only a small number of applicants during fiscal years 2007 and 2008. Moreover, based upon our review of decision notices sent to applicants who were screened out of the competition due to late submissions or improper funding requests, we found contradictory language that may confuse applicants. Specifically, the letter states that "the limited availability of funds permitted us to select only the highest scoring applications that also met all of the eligibility requirements," leaving the impression that the application, in these cases, had been evaluated and scored by a peer review panel. However, applications that are screened- out of the process before the peer review session are not evaluated or scored. When we pointed out this statement to the agency, officials agreed the language could be confusing to applicants. The runaway and homeless youth grant programs provide much needed services to a vulnerable population and the number of applications far exceed the number of grants that can be awarded with available funding. To ensure that ACF awards these grants to the most capable applicants, its award process must be fair and transparent. Without clearly organized evaluation criteria in grant announcements, applicants can have difficulty determining what their applications will be evaluated on. Furthermore, without consistent evaluation of applications in the process, there cannot be a level playing field for all applicants. All peer reviewers must have the required programmatic expertise, or not all applicants are evaluated by their peers. Additionally, unless all peer reviewers attend meetings at panel review sessions; these meetings cannot help ensure consistent evaluation of applications. Without documentation of ACF comments to peer review panels during the review process there is also a risk that the evaluation process will not be consistent. Moreover, without fully documenting the rationale for denying grants to highly scored applicants, agency grant award decisions are not transparent. Once the grant award decisions are made, it is incumbent on ACF to notify applicants of decisions in a timely manner and provide them with clear and specific information about, in particular, decision not to grant awards. Without such notification, applicants may experience planning challenges and not fully understand the reasons they were denied grants. To enhance transparency and fairness in the grant award process, and improve grantees ability to plan for services, we recommend that the Secretary of Health and Human Services direct the Assistant Secretary for the Administration for Children and Families to take the following seven actions: Clearly identify in grant announcements all the criteria that peer reviewers will use to evaluate and score applications, and ensure that peer reviewers use only those criteria during the peer review process. Select peer reviewers with expertise in the programmatic area for which they are evaluating grant applications. Make all meetings for peer reviewers, including those for new reviewers and chairs, mandatory. Document and maintain records of ACF comments to peer review panels during the review process. Document the specific reasons for denying grants to high-scoring applicants in favor of other applicants for the agency record. Provide clear information to applicants about the specific reasons their applications were not approved. Notify applicants about the outcome of their applications as soon as grant award decisions are made. We provided a draft of this report to the Department of Health and Human Services for review and comment; these appear in appendix II. In its comments, HHS disagreed with our recommendation to review and revise announcements to ensure that all evaluation criteria listed be clearly labeled as evaluation criteria and be contained in a single section of the announcement. HHS maintains that all of the criteria used to evaluate and score applications are contained in section 5 of the announcement. However during the peer review process we observed, in addition to evaluating and scoring applications based on criteria specified in the "Evaluation Criteria" section of the announcement (section 5), some of the panels evaluated and scored applications based on criteria from two other sections of the announcement. Given the difference between the agency's response to our recommendation and what we observed, we are revising our recommendation to highlight the need to ensure that the all criteria used to evaluate and score applications are clearly identified to applicants and peer reviewers, and that peer reviewers use only those criteria when evaluating and scoring applications. With regard to our recommendation to select peer reviewers with expertise in the program for which they are evaluating grant applications, HHS commented that the agency has elected to accept reviewers who are knowledgeable of the risk factors faced by runaway and homeless youth, and that many professional disciplines often intersect with runaway and homeless youth. However, we found that in the past the agency has used individuals that would not be expected to have relevant expertise, such as television and media workers. Noting our concern in this area, the agency indicated that they plan to take steps to ensure that all reviewers possess the knowledge and expertise in the particular program for which they are reviewing grant applications. In the event of a shortage of reviewers, the agency intends to staff panels with at least one peer reviewer with extensive relevant knowledge, which would continue to differ from the current policy that grants must receive an objective, advisory review by a minimum of "three" unbiased reviewers with expertise in the programmatic area for which applications are submitted. We agree that professionals in varied disciplines could have sufficient expertise to serve as reviewers and recognize that it may be difficult for the agency to find enough reviewers with expertise in a particular program. As a result, we are clarifying our recommendation to include those that have expertise in the programmatic area for which they are evaluating grant applications, and not a specific program. Regarding our recommendations to make peer review meetings mandatory, HHS indicated that all meetings for peer reviewers and chairs are already mandatory but due to unforeseen factors, it is not always possible for all reviewers to attend. Indeed, during our observation of a peer review session, not all reviewers and chairs attended the meetings. Moreover, at the time, agency officials told us that attendance at these meetings was not explicitly mandatory, but highly encouraged. They also indicated that attendance was not enforced and attendance records were not maintained. In response to this recommendation, the agency indicated that they plan to officially notify all reviewers and chairpersons participating in future reviews that all training is mandatory. In the event some reviewers and chairpersons are not able to attend the mandatory training sessions due to unforeseen circumstances, the agency intends to offer "make up" sessions. HHS did not provide comments on our recommendation to maintain records of ACF comments to peer reviewer panels during the review process. However they agreed with our recommendation to document the specific reasons for denying grants to high-scoring applicants in favor of other applicants. HHS commented that the agency plans to include more details concerning geographic distribution in the letters to applicants who are denied grants for this reason. While these efforts would be in line with our recommendation; the details supporting such decisions must be consistently documented in the agency's records to support the information provided to applicants in their letters. In response to our recommendation to provide clear information to applicants about specific reasons their applications were not approved, HHS stated that in accordance with ACF policy and procedures, every unsuccessful applicant is entitled to an explanation of why their application was not funded. In addition, the agency noted that, upon request, the Program Office will provide a debriefing to applicants. However, letters sent to unsuccessful applicants should clearly note that applicants may request a debriefing by the Program Office regarding specific reasons why their application was not funded. Currently, letters to unsuccessful applicants do not include this information. In addition, it is important to revise the language in letters to applicants that are screened out of the grant competition that implies their application was evaluated and scored. Finally, HHS agrees with our recommendation to notify applicants about the outcome of their application as soon as grant award decisions are made. As part of the grant application process, the agency plans to explain to applicants that final grant decisions depend on the results of the grant award negotiations between ACF and the prospective grantees. We recognize that these grants are discretionary and that final award decisions involve negotiations that may take time. However, every effort should be made to complete negotiations and notify both successful and unsuccessful applicants as quickly as possible. To enable applicants to efficiently and effectively manage their programs, it is important for applicants to receive their notices in a timely manner. HHS also provided technical comments, which we incorporated into the report as appropriate. We are sending copies of this report to the Secretary of HHS, relevant congressional committees, and other interested parties. In addition, the report will be made available at no charge on GAO's Web site at http://www.gao.gov. If you or your staffs have any questions about this report, please contact Kay E. Brown at (202) 512-7215 or brownke@gao.gov. GAO staff who made major contributions to this report are listed in appendix III. In addition to the contact named above, Clarita Mrena (Assistant Director) and Jacqueline Harpp (Analyst-in-Charge) managed all aspects of the assignment up to report production. Anna Kelley managed the report production and Vernette Shaw made significant contributions in all aspects of the work. Lisa Fisher and Jennifer McDonald also made significant contributions to this report. Additionally, Walter Vance and Minette Richardson provided technical support in design and methodology. James Rebbe provided legal support and Susannah Compton assisted in message and report development. James Bennett assisted with visual communications.

Summary:
The Department of Health and Human Services (HHS) awards grants to provide shelter and services to runaway and homeless youth through the Basic Center, Transitional Living and Street Outreach Programs. In response to a mandate for a review of the grant award process for these programs in the Reconnecting Homeless Youth Act of 2008 (Pub. L. No. 110-378), GAO examined (1) grant announcements and application requirements, (2) technical assistance for grant applicants, (3) how grant award decisions are made, and (4) notification of grant award decisions. GAO reviewed requirements, documents, and records associated with this process for fiscal years 2007 and 2008, observed the grant evaluation portion of this process, and interviewed applicants, peer reviewers, and agency officials. Based on GAO's review of past grant announcements for these programs, GAO found that the criteria upon which grant applications were evaluated were not clearly identified or presented in a single location in the announcement. Rather, GAO found that criteria were scattered throughout various sections of the announcement, had multiple labels, and were not presented in an orderly manner. As a result, applications that did not address the criteria from all sections were likely to receive lower evaluation scores, decreasing their chances of receiving a grant. HHS provides technical assistance to potential applicants for runaway and homeless youth grants, as required by statute. Of the 20 applicants GAO interviewed who sought technical assistance, 17 were satisfied with the help they received. Grant award decisions are primarily based on the results of the peer review process, and internal controls in place to ensure that applications are evaluated consistently were not always adequate. GAO found weaknesses in four out of the six procedures the agency relies on to ensure consistent evaluation of applications. For example, although HHS policy requires peer reviewers to be experts in the field of runaway and homeless youth programs, about one- quarter of the reviewers who evaluated applications for 2009 Street Outreach grants had little or no experience in this area. With regard to notification of grant award decisions, GAO found that they have not always been communicated to applicants in a timely manner, which can delay the start of new programs and present planning challenges for existing ones. GAO also found that the information in notification letters to applicants who were not awarded grants was not always clear or complete.