Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program
Released: 2005-11-29T00:00:00

Letter

Background

The National Guard Youth Challenge Program is a 17-month program that serves at-risk youth at 29 sites in 24 states and Puerto Rico. The purpose of the program is to improve the education, life skills, and employment potential of students by providing military-based training, supervised work experience, and knowledge in eight core program components. Students must be 16 to 18 years old, drug-free, unemployed, high school dropouts, and not in trouble with the law. NGB reports that more than 59,000 students have been graduated from the Challenge Program since it began as a pilot in 1993. The program was authorized by 32 U.S.C. §509 on a permanent basis in fiscal year 1998, at which time states were to begin paying a share of operating costs.

Challenge Program Structure

Each Challenge Program site operates two residential classes per year, one of which begins in January and the other around July. A typical graduation goal is 100 students per class, or 200 per year, although several programs graduate more students. In 2004, for example, Illinois graduated almost 800 students in 2004, and Louisiana’s three sites combined graduated more than 950 students. The residential phase of the program runs 22 weeks and includes a 2-week Pre-Challenge phase. During Pre- Challenge, applicants are assessed for their ability and motivation to complete the remaining 20 weeks of the residential program. Those who successfully complete Pre-Challenge are then formally enrolled in the Challenge Program in numbers that equal each program’s graduation target plus normal program attrition rate. In the residential phase, students receive military-based training and supervised work experience. Additionally, each state develops a curriculum that incorporates the eight core components and the tasks, conditions, and standards that students must complete to demonstrate progress in those components. Each student must receive a score of at least 80 percent on each core component to be graduated from the program.

During the 12-month post-residential phase, individuals who have successfully completed the residential phase are involved in placement activities, which include employment, education, volunteer activities, or any combination of the three or military service. The graduates work with adult mentors who were matched with them during the residential phase. These mentors provide guidance and support to the graduates and are required to contact the youths twice each month at a minimum. Program staff use the written, post-residential action plan that each student prepares and updates during the residential phase to monitor placement activities. Mentors also use this plan during their interactions with graduates. The Challenge Program reports youth placement activities at the end of the 12-month follow-up period. To further assess the long-term impact of the program, NGB has contracted with AOC Solutions to conduct a retrospective longitudinal study of program graduates as well as students who did not complete the program.

Program Funding

Prior to 1998, the federal government, through DOD, completely funded the Challenge Program. In fiscal year 1998, Congress began requiring states to provide a minimum of 25 percent of their programs’ operating costs. The state cost share increased 5 percent each year until fiscal year 2001, when it reached the current funding requirement of 40 percent. Although some states had provided more funds than required in the past, program funding each year is now determined by the 40 percent share, which is based on $14,000 for each youth targeted for graduation. In addition to the federal and state funds used to operate the program sites, DOD also provides funds for NGB management expenses such as program evaluations, contractor-provided training, and travel for training and workshops. These NGB program management costs are not subject to the federal/state 60/40 cost share requirement.

Each state submits a budget to NGB that is based on that state's target for number of graduates. Since the program’s inception, the funding provided by NGB has been based on a cost per student of $14,000. For example, if a state has a target of 100 students per class (200 per year) to graduate, the estimated program costs would be $2.8 million. The federal contribution, or 60 percent of the total, would be $1.68 million, while the state contribution would be $1.12 million.

To receive federal funding, a state must certify that it has sufficient funds to provide its 40 percent share. State funds can be composed of cash, noncash supplies, services, or a combination of these sources. States are allowed to provide additional funding (over and above the 40 percent share) to the program from sources such as individual and corporate donations, additional moneys from the state general fund or other state revenue sources, or other federal funding. Some Challenge Program sites, for example, operate as alternative schools and are reimbursed by their state education agencies for portions of their program costs.

Management, Oversight, and Support

Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for preparing the annual budget and reviewing state budgets and funding certifications. Reserve Affairs is to monitor program compliance with DOD policy, issue supplemental policy guidance, and submit the Challenge Program annual report to Congress. NGB provides day-to-day administration and oversight of the Challenge Program, issuing regulations, and submitting budgets and annual report drafts to Reserve Affairs. NGB has contracted with AOC Solutions to assist with the oversight of the Challenge Program. AOC Solutions performs the annual operational evaluations and the biennial resource management reviews. This contractor also pulls together the program information for the annual report and maintains and oversees the Data Management and Reporting System (DMARS), which is used to collect student data and report on individual and program activities. NGB has also contracted with Dare Mighty Things to provide training and technical assistance to Challenge Program staff. Finally, NGB has United States property and fiscal officers in each state who are responsible for receiving and accounting for all federal Challenge Program funds and property under control of the National Guard in that state. The property and fiscal officers are also responsible for ensuring that federal funds are properly obligated and expended.

NGB enters into cooperative agreements with governors of states approved to participate in the Challenge Program. The cooperative agreements describe the responsibilities of the states and NGB as well as the funding, costs, and regulations for operating National Guard Youth Programs. The cooperative agreements also define the eight core components and provide guidance on how to run the residential and post- residential phases and other aspects of the Challenge Program. Each Challenge Program state is also required to submit state plans and budget estimates for their state. These state plans must include details on the state’s procedures and be consistent with overall program guidance provided by DOD. For example, state plans include information on application and selection procedures, staffing and staff training, and a detailed budget.

Challenge Program Expenditures and Participation Have Increased and NGB Has Reported Positive Performance Outcomes

According to NGB, Challenge Program expenditures and state participation have increased since the program began, and the program has achieved positive program performance outcomes over time. Since the program’s inception, total expenditures have increased from about $63 million to about $107 million per year. The number of states participating in the Challenge Program has also increased, and several states have expressed interest in adding a program or expanding existing ones. Challenge sites must account for their activities throughout the year, and NGB has reported positive performance outcomes over time.

Program Expenditures Have Increased Over Time

NGB reports that overall federal expenditures for the Challenge Program have increased over time, but states have also increased their expenditures since the program was permanently authorized in fiscal year 1998. Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to about $107 million. For fiscal year 2004, for example, NGB expenses included $61.6 million for the federal cost share and $5.8 million for NGB management costs, while states contributed approximately $40.5 million. In addition, in 2000 and 2001, the Challenge Program received $5 million and $7,483,500 respectively from the Department of Justice. Reserve Affairs stated that the primary use of these funds has been to start new Challenge Program sites. Since 2001, four programs were established using these funds and three programs’ operations were maintained in 2002. In total, approximately $5.97 million remain unspent in a nonexpiring account. Officials at Reserve Affairs and NGB told us that these funds remain unspent because no new Challenge Programs have started. According to these officials, new programs have not been established because state governments have not committed the required 40 percent match. (See fig. 1 for total program expenditures from fiscal year 1998 to 2004, broken down by federal and state cost share and NGB management expenses.)

The Number of States Participating in the Challenge Program Has Increased

When the pilot program began in 1993, there were 10 Challenge Program sites in 10 states. The program has now grown to 29 sites in 24 states and Puerto Rico. In fiscal year 2005, Wyoming received funds to start up a program site. According to NGB, Wyoming will begin its first class in January 2006. In addition to those states currently operating Challenge Program sites, there are also nine states that have expressed interest in establishing new programs. For example, according to NGB officials, representatives from Washington and Indiana National Guard units have visited some existing program sites and are in the process of developing state programs. Other states are interested in expanding their programs to serve more youth at existing sites or to open new locations. On the other hand, for various reasons including difficulty meeting the state match requirement, lack of state support, and substandard facilities, four states have discontinued their Challenge Programs. Connecticut, a pilot program state, dropped its program in 1994 after two classes. Colorado discontinued its program after 1999, Missouri after 2002, and New York after 2003. Figure 2 describes the number of Challenge Program sites for each year since the program began. Appendix II identifies the individual states with Challenge Program sites.

Student participation in and graduation from the Challenge Program have also increased over time. States are required to track the number of youth who have applied to the Challenge Program, enrolled in the third week of the program (after the 2-week Pre-Challenge phase), and were graduated from the residential phase. According to NGB, the target graduation rate for 2004 was 6,961; the actual number of enrollees was 8,920; and 7,003 students were graduated, or 79 percent of those enrolled, from the Challenge Program. Figure 3 shows the target numbers, the actual number of students who were enrolled in the residential phase at week 3, and the number that graduated from the program from 2000 through 2004.

Although all Challenge Programs graduate two classes per year, the number of graduates per class varies. In addition, some states have multiple programs. For example, Louisiana has three Challenge Programs and, in 2004, graduated a total of 952 students. Figure 4 identifies those states currently participating in the Challenge Program by the number of graduates they reported for 2004.

NGB Has Reported Positive Performance Outcomes

NGB has reported positive performance outcomes in academic performance, community service activities, and post-residential placements. Program performance information is tracked by each Challenge site and submitted to NGB. Each year, the Challenge Program reports on outcomes for the two classes completing the 22-week residential phase during that reporting year and for the two preceding classes as they complete their 1-year post-residential follow-up phase. The Challenge Program sites use the same automated system, DMARS, to collect information on students and report on their progress and activities. The information collected in DMARS is reviewed by the contractor through weekly and monthly reports and during random checks of source documents during operational evaluation site visits.

Some residential phase outcomes of the Challenge Program, such as the number of graduates earning a general educational development (GED) credential or high school degree and changes in scores on standardized math and reading tests, are tied to the core component of Academic Excellence. For example, NGB reported that 70 percent of graduates in 2004 earned a GED. Figure 5 illustrates the outcomes of GED attainment for the past 5 reporting years. Students also take the Tests of Adult Basic Education, a series of tests that identify individual education levels in various academic subject areas. Each state program tests its students early in the residential phase and then toward the end of the 22-week period, and it reports the changes in test scores. In 2004, for example, NGB reported that graduating students improved 1.7 grade levels in reading and 1.8 grade levels in math during the residential phase. Another core component, Service to the Community, requires each student to perform a minimum of 40 hours of service to the community or conservation project activities. The number of community service hours performed by each student is tracked, and the total number of hours for each site is another outcome that the Challenge Program reports annually. For example, in 2004, NGB reported that Challenge Program students performed more than 590,000 hours of community service, such as maintaining historical cemeteries and parks and supporting organizations such as Special Olympics and Habitat for Humanity.

Each month of the post-residential phase, each Challenge Program graduate, or that individual’s mentor, reports on the graduate’s post- residential activities. Following the 12-month post-residential phase, each Challenge Program site reports graduate placements in continuing education, the military, or the labor force. These placements are verified with schools, the military, and employers and are documented. Program representatives are not always able to contact all graduates for placement information and therefore placement data reflect only the students contacted, not all graduates. Education placements include returning to high school or going to a post-secondary or vocational-training institution, which students may be attending full- or part-time. Some Challenge Program graduates also enter the military, into either the active or the reserve forces. Post-residential employment placements can be full- or part-time, and they include those graduates who are self-employed. Graduates can have placements in more than one of these categories. For example, an individual might be attending school and working part-time. Challenge Program sites continue to update their placement records after the 12-month follow-up period when they come in contact with former students. During the longitudinal study, for example, the contractor has been able to update placement data based on information received from state program officials and graduates. Figure 6 shows post-residential placement trends for the past 3 reporting years. The total numbers of graduates placed in these 3 years are 2,407 in 2003; 3,698 in 2004; and 4,086 in 2005.

Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program Costs or the Need to Adjust the Federal and State Cost Share

Although Reserve Affairs and NGB have expressed concern about the current program funding level and have suggested increasing both the cost basis used to determine funding needs and the federal cost share, we found that neither Reserve Affairs nor NGB has performed analyses to support the need for such changes. Good budget practices, included in the Office of Management and Budget’s Federal Financial Accounting Standards, state that agencies should determine actual costs of their activities on a regular basis and that reliable cost information is crucial for effective management of government operations. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost-share ratio.

Other than calculating how inflation has affected program costs, NGB has not analyzed data on actual program costs. Since 1993, NGB has used a cost of $14,000 per student as the basis for determining the amount of funds needed to cover program operating costs. In 2003, NGB calculated that if that amount were adjusted for inflation, it would be $18,000. The results of our survey of all Challenge Programs showed that in 2004, states actually spent between $9,300 and $31,031 per graduate with an average of $15,898 per graduate. In addition, our survey showed that, on average, states estimated that the program should be funded at approximately $16,900 per target graduate to cover all of the services in the cooperative agreements, although the estimates ranged from $14,000 to $31,800. Most Challenge Program officials also told us that increasing the cost per student funding level for the program without increasing the federal cost share would negatively impact their programs because their states would be unlikely to come up with the additional state match money. Because costs vary between states due to regional differences in salary levels, staff benefits, and facility costs, Reserve Affairs has asked NGB to determine a new funding formula for the program based on individual state needs. At the time of our review, NGB had not yet done this and Reserve Affairs has not given NGB a deadline for completion.

In addition to expressing a desire to change the amount of funding per student, Reserve Affairs, NGB, and participating states have suggested that the cost-share ratio be changed from its current 60 percent federal share to a 75 percent federal share because they believe that the current 40 percent state share is sometimes difficult for states to meet; however, neither Reserve Affairs nor NGB has analyzed states’ financial situations or the impact of adjusting the federal and state cost share. Challenge Program officials told us that increasing the federal cost share of the program would be beneficial because it would enable states to expand their existing programs; give states more flexibility in funding their programs; and allow programs to restore to students some services that had been eliminated due to budgetary constraints. Although we did not analyze how changing the cost basis or the cost-share ratio would affect specific states, we prepared hypothetical examples for illustrative purposes. Table 1 shows how changing the cost basis and the cost-share ratio would affect federal and state required funding levels.

In our survey, states reported varying views on whether they were experiencing difficulty in meeting their share of program costs. We did not verify the basis for their responses. Our survey showed that some states are able to provide funds above the required match; some states provide only the required match; and some states are unable to provide a match based on $14,000 per student and therefore fund the program at a lower level and receive less money from NGB. For example, California is able to provide additional support beyond the required match through additional money provided from the state general fund and funding from the program’s local school district. In 2004, California spent approximately $20,200 per graduate. Oregon, on the other hand, funds its program primarily through state education money. Due to recent state budget difficulties, Oregon cannot fund the program at $14,000 per student. In 2004, Oregon spent approximately $12,600 per graduate. According to the National Guard Bureau, all 29 programs are providing the services required by the cooperative agreements, and several states have added program enhancements such as field trips or vocational classes. However, some states reported that they reduced nonrequired services to stay within their budgets. For example, they implemented pay and hiring freezes; eliminated the student stipend; and eliminated program enrichment activities, such as field trips and vocational classes. Some states told us that additional funding, provided by a change in the cost- share ratio or an increase in the per student funding amount, would allow them to restore some of these services.

The National Guard Bureau Lacks a Complete Framework for Oversight of the Challenge Program, Making It Difficult to Measure Program Effectiveness

Although NGB has several mechanisms in place for overseeing the Challenge Program, it lacks a complete oversight framework, making it difficult to measure the effectiveness of the program. A complete oversight framework, as suggested by the Government Performance and Results Act of 1993 (the Results Act) and Standards for Internal Control in the Federal Government, includes performance goals and measures against which to objectively measure performance as well as a mechanism for tracking findings of audits or reviews and responding to those findings. Currently, NGB does not require participating states to establish performance goals for individual programs and therefore does not have a firm basis for evaluating program outcomes and DOD’s return on investment. In accordance with the cooperative agreements, U.S. property and fiscal officers in each state are required to conduct full audits of state Challenge Programs at least every 3 years. However, these audits have not been conducted as required; and, when audits are conducted, copies of the results are not provided to NGB for review. Without regular audits and access to results, NGB cannot be assured that programs are using federal funds appropriately and that audit findings are addressed.

The National Guard Bureau Conducts Several Oversight Activities and Recently Implemented a Mechanism for Tracking Audit Findings

NGB conducts several oversight activities for the Challenge Program. In accordance with the DOD Instruction and cooperative agreements governing the Challenge Program, NGB is responsible for the overall administration of the program, including program oversight. NGB uses both informal and formal mechanisms to oversee the program. On an informal basis, NGB frequently communicates with state program directors via e-mail and telephone calls. In addition, according to NGB, if a program director has a problem or an issue that he or she feels NGB needs to be involved with, he or she will initiate contact. Formal oversight of the program is conducted by NGB through AOC Solutions with yearly operational evaluations and biennial resource management reviews of all 29 Challenge Programs. The purpose of the operational evaluations is to assess the programs’ compliance with the cooperative agreements and the implementation of the eight core components. The resource management reviews focus on assessing programs’ financial accountability and reviewing resources including staffing levels and salaries; food service costs; and physical inventory of property.

Both the operational evaluations and the resource management reviews have identified areas for improvement and, according to NGB, changes were made to the program. For example, the operational evaluation of one state program conducted in fiscal year 2004 reported that program staff was calling the students inappropriate names. According to NGB, a staff member was dismissed as a result of this finding. Another operational evaluation of a different program conducted in fiscal year 2005 found that over 90 percent of the community service hours accumulated by the students were for kitchen patrol. The cooperative agreements state that work in the dining facility may not be counted towards community service hours. According to NGB, this program has completely revamped its community service program and students now participate in such activities as visiting with residents of the local veterans’ home and caring for a historic cemetery.

Until recently, NGB did not have a formal mechanism for tracking the findings of the reports conducted by AOC Solutions. During our review, we discussed with NGB the importance of keeping track of review findings in order to adequately respond to these findings, in accordance with Standards for Internal Control in the Federal Government. In response to our review, in October 2005, NGB provided new guidance to AOC Solutions regarding the operational evaluations and resource management reviews, which required, among other things, AOC Solutions to review findings from previous evaluation reports to determine whether corrective actions have been taken where warranted. In addition, officials at NGB told us that they currently monitor responses to audit and review findings informally with individual program directors.

The National Guard Bureau Does Not Require Participating States to Establish Performance Goals for the Challenge Program

Although NGB requires state Challenge Programs to report on certain outcome measures, such as GED attainment and number of graduates, NGB does not require states to establish any performance goals in these areas to measure the effectiveness of the program. The establishment of performance goals is consistent with the principles of effective management as set forth in the Results Act and would allow NGB to better evaluate the overall performance of the program and to assess DOD’s return on its investment. Without clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance and thereby assess DOD’s return on investment.

Although it may not be reasonable to have the same performance goal for all state programs, it would be appropriate for each state program to negotiate a performance goal for defined performance areas such as increases in standardized test scores or physical fitness levels. Similar state programs, overseen by the Department of Labor, set individual negotiated levels of performance for specified core performance measures. These measures are used to provide information for systemwide reporting and evaluation for program improvement. For example, state Workforce Investment Act programs negotiate performance measures for youth ages 14 to 18 in three areas: attainment of basic skills; attainment of high school diplomas or their equivalents; and placement in education, the workforce, or the military. For the area of diploma attainment, the goals range from 42.8 percent of participants in Louisiana to 68 percent of participants in New Hampshire.

Currently, Challenge Program states are required to submit state plans annually. These state plans are required to contain long-term and annual performance goals and are to be updated annually. However, NGB has not provided guidance on specific performance areas where states should focus their goals; therefore, states may not have goals in the same performance areas, making it difficult for NGB to compare performance across programs. For example, California’s state plan contains a goal to acquire additional sources of funding through grants and charitable contributions. Oregon’s state plan, on the other hand, contains a goal stating that 80 percent of graduates from the residential portion of the program will be placed in education, the military, or employment but does not contain any goal related to acquiring additional sources of funding. In addition, states are not currently held accountable to the goals that they do set since the evaluation process does not measure the states’ performance against their goals. NGB told us that beginning in January 2006, states would be held accountable to the goals outlined in their state plans.

U.S. Property and Fiscal Officers Have Not Conducted Required Audits and Audit Results Are Not Provided to NGB

Although the cooperative agreements governing the Challenge Program currently require U.S. property and fiscal officers to perform full audits of state Challenge Programs at least every 3 years, and prior to January 2005, the cooperative agreements required full audits every year, these audits have not been conducted as required. For example, according to NGB, out of a required 29 audits, only 14 were conducted in 2003, and only 7 audits were conducted in 2004. According to property and fiscal officers that we spoke with, audits were not conducted due to the lower priority placed on these audits compared to other audits that needed to be conducted within the state and a lack of staff to conduct the audits. Because the property and fiscal officers are responsible for ensuring that federal dollars are appropriately spent, if these audits are not conducted, it may be difficult to ensure that federal interests are adequately protected.

When the property and fiscal officers do conduct audits of the Challenge Program, they are not currently providing copies of the audit results to NGB because, according to the Chief of Property and Fiscal Affairs at NGB, there is no specific requirement to do so. Standards for Internal Control in the Federal Government states that agencies need to ensure that the findings of audits and reviews are promptly resolved. If NGB does not review these audits, it cannot ensure that audit findings are resolved or identify trends across programs that may require action at a programwide level. According to officials at NGB, the audits remain internal to the state and the property and fiscal officer works directly with the state Challenge Program site to resolve any issues. In addition, if any audit findings require action from NGB, the state property and fiscal officer will contact NGB and ask for assistance. NGB needs to be aware of all audit findings, including those reported by the property and fiscal officers to effectively manage the Challenge Program.

We reviewed the most recent property and fiscal officer audits for each Challenge Program participating state and found that they did identify areas that needed improvement. For example, an audit conducted of one program in fiscal year 2004 found unspent program dollars totaling approximately $180,000 that needed to be returned to the NGB. According to a follow-up report by the property and fiscal officer, this money was returned to NGB. Another audit conducted during fiscal year 2003 found that oversight of budget expenditures for one program was not adequate to ensure accurate, timely, and complete accounting of expenditures and recommended that key management controls over the program be identified, documented, published and tested. A follow-up audit of the program found that this recommendation was implemented.

Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding Support for the Challenge Program, While Some Participating States Have Obtained Alternative Funding Support as a Result of Their Efforts

Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding, while some participating states have obtained alternative funding support for their programs as a result of their own efforts. Because Reserve Affairs has not made a formal business case to request funds from nondefense agencies, these agencies are unable to determine whether or not they are able to fund the Challenge Program. As a result, Reserve Affairs is potentially missing out on additional sources of funding that could enhance the program. Some participating states have obtained alternative funding support to enhance or maintain services provided to Challenge Program students.

Nondefense Agencies Are Unable to Determine If They Can Fund the Challenge Program without a More Formal Request for Funding

Reserve Affairs has not made a formal business case to request funds from nondefense agencies and therefore these agencies are unable to determine whether or not they are able to fund the Challenge Program. Although the authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program, Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding. Rather, Reserve Affairs has primarily adopted informal strategies to contact agencies outside DOD to inform them about the Challenge Program and seek opportunities for partnerships. For example, Reserve Affairs officials have sent information via e-mail to officials at federal agencies with programs targeting at-risk youth (i.e. Department of Justice (DOJ), Department of Labor (DOL), and Department of Education (ED)) to inform them about the Challenge Program. These e-mails did not include specific requests for program funding. In addition, in July 2003, Reserve Affairs presented senior DOL officials with a proposal for forming an interagency partnership with the Challenge Program, but did not ask for a commitment of funding from DOL in a specified amount. Additionally, officials from these agencies have been invited to events sponsored by the National Guard Youth Foundation in support of the Challenge Program. Lastly, Reserve Affairs officials told us that they participate or have participated on a number of interagency councils and working groups that represent at-risk youth such as the President’s Crime Prevention Council, the White House Federal Interagency Working Group on Service, the Corporation for National and Community Service, the National Civilian Community Corps, the Math Science Initiative, and the Juvenile Justice and Delinquency Prevention Council.

Officials from DOJ, DOL, and ED noted that their agencies have general authority to provide funds to other programs if the transferred funds were used to support a program consistent with their agency’s interests. It is not unprecedented for an agency to transfer funds to the Challenge Program. In fiscal years 2000 and 2001, DOD and DOJ signed an interagency agreement under which DOJ agreed to provide $5 million in 2000 and $7,483,500 in 2001 to the Challenge Program. As DOJ concluded in this situation, and as officials at DOJ, DOL and ED stated, the transfer of funds to another agency must be consistent with the purpose and legal requirements of the program from which funds would be transferred. Moreover, these officials stated that sufficient funding must be available in the program from which the funds would be transferred, and making such a transfer must be in the interest of the agency.

Officials at DOJ, DOL, and ED stated that Reserve Affairs needed to present more specific information to them before they could make a determination as to whether funds could be provided to the Challenge Program. These officials stated that an executive branch agency that wishes to make a request for funding from another executive branch agency could make such a request in any number of ways, either formally or informally. At a minimum, they noted, any such request should contain the amount of funding sought and a sufficiently detailed description of the program to allow the agency receiving the request to determine whether it would be an appropriate use of funds. At the time of our review, none of the agency officials we met with were aware of any specific request from Reserve Affairs concerning this matter. Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they are able to provide funds for the Challenge Program.

Some Participating States Have Obtained Alternative Funding Support to Enhance or Maintain Services Provided to Challenge Program Students

Under cooperative agreements with NGB, states are not required to seek funding support beyond the required federal and state contributions; however, some states have made efforts to obtain alternative funding support for their programs to enhance or maintain the services provided to Challenge Program students. Under authorizing legislation for the Challenge Program, states may accept, use, and dispose of gifts or donations of money, other property, or services for the Challenge Program. Reserve Affairs and NGB officials said that they encouraged states to seek out additional funding sources for their programs if they want to enhance the services provided to program students.

To assist state programs in their efforts to obtain that additional support, NGB, through its contractor Dare Mighty Things, shares information among state programs on strategies for successfully organizing a 501 (c) (3) corporation, developing a fundraising policy that focuses on the long- term vision of the Challenge Program, accessing and sharing a grant writer with other programs, applying for National School Lunch Program funds, and educating state legislators to secure funding support. By providing information and examples of programs that are currently implementing these strategies, other state Challenge Programs have the information available to them on how to obtain additional funding support for their programs. In our review of the program, we found that some states had identified strategies for soliciting additional funding support from nonprofit organizations. For example, in the annual plans submitted to NGB, some states developed specific strategies for obtaining additional funding support such as setting a goal for the number of grants they would apply for and establishing a 501 (c) (3) corporation to raise funds on behalf of the program.

We saw further evidence of states’ efforts to obtain additional funding support through our site visits and in our survey of the 29 Challenge Program sites. For example, in one state we visited, the Challenge site was also a charter school, which allowed the program to receive additional funding from the local school district. Moreover, in four states we visited, the state legislatures provided additional funding beyond the required state match of 40 percent to pay for additional staffing costs or facilities. According to our survey of all Challenge Programs, 28 out of 29 programs identified receiving some type of funding support beyond the required federal and state contributions to pay for program expenditures incurred during 1999 through 2004. These programs identified receiving additional funding from the states’ general fund beyond the required state match as well as support from other state agencies such as the state Department of Education. Additionally, the programs we surveyed relied on funding assistance from other federal agencies such as the Department of Agriculture, which provides funding under the National School Lunch Program; nonprofit organization grants; the programs’ private 501 (c) (3) corporation; and donations from private individuals.

Results from our site visits and national survey also showed that state Challenge Programs rely on donations of goods and services to support the program. For example, states we visited relied on donations from private citizens, corporations, schools, and the states’ National Guard units for such items as computers and software, exercise equipment, books, uniforms, and shoes. Some programs also received support from the surrounding community in the form of donated services such as transportation assistance and medical services from local doctors and nurses. Our survey further showed that 21 out of 29 programs received some form of donated goods and services. Program officials we met stated that donations of goods and services were vital to programs because many times they do not have funds available to acquire equipment or services for the program. Donations of goods and services can enhance the basic program beyond the military-based training and academics required under the cooperative agreements adding little or no cost to the program.

Despite the success that many states had in obtaining additional funding support and donations of goods and services for their programs, some states reported obstacles in securing supplemental funding. For example, the cooperative agreements governing the Challenge Program do not allow states to use Challenge Program funds to hire full-time grant writers. Lastly, some programs we visited expressed concern that if they received funding support outside the federal and state required contributions, then their state governments might reduce their allocations.

Conclusions

Although Reserve Affairs, NGB, and participating states have suggested the current cost basis of $14,000 per student is not sufficient to sustain the program and that the cost-share ratio should be changed, Reserve Affairs and NGB have done little analysis to show what the actual costs of the program are and how changing the cost-share ratio for the program would impact participating states. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost-share ratio.

Until NGB establishes clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance. Without these performance goals in place, NGB does not have a firm basis for evaluating program outcomes and DOD’s return on investment.

Although property and fiscal officers are required to conduct full audits of the Challenge Program at least every 3 years, these audits have not been conducted due to competing priorities at the state level and a lack of staff. Without these audits, it may be difficult to ensure that federal interests are adequately protected. Also, because NGB does not receive copies of audits conducted by property and fiscal officers, it cannot know what the findings of these audits were and if changes were made to the programs based on these findings.

Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they can provide funds for the Challenge Program. Without a more formal process that outlines the amount of funding needed and a detailed description of the Challenge Program and its specific funding needs, Reserve Affairs is potentially missing out on additional sources of funding that could enhance the program.

Recommendations for Executive Action

To improve the management and oversight of the National Guard Youth Challenge Program, we recommend that the Secretary of Defense, direct the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau, to take the following three actions: Determine the actual costs of the Challenge Program, including states’ ability to fund their share of the program, and use this information, as appropriate, to support funding requests or a request to change the cost- share ratio.

Establish performance goals to measure the effectiveness of the Challenge Program.

Direct U.S. property and fiscal officers to conduct audits as required and require that copies of audit results are provided to the appropriate office at the National Guard Bureau in order to ensure that the results of audits are promptly reviewed and resolved.

To strengthen efforts at obtaining alternative funding in support of the National Guard Youth Challenge Program, we recommend that the Secretary of Defense direct the Undersecretary of Defense for Personnel and Readiness, in consultation with the Assistant Secretary of Defense for Reserve Affairs, to develop more formal strategies for requesting alternative funding support for the Challenge Program. Such strategies may include submitting requests for funding that include the amount of funding requested and a sufficiently detailed description of the proposed program to allow potential providers of funds, such as nondefense agencies, to determine whether it would be an appropriate use of their funds.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with our recommendations. In its overall comments, DOD asserts that our report reaffirms that the program met its congressional mandate to improve the life skills and employment potential of participants by providing military- based training and supervised work experience. Further, DOD’s comments state that our report validates the core program components of assisting participants to receive a high school diploma or its equivalent; developing leadership skills; promoting fellowship and community service; developing life coping and job skills; and improving physical fitness, health and hygiene. DOD also states that it appreciates GAO’s confirmation that it is properly executing the program.

We do not agree with DOD’s characterization of our report. Whereas our report provides background data on program performance and mentions that DOD has reported positive outcomes from the Challenge Program, we do not assess whether or not DOD met its congressional mandate; we do not validate the program’s core components, nor do we confirm that DOD is properly executing the program, as DOD’s comments suggest. Rather our objectives were to examine: (1) historical trends of the National Guard Youth Challenge Program, including program expenditures, participation, and performance; (2) the extent to which Reserve Affairs and the NGB have determined actual program costs and the need to adjust the federal and state cost-share; and (3) the extent to which the NGB has provided oversight of the program. We also determined the extent to which Reserve Affairs and participating states have made an effort to obtain alternative funding support for the Challenge Program.

In response to our recommendation to determine the actual costs of the Challenge Program, including the ability of states to fund their share, DOD concurred, but claims that the matching fund requirement makes the program vulnerable to state budget cuts. DOD also contends that until recently, sluggish state revenues made the cost share requirement more burdensome and inhibited the ability of both the NGB and the states to focus on determining the actual costs of the Challenge Program. DOD claims that the budget shortfalls required the states to reduce and/or eliminate program services and that, with state revenues slowly closing their budget gaps, restoring funding for needed program services is being considered. DOD also notes that it is currently working on a new funding formula for the program.

Although DOD’s comments suggest that adequate funding was not available for needed services, through our survey and other work, we found that all 29 programs were able to provide the services required by the cooperative agreements, and several states reported they added program enhancements such as field trips or vocational classes. However, some states reported that they reduced nonrequired services to stay within their budgets. Furthermore, the basis for DOD’s comment regarding the effect of the matching fund requirement and the level of state revenues on the program is unclear. As we reported, neither Reserve Affairs nor NGB has compiled or analyzed data on actual program costs, states’ financial situations, or the impact of adjusting the federal and state cost share. We continue to believe that any future funding request, or a request to change the cost share requirement, should be based on an analysis of the actual costs of the program.

DOD’s comments are printed in their entirety in appendix III.

We are sending copies of this report to the Secretary of Defense and interested congressional committees. We will also make copies available to others upon request. This report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions regarding this report, please contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To examine historical trends of the National Guard Youth Challenge Program, including program expenditures, participation, and performance, we interviewed officials from the Office of the Assistant Secretary of Defense for Reserve Affairs; the National Guard Bureau (NGB); and the contractor that monitors and evaluates state programs, AOC Solutions. We reviewed other documentation provided by Reserve Affairs and NGB, such as program funding summaries. We also reviewed the Challenge Program’s annual reports submitted to Congress from 1994 through the present, excluding 1997, which provided us background information on the program. For data on participation and performance outcomes, including general education development credential attainment, community service hours, and post-residential placements, we relied on the Data Management and Reporting System (DMARS) that is used by NGB to collect participant information and track individual and program activities. DMARS was implemented for the 2003 reporting year, and the contractor has performed procedures to clean up the data as far back as 2000. We conducted data reliability tests on the program’s annual reports and data management system to conclude that the data are sufficiently reliable for our purposes. We did not compare the costs or outcomes of this program to other similar youth programs currently funded by the federal government since that was not within the scope of this engagement.

To assess the extent to which Reserve Affairs and NGB determined actual program costs and analyzed states’ ability to fund their share of the program, we interviewed officials at Reserve Affairs and NGB as well as officials at state programs. We reviewed and analyzed internal budget documents prepared by state programs and submitted to NGB annually as well as documents prepared by individual program directors that discussed the funding situation in their state. We also reviewed and analyzed good budget practices as described in federal financial accounting standards. Using a semi-structured questionnaire, we interviewed eight state Challenge Program directors and their budget officers in seven states and discussed sources of funding for their programs, actual costs of operating their programs, and the types of services that are provided to program participants given the available funding. States selected for site visits or telephone interviews are shown in table 2. There were a number of factors which affected the judgmental sampling of these state programs. States were chosen because of their length of time in the program, size, geographic location, experience of the program’s director, and whether or not there were multiple programs in the state. Additionally, in July 2005, we surveyed all 29 participating Challenge Programs and asked them about the actual costs of their programs and their assessment of their states’ ability to fund the programs. This survey had a response rate of 100 percent.

To assess the extent to which NGB has provided oversight of the Challenge Program, we interviewed officials at Reserve Affairs and NGB as well as program officials at the state level. To determine the oversight responsibilities of federal and state officials involved in the program, we reviewed and analyzed pertinent laws, the Department of Defense’s (DOD) Instruction governing the program, and the NGB Master Cooperative Agreement, which defines the terms and conditions of the program in each state. We also interviewed the independent contractor hired by NGB to conduct yearly and biennial on-site program reviews, AOC Solutions. We obtained access to its data management system and analyzed the program evaluations and resource management reviews it completed on behalf of NGB since 2003. We also reviewed and analyzed individual state program’s annual plans and NGB’s strategic plan to identify the types of goals set by the Challenge program. We consulted previous GAO work regarding performance measurement and evaluations, identified best practices for establishing and measuring performance goals, and reviewed Standards for Internal Control in the Federal Government and the Government Performance and Results Act of 1993. We interviewed representatives of the U.S. Property and Fiscal Officer’s Office in six states as well as the NGB Chief of Property and Fiscal Affairs to determine the time frame for completing audits and mechanisms for reporting, tracking, and resolving issues that arise out of the PFO audits.

To determine the extent to which Reserve Affairs and participating states have obtained alternative funding support for the program, we reviewed relevant laws, policies, and reports to determine the relevant authorities for receiving and transferring funds between federal agencies. We also interviewed federal officials at Reserve Affairs and the Departments of Labor, Education, and Justice to determine the extent to which these agencies discussed the possibility of transferring funds to DOD in support of the Challenge program. To determine states’ efforts at obtaining alternative funding support for their program’s operations, we conducted a survey of all 29 participating programs to collect information about their sources of funding for the program, how funding is distributed across different program operational functions, the sources and types of donated goods and services, and descriptions of strategies programs use to obtain alternative funding support. Using a semi-structured questionnaire, we interviewed eight state Challenge Program directors and members of their staff in seven states and discussed sources of funding for their programs, the strategies these specific programs use for obtaining alternative funding support and donations, and the difficulties these states face in seeking out other funding.

We provided a draft of this report to officials at DOD for their review and incorporated their comments where appropriate. We conducted our work from January 2005 to October 2005 in accordance with generally accepted government auditing standards.

Appendix II: National Guard Youth Challenge Program Sites

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgment

Acknowledgments

In addition to the individual named above, Harold Reich, Karyn Angulo, Leslie Bharadwaja, Susan Ditto, K. Nicole Harms, Wilfred Holloway, Jessica Kaczmarek, Stanley Kostyla, Julia Matta, Renee McElveen, and John Van Schaik made key contributions to this report.