K-12 EDUCATION

Education Needs to 

Improve Oversight of 

Its 21st Century 

Program 

Report to Congressional Committees

April 2017

GAO-17-400

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-400, a report to 

congressional committees

April 2017

K-12 EDUCATION

Education Needs to Improve Oversight of Its 21st

Century Program 

What GAO Found 

The Department of Education (Education) awards 21st Century Community 

Learning Centers (21st Century) grants to states, who in turn, competitively 

award funds to local organizations, which use them to offer academic enrichment 

and other activities to improve students’ academic and behavioral outcomes. In 

their most recent grant competitions, states awarded 21st Century funds to 

nearly 2,400 organizations—including school districts and community-based

organizations—based on a variety of criteria, such as the quality of their 

proposed program designs. Relevant research we reviewed that compared 

program participants to those of non-participants suggests that the 21st Century 

program is effective in improving students’ behavioral outcomes, such as schoolday attendance and reduced disciplinary incidents, more often than their 

academic outcomes. However, because Education’s current 21st Century 

performance measures primarily focus on students’ reading and math scores on 

state tests, Education lacks useful data about whether the program is achieving 

its objectives to improve students’ behavioral outcomes such as attendance and 

discipline—the areas where the program most frequently has a positive effect. 

Education officials have not substantially revised the program’s performance 

measures since 1998, in part because its authorization lapsed from fiscal years 

2008 through 2016. Leading practices in performance measurement call for 

federal agencies to align performance measures with program objectives. 

21st Century Community Learning Centers’ Objectives and Performance Measures for 

Student Outcomes 

Education’s technical assistance to states does not adequately address 

challenges states face in evaluating their 21st Century programs and sustaining 

them when program funding ends. About a third of states reported in GAO’s 50-

state survey that they face challenges in evaluating program performance, such 

as difficulty designing evaluations that shed light on program effects. Further, the 

21st Century program was reauthorized twice, which resulted in significant 

changes to state requirements for evaluating programs. However, Education has

not provided states written guidance on developing and conducting high-quality 

evaluations since 1999. Federal standards for internal control state that when 

significant changes occur agencies should periodically review policies for 

continued relevance and effectiveness in achieving their objectives. Absent 

written guidance to states on conducting high-quality evaluations, Education may 

miss opportunities to help states improve their capacity to conduct such 

evaluations to assure the program is meeting its goals.

View GAO-17-400. For more information, 

contact Jacqueline M. Nowicki at 

(617) 788-0580 or nowickij@gao.gov.

Why GAO Did This Study

Education’s 21st Century program—

funded about $1 billion annually

since 2002—supports a broad array 

of activities outside the school day to 

improve student outcomes in highpoverty or low-performing K-12 

schools. A statement accompanying 

the Consolidated and Further 

Continuing Appropriations Act of 

2015 included a provision for GAO to 

review Education programs outside 

the regular school day.

GAO examined (1) how 21st Century 

funds are awarded and used, (2) 

what is known about the

effectiveness of these programs, (3) 

how Education manages and uses

program data to inform decision

making, and (4) Education's 

technical assistance for evaluating 

and sustaining programs. GAO 

conducted a 50-state survey of 

program officials, obtaining a 100 

percent response rate. GAO also

reviewed selected state program 

evaluations and academic studies on

student outcomes, and observed 

program activities and interviewed 

officials in four states representing a 

range of grant size and location. 

What GAO Recommends

GAO is making four 

recommendations, including that 

Education expand its performance 

measures for behavioral outcomes 

and provide written guidance to 

states on conducting high-quality

program evaluations. Education 

neither agreed nor disagreed with 

the recommendations, and outlined 

steps it is taking to address them.

 
Page i GAO-17-400 K-12 Education

Letter 1

Background 4

States Use a Variety of Criteria to Award Sub-grants for 21st 

Century Program Centers, Which Use Funds to Provide a 

Broad Array of Services 8

Research Suggests 21st Century Programs Improve Students’ 

Behavioral Outcomes More Often Than Academic Outcomes 15

Education Has Not Taken Important Steps to Ensure 21st Century 

Data Are Useful for Program Decision Making 20

Education’s Technical Assistance Does Not Yet Adequately 

Address Many States’ Challenges in Evaluating and Sustaining 

Programs 25

Conclusions 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 31

Appendix I Scope and Methodology 35

Appendix II 21st Century Grant Award Amount by States, Fiscal Year 2016 40

Appendix III Comments from the U.S. Department of Education 42

Appendix IV GAO Contact and Staff Acknowledgments 46

Table

Table 1: 21st Century Grant Award Amount by States, 

Fiscal Year 2016 40

Figures

Figure 1: Overview of Objectives and Activities of the 21st Century 

Community Learning Center’s Programs 5

Figure 2: Overview of the 21st Century Community Learning 

Centers Grant Process 6

Contents
Page ii GAO-17-400 K-12 Education

Figure 3: Number of States That Reported Providing Additional 

Points or Consideration to 21st Century Community 

Learning Centers Grant Applicants, by Selected 

Categories 9

Figure 4: Examples of Activities Provided by 21st Century 

Community Learning Centers’ Programs 12

Figure 5: Most Frequent Funding Sources Used to Supplement 

21st Century Community Learning Centers Grants 14

Figure 6: 21st Century Community Learning Centers Objectives 

and Performance Measures for Student Outcomes, 

Established by the U.S. Department of Education 21

Abbreviations

21st Century Community Learning Centers 21st Century

Department of Education Education

Elementary and Secondary Education Act of 1964 ESEA

Every Student Succeeds Act ESSA

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-400 K-12 Education

441 G St. N.W.

Washington, DC 20548

April 26, 2017

The Honorable Roy Blunt

Chairman

The Honorable Patty Murray

Ranking Member

Subcommittee on Labor, Health and Human Services,

Education, and Related Agencies

Committee on Appropriations

United States Senate

The Honorable Tom Cole

Chairman

The Honorable Rosa L. DeLauro

Ranking Member

Subcommittee on Labor, Health and Human Services,

Education, and Related Agencies

Committee on Appropriations

House of Representatives

The 21st Century Community Learning Centers (21st Century) program is 

meant to support local communities in providing learning opportunities for 

K-12 students outside the regular school day, such as before school, after 

school, and during the summer.1 The program is designed to target 

services primarily to students who are from low-income families or who 

attend schools in need of academic assistance. From fiscal years 2002 to 

2016, Congress appropriated a total of about $16 billion to the 

Department of Education (Education) to administer 21st Century funds.2

In 2015, the program was reauthorized by the Every Student Succeeds 

 1

For the purposes of this report, we use the phrase “activities outside the regular school 

day” to refer to activities “during non-school hours or periods when school is not in 

session.”

2

The program was originally authorized in 1994 and reauthorized in 2002. Pub. L. No. 

103-382, § 101, 108 Stat. 3518, 3844 (1994), Pub. L. No. 107-110, § 401, 115 Stat. 1425, 

1765 (2002). Funding authorization for the 21st Century program ended in fiscal year 

2007. For fiscal years 2008 through 2016, funding for the program was provided through 

annual appropriations.

Letter
Page 2 GAO-17-400 K-12 Education

Act (ESSA).3 According to Education, in school year 2013-2014 (the most 

recent data available at the time we did our work) the program provided 

funding for about 9,500 centers serving approximately 1.8 million students 

and 430,000 adult family members.

A statement accompanying the Consolidated and Further Continuing 

Appropriations Act of 2015 included a provision for GAO to review K-12 

education programs that extend learning time beyond the regular school 

day.4 This report examines (1) how 21st Century funds are awarded and 

used, (2) what is known about the effectiveness of 21st Century 

programs, (3) the extent to which Education has effectively managed and 

used program data to inform decisions about the 21st Century program, 

and (4) the extent to which Education’s technical assistance has helped 

states assess and sustain high-quality programs once 21st Century 

funding ends.

We focused our review on 21st Century afterschool programs because 

centers have historically provided services more during this time as 

compared to before school and during the summer. To answer all four 

research questions, we reviewed Education documents, including annual 

performance reports, studies, and budget justifications; and federal laws, 

regulations, and policies. We also interviewed Education officials and 

other researchers on the 21st Century program. To assess the reliability 

of federal 21st Century program data underlying Education’s annual 

performance reports and studies, we conducted electronic data tests, 

 3

Pub. L. No. 114-95, § 4201, 129 Stat. 1802 (2015). ESSA also allows states to use 21st

Century funds to support academic activities that are included as part of a program that 

provides additional program hours during the regular school day. Education stated that 

this provision will be implemented starting in school year 2017-2018. Prior to fiscal year

2017, authorized activities must be conducted during non-school hours or periods when 

school is not in session (i.e., before school, after school, or during summer break) and 

may not be used to extend the school day. For a limited period of time prior to enactment 

of ESSA, Education approved waivers for some states to exercise flexibility to permit their 

sub-grantees to conduct authorized 21st Century program activities during the school day 

if there was an extended learning program in place. Throughout this report, any 

references to statutory requirements of the 21st Century program are to those that existed 

prior to the program’s reauthorization by ESSA, as these were the requirements in effect 

at the time of our review.

4

See 160 Cong. Rec. H9307, H9841 (daily ed. Dec. 11, 2014) (explanatory statement), 

accompanying Pub. L. No. 113-235, 128 Stat. 2130 (2014). Under the same mandate, 

GAO issued a companion report examining what is known about Education programs that 

states and school districts can use to support extended learning time. GAO, K-12 

Education: Federal Funding for and Characteristics of Public Schools with Extended 

Learning Time, GAO-16-141 (Washington, D.C.: Nov. 30, 2015).
Page 3 GAO-17-400 K-12 Education

reviewed relevant documents, and interviewed agency officials. We 

determined that the data were sufficiently reliable for our purposes. We 

also conducted a Web-based survey of 21st Century state coordinators 

from the 50 states and the District of Columbia and received responses 

from 100 percent of the coordinators.5 We analyzed responses to survey 

questions as well as additional comments state coordinators provided in 

response to open-ended questions. In addition, we interviewed state 

officials responsible for administering the 21st Century program in four 

states: Idaho, Massachusetts, Rhode Island, and Texas; and we 

observed activities and interviewed staff at two to four 21st Century 

centers in each of those states. States were selected to represent a range 

in the amount of 21st Century grant funding, student demographics such 

as race and ethnicity, and geographic location. Information collected 

about these four states is not generalizable but provides insight into 21st 

Century program operations at the local level.

Additionally, to examine what is known about the effectiveness of 21st 

Century programs, we reviewed select state-level program evaluations 

and other studies that reported outcomes for students who participated in 

the program. To identify relevant state-level evaluations, we took a twopronged approach. First, we reviewed state evaluations that Education 

determined in a 2012 report on 21st Century program state evaluation 

practices (the most recent available Education report on this topic) to 

have used quasi-experimental methods or statistical controls, such as a 

comparison group, to account for other plausible influences on the 

outcomes reported.6 We then interviewed researchers to identify other 

quasi-experimental state evaluations published since 2012, and we 

reviewed them for the soundness of their methodology and data. 

Ultimately, we determined that four state evaluations were appropriate to 

address our research objective about the 21st Century program’s 

effectiveness. Second, we conducted a comprehensive literature review 

of 104 other studies of afterschool programs, including the 21st Century 

program. We ultimately determined that six studies were sufficiently 

rigorous and appropriately scoped to include in our review, including three 

meta-analyses that synthesized a broad array of studies on the outcomes 

 5

Throughout the report, we use “states” to refer to the 50 states and the District of 

Columbia.

6

U.S. Department of Education, Office of Elementary and Secondary Education, Review 

of 21st Century Community Learning Centers’ Grantee Evaluation Practices Final Report, 

(Washington, D.C.: Sept. 24, 2012).
Page 4 GAO-17-400 K-12 Education

of afterschool programs in general, including the 21st Century program.7

Additional information on our research objectives, scope, and 

methodology is available in appendix I.

We conducted this performance audit from April 2016 to April 2017 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

The 21st Century program is authorized to provide a wide range of 

activities for K-12 students and their families to:

1. provide opportunities for academic enrichment, including providing 

tutorial services to help students—particularly students who attend 

low-performing schools—meet state and local academic standards in 

core subjects such as reading and mathematics;

2. offer students a broad array of additional services, such as drug and 

violence prevention, counseling, art, music, recreation, and 

technology programs that are designed to reinforce and complement 

instruction in the regular school day; and

3. offer literacy and related educational development opportunities to 

families of students served.8

Education has established performance objectives for the 21st Century 

program, including objectives related to student outcomes. These 

objectives state that 21st Century participants will demonstrate 

educational and social skills and exhibit positive behavioral changes; and 

they will improve in outcomes such as academic performance, school 

attendance, and rates of disciplinary incidents and other adverse 

behaviors (see fig. 1).

 7

A meta-analysis is a statistical technique that combines findings from independent 

studies to develop a conclusion with a statistical power stronger than the analysis of any 

single study. 

8

20 U.S.C. § 7171(a).

Background
Page 5 GAO-17-400 K-12 Education

Figure 1: Overview of Objectives and Activities of the 21st Century Community Learning Centers Programs

The 21st Century program is administered by state educational agencies. 

Education provides state agencies with annual formula grants, which in 

2015 ranged from about $6 million to $132 million. The formula that 

determines the funding amount for a particular state is based in part on 

the percentage of total students from low-income families enrolled in K-12 

public schools and how much that state spends per pupil on education.9

States, in turn, competitively award funds to sub-grantees, which may be 

school districts or community-based organizations, such as those that 

focus on youth development. By law, states must award sub-grants of a 

minimum $50,000 per year for periods of 3 to 5 years.10

 9

20 U.S.C. § 7172(b)(1).

1020 U.S.C. § 7174(g), (h).

21st Century Grant Process
Page 6 GAO-17-400 K-12 Education

Sub-grantees oversee one or more physical locations, referred to as 

“centers,” where grant-funded services are provided to participating 

students and adult family members.11 Centers may be located in schools, 

churches, community centers’ or other spaces (see fig. 2). They must

provide services during non-school hours or periods when school is not in 

session, such as before school, after school, on weekends, or during 

summer vacations and school breaks.

Figure 2: Overview of the 21st Century Community Learning Centers Grant Process

In making awards, states must give priority both to applications that 

propose to serve students who attend schools identified as needing 

improvement and that are submitted jointly by at least one school district 

receiving funds under Title I, Part A of the Elementary and Secondary 

Education Act of 1964 (ESEA), as amended, and at least one communitybased organization or other public or private entity.12 In addition, states 

are authorized to include additional priorities in their sub-grant 

competitions so long as they are aligned with the statute’s requirements 

and priorities.

Education’s Office of Academic Improvement, under the Office of 

Elementary and Secondary Education, is responsible for overseeing 

states’ implementation of the 21st Century program. The Office of 

Academic Improvement is also responsible for providing ongoing 

 11For the purposes of our report, we refer to “community learning centers” as “centers.”

1220 U.S.C. § 7174(i). Title I, Part A (Title I) of ESEA provides formula grants to states for 

their school districts to improve educational programs in schools with high concentrations 

of students from low-income families. School districts that are unable to partner with 

qualified community organizations within reasonable geographic proximity are still entitled 

to receive priority under the law.

Program Monitoring and 

Evaluation
Page 7 GAO-17-400 K-12 Education

technical assistance to states and monitoring state performance through 

on-site and desk monitoring visits and ongoing communications with state 

program officials.13 The Office of Academic Improvement conducts on-site 

monitoring for the 21st Century program for each state grantee every 3 

years, or more frequently if needed.

The Office of Academic Improvement is also responsible for reviewing 

states’ grant applications for, among other things, compliance, including 

reviewing certain assurances that states are required to provide under 

ESEA. For example, Education must review states’ assurances that they 

will grant awards only to eligible entities that propose to serve students 

who primarily attend schools with at least 40 percent of students from 

low-income families.14 In addition, states must collect plans from 

applicants describing how activities funded by the sub-grant will continue 

after 21st Century funding ends, as well as plans for how centers will 

address participants’ transportation needs, among other things.15

Education collects program data from states in order to report annually on 

whether 21st Century programs have met performance targets for their 

objectives. These data include information on sub-grantee characteristics, 

center activities, and participants’ demographics and outcomes. States 

monitor sub-grantee performance and program data submission to 

Education.

Under ESEA, states are required to conduct ongoing program evaluations 

to assess the effectiveness of their 21st Century programs and activities 

and disseminate the findings.16 Education reviews state evaluations as a 

part of its monitoring process.

 13Education may reserve up to 1 percent of the 21st Century program’s annual 

appropriation to provide technical assistance to entities receiving sub-grants or to conduct 

a national program evaluation. 20 U.S.C. § 7172(a)(2).

1420 U.S.C. § 7173(a)(3)(A). 

1520 U.S.C. § 7173(a)(8)(B), (9).

1620 U.S.C. § 7173(a)(13).
Page 8 GAO-17-400 K-12 Education

Together states received more than 4,000 21st Century sub-grant 

applications and funded nearly 60 percent of them (about 2,400) in their 

most recent sub-grant competitions, according to our survey of all 50 

states and the District of Columbia.17 States’ criteria for these 

competitions vary, and most states score applications on a point system 

based on a variety of criteria. To help determine whether applicants have 

the capacity to implement high-quality programs, over half of states award 

applicants additional points based on the quality of their program design 

(33 states) and use of evidence-based practices (28 states). To sustain 

these programs once grant funding ends, about half of the states reported 

awarding additional points based on the applicants’ level of support from 

schools and school districts (25 states) and other external organizations 

(23 states). (See fig. 3.)

 17States vary in how recently they held their sub-grant competitions. In our survey, 33 

states reported that they held a sub-grant competition in school year 2015-2016. Eleven 

states reported that their most recent competition was in school year 2014-2015, and the 

remaining 6 states had not held a competition since school year 2013-2014 or earlier. One 

state did not respond to this question. We calculated the percentage of applicants funded 

by dividing the total number accepted by the total number of applicants. One state did not 

report data needed to calculate the percentage of applicants funded.

States Use a Variety 

of Criteria to Award 

Sub-grants for 21st 

Century Program 

Centers, Which Use 

Funds to Provide a 

Broad Array of 

Services

States Use Varied Criteria, 

Funding Periods, and 

Amounts for Sub-grants to 

21st Century Program 

Providers
Page 9 GAO-17-400 K-12 Education

Figure 3: Number of States That Reported Providing Additional Points or Consideration to 21st Century Community Learning 

Centers Applicants, by Selected Categories

Additionally, 10 states reported in our survey that they provided additional 

points to applicants serving schools identified as the lowest performing or 

with the largest gaps in student achievement based on comparisons by 

race or socio-economic levels.

To provide adequate time to implement successful programs, a majority 

of states reported that they offer sub-grantees 21st Century funding for 5 

years, the maximum number of years allowed under law.18 Specifically, 

39 states said in our survey that they provide 21st Century funding to subgrantees for 5 years. Education’s 2003 program guidance states that 

research suggests it takes approximately 5 years of continual revision 

and improvement for a community to fully implement a successful 21st 

 1820 U.S.C. § 7174(g).
Page 10 GAO-17-400 K-12 Education

Century program.19 Further, existing 21st Century sub-grantees are also 

eligible to re-apply for funding outside the competition period if their grant 

is close to expiring. Of the applicants that received an award in the most 

recent competition, about 45 percent (nearly 1,100) had previously 

received a 21st Century grant award.20

States have provided different levels of funding for sub-grantees and their 

centers. Fifteen states reported the minimum and maximum amounts 

sub-grantees allocated to their centers. Specifically, the amount that a 

sub-grantee’s center received ranged from a minimum of $100,000 to a 

maximum of about $660,000, with a median amount of $185,000 per 

center in school year 2015-2016. Officials in Idaho told us the amounts 

vary because each center has different needs and student numbers. They 

said their program’s average cost per student was $1,700 per year. They 

noted that they encourage centers to target students most in need of 

services instead of serving all students in a school, even if the school 

primarily serves students from low-income families. Officials in Texas told 

us that funding amounts to centers can vary based on other state 

requirements, such as a requirement to employ a full-time program 

director or coordinator at the sub-grantee or center level.

Sub-grantees have used 21st Century funds to provide a broad array of 

services to K-12 students. Education’s 2014 annual report on the 

characteristics of 21st Century programs found that the most commonly 

provided activities in school year 2012-2013 were those to enrich 

academics, provide tutoring and homework help, and offer recreational 

activities.21 According to Education’s 2003 guidance, academic 

enrichment can include tutoring in core academic subjects and providing 

extra learning opportunities so that students can practice their academic 

 19U.S. Department of Education, Office of Elementary and Secondary Education, 21st

Century Community Learning Centers Non-Regulatory Guidance (Washington, D.C.: 

February 2003).

20We calculated the percentage of applicants that had previously received a 21st Century 

grant by dividing total number accepted who had previously received an award by the total 

number of applicants who were accepted. Five states did not report data for the total 

number of applicants. 

21Learning Point Associates and Diehl Evaluation and Consulting Services, Inc., An 

Overview of the 21st Community Learning Centers Characteristic Data: 2012-2013

(Naperville, IL: Learning Point Associates, May 2014). This report uses 21st Century 

program data from school year 2012-2013. 

Sub-grantees Use 21st 

Century Funds to Provide 

Academic Enrichment and 

Recreational Activities
Page 11 GAO-17-400 K-12 Education

skills through hands-on activities. The 2014 annual report also found that 

many centers supported building students’ academic skills in reading and 

math as well as in the arts, science, technology, cultural activities, and 

health. Further, at least two states gathered information on the time spent 

on specific activities as part of their state 21st Century program 

evaluations. In New Jersey, for example, a 2015 state evaluation 

examined time spent by students on specific 21st Century activities, 

including academic improvement/remediation, tutoring, academic 

enrichment, community service, and recreation. Of these, academic 

improvement/remediation was the activity on which students spent the 

most time.22 Oregon’s 2012 state evaluation also found centers were 

most likely to offer weekly activities they categorized as enrichment, 

homework help, or recreational.23 In terms of subjects targeted by the 

program, centers were most likely to report in the state evaluation that 

their centers’ weekly activities focused on reading, math, and arts or 

music.

Officials in all 13 centers we visited in our four selected states said they 

offered a mix of academic support and enrichment activities. These 

activities generally focused on reading and math, science and technology, 

art and music, and fitness and nutrition. For example, centers we visited 

used computers and tablets to reinforce math skills or introduce new 

concepts, such as computer programming, for students who had limited 

access to technology. Officials at 7 of the 13 centers also said 21st 

Century activities can fill gaps in a school’s offerings such as providing 

music or computer instruction that schools may not have the opportunity 

to provide during the regular school day. (See fig. 4.)

 22Matthew Vinson, Samantha Sniegowski, and Feng Liu, New Jersey 21st Century 

Community Learning Centers Year 2 Report 2013-14 Program Year (Washington, D.C.: 

American Institutes for Research, November 2015).

23Deborah Moroney et al., Statewide Program Evaluation of 21st CCLC Grants: Oregon 

Department of Education (Chicago, IL: American Institutes for Research, October 2012).
Page 12 GAO-17-400 K-12 Education

Figure 4: Examples of Activities Provided by 21st Century Community Learning Centers Programs

21st Century centers serve different grade levels and use different 

staffing models. Education’s 2014 annual report and officials in the states 

we visited said that centers were primarily for elementary school students, 

but some also served middle and high school students. Further, 

Education’s report found that centers relied primarily on paid staff, but 

staff characteristics varied widely. For example, the report found that 

some centers used regular school-day teachers and non-school based 

staff, such as youth development workers or staff without college 

degrees. In the 13 centers we visited, officials told us their staff included a 

mix of certified teachers and paraprofessionals who work during the 

regular school day, as well as staff from outside the school who work in 

the program only part-time.
Page 13 GAO-17-400 K-12 Education

Education’s guidance encourages sub-grantees to identify other sources 

of related funding, and Education’s monitoring protocols ask sub-grantees 

to describe how all of these resources will be combined or coordinated to 

offer a high-quality, sustainable program.24 Eight states in our survey 

reported that they require sub-grantees to match Education funds with 

funding from other sources.25 For example, one state required subgrantees to match 30 percent of their 21st Century grant award, with at 

least 10 percent required from sources outside the participating school 

district. Even if a state does not require matching funding, 21st Century 

applicants for sub-grants must identify federal, state, and local programs 

they will combine or coordinate with on their 21st Century program to 

make the most effective use of public resources.26 In our survey, states 

reported that the two most frequent funding sources sub-grantees used to 

supplement their 21st Century funding were the local school district (23 

states) and other federal government programs (21 states), such as the 

Department of Health and Human Services’ Child Care and Development 

Fund and the Department of Agriculture’s Child and Adult Care Food 

Program.27 (See fig. 5.) Officials at several centers we interviewed also 

said they provide snacks and meals during the afterschool programs 

using Department of Agriculture funds. Ten states reported in our survey 

that sub-grantees frequently use other Education funding, such as Title I, 

Part A of ESEA, as amended, which provides financial assistance to 

districts and schools with high numbers or percentages of children from 

low-income families. In addition, officials in Idaho told us their subgrantees also use funds from Education’s Migrant Education program, 

which provides educational and support services to migrant children.

 24U.S. Department of Education, Office of Elementary and Secondary Education, 21st

Century Community Learning Centers Non-Regulatory Guidance (February 2003).

25If a state has a matching requirement, it cannot require the match to exceed the federal 

grant amount, and it must be based on a sliding scale that takes into account the relative 

poverty of the population to be targeted by the sub-grantee and the ability of the subgrantee to obtain matching funds. 20 U.S.C. § 7174(d)(1), (2).

2620 U.S.C. § 7174(b)(2)(C).

27The Child Care and Development Fund, administered by the Department of Health and 

Human Services, provides federal funding to states for child care subsidies to low-income 

parents so they can work, look for jobs, or participate in education and training activities. 

The Child and Adult Care Food Program, administered by the Department of Agriculture, 

reimburses child care providers for meals and snacks to children in child care centers and 

other locations, among other things.

Sub-grantees Leverage 

Their 21st Century 

Funding to Obtain 

Additional Funding from 

Other Sources
Page 14 GAO-17-400 K-12 Education

Officials from 46 states reported in our survey that they allow subgrantees to charge student fees. These states, however, generally had a 

policy that no child would be turned away due to a family’s inability to pay. 

Of these states, 27 reported they require sub-grantees to have a sliding 

scale for fees based on family income.

Figure 5: Most Frequent Funding Sources Used to Supplement 21st Century Community Learning Centers Grants

To encourage sub-grantees to secure funding from alternate sources, 

most states reported that they gradually reduce 21st Century sub-grants 

over the funding cycle. Specifically, officials in 37 states reported in our 

survey that they reduce funding for sub-grantees after an initial period. 

For example, one state reported that it awards full funding in the first 2 

years, but then reduces funding to 80 percent of that amount in the third 

year, 60 percent in the fourth year, and 40 percent in the fifth year. 

Further, 15 states reported that they reduce funding levels for sub-
Page 15 GAO-17-400 K-12 Education

grantees that have received 21st Century grants in previous funding 

cycles (“repeat sub-grantees”). Officials in Texas told us they fund repeat 

sub-grantees for 3 years rather than the usual 5 to preserve funds for new 

recipients. In Massachusetts, repeat sub-grantees can only apply for 50 

to 75 percent of their previous grant amount.

Few evaluations of the 21st Century program use methodologies that are 

appropriate for determining the effect of program participation on student 

outcomes. Of the 10 studies we included in our review of program 

effectiveness—4 state evaluations and 6 other studies—that do use such 

methodologies, 3 state evaluations found a positive relationship between 

program participation and school-day attendance and/or discipline.28 (See 

appendix I for more details on our process for selecting studies.) These 

state evaluations—from New Jersey, Texas, and Washington—examined 

behavioral outcomes such as school-day attendance and discipline. All 

three of these evaluations found there was a positive effect on school-day 

attendance, and two of them found there was also a positive effect on 

school-day discipline. For example, Washington’s state evaluation found 

that school-day attendance improved for students in grades 6 through 12 

who participated in the program when compared to similar students who 

had not participated.29 Similarly, Texas’s state evaluation found 

participating students in grades 4 through 11 had improved school-day 

attendance. This effect was particularly strong for students who 

 28Three of the six studies were meta-analyses of afterschool programs in general, 

including 21st Century programs.

29Neil Naftzger, et al., Washington 21st Century Community Learning Centers Program 

Evaluation: 2012-13 and 2013-14 (Chicago, IL: American Institutes for Research, October 

2015).

Research Suggests 

21st Century 

Programs Improve 

Students’ Behavioral 

Outcomes More 

Often Than Academic 

Outcomes

Research Indicates 

Improvements in Some 

Behavioral Outcomes for 

Participating Students
Page 16 GAO-17-400 K-12 Education

participated in the program 60 days or more, especially in grades 6 

through 11.30 These students had an absentee rate that was more than 

20 percent lower than non-participants. Additionally, Texas’s evaluation 

found that students in a statewide 21st Century pilot program with 

intensive academics showed improved school-day attendance.31 In New 

Jersey’s evaluation of students in grades 4 through 8, students in all 

grades had improved attendance; this improvement was generally larger 

in the higher grades, particularly 8th grade.32 Further, one of the three 

meta-analyses of afterschool programs in general also identified similar 

positive effects on school-day attendance.33 This meta-analysis, 

published by Education in 2014, synthesized the results of 30 studies, 

including 6 studies of 21st Century programs.34 It identified positive 

effects on students’ academic motivation, a broad category of outcomes 

which included measures such as school-day attendance and homework 

completion.

Two of the four state evaluations we included in our review also examined 

school-day disciplinary incidents—which include fighting, bullying, or 

disruptive conduct that result in a student’s removal from the classroom—

and both found that 21st Century program participation may improve 

outcomes, particularly for students who participated in the program for 60

days or more.35 Specifically, Washington’s state evaluation found an 

 30Elizabeth Devaney, et al., Texas 21st Century Community Learning Centers 2014-15 

Evaluation Report. (Naperville, IL: American Institutes for Research, May 2016).

31The evaluation also assessed the pilot’s effect on participating students’ academic 

outcomes and found no effect. 

32Vinson et al. (2015)

33Yael Kidron and Jim Linsday, The Effects of Increased Learning Time on Student 

Academic and Nonacademic Outcomes: Findings from a Meta-analytic Review

(Washington, D.C.: U.S. Department of Education, Institute of Education Sciences, 

National Center for Education Evaluation and Regional Assistance, Regional Educational 

Laboratory Appalachia, 2014).

34Another meta-analysis published in 2014—funded in part through grants from Education 

and the Department of Health and Human Services—examined afterschool programs that 

served at-risk students and found no observable effect on school-day attendance. The 

third meta-analysis of afterschool programs that we reviewed did not examine school-day 

attendance. Kirstin P. Kremer, et al., “Effects of After-School Programs with At-risk Youth 

on Attendance and Externalizing Behaviors: A Systematic review and Meta-analysis.” 

Journal of Youth and Adolescence, vol. 44, no.3 (2014). Sema A. Taheri and Brandon C. 

Welsh, “After-School Programs for Delinquency Prevention: A Systematic Review and 

Meta-analysis,” Youth Violence and Juvenile Justice, vol. 14, no. 3 (2016).

35The other two state evaluations that we reviewed did not examine school day discipline. 
Page 17 GAO-17-400 K-12 Education

association between program participation and fewer disciplinary 

incidents for students in grades 3 through 12 who attended 60 days or 

more.36 In Texas, student participation in the program for 60 days or more 

was also associated with fewer disciplinary incidents.37 Separately, 

Texas’s evaluation also found that the presence of certain program 

characteristics in the state’s 21st Century academic pilot program were

associated with lower rates of disciplinary incidents.38 In particular, the 

evaluation showed that centers in the pilot that taught students face-toface rather than via computer were associated with fewer disciplinary 

incidents. Moreover, this positive association was stronger for centers 

whose curricula focused on general learning strategies rather than on 

specific subject area skills.

Regarding the effect of the 21st Century program on reducing 

delinquency, a 2016 meta-analysis of 12 studies examining afterschool 

programs in general, including one on programs funded through the 21st 

Century program, found no significant effect.39 This meta-analysis 

identified levels of delinquency—incidents such as arrest rates and violent 

behavior—as an important measure for afterschool programs because 

many of them focus on reducing such behaviors by providing a safe, 

supervised environment where children spend less time with potentially 

delinquent peers. Nevertheless, when the researchers averaged the 

results of all afterschool programs in their review, they found no 

significant effect on delinquency rates.

Results of some studies we reviewed demonstrated a positive association 

between participation in 21st Century programs and improved academic 

outcomes for selected groups of students or for particular types of 

activities. For example, in Texas the program was associated with higher 

math scores for students who participated 60 days or more.40 However, 

none of the 10 studies in our review observed consistently better scores 

in either math or reading in program participants’ state assessments. 

(See appendix I for more details on our process for selecting studies.)

 36Naftzger et al. (2015).

37Devaney et al. (2016).

38Devaney et al. (2016).

39Taheri and Welsh (2016). 

40 Devaney et al. (2016).

Research Indicates Mixed 

Effects on Math and 

Reading Scores for 

Participating Students
Page 18 GAO-17-400 K-12 Education

The 10 studies we reviewed identified differing effects of participation in 

21st Century programs on students’ math scores. For example, 

evaluations for Virginia in grades 4 through 12 and for Washington in 

grades 4 through 8 found the program had no significant effect on math

scores.41,42 On the other hand, two other state evaluations indicated that 

21st Century programs were associated with improved math scores for 

certain grade levels. Specifically, New Jersey’s evaluation found a 

positive association between program participation and higher math 

scores, but it was not observable for students in 4th or 6th grade.43

Additionally, results from Texas’s evaluation found an association 

between program participation and increased math scores among middle 

school students.44

None of the state evaluations we reviewed showed a significant 

association between participation in a 21st Century program and 

increased reading scores. In fact, Texas’s evaluation showed lower 

reading scores among students who participated in the program 

compared to students who did not participate.45 In particular, participants 

in grades 4 and 5 had lower reading scores than non-participants. 

However, the Texas evaluation showed no observable decline in reading 

scores among students in middle school who attended 30 days or more. 

Evaluations for Washington, Virginia, and New Jersey found no significant 

relationship between program participation and increased reading scores 

for grades 4 through 8.46,47,48 Additionally, a 2014 academic study 

examined differences in program characteristics among 58 21st Century 

 41Margie King, et al., Virginia Department of Education Evaluation of 21st Century 

Community Learning Centers 2014-15. (Memphis, TN: University of Memphis, July 2016). 

42Although Washington’s evaluation found that the program had a positive effect on math 

scores in a prior year, we were unable to obtain information to assess the reliability of 

Washington’s data from that year. Naftzger et al. (2015)

43Vinson et al. (2015).

44Devaney et al. (2016).

45Devaney et al. (2016).

46Naftzger et al. (2015).

47King et al. (2015).

48 Vinson et al. (2015).

Program Effects on Math 

Scores

Program Effects on Reading 

Scores
Page 19 GAO-17-400 K-12 Education

programs in New York City and found no association with improved 

reading scores for students.49

Afterschool programs’ inconsistent effects on math and reading scores 

may be the result of these programs serving students with different 

needs, according to recent research. In particular, two studies we 

reviewed indicated that the impact of specific types of activities in 21st 

Century programs may differ depending on the students being served.

Specifically, they used statistical modeling to analyze the relationships 

between program features and the different educational needs among 

21st Century program participants. One study found that programs 

focusing solely on academic content were associated with larger 

increases in reading scores for students with limited English proficiency 

than programs that mixed academic content with other activities; while 

students overall benefited from both solely academic programs as well as 

those that mixed academic with other activities.50 The other study found 

that activities with structured interactions with adults—including 

opportunities for collaboration and meaningful verbal exchanges—were 

associated with increased reading scores for middle school students. For 

elementary students, however, there was no association between these 

activities and improved reading scores.51

 49Lisa M. DeBellis, Participant-Staff Relationship Quality and Academic Outcomes among

Elementary and Middle School Children in After-School Programs (Ph.D. dissertation, 

Fordham University, 2014).

50Carol Ward et al., “Assessing the 21st Century After-School Program and the 

Educational Gains of LEP Participants: A Contextual Approach,” Journal of Education for 

Students Placed at Risk (JESPAR), vol. 20, no. 4 (2015).

51Jacob Leos-Urbel, “What Works After School? The Relationship Between After-School 

Program Quality, Program Attendance, and Academic Outcomes,” Youth & Society, vol. 

47, no.5 (2015).

Program Effects for Subgroups 

of Students
Page 20 GAO-17-400 K-12 Education

While Education has developed performance measures to align with 

some 21st Century program objectives—primarily student academic 

outcomes—it has not aligned its measures with other program objectives 

related to key student behavioral and socio-emotional outcomes. As 

previously noted, these objectives describe a goal of program participants 

demonstrating improvement in three areas—educational, social, and 

behavioral—with outcomes such as improved academic performance and 

school attendance, and lower rates of disciplinary incidents and other 

adverse behaviors.

Education’s current performance measures were established in 1998. 

They address participating students’ English and math grades and state 

test scores as well as some behavioral outcomes, including homework 

completion, class participation, and classroom behavior. However, 

Education does not measure two other behavioral outcomes that are 

included in 21st Century program objectives: improved school-day 

attendance and a decrease in disciplinary incidents, although research 

has shown positive program effects for these two outcomes more often 

than for academic outcomes. Some states also recognize the importance 

of measuring behavioral outcomes associated with these programs, with 

about half of states (26 states) reporting in our survey that they choose to 

measure at least one of them.52 The remaining states, however, are not 

measuring either of these behavioral outcomes.

In addition, Education has not established any performance measures for 

socio-emotional outcomes, although social skills are also included in 

program objectives, and socio-emotional learning is an important 

component of 21st Century implementation across states we visited and 

 52The remaining 25 states reported they have either not established state-level 21st

Century measures or do not measure school-day attendance or disciplinary incidents. 

Education Has Not 

Taken Important 

Steps to Ensure 21st 

Century Data Are 

Useful for Program 

Decision Making

Education’s Performance 

Measures and Data Do 

Not Address Some Key 

21st Century Objectives

Alignment of Performance 

Measures with Program 

Objectives
Page 21 GAO-17-400 K-12 Education

surveyed. According to Education’s website, socio-emotional learning 

involves students’ knowledge and skills necessary to understand and 

manage emotions and establish positive relationships, among other 

things. Twenty-seven states reported in our survey that they currently 

measure or are developing measures for at least one socio-emotional 

outcome, including student relationships with adults or communication 

skills.53 Again, the remaining states are not measuring any of these socioemotional outcomes. At the centers we visited, programs offered various 

activities geared toward socio-emotional learning, including activities that 

paired students with adult mentors and developed students’ problemsolving skills. Officials in two states we visited told us that socio-emotional 

learning is a major component of their 21st Century programs’

philosophy. (See figure 6 for a comparison of 21st Century program 

objectives and Education’s program performance measures).

Figure 6: 21st Century Community Learning Centers Objectives and Performance 

Measures for Student Outcomes, Established by the U.S. Department of Education

 53These 27 states reported measuring at least one of the following four outcomes: 

students’ school engagement in learning (e.g., persistence on tasks), relationships with 

adults (e.g., showing respect and seeking assistance), communication skills, or problem 

solving. Twenty-three states reported that they have either not established state-level 21st

Century measures or do not measure any of these four outcomes. One state reported it 

has established state-level measures, but it did not respond to the question asking which 

types of measures it established.
Page 22 GAO-17-400 K-12 Education

Note: Education measures math scores on state tests only for students attending 21st Century middle 

and high school centers and reading scores only for students attending elementary school centers. 

For all remaining measures shown, Education collects data for all students and breaks them out by 

elementary and middle school/high school.

Several factors contributed to Education’s decision to maintain its current 

21st Century performance measures for over 15 years. In particular, 

Education officials told us they have not substantially revised 

performance measures since 1998 in part because the program’s 

authorization lapsed from 2008 through 2016, with ESSA providing 

funding authorization starting in fiscal year 2017. As a result, the officials 

said this created uncertainty about potential future program changes. In 

addition, they said they were reluctant to revise the measures given the 

costs to states of tracking and reporting on new measures, as well as 

concerns about maintaining the continuity of data collected over time. 

Education officials told us they may revise the program’s performance 

measures when they implement program changes pursuant to ESSA and 

that in doing so, they will consider including additional behavioral 

performance measures as well as socio-emotional measures.

Education’s lack of 21st Century performance measures for some key 

program objectives, however, has limited the usefulness of its 

performance data. Specifically, without measures for student behavioral 

and socio-emotional outcomes, Education lacks useful data on the extent 

to which the program is achieving its stated objectives, especially in areas 

where the program is likely to have positive effects on student outcomes. 

While about half of states reported they already measure at least one 

behavioral or socio-emotional outcome, absent measures for all states, 

Education will continue to report incomplete program results to Congress 

and the public. Further, leading practices in performance management 

call for agencies to use performance information as the basis for decision 

making.54 These practices also state that aligning performance measures 

 54GAO, Executive Guide: Effectively Implementing the Government Performance and 

Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996).
Page 23 GAO-17-400 K-12 Education

with program objectives can enhance the use of information for 

management decision making.55

While Education has taken steps to promote data quality, including data 

accuracy and completeness, it lacks reasonable assurance that the data 

submitted by states are accurate. Education officials told us they began 

using a new online data system in school year 2014-2015, called 21APR, 

in part to improve data quality. Among other changes, 21APR does not 

permit users to advance to the next page until the data on the current 

page are complete. Education officials said that this can help prevent 

missing data—a feature that was absent in the previous data system. In 

addition, in 2016 Education provided states with regional training 

sessions and instructional documents for data entry. However, Education 

has not independently assessed the accuracy of 21st Century data 

submitted in 21APR.

Education officials told us they require states to check their data 

submissions rather than Education conducting these checks because the 

agency does not have access to sub-grantees’ and centers’ source 

information. Education’s own Information Quality Guidelines state that 

data should be processed and edited to help ensure they are accurate.56

Performing accuracy checks would not necessarily require that Education 

have access to the source data. For example, it could perform basic logic 

checks between fields—such as verifying that the number of student 

participants in one grade is fewer than the total number of participants in 

all grades. Education officials told us they may explore additional checks 

for accuracy, but they did not provide a timeframe for doing so.

Education officials do review states’ procedures for data quality 

assurance, although some states have expressed concerns about the 

quality of data they receive from sub-grantees. Since 2015, when 

 55GAO, Managing for Results: Enhancing Agency Use of Performance Information for 

Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005). Further, 

measures should demonstrate results, address important aspects of program 

performance, and provide useful information for decision making. GAO, Pipeline Safety:

Management of the Office of Pipeline Safety’s Enforcement Program Needs Further 

Strengthening, GAO-04-801 (Washington, D.C.: July 23, 2004). GAO/GGD/AIMD-99-69. 

GAO/GGD-96-118.

56U.S. Department of Education, Information Quality Guidelines, accessed February 28, 

2017, https://www2.ed.gov/policy/gen/guid/iq/infoqualguide.pdf

Education Does Not Have 

Reasonable Assurance 

That 21st Century 

Program Data Submitted 

by States Are Accurate
Page 24 GAO-17-400 K-12 Education

Education implemented new monitoring protocols that included a review 

of states’ data quality procedures, Education officials have monitored 20 

states and have not found any states that are noncompliant. However, in 

our interviews and survey responses, officials from 10 states commented 

that they had concerns about the quality of program data they collect from 

sub-grantees. For example, states are unable to upload data to the 

21APR system, unlike in Education’s previous data system, and several 

states expressed concerns that they must enter data manually. One state 

official commented that this can increase the risk of errors.

Without independently checking accuracy, Education may be unable to 

detect and address potential data errors in the 21APR system—data that 

Education eventually needs to use to conduct analyses that inform 

management decisions, identify sub-grantees needing technical 

assistance, and contribute to annual performance reports and budget 

justifications. Without reasonable assurance that data in the 21APR 

system are of sufficient quality, Education may not be able to effectively 

use the data for such decision making and reporting going forward, 

despite the fact that the new system was created, in part, to improve data 

quality. Further, absent better information on the quality of its data, 

Education will be unable to communicate any data limitations in its reports 

to Congress and the public. Our prior work has identified leading 

practices in performance management, which state that agencies should 

disclose limitations associated with data used in reporting and indicate 

what action they plan to take to address the limitations.57

 57GAO/GGD/AIMD-99-69. GAO, Results-Oriented Government: GPRA Has Established a 

Solid Foundation for Achieving Greater Results, GAO-04-38 (Washington, D.C.: Mar. 10, 

2004).
Page 25 GAO-17-400 K-12 Education

States in our survey gave mixed reviews on the usefulness of Education’s 

technical assistance for developing and conducting state-level 

evaluations. Twenty-nine states reported that they found Education’s 

technical assistance on conducting state evaluations to be very or 

moderately useful. On the other hand, 12 states reported they found it 

only slightly or not at all useful; 8 states reported they had not received 

any technical assistance on this topic; and one state said it was unaware 

that Education provided such technical assistance.58

Further, 40 states commented in our survey that they face challenges in 

evaluating sub-grantee performance that may limit their capacity to 

conduct high-quality evaluations. For example, six states reported they 

have difficulty designing evaluations that can determine whether a 

change in student outcomes results from a student’s participation in a 

21st Century program or from other factors, such as interventions during 

the school day. This difficulty is evidenced by the fact that we identified 

very few state evaluations whose evidence standards allowed us to 

include them in our review on program effectiveness, as noted above. In 

addition, four states reported that they face challenges in defining and 

establishing performance measures for students who participate in the 

program, such as measures for behavioral and academic outcomes. For 

example, one state commented that it is difficult to measure the benefits 

of enrichment activities on academic performance.

Education also found that states are having difficulty conducting their 

required evaluations when Education reviewed them as part of its 

monitoring process. For example, in recent monitoring findings, Education 

officials said they found that five states did not have plans in place to 

 58One state did not report any information on this question.

Education’s Technical 

Assistance Does Not 

Yet Adequately 

Address Many States’

Challenges in 

Evaluating and 

Sustaining Programs

States Reported Mixed 

Reviews on Education’s 

Technical Assistance for 

Evaluating their 21st 

Century Programs
Page 26 GAO-17-400 K-12 Education

conduct these evaluations. In addition, according to technical assistance 

reports provided as a part of the monitoring process, three states had 

difficulty conducting comprehensive evaluations such as establishing the 

appropriate scope of work and timeframes for conducting evaluations and 

developing appropriate performance measurement tools.

According to Education officials, it has not provided written guidance to 

states on 21st Century program evaluations since 1999, when it provided 

states non-regulatory guidance in the form of a guide to help them 

evaluate their programs and use the results to make program 

improvements.59 However, since 1999, there have been significant 

changes in the 21st Century program. Specifically, the program was 

reauthorized twice, resulting in changes to requirements for measuring 

and evaluating program performance. First, the 2002 reauthorization 

added new requirements for states to describe in their funding 

applications how the state will evaluate the effectiveness of programs, 

including developing performance indicators and measures to evaluate 

programs and activities.60 It also required sub-grantees to conduct 

periodic local evaluations to assess progress toward achieving goals.61

Second, as a result of requirements under ESSA, which reauthorized the 

21st Century program in 2015, additional changes are planned starting in 

school year 2017-2018. Specifically, states will have to evaluate their 

programs in conjunction with new data collection and evaluation planning 

requirements, including requirements to track student progress over time 

and to include state standardized test scores and other indicators of 

student success, such as improved school-day attendance.62

Federal standards for internal control state that when significant changes 

occur in how an agency achieves its objectives it should periodically 

review policies and procedures for continued relevance and effectiveness 

 59N. Pane, I. Mulligan, A. Ginsburg, & A. Lauland, A Guide to Continuous Improvement 

Management (CIM) for 21st Century Community Learning Centers (Washington, D.C.: 

U.S. Department of Education, 1999).

60No Child Left Behind Act of 2001, Pub. L. No. 107-110, § 401, 115 Stat. 1425, 1768 

(creating 20 U.S.C. § 7173(a)(13)) (2002).

61Pub. L. No. 107-110, § 401, 115 Stat. 1425, 1772 (creating 20 U.S.C. § 7175(b)(2)) 

(2002).

62 Pub. L. No. 114-95, § 4201, 129 Stat. 1802, 1986-1987 (2015). In addition, ESSA 

increased the amount that states can set aside for activities including monitoring and 

evaluation from 3 to 5 percent of their 21st Century funds. 
Page 27 GAO-17-400 K-12 Education

in achieving its objectives or addressing related risks. Further, it states 

that written policies and procedures can help ensure that necessary 

actions are taken to address risks to achieving the entity’s objectives.63 In 

addition, our prior work has identified several leading practices in 

agencies’ use of evaluations, which state that agencies should promote 

capacity building to evaluate program activities as a key strategy for 

supporting objectives.64 Education officials told us they are developing a 

technical assistance plan to improve the support the agency provides to 

states to implement the 21st Century program. These officials told us they 

are considering the topics and areas of concern that they will address 

including developing and conducting effective evaluations and 

assessments, and they expect to finalize the list of topics in summer 

2017. However, Education officials told us that the agency has not 

determined what type of technical assistance it will provide to states on 

conducting evaluations. Absent new written, non-regulatory guidance to 

states that addresses the areas in which they struggle most, Education 

may miss opportunities to help states improve their capacity to conduct 

high-quality evaluations and ultimately improve their programs.

Education’s technical assistance to states does not effectively address 

the challenges most states face in helping their 21st Century subgrantees continue to operate their programs once grant funding ends, 

according to state and sub-grantee officials. In our survey, 35 states 

reported that their centers often face challenges providing the same level 

of services to students when the grant funding ends. Officials in 20 states 

reported that centers in their states generally reduce the level of services 

or cease to operate once 21st Century funding ends. Two states 

commented that less than 10 percent of centers had been able to 

continue operating after the 21st Century grants expired. These concerns 

were echoed by officials at 10 of the 13 21st Century centers we visited, 

who told us they had major concerns about sustaining their programs.

Twenty centers that commented they will be unable to sustain operations 

or maintain the same level of services after 21st Century funding ends 

had difficulty securing funds from other sources. For example, officials in 

 63GAO, Standards for Internal Control in the Federal Government, GAO-14-704G

(Washington, D.C.: September 2014).

64GAO, Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in 

Program Management and Policy Making, GAO-13-570 (Washington, D.C.: June 26, 

2013).

Education’s Technical 

Assistance Does Not 

Effectively Address States’

Difficulty in Sustaining 21st 

Century Programs
Page 28 GAO-17-400 K-12 Education

one state commented that it is often difficult for centers to obtain funding 

from local school districts because district budgets are already strained by 

supporting school-day programs. Officials in three other states 

commented that there is no state funding dedicated exclusively to 

programs outside the regular school day.

By law, states must require sub-grant applicants to submit a preliminary 

plan for how their programs will continue to operate once grant funding 

ends.65 Education’s monitoring protocols call for Education officials to ask 

states if applicants have sustainability plans and how states monitor subgrantees’ implementation of the plans. A majority of states (42 states) 

reported in our survey that they require sub-grantees to provide a written 

sustainability plan, while 6 states reported they do not.66 However, in its 

monitoring efforts, Education told us it did not identify any states that are 

currently out of compliance with this requirement.

Education may be missing opportunities in its monitoring efforts to collect 

information on states’ strategies and practices for program 

sustainability—information that could be useful for sharing promising 

practices across states. As a part of its monitoring process, Education 

officials told us they discuss with state officials how they ensure 

compliance with the requirement to have such sustainability plans in 

place, but they do examine the quality of the plans. Officials in 11 states 

in our survey commented they would benefit from opportunities to 

collaborate across states. States do appear to have valuable information

on sustainability to share with each other. For example, 25 states—in our 

survey of all 50 states and the District of Columbia—reported providing 

additional points or credit based on applicants’ ability to leverage external 

funds and sustain programs when selecting sub-grantees. Further, in the 

states that collect information on what happens after 21st Century funding 

ends, most reported centers having some success in finding private or 

nonprofit funds to help replace Education’s grant funds. States reported 

that the most common sources of replacement funding were student fees 

charged to participants (17 states), private foundation funding (14 states), 

and non-profit funding (e.g., universities and community organizations) 

(13 states).

 6520 U.S.C. § 7174(b)(2)(K).

66Three states did not report data on whether they require sub-grantees to provide a 

written sustainability plan.
Page 29 GAO-17-400 K-12 Education

Education officials told us they provide technical assistance on 21st 

Century program sustainability through interactions such as an annual 

summer training conference on the program and an online learning 

portal.67 However, Education’s July 2016 training conference—a forum 

where 21st Century participants share experiences and best practices—

was generally not focused on state practices and policies. The sessions 

were generally focused on centers’ collaboration with the school or 

community, centers’ activities (e.g., literacy, science, and math), or how to 

report performance data. Officials in about a third of states reported in our 

survey that they did not know about or had no basis to judge Education’s 

technical assistance on sustainability. Another third of states reported that 

Education’s technical assistance in this area was slightly or not at all 

useful. For example, two states reported that the content of Education’s 

professional development, training, and other assistance is heavily 

geared toward sub-grantees.

Education officials said they do have other interactions and forums, such 

as regional meetings, to share information with states. For example, 

Education started in-depth training in 2016 for state-level officials at four 

regional meetings for the Midwest, Northwest, South, and East, but the 

sessions did not cover program sustainability.

Federal standards for internal control state that information should be 

communicated in a form that enables an agency to achieve its 

objectives.68 In addition, our prior work has identified leading practices in 

federal collaboration that have shown that information sharing among 

grant participants, such as states, is important for effective grants 

management.69 Education’s efforts to help facilitate information sharing 

among states can help states identify practices that can be tailored to 

meet their individual needs and leverage their knowledge to address 

common challenges in continuing their programs over the long-term. 

 67You For Youth is an online service hosted by Education that provides professional 

development and technical assistance on the 21st Century program. See 

https://y4y.ed.gov/.

68GAO, Standards for Internal Control in the Federal Government, GAO-14-704G 

(Washington, D.C.: September 2014). 

69GAO, Municipalities in Fiscal Crisis: Federal Agencies Monitored Grants and Assisted 

Grantees, but More Could Be Done to Share Lessons Learned, GAO-15-222 

(Washington, D.C.: Mar. 20, 2015); and Grants to State and Local Governments: An 

Overview of Federal Funding Levels and Selected Challenges, GAO-12-1016 

(Washington, D.C.: Sept. 25, 2012).
Page 30 GAO-17-400 K-12 Education

Unless Education undertakes such efforts for the 21st Century program, 

students and families who participate may be at greater risk of not 

receiving the full range program benefits for the longest amount of time.

The 21st Century program is designed to fund programs outside the 

regular school day to improve academic and behavioral outcomes for K12 students who are from low-income families or who attend lowperforming schools. High-quality research on the effectiveness of this 

program is very limited due to several factors, including difficulty 

determining whether a change in students’ outcomes results from their 

participation in a 21st Century program or from other factors, such as 

interventions during the school day. Although existing research on 

effectiveness points to greater positive behavioral effects than academic 

effects, Education’s current performance measures do not address some 

key behavioral outcomes. The lack of performances for behavioral 

outcomes makes it difficult to determine whether the program is achieving 

some of its stated objectives—especially in areas where research has 

shown that the program is likely to have the greatest effect on student 

outcomes. Specifically, Education does not measure socio-emotional 

outcomes or two other behavioral outcomes included in program 

objectives: improved school-day attendance and discipline in the 

classroom. Absent these performance measures, Education is missing an 

opportunity to assess the full range of benefits of this program.

In addition, Education has not sufficiently assessed the quality of its 

program data, further limiting its ability to assess program effectiveness. 

Another factor hindering the agency’s ability to determine program 

effectiveness is the significant difficulty that states are experiencing in 

evaluating their programs. Although Education officials told us they are 

considering topics and areas of concern for additional technical 

assistance, Education has not yet provided states with sufficient guidance 

on developing rigorous evaluations. Unless Education takes steps to 

reasonably ensure the accuracy of data and provides written guidance to 

help states develop high-quality evaluations, it cannot ensure the program 

is effectively meeting its goals.

Lastly, states are experiencing substantial difficulty in sustaining their 

programs after 21st Century funding ends, and many states expressed 

interest in collaborating with other states to address such challenges. 

Education is uniquely situated to take the lead in sharing information with 

states to help them address their sustainability challenges by allowing 

them to identify state policies and practices that have had some success. 

Conclusions
Page 31 GAO-17-400 K-12 Education

Sharing such information could help states address the challenges they 

face in continuing their programs over the long-term.

We recommend that the Secretary of Education direct the Office of 

Academic Improvement to:

1. Expand its performance measures for the 21st Century program to 

address all program objectives. Specifically, Education should 

establish performance measures related to key behavioral, including 

student attendance and disciplinary incidents, and socio-emotional 

outcomes.

2. Conduct federal-level data checks on the accuracy of 21st Century 

program data submitted by states. Such checks could test for logical 

relationships between fields. Education should also publicly disclose 

and address any data limitations it identifies, as appropriate.

3. Provide written, non-regulatory guidance to states on developing and 

conducting high-quality 21st Century state evaluations to help address 

the difficulties states face in measuring program performance and 

effectiveness.

4. Use the information it collects from its monitoring visits and ongoing 

interactions with states to share effective practices across states for 

sustaining their 21st Century programs once program funding ends. 

This information could be shared using existing mechanisms such as 

Education’s meetings with 21st Century state coordinators. 

We provided a draft of this report to the Department of Education for 

comment, and its written comments are reproduced in appendix III. 

Education also provided technical comments that we incorporated in the 

report as appropriate. Education neither agreed nor disagreed with our 

recommendations; rather, it generally noted that it will keep our 

recommendations in mind as it continues to implement changes in the 

program as a result of ESSA, and outlined steps it will take to address our 

recommendations. 

In response to our recommendation that it expand its performance 

measures for the 21st Century program to address all program objectives, 

including those related to behavioral and socio-emotional outcomes,

Education said it will keep our recommendation in mind. Specifically, the 

department stated that it is in the process of re-examining whether 

additional or revised measures should be developed to align more 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
Page 32 GAO-17-400 K-12 Education

significantly with the program’s statutory objectives under ESSA. 

Education also acknowledged the need to develop measures that provide 

sufficient information and data on program outcomes. As stated in our 

draft report, the Department currently measures some student behavioral 

outcomes; in its response, Education described one measure as: the 

percentage of all program participants with teacher-reported 

improvements in behavior. Therefore, we modified our recommendation 

to focus on measuring the program’s key behavioral outcomes. In its 

response, Education also expressed concern about collecting data on 

student attendance and disciplinary measures, noting that it will require 

effective collaboration between states, districts, and other eligible entities.

However, as we stated in our report, about half of states already collect 

data on at least one of these two measures. Further, as we stated, 

research has shown that 21st Century programs more often have positive 

effects on student attendance and reducing disciplinary incidents than on 

improving students’ academic outcomes. Given these effects, we 

continue to believe that it is critical for Education to measure student 

attendance and disciplinary incidents to obtain more complete, accurate 

information on this program’s effect on student outcomes. 

In response to our recommendation that it conduct federal-level data 

checks on the accuracy of 21st Century program data submitted by 

states, Education commented that it plans to build in additional data 

checks into the data system beyond its current checks on the data’s 

completeness. Specifically, Education anticipates that new technology 

enhancements in the data system will be designed to flag for 

inconsistencies in data reporting. For example, the system may send a 

“flag” that participation data is significantly lower or higher than previously 

reported participation data. Further, Education indicated that it will 

consider whether auditors performing audits under the Single Audit Act 

can be asked and guided to do more checks on the accuracy and 

reliability of 21st Century program data. 

Regarding our recommendation to provide written, non-regulatory 

guidance to states on developing and conducting high-quality 21st 

Century state evaluations, Education outlined several steps it has taken 

to assist states in the past. For example, Education said that it provided 

six states with individualized technical assistance on strategies related to 

developing statewide evaluations and measures. Education also noted 

that, to date, it has conducted two webinars on state evaluations and is in 

the process of including presentations from those webinars on its online 

learning portal so that states will have easy access to the information. In 

addition, Education stated that it included presentations on evaluation 
Page 33 GAO-17-400 K-12 Education

strategies in the past during its Summer Institute. Education also said it 

would consider whether additional guidance for all states was needed. 

While these are important steps, we do not believe they are sufficient. 

Twenty-one of 51 states (41 percent of states) reported in our 2016 

survey that they found Education’s technical assistance for developing 

and conducting state-level evaluations only slightly or not at all useful; 

they had not received any technical assistance on this topic; or they were 

unaware that Education provided such assistance. Therefore, we 

continue to believe that Education should prepare written guidance to 

assist all states in developing and conducting high-quality program 

evaluations. 

Finally, regarding our recommendation to share effective practices across 

states for sustaining their 21st Century programs once program funding 

ends, Education stated that it hosts meetings twice a year for 21st 

Century state coordinators. At these meetings, Education officials share 

strategies with states related to program sustainability. Education stated 

that these meetings covered topics such as reducing the amounts of 21st

Century grant awards by a percentage each year. However, Education 

officials told us in February 2017 that these meetings have not focused on 

topics on program sustainability for several years. We continue to believe 

that Education should take the lead in sharing information with states to 

help them address their sustainability challenges by sharing information 

on state policies and practices that have shown some success. In its 

comments, Education stated that it has not held regional meetings with 

states for several years; however, in 2016 it held four regional meetings

with states on implementing its new 21APR data system. Therefore, we 

modified our recommendation to emphasize that such information be 

shared through these types of existing mechanisms.
Page 34 GAO-17-400 K-12 Education

We are sending copies of this report to the appropriate congressional 

committees, the Secretary of Education, and other interested parties. In 

addition, the report will be available at no charge on GAO’s website at 

http://www.gao.gov.

If you or your staff should have any questions about this report, please 

contact me at (617) 788-0580 or nowickij@gao.gov. Contact points for 

our Offices of Congressional Relations nowickij@gao.gov Public Affairs 

may be found on the last page of this report. GAO staff who made key 

contributions to this report are listed in appendix IV.

Jacqueline M. Nowicki

Director, Education, Workforce,

and Income Security Issues
Appendix I: Scope and Methodology

Page 35 GAO-17-400 K-12 Education

Our study of the 21st Century program was framed around four 

objectives: (1) how 21st Century funds are awarded and used, (2) what is 

known about the effectiveness of 21st Century programs, (3) the extent to 

which Education has effectively managed and used program data to 

inform decisions about the 21st Century program, and (4) the extent to 

which Education’s technical assistance has helped states assess and 

sustain high-quality programs once 21st Century funding ends. We 

focused our review on 21st Century afterschool programs because 

centers have historically provided services more during this time as 

compared to before school and during the summer. To address our four 

objectives, we used a variety of methods, including a Web-based survey 

of 21st Century state coordinators; a review of selected state evaluations 

and academic literature; a review of federal laws, regulations, and agency 

documents such as annual performance reports and guidance; interviews 

with federal and other officials; and site visits to four states.

To obtain information about the state sub-grantee award process, 

program sustainability and technical assistance of the 21st Century 

program, we conducted a Web-based survey of the 21st Century state 

coordinator at each state educational agency in all 50 grantee states and 

the District of Columbia. We conducted the survey from August 2016 

through November 2016. The survey covered several topics, including 

state processes for awarding 21st Century sub-grants to local 

communities, state-level program evaluations, performance measures, 

and sub-grantees’ financial sustainability, among other things. We 

received responses from all 50 states and the District of Columbia for a 

100 percent response rate. The survey included an introductory 

statement specifying that the survey focused on afterschool programs 

funded by the 21st Century program and collects information about 

federal-level guidance provided to state officials on this program.

The quality of survey data can be affected by nonsampling error, which 

includes variations in how respondents interpret questions, respondents’

willingness to offer accurate responses, and data collection and 

processing errors. To minimize such error, we included the following 

steps in developing the survey and in collecting and analyzing survey 

data: in developing the Web survey, we pre-tested draft versions of the 

instrument with 21st Century program officials in four states to check the 

clarity of the questions and the flow and layout of the survey. On the basis 

of the pretests, we made revisions to the survey. Further, using a Webbased survey and allowing 21st Century coordinators to enter their 

responses directly into an electronic instrument created an automatic 

Appendix I: Scope and Methodology

Survey of State 

Coordinators
Appendix I: Scope and Methodology

Page 36 GAO-17-400 K-12 Education

record for each state in a data file and eliminated the errors associated 

with a manual data entry process. In addition, the program used to 

analyze the survey data was independently verified to ensure the 

accuracy of this work.

In order to examine what is known about the effectiveness of 21st 

Century programs, we reviewed select state-level program evaluations 

and academic studies that reported outcomes for students who 

participated in the program. To identify relevant research, we took a twopronged approach. First, we reviewed three state evaluations that 

Education determined in a 2012 report on 21st Century program grantee 

evaluation practices (the most recent available Education report on this 

topic) to have used quasi-experimental methods or statistical controls, 

such as a comparison group, to account for other plausible influences on 

the outcomes reported. We retrieved the most recent evaluations from 

those three states and through interviews with researchers in the field; we 

identified three other quasi-experimental state evaluations published 

since 2012. We reviewed each of these state evaluations for the 

soundness of its methodology and data. In four cases where the 

published evaluation did not include enough information to complete our 

review, we contacted the states and the researchers to obtain additional 

data from their samples. Ultimately, we determined that four of the six 

state evaluations were appropriate for purposes of our research objective 

about the 21st Century program’s effectiveness.

Second, we conducted a comprehensive literature review of 104 

academic studies of afterschool programs, including the 21st Century 

program, and we ultimately determined that six of these academic studies 

were sufficiently rigorous and appropriately scoped to include in our 

review. To identify academic studies on the effectiveness of 21st Century 

programs, we conducted a literature search through ProQuest. Our initial 

search terms were adapted from a 2014 report produced by Education 

that looked at the effect of 21st Century programs, along with other 

programs that increased the amount of time that children spent in school.1

Because our review was focused on afterschool programs, rather than 

 1 Yael Kidron and Jim Linsday, The Effects of Increased Learning Time on Student 

Academic and Nonacademic Outcomes: Findings from a Meta-analytic Review

(Washington, DC: U.S. Department of Education, Institute of Education Sciences, National 

Center for Education Evaluation and Regional Assistance, Regional Educational 

Laboratory Appalachia, 2014).

Review of State 

Evaluations and Academic 

Literature
Appendix I: Scope and Methodology

Page 37 GAO-17-400 K-12 Education

the broader range of out of school time or extended learning time 

programs covered in that study, we only searched for studies of 

afterschool programs and studies of 21st Century programs particularly.

We identified 104 papers initially and selected 25 of them on the basis of 

the following criteria:

• Topic relevance–the studies covered 21st Century programs or 

afterschool programs that measured student outcomes and had some 

instructional component (i.e. programs were not only a study hall or a 

sports program).

• Timeframe relevance–studies must have been published since 2012, 

in order to find studies that may not have been included in Education’s 

2014 study, which examined studies published since 1998.

• Publication status–studies were in their final form, not drafts.

• Sample relevance–studies looked at afterschool programs in K-12 

settings in the United States. Foreign school systems were excluded.

• Design relevance–studies used experimental or quasi-experimental 

designs with well-formulated comparison groups or controls; or they 

used some statistical method to account for other plausible influences 

on the outcomes of the study.

Next, two analysts reviewed those studies a second time, retaining 13 

studies that had a sample size above 100 students in the intervention 

group and evaluated more than one program or intervention site. The 

analysts further narrowed this sample to six studies, which we selected 

because they were focused on 21st Century programs. In addition to 

reviewing Education’s 2014 meta-analysis, in our initial search of 104 

papers, the analysts also identified two additional meta-analyses focused 

on afterschool programs, which we included in our sample because they 

were of appropriate quality and scope to assess afterschool program 

effects. We then conducted detailed reviews of the studies. These 

reviews entailed an evaluation of each study’s research methodology, 

including its research design, sampling frame, selection of measures, 

data quality, limitations, and analytic techniques, as well as a summary of 

its major findings. We also assessed the extent to which each study was 

relevant to assessing what is known about the effectiveness of 21st 

Century programs. Three studies had major research design limitations 

resulting from the lack of a rigorously formed comparison group. As such, 

these studies were not able to demonstrate whether the programs were 

responsible for the effects being measured or whether other factors may 
Appendix I: Scope and Methodology

Page 38 GAO-17-400 K-12 Education

have contributed. After eliminating the three studies with major research 

design limitations, six studies remained in our review.

For all four objectives, we reviewed agency documents and interviewed 

agency officials and researchers who had conducted work on the 21st 

Century program. To examine how states and other entities have used 

21st Century funds, we reviewed Education’s 2014 annual characteristic 

report for the 21st Century program. The reports provide summaries and 

analyses over time, based on program data collected from states in 

school years 2001-2002 through 2012-2013 using data from the Profile 

and Performance Information Collection System. In order to assess the 

reliability of the data underlying the reports, we reviewed agency 

documents regarding this data system, spoke with agency officials, and 

conducted electronic tests of data from the system for select data 

elements. We determined that the underlying data summarized and 

analyzed in Education’s 2014 annual characteristic report was sufficiently 

reliable for our purpose of reporting on program characteristics.

To examine Education’s management of program data, we also reviewed 

Education documents including budget justifications and documents on 

the 21st Century data systems such as user guides, technical assistance 

documents for states, data dictionaries, and a 2013 Education Office of 

Inspector General report on federal and state oversight for the 21st 

Century program.

To examine Education’s technical assistance, we reviewed Education’s 

program guidance to states, monitoring protocols and tools, and annual 

grantee satisfaction survey results which generally includes Education’s 

largest grant programs including the 21st Century program. In order to 

compare the program to established criteria, we reviewed relevant 

legislation; Office of Management and Budget guidance; federal internal 

control standards; and past GAO work pertaining to leading practices in 

performance management, information quality, program evaluation, and 

federal collaboration for grants management.

We interviewed federal officials from Education’s Office of Academic 

Improvement, within the Office of Elementary and Secondary Education, 

regarding data management and use, research and evaluation, and 

program challenges. In addition, we interviewed stakeholders of the 21st 

Century program, including contractors and researchers. Finally, we 

observed Education’s annual Summer Institute, where Education 

Review of Agency 

Documents and Interviews 

with Agency Officials
Appendix I: Scope and Methodology

Page 39 GAO-17-400 K-12 Education

provides technical assistance and training to state, sub-grantee, and 

center officials.

To learn about program activities and challenges, we conducted site visits 

to four states—Idaho, Massachusetts, Rhode Island, and Texas—to visit 

21st Century centers and speak with program staff and state officials. We 

selected these states for diversity in geographic region, 21st Century 

formula grant funding levels to states, and student demographics. In each 

state, we interviewed the 21st Century state coordinator and other 

officials responsible for the program. In total, we visited 13 centers in the 

four states. During these visits, we interviewed center staff and observed 

afterschool program activities.

Information we gathered on our site visits represents only the conditions 

present in the states and local areas at the time of our visits. Furthermore, 

our fieldwork focused on in-depth analysis of only a few selected states. 

On the basis of our site visit information, we cannot generalize our 

findings beyond the states we visited.

We conducted this performance audit from April 2016 to April 2017 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

Site Visits
Appendix II: 21st Century Grant Award Amount 

by States, Fiscal Year 2016

Page 40 GAO-17-400 K-12 Education

Table 1: 21st Century Grant Award Amount by States, Fiscal Year 2016

State Educational Agency (SEA) Grant award amount (in dollars)

California 132,439,027

Texas 101,389,315

New York 84,279,065

Florida 61,414,141

Illinois 50,808,494

Ohio 43,888,443

Pennsylvania 42,558,875

Georgia 39,347,084

Michigan 38,833,081

North Carolina 32,539,202

Arizona 24,696,549

New Jersey 23,666,114

Louisiana 22,316,104

Tennessee 21,760,677

Indiana 20,236,679

Missouri 18,194,441

Virginia 18,141,534

Kentucky 17,188,889

Alabama 17,135,242

South Carolina 16,787,291

Washington 16,694,742

Massachusetts 16,671,886

Wisconsin 16,137,201

Maryland 15,604,645

Mississippi 14,134,129

Arkansas 12,195,332

Oklahoma 11,926,077

Colorado 11,925,141

Oregon 11,429,471

Minnesota 11,253,198

Nevada 9,133,188

Connecticut 8,966,295

New Mexico 8,392,219

Kansas 8,286,212

Utah 6,982,788

Appendix II: 21st Century Grant Award 

Amount by States, Fiscal Year 2016
Appendix II: 21st Century Grant Award Amount 

by States, Fiscal Year 2016

Page 41 GAO-17-400 K-12 Education

West Virginia 6,849,474

Iowa 6,572,166

Maine 5,643,198

New Hampshire 5,643,198

Rhode Island 5,643,198

Vermont 5,643,198

Delaware 5,643,198

DC 5,643,198

Nebraska 5,643,198

North Dakota 5,643,198

South Dakota 5,643,198

Alaska 5,643,198

Hawaii 5,643,198

Idaho 5,643,198

Montana 5,643,198

Wyoming 5,643,198

Source: GAO analysis of Education documents. GAO 17-400
Appendix III: Comments from the U.S. 

Department of Education

Page 42 GAO-17-400 K-12 Education

Appendix III: Comments from the U.S. 

Department of Education
Appendix III: Comments from the U.S. 

Department of Education

Page 43 GAO-17-400 K-12 Education
Appendix III: Comments from the U.S. 

Department of Education

Page 44 GAO-17-400 K-12 Education
Appendix III: Comments from the U.S. 

Department of Education

Page 45 GAO-17-400 K-12 Education
Appendix IV: GAO Contact and 

Staff Acknowledgments

Page 46 GAO-17-400 K-12 Education

Jacqueline M. Nowicki, (617) 788-0580 or nowickij@gao.gov

In addition to the contact named above, Elizabeth Sirois (Assistant 

Director), Sheranda Campbell (Analyst-in-Charge), Lucas Alvarez, Ashley 

Chaifetz, Jamila Kennedy, and Kelsey Kennedy made significant 

contributions to this report. Assistance, expertise, and guidance were 

provided by Susan Aschoff, Carl Barden, Deborah Bland, James Bennett, 

Angie Jacobs, Thomas James, Kirsten Lauber, Ben Licht, Edward 

Malone, John Mingus, Amy Moran Lowe, Sara Pelton and James Rebbe.

Appendix IV: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(100498)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov and read The Watchblog.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.