21st Century Community Learning Centers 

Descriptive Study of Program Practices 

U.S. Department of Education 

Office of Planning, Evaluation and Policy Development 

Policy and Program Studies Service 

Prepared by: 

William R. Penuel 

Raymond McGhee, Jr. 

SRI International 

Menlo Park, Calif. 

2010
This report was prepared for the U.S. Department of Education under Contract Number ED-01-CO0059/0005 by SRI International. Beth Yeh served as the contracting officer’s representative. The views 

expressed herein do not necessarily represent the positions or policies of the U.S. Department of 

Education. No official endorsement by the U.S. Department of Education is intended or should be 

inferred. 

U.S. Department of Education 

Arne Duncan 

Secretary 

Office of Planning, Evaluation and Policy Development 

Carmel Martin 

Assistant Secretary 

Policy and Program Studies Service 

Alan Ginsburg 

Director 

July 2010 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While 

permission to reprint this publication is not necessary, the suggested citation is: U.S. Department of 

Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service, 

21st Century Community Learning Centers Descriptive Study of Program Practices, Washington, D.C., 

2010. 

To order copies of this report, write to 

ED Pubs 

Education Publications Center 

U.S. Department of Education 

P.O. Box 22207 

Alexandria, VA 22304 

Or fax your request to 703-605-6794. 

Or call in your request toll-free to1-877-433-7827 (1-877-4-ED-PUBS). Those who use a 

telecommunications device for the deaf (TDD) or a teletypewriter (TTY) should call 1-877-576-7734. If 

877 service is not yet available in your area, call 1-800-872-5327 (1-800-USA-LEARN). 

Or order online at http://edpubs.gov. 

On request, this publication is available in alternate formats, such as Braille, large print, or computer 

diskette. For more information, please contact the U.S. Department of Education’s Alternate Format 

Center at 202-260-0852 or 202-260-0818. 
Contents i 

Contents 

Acknowledgments ........................................................................................................................................ vii 

Executive Summary ...................................................................................................................................... ix 

Study Background and Overview ............................................................................................................. ix 

Key Findings ............................................................................................................................................. x 

Nature and Quality of Reading and Mathematics Activities ............................................................... x 

Student Attendance in Center Activities ............................................................................................. xiii 

Organizational Supports for Instructional Quality .............................................................................. xv 

Conclusions .............................................................................................................................................. xvii 

1. Study Background, Goals and Methods .............................................................................................. 1 

The 21st Century Community Learning Centers Program ....................................................................... 2 

Background ............................................................................................................................................... 4 

Conceptual Framework for the Study ....................................................................................................... 5 

Study Design and Data Collection Methods ............................................................................................. 7 

Evaluation Questions ......................................................................................................................... 7 

Study Design ...................................................................................................................................... 8 

Data Sources and Data Collection Procedures ................................................................................. 8 

Weighting .................................................................................................................................................. 10 

Limitations of the Study ............................................................................................................................ 10 

Presentation of Survey Data in the Report ............................................................................................... 12 

2. Reading and Mathematics Activities in Centers Serving Elementary School Students ................. 13 

Key Findings ............................................................................................................................................. 13 

Exposure to Academics After School ....................................................................................................... 14 

Reading Activities ..................................................................................................................................... 15 

Reading Skills Emphasized ............................................................................................................... 15 

Reading Skills Emphasized in Observed Activities ............................................................................ 16 

Use of Instructional Strategies for Teaching Vocabulary, Fluency and Comprehension .................. 17 

Mathematics Activities ............................................................................................................................. 20 

Mathematics Skills Emphasized ........................................................................................................ 20 

Mathematics Content Emphasized in Observed Activities ................................................................ 21 

Use of Instructional Strategies for Teaching Mathematics ................................................................ 22 

Mathematics Instructional Strategies in Observed Activities ............................................................. 24 

Instructor Education and Experience ....................................................................................................... 26 

Instructor Guidance in Observed Activities .............................................................................................. 27 

Feedback to Students and Parents on Student Progress ........................................................................ 28 

Summary .................................................................................................................................................. 29 

3. Student Targeting and Attendance ....................................................................................................... 31 

Key Findings ............................................................................................................................................. 31 

Priorities Regarding Targeting .................................................................................................................. 32 

Policies Regarding Attendance ................................................................................................................ 33 

Regular Attendance .................................................................................................................................. 34 

Student Engagement in Observed Activities ............................................................................................ 39 

Summary .................................................................................................................................................. 39 
Contents ii 

4. Organizational Supports for Instructional Quality .............................................................................. 41 

Key Findings ............................................................................................................................................. 41 

Center Staffing .......................................................................................................................................... 41 

Recruiting and Retaining High-Quality Staff ...................................................................................... 41 

Professional Development ................................................................................................................. 45 

Alignment to Student Needs ..................................................................................................................... 47 

Coordination With the School Day ..................................................................................................... 47 

Use of Student Assessment Data ..................................................................................................... 48 

Adult Support: Fostering Positive Relationships Between Adults and Students ...................................... 52 

Coordination with Supplemental Educational Services ..................................................................... 53 

Summary .................................................................................................................................................. 53 

5. Conclusions and Implications for Future Research and Evaluation ................................................ 55 

Nature and Quality of Reading and Mathematics Activities .................................................................... 55 

Attendance and Student Participation ...................................................................................................... 55 

Organizational Supports for Instructional Quality ..................................................................................... 55 

Key Differences Between Different Types of Centers .............................................................................. 56 

Recommendations for Future Research and Evaluation ......................................................................... 56 

References ..................................................................................................................................................... 59 

Appendices 

A: Descriptions of Academic Instruction for Centers Serving All Students ..................................................... 65 

B: Case Study Observations ........................................................................................................................... 79 

C: Sampling Approach and Weighting of Surveys .......................................................................................... 85 

D: Survey Instruments ..................................................................................................................................... 95 

E: Supplementary Tables for the Surveys ...................................................................................................... 131 

F: 21st CCLC Quality Study—Quality Observation Instrument ...................................................................... 151 
Contents iii 

Exhibits 

Exhibit ES-1: Percentage of Centers Serving Elementary School Students That Reported Participation in 

Activities by a Typical Student, by Type of Activity and Amount of Time of Engagement per 

Week ........................................................................................................................................ xii 

Exhibit 1-1: Characteristics and Type of and Grade Levels Served by Funded and Sampled Centers ..... 4 

Exhibit 1-2: Framework for the Study: Supports for Student Participation in Quality Academic 

Instruction That Leads to Increased Achievement ................................................................... 6 

Exhibit 2-1: Percentage of Centers Serving Elementary School Students That Reported Participation in 

Activities by a Typical Student, by Type of Activity and Amount of Time of Engagement per 

Week ........................................................................................................................................ 15 

Exhibit 2-2: Percentage of Centers Serving Elementary School Students That Reported Reading 

Activities Emphases, by Reading Skill Emphasized and Inclusion Level ................................ 16 

Exhibit 2-3: Percentage of Observations of Reading Activities in Centers Serving Elementary School 

Students, by Reading Skill Emphasized .................................................................................. 17 

Exhibit 2-4: Percentage of Centers Serving Elementary School Students That Reported Student 

Participation in Instructional Strategies in Reading Activities, by Strategy and Inclusion 

Level ......................................................................................................................................... 18 

Exhibit 2-5: Percentage of Observations of Reading Activities in Centers Serving Elementary School 

Students, by Instructional Strategy Emphasized ..................................................................... 19 

Exhibit 2-6: Percentage of Centers Serving Elementary School Students That Reported Emphases in 

Math Activities, by Type of Content Emphasized and Inclusion Level ..................................... 21 

Exhibit 2-7: Percentage of Observations of Mathematics Activities in Centers Serving Elementary 

School Students That Emphasized Skills, by Type of Math Skill Emphasized ........................ 22 

Exhibit 2-8: Percentage of Centers Serving Elementary School Students That Reported Student 

Participation in Instructional Strategies in Math Activities, by Strategy and Inclusion Level ... 24 

Exhibit 2-9: Percentage of Observations of Mathematics Activities in Centers Serving Elementary 

School Students That Emphasized Instructional Strategies, by Strategy Emphasized ........... 25 

Exhibit 2-10: Percentage of Observations in Reading and Mathematics Activities in Centers Serving 

Elementary School Students That Emphasized Use of Active Teaching Strategies, by 

Strategy Emphasized and Activity Type ................................................................................... 27 

Exhibit 2-11: Percentage of Centers Serving Elementary School Students That Provided Feedback 

About Their Progress, by Type of Feedback and Center Type ................................................ 28 

Exhibit 3-1: Percentage of Centers That Reported Top-Priority Target Groups, by Target Groups 

and Center Type ....................................................................................................................... 33 

Exhibit 3-2: Percentage of Centers That Reported Attendance Requirements, by Grade Level and 

Attendance Requirement.......................................................................................................... 34 

Exhibit 3-3: Percentage of Centers That Reported That They Required Participation of All Students in 

Specific Activities When Students Were Present, by Type of Activity ..................................... 35 

Exhibit 3-4: Percentage of Centers That Reported the Number of Days of Attendance in 2005–06, 

by Grade Level and Days of Attendance ................................................................................. 37 

Exhibit 3-5: Percentage of Centers That Reported Consequences for Student Absences, by 

Consequence Type ...................................................... ................................................ 38 

Exhibit 4-1: Percentage of Centers That Paid Hourly Rates to Staff Who Led Instructional Activities, 

by Type of Staff and Rates ....................................................................................................... 42 

Exhibit 4-2: Percentage of Staff Who Led Instructional Activities at Centers, by Hours Worked per Week 43 

Exhibit 4-3: Percentage of Centers That Reported Offering Benefits to Full- and Part-Time Staff Who 

Led Instructional Activities, by Benefit Offered and Staff Status .............................................. 44 

Exhibit 4-4: Percentage of Centers That Reported Reasons for Staff Departures, by Reason for 

Departure and Center Type ..................................................................................................... 45 

Exhibit 4-5: Percentage of Centers That Reported That They Offered Paid Professional Development 

Opportunities for Full- and Part-Time Staff Who Led Instructional Activities, by Staff Status 

and Type of Development ........................................................................................................ 46 

Exhibit 4-6: Percentage of Centers That Reported That State Assessments Data Was Available for 

Use in Tracking Student Outcomes, by Subject Tested and Center Type .............................. 48 
Contents iv 

Exhibit 4-7: Percentage of Centers That Reported Student Assessment Sources, by Type and 

Frequency of Assessment ........................................................................................................ 49 

Exhibit 4-8: Percentage of Centers That Reported Uses of Student Assessment Data, by Type of 

Use ........................................................................................................................................... 50 

Exhibit 4-9: Percentage of Centers That Reported Challenges in Implementing High-Quality 

Programming, by Type of Challenge and Center Type ............................................................ 51 

Exhibit 4-10: Percentage of Observations in Reading and Math Activities That Demonstrated StudentStaff Relationship-Building Efforts, by Relationship-Building Effort and Type of Activity ....... 52 

Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 

Exhibit A-1: Percentage of Centers That Reported Participation in Activities by a Typical Student, 

by Type of Activity and Hours of Engagement per Week ....................................................... 68 

Exhibit A-2: Percentage of Centers Serving Middle School Students That Reported Reading Activities 

Emphases, by Reading Skill Emphasized and Inclusion Level................................................ 69 

Exhibit A-3: Percentage of Centers Serving High School Students That Reported Reading Activities 

Emphases, by Reading Skill Emphasized and Inclusion Level................................................ 70 

Exhibit A-4: Percentage of Centers That Reported Student Participation in Instructional Strategies 

in Reading Activities, by Strategy and Inclusion Level ............................................................. 71 

Exhibit A-5: Percentage of Centers Serving Middle School Students That Reported Emphases in Math 

Activities, by Type of Content Emphasized and Inclusion Level .............................................. 72 

Exhibit A-6: Percentage of Centers Serving High School Students That Reported Emphases in Math 

Activities, by Type of Content Emphasized and Inclusion Level .............................................. 73 

Exhibit A-7: Percentage of Centers That Reported Student Participation in Instructional Strategies in 

Math Activities, by Strategy and Inclusion Level ...................................................................... 75 

Exhibit A-8: Percentage of Centers That Provided Feedback About Their Progress, by Type of 

Feedback and Center Type ...................................................................................................... 76 

Appendix B: Case Study Observations 

Exhibit B-1: Ratings of Observations in Enrichment Activities .................................................................... 81 

Exhibit B-2: Ratings of Observations in Reading Activities ......................................................................... 82 

Exhibit B-3: Ratings of Observations in Mathematics Activities .................................................................. 83 

Appendix E: Supplementary Tables for the Surveys 

Exhibit E-1: Centers Serving Elementary School Students That Reported Participation in Activities by 

a Typical Student, by Type of Activity and Amount of Time of Engagement per Week .......... 134 

Exhibit E-2: Centers Serving Elementary School Students That Reported Reading Activities Emphases, 

by Reading Skill Emphasized and Inclusion Level ................................................................... 135 

Exhibit E-3: Centers Serving Elementary School Students That Reported Student Participation in 

Instructional Strategies in Reading Activities, by Strategy and Inclusion Level ....................... 136 

Exhibit E-4: Centers Serving Elementary School Students That Reported Emphases in Math Activities, 

by Type of Content Emphasized and Inclusion Level .............................................................. 137 

Exhibit E-5: Centers Serving Elementary School Students That Reported Student Participation in 

Instructional Strategies in Math Activities, by Strategy and Inclusion Level ............................ 138 

Exhibit E-6: Centers Serving Elementary School Students That Provided Feedback About Their 

Progress, by Type of Feedback and Center Type and ............................................................ 139 

Exhibit E-7: Centers That Reported Top-Priority Target Groups, by Target Groups and Center Type ...... 140 

Exhibit E-8: Centers That Reported Attendance Requirements, by Grade Level and Attendance 

Requirement ............................................................................................................................. 141 

Exhibit E-9: Centers That Reported That They Required Participation of All Students in Specific 

Activities When Students Were Present, by Type of Activity ................................................... 142 

Exhibit E-10: Centers That Reported the Number of Days of Attendance in 2005–06, by Grade Level 

and Days of Attendance ........................................................................................................... 142 

Exhibit E-11: Centers That Reported Consequences for Student Absences, by Consequence 

Type...................................................... ........................................................................ 143 

Exhibit E-12: Centers That Paid Hourly Rates to Staff Who Led Instructional Activities, by Type of 

Staff and Rates ......................................................................................................................... 144 
Contents v 

Exhibit E-13: Staff Who Led Instructional Activities at Centers, by Hours Worked per Week ....................... 145 

Exhibit E-14: Centers That Reported Offering Benefits to Full- and Part-Time Staff Who Led Instructional 

Activities, by Benefit Offered and Staff Status .......................................................................... 145 

Exhibit E-15: Centers That Reported Reasons for Staff Departures, by Reason for Departure and 

Center Type .............................................................................................................................. 146 

Exhibit E-16: Centers That Reported That They Offered Paid Professional Development Opportunities for 

Full- and Part-Time Staff Who Led Instructional Activities, by Staff Status and Type of 

Development ............................................................................................................................ 147 

Exhibit E-17: Centers That Reported That State Assessments Data Was Available for Use in Tracking 

Student Outcomes, by Subject Tested and Center Type ......................................................... 147 

Exhibit E-18: Centers That Reported Student Assessment Sources, by Type and Frequency of 

Assessment .............................................................................................................................. 148 

Exhibit E-19: Centers That Reported Uses of Student Assessment Data, by Type of Use .......................... 149 

Exhibit E-20: Centers That Reported Challenges in Implementing High-Quality Programming, by Type of 

Challenge and Center Type ..................................................................................................... 150 

 
Contents vi 
Contents vii

Acknowledgments 

The authors wish to thank the many individuals who assisted with this study. 

SRI International: Nancy Adelman, Sarah Bardack, Nancy Kamprath Bier, Sara Carriere, Reina 

Fujii, Larry Gallagher, Bowyee Gong, Bonnee Groover, Catherine Hall, Meredith Ittner, Karla 

Jones, Christine Korbak, Carlin Llorente, Bladimir Lopez-Prado, Nicolette Mayes, 

Samantha Murray, Natalie Nielsen, Ron Orpitelli, Melia Repko, Angeline Reyes, Patrick 

Shields, Michael Smith and Stephanie Stone. 

Policy Studies Associates: Kolajo Afolabi, Monica Mielke, Ellen Pechman, Christina Russell, 

Dwayne Smith and Mark Wilson. 

U.S. Department of Education: Adriana de Kanter, Alan Ginsburg, David Goodwin, Daphne 

Kaplan, Jessica Ramakis, Margery Yeager and Beth Yeh. 

Technical Work Group and Consultants: David Fleming, Andria Fletcher, Denise Huang, 

Priscilla Little, Neil Naftzger, Mary Parella, Marguerite Peebles and Karyl Resnick. 

And finally, our deep appreciation to the center directors and center staff who participated in the 

study’s data collection efforts. 

 
Contents viii
Executive Summary ix

Executive Summary 

Study Background and Overview 

The 21st Century Community Learning Centers (21st CCLC) program is a federal program 

authorized in 1994 under Title IV, Part B, of the Elementary and Secondary Education Act of 

1965 (ESEA), as amended. The program was originally intended to provide funding to school 

districts to support continuing education and lifelong learning opportunities to children and 

adults to help keep the country’s workforce competitive for the 21st century. The program’s 

authorizing statute was amended by the reauthorization of ESEA in 2002 to provide before- and 

after-school and summer academic enrichment opportunities for children, particularly children 

attending low-performing public schools, and to eligible private school students. The goal of the 

program is to help students meet local and state academic standards in core subjects, such as 

reading and mathematics. In addition to adding to academic content, programs may also provide 

youth development activities, drug and violence prevention, technology education, art and music 

activities, character education, counseling and recreation. The federal program is administered by 

the United States Department of Education, which awards grants to states by formula. States, in 

turn, award subgrants to eligible entities on a competitive basis. Eligible entities include 

education agencies, community-based organizations and other entities operating in either school 

or community settings. 

The previous national evaluation of the 21st Century Community Learning Centers program 

(Dynarski et al., 2003; James-Burdumy et al., 2005), which examined centers funded under the 

pre-2002 legislative authority, found that the centers did not focus on academic achievement and 

had no effects on participants’ academic outcomes. These findings raised questions about the 

level of program quality in after-school programs. The requirements changed to focus more on 

academics, so a study conducted today of program impacts might not find the same results. 

In 2004, the U.S. Department of Education’s Policy and Program Studies Service contracted 

with SRI International and its partner, Policy Studies Associates, to undertake an evaluation of 

the 21st Century Community Learning Centers The following evaluation questions informed this 

study: 

1. What is the nature of activities in centers that are designed to promote the academic 

development of students? 

2. How do centers vary with respect to regular attendance? 

3. How do center leaders staff their centers, coordinate with other service providers and use 

data to improve programming? 

The sources of data for the study were surveys and site visits. A sample of 516 center 

directors intended to be nationally representative of centers offering academic activities 

completed a written survey in the 2006–07 school year. A subsample of administrators and 

program staff members from 122 centers completed a more in-depth telephone survey on 

attendance and staff characteristics in the same school year. Site visits in fall 2006 and spring 

2007 provided data on the nature of instruction to compare with the survey data; the visits also 

provided observational data on instruction and student participation. The study team interviewed 

program staff members and observed after-school programming at 12 sites (11 served elementary 
Executive Summary x

school students; one served middle school students). The site visit data are not nationally 

representative and observation protocols used have not been related to any outcomes of 

importance to the program. The study also used grantee- and center-level data collected by the 

21st CCLC program office at the U.S. Department of Education, through the Profile and 

Performance Information Collection System (PPICS) database, to identify the basic center 

characteristics nationwide and to construct the survey and case study samples. 

The study provides descriptive information on the 21st CCLC program; it does not provide 

information on program outcomes or impacts. In addition, the study’s original sampling strategy 

and its reliance on self-reported data from surveys limit the generalizability of the findings and 

provide no basis for making causal inferences. To ensure that all centers surveyed could respond 

to questions about academics, the sampling plan limited the sample to centers that were funded at 

the time of the study, and that offered instruction in reading, mathematics and technology; thus, 

the sample was not nationally representative of all 5,122 centers funded at the time of the study. 

To address this limitation, the data were poststratified to reflect the full population of centers. 

Respondents’ self-reported answers to survey questions may reflect unreliable memory of past 

events and may include responses on academic instruction that center staff considered socially 

desirable. This report presents findings on academic instruction only for centers serving 

elementary school students in which the study team collected observation data. Findings on 

academic instruction for centers serving middle and high school students are reported in 

Appendix A. 

Key Findings 

This report on the 21st Century Community Learning Centers program focuses on the 

implementation of reading and mathematics activities, student attendance and hiring and 

retaining qualified staff in centers from which data were obtained. 

Nature and Quality of Reading and Mathematics Activities 

The program statute requires that centers focus on academics and use research-based 

strategies for instruction. The law requires that students participate in academic activities at a 

frequency that is “sufficient to influence their learning.” 
Executive Summary xi

According to grantee performance data, nearly all centers funded offered reading1 and 

mathematics activities. 

Ninety-eight percent of all centers funded as of the 2006–07 school year (the time of the 

study) reported that they offered activities in reading, and 94 percent of all centers offered 

activities in mathematics. Whether students are required to participate in these activities, 

however, varies by center.

Three-quarters of the centers reported that a typical student participated in reading 

activities (75 percent) and mathematics activities (81 percent) for less than 4 hours per 

week. 

Centers serving elementary school students reported that the average student spent the most 

amount of time per week doing homework in a group setting (36 percent) or participating in arts 

or recreation activities (33 percent) (Exhibit ES-1). One-quarter of centers reported that a typical 

student received instruction in reading or practiced reading skills, and 19 percent of centers 

indicated that a typical student engaged in mathematics activities for more than 4 hours per 

week. Thirty-six percent of centers reported that a typical student worked on homework in a 

group setting (which could also include reading and mathematics activities), and 33 percent said 

that the typical student was involved in arts/recreation activities for more than 4 hours per week. 

Because centers were open for about 16 hours per week, on average, student participation for 4 

hours per week in a particular activity represented 25 percent of the available time. 

 

1

 Reading enrichment activities are defined as structured activities designed to build students’ literacy skills. 

Reading enrichment may include scheduled time for independent student reading, writing and literacy enrichment 

activities but not homework assistance. However, homework assistance activities could include reading activities. 
Executive Summary xii

Exhibit ES-1 

Percentage of Centers Serving Elementary School Students That Reported Participation 

in Activities by a Typical Student, by Type of Activity and Amount of Time of Engagement 

per Week 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 5. 

n = 389 

Exhibit reads: Thirty-six percent of centers serving elementary school students reported that a typical student 

participated in homework activities for more than 4 hours per week. 

A majority of centers serving elementary school students reported that reading activities 

included the five essential components of reading instruction. 

Many centers serving elementary school students reported emphasizing comprehension, 

fluency, vocabulary, phonics and phonemic awareness in at least some activities. Seventy-four 

percent of centers serving elementary school students reported that they focused on 

comprehension in all or most instructional activities in reading, compared with 52 percent that 

concentrated on phonics skills in all or most activities. Observational data were consistent with 

the pattern reported by centers serving elementary school students: 86 percent of observed 

activities focused on comprehension, compared with 46 percent that focused on phonics. 

3%

6%

12%

19%

25%

33%

36%

15%

21%

31%

41%

47%

38%

36%

60%

58%

46%

36%

28%

28%

24%

23%

16%

11%

4%

1%

1%

3%

0% 20% 40% 60% 80% 100%

Health or social services

Science

Tutoring

Mathematics

Reading

Arts or recreation enrichment

Homework

Percent of Centers Reporting

Reported Student Activities

More than 4 hours

2-4 hours

Less than 2 hours

Not offered

Hours per Week
Executive Summary xiii

However, the observation data were collected from a small number of purposively sampled 

sites.2

 Thus the comparison may not be useful. 

Centers reported that mathematics activities focused on basic skills. 

Seventy-one percent of centers surveyed serving elementary school students reported 

stressing operations with whole numbers in all or most mathematics activities. Centers serving 

elementary school students also reported that they were more likely to engage students in tasks 

that required simple rather than complex problem solving. Sixty-eight percent of centers reported 

asking students to practice basic facts in all or most instructional activities in mathematics. 

Observational data from a small, purposive sample of case study sites were consistent with the 

pattern of emphasizing basic mathematics facts: 83 percent of observed activities involved 

practice with basic facts. 

Observers in case study sites found that staff providing instruction used active teaching 

strategies in academic activities, communicating goals clearly to students in most activities. 

However, staff were more likely to use multiple teaching strategies in reading than in 

mathematics. Staff providing instruction communicated the goals, purposes and expectations of 

activities to students more often in mathematics activities (89 percent) than in reading activities 

(60 percent). Staff providing instruction used multiple strategies in 22 percent of the mathematics 

activities observed, compared with 53 percent of the reading activities observed. 

Student Attendance in Center Activities 

Researchers have linked regular participation to better outcomes for students in after-school 

programs (e.g., Lauer et al., 2006). Although ED’s annual performance reporting guidelines 

define regular attendance as 30 days or more per year, the number of days required to have an 

effect on academic achievement is not known. 

Centers reported that about half of their students attended roughly 2 days a week or more. 

The study team asked a random subset of 140 centers in the study to report on student 

attendance and participation. Just 75 of the 119 centers (63 percent) that completed surveys 

indicated they could track these data. The centers that could track attendance indicated that 44 

percent of all center students attended 60 days (roughly 2 days per week) or more in the last year. 

 

2

 It is important to note that the study team asked center directors to obtain these data from someone familiar with 

the activities, but they may have responded without consulting an expert. Additionally, center directors may have 

had reasons to provide socially desirable responses to a U.S. Department of Education (ED) survey. 
Executive Summary xiv

More than half of all centers reported having policies that required students to attend at 

least 2 days a week, but attendance policies were not associated with greater attendance. 

Centers’ attendance policies varied. About half of all centers (56 percent) had policies 

requiring student attendance at least 2 to 3 days per week in order to remain enrolled in the afterschool program.3

 Thirty-eight percent of centers reported requiring attendance daily. Twenty-six 

percent did not require attendance. There were no significant differences in attendance 

requirements between school-based and nonschool-based centers. A higher percentage of centers 

serving elementary school students (41 percent) were more likely to have policies requiring 

attendance than those centers not serving this age group. In contrast, 31 percent of centers 

serving middle school students and 22 percent of centers serving high school students required 

attendance every day. Beyond basic attendance requirements, more than half of the centers 

required the participation of all students in specific academic activities, including homework 

help (73 percent), reading activities (60 percent), mathematics activities (58 percent) and tutoring 

(14 percent). 

In centers that required attendance, 66 percent of students attended for more than 30 days; 

62 percent of students attended for the same duration in centers that did not require attendance. 

No relationship was found between any kind of attendance policy and actual attendance for 

centers overall or for centers serving elementary school students. For the middle grades, centers 

that required attendance every day had higher attendance than those that did not require 

attendance. In high school, centers that required attendance 2 to 3 days per week had higher 

attendance than those that did not require attendance. 

Although attendance rates varied little by center type, elementary school students were 

more likely to attend center programming than older students. 

The pattern of higher attendance for elementary-serving centers than for secondary-serving 

centers mirrors that of the previous national evaluation. 

Centers that served elementary school students and had adequate tracking systems reported 

that 48 percent of students attended 60 days or more in the 2005–06 school year, or roughly 2 

days per week. Centers serving the middle grades indicated that 36 percent of students attended 

this often, and centers serving high school students cited 30 percent of their students’ meeting 

this attendance level. Researchers conducting the previous national evaluation found a similar 

pattern of results for elementary school students but not for middle school students: In their 

study, 55 percent of elementary students attended 51 or more days, but just 20 percent of middle 

school students attended that often (Dynarski et al., 2004). Their study used different methods to 

study attendance, however. 

 

3

 Although requiring attendance was not defined on the survey, respondents may have varied in their interpretations 

of this question, depending upon the consequences attached to nonattendance; therefore, data are included in a 

separate question about consequences of nonattendance. “Require attendance” may mean that center directors have a 

rule that only students who attend five days per week may enroll in the program; in other instances, requiring 

attendance could simply mean to center directors that there is a norm that students will attend regularly. 
Executive Summary xv

Organizational Supports for Instructional Quality 

Offering high-quality instruction in reading and mathematics requires recruiting, developing 

and retaining high-quality staff, as well as developing policies and programs that attract and 

retain students and reflect students’ academic needs. 

Centers reported that they relied primarily on part-time staff, who were unlikely to receive 

benefits from their work in the center. This latter finding is not surprising, as the centers 

are open for an average of only 16 hours per week. 

Seventy-six percent of program staff members in centers who led instructional activities 

reported working fewer than 20 hours per week. Centers infrequently offered job benefits for 

part-time staff members. Fourteen percent of centers reported offering a retirement savings plan 

to part-time staff, 11 percent offered paid time off for vacation and sick leave, 8 percent offered 

health insurance and 4 percent offered tuition reimbursement. It is important to note that parttime staff may include individuals who have full-time teaching positions in addition to working 

at the 21st CCLC center. 

To provide professional development to staff, about half of centers reported offering 

opportunities through training courses or conferences. 

Centers indicated that the school-day teachers they employed may have had increased 

opportunities for staff development through activities offered by their districts and schools than 

did other types of center employees. Sixty-two percent of centers offered other paid training or 

professional development to full-time staff, and 55 percent did so for part-time staff. Fifty-three 

percent of centers offered paid conference attendance to full-time staff; 39 percent offered this 

opportunity to part-time staff. In the case studies, center directors in school-based programs said 

they relied on professional development opportunities the staff received through the district. 

They also reported that they encouraged staff to share their ideas for innovative programming 

through weekly staff meetings that served as school-based professional development 

opportunities. 

Across all centers, 29 percent of staff had worked at the center for less than 1 year. 

About half of centers (48 percent) reported that the primary reason for staff turnover was 

graduation from school or completion of a program of study. Other commonly reported reasons 

for staff turnover were lack of benefits and the centers’ inability to offer full-time positions. 

Centers reported that more than two-thirds of staff providing instruction in reading and 

mathematics had prior experience as certified classroom teachers or as instructional 

specialists in reading or mathematics. 

Fifty-five percent of the staff who provided instruction in reading or mathematics had been or 

were, at the time of the study, regular classroom teachers, and 23 percent had been instructional 

specialists in reading or mathematics. Twenty-three percent of staff providing instruction in 

reading or mathematics were currently or had served as classroom aides. 
Executive Summary xvi

Centers reported that nearly two-thirds of reading and mathematics instructors had a 

bachelor’s degree or higher (64 percent for reading and 63 percent for mathematics). 

Centers reported that a little more than a third of their instructors reported having only a 

bachelor’s degree (32 percent for reading and 33 percent for mathematics). 

A majority of centers reported using assessment data to improve existing program 

offerings and evaluate program success. 

Seventy-one percent of centers reported having access to whole-school state assessment data. 

In addition, more than four-fifths reported that they received state assessment results for 

individual students at their centers (83 percent in reading and 82 percent in mathematics). 

Centers said that they used a variety of data to assess student academic growth, make program 

adjustments or evaluate program success periodically. Almost half (47 percent) of centers noted 

that once or twice a year they used results from tests administered at the students’ school, while 

34 percent of centers said that written reports from students’ teachers were used to assess 

academic growth once or twice annually. 

About 40 percent of centers reported some involvement with supplemental educational 

services (SES). Like 21st Century Community Learning Centers, SES is intended to 

provide after-school academic activities to students. A small percentage (9 percent) of 

centers said they coordinated their activities with SES providers. Fifteen percent of centers 

reported being authorized to provide SES themselves. 

On average, each center that was an SES provider reported offering supplemental 

instructional services to 38 students in reading and mathematics. Just 9 percent of all centers 

reported coordinating with one or two providers, and only 5 percent of all centers reported that 

their coordination activities focused on aligning schedules with the providers, while 4 percent 

indicated they coordinated their academic support activities with the providers. 

Just under one-third of all centers reported that coordination with staff from the schoolday instructional program was a challenge to implementing high-quality programming. 

However, the percentage was higher for nonschool-based centers. 

Lack of information about students’ academic needs, school-day teachers’ lack of 

responsiveness to requests from after-school staff for information and lack of information about 

the school-day curriculum were cited as barriers to implementing high-quality programming for 

22 percent to 32 percent of all centers. Barriers to obtaining information about student needs 

were greater for nonschool-based centers than for school-based centers. Thirty-six percent of 

nonschool-based centers reported lack of information as a barrier, compared to 20 percent of 

school-based centers. Barriers were also greater for nonschool-based centers with respect to 

responsiveness of school staff: 39 percent of nonschool-based centers identified lack of 

responsiveness as a barrier, compared to 29 percent of school-based centers. 
Executive Summary xvii

Conclusions 

Survey data and site observations indicate that 21st Century Community Learning Centers 

focused on reading and mathematics enrichment. Compared with the breadth of reading skills 

emphasized, mathematics instruction covered a narrower range of basic skills. Centers reported 

that 44 percent of students attended 60 days or more in the last year, which amounts to roughly 2 

days per week. Elementary school students attended after-school activities for more days in the 

school year than did middle and high school students. About half of the centers reported using 

data for a variety of purposes, including program evaluation and ongoing review of programming 

activities. The majority of centers reported having access to state assessment data results on 

individual students and many reported using this and other information to inform program 

practice. 

Although there were few differences between school-based and nonschool-based centers, 

school-based centers were more likely to report emphasizing higher-order skills such as asking 

students to make predictions about something they were reading and talking or writing about 

answers to questions related to something they had read. At the same time, nonschool-based 

centers had students read teacher-selected books more often than school-based centers did. 

The findings of this report suggest three challenges that centers face in implementing their 

programs: (a) staff departures after graduating from school or completing a program of study, as 

the lack of benefits makes it difficult to retain high-quality staff; (b) a lack of up-to-date 

information about students’ individual needs and (c) low attendance rates, the remedy for which 

requires more than simply having attendance policies. 

The study did not directly measure instructors’ knowledge or skills, but future studies could 

examine detailed measures of instructor knowledge to predict differences in instructional quality. 

Future studies could also examine the content of professional development provided for center 

staff. Finally, future work that examines the quality of reading programming could focus on how 

best to capture information on instruction in after-school programs since this study had 

challenges in this area. Future studies could also identify effective practices for improving 

attendance levels among participants. 

 
Executive Summary xviii
1. Study Background, Goals and Methods 1

1. Study Background, Goals and Methods 

Since the reauthorization of the Elementary and Secondary Education Act (ESEA) of 2002, a 

stronger emphasis has been placed on out-of-school learning to improve student academic 

achievement. As policy makers, schools and districts contemplate possible interventions, one 

option is to increase the amount of time spent on learning, such as through before- and afterschool programs. Additional hours provide more opportunities for instruction, academic 

assistance and enrichment or experiential activities to support learning. 

The body of rigorous research on the effects of extended learning time on student academic 

achievement is not extensive, but it suggests the potential for a positive effect on academic 

performance (e.g., Lauer et al., 2006; Zimmer et al., 2007). How the additional time is used is 

what matters. For example, a review of the literature on time and learning found that there was 

no relationship between allocated time and student academic achievement, some relationship 

between engaged time and achievement and a greater relationship between time spent on 

academic learning and achievement (Aronson, Zimmerman, and Carlos, 1999). Large-scale 

survey studies find a similar relationship between time spent on academics in specific content 

areas and academic achievement in those areas (Rowan, Correnti, and Miller, 2002). 

The 21st Century Community Learning Centers (21st CCLC) program is an example of a 

federal effort to improve academic achievement by extending learning opportunities. The 

program, which is funded by the U.S. Department of Education and administered by the states, 

seeks to provide opportunities for academic enrichment, youth development and other activities 

for students before and after school, as well as during the summer, and to offer the families of the 

students served opportunities for literacy and related educational development. Past program 

evaluations have generated questions about the impact of federally funded after-school services 

on student academic achievement. In the previous 2-year national evaluation study of the 21st 

CCLC program, which was conducted early in the program’s history, researchers found that 

attendance was irregular in many centers. In addition, the evaluators reported that academic 

opportunities in core subject areas such as reading and mathematics were limited and that 

participation in the program had no effect on student academic achievement (Dynarski et al., 

2003; Dynarski et al., 2004; James-Burdumy et al., 2005). 

The current study examined characteristics of programming, building on the findings from 

the earlier evaluation. In 2004, the U.S. Department of Education’s Policy and Program Studies 

Service contracted with SRI International and its partner, Policy Studies Associates, to evaluate 

the programming that the 21st CCLC program offers students and their families. One purpose of 

the evaluation was to provide evidence regarding how 21st CCLC program grantees, center 

directors and center staff implemented activities—in particular, academic activities focused on 

reading and mathematics. In addition, the study examined organizational supports that previous 

research suggests may be associated with high-quality programs as indicated, for example, by the 

hiring and retention of staff with experience and expertise relevant to their roles in the centers. 

This report presents the findings of the evaluation. 
1. Study Background, Goals and Methods 2

The 21st Century Community Learning Centers Program 

The Elementary and Secondary Education Act of 1965 (ESEA), as amended by the 

Improving America’s Schools Act of 1994 (P.L. 103-382), initially authorized the 21st CCLC 

program as a small discretionary grant program. In the authorizing legislation, Congress 

described local public schools—particularly those in rural areas and inner cities—as valued 

institutions to which communities already turned for educational and other essential services; 

thus, funding would go to school districts to develop centers. Centers were to provide continuing 

education and lifelong learning opportunities “to individuals of all ages” and were to help keep 

the country’s workforce “competitive and successful ... [in] the high technology, global 

economy of the 21st century.” Each center was required to offer no fewer than 4 of 13 authorized 

activities, which included: services for school-age students; senior citizen programs; children’s 

day care services; and employment counseling, training and placement for adults. In 1998, an 

absolute priority was added for “activities that offer expanded learning opportunities for children 

and youth in the community.” 

The 21st CCLC program underwent a significant transformation when Congress reauthorized 

it in January 2002 as Title IV, Part B, of the ESEA (P.L. 107-110). The reauthorization 

emphasized improving academic achievement for low-performing schools with high percentages 

of low-income students. 

The reauthorization also established a new set of authorized activities that centers could 

offer: 

 Remedial education activities and academic enrichment learning programs, including 

providing additional assistance to students to improve their academic achievement 

 Mathematics and science education activities 

 Arts and music education activities 

 Entrepreneurial education programs 

 Tutoring services (including those provided by senior citizen volunteers) and mentoring 

programs 

 Activities that emphasized language skills and academic achievement for students with 

limited English proficiency 

 Recreational activities 

 Telecommunications and technology education programs 

 Expanded library service hours 

 Programs that promoted parental involvement and family literacy 

 Programs that provided assistance to students who had been truant, suspended or 

expelled, to allow them to improve their academic achievement 

 Drug- and violence-prevention programs, counseling programs and character education 

programs 
1. Study Background, Goals and Methods 3

As significant as these changes to the goals and authorized activities were, changes relating 

to program administration following reauthorization were at least as important. Congress 

transferred the responsibility for administering the program from the U.S. Department of 

Education to each state when funding rose to $1 billion in fiscal year (FY) 2002. The U.S. 

Department of Education currently awards grants to states using a formula based on a state’s 

allocation under Title I, Part A, of the ESEA, which takes into account the state’s size and the 

proportion of students from low-income families. Local education agencies and a variety of 

community-based organizations, including for-profit businesses, nonschool-based organizations 

and faith-based organizations, are eligible to apply for subgrants from the state. Each state is 

required to submit an application for implementing the program, and the U.S. Department of 

Education is responsible for approving those applications. States then operate their own grant 

competitions, according to their approved applications, in which they determine the duration of 

program grants (between 3 and 5 years) and specify their own requirements for matching funds 

(up to a one-to-one maximum). 

Individual programs also have flexibility under the reauthorized program. States can allow 

programs (subgrantees) to apply to implement programs in various locations—for example, 

choosing to operate in elementary or middle school buildings, or in a location more accessible to 

students and their families. This policy change made it more likely that nonschool-based private 

and public entities could operate with support from the 21st CCLC program. In addition, 

applicants for subgrants have some flexibility in setting program goals. As part of their grant 

applications, subgrantees must identify the community need for their program and base their 

offerings on the need identified. As principal grantees, states are also required to select programand student-level outcomes. 

Although the federal guidelines specify a range of measures for which grantees must report 

data, states have considerable flexibility in determining how to hold local programs accountable 

for results. Certain states allow local programs to set program goals and measures within the 

bounds of federal and state priorities. Centers are required to report data on the Government 

Performance Results Act measures for the program through the Profile and Performance 

Information Collection System (PPICS) system. They are given flexibility on whether they 

report on grades or state assessment results, but they are required to report on one or the other. 

To monitor program progress, the U.S. Department of Education contracted with Learning 

Point Associates in October 2003 to develop the PPICS. This online data collection and reporting 

system gathers grantee- and center-level data aligned with overall program goals to enable local 

programs to report on progress toward the outcomes they set out to achieve as part of their 

grants. This study used the PPICS database to identify basic characteristics of all centers to 

construct the survey and case study samples. In addition, PPICS was the source of contextual 

information for the survey data and the case study sites. 

In FY 2009, Congress appropriated more than $1.1 billion for the 21st CCLC program. 

Currently, the federal program supports 1,585 local programs within more than 9,500 centers in 

53 states and U.S. territories. Roughly two-thirds of all centers serve students in each of the 

elementary grades; slightly fewer than half serve students in grades 6, 7 and 8 and one in six 

centers serve students in the high school grades. Exhibit 1-1 summarizes basic information about 
1. Study Background, Goals and Methods 4

funded centers that were in the population at the time of this study, i.e. 2005–06, and about 

centers targeted by the study. 

Exhibit 1-1 

Characteristics and Type of and Grade Levels Served by Funded and Sampled Centers 

Characteristic Funded Centers Centers in Sample 

Located in an urbanized area 74% 74% 

Minority population in area served 37% 36% 

Center Type 

Located in a school 92% 71% 

Located in a nonschool setting 8% 29%* 

Grade Level Served 

K 45% 45% 

1 57% 57% 

2 60% 60% 

3 66% 64% 

4 67% 64% 

5 68% 64% 

6 54% 54% 

7 39% 41% 

8 38% 41% 

9 13% 41% 

10 11% 13% 

11 11% 12% 

12 11% 12% 

* For this study, nonschool-based centers are overrepresented to facilitate the 

examination of differences between nonschool-based centers and school-based centers. 

Source: Profile and Performance Information Collection System database. 

n= 5,122 for funded; n = 516 for sample 

Exhibit reads: Seventy-four percent of centers are located in urbanized areas. 

Background 

This study attempts to analyze the extent to which activities funded by the 21st CCLC 

program have characteristics known to be associated with positive student academic 

achievement. Throughout much of their history, after-school programs have been defined by 

their diversity of organizational sponsors, voluntary youth participation, modest levels of adult 

direction and commitment to meeting the social and developmental needs of low-income youth 

(Halpern, 2002; Muller and Frisco, 1998). Over time, however, programs have shifted away from 

voluntary and toward mandatory participation, and they have increased the level of adult 

direction, especially with respect to homework help (Halpern, 2002). The pressure on schools to 

meet the academic needs of disadvantaged youth has also led to increased emphasis on 

academics in after-school programs (Brown, 1999). 

This study focuses narrowly on academic activities in two areas, reading and mathematics, in 

after-school programs and does not address practices associated with outcomes other than 

academic achievement. This narrow focus is significant, since research syntheses of 

experimental and quasi-experimental research studies suggest that after-school programs may 

produce positive outcomes in areas other than student academic achievement (Durlak and 

Weissberg, 2007; Lauer et al., 2006). The authors of these syntheses also conclude that programs 

do not have to be exclusively academic in focus to have positive impacts on academic 
1. Study Background, Goals and Methods 5

achievement. This study focuses on reading and mathematics activities, however, because the 

2004 program evaluation identified these areas as ones where programs might be strengthened. 

Evidence has shown that programs that take place outside the school day can help improve 

reading and mathematics achievement. Lauer and colleagues (2006) conducted a meta-analysis 

of studies that examined the effectiveness of after-school and summer strategies for improving 

achievement in reading and mathematics. Their study, conducted under contract for the U.S. 

Department of Education’s Institute of Education Sciences, was limited to studies that measured 

achievement in reading or mathematics and used comparison-group designs. They found small 

but significant positive effects of programs on achievement in both reading and mathematics, and 

larger positive effect sizes for programs with specific characteristics such as tutoring in reading. 

Conceptual Framework for the Study 

Because the enhanced focus on academics is relatively new in the field of after-school 

programming, few studies have been conducted of the program quality of after-school academic 

instruction. Until now, studies have principally relied on frameworks for studying youth 

development in community settings (e.g., Leffert, Benson, and Roehlkepartain, 1997; National 

Research Council & Institute of Medicine, 2002), rather than on frameworks that focused on 

specific academic content or instructional strategies. The framework that guided this study, 

however, drew on both research conducted in community settings and research conducted in 

schools about instructional quality. In this respect, the study framework was similar to Huang’s 

(2001) system of program quality indicators for middle and high school after-school programs, 

which were based on research carried out in both schools and after-school programs (see also 

Little, 2007). 

Each subsequent chapter first briefly describes the relevant components of the framework 

that guided the study (see Exhibit 1-2) and the research that supports the inclusion of each 

component of the framework. The chapters then set forth relevant study findings. It is important 

to note that while this study is informed by the broader universe of after-school programming, it 

is focused on only the 21st Century Community Learning Centers program, and its findings are 

not intended to be applied to other after-school programs. 

The framework below helped organize the presentations of findings, but it has some 

limitations. Its principal limitation is its narrow focus on those elements needed to promote 

participation in reading and mathematics instructional activities in 21st CCLC programs, which 

were the focus of the study. The framework is by no means intended to be inclusive of the broad 

range of elements of program quality that may be needed to improve academic achievement or 

related school outcomes, such as attendance. Nor does the framework encompass all research 

relevant to judging program quality in after-school programs. Research about which program 

elements contribute to social and emotional outcomes for youth, for example, is excluded from 

the framework because those findings fall outside the focus of the study. 
1. Study Background, Goals and Methods 6

Exhibit 1-2 

Framework for the Study: Supports for Student Participation in Quality Academic 

Instruction That Leads to Increased Achievement 

Exhibit reads: In the framework for the study, student participation in quality academic instruction is 

hypothesized to lead to desired student outcomes for the program. 

Chapter 24 addresses the components of high-quality academic instruction, which is one of 

the most important influences on academic achievement (Cohen, Raudenbush, and Ball, 2003): 

 Sufficient exposure to academics after school on a consistent basis. The frequency and 

intensity of student participation in academic activities at centers needs to be sufficient to 

influence their learning. 

 Focus on core academic content. Research in reading has identified core skills (e.g., 

phonemic awareness) that are essential to literacy. Likewise, research in mathematics has 

identified several content strands that build students’ mathematical proficiency (e.g., 

peer-assisted learning strategies, solving cognitively complex problems, using computers 

and graphing calculators to solve problems). 

 

4

 The information presented in this chapter is limited to the study’s elementary school findings. Data from programs 

serving middle and high school students are presented separately in Appendix A. Comparisons between schoolbased and nonschool-based centers are based on elementary-level findings. 

INSTRUCTIONAL 

SUPPORTS

Center Staffing 

and Policies 

Recruit and retain quality 

staff 

Provide opportunities for 

staff development 

Policies requiring 

attendance 

Alignment to 

Student Needs 

Coordinate activities with 

school and with support 

services 

Use of data for program 

improvement 

Adult Support 

Maintain positive 

relationship with youths 

QUALITY ACADEMIC 

INSTRUCTION 

Intense exposure to academics 

after school on a consistent 

basis 

Focus on core academic 

content 

Use of research-based 

instructional strategies 

Guidance from instructors with 

education and experience 

related to their roles 

Feedback to students and 

parents on academic progress

Student 

Participation 

Regular attendance 

Student 

engagement in 

academic activities 

Participant 

Outcomes 

Increased 

achievement 
1. Study Background, Goals and Methods 7

 Guidance from instructors with education and experience related to their roles. 

Instructors with backgrounds in helping students in reading and mathematics are likely to 

be better prepared to offer high-quality instruction matched to student needs in these 

domains. Staff members’ active guidance of student learning is likely to be more 

effective than passive supervision of students. 

 Feedback to students and parents on academic progress. Feedback to students helps 

develop their metacognitive skills and motivates improvement. Parent feedback supports 

caregiver involvement in student learning, a key component of academic success. 

Chapter 3 addresses the components of student participation necessary to affect academic 

achievement outcomes: 

 Regular attendance. Regular center attendance is necessary for students to benefit from 

after-school instruction. 

 Student engagement in academic activities. In addition to attendance, being on task and 

focused on academics when present support students’ motivation to learn and to 

appreciate the importance of developing proficiency in reading and mathematics. 

Chapter 4 addresses the necessary supports for high-quality instruction: 

 Center staffing and policies. Centers are responsible for providing pay and benefits at 

levels that help attract and retain qualified staff, and for providing opportunities for staff 

to develop and enhance skills relevant to their roles. 

 Alignment of activities to student needs. High-quality instruction is aligned to student 

needs. To understand student needs, after-school programs need to be well coordinated 

with school-day activities and use data from student assessments (or conduct their own 

student assessments) to adjust program offerings accordingly. 

 Adult support. After-school programs provide an opportunity to extend students’ learning 

beyond the school day, but the effectiveness of that help depends on the quality of the 

relationships between adults and students. 

Study Design and Data Collection Methods 

Evaluation Questions 

This evaluation of program quality addressed three main questions: 

1. What is the nature of activities in centers that are designed to promote academic 

development of students? 

2. How do centers differ with respect to regular attendance? 

3. How do center leaders staff their centers, coordinate with other service providers and 

allocate their resources to provide quality programming? 
1. Study Background, Goals and Methods 8

Study Design 

The evaluation included the following components: (a) a national survey of center directors, 

sent to a sample of 600 centers in 2006–07, (b) a focused telephone survey of center directors 

conducted among a subsample of 140 of those centers in 2006–07 and (c) case study visits to 12 

sites in fall 2006 and spring 2007. The focused telephone survey sample was nested within the 

survey sample, which was nationally representative of centers offering instruction in reading, 

mathematics, and technology (see explanation below on weighting). Approximately 70 percent 

of the sample consisted of school-based sites. Nonschool-based sites were deliberately 

overrepresented to enable statistical comparison with school-based sites.5

The following outline briefly describes the sources of data, data collection procedures and 

response rates, along with a brief explanation of how to interpret the data presented in the 

exhibits. (See Appendix C for information about the selection of samples for the national survey, 

focused survey and case studies.) 

Data Sources and Data Collection Procedures 

National survey. The survey provided data from a nationally representative sample of 600 

center directors to whom surveys were sent, of whom 516 responded. The survey elicited 

information regarding centers’ objectives and approaches and basic staffing information, as well 

as the indicators that they use to measure success, their methods for tracking outcomes and their 

methods for communicating with parents about student progress. In addition, the survey 

collected data on the organization of activities aimed at improving students’ reading and 

mathematics skills. In selecting centers, we did not distinguish among centers offering summer, 

before- or after-school programming. 

The surveys were mailed in September 2006 to the center director in charge of after-school 

services at each of the 600 selected centers. Survey responses were accepted until January 19, 

2007. After eliminating centers that could not be reached or were ineligible for the study (e.g., 

because funding had expired), the size of the sample for the national survey was 558. Directors 

of 516 centers completed the national survey, a response rate of 92.5 percent. 

Focused telephone survey. To provide more in-depth information on program quality, a 

focused telephone survey collected data from a nationally representative subsample of 140 center 

directors from the 600 centers surveyed by mail. The telephone survey gathered information 

about the extent to which centers tracked students’ level of proficiency on state tests in reading 

and mathematics, the targeting of services and centers’ coordination with local supplemental 

educational services providers. Questions were also asked about student attendance and the 

qualifications of center directors and staff. To collect data on staff qualifications, each center 

director used a method provided by the research team to randomly select three staff members. 

All centers were asked to provide data on the chosen staff members. If centers had fewer than 

three staff members, directors completed information regarding all staff members. Printed 

versions of the focused telephone surveys were mailed, along with the national surveys, to the 

 

5

 School-based sites are not always administered by schools or school districts and nonschool-based sites are not 

necessarily administered by a community-based or nonprofit organization. Of the 516 schools in our final sample, 

nonschool grantees administered 210 programs (101 in school settings; 109 in nonschool settings) and schools or 

school districts administered 306 programs (266 in school settings; 40 in nonschool settings). 
1. Study Background, Goals and Methods 9

140 selected centers, and respondents had the option of mailing in their completed surveys or 

communicating their responses by telephone during the data collection period. Survey responses 

were accepted until January 19, 2007. After eliminating ineligible and unreachable centers, 132 

centers were chosen for the telephone survey. Center directors completed 122 focused telephone 

surveys—a response rate of 92.4 percent. 

Site visits. The study team visited 12 sites between fall 2006 and March 2007. Eleven of the 

site visits were to elementary schools; the 12th was to a middle school site. Two sites were 

visited in each of California, North Carolina, South Dakota and Maryland, and four sites were 

visited in Florida. To ensure comparability of data collected across sites, all site visitors received 

training at the beginning of the study. Two researchers then visited each site, where they 

conducted program observations, document review and interviews with center directors, staff 

and school-day instructors. After the site visits, each pair of researchers completed a debriefing 

form to ensure that they had captured comparable data aligned with the research questions. 

Site visit protocols included interviews with grant directors, center directors, program staff 

and school personnel. The protocols also included a structured observation of academic and 

enrichment activities. The case studies provided additional evidence about the nature of 

academic instruction and about how well center practices aligned with policy goals for the 

program and with previous research on program quality. The observation form used in the study 

was adapted from an earlier study Policy Studies Associates conducted of after-school programs 

funded by The After School Corporation (TASC) (Birmingham, et al., 2005). That instrument 

included three constructs used in this study: active teaching strategies, teacher relationships and 

engagement. Four items comprised the active teaching strategies construct: communicating 

goals, using multiple teaching strategies, challenging students to move beyond current skill 

levels and asking students to expand on answers and ideas. For the teacher relationships 

construct, there were also four items: using positive behavior management techniques, showing 

positive affect toward students, attentively listening to and/or observing students and 

encouraging students to share ideas, opinions and concerns. Student engagement had one item. 

For each observation, the study team rated each of these items on a four-point scale, ranging 

from “did not occur” to “highly characteristic.” A concurrent validity study of this instrument 

conducted in 2006 found ratings on these constructs to be significantly correlated with student 

ratings of settings observed by researchers (Pechman, et al., 2008). Each item is reported 

separately but in a group with other items in the constructs identified in the earlier research; no 

weights were assigned to the items. The observation instrument can be found in Appendix F. 

Because this study focused on academic instruction, the study team added content to the 

observation form related to reading and mathematics (in Appendix F). The items added focus on 

reading skills and activities and mathematics skills and activities asked about in the national 

survey. For each activity, observers rated whether there was an explicit focus on the skills and 

activities, an implicit focus or no focus. The distinction between explicit and implicit focus is an 

important one for some after-school programs, which may make academic goals implicit in an 

effort to attract participants and increase engagement (Vadeboncoeur, 2006). A limitation of 

these observational constructs is that separate validity data are not available, as this was the first 

time the constructs were used. 
1. Study Background, Goals and Methods 10

SRI provided training to all case study researchers in the instrument, and researchers 

conducted joint observations in the field to establish the reliability of the instrument. The training 

session provided guidance on the definitions of categories used in the instrument; participants 

also reviewed a codebook with examples of types of activities to help them learn how to use it. A 

total of 13 joint observations took place on site visits. For the active teaching strategies, raters 

agreed on the exact rating in 64 percent of observations and were within one point on the fourpoint scale 94 percent of the time. For on-task ratings, raters agreed on an exact rating 69 percent 

of the time, and 100 percent of the ratings were within one point. Agreement was lower for 

reading skills (63 percent exact agreement) and activities (65 percent), but higher for 

mathematics content (96 percent) and skills (97 percent). 

Weighting 

All the survey data were weighted to reflect the full population of eligible centers and to take 

into account nonresponse. Centers included in the original sampling frame for this study had to 

meet all of the following four criteria: (a) receiving funding at the time the sample was drawn; 

(b) offering reading activities; (c) offering mathematics activities and (d) offering technologyrelated activities.6

 Nonschool-based centers were deliberately oversampled and were weighted to 

reflect their percentage in the population of funded centers through the variables discussed 

below. To arrive at conclusions about the national population of centers, the data were 

poststratified to reflect the full population of funded centers by using the variables discussed 

below. The study team also used Deming’s procedure to weight the respondents so that the 

weighted marginal counts were the same as in the universe of sites funded at the time of the 

study and in operation for at least a year. (For example, the weighted number of responding 

elementary schools was the same as the number of elementary schools in the universe.) 

The specific characteristics used to poststratify the respondents came from the PPICS 

database and from U.S. census data on characteristics of persons living in ZIP codes where 

centers were located. The variables from the PPICS database were the numbers of students 

served, program longevity, centers’ history of offering academic services, primary grade levels 

served and number of weeks per year centers were open. Data from U.S. census ZIP code files 

were the percentage of the ZIP codes’ population living in an urbanized area and the percentage 

of minority population in the area. 

Limitations of the Study 

One limitation of the study pertains to the representativeness of the sample. Because of the 

original emphasis on the quality of instructional services in reading, mathematics and 

technology, the study team excluded sites that did not offer all those services. Although nearly 

all sites offered reading and mathematics activities, 19 percent of centers did not offer 

technology services and thus were excluded from the sample. 

The specific characteristics used in poststratifying the respondents, using the PPICS database 

and U.S. census data from ZIP codes where centers were located, equalized the weighted sample 

and the universe of centers in regard to all characteristics expected to influence survey responses. 

The reweighting thus essentially eliminated the biases caused by these variables. A potential 

 

6

 Centers provided self-reported answers to these questions through the PPICS database. 
1. Study Background, Goals and Methods 11

limitation is that reweighting did not eliminate biases not associated with these variables, nor did 

it eliminate biases that were functions of variable interactions, such as between the size of the 

community served and a center’s academic offerings. 

A second limitation was the study’s heavy reliance on self-reported data. Self-reported data 

on instruction can be inaccurate, and observation data do not always confirm what instructors 

report they are doing in their classrooms (Burstein et al., 1995). One source of bias is instructors’ 

memory, which has been found to be poor for instructional events that are rare or occur 

infrequently (Hoppe et al., 2000; Rowan, Camburn, and Correnti, 2004; Sudman and Bradburn, 

1982). Response choices can also influence survey respondents’ answers, either by cueing 

respondents to supply socially desirable answers or by suggesting estimation strategies that 

systematically bias the obtained reports (Schwartz, 1999; Schwartz and Oyserman, 2001). 

Questions about frequency of events or the number of hours spent in certain academic activities 

may be particularly biased, because respondents tend to answer these questions quickly at a cost 

of accuracy (Burton and Blair, 1991; Marquis, Marquis, and Polich, 1986). 

The study team sought to reduce the impact of self-report bias by using established 

techniques. The questionnaire asked center directors to ask their instructors about specific 

pedagogies and behaviors, a strategy known to increase the reliability of teacher self-reported 

data (Garet, et al., 2001; Herman, Klein and Abedi, 2000; Koziol and Burns, 1986; Mullens, 

1998; Ross et al., 1997). In addition, respondents were asked to skip items if earlier items on the 

survey established that respondents could not report accurately. If, for example, respondents to 

the telephone survey reported that they did not keep accurate records of the number of 

individuals who attended in the past week, they were asked not to report on the number of 

students who attended 90 days or more in the past year. This approach resulted in some 

additional missing data, but it likely reduced the bias in the responses obtained. 

When possible, the team checked the consistency of reporting across questions and against 

data reported by centers in their annual reports. For example, centers provided information about 

their operating hours in annual reports, and they also answered questions about the number of 

hours typical students participated per week. Despite these precautions, bias may still have 

affected the responses to some items, particularly those related to academic instruction. Center 

directors who were mailed the questionnaire may not have asked their instructors to complete it 

as instructed, but instead may have completed the surveys themselves. Center directors, given the 

roles they play in centers, may be reliable informants about record keeping, data use and 

organizational supports for program quality. They rarely provide instruction to students, 

however, so they may not have been good sources for information about instruction. Because we 

were able to compare survey results on instruction with observation results for the centers 

serving elementary school students, we have focused in Chapter 2 of this report on findings for 

centers serving elementary students. Descriptions of academic instruction for centers serving 

middle and high school students appear in Appendix A. Appendix E provides detailed statistics 

for data presented in the tables in Chapter 2. 

A third limitation is that the study team did not distinguish among before-, after- and summer 

school programs. This limitation is important, because a review by Lauer and colleagues (2006) 

found that summer programs were more likely to emphasize academics than programs offered 

during the regular school year. Because this study does not distinguish among different types of 
1. Study Background, Goals and Methods 12

programs, the data collected in it may be of limited utility in addressing concerns about 

attendance raised within the 21st CCLC program. 

There are also three important limitations of the observation data. First, the lower agreement 

among ratings for reading skills and activity implies these data are less reliable than data 

associated with the other ratings. Raters had difficulty agreeing on which skills were explicit and 

which ones were implicit; thus, ratings of reading activities were measured with a higher level of 

error than other ratings and should be interpreted with caution. Second, observations were 

purposively sampled, so the data collected are not nationally representative and observation 

protocols used have not been related to any outcomes of importance to the program. We initially 

sampled sites for the study using the promising practices framework (Luce and Thompson 2005). 

The evaluation team analyzed the data from the case studies with the intent of comparing 

promising versus typical sites along such dimensions as center goals, staffing and professional 

development; attendance patterns and policies; academic program offerings and resource 

management. However, the analysis yielded few meaningful differences between typical and 

promising centers; therefore, the focus of data analysis shifted to examining consistency between 

observation and self-reported survey data. Because the observations were few and purposively 

sampled, we cannot draw inferences from these data about program practices nationwide. Data 

from the observations can be used, however, to qualitatively examine whether the patterns are 

similar between the observation and survey data. Descriptions of observation ratings for site 

visits appear in Appendix B. A third limitation is that although analyses were conducted that 

demonstrated the consistency of the OST scales, no analyses have been conducted to establish 

the predictive validity of the scales (i.e., there is no evidence about the relationship between the 

measure and outcomes of interest such as youth behavior and academic outcome measures). 

Presentation of Survey Data in the Report 

Throughout the report, differences among different types of centers are noted. Unless 

otherwise stated, all differences reported in the text are statistically significant. Statistically 

significant differences are identified by asterisks and the inclusion of p values in the report’s 

exhibits. “School-based” and “nonschool-based” refer to the location where grantees provide 

services for students. Moreover, although some centers serve only a single school level 

(elementary school, middle school or high school), many centers serve multiple school levels. 

All comparisons related to levels, therefore, analyze differences between centers serving a 

particular level (e.g., elementary) and those not serving that level (e.g., nonelementary). 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 13 

2. Reading and Mathematics Activities in Centers Serving 

Elementary School Students 

Key Findings 

The following are key findings for centers serving elementary school students. Findings for 

centers serving middle and high school students appear in Appendix A. 

 Sufficient exposure to academics. To have an impact on academic outcomes, centers need 

to focus on academics, and students need to participate in academic activities at centers at 

a frequency that is sufficient to influence their learning (Dynarski et al., 2003; Fleming 

and Zhang, 2005). Grossman and colleagues (2002) found that homework help activities 

implemented well provided adult support to students and helped them to build academic 

skills. A recent meta-analysis found that after-school programs that provided one-on-one 

tutoring to students had positive effects on student academic achievement. The effect 

sizes ranged from 1 and 4/5 of a standard deviation, which are relatively large effects for 

educational programs (Lauer et al., 2006). More than 70 percent of centers serving 

elementary school students reported their students participated in reading activities on a 

weekly basis for at least two hours per week; 60 percent reported their students 

participated weekly in mathematics activities for at least two hours per week. Centers 

reported that a typical student spent 4 or more hours per week on reading activities (25 

percent of centers) and mathematics activities (19 percent of centers). Centers also 

reported that a typical student spent 4 or more hours per week working on homework (36 

percent of centers) and tutoring (12 percent of centers), both of which could include 

reading or mathematics activities. None of the observed activities involved one-on-one 

tutoring, but more than half of (58 percent) of observed reading activities involved some 

small-group (three to five students) tutoring activities (n = 23). 

 Focus on core academic content. Research in reading has identified core skills that are 

essential to literacy: phonemic awareness, phonics, vocabulary, fluency and 

comprehension (Snow, Burns and Griffin 1998). Likewise, research in mathematics has 

identified content strands that build students’ mathematical proficiency: number sense, 

operations, measurement, geometry, data analysis, statistics, probability and algebra 

(National Research Council, 2001, 2005). The data from surveys indicated that reading 

activities in centers serving elementary school students included all skills emphasized by 

the National Reading Panel (2000). Instruction in mathematics covered a narrower range 

of skills, with little emphasis on more complex problem solving. 

 Guidance from instructors. Instructors with backgrounds in helping struggling students in 

reading and mathematics are likely to be better prepared to offer high-quality instruction 

matched to student needs (Intercultural Center for Research in Education & National 

Institute on Out-of-School Time, 2005). Staff members’ active guidance of student 

learning is likely to be more effective than passive supervision of students (Brophy and 

Good, 1986; Good and Brophy, 2008; Rowan et al., 2002). Observers in case studies 

found that instructors in centers serving elementary school students used active teaching 

strategies in academic activities, communicating goals clearly to students in most 

activities (in 60 percent of reading activities and in 89 percent of mathematics activities). 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 14 

However, staff were less likely to use multiple teaching strategies in observed 

mathematics activities than in observed reading activities. The case studies were 

conducted in a small number of purposively sampled schools, so the data are not 

representative of all 21st CCLC sites. 

 Feedback to students and parents. Feedback to students helps develop their 

metacognitive skills and motivates improvement (Butler and Winne, 1995). Parent 

feedback supports caregiver involvement in student learning, a key component of 

academic success (Epstein, 2008; Sheldon, 2008). Most centers serving elementary 

school students reported providing frequent verbal feedback to students and parents, but 

written feedback was less common. 

Exposure to Academics After School 

Center activities need to provide sufficient exposure to academics if they are to increase the 

chances of improving student academic achievement (Dynarski et al., 2003; Fleming and Zhang, 

2005). Reviews of academic time-on-task in schools (Wang, Haertel and Walberg, 1993) as well 

as evaluations of after-school programs (Baker and Witt, 1995; Huang, et al., 2000) have found a 

relationship between sufficient exposure to academic content and a variety of positive academic 

outcomes. Lauer and colleagues’ (2006) meta-analysis found that reading programs with 

durations of 44 to 84 hours and mathematics programs with durations of 46 to 75 hours had the 

largest effect sizes. They also found that more exposure does not always result in higher 

academic achievement, making it difficult to say how much exposure to academics in an afterschool program is ideal to improve academic achievement. 

According to the PPICS database, only 2 percent of all centers reported that they did not 

intend to offer activities that focused on reading, and 6 percent did not intend to offer activities 

that focused on mathematics. By contrast, although all the centers included in the survey sample 

intended to offer services in both reading and mathematics, of the surveyed centers serving 

elementary school students, 1 percent indicated that they did not offer reading activities and 4 

percent did not offer mathematics activities. Not all students in a center offering these activities 

are required to participate in both of these activities. 

Surveyed centers serving elementary school students reported that a typical student 

spent time weekly in academic support activities like homework and in reading or 

mathematics activities. Centers reported that a typical student participated in homework help 

(36 percent), reading activities (25 percent), mathematics activities (19 percent) and tutoring (12 

percent of centers) for more than 4 hours per week (see Exhibit 2-1; see also Exhibit E-1 in 

Appendix E). Additionally, 33 percent of centers reported that a typical student engaged in arts 

or recreation activities for more than 4 hours weekly. Because centers were open slightly less 

than 16 hours per week, on average, student participation of 4 hours per week in a particular 

activity amounted to about 25 percent of the available time. These data are different from and 

more detailed than those recorded in PPICS; one reason may be that PPICS data used for 

identifying the sample were recorded the year before the study team fielded the survey. 

 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 15 

Exhibit 2-1 

Percentage of Centers Serving Elementary School Students That Reported Participation 

in Activities by a Typical Student, by Type of Activity and 

Amount of Time of Engagement per Week 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 5. 

n = 389 

Exhibit reads: Thirty-six percent of centers serving elementary school students reported that a typical student 

participated in homework activities for more than 4 hours per week. 

Reading Activities 

Reading Skills Emphasized 

Proficiency in reading requires a range of skills, including awareness of print features, 

phonemic awareness, phonics, vocabulary, fluency and comprehension (Snow et al., 1998). 

Concentrating on only one of these reading skills is likely to be insufficient to improve reading 

achievement because evidence indicates that decoding and language comprehension skills make 

independent, but related, contributions to reading comprehension (Oakhill, Cain and Bryant, 

2003). 

3%

6%

12%

19%

25%

33%

36%

15%

21%

31%

41%

47%

38%

36%

60%

58%

46%

36%

28%

28%

24%

23%

16%

11%

4%

1%

1%

3%

0% 20% 40% 60% 80% 100%

Health or social services

Science

Tutoring

Mathematics

Reading

Arts or recreation enrichment

Homework

Percent of Centers Reporting

Reported Student Activities

More than 4 hours

2-4 hours

Less than 2 hours

Not offered

Hours per Week
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 16 

A majority of centers serving elementary school students reported that most or all 

reading activities7 included the skills emphasized by the National Reading Panel (2000).

Center directors provided these data, which the study team asked them to obtain from someone 

familiar with the activities. Among centers serving elementary students, the largest percentage 

reported focusing on comprehension and vocabulary in all or most activities (74 percent and 69 

percent, respectively) (see Exhibit 2-2; see also Exhibit E-2 in Appendix E). 

Exhibit 2-2 

Percentage of Centers Serving Elementary School Students That Reported Reading 

Activities Emphases, by Reading Skill Emphasized and Inclusion Level

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 22. 

n = 388 

Exhibit reads: Seventy-four percent of centers serving elementary students reported emphasizing comprehension 

in most or all instructional activities in reading. 

Reading Skills Emphasized in Observed Activities 

Site visit observations of centers that served elementary school students, though not 

nationally representative, were consistent with the pattern of the survey data. Of the 35 reading 

activity observations conducted, 86 percent stressed reading comprehension, and 71 percent 

emphasized vocabulary (see Exhibit 2-3). The observation data revealed a greater emphasis on 

fluency than did the survey data: 77 percent of observed reading activities focused on fluency, 

but 59 percent of centers reported that all or most reading activities emphasized fluency (see 

Exhibit 2-2). 

 

7

 Reading enrichment activities are defined as structured activities designed to build students’ literacy skills. 

Reading enrichment may include scheduled time for independent student reading, writing and literacy enrichment 

activities, but not homework assistance. 

49%

52%

59%

69%

74%

43%

41%

35%

28%

24%

8%

7%

6%

3%

3%

0% 20% 40% 60% 80% 100%

Phonemic awareness

Phonics

Fluency

Vocabulary

Comprehension

Percent of Centers Reporting

Reading Skills Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 17 

Exhibit 2-3 

Percentage of Observations of Reading Activities in 

Centers Serving Elementary School Students, by Reading Skill Emphasized 

Source: Site visit observations. 

n = 35 observations 

Exhibit reads: Eighty-six percent of observed reading activities in centers serving elementary school students 

focused on developing students’ comprehension skills. 

Use of Instructional Strategies for Teaching Vocabulary, Fluency and 

Comprehension 

The National Reading Panel (2000) summarized existing rigorous evidence on strategies for 

the teaching of vocabulary, fluency and reading comprehension.8 To develop vocabulary skills, 

children need rich, explicit instruction (Beck, McKeown and Kucan, 2002), but children can also 

learn vocabulary incidentally, as part of authentic reading activity (National Reading Panel, 

2000). Both forms of instruction may be effective because children need extensive practice and 

multiple exposures to words in order to learn them (Coyne, et al., 2004). Frequent opportunities 

to read aloud with guidance from staff, peers or parents have been found to have a significant 

and positive effect on word recognition, fluency and comprehension across a range of grade 

levels (Coyne, et al., 2004). Finally, research suggests that strategies such as asking students to 

recall a text, answer and generate questions about it, make predictions about it and summarize it, 

improve comprehension (Biancarosa and Snow, 2006; National Reading Panel, 2000). Recent 

experimental research has also shown the advantages of selecting books at students’ reading 

level for them to read independently (Borman and Dowling, 2006). 

 

8

 This study focused on these components of reading in order to select survey items from the National Assessment of 

Educational Progress Teacher Background Survey related to reading instruction, and to design interview items that 

could measure the extent to which programs addressed the major processes involved in learning how to read, 

according to the findings of the National Reading Panel. 

26%

46%

71%

77%

86%

0% 20% 40% 60% 80% 100%

Phonemic awareness

Phonics

Vocabulary

Fluency

Comprehension

Percent of Observations

Reading Skills Emphasized
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 18 

Although 99 percent of centers serving elementary school students reported using at 

least one strategy in reading, the most commonly reported reading activities were designed 

to foster more independent reading and greater comprehension. Roughly half of centers 

serving elementary school students indicated that in all or most reading activities, they asked 

students to explain or support statements about something they read (50 percent) or had students 

talk about or write answers to questions on something they had read (42 percent). More than a 

third of these centers also reported asking students to make predictions about a text they were 

about to read (38 percent), or to talk with peers about what they had read (36 percent) in all or 

most activities (see Exhibit 2-4; see also Exhibit E-3 in Appendix E). 

Exhibit 2-4 

Percentage of Centers Serving Elementary School Students That Reported Student 

Participation in Instructional Strategies in Reading Activities, by Strategy and Inclusion 

Level

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 21. 

n = 388 

Exhibit reads: Fifty percent of centers serving elementary school students reported having students explain or 

support their understanding of what they had read in most or all reading activities. 

18%

19%

29%

30%

31%

35%

35%

36%

36%

38%

42%

50%

72%

53%

57%

60%

62%

55%

57%

62%

57%

51%

52%

43%

10%

28%

14%

10%

8%

10%

7%

2%

7%

11%

6%

7%

0% 20% 40% 60% 80% 100%

Read books selected by the leader

Describe the style or structure of a text they have 

read

Discuss different interpretations of what they 

have read

Do a group activity or project about what they 

have read

Write about something they have read

Make generalizations and draw inferences about 

something they have read

Read aloud

Talk with peers about what they have read

Help other students understand new words

Make predictions about something they are 

reading before reading or as part of reading

Talk or write about answers to questions about 

something they have read

Explain or support their understanding of what 

they have read

Percent of Centers Reporting

Reading Activity Strategies

Most or all activities

Some activities

None or almost none

Inclusion Levels
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 19 

School-based centers were more likely than nonschool-based centers to report that their 

instructors asked students to make predictions about something they were reading, and talk about 

or write answers to questions related to something they had read. Observation data, while not 

nationally representative, were generally consistent with the survey data. Two instructional 

strategies were found in more than half of the observations conducted on site visits: having 

students help other students understand new words (80 percent) and asking students to make 

generalizations (63 percent) (see Exhibit 2-5). In contrast, as shown previously in Exhibit 2-4, 

only slightly more than a third of centers reported that most or all reading activities employed 

these strategies. At the same time, the proportions of observed activities in which students were 

asked to make predictions (49 percent) and read aloud (46 percent) were generally aligned with 

the survey data for these activities. 

Exhibit 2-5 

Percentage of Observations of Reading Activities 

in Centers Serving Elementary School Students, by Instructional Strategy Emphasized 

Source: Site visit observations. 

n = 35 observations 

Exhibit reads: Eighty percent of observations of reading activities in centers serving elementary school students 

focused on students helping other students understand new words. 

 

9%

14%

29%

37%

46%

49%

63%

80%

0% 20% 40% 60% 80% 100%

Write answers to structured questions about 

something they have read

Describe the style or structure of a text they have 

read

Discuss different interpretations of what they have 

read

Do a group activity or project about what they have 

read

Read aloud to peers or staff

Make predictions about something they are reading

Make generalizations and draw inferences about 

something they have read

Work on understanding new words

Percent of Observations

Reading Instructional Strategies Emphasized

Work on understanding new words

Make generalizations and draw inferences about 

something they have read

Make predictions about something they are reading

Read aloud to peers or staff

Do a group activity or project about what they have read

Discuss different interpretations of what they have read

Describe the style or structure of a text they have read

Write answers to structured questions about something 

they have read
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 20 

Mathematics Activities 

Mathematics Skills Emphasized 

The content of mathematics instruction is a significant predictor of gains in student 

achievement in mathematics (Gamoran, et al., 1998; Porter, 2002; Rowan et al., 2002). Syntheses 

of research have also pointed to the central importance of content in mathematics, drawing 

attention, for example, to the foundational role of numbers and procedures with numbers in 

mathematics (National Research Council, 2001, 2005). Instruction needs to emphasize more than 

just one strand of mathematics content, however; it is also important for students in the 

elementary and middle grades to learn about the foundations for algebra, which include a focus 

on fractions, measurement and geometry (National Research Council, 2001; National 

Mathematics Advisory Panel, 2008, pp. xvii, 18). The National Mathematics Advisory Panel 

recommends that the content should build over grade levels; failure to address any of the 

foundations, even in the early grades, may cause many students to fall behind (Roschelle, et al., 

2009). In addition, large-scale studies of mathematics instruction have shown that the use of 

more rigorous content was a significant predictor of gains in student academic achievement 

(Gamoran et al., 1998; Rowan et al., 2002). 

Mathematics instruction focused on a relatively narrow range of content strands. 

Among centers serving elementary students, 71 percent reported emphasizing operations such as 

addition, subtraction, multiplication and division in all or most activities9

 (see Exhibit 2-6; see 

also Exhibit E-4 in Appendix E). Fifty-nine percent reported that number sense (i.e., 

understanding of numerical relationships as expressed in ratios, proportions and percentages) 

was stressed in all or most activities. Operations with decimals or fractions, a key target of 

instruction according to the National Mathematics Advisory Panel, ranked third as an area of 

emphasis in center programming. Fewer activities targeted complex mathematics: 15 percent of 

centers emphasized algebraic concepts, and 8 percent focused on data analysis in all or most 

activities. 

 

9

 Mathematics activities are defined as structured activities designed to build students’ mathematics skills. 

Mathematics activities may include scheduled time for independent problem solving and instructor-led enrichment 

activities, but not homework assistance. 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 21 

Exhibit 2-6 

Percentage of Centers Serving Elementary School Students That Reported Emphases in 

Math Activities, by Type of Content Emphasized and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 24. 

n = 387 

Exhibit reads: Seventy-one percent of centers serving elementary school students emphasized operations with whole 

numbers in most or all mathematics activities. 

School-based centers were more likely than nonschool-based centers to emphasize advanced 

topics in mathematics. In particular, school-based centers were significantly more likely than 

nonschool-based centers to have at least some emphasis on geometry (84 percent versus 70 

percent), algebra (75 percent versus 59 percent) and data analysis (70 percent versus 60 percent). 

However, from both types of centers, the reported focus on foundational topics such as number 

sense and operations was similar. 

Mathematics Content Emphasized in Observed Activities 

Observation data from the site visits, while not nationally representative, generally showed 

that centers primarily focused on teaching operations with whole numbers and number sense 

(see Exhibit 2-7). The emphasis on these basic skills may reflect a focus on building a strong 

foundation in basic math facts. In an interview at one of the centers visited, the mathematics 

instructor explained, “Because these kids are struggling, they need explicit systematic 

instruction. They have to get their foundation solid first.” In addition, a center’s ability to deliver 

instruction in higher-order mathematics may be limited by the knowledge of its tutors and staff. 

8%

12%

15%

22%

22%

59%

71%

62%

70%

58%

75%

68%

36%

28%

31%

17%

27%

3%

10%

5%

2%

0% 20% 40% 60% 80% 100%

Data analysis

Geometric concepts or geometry

Algebraic concepts or algebra

Measurement

Operations with decimals or fractions

Number sense

Operations with whole numbers

Percent of Centers Reporting

Math Activity Content Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 22 

In another center, one classroom teacher commented that tutors struggled with the level of 

content knowledge necessary to successfully assist older students with more complex 

mathematics problems, but did not experience the same challenges in literacy or language arts. 

Exhibit 2-7 

Percentage of Observations of Mathematics Activities in 

Centers Serving Elementary School Students That Emphasized Skills, by Type of Math 

Skill Emphasized 

Source: Site visit observations. 

n = 18 observations 

Exhibit reads: Eighty-three percent of observed mathematics activities in centers serving elementary schools focused 

on developing students’ skills in performing operations with whole numbers. 

Use of Instructional Strategies for Teaching Mathematics 

In mathematics, research on effective instructional strategies has focused on how best to 

develop the full range of skills students need to be mathematically proficient (National Research 

Council, 2001, 2005; National Mathematics Advisory Panel, 2008). One strand of research has 

examined the relationship between student academic achievement and the cognitive complexity 

of tasks instructors in regular classrooms assign to students. Cognitive complexity refers to the 

demands instructors place on students to engage in thinking about particular academic content. A 

low-complexity task in mathematics might involve asking students to memorize facts, definitions 

or formulas; in contrast, a high-complexity task might involve asking students to prove a 

theorem or solve a nonroutine problem that requires many steps (Porter, 2002). Although an 

emerging consensus views both procedural fluency and conceptual understanding as 

fundamental, interwoven aspects of mathematical proficiency (National Research Council, 

2001), strong experimental and correlational evidence indicates that when instructors require 

more complex tasks of students in a particular content area, student academic achievement gains 

are higher (Carpenter, et al., 1989; Gamoran et al., 1998). 

6%

6%

22%

29%

78%

83%

0% 20% 40% 60% 80% 100%

Operations with decimals or fractions

Data analysis

Measurement

Algebraic concepts

Number sense

Operations with whole numbers

Percent of Observations

Math Skills Emphasized
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 23 

Other evidence for effective strategies in mathematics comes from the National Mathematics 

Advisory Panel (2008), which was established to promote American students’ knowledge of 

mathematics and improve their performance in the subject. Having analyzed relevant research, 

the panel concluded that providing students the opportunity to explain the reasoning behind a 

problem solution helps to build proficiency in computation and translation of word problems and 

helps teachers to ensure that students have the conceptual framework necessary for 

understanding grade-level mathematics. The National Mathematics Advisory Panel (2008) also 

concluded that cooperative learning strategies, including peer-assisted learning strategies and 

heterogeneous grouping, have significant effects on student outcomes, particularly in teaching 

young children mathematical operations. Finally, other reviews of experimental research studies 

indicate that computers and graphing calculators can positively affect mathematics achievement 

(Dixon, et al., 1998). 

Although 99 percent of centers serving elementary school students reported using at 

least one strategy, the most common strategies used in all or most mathematics activities 

were practicing mathematical facts (68 percent) and asking students to explain their 

reasoning in solving a problem (46 percent). Twenty-four percent of centers reported having 

students use computers to support mathematics instruction or to solve open-ended problems in 

mathematics (see Exhibit 2-8; see also Exhibit E-5 in Appendix E). 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 24 

Exhibit 2-8 

Percentage of Centers Serving Elementary School Students That Reported Student 

Participation in Instructional Strategies in Math Activities, 

by Strategy and Inclusion Level

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 23. 

n = 388 

Exhibit reads: Sixty-eight percent of centers serving elementary school students reported asking students to 

practice basic mathematics facts in all or most instructional activities in mathematics. 

Mathematics Instructional Strategies in Observed Activities 

Observation data, though not nationally representative, showed a pattern similar to the survey 

data. Consistent with the mathematics strategies that centers reported on the survey, practicing 

basic mathematics facts was a strategy in 83 percent of the observed mathematics activities. 

Solving problems in which students had to select a method was assigned in 47 percent of 

observed activities, and students had to explain the reasoning behind their problem-solving 

strategy in 47 percent of observed activities. There were a few differences, however. For 

example, site visitors reported that students engaged in more complex tasks in mathematics than 

reported on the survey, but were asked to write mathematical equations less often than reported 

(see Exhibit 2-9). 

2%

18%

24%

24%

30%

42%

46%

68%

27%

71%

61%

54%

53%

50%

47%

29%

70%

11%

15%

22%

17%

8%

6%

3%

0% 20% 40% 60% 80% 100%

Use graphing calculators to solve problems

Make charts, tables, or graphs

Solve problems that have many possible 

correct answers

Use computers to practice mathematics

Write mathematical equations to solve a 

problem

Solve problems in which students have to 

figure out what method to use to solve them

Explain the reasoning behind how they 

solved a problem

Practice basic mathematics facts

Percent of Centers Reporting

Instructional Strategies in Math

Most or all activities

Some activities

None or almost none

Practice basic mathematics facts

Explain the reasoning behind how they solved a 

problem

Solve problems in which students have to figure 

out what method to use to solve them

Write mathematical equations to solve a problem

Use computers to practice mathematics

Solve problems that have many possible correct 

answers

Make charts, tables or graphs

Use graphing calculators to solve problems

Inclusion Levels 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 25 

Exhibit 2-9 

Percentage of Observations of Mathematics Activities in Centers 

Serving Elementary School Students That Emphasized Instructional Strategies, 

by Strategy Emphasized 

Source: Site visit observations. 

n = 18 observations 

Exhibit reads: Eighty-three percent of elementary school center mathematics activities observed involved students 

practicing basic mathematics facts. 

12%

12%

24%

35%

47%

47%

83%

0% 20% 40% 60% 80% 100%

Make charts, tables or graphs

Write mathematical equations to solve 

problems

Solve problems that have many possible 

correct answers

Use computers, calculators to practice 

basic mathematics skills

Explain the reasoning behind how they 

solved a problem

Solve problems in which they figure out what 

method to use

Practice basic mathematics facts

Percent of Observations

Instructional Strategies in Math

Practice basic mathematics facts

Solve problems in which they figure out what 

method to use

Explain the reasoning behind how they solved a 

problem

Use computers, calculators to practice basic 

mathematics skills

Solve problems that have many possible correct 

answers

Write mathematical equations to solve problems

Make charts, tables or graphs
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 26 

Instructor Education and Experience 

Research on after-school programs suggests that when students receive guidance from 

instructors with education and experience relevant to their roles, students are more likely to 

benefit from program activities. A case study comparing after-school programs assessed to be 

“high quality” and “low quality” found that these programs differed on key program features like 

staff commitment, amount of staff training and staff background (Vandell, et al., 2004). A study 

of after-school programs in Massachusetts found that the programs that demonstrated higher 

outcomes relied on staff with high levels of educational attainment and often employed a number 

of instructors with teaching certificates (Intercultural Center for Research in Education & 

National Institute on Out-of-School Time, 2005). In addition to providing relevant expertise, 

certified instructors who also teach after-school participants during the regular school day may 

have a deeper knowledge of students’ needs (Little, 2006). 

Research on the need for certified instructors to offer high-quality instruction is by no means 

definitive, however. For example, a large-scale study of elementary schools found no 

relationship between student academic achievement gains and special certification for instructors 

for either reading or mathematics (Rowan et al. 2002). It is also important to note that the 

authorizing legislation for the 21st CCLC program does not require centers to use highly 

qualified teachers. 

Centers serving elementary school students reported that more than two-thirds of staff 

providing instruction in reading and mathematics had prior experience as certified 

classroom teachers or as instructional specialists in reading or mathematics. Fifty-two 

percent of the staff who provided instruction in reading and mathematics were at the time of the 

study or in the past regular classroom teachers, and 16 percent had been instructional specialists 

in reading or mathematics. Twenty-two percent of staff providing instruction in reading or 

mathematics were at the time of the study or had previously been classroom aides. 

During site visits, instructors in observed reading activities were more likely to hold a 

teaching certificate than instructors in mathematics activities. In addition, the percentage of 

certified teachers in reading was higher than reported in the survey. Currently certified teachers 

were present in 94 percent (n = 35) of observed reading activities (compared with 56 percent 

reported by the center surveys). Currently certified teachers were present in 50 percent (n = 18) 

of observed mathematics activities. The discrepancies may reflect the fact that the data from the 

observations were not nationally representative. 

Centers reported that nearly two-thirds of reading and mathematics instructors had a 

bachelor’s degree or higher (64 percent for reading and 63 percent for mathematics).

Approximately one third of the centers indicated that their instructors had only a bachelor’s 

degree (32 percent for reading and 33 percent for mathematics) and a little more than a third of 

the centers said that their instructors had some graduate training (36 percent for reading and 37 

percent for mathematics). 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 27 

Instructor Guidance in Observed Activities 

Research on active teaching has found that in classrooms where instructors actively 

supervise, instruct, support and provide feedback to students, students make greater learning 

gains in both reading and mathematics (Brophy and Good, 1986; Good and Brophy, 2008; 

Rowan, et al., 2002). Active teaching is contrasted with teaching in which students primarily 

work independently or on tasks unrelated to academic activities (e.g., generating a list of 

behavior rules for the class). 

Instructors used active teaching strategies in 60 percent of all observed reading 

activities and in about 90 percent of all observed mathematics activities (see Exhibit 2-10). 

In 60 percent of reading activities, observers noted that instructors communicated the goals, 

purposes and expectations about activities to students. In more than half of all reading activities 

(53 percent), instructors used two or more strategies to teach a topic. In 89 percent of 

mathematics activities, observers rated instructors as providing guidance to students about the 

goals, purposes and expectations of activities. In 44 percent of observed mathematics activities, 

instructors challenged students to extend their current level of skill. In 22 percent of mathematics 

activities, instructors were observed to use more than one strategy to teach a skill. 

Exhibit 2-10 

Percentage of Observations in Reading and Mathematics Activities in Centers Serving 

Elementary School Students That Emphasized Use of Active Teaching Strategies, 

by Strategy Emphasized and Activity Type

Source: Site visit observations. 

n = 35 observations of reading activities; n = 18 observations of mathematics activities 

Exhibit reads: Instructors communicated the goals, purposes and expectations of activities to students in 

60 percent of the reading activities observed in centers serving elementary school students. 

28%

44%

22%

89%

15%

26%

53%

60%

0% 20% 40% 60% 80% 100%

Ask students to expand on answers and ideas

Challenge students to move beyond current skill levels

Employ multiple teaching strategies

Communicate goals, purposes and expectations

Percent of Observations

Active Teaching Strategies

Reading activities

Mathematics activities

Activity Type
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 28 

Feedback to Students and Parents on Student Progress 

Research on how people learn (National Research Council, 1999, 2005) emphasizes the need 

to provide feedback to learners about their progress. Feedback supports students’ developing 

skill in monitoring their own learning (Butler and Winne, 1995). Feedback can also prompt 

learners to correct misunderstandings (Bangert-Downs, et al., 1991), motivate students’ interest 

in seeking help (Butler and Neuman, 1995) and increase students’ interest in improving their 

understanding or skill (Butler, 1987). Feedback to parents about students’ performance can also 

support improvement in learning and serve as an important vehicle for parent involvement 

(Epstein, 2008; Sheldon, 2008). Evidence suggests that delayed feedback—written comments on 

work as opposed to immediate verbal feedback—is better for long-term recall and retention 

(Butler and Winne, 1995). None of this research, however, was conducted specifically in afterschool programs. 

Most centers serving elementary school students reported providing informal feedback 

to students; written or formal feedback was less common. For instance, 87 percent of schoolbased centers reported providing verbal feedback to students on work assigned by the school, as 

well as on their behavior. In contrast, about a quarter of school-based centers gave students 

written feedback on work assigned by the school (22 percent) and on work assigned in the center 

(26 percent). More than two-thirds of nonschool-based centers reported offering some kind of 

reward or recognition for program accomplishments, more than did school-based centers (see 

Exhibit 2-11; see also Exhibit E-6 in Appendix E). 

Exhibit 2-11 

Percentage of Centers Serving Elementary School Students That Provided Feedback 

About Their Progress, by Type of Feedback and Center Type 

Feedback 

Center Type 

Nonschoolbased Schoolbased 

Verbal feedback from center staff 

On work assigned by the school 86% 87% 

On work assigned at the center 74% 72% 

On student behavior 87% 87% 

Written feedback from center staff 

On work assigned by the school 26% 22% 

On work assigned at the center 34% 26% 

On student behavior 42% 48% 

“Points” or rewards for program accomplishments 65%* 48%* 

Certificates or awards for accomplishments 72% 62% 

Source: National Survey, Item 18. 

n = 393 

*p < 0.05 

Exhibit reads: Eighty-six percent of nonschool-based centers serving elementary school students provided verbal 

feedback to students on work assigned by the students’ schools. 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 29 

A majority of centers serving elementary school students reported providing frequent, 

informal feedback about student progress to parents. Sixty-two percent of centers serving 

elementary students reported talking with parents nearly every week when the parents picked up 

or dropped off their child. Four percent sent home written reports of students’ progress in the 

after-school program nearly every week, but 53 percent never did so or did so rarely. One 

percent of centers serving elementary school students indicated that they held parent–center staff 

conferences nearly every week, and 56 percent never or rarely held conferences. Nonschoolbased centers were more likely to hold parent–center staff conferences than school-based centers, 

but less likely to contact parents by telephone or e-mail. These differences were statistically 

significant. 

Summary 

Overall, the reading and mathematics activities implemented in 21st CCLC programs that 

serve elementary students reflected a focus on academics. Most students were exposed to a 

variety of activities in reading and mathematics. Additionally, centers reported offering feedback 

to students and parents on a regular basis about their progress. The feedback was largely 

informal, however, and limited primarily to verbal exchanges about progress on work assigned 

by the school. 

Reading activities covered a broader range of skills and used a wider variety of strategies 

than did mathematics activities. For example, instruction covered a broader range of reading 

skills, and centers were likely to use a range of strategies for teaching reading skills. In contrast, 

mathematics instruction focused primarily on two content strands and engaged students in tasks 

with low levels of cognitive complexity. The emphasis on operations and number sense in 

mathematics activities may have resulted from a need to build a strong foundation of basic 

mathematics skills among students. These differences in the quality of reading and mathematics 

activities cannot be explained using data from the surveys about staff qualifications, because 

instructors in reading and mathematics had similar levels of formal education. 

 
2. Reading and Mathematics Enrichment in Centers Serving Elementary School Students 30 
3. Student Targeting and Attendance 31 

3. Student Targeting and Attendance 

In general, 21st Century Community Learning Centers operate before and after school as 

well as during the summer. According to a recent report (Naftzgeret, et al., 2006), the operating 

hours for centers (typically open Monday through Thursday during the school year) averaged 

14.2 hours per week during the school year and 23.5 hours per week during the summer. On 

average, centers were open for 150 days during the school year. 

Key Findings 

 Setting priorities and policies regarding services and participation. Centers’ policies are 

important organizational supports, particularly for setting expectations regarding student 

attendance (Crollick, Zhang and Fleming, 2005). More than half of all centers had 

policies that required students to attend, but mandatory attendance policies were not 

found to relate to greater attendance.10,11 About two-thirds of centers had tracking 

systems capable of identifying how many days individual students participated in the 

program (i.e., could generate attendance reports easily, within 1–2 hours’ time). 

 Regular attendance. Regular attendance by students who are most in need of after-school 

programming is necessary for them to benefit from the instruction (Reisner, et al., 2004). 

Although all centers served students who attended Title I schools, the centers did not 

focus solely on serving low-performing students. Most allowed any interested students to 

participate. School-based centers were more likely to give priority to low-performing 

students. 

 Attendance rates and grade level. Of the centers that could generate daily attendance 

reports easily (within 1–2 hours’ time) (75 of 122 indicated that they could generate 

attendance reports easily), centers serving elementary school students reported that 48 

percent of students attended 60 days or more in 2005–06. Centers serving middle grades 

reported 36 percent of students attended this often, and centers serving high school 

students reported 30 percent of their students met this standard. These findings are not 

generalizable to all centers; it is possible that centers that are capable of reporting 

attendance data may be more likely to have higher attendance.

 Student engagement in academic activities. Beyond attendance, being on task and 

focusing on academics when present support students’ motivation to learn and appreciate 

the importance of developing proficiency in reading and mathematics (Chaput, Little and 

Weiss, 2004; Mahoney, Parente and Lord, 2007). Students were more engaged in 

observed reading activities during site visits than in observed mathematics activities. 

However, these data are not nationally representative. 

 

10 The National Survey Item 10 asked center directors: “What policies regarding attendance does the center have for 

students?” Possible answers were: “Attendance is not required; students can drop in and out”; “We require students 

to attend at least 1 day per week”; “We require students to attend 2–3 days per week”; “We require students to 

attend every day we offer services”; and “Other.” 

11 Although requiring attendance was not defined on the survey, respondents may have varied in their interpretations 

of this question, depending upon the consequences attached to nonattendance; therefore, data are included in a 

separate question about consequences of nonattendance. 
3. Student Targeting and Attendance 32 

Priorities Regarding Targeting 

Historically, economically advantaged students have had greater access to after-school 

programs than students from low-income families (Simpkins, et al., 2005). The legislation that 

authorizes the 21st CCLC program requires grantees to primarily target students who attend Title 

I schools in which 40 percent or more of the students are from low-income families and the 

families of such students. At the same time, the program gives local grantees some discretion in 

how their programming targets students who attend these schools.

In general, centers reported that they sought to serve a variety of students and used a mix of 

criteria to target students for participation. Overall, 85 percent of centers reported that they 

sought to serve students identified by the school as needing special assistance in reading and/or 

mathematics, and 76 percent sought to serve students scoring “below proficient” on state tests. 

Sixty-eight percent of centers sought to serve students who were eligible for free or reducedprice lunches. 

When centers were asked to rank their top three priority groups, the most commonly reported 

groups were students who scored below proficient on state assessments (44 percent of centers) 

and students whom the school identified as needing special assistance in reading and 

mathematics (21 percent of centers). By contrast, a relatively small proportion of centers (2 

percent of school-based centers and 3 percent of nonschool-based centers) reported that students 

who received free or reduced-price lunch were among the top three priority groups. 

Top-priority groups differed in certain respects for school-based and nonschool-based centers 

(see Exhibit 3-1; see also Exhibit E-7 in Appendix E). Both types of centers targeted students 

scoring below proficient on state tests, but nonschool-based centers were more likely than 

school-based centers to target this group (79 percent versus 46 percent). Nonschool-based 

centers were also more likely than school-based centers to report their top priorities were 

students whom their teacher or counselor identified as in academic need (88 percent versus 55 

percent). 
3. Student Targeting and Attendance 33 

Exhibit 3-1 

Percentage of Centers That Reported Top-Priority Target Groups, 

by Target Groups and Center Type

Source: Focused Telephone Survey, Item 1. 

n = 102 

*p < 0.05 

Exhibit reads: A higher percentage of centers in school-based settings (75 percent) reported they had open 

enrollment for all interested students than nonschool-based centers (60 percent). 

Policies Regarding Attendance 

Data on general attendance policies were collected for all centers, but data on attendance 

were collected using the focused survey from a random sample of centers sampled for the 

national survey. The study team asked centers to report attendance only if they had adopted 

methods for tracking attendance and only if staff kept daily records of individual students’ 

attendance. Sixty-one percent of the centers tracked attendance in a way that allowed them to 

report easily (within 1–2 hours’ time) how many days an individual student attended in the past 

year. 

More than half of all centers (56 percent) in the full sample of centers reported having 

policies that require students to attend 2 days per week or more to remain enrolled in the 

program.12 Thirty-eight percent of centers required daily attendance, and 26 percent of centers 

reported that attendance was on a drop-in basis. Centers serving elementary school students were 

more likely to have policies that required attendance than did those serving older students (see 

 

12 Although requiring attendance was not defined on the survey, respondents may have varied in their interpretations 

of this question, depending upon the consequences attached to nonattendance; therefore, data are included in a 

separate question about consequences of nonattendance. 

9%

79%*

88%*

69%

59%

60%

12%

46%*

55%*

58%

70%

75%

0% 20% 40% 60% 80% 100%

Other 

Students who scored "below proficient" on state 

assessments

Students identified by the school as needing 

special assistance in reading and/or math

Students who receive free or reduced price 

meals

Students who are recommended by teachers or 

counselors at the feeder schools

Open enrollment for all interested students

Percent of Centers Reporting

Students Targeted

School-based centers

Nonschool-based centers

Open enrollment for all interested students

Students who are recommended by teachers or 

counselors at the feeder schools

Students who receive free or reduced price meals

Students identified by the school as needing special 

assistance in reading and/or math

Students who scored “below proficient” on state 

assessments

Other

Center Type 
3. Student Targeting and Attendance 34 

Exhibit 3-2; see also Exhibit E-8 in Appendix E). Forty-one percent of centers serving 

elementary school students required students to attend the program every day services were 

offered, compared with 31 percent of centers serving students in middle school and 22 percent of 

centers serving high school students. These data are for the sample that completed the national 

survey. 

Exhibit 3-2 

Percentage of Centers That Reported Attendance Requirements, 

by Grade Level and Attendance Requirement

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 10. 

n = 499 

*p < 0.05 

Exhibit reads: Forty-one percent of centers serving elementary school students reported that they required 

the students to attend every day the center-offered services. 

22%*

31%*

41%*

19%*

25%*

19%*

13%*

14%*

13%*

45%*

29%*

26%*

0% 20% 40% 60% 80% 100%

High Schools

Middle Schools

Elementary Schools

Percent of Centers Reporting

Type of School

Require every day attendance

Require attendance 1-3 days per week

Other attendance requirement

Do not require attendance at all

Attendance Requirement 
3. Student Targeting and Attendance 35 

More than half the centers reported requiring participation for all students in specific 

academic activities when students were present, including homework help (73 percent), 

reading activities (60 percent), mathematics activities (58 percent) and tutoring (14 percent) (see 

Exhibit 3-3; see also Exhibit E-9 in Appendix E). Programs serving high school students were 

less likely than those not serving this age group to require participation in these activities. A 

higher proportion of school-based centers reported that they required participation in reading 

activities (10 percent, compared with 2 percent of nonschool-based centers) for all students 

attending the center. 

Exhibit 3-3 

Percentage of Centers That Reported That They Required Participation of All Students in 

Specific Activities When Students Were Present, by Type of Activity

Source: Focused Telephone Survey, Item 9. 

n = 114 

Exhibit reads: Seventy-three percent of centers reported that homework time was mandatory. 

Center directors at case study sites noted that they felt it important to consider carefully what 

would engage students and encourage them to keep coming to the program. For example, in a 

Florida center, the director offered engaging recreational activities as a way to entice students to 

attend the academic components of the program, which were voluntary. “The main issue is if you 

only offer academics, then your attendance is affected.... We’ve offered a professional football 

player doing flag football games, and that generated so much interest among kids.... They’re 

dying to play and consequently are attending after school.” 

Regular Attendance 

Research on after-school programs generally has found a significant association between 

regular program attendance and positive student outcomes. For example, participation in afterschool activities sponsored by The After School Corporation in New York City was linked to 

higher grades and increased school attendance, especially for students who participated regularly 

in the corporation’s programs over 2 consecutive years (Reisner, et al., 2004). Researchers 

studying attendance in before- and after-school programs have argued that it is important for 

evaluation studies to distinguish among intensity, duration and breadth of student participation 

14%

58%

60%

73%

0% 20% 40% 60% 80% 100%

Tutoring

Mathematics activities

Reading activities

Homework time

Percent of Centers Reporting

Activity Required
3. Student Targeting and Attendance 36 

(Chaput, et al., 2004). Intensity refers to how often students participate in a program during a 

given time period. Duration refers to the length of time (in days, months or years) students 

participate in programs. Other studies have found a relationship between regular attendance in 

after-school programs and educational attainment in high school (Fredricks and Eccles 2006), inclass attendance (Grossman et al., 2002; Huang et al., 2000) and reading and mathematics 

achievement (Lauer et al., 2006). 

Although most research tends to focus on just one aspect of participation, each type of 

measure has been associated with improved outcomes for youth. For example, in their evaluation 

study of two after-school programs, Baker and Witt (1995) found a relationship between 

intensity of participation and a variety of academic outcomes (see also Huang, et al., 2000). 

Duration appears to be particularly significant for arts-related programming, where interactions 

may not be intense but take place over many years (Catterall, 1998). A study examining the 

relationship between attendance and academic achievement of students participating in Los 

Angeles’ BEST after-school program found that elementary school students attending program 

activities for 50 or more days throughout the school year demonstrated positive academic 

achievement growth (Huang, et al., 2008). Other researchers have studied how attendance in 

after-school programs, in combination with other home-based and community activities, is 

linked to positive behavioral and academic outcomes (Vandell, Reisner and Pierce, 2007). More 

attendance is not always better, however, as Lauer and colleagues (2006) reported in their 

synthesis: Students who received more than 210 hours of instruction did not show gains in 

reading. 

As stated earlier, 21st Century Community Learning Centers operate before and after school 

as well as during the summer. According to Naftzger and colleagues (2006), centers were open 

an average of 14.2 hours per week during the school year and 23.5 hours per week during the 

summer. On average, centers were open for 150 days during the school year. 

Regular attendance has been a concern within the 21st CCLC program, especially since the 

publication of the first national evaluation (Dynarski, et al., 2003). That study found that many 

programs had lower-than-expected attendance: 55 percent of students in the elementary school 

treatment group of this evaluation attended 51 or more days per year, and 20 percent of middle 

school students in the treatment group attended that often. 

Although this study sought to better understand the nature of attendance in centers, the data 

collected from this study may be of limited utility in addressing concerns about attendance raised 

within the 21st CCLC program, since it was not collected in the same way as the national 

evaluation and did not distinguish among different types of programs (summer, before- or afterschool programs). 

Of the centers that reported on attendance in the current study (approximately 61 

percent of centers surveyed), 44 percent of students attended 60 days or more in the last 

year. Centers serving elementary school students reported higher attendance than did centers 

serving middle and high school students (see Exhibit 3-4; see also Exhibit E-10 in Appendix E). 

Centers serving elementary school students reported that 48 percent of students attended 60 days 

or more in the last year, which amounts to approximately 2 days a week. Centers serving middle 

grades reported 36 percent of students attended this often (60 days or more), and centers serving 
3. Student Targeting and Attendance 37 

high school students reported 30 percent of their students met this standard. In the previous 

evaluation, the average elementary student attended programming 2 to 3 days a week or 56 days 

per year during the year of the study (Dynarski et al., 2004), and the typical middle school 

student attended 1 day a week, or 32 days a year (Dynarski et al., 2003). The results of this study 

cannot be compared directly with the results of the previous evaluation because the two studies 

did not measure attendance using the same method.13 

Exhibit 3-4 

Percentage of Centers That Reported the Number of Days of Attendance in 2005–06, 

by Grade Level and Days of Attendance

Note: Percentages may not equal 100 percent due to rounding. 

Source: Focused Telephone Survey, Item 8. 

n = 75 

Exhibit reads: Centers serving elementary school students reported that 17 percent of students attended 120 days or 

more. 

The proportion of students attending 60 days or more was higher in school-based 

centers than in nonschool-based centers (52 percent versus 44 percent). A third of students 

in school-based centers serving elementary school students attended fewer than 30 days, 

compared with 14 percent in nonschool-based centers serving elementary school students. About 

half of students in school-based centers serving high school students attended fewer than 30 

days, compared with 89 percent in nonschool-based centers serving high school students. 

 

13 Data collection methods varied between this study and the previous study. The previous national evaluation 

collected program attendance records of individual students across 2 years, while the current study relied on survey 

reports of attendance during 1 year by program directors. Hence, the attendance rates in the previous evaluation are 

likely to be more accurate than the self-reported attendance rates reported in this study. 

6%

11%

17%

14%

11%

17%

10%

14%

14%

9%

28%

21%

61%

36%

32%

0% 20% 40% 60% 80% 100%

High Schools

Middle Schools

Elementary Schools

Percent of Centers Reporting

Type of School

120 or more days

90-199 days

60-89 days

30-59 days

Fewer than 30 days

Number of Attendance Days
3. Student Targeting and Attendance 38 

Overall attendance rates were similar for centers that did and did not require some 

level of attendance to remain enrolled in the program, but attendance policies were related 

to attendance in centers serving secondary-level students. In centers that required some level 

of attendance, 66 percent of students attended for more than 30 days; 62 percent attended for the 

same duration in centers that did not require attendance. There was no relation between any kind 

of attendance policy and actual attendance for centers overall or for centers serving elementary 

school students. For the middle grades, centers that required attendance every day had higher 

attendance than did centers with other attendance policies. In high school, centers that required 

attendance 2 to 3 days per week had higher attendance. 

Staff members address absences from the programs primarily by talking with parents 

and students (see Exhibit 3-5; see also Exhibit E-11 in Appendix E). Fifty-eight percent of staff 

members reported they notified parents or caregivers if the student did not meet the attendance 

requirements of the center. Thirty-eight percent said they spoke to students after several 

absences. Few centers punished students for absences by restricting privileges (6 percent) or by 

restricting participation in particular activities (10 percent). 

Exhibit 3-5 

Percentage of Centers That Reported Consequences for Student Absences, 

by Consequence Type

Source: National Survey, Item 11. 

n = 503 

Exhibit reads: Fifty-eight percent of centers reported notifying parents when a student is absent from the program. 

6%

10%

13%

26%

29%

38%

58%

0% 20% 40% 60% 80% 100%

The student receives “points” or some 

other kind of penalty that restricts ...

The student is restricted from participating 

in other activities at the center for a ...

A staff member notifies the student’s 

school

Not applicable; we do not require 

attendance

A staff member speaks to the student 

about an absence after each absence

A staff member speaks to the student 

about an absence after a certain number ...

A staff member notifies the student’s 

parents or caregivers

Percent of Centers Reporting

Consequences for Absences

A staff member notifies the student’s parents or 

caregivers

A staff member speaks to the student about an absence 

after a certain number of absences

A staff member speaks to the student about an absence 

after each absence

Not applicable; we do not require attendance

A staff member notifies the student’s school

The student is restricted from participating in other 

activities at the center for a period of time

The student receives “points” or some other kind of 

penalty that restricts privileges at the center
3. Student Targeting and Attendance 39 

Student Engagement in Observed Activities

Beyond attendance, learning requires student engagement in after-school activities (Chaput et 

al., 2004; Mahoney et al., 2007). Cognitive engagement in learning tasks is closely related to 

self-regulation in learning: by engaging with classroom activities, students are better able to track 

new information and assimilate feedback from instructors (Corno and Mandinach, 1983). 

Engagement is also related to students’ beliefs about the value of effort in learning (Mahoney et 

al., 2007; Pintrich and Schrauben, 1992). Motivation and engagement are important to 

developing students’ disposition to see mathematics as valuable and worthwhile (National 

Research Council, 2001) and in developing students’ reading skills (National Reading Panel, 

2000). 

Observers in the case studies rated students as “on task” in most activities in reading, 

but not in mathematics. In 80 percent of reading activities, observers judged students to be on 

task and engaged in those activities, compared with 42 percent of mathematics activities. 

Summary 

Centers reported seeking to serve low-income students and students who scored low on state 

assessments or whom instructors judged to need special assistance in reading and mathematics. 

Centers’ reports on student attendance revealed a familiar pattern. On average, elementary 

schools attended programming more than middle and high school students did. With respect to 

regular attendance, this evaluation found that, on average, elementary students attended 

programming more than middle and high school students did. These findings replicated a general 

trend shown in other studies (Borman and Dowling, 2006; Grossman et al., 2002; Reisner et al., 

2004; Simpkins et al., 2005): Elementary school students attended after-school activities for 

more days than did middle and high school students. These data underscore the challenge centers 

face in encouraging and sustaining regular student attendance. 

Engagement was generally high in reading activities but lower in mathematics activities. 

These data help underline the differences between these two subjects. Observers in the case 

studies rated students as “on task” in most activities in reading, but not in mathematics. Lower 

engagement in mathematics activities may be related to the narrower range of mathematics skills 

targeted by these activities. 

 
3. Student Targeting and Attendance 40 
4. Organizational Supports for Instructional Quality 41 

4. Organizational Supports for Instructional Quality 

Key Findings 

 Center staffing. Research on staff compensation suggests that adequate pay and benefits 

can help attract and retain qualified staff to support high-quality after-school 

programming (Wechsler et al., 2001). Additionally, it is important for after-school 

programs to provide sufficient staff development to enhance staff skills and maximize 

their expertise relevant to their roles (Crollick et al., 2005). Centers relied primarily on 

part-time staff, who were unlikely to receive benefits as a result of their work in the 

center. This is not surprising, since centers offered programming 16 hours per week, on 

average. It is also possible that some of these part-time staff were employed full-time as 

teachers or paraprofessionals at the time of the study. The majority of centers reported 

providing paid professional development opportunities to full-time and part-time staff. 

Across all centers, 29 percent of the staff had worked at the center for less than 1 year. 

About half of centers (48 percent) reported that the primary reason for staff turnover was 

graduation from school or completion of a program of study. 

 Alignment of activities with student needs. After-school programs provide an opportunity 

to extend students’ learning beyond the school day, but the effectiveness of that help 

depends, in part, on the quality of the relationships between adults and students 

(Birmingham et al., 2005; Carruthers and Busser, 2000; Grossman et al., 2002; Huang et 

al., 2000; Intercultural Center for Research in Education & National Institute on Out-ofSchool Time, 2005; Pierce, Hamm and Vandell, 1999). Centers reported using state 

assessment data to assess student growth and to identify different strategies for matching 

activities intended to meet student needs. About one-quarter of the centers coordinated 

activities with supplemental educational services (SES) providers; 9 percent reported 

coordinating schedules and academic activities with providers. More than two-thirds of 

centers reported having access to whole-school achievement data for reading and 

mathematics. 

 Coordination with the school day. High-quality instruction is aligned with student needs; 

however, to understand student needs, after-school programs need to be well coordinated 

with school-day activities and use data from student assessments to adjust program 

offerings (de Kanter, et al., 2003; Noam, 2003). Respondents from nonschool-based 

centers were more likely than school-based centers to report challenges aligning activities 

with the school day because they lacked information about students’ needs and because 

school staff were not responsive to the centers’ requests. 

Center Staffing 

Recruiting and Retaining High-Quality Staff 

An important role for center directors in 21st Century Community Learning Centers 

programs is recruiting and retaining high-quality staff with expertise matched to their roles 

(Crollick et al., 2005). Past research has focused primarily on the role of competitive salaries in 

retaining staff (Halpern, Barker and Mollard, 2000). This section focuses on both staff pay and 

benefits at centers and on center directors’ perceptions about the relationship between staff pay 

and benefits and staff turnover in centers. 
4. Organizational Supports for Instructional Quality 42 

About half of centers paid staff with current teaching certificates $15 to $25 per hour 

(54 percent) and staff without current teaching certificates $10 to $15 per hour (52 percent) 

(see Exhibit 4-1; see also Exhibit E-12 in Appendix E). Currently certified teachers who worked 

in nonschool-based sites were paid less than staff without teaching certificates working in 

school-based settings, but pay for staff without teaching certificates was comparable across 

center types. Data from the Bureau of Labor Statistics (Occupational Online Network, 2008) 

indicate that the median wage for paraprofessionals working in schools in 2008 was 

approximately $11.45 per hour (based on an annual salary of $22,500). Rates are not likely to be 

comparable across different regions of the country; however, no objective data are readily 

available about how competitive wages are for center staff. 

Exhibit 4-1 

Percentage of Centers That Paid Hourly Rates to Staff Who Led Instructional Activities, 

by Type of Staff and Rates 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 27. 

n = 453 

Exhibit reads: One percent of all centers paid staff who are certified teachers less than $10 per hour. 

<$10/hour

1%

$10-

15/hour

11%

$15-

20/hour

27%

$20-

25/hour

27%

$25-

30/hour

20%

>$30/hour

10%

Don't 

know

6%

Staff Who Are Certified Teachers

<$10/hour

$10-15/hour

$15-20/hour

$20-25/hour

$25-30/hour

>$30/hour

Don't know

<$10/hour

29%

$10-

15/hour

52%

$15-

20/hour

10%

$20-

25/hour

3%

$25-

30/hour

1% >$30/hour

0%

Don't 

know

5%

Other Staff

<$10/hour

$10-15/hour

$15-20/hour

$20-25/hour

$25-30/hour

>$30/hour

Don't know

Pay Rate Pay Rate 
4. Organizational Supports for Instructional Quality 43 

Seventy-six percent of staff in all centers reported working fewer than 20 hours per 

week at the center (see Exhibit 4-2; see also Exhibit E-13 in Appendix E). Ten percent of staff 

members offering academic instruction worked in full-time positions. Another 14 percent 

worked half time or more. Nonschool-based centers were more likely to employ staff in halftime (21 percent) or full-time (34 percent) positions, compared with school-based centers (11 

percent half-time and 8 percent full-time). Centers serving high school students were more likely 

to have staff members who worked full-time than were centers not serving this age group. Center 

staff could be employed in other jobs, such as working as teachers during the school day. 

Exhibit 4-2 

Percentage of Staff Who Led Instructional Activities at Centers, 

by Hours Worked per Week 

Note: Percentages may not equal 100 percent due to rounding. 

Source: Focused Telephone Survey, Items 26, 32 and 38. 

n = 349 

Exhibit reads: Ten percent of staff worked full-time at the center (more than 35 hours 

per week). 

<10 

hours/week

32%

10-19 

hours/week

44%

20-34 

hours/week

14%

>35 

hours/week

10%

<10 hours/week

10-19 hours/week

20-34 hours/week

>35 hours/week

Hours Worked Per Week 
4. Organizational Supports for Instructional Quality 44 

About half of centers reported offering some type of benefits to either full- or part-time 

staff members. The most commonly offered benefits related to health insurance (offered to fulltime staff in 43 percent of centers and to part-time staff in 8 percent of centers) and retirement 

plans (offered to full-time staff in 41 percent of centers and part-time staff in 14 percent of 

centers) (see Exhibit 4-3; see also Exhibit E-14 in Appendix E). About half of centers reported 

that they did not offer health insurance (49 percent) or paid time off for vacation and sick leave 

(49 percent) to any staff members. Nonschool-based centers were more likely than school-based 

centers to provide their full-time staff with health insurance (66 percent versus 40 percent), a 

retirement savings plan (59 percent versus 39 percent) and paid time off for vacation and sick 

leave (72 percent versus 37 percent). Centers serving high school students were more likely to 

provide their full-time staff with health insurance and paid time off than centers not serving this 

age group, but less likely to provide this benefit to their part-time staff than centers not serving 

high school students. Part-time staff at centers serving high school students were also less likely 

than staff at centers not serving this age group to receive tuition reimbursement as a benefit. 

Finally, centers serving middle school students were more likely than centers not serving this 

group to provide health insurance to their part-time staff. 

Exhibit 4-3 

Percentage of Centers That Reported Offering Benefits to Full- and Part-Time Staff Who 

Led Instructional Activities, by Benefit Offered and Staff Status 

Source: National Survey, Item 28. 

n = 505 

Exhibit reads: Forty-three percent of centers offered health insurance to full-time staff. 

4%

11%

14%

8%

12%

40%

41%

43%

0% 20% 40% 60% 80% 100%

Tuition reimbursement

Paid time off for vacation and sick leave

Retirement savings plan

Health insurance

Percent of Centers Reporting

Benefits Offered

Offered to full-time staff

Offered to part-time staff

Staff Status 
4. Organizational Supports for Instructional Quality 45 

Over one-quarter of staff members had been in their positions for less than 1 year.

Across all centers, 29 percent of the staff had worked at the center for less than 1 year.14

The most commonly reported reasons for staff turnover were graduation from school 

or completion of a program, but staff also left for lack of benefits and because centers were 

unable to offer a full-time position. More nonschool-based centers than school-based centers 

said that staff left the program because of the lack of benefits (44 percent versus 25 percent) and 

because of low pay (18 percent versus 8 percent) (see Exhibit 4-4; see also Exhibit E-15 in 

Appendix E). 

Exhibit 4-4 

Percentage of Centers That Reported Reasons for Staff Departures, 

by Reason for Departure and Center Type 

Source: National Survey, Item 30b. 

n = 466 

*p < 0.05 

Exhibit reads: Forty-nine percent of school-based centers reported that the primary reason why staff left was 

because they had completed a program (e.g., AmeriCorps) or graduated from school compared with 42 percent of 

nonschool-based centers. 

Professional Development 

Ensuring staff quality in schools is widely believed to require ongoing professional 

development in evidence-based practice (National Reading Panel, 2000; National Research 

Council, 2001). Similarly, researchers of after-school programs have argued that professional 

development is necessary for staff to learn principles of youth development (Huebner, Walker 

 

14 Since the National Survey Item 29 asked only about “all staff,” the tenure for certain types of staff may vary. 

28%*

9%

18%*

24%

44%*

42%

12%*

13%

8%*

22%

25%*

49%

0% 20% 40% 60% 80% 100%

We have not had staff leave our program

Don't know

Low pay

Inability to offer a full-time position

Lack of benefits

Completion/graduation from a program or school

Percent of Centers Reporting

Reason for Departure

School-based centers

Nonschool-based centers

Types of Centers 
4. Organizational Supports for Instructional Quality 46 

and McFarland, 2003), develop skills for building positive relationships with youth (Carruthers 

and Busser, 2000) and acquire strategies for supporting youth skill building (Carruthers and 

Busser, 2000). For programs seeking to enhance students’ academic achievement, professional 

development activities need to be aligned with student needs identified by school-day staff and 

administrators (Huang, 2001). An analysis of The After School Corporation’s most promising 

programs with respect to youth outcomes indicated that those programs all had a leader who 

provided ongoing opportunities for professional development to staff members (Birmingham et 

al., 2005). 

The majority of centers reported providing paid professional development 

opportunities to full-time and part-time staff (see Exhibit 4-5; see also Exhibit E-16 in 

Appendix E). Professional development often included conference attendance paid for by the 

center (53 percent of centers for full-time staff and 39 percent of centers for part-time staff). 

Centers were more likely to pay for training or professional development to full-time staff (62 

percent) than to part-time staff (55 percent). At the same time, for both full-time and part-time 

staff, school-based centers were less likely than were nonschool-based centers to pay for 

attendance at conferences. Centers serving students in the middle grades were less likely to pay 

for attendance at conferences to their part-time staff than were centers not serving this age group. 

Exhibit 4-5 

Percentage of Centers That Reported That They Offered Paid Professional Development 

Opportunities for Full- and Part-Time Staff 

Who Led Instructional Activities, by Staff Status and Type of Development 

Source: National Survey, Item 28. 

n = 505 

Exhibit reads: Sixty-two percent of centers reported that they offered paid professional development to fulltime staff. 

Site visit data suggest that professional development offered by centers may have 

focused on nonacademic areas. Four of 12 case study centers provided just 1 to 2 days of 

training or orientation to staff members, and that training focused primarily on logistics and 

program implementation. At first glance, these findings appear to contradict the survey findings, 

but the survey did not include questions about the content of the professional development. 

Additionally, the case study data are not nationally representative. 

39%

53%

55%

62%

0% 20% 40% 60% 80% 100%

Offered to part-time staff

Offered to full-time staff

Percent of Centers Reporting

Staff Status

Paid professional development

Paid attendance at conferences

Types of Benefits 
4. Organizational Supports for Instructional Quality 47 

Among the case study sites, few centers reported that they offered professional development 

to staff, beyond that which was available to regular school-day staff members. Center directors in 

the case studies said they relied on the professional development opportunities their instructional 

staff received through the district, and they thought additional professional development specific 

to the center’s program might be a burden for staff. For example, one director noted that most of 

the center’s instructional staff were “already certified instructors who have to go to so many 

workshops that we don’t feel the need to give them more professional development.” During site 

visits, directors reported that rather than participating in formal professional development 

workshops or conferences, staff were encouraged to share their ideas for innovative 

programming in weekly staff meetings. 

Alignment to Student Needs 

Coordination With the School Day 

A central challenge for centers is to coordinate their activities with the school-day 

instructional program in ways that are aligned with student needs and with the school-day 

curriculum (de Kanter et al., 2003; Noam, 2003). Studies of after-school programs show that 

when programs align their activities to complement school-day activities, they can contribute to 

student success (Little, 2006). At the same time, studies also indicate that such coordination can 

be difficult, especially when programs are not based in schools, because the programs may have 

limited access to the curricular materials used in the school day and to school-day staff who 

could inform them about particular students’ progress and challenges. 

More than two-thirds of the centers (71 percent) had access to whole-school state 

assessment data in reading, and 69 percent had access to this information for mathematics.

Of those centers that received these data, 83 percent also had access to individual students’ level 

of proficiency on state assessments. 

Significantly more school-based centers than nonschool-based centers had access to 

whole-school data. More school-based centers reported having access to state assessment data at 

the school level on reading (73 percent) and mathematics (71 percent) than did nonschool-based 

centers in those two subject areas (53 percent and 52 percent, respectively) (see Exhibit 4-6; see 

also Exhibit E-17 in Appendix E). Centers did not differ by type with respect to their access to 

individual-level data, however. 
4. Organizational Supports for Instructional Quality 48 

Exhibit 4-6 

Percentage of Centers That Reported That State Assessments Data Was Available F 

for Use in Tracking Student Outcomes, by Subject Tested and Center Type 

Source: National Survey, Item 13. 

n = 467 

*p < 0.05 

Exhibit reads: Seventy-three percent of school-based centers reported that they had access to test results from No 

Child Left Behind (NCLB) state reading assessments for the whole school. 

Use of Student Assessment Data 

Instructors should be able to assess the needs of individual students and tailor instruction to 

meet specific needs (National Reading Panel, 2000). Likewise, in mathematics, emphasis should 

be on assessing and engaging students’ prior knowledge (National Research Council, 1999, 

2005). In terms of providing quality programming, a key use of assessment data, then, is to 

adjust instructional practice and design new offerings that better meet identified student needs. 

Center directors reported using several sources of information to assess student 

academic growth, make program adjustments or evaluate program success. Almost half of 

the centers reported using results from tests administered at students’ schools to assess students’ 

academic growth in the 21st CCLC program. More centers reported using informal staff 

observations (28 percent of centers reported using them almost daily) and verbal reports from 

teachers (13 percent reported using them almost daily) to assess student progress with greater 

frequency than more formal assessments. Only 6 percent of centers reported that they used 

results from tests administered by center staff or student self-assessments (17 percent) at least 

weekly to assess students or evaluate program success (see Exhibit 4-7; see also Exhibit E-18 in 

Appendix E). 

52%*

53%*

71%*

73%*

0% 20% 40% 60% 80% 100%

Mathematics

Reading

Percent of Centers Reporting

Subject Tested

School-based centers

Nonschool-based centers

Types of Centers 
4. Organizational Supports for Instructional Quality 49 

Exhibit 4-7 

Percentage of Centers That Reported Student Assessment Sources, 

by Type and Frequency of Assessment 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 15. 

n = 502 

Exhibit reads: Twenty-eight percent of centers reported using informal staff observations on an almost daily basis. 

More than half of the centers reported using assessments of students’ academic growth 

for at least one of the following purposes: to identify new offerings (59 percent) or improve 

practices in existing program offerings (69 percent), to assign students to particular staff 

members (54 percent) and to evaluate the success of their programs (66 percent)Two-thirds 

of centers reported using assessment information to improve practices in existing program 

offerings (see Exhibit 4-8; see also Exhibit E-19 in Appendix E). 

Nonschool-based centers were significantly more likely than school-based centers (67 

percent versus 52 percent) to report using formative assessment data collected by the center to 

assign students to particular staff members who could meet their specific academic needs. For 

example, at one site visited, staff used data on students’ mathematics assessments administered 

by the center to assign them to mathematics tutors who were also certified instructors. Centers 

serving elementary school students were significantly less likely than were centers not serving 

this age group to use center-administered assessment data to identify new program offerings (56 

percent of centers serving elementary school students versus 60 percent of centers serving 

middle school students and 64 percent of centers serving high school students). 

1%

1%

3%

7%

8%

13%

28%

5%

7%

9%

16%

9%

26%

20%

11%

13%

19%

22%

10%

23%

14%

24%

47%

34%

22%

22%

11%

12%

58%

31%

36%

32%

50%

26%

26%

0% 20% 40% 60% 80% 100%

Results from tests administered by 

center staff

Results from tests administered at 

students’ schools

Written reports from teachers 

Informal discussions with family 

members

Student self-report or selfassessment

Verbal reports from teachers

Informal staff observations

Percent of Centers Reporting

Assessment Type

Almost daily

Weekly

1-2 times per month

1-2 times per year

Not used

Informal staff observations 

Verbal reports from teachers 

Student self-report or selfassessment 

Informal discussions with family 

members 

Written reports from teachers 

Results from tests administered 

at students’ schools 

Results from tests administered 

by center staff 

Frequency of Assessment 
4. Organizational Supports for Instructional Quality 50 

Exhibit 4-8 

Percentage of Centers That Reported Uses of Student Assessment Data, by Type of Use 

Source: National Survey, Item 16. 

n = 516 

Exhibit reads: Sixty-nine percent of school-based centers reported using students’ assessments of academic growth 

to improve existing program offerings. 

Case study data illustrate ways that programs used data to target instruction. For example, 

one Florida grantee used extensive student-level data to track student progress and growth, 

including results from the two benchmark assessments administered by the district. Staff in this 

center looked at the subscale scores to identify areas of weakness in students’ skills and 

determine which skills to target during the after-school program. Similarly, a North Carolina 

grantee used the pre- and post-test data associated with the reading and mathematics intervention 

curricula provided by the center to monitor student progress from week to week. The center 

leadership worked with staff members to identify the trends in these data and adjust instruction 

to meet students’ needs. 

Beyond the use of individual assessment to target specific instruction to students, the case 

study data support the survey findings that staff in some centers used data for program-level 

improvement. For example, one Maryland grantee employed an onsite evaluator to monitor 

program activities. That program evaluator used a narrative reporting form to document weekly 

program visits, as well as surveys of parents, students and classroom instructors. The evaluator 

then used the data collected by these means to “zero in on the [progress related to] homework 

and academic performance and to assess how students are getting along” in order to “analyze it 

by school and communicate it back to the coordinators at each site.” On the basis of this 

feedback, the coordinators adjusted programming activities, targeting students in need. 

 

1%

48%

54%

59%

66%

69%

0% 20% 40% 60% 80% 100%

Do not use

Assign students to activities

Assign students to staff

Identify new offerings

Evaluate program success

Improve existing offerings

Percent of Centers Reporting 

Use of Assessment Data
4. Organizational Supports for Instructional Quality 51 

Use of data by some centers in the case studies was more limited. For instance, a site in 

South Dakota made little use of student assessment data. According to the center director, staff at 

this nonschool-based site did not use data in their decision-making processes. All data collected 

were used for public relations or for reporting to grantee agencies. The outreach director reported 

that he did not find the data collected to be useful: “I found next to no useful information from 

that evaluation.... For me, the purpose of evaluation is to make the program better, and if I’m not 

going to learn how to make the program better, I don’t need it.” 

Despite the achievement data available to many centers, about a third of centers 

reported lacking information about students’ academic needs, a lack of responsiveness by 

school staff to center requests and a lack of information about the school’s curriculum as 

major challenges in implementing high-quality programming. Nonschool-based centers were 

more likely to report these challenges than were school-based centers (see Exhibit 4-9; see also 

Exhibit E-20 in Appendix E). A significantly higher percentage of nonschool-based center 

directors (36 percent) reported they had difficulty obtaining information about student needs 

from schools than did directors of school-based centers (20 percent). Similarly, 39 percent of 

nonschool-based centers reported a lack of responsiveness by school staff to their requests, 

compared with 29 percent of school-based centers. Directors of centers serving elementary 

school students reported less difficulty getting information about student needs and the school’s 

curriculum than did directors of centers serving older groups. Difficulties getting this 

information were greater among centers serving middle school students than they were for 

centers serving other age groups. 

Exhibit 4-9 

Percentage of Centers That Reported Challenges in Implementing High-Quality 

Programming, by Type of Challenge and Center Type 

Source: National Survey, Item 31. 

n = 499 

*p < 0.05 

Exhibit reads: Twenty percent of school-based centers cited lack of information about students’ academic needs 

as a challenge to implementing high-quality programming. 

40%

39%*

36%*

31%

29%*

20%*

0% 20% 40% 60% 80% 100%

Lack of information about the school-day curriculum

Lack of responsiveness by school staff to center 

requests

Lack of information about students' academic needs

Percent of Centers Reporting

Implementation 

Challenges

School-based centers

Nonschool-based centers

Types of Centers 

Lack of information about students' academic needs

Lack of responsiveness by school staff to center 

requests

Lack of information about the school-day curriculum
4. Organizational Supports for Instructional Quality 52 

Adult Support: Fostering Positive Relationships Between Adults and Students 

One of the most consistent findings about after-school programs is that youth engagement in 

activities depends on positive relationships between adult staff and participants (Birmingham et 

al., 2005; Carruthers and Busser, 2000; Grossman et al., 2002; Huang et al., 2000; Intercultural 

Center for Research in Education & National Institute on Out-of-School Time, 2005; Pierce et al. 

1999). The after-school programs that researchers identified as more successful in offering 

quality programming hire staff who are able to create warm and friendly relationships with 

youth, and provide a challenging and motivating intellectual environment for them (Grossman et 

al., 2002). Positive staff relationships with students are key to retaining students in a program, 

especially among older youth (Roffman, Papagano and Hirsch, 2001), and one research study has 

linked such positive relationships to positive behavioral adjustment in school (Pierce et al., 

1999). See Appendix F for the observation form that describes the dimensions of adult support. 

In the large majority of observed activities in reading and mathematics (n = 59), staff used 

positive behavior management strategies and showed warm affect toward students, but 

infrequently sought to determine students’ concerns and ideas. In 88 percent of reading activities 

and 84 percent of mathematics activities, staff were observed using positive behavior 

management strategies, such as reinforcement of good behavior (see Exhibit 4-10). However, 

instructors encouraged students to share their ideas in only 30 percent of reading activities and 

21 percent of mathematics activities. 

Exhibit 4-10 

Percentage of Observations in Reading and Math Activities That Demonstrated StudentStaff Relationship-Building Efforts, by Relationship-Building Effort and Type of Activity 

Source: Site visit observations. 

n = 40 for reading; n = 19 for mathematics 

Exhibit reads: In 88 percent of observed reading activities, staff used positive behavior management techniques. 

21%

90%

95%

84%

30%

78%

88%

88%

0% 20% 40% 60% 80% 100%

Encourage students to share their ideas, 

opinions, and concerns

Attentively listen to and/or observe 

students

Show positive affect 

toward students

Use positive behavior management 

techniques

Percent of Observations

Relationship-Building Efforts

Reading activities

Mathematics activities

Use positive behavior management 

techniques

Show positive affect toward students

Attentively listen to and/or observe 

students

Encourage students to share their 

ideas, opinions, and concerns

Type of Activity 
4. Organizational Supports for Instructional Quality 53 

Coordination with Supplemental Educational Services 

To strengthen their links with schools, leverage their resources and enhance their services, 

some 21st Century Community Learning Centers coordinated with or became SES providers. 

Fifteen percent of centers were authorized to provide supplemental educational 

services. Each center providing SES did so for an average of 38 students in the core subjects of 

reading and mathematics while, according to the PPICS database, delivering services for a 

median of 100 students across all subjects and activities, including reading and mathematics. Just 

9 percent of all centers reported coordinating with one or two providers, and only 5 percent of all 

centers reported that their coordination activities focused on aligning schedules with the 

providers, while 4 percent indicated they coordinated their academic support activities with the 

providers. 

The vast majority of instructional practices reported for centers providing SES and for 

centers not providing SES were similar. However, SES-providing centers responded differently 

regarding two practices in mathematics and one in tutoring than centers that were not providers. 

The SES providers were less likely than non-SES providers to have students practice basic 

mathematics operations in all or most activities (46 percent versus 68 percent). Furthermore, 

SES-providing centers were less likely to have students solve mathematical equations in some 

activities than non-SES providers (59 percent versus 85 percent). However, SES providers were 

more likely to offer 2–4 hours of tutoring weekly than non-SES providers (56 percent versus 36 

percent). 

Summary 

Instructional supports for program quality were more widely available in school-based 

centers than in nonschool-based centers. School-based centers paid staff who held teaching 

certificates more than did nonschool-based programs. About one-third of the staff had joined the 

center within the past year. Low pay and the inability to offer staff full-time positions were 

greater challenges to retention in nonschool-based centers than in school-based centers. With 

respect to coordinating their activities with the regular school day, nonschool-based centers faced 

greater barriers to obtaining information about student needs than did school-based centers. This 

is not surprising, given the beneficial location that school-based centers possess. 

Professional development was not emphasized in the centers. Although more than half of all 

centers reported that they offered paid professional development opportunities to their staff, site 

visit data suggest that those offerings were limited in time and scope. Four of 12 case study 

centers provided just 1 to 2 days of training or orientation to staff members, and the training 

focused primarily on logistics and program implementation. Content-focused professional 

development is important because it has been linked, in correlational studies, to changes in 

teacher knowledge and practice (Garet et al., 2001). 
4. Organizational Supports for Instructional Quality 54 

It was significant that half the centers reported using some form of student- or school-level 

data to improve program practice, given increasing attention to the use of data to monitor 

progress and enhance the quality of before and after-school programs (Yohalem, WilsonAhlstrom and Yu, 2005). The logic behind these efforts is straightforward: With greater 

accountability for results, educators pay more attention to those results and to their significance 

for ongoing efforts to improve practice. Research in this area, however, is sparse and has not 

been linked to outcomes. 

In nearly all observed activities in reading and mathematics (n = 59), staff used positive 

behavior management strategies and showed warm affect toward students. Such high-quality 

relationships are important because they are linked to greater student engagement with afterschool programming activities and positive youth outcomes, including improved behavior in 

school (Pierce et al., 1999). 
5. Conclusions and Implications for Future Research and Evaluation 55 

5. Conclusions and Implications for Future Research 

and Evaluation 

Nature and Quality of Reading and Mathematics Activities 

Considering that the previous national evaluation noted the centers’ limited focus on 

academics (Dynarski et al., 2003; Dynarski et al., 2004), the fact that nearly all the centers 

funded at the time of this study reported providing activities in reading and mathematics may be 

regarded as a significant shift in approach. The center directors’ responses indicated that reading 

activities were aligned with scientifically based practices: Instruction focused on a broad range 

of skills and employed several strategies for teaching vocabulary, fluency and comprehension. 

Additionally, many centers reported that they asked students to talk or write about something 

they had read. 

Survey findings and observations indicated that mathematics activities may be more limited 

than reading activities. Mathematics instruction focused primarily on two content strands and 

engaged students in tasks with low levels of cognitive complexity. Observation data, while not 

nationally representative, suggest that a more limited range of strategies was used in mathematics 

instruction. Furthermore, in the mathematics activities observed on site visits, students were less 

likely to be on task than were students in reading activities. 

Attendance and Student Participation 

With respect to regular attendance, centers in a nonrepresentative sample reported that 48 

percent of students in centers serving elementary school students attended 60 days or more in the 

last year, or roughly 2 days per week. In this study, 36 percent of middle school students 

attended 60 days or more in the last year. Twenty-eight percent of middle school students in this 

study attended 30–60 days and 36 percent attended fewer than 30 days. In the earlier evaluation, 

20 percent of middle school students attended 51 days or more. Attendance was higher than 

reported in the earlier study for school-based centers but not for nonschool-based centers. These 

data also replicated a general trend found in the other studies: Elementary students attend afterschool activities for more days in the school year than do middle and high school students. These 

data are of limited utility, however, compared with other studies that measured attendance 

directly and distinguished among attendance in different types of programs (summer, before and 

after school). 

Organizational Supports for Instructional Quality 

Two ways that centers’ organizational practices strongly supported instructional quality were 

their use of data and positive staff–student relationships. About half of all centers reported 

making use of data for a variety of purposes, including program evaluation and ongoing review 

of programming activities. Site visitors also observed that staff relationships with students were 

characterized by high levels of warmth and positive behavior management strategies. 

At the same time, professional development was an area that needed greater attention. 

Although more than half of all centers reported offering paid professional development 

opportunities to their staff, site visit data suggested that, in some centers, those opportunities 

were limited in time and scope. 
5. Conclusions and Implications for Future Research and Evaluation 56 

Centers faced three additional challenges to offering high-quality reading and mathematics 

activities: (a) low pay and lack of benefits were barriers to retaining high-quality staff; (b) 

centers did not always have the information they needed about students’ needs and (c) attendance 

policies did not necessarily result in better attendance rates. 

Key Differences Between Different Types of Centers 

There were few reported differences between school-based and nonschool-based centers’ 

math and reading instruction, except that school-based centers were generally more likely to 

report emphasizing higher-order skills. School-based centers were more likely than nonschoolbased centers to report that their instructors asked students to make predictions about something 

they were reading, and answer questions, orally or in writing, on something they had read. At the 

same time, nonschool-based centers had students read teacher-selected books more often than 

school-based centers did. In mathematics as well, school-based centers were more likely than 

nonschool-based centers to report emphasizing advanced topics. School-based centers were also 

significantly more likely than nonschool-based centers to ask students to make charts, graphs and 

tables in all or most activities. 

One of the key differences between school-based and nonschool-based centers was the level 

of instructional support for program quality. School-based centers paid staff with teaching 

certificates at higher rates than nonschool-based programs did. For example, school-based 

centers reported that low pay and lack of benefits were less important in retaining staff than they 

were for nonschool-based centers.. With respect to coordinating their activities with the regular 

school day, nonschool-based centers faced greater barriers to obtaining information about student 

needs and performance than did school-based centers. These barriers present formidable 

challenges to nonschool-based centers in partnering with the school-day instructional staff to 

deliver high-quality programming to address individual student needs. 

Significant differences existed in attendance policies and patterns of attendance based on the 

grade level of students served. In centers serving the elementary grades, students were more 

likely to attend regularly; these centers were also more likely to have policies requiring students 

to attend the program for a minimum number of days to remain enrolled. In contrast, centers 

serving high school students were more likely to have a high proportion of students who attended 

fewer than 30 days in 2005–06 than centers not serving that age group. At the same time, centers 

that served middle and high school grades and that had attendance policies reported higher levels 

of attendance than did centers serving these grades that allowed students to participate on a dropin basis. 

Recommendations for Future Research and Evaluation 

Recommended areas of future study include analyzing the role of instructors’ 

qualifications, the quality of reading programming and students’ and families’ needs for afterschool programs. Finer-grained measures of instructor qualifications may prove more effective in 

predicting differences in instructional quality than the broader measure of teacher certification or 

specialist status. Measures of instructors’ relative confidence in teaching reading and 

mathematics and their knowledge of mathematics teaching may explain these differences (Hill, 

Rowan and Ball, 2005). The current study did not directly measure instructors’ knowledge or 

skills, but future studies could examine those aspects. The content of the professional 
5. Conclusions and Implications for Future Research and Evaluation 57 

development provided for center staff could also be considered as part of instructors’ 

qualifications. Future work examining the quality of reading programming could focus on how 

best to capture information on reading when respondents are after-school program administrators 

(rather than school-day administrators) who may be less familiar with specific instructional 

strategies used in academic programming. Future studies concerned with student and family 

demand for after-school programs could collect information on program enrollment capacity and 

waiting lists, as well as reasons why parents or students chose to enroll in after-school programs. 

Effective strategies that centers implement to increase participation in their programs could also 

be analyzed in this regard. 

Last, because all students who take part in 21st Century Community Learning Centers 

programs also participate in learning experiences at home and at school, and some also 

participate in other after-school programs, longitudinal studies that follow participants over an 

extended period could provide a better understanding of how these learning experiences may 

complement one another and enhance academic achievement. Emerging frameworks (e.g., 

Barron, 2006) within the learning sciences that highlight the need to understand how activities, 

resources and interactions outside of school relate to learning within schools could guide such a 

study. Studying how learning experiences in one setting lead students to pursue learning in 

another setting could be the focus of a specific longitudinal study. Connections might be found at 

21st Century Community Learning Centers programs with a strong focus on developing interest, 

knowledge and skill in a particular area and strong connections to the school day and other 

service providers. A 3- to 5-year study of students who participate in such programs could yield 

important new insights into how 21st Century Community Learning Centers programs can 

enhance outcomes for students. 

 
5. Conclusions and Implications for Future Research and Evaluation 58 
References 59 

References 

Aronson, J., Zimmerman, J., & Carlos, L. (1999). Improving student achievement by extending 

school: Is it just a matter of time? San Francisco: WestEd. 

Baker, D. A., & Witt, P. A. (1995). Evaluation of the impact of two after-school programs. 

Journal of Park and Recreation Administration, 14(3), 60–81. 

Bangert-Downs, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. T. (1991). The instructional 

effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238. 

Barron, B. J. S. (2006). Interest and self-sustained learning as catalysts of development: A 

learning ecology perspective. Human Development, 49, 193–224. 

Beck, I., McKeown, M., & Kucan, L. (2002). Bringing words to life. New York: Guilford. 

Biancarosa, C., & Snow, C. E. (2006). Reading next—A vision for action and research in middle 

and high school literacy: A report to Carnegie Corporation of New York. Washington, DC: 

Alliance for Excellent Education. 

Birmingham, J., Pechman, E. M., Russell, C. A., & Mielke, M. (2005). Shared features of highperforming after-school programs: A follow-up to the TASC evaluation. Washington, DC: 

Policy Studies Associates. 

Borman, G. D., & Dowling, N. M. (2006). Longitudinal achievement effects of multiyear 

summer school: Evidence from the Teach Baltimore randomized field trial. Educational 

Evaluation and Policy Analysis, 28(1), 25–48. 

Brophy, J., & Good, T. L. (1986). Teacher behavior and student achievement. In M. C. Wittrock 

(Ed.), Handbook of research on teaching (3rd ed., pp. 328–375). New York: Simon & 

Schuster. 

Brown, C. (1999). The role of schools when school is out. The Future of Children, 9(2), 139–

142. 

Burstein, L., McDonnell, L., Van Winkle, J., Ormseth, T., Mirocha, J., & Guiton, G. (1995). 

Validating national curriculum indicators. Report prepared for the National Science 

Foundation. Santa Monica, CA: RAND. 

Burton, S., & Blair, E. (1991). Task conditions, response formulation processes, and response 

accuracy for behavioral frequency questions in surveys. The Public Opinion Quarterly, 

55(1), 50–79. 

Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical 

synthesis. Review of Educational Research, 65, 245–281. 

Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different 

feedback conditions on motivational perceptions, interest, and performance. Journal of 

Educational Psychology, 79(4), 474–482. 

Butler, R., & Neuman, O. (1995). Effects of task and ego-achievement goals on help-seeking 

behaviours and attitudes. Journal of Educational Psychology, 87(2), 261–271. 

Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, P. L., & Loef, M. (1989). Using 

knowledge of children's mathematics thinking in classroom teaching: An experimental study. 

American Educational Research Journal, 26(4), 499–531. 

Carruthers, C. P., & Busser, J. A. (2000). A qualitative outcome study of Boys and Girls Club 

program leaders, club members, and parents. Journal of Park and Recreation Administration, 

18(1), 50–67. 
References 60 

Catterall, J. S. (1998). Involvement in the arts and success in secondary school. Washington D. 

C., Americans for the Arts. 

Chaput, S. S., Little, P. M. D., & Weiss, H. B. (2004). Understanding and measuring attendance 

in out-of-school time programs. Cambridge, MA: Harvard Family Research Project. 

Cohen, D., Raudenbush, S., and Ball, D. (2003). Resources, Instruction, and Research 

Educational Evaluation and Policy Analysis, 25, 119-142. 

Corno, L., & Mandinach, E. B. (1983). The role of cognitive engagement in classroom learning 

and motivation. Educational Psychologist, 18(2), 88–108. 

Coyne, M. D., Kameenui, E., Simmons, D. C., & Harn, B. A. (2004). Beginning reading 

intervention as inoculation or insulin: First grade reading performance of strong responders 

to kindergarten intervention. Journal of Learning Disabilities, 37, 90–104. 

Crollick, J., Zhang, J. J., & Fleming, D. S. (2005, April). Program staffing: A critical element for 

after school program quality. Paper presented at the 2005 AAHPERD National Convention 

and Exposition, Chicago. 

de Kanter, A., Adair, J. K., Chung, A.-M., & Stonehill, R. M. (2003). Ensuring quality and 

sustainability in after-school programs: How partnerships play a key role. Yearbook of the 

National Society for the Study of Education, 102(2), 201–220. 

Dixon, R. C., Carnine, D., Lee, D.-S., Wallin, J., & Chard, D. (1998). Report to the California 

State Board of Education and addendum to principal report: Review of high quality 

experimental mathematics research. Eugene, OR, and Austin, TX: National Center to 

Improve the Tools of Educators, University of Oregon and University of Texas at Austin. 

Durlak, J. A., & Weissberg, R. P. (2007). The impact of after-school programs that promote 

personal and social skills. Chicago, IL: Collaborative for Academic, Social, and Emotional 

Learning. 

Dynarski, M., James-Burdumy, S., Moore, M., Rosenberg, L., Deke, J., Mansfield, W., et al. 

(2004). When schools stay open late: The national evaluation of the 21st Century Community 

Learning Centers program: New findings. Washington, DC: Institute of Education Sciences, 

National Center for Education Evaluation and Regional Assistance, U.S. Department of 

Education. 

Dynarski, M., Moore, M., Mullens, J., Gleason, P., James-Burdumy, S., Rosenberg, L., et al. 

(2003). When schools stay open late: The national evaluation of the 21st Century Community 

Learning Centers program: Year 1 report. Princeton, NJ: Mathematica Policy Research. 

Epstein, J. L. (2008). Research meets policy and practice: How are school districts addressing 

NCLB requirements for parental involvement? In A. R. Sadovnik, J. O'Day, G. W. 

Bohrnstedt & K. Borman (Eds.), No Child Left Behind and the reduction of the achievement 

gap: Sociological perspectives on federal educational policy (pp. 261-280). New York: 

Routledge. 

Fleming, D. S., & Zhang, J. J. (2005, April). Program content and operation of after school 

programs. Paper presented at the 2005 AAHPERD National Convention and Exposition, 

Chicago. 

Fredricks, J. A., & Eccles, J. S. (2006). Extracurricular involvement and adolescent adjustment: 

Impact of duration, number of activities, and breadth of activities. Applied Developmental 

Science, 10(3), 132–146. 

Gamoran, A., Porter, A. C., Smithson, J. L., & White, P. A. (1998). Upgrading high school 

mathematics instruction: Improving learning opportunities for low-achieving, low-income 

youth. Educational Evaluation and Policy Analysis, 19(4), 325–338. 
References 61 

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes 

professional development effective? Results From a national sample of teachers. American 

Educational Research Journal, 38(4), 915–945. 

Good, T. L., & Brophy, J. (2008). Looking in classrooms. Boston: Allyn and Bacon. 

Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. Z., Kotloff, L. J., Raley, R., et al. (2002). 

Multiple choices after school: Findings from the Extended-Service Schools Initiative. 

Philadelphia and New York: Public/Private Ventures and Manpower Research and 

Development Corporation. 

Halpern, R. (2002). A different kind of child development institution: The history of after-school 

programs for low-income children. Teachers College Record, 104(2), 178–211. 

Halpern, R., Barker, G., & Mollard, W. (2000). Youth programs as alternative spaces to be: A 

study of neighborhood youth programs in Chicago’s West Town. Youth and Society, 31(4), 

469–505. 

Herman, J. L., Klein, D. C. D., & Abedi, J. (2000). Assessing students’ opportunity to learn: 

Teacher and student perspectives. Educational Measurement: Issues and Practice, 19(4), 16–

24. 

Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for 

teaching on student achievement. American Educational Research Journal, 42(2), 371–406. 

Hoppe, M., Gillmore, M., Valadez, D., Civic, D., Hartway, J., & Morrison, D. (2000). The 

relative costs and benefits of telephone interviews versus self-administered diaries for daily 

data collection. Evaluation Review, 24(1), 102–116. 

Huang, D. (2001). An after-school evaluation system for middle and high school programs. 

NAASP Bulletin, 45(626), 45–61. 

Huang, D., Gribbons, B., Kim, K. S., Lee, C., & Baker, E. (2000). A decade of results: The 

impact of the LA’s BEST after school enrichment program on subsequent student 

achievement and performance. Los Angeles: CRESST. 

Huang, D., Leon, S., La Torre, D., & Mostafavi, S. (2008). Examining the relationship between 

LA’s BEST program attendance and academic achievement of LA’s BEST students. Los 

Angeles: CRESST. 

Huebner, A. J., Walker, J. A., & McFarland, M. (2003). Staff development for the youth 

development professional: A critical framework for understanding the work. Youth and 

Society, 35(2), 204–225. 

Intercultural Center for Research in Education & National Institute on Out-of-School Time. 

(2005). Pathways to success for youth: What counts in after-school. Massachusetts AfterSchool Research Study (MARS) report. Boston: United Way of Massachusetts Bay. 

James-Burdumy, S., Dynarski, M., Moore, M., Deke, J., Mansfield, W., Pistorino, C., et al. 

(2005). When schools stay open late: The national evaluation of the 21st Century Community 

Learning Centers program. Washington, DC: U.S. Department of Education, Institute of 

Education Sciences, and National Center for Education Evaluation and Regional Assistance. 

Koziol, S. M., & Burns, P. (1986). Teachers’ accuracy in self-reporting about instructional 

practices using a focused self-report inventory. Journal of Educational Research, 79(4), 205–

209. 

Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. 

(2006). Out-of-school time programs: A meta-analysis of effects for at-risk students. Review 

of Educational Research, 76, 275–313. 
References 62 

Leffert, N., Benson, P., & Roehlkepartain, J. (1997). Starting out right: Developmental assets for 

children. Minneapolis, MN: Search Institute. 

Little, P. M. D. (2006). Beyond basic training: Promising strategies for connecting out-of-school 

time programs to schools: Learning what works [Electronic version]. The Evaluation 

Exchange, 12. Retrieved May 15, 2010 from http://www.hfrp.org/evaluation/the-evaluationexchange/issue-archive/building-and-evaluating-out-of-school-time-connections/promisingstrategies-for-connecting-out-of-school-time-programs-to-schools-learning-what-works 

Little, P. M. D. (2007). The quality of school-age child care in after-school settings. New York 

and Ann Arbor, MI: National Center for Children in Poverty, Columbia University, and 

Institute for Social Research, University of Michigan. 

Luce, T., & Thompson, L. (2005). Do what works: How proven practices can improve America’s 

public schools. Dallas, TX: Ascent Education Press. 

Mahoney, J. L., Parente, M. E., & Lord, H. (2007). Afterschool program engagement: Links to 

child competence and program quality and content. The Elementary School Journal, 107(4), 

385–404. 

Marquis, K. H., Marquis, M. S., & Polich, J. M. (1986). Response bias and reliability in sensitive 

topic surveys. Journal of the American Statistical Association, 81, 381–389. 

Mullens, J. E. (1998, October). Validating teacher responses on instructional practice self-report 

surveys: Findings from a recent field test. Paper presented at the Association for Public 

Policy Analysis and Management Fall Research Conference, New York. 

Muller, C., & Frisco, M. (1998). Social institutions serving adolescents. In K. Borman & B. 

Schneider (Eds.), The adolescent years: Social influences and educational challenges (pp. 

142–159). Chicago: National Society for the Study of Education. 

Naftzger, N., Kaufman, S., Margolin, J., & Ali, A. (2006). 21st Century Community Learning 

Centers (21st CCLC) analytic support for evaluation and program monitoring: An overview 

of the 21st CCLC Program: 2004–05. Naperville, IL: Learning Point Associates. 

National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the 

National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education. 

National Reading Panel. (2000). Report of the National Reading Panel: Teaching children to 

read. Washington, DC: National Institute of Child Health and Human Development, and 

National Institutes of Health. 

National Research Council. (1999). How people learn: Brain, mind, experience. Washington, 

DC: National Academies Press. 

National Research Council. (2001). Adding it up: Helping children learn mathematics. 

Washington, DC: National Academies Press. 

National Research Council. (2005). How students learn: History, mathematics, and science in 

the classroom. Washington, DC: National Academies Press. 

National Research Council & Institute of Medicine. (2002). Community programs to promote 

youth development. Washington, DC: National Academies Press. 

Noam, G. G. (2003). Learning with excitement: Bridging school and after-school worlds and 

project-based learning. In K. J. Pittman, N. Yohalem, & J. Tolman (Eds.), When, where, 

what, and how youth learn: Blurring school and community boundaries (Vol. 97, pp. 121–

138). San Francisco: Jossey-Bass. 

Oakhill, J., Cain, K., & Bryant, P. E. (2003). The dissociation of word reading and text 

comprehension: Evidence from component skills. Language and Cognitive Processes, 18, 

443–468. 
References 63 

Occupational Online Network (O*NET) Online Summary Report for: 25-9041.00 - Teacher 

Assistants, in Bureau of Labor Statistics 2008 wage data, downloaded on August 13, 2009 

from: http://online.onetcenter.org/link/summary/25-9041.00 

Pechman, E. M., Mielke, M. B., Russell, C., White, R. N., & Cooc, N. (2008). Out-of-school 

time (OST) observation instrument: Report of the validation study. Washington, DC: Policy 

Studies Associates, Inc. 

Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and 

children’s adjustment in first-grade classrooms. Child Development, 70(3), 756–767. 

Pintrich, P. R., & Schrauben, B. (1992). Students’ motivational beliefs and their cognitive 

engagement in classroom academic tasks. In D. Schunk & J. Meece (Eds.), Student 

perceptions in the classroom: Causes and consequences (pp. 149–183). Hillsdale, NJ: 

Erlbaum. 

Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. 

Educational Researcher, 31(7), 3–14. 

Reisner, E. R., White, R., Russell, C. A., & Birmingham, J. (2004). Building quality, scale, and 

effectiveness in after-school programs. Washington, DC: Policy Studies Associates. 

Roffman, J. G., Papagano, M. E., & Hirsch, B. J. (2001). Youth functioning and experiences in 

inner-city after-school programs among age, gender, and race groups. Journal of Child and 

Family Studies, 10(1), 85–100. 

Roschelle, J., Singleton, C., Sabelli, N., Pea, R. D., & Bransford, J. D. (2009). Mathematics 

worth knowing, resources worth growing, research worth noting: A response to the National 

Mathematics Advisory Panel Report. Educational Researcher, 37(9), 610-617. 

Ross, S. M., Troutman, A., Horgan, D., Maxwell, S., Laitinen, R., & Lowther, D. (1997). The 

success of schools in implementing eight restructuring designs: A synthesis of first-year 

evaluation outcomes. School Effectiveness and School Improvement, 8(1), 95–124. 

Rowan, B., Camburn, E., & Correnti, R. (2004). Using teacher logs to measure the enacted 

curriculum: A study of literacy teaching in third-grade classrooms. Elementary School 

Journal, 105(1), 75–102. 

Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about 

teacher effects on student achievement: Insights from the Prospects Study of Elementary 

Schools. Teachers College Record, 104(8), 1525–1567. 

Schwartz, N. (1999). How the questions shape the answers. American Psychologist, 54, 93–105. 

Schwartz, N., & Oyserman, D. (2001). Asking questions about behavior: Cognition, 

communication, and questionnaire construction. American Journal of Evaluation, 22(2), 

127–160. 

Sheldon, S. B. (2008). Getting families involved with NCLB: Factors affecting schools' 

enactment of federal policy. In A. R. Sadovnik, J. O'Day, G. W. Bohrnstedt & K. Borman 

(Eds.), No Child Left Behind and the reduction of the achievement gap: Sociological 

perspectives on federal educational policy (pp. 281-294). New York: Routledge. 

Simpkins, S. D., Ripke, M., Huston, A. C., & Eccles, J. S. (2005). Predicting participation and 

outcomes in out-of-school activities: Similarities across social ecologies. In H. B. Weiss, P. 

M. D. Little, & S. M. Bouffard (Eds.), Participation in youth programs: Enrollment, 

attendance, and engagement (pp. 51–69). San Francisco: Jossey-Bass. 

Snow, C., Burns, M. S., & Griffin, P. (1998). Preventing reading difficulties in young children. Washington, DC: National Academies Press. 
References 64 

Sudman, S., & Bradburn, N. M. (1982). Asking questions. A practical guide to questionnaire 

design. San Francisco: Jossey-Bass. 

Vadeboncoeur, J. A. (2006). Engaging young people: Learning in informal contexts. Review of 

Research in Education, 30(1), 239–278. 

Vandell, D. L., Reisner, E. R., & Pierce, K. M. (2007). Outcomes linked to high-quality 

afterschool programs: Longitudinal findings from the Study of Promising Afterschool 

Programs. In The afterschool impact: Making every minute count. Policy briefing, U.S. 

Congress, Washington, DC. 

Vandell, D. L., Shumow, L., & Posner, J. K. (2004). After-school programs for low-income 

children: Differences in program quality. In J. L. Mahoney, J. S. Eccles, & R. W. Larson 

(Eds.), Organized activities as contexts for development: Extracurricular activities, afterschool, and community programs (pp. 437–456). Mahwah, NJ: Erlbaum. 

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1993). Toward a knowledge base for school 

learning. Review of Educational Research, 63(3), 249–294. 

Wechsler, Samantha, et al. (2001). Meeting the challenge: Financing out-of-school time 

programming in Boston and Massachusetts. Parents United for Child Care. 

Yohalem, N., Wilson-Ahlstrom, A., & Yu, D. (2005). Youth program quality assessment and 

improvement: Celebrating progress and surfacing challenges. A meeting report. Washington, 

DC: The Forum for Youth Investment, Impact Strategies. 

Zimmer, R., Gill, B., Raquin, P., Booker, K., & Lockwood, J. R. (2007). State and local 

implementation of the No Child Left Behind Act: Volume I—Title I school choice, 

supplemental educational services, and student achievement. Washington, DC: U.S. 

Department of Education. 
References 65 

Appendix A 

Descriptions of Academic Instruction for 

Centers Serving All Students 

 
References 66 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 67 

This appendix contains descriptions of academic instruction for all centers, with separate 

descriptions for centers serving middle and high school students. The organization of the 

discussion parallels Chapter 2, which presents data for centers serving elementary school 

students only. Information presented on school-based versus nonschool-based is for all centers, 

including centers that serve elementary, middle or high school students or students at multiple 

levels. Data presented on centers serving middle school students cover any center that included 

middle school students, and data presented on centers serving high school students cover any 

center that served high school students. 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 68 

Exposure to Academics After School 

Across all centers, students spent similar amounts of time per week in reading and 

mathematics activities, but they spent more time working on homework and in arts or 

recreation activities. Overall, almost a quarter of centers reported that students participated in 

reading activities and 19 percent of centers conducted mathematics activities for more than 4 

hours per week (see Exhibit A-1). In contrast, higher percentages of centers indicated that 

students participated in both homework (35 percent) and arts or recreation activities (35 percent) 

for more than 4 hours. Because centers were open slightly less than 16 hours per week, on 

average, student participation of 4 hours per week in a particular activity amounted to about 25 

percent of the available time. 

Exhibit A-1 

Percentage of Centers That Reported Participation in Activities by a Typical Student, by 

Type of Activity and Hours of Engagement per Week 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 5. 

n = 504 

Exhibit reads: Thirty-five percent of all centers reported that a typical student participated in homework 

activities for more than 4 hours per week. 

3%

6%

11%

19%

23%

35%

35%

15%

20%

31%

39%

44%

36%

39%

61%

58%

46%

38%

29%

28%

24%

21%

17%

11%

5%

3%

1%

2%

0% 20% 40% 60% 80% 100%

Health or social services

Science 

Tutoring

Mathematics

Reading

Arts or recreation enrichment

Homework

Percent of Centers Reporting

Reported Student Activities

More than 4 hours

2 - 4 hours

Less than 2 hours

Not offered

Hours per Week 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 69 

Reading Activities 

Reading Skills Emphasized 

More than half of centers reported that most or all reading activities15 included the 

skills emphasized by the National Reading Panel (2000). Although phonemic awareness and 

phonics instruction were emphasized in secondary reading activities, centers reported very few 

differences in the skills that were emphasized in different grade levels. The largest percentage of 

centers serving middle schools reported emphasizing comprehension and vocabulary in all or 

most activities (65 percent and 62 percent, respectively). In particular, there was a greater-thanexpected emphasis on phonics and phonemic awareness at the secondary level. Somewhat 

surprisingly, 39 percent reported focusing on phonics skills, and 37 percent reported focusing on 

phonemic awareness in all or most activities (see Exhibit A-2). 

Exhibit A-2 

Percentage of Centers Serving Middle School Students That Reported Reading Activities 

Emphases, by Reading Skill Emphasized and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 22. 

n = 315 

Exhibit reads: Sixty-five percent of centers serving middle school students reported emphasizing comprehension 

in most or all enrichment activities in reading. 

 

15 Reading enrichment activities are defined as structured activities designed to build students’ literacy skills. 

Reading enrichment may include scheduled time for independent student reading, writing and literacy enrichment 

activities, but not homework assistance. 

37%

39%

49%

62%

65%

45%

45%

41%

33%

32%

18%

17%

9%

5%

4%

0% 20% 40% 60% 80% 100%

Phonemic awareness

Phonics

Fluency

Vocabulary

Comprehension

Percent of Centers Reporting

Reading Skills Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 70 

Centers serving students in high schools revealed few differences from centers serving 

younger students in the reading skills being emphasized. The largest percentage reported 

stressing comprehension and vocabulary in all or most activities (74 percent and 71 percent, 

respectively) (see Exhibit A-3). Although teaching phonics and phonemic awareness are 

typically not part of local and state standards at the secondary level, a large percentage of centers 

reported emphasizing those skills in their academic activities. Forty-five percent of centers 

serving high school students reported focusing on phonics in all or most reading activities, and 

39 percent of centers serving this age group stressed phonemic awareness. The study team did 

not collect site visit data in high school–serving centers, so the reported data cannot be confirmed 

or challenged by observations. Nonetheless, the study team finds it likely that high school 

directors overreported the frequency with which academic activities focused on these skills. 

Another possible explanation for the pattern is that students in these high schools struggled with 

the most basic of reading skills; enrichment targeted to them reflected their needs. 

Exhibit A-3 

Percentage of Centers Serving High School Students That Reported Reading Activities 

Emphases, by Reading Skill Emphasized and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 22. 

n = 107 

Exhibit reads: Seventy-four percent of centers serving high school students reported emphasizing comprehension 

in most or all enrichment activities in reading. 

Use of Instructional Strategies for Teaching Vocabulary, Fluency and 

Comprehension 

Although 99 percent of all centers reported using at least one strategy in reading, the 

most commonly reported reading activities were designed to foster more independent 

reading and greater comprehension. About half of centers (46 percent) reported that they 

asked students to explain or support statements about something they read in all or most reading 

activities. About a third of centers had students talk about or write answers to questions on 

39%

45%

52%

71%

74%

44%

36%

35%

23%

20%

17%

18%

13%

6%

6%

0% 20% 40% 60% 80% 100%

Phonemic awareness

Phonics

Fluency

Vocabulary

Comprehension

Percent of Centers Reporting

Reading Skills Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 71 

something they had read (39 percent), asked students to help other students understand new 

words (35 percent) and asked students to make predictions about a text they are reading (35 

percent) (see Exhibit A-4). 

Exhibit A-4 

Percentage of Centers That Reported Student Participation in Instructional Strategies in 

Reading Activities, by Strategy and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 21. 

n = 501 

Exhibit reads: Forty-six percent of centers reported having students explain or support their understanding of what 

they had read in most or all reading enrichment activities. 

Reports of instructional strategies differed by grade level and center type. Centers serving 

elementary students were less likely than centers serving other grade levels to have students talk 

with peers about what they read and more likely to have students explain or support something 

they read. These centers were also more likely to have students make prereading predictions. 

Centers serving middle school grades were less likely than those serving other grades to have 

18%

18%

27%

28%

33%

33%

33%

35%

35%

39%

46%

69%

54%

62%

56%

59%

56%

63%

52%

56%

54%

47%

13%

28%

11%

16%

9%

11%

4%

13%

9%

7%

7%

0% 20% 40% 60% 80% 100%

Read books selected by the leader

Describe the style or structure of a text they have 

read

Do a group activity or project about what they 

have read

Discuss different interpretations of what they 

have read

Read aloud

Generalize or make inferences about something 

they have read

Talk with peers about what they have read

Make predictions about something they are 

reading

Help other students understand new words

Talk/write about answers to questions about 

what they have read

Explain or support their understanding of what 

they have read

Percent of Centers Reporting

Reading Activity Strategies

Most or all activities

Some activities

None or almost none

Explain or support their understanding of what they 

have read

Talk/write about answers to questions about what 

they have read

Help other students understand new words

Make predictions about something they are reading

Talk with peers about what they have read

Generalize or make inferences about something they 

have read

Read aloud

Discuss different interpretations of what they have 

read

Do a group activity or project about what they have 

read

Describe the style or structure of a text they have 

read

Read books selected by the leader

Inclusion Levels 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 72 

students explain or support something they read. All these differences are consistent with 

differences in reading skill and are most likely associated with student grade level. School-based 

centers were more likely than nonschool-based centers to report that their instructors asked 

students to make predictions about something they were reading. At the same time, nonschoolbased centers had students read teacher-selected books more often than did school-based centers. 

Mathematics Activities 

Mathematics Skills Emphasized 

Among centers serving middle school students, 64 percent reported that they emphasized 

operations with whole numbers (such as addition, subtraction, multiplication and division) in all 

or most mathematics activities (see Exhibit A-5). About half targeted number sense in all or most 

mathematics activities (53 percent). Centers serving middle school students gave less attention to 

advanced content than to operations with numbers and other content: 18 percent emphasized 

algebraic concepts, 14 percent focused on geometry and 11 percent focused on data analysis in 

all or most activities. 

Exhibit A-5 

Percentage of Centers Serving Middle School Students That Reported Emphases in Math 

Activities, by Type of Content Emphasized and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 24. 

n = 314 

Exhibit reads: Sixty-four percent of centers serving middle school students emphasized operations with whole 

numbers in most or all mathematics enrichment activities. 

11%

14%

18%

24%

26%

53%

64%

66%

71%

62%

73%

66%

40%

34%

22%

15%

19%

3%

8%

7%

2%

0% 20% 40% 60% 80% 100%

Data analysis

Geometric concepts or geometry

Algebraic concepts or algebra

Measurement

Operations with decimals or fractions

Number sense

Operations with whole numbers

Percent of Centers Reporting

Math Activity Content Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 73 

Among centers serving high school students, 69 percent reported that they emphasized 

operations with whole numbers in all or most mathematics activities (see Exhibit A-6). Fiftyeight percent stressed number sense in all or most activities. Geometry and data analysis received 

the least amount of attention among centers serving high school students: 20 percent focused on 

geometric concepts and 16 percent emphasized data analysis in all or most activities. 

Exhibit A-6 

Percentage of Centers Serving High School Students That Reported Emphases in Math 

Activities, by Type of Content Emphasized and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 24. 

n = 117 

Exhibit reads: Sixty-nine percent of centers serving high school students emphasized operations with whole 

numbers in most or all mathematics enrichment activities. 

Across all grade levels, school-based centers were more likely than nonschool-based centers 

to emphasize advanced topics in mathematics. In particular, school-based centers were 

significantly more likely than nonschool-based centers to give at least some emphasis to 

geometry (86 percent versus 72 percent), algebra (79 percent versus 65 percent) and data 

analysis (74 percent versus 62 percent). For foundational topics such as number sense and 

operations, however, both types of centers had similar emphases. 

16%

20%

29%

30%

30%

58%

69%

65%

68%

66%

63%

52%

34%

27%

19%

12%

6%

7%

18%

8%

5%

0% 20% 40% 60% 80% 100%

Data analysis

Geometric concepts or geometry

Operations with decimals or fractions

Measurement

Algebraic concepts or algebra

Number sense

Operations with whole numbers

Percent of Centers Reporting

Math Activity Content Emphasized

Most or all activities

Some activities

None or almost none

Inclusion Levels 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 74 

Use of Instructional Strategies for Teaching Mathematics 

The observed mathematics activities in the learning centers corresponded to instructional 

strategies. In particular, the incorporation of low-complexity cognitive tasks such as practicing 

basic mathematics facts as well as high-complexity tasks such as multistep problem solving in 

the learning centers was aligned with research identifying these methods as effective 

instructional strategies (Porter, 2002). Similarly, providing students the opportunity to explain 

the reasoning behind a problem solution helps to build proficiency in computation and 

translation of word problems. It also helps teachers to ensure that students have the conceptual 

framework necessary for understanding the grade-level mathematics (National Mathematics 

Advisory Panel, 2008). Experimental research studies have shown that computers and graphing 

calculators have a positive effect on mathematics achievement (Dixon et al., 1998). 

The National Mathematics Advisory Panel (2008), which was established to promote 

American students’ knowledge of mathematics and improve their performance in the subject, 

concluded that cooperative learning has significant effects on student outcomes, particularly in 

teaching young children mathematical operations. Strategies include team-assisted 

individualization and peer-assisted learning strategies in which heterogeneous students are 

grouped so that those who are weak in a specific skill can work with others who are strong in 

that area. Additional evidence suggests that teaching mathematics from a real-world perspective 

positively affects student academic achievement.

Although 98 percent of all centers reported using at least one strategy, the most 

common strategies used in all or most mathematics activities were practicing mathematical 

facts (63 percent) and asking students to explain their reasoning in solving a problem (47 

percent). Twenty-three percent of all centers reported having students use computers to support 

mathematics activities or to solve open-ended problems in mathematics (see Exhibit A-7). 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 75 

Exhibit A-7 

Percentage of Centers That Reported Student Participation in Instructional Strategies in 

Math Activities, by Strategy and Inclusion Level 

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 23. 

n = 501 

Exhibit reads: Sixty-three percent of all centers reported asking students to practice basic mathematics facts in most 

or all instructional activities in mathematics. 

Mathematics activities differed in certain respects among center types and the grade levels 

served. For example, school-based centers were significantly more likely than nonschool-based 

centers to ask students to make charts, graphs and tables in all or most activities (20 percent 

versus 10 percent). Centers serving elementary school students were less likely than centers 

serving older groups to report using graphing calculators in all or most activities. 

Instructor Education and Experience 

Centers reported that more than half of staff providing instruction in reading and 

mathematics had prior experience as certified classroom teachers. Fifty-five percent of the 

staff who provided instruction in reading or mathematics had been regular classroom teachers, 

and 23 percent had been instructional specialists in reading or mathematics. Twenty-three 

percent of staff providing instruction in reading or mathematics had served as classroom aides. 

5%

19%

23%

24%

28%

42%

47%

63%

33%

69%

56%

61%

54%

50%

46%

33%

63%

12%

21%

15%

17%

8%

8%

4%

0% 20% 40% 60% 80% 100%

Use graphing calculators to solve problems

Make charts, tables or graphs

Use computers to practice mathematics

Solve problems that have many possible correct 

answers

Write mathematical equations to solve a problem

Solve problems in which students figure out solving 

method

Explain the reasoning behind how they solved a 

problem

Practice basic mathematics facts

Percent of Centers Reporting

Instructional Strategies in Math

Most or all activities

Some activities

None or almost none

Inclusion Levels 

Practice basic mathematics facts

Explain the reasoning behind how they solved a 

problem

Solve problems in which students figure out 

solving method

Write mathematical equations to solve a problem

Solve problems that have many possible correct 

answers

Use computers to practice mathematics

Make charts, tables or graphs

Use graphing calculators to solve problems
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 76 

Centers reported that nearly two-thirds of reading and mathematics instructors had a 

bachelor’s degree or higher (64 percent for reading and 64 percent for mathematics). A 

third of instructors reported having a bachelor’s degree or a bachelor’s degree plus some 

graduate training (36 percent for reading and 37 percent for mathematics). 

Feedback to Students and Parents on Student Progress 

The majority of centers providing academic instruction reported providing informal 

feedback to students; written or formal feedback was less common. For instance, across all 

grade levels, 86 percent of school-based centers reported providing verbal feedback to students 

on work assigned by the school and 85 percent provided feedback on their behavior. In contrast, 

about a quarter of centers provided students written feedback on work assigned by the school (23 

percent) and on work assigned in the center (25 percent). Nonschool-based centers were 

significantly more likely to provide written feedback on work assigned in the center than were 

school-based centers (35 percent versus 25 percent) (see Exhibit A-8). 

Exhibit A-8 

Percentage of Centers That Provided Feedback About Their Progress, by Type of 

Feedback and Center Type 

Feedback 

Center Type 

Nonschoolbased Schoolbased 

Verbal feedback from center staff: 

On work assigned by the school 81% 86% 

On work assigned at the center 76% 67% 

On student behavior 86% 85% 

Written feedback from center staff: 

On work assigned by the school 25% 23% 

On work assigned at the center 35%* 25%* 

On student behavior 44% 44% 

“Points” or rewards for program accomplishments 66%* 48%* 

Certificates or awards for accomplishments 74%* 62%* 

Source: National Survey, Item 18. 

n = 500 

*p < 0.05 

Exhibit reads: Eighty-six percent of all school-based centers provided verbal feedback to students on work assigned 

by the student’s school. 

Centers serving elementary school students were significantly more likely than centers 

serving older students to report providing verbal feedback to students on work assigned at the 

center. Seventy-two percent of centers serving elementary school students reported providing 

this type of feedback, compared with 66 percent of middle school centers and 67 percent of high 

school centers. Significantly more centers serving elementary school students than centers 

serving other age groups also reported providing written feedback on student behavior. 

A majority of centers reported providing frequent, informal feedback about student 

progress to parents. More than half (55 percent) of centers reported talking with parents nearly 

every week when the parents picked up or dropped off their child. Five percent sent home 

written reports of students’ progress in the after-school program nearly every week, but 54 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 77 

percent never did so or did so rarely. Similarly, 2 percent of centers held parent–center staff 

conferences nearly every week, and 56 percent never or rarely held conferences. There were no 

substantive differences by grade level served or type of center. Elementary-serving centers were 

more likely than centers not serving this age group to provide feedback to parents about student 

behavior. Nonschool-based centers were more likely than school-based centers to provide 

feedback on work assigned and to use points and certificates for accomplishments. 

 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 78 
Appendix B 

Case Study Observations 

 
Appendix B: Case Study Observations 80 

This appendix presents analyses conducted by coders working independently to characterize 

organizational goals and practices of 21st Century Community Learning Centers programs 

visited in fall 2006 and spring 2007. The focus of the analysis is identifying and describing 

practices in after-school programming, and identifying the factors that influence those practices. 

The observation instrument used to collect this data is available in Appendix F. 

Case Study Observations: Identifying Practices 

Alignment of Observed Enrichment Activities With Practices 

Observed enrichment activities were partially aligned with practices. Analysts rated 

enrichment activities observed during the site visits for their alignment with practices in four 

broad domains: content, instruction, engagement and relationships. For each of these domains, 

the observed activities were rated on a four-point scale. Raters assigned one point if the observed 

activity did not occur, two points if the activity focused on the feature but there was no 

alignment, 3 points if the activity focused on the feature and there was partial alignment and four 

points if the feature was a focus and there was good alignment with the definition of practice. 

The average ratings of quality were highest in engagement and relationships (see Exhibit 

B-1). The typical activity was partially aligned in terms of engagement (average rating = 3.6) and 

relationships (average rating = 3.3). There was more limited alignment in content (average rating 

= 2.9) and instruction (average rating = 2.7). 

Although different from classroom instruction, enrichment programming in the after-school 

setting maintains connections across learning environments, integrates learning goals and 

deepens exploration and skill acquisition for children (Noam, 2003). In addition to focusing on 

academics, the observed enrichment activities actively engaged learners in constructing 

knowledge and building skills, and targeted particular social or personal skills. Although it is not 

possible to generalize the case study findings to the entire population of 21st Century 

Community Learning Centers, these findings suggest that centers are employing practices and 

approaches that are aligned with research on program quality. 
Appendix B: Case Study Observations 81 

Exhibit B-1

Ratings of Observations in Enrichment Activities 

Source: Observation Activity Rating Rubric. 

n = 57 

Scale: 1 = did not occur; 2 = focus, no alignment; 3 = focus, partial alignment; 4 = focus, good alignment 

Exhibit reads: The level of student engagement for enrichment activities was rated 3.6 by observers in terms 

of alignment with practice. 

Reading and Mathematics Activities 

Alignment of Observed Reading Activities With Practices 

Analysts rated the reading activities observed during the site visits for their alignment with 

practices in four broad domains: content, instruction, engagement and relationships. For each of 

these domains, the observed activities were rated on a four-point scale as described earlier. The 

“content” and “instruction” categories are quality ratings made by observers and not simply 

occurrences of content. 

Observed reading activities rated high in terms of student engagement (see Exhibit B-2). 

The typical activity had good alignment in engagement (average rating = 3.8) and partial 

alignment in relationships (average rating = 3.3) and content (average rating = 3.3). There was 

more limited alignment with instructional practices (average rating = 2.9). 

2.7

2.9

3.3

3.6

1234

Instruction

Content

Relationships

Engagement

Average Rating

Practices
Appendix B: Case Study Observations 82 

Exhibit B-2 

Ratings of Observations in Reading Activities 

Source: Observation Activity Rating Rubric. 

n = 38 

Scale: 1 = did not occur; 2 = focus, no alignment; 3 = focus, partial alignment; 4 = focus, good alignment 

Exhibit reads: The level of student engagement for reading enrichment activities was rated 3.8 by 

observers in terms of alignment with practice. 

Mathematics Activities 

Alignment of Observed Mathematics Activities With Practices 

Analysts rated the mathematics activities observed during the site visits for their alignment 

with practices in four broad domains: content, instruction, engagement and relationships, as 

described earlier. The “content” and “instruction” categories are quality ratings made by 

observers and not simply occurrences of content. 

The content of observed mathematics activities aligned with practice (see Exhibit B-3). 

The typical activity had good alignment in content (average rating = 3.7) and partial alignment in 

engagement (average rating = 3.3) and relationships (average rating = 3.1). Alignment was more 

limited in instruction (average rating = 2.9). 

2.9

3.3

3.3

3.8

1234

Instruction

Content

Relationships

Engagement

Average Rating

Practices
Appendix B: Case Study Observations 83 

Exhibit B-3 

Ratings of Observations in Mathematics Activities 

Source: Observation Activity Rating Rubric. 

n = 19 

Scale: 1 = did not occur; 2 = focus, no alignment; 3 = focus, partial alignment; 4 = focus, good alignment 

Exhibit reads: The level of student engagement for mathematics enrichment activities was rated 3.3 by 

observers in terms of alignment with practice. 

 

2.9

3.1

3.3

3.7

1234

Instruction

Relationships

Engagement

Content

Average Rating

Practices
Appendix B: Case Study Observations 84 

Example of a Mathematics Activity 

In a mathematics activity that scored a 4 on the content and engagement observation rating scales, 

indicating good alignment with practices, a center used computerized instruction aligned with the 

state’s content standards. The program, which was also used during the school day, included a 

continuing assessment for students so that they were directed to new skills as they successfully 

mastered concepts. It also prompted students to call on a teacher when they repeated errors and 

could benefit from direct instruction. 

In the observed activity, after-school students worked at the computer under the supervision of their 

regular classroom teacher. Students worked at different skill levels on different topics, including 

number sense, expanded notation, measurement and place value. As they worked independently, the 

teacher circulated, explaining math concepts and procedures to individual students as necessary. 

When students mastered objectives, the program presented new material to them. If they stumbled, 

the computer alerted the teacher’s diagnostic screen, and the teacher moved to the struggling 

students with direct assistance. When students faltered, a pop-up “Teacher Time” window appeared, 

and the student summoned the teacher to diagnose the error. The teacher, together with the 

computerized program, kept students on task and moving swiftly through the instruction. When 

students made mistakes, the teacher assured them that “making mistakes is how we learn the most.” 

In summary, these findings suggest that reading activities address a broader range of 

processes than mathematics activities. Although reading instruction across center types focused 

on each of the reading processes emphasized by the National Reading Panel, the content strands 

of mathematics were not well represented in center activities. From the observation data, 

instructional strategies of mathematics activities appeared to be limited, suggesting a narrow 

range of instruction. Furthermore, the cognitive demand of the reading activities observed—both 

as part of formal instruction and in enrichment activities—was generally higher than the demand 

in activities where mathematics was explicitly taught or embedded. This trend was especially 

evident among nonschool sites, which also had fewer certified teachers offering instruction.
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 85 

Appendix C 

Sampling Approach and Weighting of Surveys 

 
Appendix A: Descriptions of Academic Instruction for Centers Serving All Students 86 
Appendix C: Sampling Approach and Weighting of Surveys 87 

Center Director National Survey 

Intended Survey Respondents 

The national survey was administered to the center director in charge of after-school services 

at sampled sites. A “site” was generally defined as a location where 21st Century Community 

Learning Centers services were offered. 

Sampling Approach 

The evaluation team used the Profile and Performance Information Collection System 

(PPICS) database, an online data collection and reporting system that collects grantee- and 

center-level data, to design the sample. To ensure adequate representation across characteristics 

of interest, the sample was stratified by the school levels the center served (elementary, 

secondary or both), the center location (school-based or nonschool-based) and the percentage of 

students eligible for free or reduced-priced lunch. The team identified 5,122 centers that met the 

following survey population-inclusion criteria: 

1. At least one center was expected to be funded for the duration of the study. 

2. The center offered academic enrichment services in mathematics, reading and 

technology. 

3. The center had been in operation for at least 1 full school year by the time of the survey. 

A total of 4,347 centers did not meet the survey population-inclusion criteria. Although nearly all 

sites offered reading and mathematics instruction, 19 percent of centers did not offer technology 

services and thus were excluded from the sample. 

Degree of Precision 

A simple random sample of 600 centers with 516 respondents allowed 95 percent confidence 

intervals to be developed for proportions (e.g., the proportion of centers that had organized 

homework activities) with a half-width of no more than 4.3 percentage points.16 However, 

because of the need to weight the sample to account for nonresponse, the target “effective” 

sample size was approximately 407, or about 79 percent of the number of respondents. That is, 

the 516 respondents for this study provided the same precision of estimation as 407 respondents 

who were selected via a simple random sampling method and were equally weighted. Thus, the 

95 percent confidence intervals were designed to have a half-width of approximately 4.9 

percentage points. If two approximately equally sized groups of centers were compared, 

differences in the means group would have a 95 percent confidence interval no larger than 9.7 

percentage points. That is, descriptive differences between groups of more than 9.7 percentage 

points would be large enough to be practically important and statistically significant. 

 

 

16 If the proportion of the sample with a certain characteristic is greater than or less than 50 percent, the half-width of 

the confidence interval is less than 4.3 percentage points. For example, for a proportion of 10 percent or 90 

percent, the half-width would be 2.6 percentage points. 
Appendix C: Sampling Approach and Weighting of Surveys 88 

The effect size that is detectable with 80 percent power (using a two-sided 5 percent alpha 

level) is a function of the effective sample size for each subgroup. (In general, the study team 

expected a design effect of about 1.2 corresponding to an effective sample size that was about 85 

percent of the actual number of respondents.) 

Selecting the Sample 

From the initial pool of 5,122 centers, the study team selected a random sample of 600 

centers stratified on school-based or nonschool-based centers (with oversampling of the 

community-based centers to derive more precise estimates about the practices of those centers). 

Response Rates 

During follow-up communication with survey nonrespondents, the project directors 

determined that 43 centers should be eliminated from the sample: 11 centers were eliminated 

because they had been defunded; 21 because they had not received program funds, did not serve 

the K–12 population or had no center at the school address supplied; 6 because they were 

duplicates of other centers and 4 because all efforts to obtain correct contact information failed. 

Thus, the size of the sampling frame after eliminating ineligible centers was 558 for the national 

survey. For the national survey, 516 surveys were completed, resulting in a response rate of 92.5 

percent. 

Weighting 

The national survey sample was weighted to reflect the stratified design. However, to be able 

to make statements about the national population of centers, the data were poststratified to reflect 

the full population of eligible centers. The study team also used Deming’s procedure to weight 

the respondents so that the weighted marginal counts were the same as in the universe of sites 

that were funded at the time of the study and that had been in operation at least a year (e.g., the 

weighted number of responding centers serving elementary school students was the same as the 

number of centers serving elementary school students in the universe). The specific 

characteristics used to poststratify the respondents came from the PPICS database and from U.S. 

Census data ZIP code files. The specific variables used from the PPICS database were the 

numbers of students served, program longevity, prior history with offering academic services, 

the primary grade levels served and the number of weeks per year that centers were open. Data 

from the U.S. census ZIP code files used to poststratify respondents were the percentage of a ZIP 

code’s population living in an urbanized area and the percentage of minority population in that 

area. 

In addition, to correct for biases introduced by differences among some subpopulations (e.g., 

elementary and high schools may have different response rates), the evaluation team 

poststratified the respondents on the basis of characteristics available in the sampling frame and 

used Deming’s procedure to weight the respondents so that the weighted marginal counts were 

the same as in the universe (e.g., the weighted number of responding centers serving elementary 

school students was the same as the number of centers serving elementary school students in the 

universe). 
Appendix C: Sampling Approach and Weighting of Surveys 89 

Center Director Focused Telephone Survey 

Intended Survey Respondents 

The focused telephone survey targeted the center director (the same respondent to whom the 

national survey was sent) as the respondent. In addition to asking for information from the center 

director, the telephone survey asked a series of questions designed to elicit information about 

three staff members. The research team supplied the center director with an algorithm for 

randomly selecting appropriate staff members as targets for the questions in the survey. 

Sampling Approach 

Determining the Sample Size 

To obtain a response rate of 85 percent, 140 centers were selected to receive a focused 

telephone survey along with the national mail survey. Of these 140 centers, 116 completed both 

surveys. 

Degree of Precision 

A simple random sample of 140 centers with 122 respondents would allow 95 percent 

confidence intervals to be developed for proportions (e.g., the proportion of centers that had 

organized homework activities) with a half-width of no more than 9.0 percentage points. 

However, because of the need to weight the sample to account for stratification (into schoolbased and nonschool-based strata) and differential nonresponse between the two types of centers, 

the target “effective” sample size was approximately 84 or about 69 percent of the number of 

respondents. That is, the 122 respondents for this survey would provide the same precision of 

estimation as 69 respondents who were selected via a simple random sample and were equally 

weighted. Thus, the 95 percent confidence intervals would have a half-width of approximately 

12.0 percentage points. If two approximately equally sized groups of centers were compared, 

differences in the proportion of centers in each group with a certain characteristic would have a 

95 percent confidence interval no larger than 24.3 percentage points.17 That is, differences of 

more than 24.3 percentage points would be considered statistically significant. Thus, differences 

that were large enough to be practically important would also be statistically significant. 

Selecting the Sample 

The evaluation team used a sequential sampling approach (similar to that described for 

selecting the national mail survey sample) to select the sample of centers that were surveyed 

using the focused telephone questionnaire. 

 

17 These results are the consequence of general statistical principles. They would apply to any survey with 122 

respondents and a sampling efficiency of 69 percent. Even if the sampling efficiency were 100 percent, the halfwidth would be (at most) 17.4 percentage points. Sampling efficiency could be as high as 100 percent, but would 

not be known until the responses were obtained (allowing us to ascertain the characteristics of respondents versus 

nonrespondents relative to information known about the universe). 
Appendix C: Sampling Approach and Weighting of Surveys 90 

The focused telephone survey collected data on staff qualifications for up to three staff 

members for each center. Statisticians used all the responses about staff in calculating staff 

characteristics. Because each center nominated three staff on average, no center effect had to be 

accounted for in the data. 

To select the target staff members, the evaluation team asked the center director to provide a 

list of staff whose primary responsibilities were planning and delivering academically oriented 

content to students. From that list, the evaluation team randomly selected three individuals and 

asked the center director to provide background information about their roles, credentials and 

qualifications. 

Response Rates 

During follow-up procedures with nonrespondents, the project directors determined that 8 of 

the 140 centers were ineligible for inclusion. Thus, the size of the sampling frame after 

eliminating ineligible centers was 132 for the focused survey. For the focused survey, 122 

surveys were completed, resulting in a response rate of 92.4 percent. A total of 116 centers 

completed both a national and a focused survey, for a response rate of 87.8 percent. 

Weighting 

The telephone survey sample was weighted to reflect the stratified design. In addition, the 

response rate among some subpopulations differed (e.g., elementary and high school centers had 

different response rates). To correct for biases that such differences could introduce, the 

statisticians poststratified the respondents on the basis of characteristics available in the sampling 

frame and used Deming’s procedure to weight the respondents so that the weighted marginal 

counts were the same as in the universe (e.g., the weighted number of responding centers serving 

elementary school students would be the same as the number of centers serving elementary 

school students in the universe). 
Appendix C: Sampling Approach and Weighting of Surveys 91 

Case Study Site Visits 

Purpose of the Site Visits 

The purpose of the case study data collection and analysis in fall 2006 and spring 2007 was 

to find after-school centers whose students’ academic outcomes had improved—“promising 

sites”—compare them with more typical sites, examine the activities and practices of the 

promising centers and make their successful strategies accessible to others. We visited 12 sites at 

elementary and secondary schools. Of particular interest to the Department of Education for the 

site visits was consideration of the links between the degree of academic focus and academic 

outcomes, alignment of student needs with academic activities and the links between schools and 

after-school centers. Overall, the analysis revealed few differences between promising and 

typical sites across the dimensions examined. The analysis used to distinguish between 

promising and typical sites did not provide sufficient warrant for making judgments about which 

goals, strategies and organizational practices were associated with more promising results. As a 

result, the case studies were analyzed with respect to how each case aligned with policy or 

research on best practices. Cross-case analyses were conducted to illustrate the different ways in 

which sites were aligned or seeking to align with policy and research. 

Intended Respondents 

The site visits included at least three interviews with center staff and school personnel (i.e., 

with the center director, the lead instructor at the after-school center and at least one school staff 

member linked to the center). The site visits also included observations of a minimum of six 

after-school activities at the centers, with primary focus on reading and mathematics activities. 

Finally, as part of the site visit, the researchers collected relevant center artifacts (e.g., schedules, 

student worksheets) and existing student academic achievement data to assess whether test 

scores and academic skills were improving (e.g., data collected by the after-school center, feeder 

schools’ Adequate Yearly Progress status). 

An initial planning conversation with the center director helped to identify the appropriate 

individuals for site visit interviews, and the study team developed interview guides for four types 

of staff associated with each center: 

Center director. The center director was the best-placed individual to have salient 

information about the operations and organization of learning at a 21st Century Community 

Learning Centers program. 

Lead center instructor. With the assistance of the center director or main contact, the site 

visitors selected a key staff member who provided academically oriented programming at the 

center. 

School staff. With the assistance of the center director, the site visitors identified a staff 

member from the school who was familiar with the center’s after-school program and/or who 

provided coordination between the school and the center. 
Appendix C: Sampling Approach and Weighting of Surveys 92 

Grant director. In most cases, 21st Century Community Learning Centers have more than 

one location where services are offered, and a grant director coordinates the overall efforts of 

those locations. In those cases, the site visitors interviewed the grant director. When the center 

director and grant director were the same person, the site visitors combined the two interview 

protocols to avoid asking duplicate questions. 

Selecting Case Study Sites 

A top priority in this study was to identify and visit centers that had a strong academic focus 

as evidenced by their student outcomes and other components of practice. Because survey data 

were not available to select cases, data reported in the PPICS database provided the best 

evidence about student outcomes for use in selecting promising sites to visit. Initially, the study 

team selected the following three data elements from PPICS as the primary criteria in this 

selection process: 

1. Percentage of students whose reading and/or mathematics assessment scores increased as 

of the 2003–04 reporting period 

2. Number of hours in reading and/or mathematics instruction per week 

3. Student attendance at a center (for at least 30 days in the 2003–04 reporting period) 

Centers with these promising characteristics were selected as a subset of the stratified sample 

of centers to be surveyed. For the entire sample of centers in the PPICS database, the researchers 

first identified, separately for reading and for mathematics, a mean and standard deviation of 

percentage of students whose scores increased. Next, the researchers identified as “promising” 

two samples of centers: one in which the percentage of students was at least 1 standard deviation 

above the mean for reading, and a second in which the percentage of students was at least 1 

standard deviation above the mean for mathematics. For sites in the promising category, the 

evaluation team then eliminated sites that offered fewer than 5 hours of instruction in its category 

(reading or mathematics) and that indicated that fewer than 75 students had attended for at least 

30 days in the previous year. For each promising site, the researchers also identified a paired site 

to serve as a more typical one for purposes of comparison. Typical sites had to have similar 

ethnic diversity among students; be in a locale of similar size and in the same state and be within 

1 standard deviation, plus or minus, of the average reported percentage of students whose test 

scores increased. Thus, each pair had two sites serving relatively similar populations, but with 

different levels of test score improvement. 

Sorting against these three criteria resulted in a subset of possible centers from which sites 

were selected for further screening. From this subset, the evaluation team identified a pool of 40 

sites for further screening: 10 pairs of reading sites and 10 pairs of mathematics sites. 
Appendix C: Sampling Approach and Weighting of Surveys 93 

Telephone Screening Process 

The next step in the site selection process consisted of conducting telephone screening 

interviews with center directors from the identified centers. In the interviews, trained callers 

collected information about each site’s availability for a site visit and validated the PPICS data 

used to identify the site initially. The callers gathered information about the center’s academic 

goals and activities and about its use of data for assessing program quality and student outcomes. 

They also explored instructional strategies, student participation, links with the school day, 

staffing, student–instructor relationships and staff development. Callers entered the responses 

directly into a database developed for this purpose. Research analysts summarized these data and 

used them to develop a one-page center profile for each of the initial pairs of sites contacted for 

presentation to the Department as candidate sites. 

Joint Review and Final Selection Process 

The research team used the criteria described above to create a subset of centers to visit. 

However, many of the sites proved to be in the same state. To ensure a geographically dispersed 

group of sites, the project directors selected no more than one pair of sites for each subject area 

in a single state.18 This additional criterion reduced the ability to use the telephone screening data 

to select sites; the researchers then prepared a set of recommended sites on the basis of their 

availability and the initial PPICS screening for the Department to review. Twelve centers were 

selected for site visits. 

Site Notification Process 

The Department contacted all the primary sites by letter and by telephone, inviting their 

participation in the case studies. Within a week of mailing the letters, the study team confirmed 

by telephone whether or not the center was willing to participate in the site visits. If a center 

declined to participate in the case studies, the researchers contacted a corresponding alternate 

center immediately to seek to replace the nonparticipating center. 

 

 

18 Florida has two pairs of sites in the sample because the research team had visited one pair of sites in Florida and 

one in California before modifying the selection criteria. 
Appendix C: Sampling Approach and Weighting of Surveys 94 
Appendix D 

Survey Instruments 
Appendix D: Survey Instruments 96 

OMB No. 1875-0239 Approval Expires 10/31/07 

21st Century Community Learning Centers Program Quality Study 

Center Director National Survey 

 

Paperwork Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to 

respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB 

control number for this information collection is 1875-0239. The time required to complete this information 

collection is estimated to average 1.5 hours per response, including the time to review instructions, search existing 

data resources, gather the data needed, and complete and review the information collection. If you have any 

comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: 

U.S. Department of Education, Washington, D.C. 20202-4651. If you have comments or concerns regarding the 

status of your individual submission of this form, write directly to: Margery Yeager, Policy and Program Studies 

Service, U.S. Department of Education, 400 Maryland Ave, S.W., Room 6W213, Washington, D.C. 20202. 
Appendix D: Survey Instruments 97 

Center Director NATIONAL Survey 

March 2006 

 

Note to Survey Recipient: 

This survey should be completed primarily by the director of the 21st Century Community 

Learning Center. Most of the questions can be easily answered by or are specific to the center 

director. However, we encourage you to confer with appropriate staff members for questions 6-7 

and 21-26 of this survey and with any other appropriate staff members for other items that may 

be difficult for you to answer. Answer all questions in the survey only for the center indicated 

in the cover sheet to the survey. 

Topics addressed in the survey include: 

Overall Description of Program 

Program Goals 

Program Approach 

Center Structure and Policies 

Tracking Outcomes 

Student Academic Support 

Reading Support Activities 

Mathematics Support Activities 

Use of Technology Resources 

Staffing 

Staff Pay and Benefits 

Staff Retention 

Please record the names and titles of all respondents in the space below. 

Name(s)/Title(s) of the person(s) responding to 

this survey: 

Primary Respondent 

Name: _____________________________ 

Title: ______________________________ 

Respondent for Items 6-7, 21-26 

Name: _____________________________ 

Title: ______________________________ 

Other Respondents: 

Name: _____________________________ 

Title: ______________________________ 

Contact information (telephone or e-mail): 

Telephone: __________________________ 

E-mail: ______________________________ 

Telephone: __________________________ 

E-mail: ______________________________ 
Appendix D: Survey Instruments 98 

IMPORTANT NOTE: 

Please use a BLACK pen. Blue or red pens and pencil cannot be read by our scanners. 

When asked to mark boxes, make an "X" through the box. 

Sample: Right  Wrong 

Use block printing when you complete any text or numeric responses. If you wish to change 

a response, please mark the correct response and CIRCLE it. 

Overall Description of Program 

Program Goals 

1. To what extent is each of the following an objective or goal of the programs at your center? 

(Mark ONE in each row.)

Not an 

objective 

Minor 

objective 

Major 

objective 

a. Provide a safe environment for students   

b. Help students improve their academic performance 

(e.g., grades, test scores)   

c. Help students develop socially   

d. Provide opportunities for cultural enrichment   

e. Provide recreational activities for students   

f. Provide hands-on activities to supplement academic 

knowledge learning   

g. Help parents and/or other adults with literacy or 

other skills (e.g., parenting)   

h. Help connect students to their community through 

service or other special projects   

i. Identify health or social services that students need   

j. Provide students with positive adult guidance and/or 

mentors 

  
Appendix D: Survey Instruments 99 

2a. Which of the goals or objectives listed in question 1 are your center’s three MOST 

IMPORTANT ones? 

(Select THREE goals or objectives from question 1 and indicate their importance by 

marking the boxes under their corresponding letters in the rows below. Mark ONE in each 

row.)

a b c d e f g h i j 

Most important goal          

Second most important goal          

Third most important goal          

2b. Which of the goals or objectives listed in question 1 do evaluation data show your center is 

making the MOST PROGRESS in achieving? 

(Select THREE goals or objectives from question 1 and indicate your progress toward them 

by marking the boxes under their corresponding letters in the rows below. Mark ONE in 

each row.)

a b c d e f g h i j 

Highest level of progress made 

for this goal          

Second highest level of progress 

made for this goal          

Third highest level of progress 

made for this goal          

3. What indicators does your center use to measure your program’s success? (Mark ALL that 

apply.)

 a. Staff observations of youth 

 b. Student questionnaires 

 c. Parent questionnaires 

 d. Teacher reports 

 e. Assessments administered to students as part of the before- or afterschool program 

 f. Student grades 

 g. Student achievement test scores 

 h. Other (specify): _________________________________________ 
Appendix D: Survey Instruments 100 

Program Approach 

For the following questions, answer for the most recent program year in your program. 

4a. Our program year began: 4b. And it ends or ended: 

Summer 2005 

Fall 2005 

Winter 2005 

Spring 2006 

Summer 2006 

5a. On average, for how many hours per week does a typical student participate in activities at 

the center? 

____ hours 

5b. On average, for how many hours per week does a typical student participate in the following 

activities at your center? (Mark ONE in each row.)

Not 

offered 

No 

more 

than 2 

hours 

2-4 

hours 

5-6 

hours 

More 

than 6 

hours 

a. Homework help in a group setting     

b. Tutoring (one-to-one or peer)     

c. An academic activity in which reading 

instruction or practice takes place (not 

tutoring) 

    

d. An academic activity in which 

mathematics instruction or practice takes 

place (not tutoring) 

    

e. An arts or recreation enrichment activity 

(e.g., sports, outdoor games, crafts, 

theater, music) 

    

f. A science activity, project, or sciencerelated field trip (e.g., to a local science 

museum) 

    

g. A service or other supplemental activity 

(e.g., counseling, drug and violence 

prevention, mentoring) 

    

h. An activity that combines two or more of 

the types of activities listed above. 

(Specify letters of the items:   ) 

    
Appendix D: Survey Instruments 101 

6. What strategies do you use to help meet students’ academic goals? (Mark ALL that apply.) 

 The center I direct seeks to help students meet schools’ academic goals by: 

 a. Adopting the same textbooks and strategies as used during the school day to help 

students with the academic subjects they find difficult 

 b. Supplementing subject-matter instruction with exploratory activities, field trips, 

and/or applied learning 

 c. Providing arts, physical fitness, field trips, and active enrichment learning 

opportunities that our feeder schools (i.e., schools students we serve attend) no longer 

offer 

 d. Providing opportunities for students to participate in subjects not traditionally offered 

in schools 

 e. Providing leadership and/or community service opportunities for students 

 f. Helping parents become more involved in their children’s education 

 g. Providing students with a place where they have freedom to talk about their academic 

difficulties in a safe environment 

 h. Other 

(specify):___________________________________________________________ 

7. Which of the above statements BEST describes your approach to helping students meet their 

academic goals? (Mark ONE.)

a b c d e f g h 

       

Center Structure and Policies 

8. For how many years have before- or afterschool services been offered at this particular center 

(including before the beginning of the 21st Century Community Learning Centers grant)? 

(Mark ONE.)

 a. Never 

 b. 1-2 years 

 c. 3-5 years 

 d. 6-10 years 

 e. More than 10 years 
Appendix D: Survey Instruments 102 

9. For how many years have academically focused before- or afterschool services been offered 

at this particular center (including before the beginning of the 21st Century Community 

Learning Centers grant)? 

(Mark ONE.)

 a. Never 

 b. 1-2 years 

 c. 3-5 years 

 d. 6-10 years 

 e. More than 10 years 

10. What policies regarding attendance does the center have for students? (Mark ONE.)

 a. Attendance is not required; students can drop in and out. 

 b. We require students to attend at least 1 day per week. 

 c. We require students to attend 2-3 days per week. 

 d. We require students to attend every day we offer services. 

 e. Other (specify): _______________________________________ 

11. What consequences are there for students who do not meet the general attendance 

requirements of the center? (Mark ALL that apply.)

 a. Not applicable; we do not require attendance. 

 b. A staff member speaks to the student about an absence after each absence. 

 c. A staff member speaks to the student about an absence after a certain number of 

absences. 

 d. The student is restricted from participating in other activities at the center for a 

period of time. 

 e. The student receives “points” or some other kind of penalty that restricts privileges 

at the center. 

 f. A staff member notifies the student’s parents or caregivers. 

 g. A staff member notifies the student’s school. 

 h. Other (specify): ____________________________________________ 

12. Does the program charge a fee for students to attend, and if so, how much is the fee 

(calculated on a monthly basis)? (Mark ONE.)

 a. The program does not charge a fee 

 b. Under $10 per month 

 c. Between $10 and $25 per month 

 d. Between $26 and $50 per month 

 e. More than $50 per month 
Appendix D: Survey Instruments 103 

Tracking Outcomes 

13. What information is available to the center from school or district officials on student 

performance? (Mark ALL that apply in each row.)

Reading/ 

language arts Mathematics 

Other 

subjects 

a. Test results from the NCLB-required 

state assessment for the whole school   

b. NCLB state assessment test results for 

the individual students at your center   

c. Information about students’ grades   

d. Individual information from schooladministered diagnostic tests   

e. Data from students’ academic portfolios   

f. Quarterly performance test results   

g. End-of-course test results   

h. We do not receive information on student 

performance.   

14. Do you assess students’ academic growth in the program as part of your program 

activities? 

 Yes No If no, skip to Question 17.
Appendix D: Survey Instruments 104 

15. How often does your center use the following information to assess students’ academic 

growth in the program, either for making adjustments to the program or for evaluating the 

program’s success? (Mark ONE in each row.)

Not 

used 

Use 1-2 

times a 

year 

Use 1-2 

times 

per 

month 

Use 

nearly 

every 

week 

Use almost 

daily 

a. Informal staff observations     

b. Written reports from teachers     

c. Verbal reports from teachers     

d. Results from tests administered by 

center staff     

e. Results from tests administered at 

students’ schools     

f. Student self-report or self-assessment     

g. Informal discussions with family 

members     

h. Other (specify):___________________     

16. How does your center use the results of information from student assessments of academic 

growth? (Mark ALL that apply.)

 a. To identify needs for new program offerings. 

 b. To make improvements to practices within existing program offerings. 

 c. To assign students to particular activities. 

 d. To assign students to particular staff who are qualified to help students in identified 

academic areas of need. 

 e. To assign students to particular groups or to older youth in the program. 

 f. To evaluate the success of the program. 

 g. We do not use these data in our programming. 

17. How do you use attendance data in your center? (Mark ALL that apply.)

 a. We keep the attendance data on file for the grant director or evaluator to use. 

 b. We track individual student attendance to ensure that they come regularly. 

 c. We use data on attendance to adjust program offerings. 

 d. Other (specify): _____________________________ 
Appendix D: Survey Instruments 105 

18. What kind of feedback do students receive about the progress they are making or their 

accomplishments at the center? (Mark ALL that apply.)

 a. Verbal feedback from center staff on work assigned by the school 

 b. Written feedback from center staff on work assigned by the school 

 c. Verbal feedback from center staff on work they assign at the center 

 d. Written feedback from staff on work they assign at the center 

 e. Verbal feedback from center staff on behavior 

 f. Written feedback from center staff on student behavior 

 g. “Points” or rewards for program accomplishments 

 h. Certificates or awards for accomplishments 

19. What kind of feedback does the center send to parents about the progress their children are 

making in the before- or afterschool program? (Mark ALL that apply.)

 a. Feedback from center staff on work assigned by the school 

 b. Feedback from center staff on work assigned by the center 

 c. Feedback from center staff on behavior 

 d. “Points” or rewards for program accomplishments 

 e. Certificates or awards for accomplishments 

 f. We do not send written or verbal feedback to parents. 

20. How often do center staff communicate with parents about their students’ needs and 

progress in each of the ways listed below? (Mark ONE in each row.)

Never or 

hardly 

ever 

Once or 

twice a 

term 

Once or 

twice a 

month 

Almost 

every 

week 

a. Talk with parents when they pick up 

their child about the child’s 

difficulties and/or progress 

   

b. Contact parents by phone or e-mail    

c. Send written reports of progress at the 

before- or afterschool program home 

with students 

   

d. Hold center staff-parent conferences    

e. Involve parents in planning new 

programming to match students’ 

needs 

   

f. Hold an event in which parents can 

come and see student academic work    
Appendix D: Survey Instruments 106 

Student Academic Support Activities 

Reading Support Activities 

Academic support in reading refers to structured activities designed to build students’ literacy 

skills. Academic support activities may include scheduled time for independent student reading, 

writing, and literacy enrichment activities, but NOT homework assistance. Note: Depending 

upon the age groups and needs of students you serve, some of these reading tasks may not be 

appropriate for your students. For these questions, please consult with the staff member(s) who 

lead reading support activities at your center. 

21. In what proportion of your reading activities are students asked to do the following? 

(Mark ONE in each row.) 

None 

or 

almost 

none 

Some 

activities 

Most 

activities

All 

activities

a. Read aloud    

b. Talk with peers about what they have read    

c. Write about something they have read    

d. Work in a reading workbook or on a worksheet    

e. Read books students have chosen themselves    

f. Read books selected by the leader    

g. Do a group activity or project about what they have 

read    

h. Discuss different interpretations of what they have read    

i. Explain or support their understanding of what they 

have read    

j. Watch videos/DVDs or listen to music    

k. Help students understand new words    

l. Talk or write about answers to questions about 

something they have read    

m. Make predictions about something they are reading 

before reading or as part of reading    

n. Make generalizations and draw inferences about 

something they have read    

o. Describe the style or structure of a text they have read    
Appendix D: Survey Instruments 107 

22. In what proportion of activities at your center aimed at promoting reading or literacy skills 

do instructors emphasize to the following reading processes? (Mark ONE in each row.)

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

a. Awareness of text or print features    

b. Phonemic awareness    

c. Phonics    

d. Vocabulary    

e. Fluency    

f. Comprehension    

g. Writing    

Mathematics Support Activities 

Academic support in mathematics refers to structured activities dedicated to increasing students’ 

mathematics skills. Academic support activities may include applied enrichment, skill building, 

mathematics games, and scheduled time for tutoring, but NOT homework assistance. Note: 

Depending upon the age groups and needs of students you serve, some of these mathematics 

tasks may not be appropriate for your students. For these questions, please consult with the 

staff member(s) who lead mathematics support activities at your center. 

23. In what proportion of your mathematics activities are students asked to do the following? 

(Mark ONE in each row.) 

None or 

almost 

none 

Some 

activities 

Most 

activities

All 

activities

a. Explain the reasoning behind how they 

solved a problem    

b. Make charts, tables, or graphs    

c. Solve problems in which students have to 

figure out what method to use to solve them    

d. Solve problems that have many possible 

correct answers 

   

e. Use computers to practice math    

f. Write mathematical equations to solve a 

problem    

g. Practice basic math facts (e.g., addition, 

subtraction, multiplication, and division)    

h. Use graphing calculators to solve problems    
Appendix D: Survey Instruments 108 

24. In what proportion of activities at your center aimed at promoting mathematics skills do 

instructors give emphasis to the following? (Mark ONE in each row.)

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

a. Number sense    

b. Operations (e.g., addition, subtraction, 

multiplication, division) with whole 

numbers 

   

c. Operations with decimals or fractions    

d. Measurement    

e. Algebraic concepts or algebra (e.g., 

solving equations)    

f. Geometric concepts or geometry (e.g., 

area and perimeter of shapes)    

g. Data analysis (e.g., mean, median, 

mode)    

Use of Technology Resources 

25. How many Internet-connected computers (desktop or laptop) are available for student use 

at this center? __________ 

 a. None 

 b. Less than 5 

 c. 6 - 10 

 d. More than 10 

26. On average, how often have students at your center used the following kinds of software or 

Internet applications in the past programming year? (Mark ONE in each row.)

Never or 

hardly 

ever 

Once or 

twice a 

month 

Once or 

twice a 

week 

Almost 

every 

day 

a. Internet search tools (e.g., Google, Yahoo!)    

b. Word processing, spreadsheet, or 

presentation software (e.g., Microsoft 

Word, Excel, PowerPoint) 

   

c. Drill and practice software for mathematics 

(e.g., Achieve Now, SuccessMaker)    

d. Drill and practice software for reading (e.g., 

Destination Reading, READ 180)    
Appendix D: Survey Instruments 109 

e. Simulation or modeling software or applets 

that run over the Internet    

f. Photo or video editing software (e.g., 

Photoshop, iMovie)    

g. Web page design tools (e.g., Dreamweaver)    

Staffing 

Staff Pay and Benefits 

27. How many of the following types of staff does your center employ? (Mark ALL that 

apply.)

 Certified teachers: ____ 

 School paraprofessional staff: ____ 

 Other professionals (e.g., youth-development specialists): ____ 

 Other paraprofessionals (e.g., clerks, administrative assistants): _ 

 Unpaid volunteers: ___ 

28. In addition to wages, what benefits are offered to full- and part-time staff? 

(Mark ALL that apply in each row.)

Full-time 

staff 

Part-time 

staff 

Not 

offered 

Don’t 

know 

a. Paid time off for vacation and sick 

leave    

b. Health insurance    

c. Retirement savings plan    

d. Paid attendance at conferences    

e. Paid training or professional 

development    

f. Tuition reimbursement    
Appendix D: Survey Instruments 110 

Staff Retention 

29. Please list the number of staff, aside from you, who have worked at the center (at least 5 

hours per week) for the following durations. (Write a number on each line or check “Don’t 

know.”) 

Number of staff Don’t know 

a. Up to 1 school term __________ 

b. More than 1 school term up to 1 year __________ 

c. More than 1 year up to 2 years __________ 

d. More than 2 years up to 5 years __________ 

e. More than 5 years __________ 

30a. How many staff who work at least 5 hours per week have left the program since the start of 

the 2005–06 school year? 

____ staff 

30b. What are the primary reasons why staff leave the program? (Mark ALL that apply.)

 a. We have not had staff leave our program. 

 b. Low pay 

 c. Lack of benefits 

 d. Inability to offer a full-time position 

 e. Completion/graduation from a program (e.g., Americorps) or from school 

 f. Other (specify: ) 

 g. Don’t know 
Appendix D: Survey Instruments 111 

Overall Program Challenges 

31. How much of a challenge to implementing high-quality programming at your center is each 

of the following? (Mark ONE in each row.)

Not a 

challenge 

A minor 

challenge 

A major 

challenge 

a. The school would like our program to be more academically 

focused.   

b. The space available for center programs is inadequate, 

inappropriate, or unsafe.   

c. We cannot find staff with expertise in teaching the academic 

subjects we offer.   

d. We cannot find volunteers with time and expertise to support 

academic activities at our center.   

e. We cannot afford to offer competitive salaries to staff who 

are qualified to provide supplementary academic instruction 

at our center. 

  

f. We cannot afford to offer potential staff enough hours of 

paid employment.   

g. Staff do not come to work on a reliable schedule.   

h. Volunteers are not available on a reliable schedule.   

i. Staff do not have the skills to help English-language learners 

with their academic development.   

j. There are limited professional development opportunities 

for staff.   

k. We have inadequate instructional materials or programming 

ideas.   

l. We receive insufficient information about how to help center 

participants with the school day curriculum.   

m. There is insufficient information about the academic needs 

of students at our center.   

n. Students face transportation barriers in getting to the center.   

o. Students do not attend the center regularly enough to make 

academic improvements.   

p. Students are not interested in coming to the program.   

q. Students drop out because they lose interest in doing 

academic work after school.   

r. The school’s staff do not respond to center requests to 

coordinate services or resources for participants.   

s. Families are not sufficiently involved in monitoring the 

academic or behavior progress their children make at the 

center. 

  

t. There is too much competition for students’ time from other 

activities, such as jobs or sports.   
Appendix D: Survey Instruments 112 

Thank you for your participation.
Appendix D: Survey Instruments 113 

21st Century Community Learning Centers Program 

Quality Study 

Center Director Focused Survey 

Paperwork Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 

information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 

1875-0239. The time required to complete this information collection is estimated to average 1.5 hours per response, including the time to 

review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have 

any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of 

Education, Washington, D.C. 20202-4651. If you have comments or concerns regarding the status of your individual submission of this form, 

write directly to: Jessica Hausman, Policy and Program Studies Service, U.S. Department of Education, 400 Maryland Ave, S.W., Room 

6W227, Washington, D.C. 20202. 
Appendix D: Survey Instruments 114 

21st Century Community Learning Centers 

Program Quality Study 

Center Director FOCUSED Survey 

March 2006 

Note to Survey Recipient: 

This survey should be completed primarily by the director of the 21st Century Community 

Learning Center. Most of the questions can be easily answered by or are specific to the center 

director. For questions 15-20, we encourage you to discuss responses with the grant director (if 

that is not you). In addition, we encourage you to confer with the three previously identified 

staff members for questions 25-45 of this survey and with any other appropriate staff members 

for other items that may be difficult for you to answer. Answer all questions in the survey only 

for the center indicated in the cover sheet to the survey. 

Topics addressed in the survey include: 

Student Participation 

Recruitment 

Attendance 

Targeting Services 

Tracking of Student Scores on State Tests 

Coordination with Supplemental Educational Services Providers 

Staffing and Training 

Center Director’s Background and Experience 

Roles, Backgrounds, and Experiences of Specific Staff Members 

Organization of Instruction in Reading and Mathematics Activities 

Please record the names and titles of all respondents in the spaces below. 

Name(s)/Title(s) of the person(s) responding to 

this survey: 

Primary Respondent 

Name: _____________________________ 

Title: _____________________________ 

Respondent for Items 25-45 

Name: _____________________________ 

Title: ______________________________ 

Contact information (telephone or e-mail): 

Telephone__________________________ 

E-mail_____________________________ 

Telephone__________________________ 

E-mail_____________________________ 
Appendix D: Survey Instruments 115 

Student Participation

Recruitment 

1. Which of the following groups of students does your center seek to serve? 

(Mark ALL that apply.)

 a. We have open enrollment for all interested students. 

 c. Students who scored “below proficient” on state assessments. 

 d. Students identified by the school as needing special assistance in reading and/or 

math. 

 e. Students who are English-language learners. 

 f. Students who receive free or reduced-priced meals. 

 g. Students who are recommended by teachers or counselors at the feeder schools. 

 h. Students with siblings already attending. 

 i. Other (specify): _____________________________________________ 

2. Which, if any, of the groups listed in question 1 have priority for the center’s services? 

(Select THREE groups from question 1 and indicate them by marking the corresponding 

letter in the rows below. List the top priority group first, followed by the second and third 

priority groups. Mark ONE in each row.)

a b c d e f g h 

a. Top priority        

b. Second priority        

c. Third priority        

3. How many total students are currently enrolled at this center? ________ 

4. What is the maximum number of students the center can serve? ________ 
Appendix D: Survey Instruments 116 

5. Which of the following strategies do you use to engage and retain students at the center? 

(Mark ALL that apply.)

 a. Offering targeted academic instruction in reading and mathematics. 

 b. Offering academic enrichment activities, such as free reading, hands-on projects in 

content areas (e.g., science, history, geography projects), or field trips. 

 c. Providing individualized or small group tutoring. 

 d. Allowing students to choose the activities they participate in. 

 e. Providing both space and time at the center for recreation, snacks, and talking with 

friends. 

 f. Providing a full meal after school. 

 g. Providing family literacy assistance. 

 h. Providing transportation to the center. 

 i. Offering incentives (e.g., field trips, movie tickets) for participation. 

Attendance 

6. How does this center track attendance? (Mark ONE.)

 a. Pen and paper records. 

 b. Using a database management program on a single computer. 

 c. Using a Web-based data system for recording and reporting data on a computerized 

network. 

 d. Using student IDs with barcodes and an electronic scanner linked to computers. 

 e. Other (specify): ______________________________________________________ 

7. What kinds of attendance reports can your center generate easily (within 1-2 hours time)? 

(Mark ALL that apply.)

 a. Total number of students who attended the center in the past week. 

 b. Total number of students who attended the center in the past school year. 

 c. Number of times each student attends, recorded by the week. 

 d. Number of times each student attends, recorded by the day. 

 e. Number of special education students who have attended the program since the 

school year began. 

8. If you selected “d” above in Question 7, answer this question: Last year (2004–05), how 

many students attended this center for the following amounts of time? (Write a number on 

each line below. Write “0” if there are no students in a category.) 

 a. This information is not available in this format for our center. 

 
Appendix D: Survey Instruments 117 

Number of students who: 

Elementary 

school students 

(typically 

grades K-5) 

Middle 

school students 

(typically 

grades 6-8) 

Secondary 

school students 

(grades 9-12) 

b. Attended fewer than 30 days _________ _________ _________ 

c. Attended 30 to 59 days _________ _________ _________ 

d. Attended 60 to 89 days _________ _________ _________ 

e. Attended 90 to 119 days _________ _________ _________ 

f. Attended 120 or more days _________ _________ _________ 

9. For each of the activities below, mark the box that most accurately corresponds to your 

center’s policies regarding student attendance for the activities indicated. 

(Mark ONE in each row.) 

 a. This information is not available in this format for our center. 

Activity 

not 

offered 

Optional for 

all students 

Required 

for lowachieving 

students 

Required 

of ALL 

students 

b. Homework help in a group 

setting    

c. Tutoring (one-to-one or peer)    

d. Academic enrichment activities 

in which reading instruction or 

practice takes place (not 

tutoring) 

   

e. Academic enrichment activities 

in which mathematics 

instruction or practice takes 

place (not tutoring) 

   
Appendix D: Survey Instruments 118 

Tracking of Student Scores on State Tests 

10. Has your center received data from participants’ schools that tells how they performed on 

the state assessment test used for NCLB in 2004–05? (Mark ONE.)

 a. I don’t know. Skip to Question 13. 

 b. No. Skip to Question 13. 

 c. Yes. 

11. Do you have knowledge of the level of proficiency of individual students on these tests? 

(Mark ONE.)

 a. No. Skip to Question 13. 

 b. Yes. 

12. Approximately what percent of participants at your center scored below proficient on the 

most recent assessment that the state administers for NCLB accountability? 

 a. I don’t know. 

 b. Percent scoring below proficient in READING: ____________ 

 c. Percent scoring below proficient in MATHEMATICS: ____________ 

13. Have you or has the school administered any mid-year assessments that could help predict 

whether students scoring “below proficient” are likely to score “proficient” or higher on the 

2005–06 state tests used for NCLB? (Mark ONE.)

 a. I don’t know. Skip to Question 15. 

 b. No. Skip to Question 15. 

 c. Yes. 

14. Approximately what percent of participants enrolled in your center are likely to move from 

scoring “below proficient” to “proficient” on 2005–06 state tests, compared with last year? 

 a. I don’t know. 

 b. Percent likely to move from “below proficient” to “proficient” in READING: ____ 

 c. Percent likely to move from “below proficient” to “proficient” in 

MATHEMATICS: _____ 
Appendix D: Survey Instruments 119 

Coordination with Supplemental Services Providers 

15. This year, in 2005–06, is your center a state-approved provider of supplemental educational 

services (SES)? (Mark ONE.)

 a. I don’t know. Skip to Question 17. 

 b. No. Skip to Question 17. 

 c. Yes. 

16. If your center is a state-approved provider of supplemental educational services under 

NCLB, how many SES students does your center serve THIS SEMESTER? 

 a. I don’t know. 

 b. Our center serves _____________ students in READING under SES this semester. 

 c. Our center serves _____________ students in MATHEMATICS under SES this 

semester. 

17. This year, in 2005–06, does your center collaborate with other supplemental education 

services providers to offer services to participants at your center? (Mark ONE.)

 a. No. Skip to Question 20. 

 b. Yes. 

18. With how many different supplemental service providers does your center collaborate? 

 ______ 

19. How do you coordinate services with those SES providers? (Mark ALL that apply.)

 a. The SES provider(s) help us decide the focus of our programs. 

 b. We coordinate our schedule with the SES provider(s). 

 c. We coordinate academic support activities such as tutoring for individual students. 

 d. We help each other recruit students. 

 e. We work together to communicate with students’ schools. 

 f. Other (specify): _____________________________ 

20. Do students who attend activities at your center receive SES as defined by NCLB from a 

provider other than your center? (Mark ONE.)

 a. I don’t know. 

 b. No. 

 c. Yes. 
Appendix D: Survey Instruments 120 

Staffing and Training 

Center Director’s Background and Experience 

21. What is the highest level of education you have completed? (Mark ONE.)

 a. Less than high school 

 b. High school or GED 

 c. Some college, other classes/training not related to a degree 

 d. Completed 2-year college degree. Major: ____________________ 

 e. Completed 4-year college degree. Major: ____________________ 

 f. Some graduate work. Discipline: ____________________ 

 g. Master’s degree. Discipline: ____________________ 

 h. Ph.D. Discipline: ____________________ 

22. Are you certified to teach in school? (Mark ONE.)

 a. No. 

 b. I have a provisional or emergency teaching credential. 

 c. I have a single-subject certification. 

 d. I have a multisubject certification. 
Appendix D: Survey Instruments 121 

23. What types of experience did you have before you assumed your current position at this 

21st Century Center? How long were you in these positions? (Mark ONE in each row.)

Your previous experience None 

Less 

than a 

year 

1-3 

years 

4-6 

years 

More 

than 6 

years 

a. Classroom teacher     

b. Classroom aide/teaching assistant 

(paraprofessional)     

c. Instructional specialist in reading or 

mathematics     

d. Instructional specialist in arts or 

recreation     

e. Pupil support staff (e.g., school 

counselor, social worker, 

psychologist) 

    

f. School administrator     

g. Administrator at a child/youth center 

or at a park or recreation center     

h. Administrator in a social services 

organization (e.g., health services, 

welfare office) 

    

i. Recreation, youth, or childcare 

worker     

j. Social services or health services 

provider     

k. Camp counselor/leader     

l. Other (specify):_________________     
Appendix D: Survey Instruments 122 

24. What roles and responsibilities do you have at the center? (Mark ALL that apply.)

 a. Supervising paid staff 

 b. Helping develop or train staff 

 c. Coaching and mentoring staff 

 d. Coordinating with school day staff or activities 

 e. Securing and maintaining space for activities 

 f. Managing center budgets 

 g. Fundraising or grant writing 

 h. Community outreach and partnership development 

 i. Internal evaluation of programs 

 j. Instructor for reading activities 

 k. Instructor for mathematics activities 

 l. Instructor for other enrichment activities 

 m. Tutor for individual students 

 n. Counselor or social worker 

Staff Roles, Backgrounds, and Experience 

Please answer each of the following sets of questions for the three (3) staff members that we 

identified from the roster you submitted to us. For each, first indicate the staff member’s role at 

the center, and then answer the questions about his/her background and professional 

development experiences. If your center does not have three staff members, answer questions 

about as many staff members as we identified from the roster you submitted. If you do not have 

the information for a staff member, a researcher can contact that staff member directly by 

telephone or e-mail to set up a time to gather that information. 

25. STAFF MEMBER 1: What roles does this staff member play at the center? (Mark ALL 

that apply.)

 a. Provides homework assistance 

 b. Provides literacy- or reading-related services or instruction 

 c. Provides mathematics-related services or instruction 

 d. Leads activities in (specify): _____________________ 

 e. Other role (specify): _____________________ 
Appendix D: Survey Instruments 123 

26. Approximately how many hours each week is this staff member paid to work at this center, 

including any paid time for training and staff development? (Mark ONE.)

 a. Full-time (35 hours or more per week) 

 b. Half-time (20-34 hours per week) 

 c. Part-time (10-19 hours per week) 

 d. Occasionally (less than 10 hours per week) 

27. How long has this staff member been in this position? (Mark ONE.)

 a. Up to 1 school term 

 b. More than 1 school term up to 1 year 

 c. More than 1 year up to 2 years 

 d. More than 2 years up to 5 years 

 e. More than 5 years 

28. What is the highest level of education this staff member has completed? (Mark ONE.)

 a. Less than high school 

 b. High school or GED 

 c. Some college, other classes/training not related to a degree 

 d. Completed 2-year college degree. Major: ____________________ 

 e. Completed 4-year college degree. Major: ____________________ 

 f. Some graduate work. Discipline: ____________________ 

 g. Master’s degree. Discipline: ____________________ 

 h. Ph.D. Discipline: ____________________ 

29. Is this staff member certified to teach in public school? (Mark ONE.)

 a. No. 

 b. He/she has a provisional or emergency teaching credential. 

 c. He/she has a single-subject certification. 

 d. He/she has a multisubject certification. 
Appendix D: Survey Instruments 124 

30. What types of experience did this staff member have before joining the staff of the center? 

How long was the staff member in each position? (Mark ONE in each row.)

Previous experience None 

Less 

than a 

year 

1-3 

years 

4-6 

years 

More 

than 6 

years 

a. Paid recreation, youth, or 

childcare worker in other beforeor afterschool program 

    

b. Volunteer youth worker in other 

before or afterschool program     

c. Noninstructional paraprofessional 

(e.g., security, cafeteria, social 

services worker) 

    

d. Paraprofessional classroom aide     

e. Instructional specialist in reading or 

mathematics     

f. Instructional specialist in arts or 

recreation     

g. Certified classroom teacher     

h. Administrator at a child/youth 

center, a park or recreation center, or 

social services organization 

    

i. Administrator in a social services 

organization (e.g., health services, 

welfare office) 

    

j. Camp counselor/leader     

k. Other (specify):________________     

31. STAFF MEMBER 2: What roles does this staff member play at the center? (Mark ALL 

that apply.)

 a. Provides homework assistance 

 b. Provides literacy- or reading-related services or instruction 

 c. Provides mathematics-related services or instruction 

 d. Leads activities in (specify): _____________________ 

 e. Other role (specify): _____________________ 
Appendix D: Survey Instruments 125 

32. Approximately how many hours each week is this staff member paid to work at this center, 

including any paid time for training and staff development? (Mark ONE.)

 a. Full-time (35 hours or more per week) 

 b. Half-time (20-34 hours per week) 

 c. Part-time (10-19 hours per week) 

 d. Occasionally (less than 10 hours per week) 

33. How long has this staff member been in this position? (Mark ONE.)

 a. Up to 1 school term 

 b. More than 1 school term up to 1 year 

 c. More than 1 year up to 2 years 

 d. More than 2 years up to 5 years 

 e. More than 5 years 

34. What is the highest level of education this staff member has completed? (Mark ONE.)

 a. Less than high school 

 b. High school or GED 

 c. Some college, other classes/training not related to a degree 

 d. Completed 2-year college degree. Major: ____________________ 

 e. Completed 4-year college degree. Major: ____________________ 

 f. Some graduate work. Discipline: ____________________ 

 g. Master’s degree. Discipline: ____________________ 

 h. Ph.D. Discipline: ____________________ 

35. Is this staff member certified to teach in public school? (Mark ONE.)

 a. No. 

 b. He/she has a provisional or emergency teaching credential. 

 c. He/she has a single-subject certification. 

 d. He/she has a multisubject certification. 
Appendix D: Survey Instruments 126 

36. What types of experience did this staff member have before joining the staff of the center? 

How long was the staff member in each position? (Mark ONE in each row.)

Previous experience None 

Less 

than a 

year 

1-3 

years 

4-6 

years 

More 

than 6 

years 

a. Paid recreation, youth, or childcare 

worker in other before- or 

afterschool program 

    

b. Volunteer youth worker in other 

before or afterschool program     

c. Noninstructional paraprofessional 

(e.g., security, cafeteria, social 

services worker) 

    

d. Paraprofessional classroom aide     

e. Instructional specialist in reading or 

mathematics     

f. Instructional specialist in arts or 

recreation     

g. Certified classroom teacher     

h. Administrator at a child/youth 

center, a park or recreation center, or 

social services organization 

    

i. Administrator in a social services 

organization (e.g., health services, 

welfare office) 

    

j. Camp counselor/leader     

k. Other (specify):________________     

37. STAFF MEMBER 3: What roles does this staff member play at the center? (Mark ALL 

that apply.)

 a. Provides homework assistance 

 b. Provides literacy- or reading-related services or instruction 

 c. Provides mathematics-related services or instruction 

 d. Leads activities in (specify): _____________________ 

 e. Other role (specify): _____________________ 
Appendix D: Survey Instruments 127 

38. Approximately how many hours each week is this staff member paid to work at this center, 

including any paid time for training and staff development? (Mark ONE.)

 a. Full-time (35 hours or more per week) 

 b. Half-time (20-34 hours per week) 

 c. Part-time (10-19 hours per week) 

 d. Occasionally (less than 10 hours per week) 

39. How long has this staff member been in this position? (Mark ONE.)

 a. Up to 1 school term 

 b. More than 1 school term up to 1 year 

 c. More than 1 year up to 2 years 

 d. More than 2 years up to 5 years 

 e. More than 5 years 

40. What is the highest level of education this staff member has completed? (Mark ONE.)

 a. Less than high school 

 b. High school or GED 

 c. Some college, other classes/training not related to a degree 

 d. Completed 2-year college degree. Major: ____________________ 

 e. Completed 4-year college degree. Major: ____________________ 

 f. Some graduate work. Discipline: ____________________ 

 g. Master’s degree. Discipline: ____________________ 

 h. Ph.D. Discipline: ____________________ 

41. Is this staff member certified to teach in public school? (Mark ONE.)

 a. No. 

 b. He/she has a provisional or emergency teaching credential. 

 c. He/she has a single-subject certification. 

 d. He/she has a multisubject certification. 
Appendix D: Survey Instruments 128 

42. What types of experience did this staff member have before joining the staff of the center? 

How long was the staff member in each position? (Mark ONE in each row.)

Previous experience None 

Less 

than a 

year 

1-3 

years 

4-6 

years 

More 

than 6 

years 

a. Paid recreation, youth, or childcare 

worker in other before or afterschool 

program 

    

b. Volunteer youth worker in other 

before- or afterschool program     

c. Noninstructional paraprofessional 

(e.g., security, cafeteria, social 

services worker) 

    

d. Paraprofessional classroom aide     

e. Instructional specialist in reading or 

mathematics     

f. Instructional specialist in arts or 

recreation     

g. Certified classroom teacher     

h. Administrator at a child/youth 

center, a park or recreation center, or 

social services organization 

    

i. Administrator in a social services 

organization (e.g., health services, 

welfare office) 

    

j. Camp counselor/leader     

k. Other (specify):________________     
Appendix D: Survey Instruments 129 

Organization of Instruction in Reading and Mathematics Activities 

Please describe one example of a READING activity at your center that you believe is 

particularly effective in reaching youth. Be sure to describe the content taught, the roles and 

responsibilities of staff, how students are grouped, student-to-adult ratio, and at least one lesson 

or activity that would help an outsider understand what it would be like to participate in the 

activity as a student. 

Please describe one example of a MATHEMATICS activity at your center that you believe is 

particularly effective in reaching youth. Be sure to describe the content taught, the roles and 

responsibilities of staff, how students are grouped, student-to-adult ratio, and at least one lesson 

or activity that would help an outsider understand what it would be like to participate in the 

activity as a student. 

Thank you for your participation. 

 
Appendix D: Survey Instruments 130 
Appendix E 

Supplementary Tables for the Surveys 

Appendix E: Supplementary Tables for the Surveys 133 

Background for This Appendix 

This appendix presents exhibits with descriptive and inferential statistics that have been used 

as the source of data for the exhibits presented in the main text of the 21st Century Community 

Learning Centers Program Quality study. Evaluation team analysts constructed the exhibits from 

responses to both the national survey and the focused telephone survey. 

Many of these exhibits compare the responses of centers in different settings (school-based 

versus nonschool-based) and different levels of students served (elementary, middle and high 

school). The overall results are presented only for items for which there were no statistically 

significant differences between center types or among grade levels. When such statistically 

significant differences did occur, in comparing results for center types and levels with 

statistically significant differences (at the p < 0.05 level), separate exhibits were constructed. 

Statistically significant findings have been marked with an asterisk (*). 

Interpreting differences among levels, however, was complicated because many centers 

served students at multiple levels. For example, some sites served both elementary and middle 

school students. A statistical comparison that did not take this fact into account would have been 

likely to lead to fewer observed differences among sites, and it would also have violated the 

assumption that observations were independent from one another. Therefore, when comparing 

levels, the team addressed this problem by statistically comparing sites that served elementary 

school children against all sites that did not serve children of that age range. Analysts similarly 

compared sites that did and did not serve middle school students and sites that did and did not 

serve high school students. An asterisk appearing in the elementary school column of an exhibit 

indicates that the responses of centers serving elementary school children were statistically 

significantly different from those centers that did not serve elementary school children. 
Appendix E: Supplementary Tables for the Surveys 134 

Exhibit E-1 

Centers Serving Elementary School Students That Reported Participation in Activities by 

a Typical Student, by Type of Activity and Amount of Time of Engagement per Week 

Activities 

Not 

offered 

Less 

than 2 

hours 

2–4 

hours 

More 

than 4 

hours 

Homework help in a group setting 

3% 24% 36% 36%

(0.9%) (2.5%) (2.8%) (2.7%)

n = 10 n = 91 n = 134 n = 154

An arts or recreation enrichment activity (e.g., 

sports, outdoor games, crafts, theater, music) 

1% 28% 38% 33%

(0.4%) (2.6%) (2.8%) (2.7%)

n = 3 n = 115 n = 138 n = 133

An academic enrichment activity in which reading 

instruction or practice takes place (not tutoring) 

1% 28% 47% 25%

(0.6%) (2.6%) (2.9%) (2.5%)

n = 3 n = 116 n = 173 n = 92

An academic enrichment activity in which 

mathematics instruction or practice takes place 

(not tutoring) 

4% 36% 41% 19%

(1.0%) (2.8%) (2.8%) (2.3%)

n = 14 n = 149 n = 148 n = 77

Tutoring (one-to-one or peer) 

11% 46% 31% 12%

(1.9%) (2.9%) (2.7%) (1.9%)

n = 39 n = 179 n = 118 n = 46

A science activity, project or science-related field 

trip (e.g., to a local science museum) 

16% 58% 21% 6%

(2.1%) (2.9%) (2.4%) (1.3%)

n = 68 n = 223 n = 72 n = 21

A service or other supplemental activity (e.g., 

counseling, drug and violence prevention, 

mentoring) 

23% 60% 15% 3%

(2.4%) (2.8%) (2.0%) (0.9%)

n = 80 n = 231 n = 60 n = 14

Source: National Survey, Item 5. 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 24 percent of centers reported students 

participated in homework help in a group setting less than 2 hours a week. 
Appendix E: Supplementary Tables for the Surveys 135 

Exhibit E-2 

Centers Serving Elementary School Students That Reported Reading Activities 

Emphases, by Reading Skill Emphasized and Inclusion Level 

Reading Skills 

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

Phonemic awareness 

8% 43% 35% 15%

(1.5%) (2.9%) (2.8%) (2.1%)

n = 32 n = 168 n = 132 n = 53

Phonics 

7% 41% 38% 14%

(1.4%) (2.9%) (2.8%) (2.1%)

n = 27 n = 158 n = 148 n = 52

Vocabulary 

3% 28% 44% 25%

(0.9%) (2.6%) (2.9%) (2.5%)

n = 12 n = 107 n = 172 n = 97

Fluency 

6% 35% 39% 20%

(1.3%) (2.8%) (2.8%) (2.4%)

n = 25 n = 139 n = 152 n = 72

Comprehension 

3% 24% 43% 30%

(0.9%) (2.4%) (2.8%) (2.7%)

n = 10 n = 96 n = 163 n = 119

Note: Percentages may not equal 100 percent due to rounding. 

Source: National Survey, Item 22. 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 43 percent or 168 of those centers reported 

students’ phonemic awareness was emphasized in some reading enrichment activities. 
Appendix E: Supplementary Tables for the Surveys 136 

Exhibit E-3 

Centers Serving Elementary School Students That Reported Student Participation in 

Instructional Strategies in Reading Activities, by Strategy and Inclusion Level 

Instructional Strategies 

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

Read aloud 

7% 57% 30% 6%

(1.6%) (2.9%) (2.7%) (1.3%)

n = 25 n = 231 n = 110 n = 20

Talk with peers about what they have read 

2% 62% 29% 7%

(0.8%) (2.8%) (2.6%) (1.5%)

n = 14 n = 239 n = 106 n = 24

Write about something they have read 

8% 62% 27% 4%

(1.6%) (2.9%) (2.6%) (1.3%)

n = 30 n = 240 n = 97 n = 14

Read books selected by the leader 

10% 72% 16% 2%

(1.7%) (2.6%) (2.1%) (0.8%)

n = 46 n = 267 n = 63 n = 8

Do a group activity or project about what they 

have read 

10% 60% 26% 5%

(1.7%) (2.8%) (2.6%) (1.2%)

n = 41 n = 235 n = 90 n = 19

Discuss different interpretations of what they have 

read 

14% 57% 24% 5%

(2.0%) (2.9%) (2.5%) (1.2%)

n = 54 n = 230 n = 83 n = 18

Explain or support their understanding of what 

they have read 

7% 43% 42% 8%

(1.4%) (2.8%) (2.8%) (1.6%)

n = 32 n = 173 n = 152 n = 31

Help other students understand new words 

7% 57% 25% 11%

(1.3%) (2.9%) (2.5%) (1.9%)

n = 33 n = 220 n = 89 n = 43

Talk or write about answers to questions about 

something they have read 

6% 52% 31% 11%

(1.3%) (2.9%) (2.7%) (1.8%)

n = 29 n = 204 n = 111 n = 42

Make predictions about something they are 

reading before reading or as part of reading 

11% 51% 28% 10%

(1.7%) (2.9%) (2.6%) (1.8%)

n = 57 n = 201 n = 96 n = 32

Make generalizations and draw inferences about 

something they have read 

10% 55% 27% 7%

(1.8%) (2.9%) (2.6%) (1.6%)

n = 43 n = 223 n = 96 n = 25

Describe the style or structure of a text they have 

read 

28% 53% 15% 4%

(2.6%) (2.9%) (2.1%) (1.2%)

n = 118 n = 201 n = 51 n = 15

Source: National Survey, Item 21. 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 57 percent of centers reported that students were 

asked to read aloud in some reading enrichment activities. 
Appendix E: Supplementary Tables for the Surveys 137 

Exhibit E-4 

Centers Serving Elementary School Students That Reported Emphases in Math 

Activities, by Type of Content Emphasized and Inclusion Level 

Mathematics Content 

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

Number sense 

5% 36% 41% 18%

(1.2%) (2.8%) (2.8%) (2.2%)

n = 18 n = 138 n = 156 n = 73

Operations (e.g., addition, subtraction, 

multiplication, division) with whole numbers 

2% 28% 46% 24%

(0.7%) (2.6%) (2.9%) (2.5%)

n = 8 n = 109 n = 172 n = 97

Operations with decimals or fractions 

10% 68% 18% 4%

(1.7%) (2.7%) (2.2%) (1.0%)

n = 40 n = 255 n = 74 n = 16

Measurement 

3% 75% 19% 3%

(0.8%) (2.5%) (2.3%) (0.9%)

n = 15 n = 279 n = 76 n = 12

Algebraic concepts or algebra 

27% 58% 12% 3%

(2.5%) (2.8%) (1.9%) (0.9%)

n = 116 n = 210 n = 47 n = 13

Geometric concepts or geometry (e.g., area and 

perimeter of shapes) 

17% 70% 11% 1%

(2.1%) (2.6%) (1.8%) (0.6%)

n = 83 n = 253 n = 42 n = 8

Data analysis (e.g., mean, median, mode) 

31% 62% 6% 1%

(2.6%) (2.8%) (1.4%) (0.5%)

n = 130 n = 223 n = 27 n = 7

Source: National Survey, Item 24. 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 36 percent of centers reported that number sense 

was emphasized in some mathematics enrichment activities. 
Appendix E: Supplementary Tables for the Surveys 138 

Exhibit E-5 

Centers Serving Elementary School Students That Reported Student Participation in 

Instructional Strategies in Math Activities, by Strategy and Inclusion Level 

Instructional Strategies 

None or 

almost 

none 

Some 

activities 

Most 

activities 

All 

activities 

Explain the reasoning behind how they solved a 

problem 

6% 47% 32% 14%

(1.4%) (2.9%) (2.7%) (2.0%)

n = 26 n = 184 n = 123 n = 54

Make charts, tables or graphs 

11% 71% 16% 2%

(1.7%) (2.6%) (2.2%) (0.7%)

n = 56 n = 271 n = 52 n = 8

Solve problems in which students have to figure 

out what method to use to solve them 

8% 50% 34% 8%

(1.5%) (2.9%) (2.8%) (1.5%)

n = 37 n = 196 n = 124 n = 30

Solve problems that have many possible correct 

answers 

15% 61% 22% 2%

(2.0%) (2.8%) (2.5%) (0.8%)

n = 63 n = 232 n = 81 n = 10

Use computers to practice mathematics 

22% 54% 20% 4%

(2.4%) (2.9%) (2.3%) (1.1%)

n = 82 n = 217 n = 74 n = 15

Write mathematical equations to solve a problem 

17% 53% 26% 4%

(2.1%) (2.9%) (2.6%) (1.2%)

n = 75 n = 205 n = 85 n = 20

Practice basic mathematics facts (e.g., addition, 

subtraction, multiplication and division) 

3% 29% 46% 22%

(1.0%) (2.6%) (2.9%) (2.4%)

n = 12 n = 116 n = 171 n = 89

Use graphing calculators to solve problems 

70% 27% 2% 0%

(2.6%) (2.6%) (0.8%) (0.1%)

n = 270 n = 102 n = 12 n = 1

Source: National Survey, Item 23. 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 47 percent of centers reported that having 

students explain the reasoning behind how they solved a problem was emphasized in some mathematics enrichment 

activities. 
Appendix E: Supplementary Tables for the Surveys 139 

Exhibit E-6 

Centers Serving Elementary School Students That Provided Feedback About Their 

Progress, by Type of Feedback and Center Type 

Feedback Overall 

Center Type 

Nonschoolbased Schoolbased 

Verbal feedback from center staff on work assigned by 

the school 

87% 86% 87%

(1.9%) (3.3%) (2.1%)

n = 339 n = 106 n = 233

Written feedback from center staff on work assigned by 

the school 

22% 26% 22%

(2.4%) (4.2%) (2.6%)

n = 93 n = 33 n = 60

Verbal feedback from center staff on work assigned at 

the center 

72% 74% 72%

(2.6%) (4.1%) (2.8%)

n = 279 n = 88 n = 191

Written feedback from staff on work assigned at the 

center 

27% 34% 26%

(2.5%) (4.5%) (2.7%)

n = 117 n = 44 n = 73

Verbal feedback from center staff on student behavior 

87% 87% 87%

(1.9%) (3.2%) (2.1%)

n = 336 n = 105 n = 231

Written feedback from center staff on student behavior 

47% 42% 48%

(2.9%) (4.7%) (3.1%)

n = 184 n = 54 n = 130

“Points” or rewards for program accomplishments 

50% 65%* 48%*

(2.9%) (4.6%) (3.1%)

n = 207 n = 80 n = 127

Certificates or awards for accomplishments 

63% 72% 62%

(2.8%) (4.4%) (3.0%)

n = 254 n = 89 n = 165

Source: Focused Survey, Item 18. 

*p<0.05 

Numbers in parentheses are standard errors. 

Exhibit reads: Of those centers serving elementary school students, 87 percent of centers reported providing verbal 

feedback from center staff on work assigned by the school. 
Appendix E: Supplementary Tables for the Surveys 140 

Exhibit E-7 

Centers That Reported Top-Priority Target Groups, by Target Groups and Center Type 

Top-Priority Groups 

Center Type 

Nonschoolbased Schoolbased 

We have open enrollment for all interested students 

60% 75%

(14.9%) (5.5%)

n = 27 n = 68

Students who scored “below proficient” on state assessments 

79%* 46%*

(4.4%) (15.3%)

n = 66 n = 19

Students identified by the school as needing special assistance in 

reading and/or math 

88%* 55%*

(3.6%) (5.5%)

n = 76 n = 20

Students who receive free or reduced-priced meals 

69% 58%

(5.2%) (15.0%)

n = 57 n = 24

Students who are recommended by teachers or counselors at the 

feeder schools 

59% 70%

(5.9%) (12.5%)

n = 52 n = 20

Other 

9% 12%

(3.1%) (9.8%)

n = 8 n = 6

Source: Focused Survey, Item 1. 

*p<0.05 

Numbers in parentheses are standard errors. 

Exhibit reads: Of nonschool-based centers, 60 percent reported having open enrollment for all interested students. 
Appendix E: Supplementary Tables for the Surveys 141 

Exhibit E-8 

Centers That Reported Attendance Requirements, by Grade Level and Attendance 

Requirement 

Attendance Requirements Elementary Middle High 

Attendance is not required; students can drop in and 

out * 

26% 29% 45%

(2.6%) (2.9%) (5.8%)

n = 111 n = 104 n = 52

We require students to attend at least 1 day per week * 

3% 5% 2%

(1.0%) (1.4%) (1.7%)

n = 12 n = 17 n = 2

We require students to attend 2–3 days per week * 

16% 20% 17%

(2.1%) (2.5%) (4.4%)

n = 63 n = 64 n = 19

We require students to attend every day we offer 

services * 

41% 31% 22%

(2.8%) (2.9%) (4.9%)

n = 147 n = 97 n = 25

Other * 

13% 14% 13%

(2.0%) (2.2%) (3.7%)

n = 52 n = 44 n = 14

Source: National Survey, Item 10. 

Numbers in parentheses are standard errors. 

* Significant association (p<0.05) between grade level of school and distribution of attendance items. 

Exhibit reads: Among centers serving elementary school students, 26 percent of centers reported that attendance is 

not required; students can drop in and out. 
Appendix E: Supplementary Tables for the Surveys 142 

Exhibit E-9 

Centers That Reported That They Required Participation of All Students in Specific 

Activities When Students Were Present, by Type of Activity 

Activities 

Required of 

All 

Students 

Homework help in a group setting 

73%

(4.6%)

n = 79

Tutoring (one-to-one or peer) 

14%

(3.8%)

n = 18

Academic enrichment activities in which reading instruction or practice takes place (not 

tutoring) 

60%

(5.5%)

n = 66

Academic enrichment activities in which mathematics instruction or practice takes 

place (not tutoring) 

58%

(5.5%)

n = 63

Source: Focused Survey, Item 9. 

Numbers in parentheses are standard errors. 

Exhibit reads: Seventy-three percent of centers reported requiring participation in homework help in a group setting 

when students were present. 

Exhibit E-10 

Centers That Reported the Number of Days of Attendance in 2005–06, 

by Grade Level and Days of Attendance 

Duration of Attendance 

Elementary

N = 75 

Middle 

N = 75 

High 

N = 75 

Fewer than 30 days 32% 36% 61%

(3.2%) (2.6%) (1.1%)

30–59 days 21% 28% 9%

(1.9%) (2.3%) (0.2%)

60–89 days 14% 14% 10%

(1.6%) (1.1%) (0.2%)

90–199 days 17% 11% 14%

(3.3%) (1.2%) (0.4%)

120 or more days 17% 11% 6%

(2.8%) (1.0%) (0.2%)

Source: Focused Survey, Item 8. 

Numbers in parentheses are standard errors. 

Exhibit reads: Thirty-two percent of elementary school students attended fewer than 30 days. 
Appendix E: Supplementary Tables for the Surveys 143 

Exhibit E-11 

Centers That Reported Consequences for Student Absences, by Consequence Type 

Consequences All Centers 

A staff member notifies the student’s parents or caregivers 

58%

(2.4%)

n = 292

A staff member speaks to the student about an absence after a certain number of 

absences 

38%

(2.4%)

n = 192

A staff member speaks to the student about an absence after each absence 

29%

(2.3%)

n = 150

Not applicable; we do not require attendance 

26%

(2.2%)

n = 145

A staff member notifies the student’s school 

13%

(1.7%)

n = 67

The student is restricted from participating in other activities at the center for a period 

of time 

10%

(1.5%)

n = 56

The student receives “points” or some other kind of penalty that restricts privileges at 

the center 

6%

(1.1%)

n = 32

Source: National Survey, Item 11. 

n = 503 

Numbers in parentheses are standard errors. 

Exhibit reads: Fifty-eight percent of center directors report notifying parents when a student is absent from the 

program. 
Appendix E: Supplementary Tables for the Surveys 144 

Exhibit E-12 

Centers That Paid Hourly Rates to Staff Who Led Instructional Activities, 

by Type of Staff and Rates

Pay Rate 

Certified 

Teachers Other Staff 

Less than $10 per hour 

1% 29%

(0.4%) (2.3%)

n = 12 n = 162

$10–$15 per hour 

11% 52%

(1.5%) (2.5%)

n = 63 n = 256

$15.01–$20 per hour 

27% 10%

(2.3%) (1.5%)

n = 129 n = 44

$20.01–$25 per hour 

27% 3%

(2.2%) (0.8%)

n = 128 n = 12

$25.01–$30 per hour 

20% 1%

(2.0%) (0.5%)

n = 89 n = 4

More than $30 per hour 

10% 0%

(1.5%) (0.3%)

n = 41 n = 2

Don’t know 

6% 5%

(1.1%) (1.2%)

n = 33 n = 21

Source: National Survey, Item 27. 

Numbers in parentheses are standard errors. 

Exhibit reads: Among all centers, 1 percent reported that certified teachers leading instructional activities were paid 

less than $10 an hour. 
Appendix E: Supplementary Tables for the Surveys 145 

Exhibit E-13 

Staff Who Led Instructional Activities at Centers, by Hours Worked per Week 

Hours Worked All Centers 

Full-time (35 hours or more per week) 

10%

(2.4%)

n = 50

Half-time (20–34 hours per week) 

14%

(2.8%)

n = 51

Part-time (10–19 hours per week) 

44%

(4.3%)

n = 140

Occasionally (less than 10 hours per week) 

32%

(4.4%)

n = 110

Source: Focused Telephone Survey, Items 26, 32 and 38. 

Numbers in parentheses are standard errors. 

Exhibit reads: Ten percent of centers reported staff leading instructional activities worked full-time (35 hours or 

more per week). 

Exhibit E-14 

Centers That Reported Offering Benefits to Full- and Part-Time Staff Who Led 

Instructional Activities, by Benefit Offered and Staff Status 

Benefits 

Full-Time 

Staff 

Part-Time 

Staff 

Paid time off for vacation and sick leave 

40% 11%

(2.4%) (1.5%)

n = 244 n = 66

Health insurance 

43% 8%

(2.4%) (1.3%)

n = 248 n = 38

Retirement savings plan 

41% 14%

(2.4%) (1.7%)

n = 234 n = 69

Tuition reimbursement 

12% 4%

(1.6%) (0.9%)

n = 65 n = 21

Source: National Survey, Item 28. 

Numbers in parentheses are standard errors. 

Exhibit reads: Forty percent of centers reported that full-time staff leading instructional activities were offered paid 

time off for vacation and sick leave. 

 
Appendix E: Supplementary Tables for the Surveys 146 

Exhibit E-15 

Centers That Reported Reasons for Staff Departures, 

by Reason for Departure and Center Type 

Reasons for Leaving 

Center Type 

Nonschoolbased Schoolbased 

Low pay 

18%* 8%*

(3.4%) (1.5%)

n = 27 n = 32

Lack of benefits 

44%* 25%*

(4.3%) (2.3%)

n = 66 n = 92

Inability to offer a full-time position 

24% 22%

(3.5%) (2.2%)

n = 40 n = 80

Completion/graduation from a program (e.g., AmeriCorps) or from school 

42% 49%

(4.3%) (2.7%)

n = 64 n = 179

Don’t know 

9% 13%

(2.9%) (1.8%)

n = 10 n = 48

We have not had staff leave our program 

28%* 12%*

(3.9%) (1.7%)

n = 43 n = 46

Source: National Survey, Item 30b. 

*p<0.05 

Numbers in parentheses are standard errors. 

Exhibit reads: Among school-based centers, 8 percent reported low pay as the primary reason that staff leading 

instructional activities left. 
Appendix E: Supplementary Tables for the Surveys 147 

Exhibit E16 

Centers That Reported That They Offered Paid Professional Development Opportunities 

for Full- and Part-Time Staff Who Led Instructional Activities, 

by Staff Status and Type of Development 

Professional Development 

Full-Time 

Staff 

Part-Time 

Staff 

Paid attendance at conferences 

53% 39%

(2.5%) (2.4%)

n = 304 n = 223

Paid training or professional development 

62% 55%

(2.4%) (2.5%)

n = 338 n = 307

Source: National Survey Item 28. 

Numbers in parentheses are standard errors. 

Exhibit reads: Fifty-three percent of centers reported that paid attendance at conferences was a professional 

development opportunity for full-time staff leading instructional activities. 

Exhibit E-17 

Centers That Reported That State Assessments Data Was Available for Use in Tracking 

Student Outcomes, by Subject Tested and Center Type 

State Assessment Results for School 

Center Type 

Nonschoolbased Schoolbased 

Reading 

53%* 73%*

(4.3%) (2.4%)

n = 79 n = 268

Mathematics 

52%* 71%*

(4.3%) (2.4%)

n = 77 n = 261

Source: National Survey, Item 13. 

*p<0.05 

Numbers in parentheses are standard errors. 

Exhibit reads: Among school-based centers, 73 percent reported that centers could track student outcomes in reading 

using state assessment results. 
Appendix E: Supplementary Tables for the Surveys 148 

Exhibit E-18 

Centers That Reported Student Assessment Sources, 

by Type and Frequency of Assessment 

Benefits Not used 

1–2 

times 

per year 

1–2 

times 

per 

month Weekly 

Almost 

daily 

Informal staff observations 

26% 12% 14% 20% 28%

(2.2%) (1.7%) (1.7%) (2.0%) (2.3%)

n = 124 n = 54 n = 76 n = 92 n = 149

Written reports from teachers 

36% 34% 19% 9% 3%

(2.5%) (2.4%) (2.0%) (1.4%) (0.8%)

n = 161 n = 165 n = 97 n = 39 n = 12

Verbal reports from teachers 

26% 11% 23% 26% 13%

(2.2%) (1.6%) (2.1%) (2.2%) (1.7%)

n = 133 n = 61 n = 118 n = 123 n = 60

Results from tests administered by 

center staff 

58% 24% 11% 5% 1%

(2.5%) (2.2%) (1.6%) (1.1%) (0.5%)

n = 260 n = 123 n = 62 n = 24 n = 11

Results from tests administered at 

students’ schools 

31% 47% 13% 7% 1%

(2.3%) (2.5%) (1.7%) (1.3%) (0.6%)

n = 155 n = 242 n = 62 n = 30 n = 7

Student self-report or self-assessment 

50% 22% 10% 9% 8%

(2.6%) (2.1%) (1.6%) (1.5%) (1.4%)

n = 235 n = 104 n = 51 n = 47 n = 39

Informal discussions with family 

members 

32% 22% 22% 16% 7%

(2.4%) (2.1%) (2.1%) (1.8%) (1.3%)

n = 151 n = 106 n = 104 n = 88 n = 38

Other 

87% 7% 4% 0% 2%

(2.8%) (2.2%) (1.7%) (0.2%) (0.9%)

n = 136 n = 14 n = 7 n = 2 n = 4

Source: National Survey, Item 15. 

Numbers in parentheses are standard errors. 

Exhibit reads: Twenty-six percent of centers reported informal staff observations are not used to assess students’ 

academic growth. 

 
Appendix E: Supplementary Tables for the Surveys 149 

Exhibit E-19 

Centers That Reported Uses of Student Assessment Data, by Type of Use 

Indicators All Centers

We do not use these data in our program 

1%

(0.5%)

n = 7

To identify needs for new program offerings 

59%

(2.4%)

n = 310

To make improvements to practices within existing program offerings 

69%

(2.3%)

n = 360

To assign students to particular activities 

48%

(2.5%)

n = 258

To assign students to particular staff who are qualified to help students in identified 

academic areas of need 

54%

(2.5%)

n = 294

To assign students to particular groups or to older youth in the program 

27%

(2.1%)

n = 155

To evaluate the success of the program 

66%

(2.4%)

n = 345

Source: National Survey, Item 16. 

Numbers in parentheses are standard errors. 

Exhibit reads: One percent of centers reported not using assessment data in their program. 
Appendix E: Supplementary Tables for the Surveys 150 

Exhibit E-20 

Centers That Reported Challenges in Implementing High-Quality Programming, by Type of 

Challenge and Center Type

Challenges 

Center Type 

Nonschoolbased Schoolbased 

Lack of information about the school-day curriculum 

40% 31%

(4.3%) (2.5%)

n = 57 n = 113

School staff not responsive to requests from the center 

39%* 29%*

(4.3%) (2.4%)

n = 55 n = 108

Lack of information about students' academic needs 

36%* 20%*

(4.2%) (2.1%)

n = 54 n = 77

Source: National Survey, Item 31. 

*p<0.05 

Numbers in parentheses are standard errors. 

Exhibit reads: Among nonschool-based centers, 40 percent reported lack of information about the school-day 

curriculum as a challenge in implementing high-quality programming. 
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 151

Appendix F 

21st CCLC Quality Study—Quality Observation Instrument

 
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 152
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 153
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 154
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 155
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 156
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 157
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 158
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 159
Appendix F: 21st CCLC Quality Study - Quality Observation Instrument 160