Supplemental Educational Services and

Student Achievement in Five Waiver Districts

Submitted to

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

Prepared by

Andrea Berger

Juliette-Marie deSousa

Gur Hoshen

Stephanie Lampron

Kerstin Carlson Le Floch

Megan Petroccia

Jamie Shkolnik

American Institutes for Research

1000 Thomas Jefferson Street, N.W.

Washington, DC 20007

and

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709-2194

2011
This report was prepared for the U.S. Department of Education under Contract Number 

ED-04-CO-0036/002/02 with RTI International. Erica Lee served as the contracting officer’s 

representative. The views expressed herein do not necessarily represent the positions or policies of the 

Department of Education. No official endorsement by the U.S. Department of Education is intended or 

should be inferred.

U.S. Department of Education

Arne Duncan

Secretary

Office of Planning, Evaluation and Policy Development

Carmel Martin

Assistant Secretary

Policy and Program Studies Service

Stuart Kerachsky

Director

March 2011

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. Although permission to 

reprint this publication is not necessary, the citation should be U.S. Department of Education, Office of Planning, 

Evaluation and Policy Development, Policy and Program Studies Service, Supplemental Educational Services and Student 

Achievement in Five Waiver Districts, Washington, D.C. 2011. 

This report is also available on the Department’s Web site at 

http://www.ed.gov/about/offices/list/opepd/ppss/index.html

On request, this publication is available in alternative formats, such as Braille, large print, or computer diskette. For more 

information, please contact the Department’s Alternate Format Center at (202) 260-0852 or (202) 260-0818.
Contents iii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Contents

Exhibits......................................................................................................................................v

Acknowledgments ...................................................................................................................xv

Executive Summary...............................................................................................................xvii

Key Findings..................................................................................................................................................xviii

I. Introduction.........................................................................................................................1

Overview of the SES Provisions of the ESEA.............................................................................................1

Overview of Recent Literature.........................................................................................................................2

Evaluation Questions........................................................................................................................................4

Data Sources.......................................................................................................................................................4

Analyses of Achievement Gains......................................................................................................................9

II. Eligibility for and Participation in Supplemental Educational Services ..........................13

Eligibility for Supplemental Educational Services ......................................................................................14

Participation in Supplemental Educational Services...................................................................................16

Number and Size of Non-District Providers...............................................................................................18

SES Participation With District and Non-District Providers ...................................................................19

Summary............................................................................................................................................................26

III.Student Achievement......................................................................................................... 27

SES Participation and Student Achievement Gains...................................................................................27

SES Providers and Student Achievement Gains.........................................................................................29

Summary............................................................................................................................................................36

IV. District Communication With Parents.............................................................................. 37

General Readability..........................................................................................................................................38

Balanced Information......................................................................................................................................39

Recruitment Activities.....................................................................................................................................39

Enrollment Opportunities..............................................................................................................................40

Changes in District Communication Over Time ........................................................................................41

Summary............................................................................................................................................................41

V. Summary ............................................................................................................................ 43

Bibliography ............................................................................................................................ 45
Contents iv Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix A: Additional Exhibits ..........................................................................................A–1

Appendix B: Supplemental Exhibits for Anchorage.............................................................B–1

Appendix C: Supplemental Exhibits for Boston...................................................................C–1

Appendix D: Supplemental Exhibits for Charlotte-Mecklenburg....................................... D–1

Appendix E: Supplemental Exhibits for Chicago................................................................ E–1

Appendix F: Supplemental Exhibits for Hillsborough.........................................................F–1

Appendix G: Comparison of Full and Partial Samples on SES Provider Type 

Achievement Gains ......................................................................................................... G–1

Appendix H: Average Achievement Gains Associated With SES Participation Across 

Five Districts...................................................................................................................H–1
Exhibits v Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibits

Executive Summary

Exhibit S.1 Number and Percentage of Eligible Students Participating in SES, 

by School District, Immediately Before and After the Waiver.....................................xix

Exhibit S.2 Number and Percentage of SES Participants Who Were Served 

by Non-District Providers, by School District, Immediately Before 

and After the Waiver...........................................................................................................xxi

Exhibit S.3 Mathematics and Reading Achievement Gains Associated With SES 

Participation, by School District and Subject, Overall and by Provider Type .........xxiv

Exhibit S.4 Differences in Achievement Gains Associated With SES Participation,

Relative to Nonparticipation, for Students Served by District and Non-District 

Providers, by Subject..........................................................................................................xxv

I. Introduction

Exhibit 1 Data Analyzed From Anchorage, Boston, Charlotte-Mecklenburg, Chicago 

and Hillsborough ....................................................................................................................7

II. Eligibility for and Participation in Supplemental Educational Services

Exhibit 2 Number and Percentage of Students Eligible for SES, Participating in SES 

and Served by Non-District Providers, by School District 

and by Academic Year .........................................................................................................15

Exhibit 3 Number and Percentage of Eligible Students Participating in SES, 

by School District, Immediately Before and After the Waiver......................................17

Exhibit 4 Number of Non-District SES Providers, by Year and by School District ..................19

Exhibit 5 Number and Percentage of SES Participants Who Were Served

by Non-District Providers, by School District, Immediately Before 

and After the Waiver............................................................................................................20

Exhibit 6 Average Number of Hours of SES Received by Participants, by School District,

by Provider Type and by Year............................................................................................22

Exhibit 7 Demographic Characteristics of Students Participating in SES in 2008–09, 

by School District and by Provider Type..........................................................................24

Exhibit 8 Average Prior (2007–08) Achievement Scores of Students Participating in SES 

in 2008–09 With District and Non-District Providers, and the Difference 

Between Provider Types, by School District and by Subject.........................................26

III.Student Achievement

Exhibit 9 Overall Mathematics and Reading Achievement Gains Associated With SES 

Participation, Relative to Nonparticipation ......................................................................29

Exhibit 10 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Relative to Nonparticipation, by Provider Type .....................................32
Exhibits vi Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 11 Differences in Achievement Gains Associated With SES Participation,

Relative to Nonparticipation, for Students Served by District and Non-District 

Providers, by Subject............................................................................................................34

Appendix A: Additional Exhibits

Exhibit A.1 Number of Non-District SES Providers, by School District and by Number 

of Students Served, Post-Waiver Year and Most Recent Year...................................A–1

Exhibit A.2 Distribution of Students Participating in SES, by School District 

and by Grade Level, 2008–09 ..........................................................................................A–2

Exhibit A.3 Percentage of Students Participating in SES, by School District, 

by Provider Type and by Grade Level, 2008–09 ..........................................................A–3

Exhibit A.4 Average Prior (2007–08) Achievement Scores, by School District, 

by 2008–09 Eligibility and by Subject.............................................................................A–4

Exhibit A.5 Average Prior (2007–08) Achievement Scores, by School District,

by 2008–09 Participation Status and by Subject ...........................................................A–4

Exhibit A.6 Mathematics and Reading Achievement Gains Associated With Single-Year 

or Multiple-Year SES Participation.................................................................................A–5

Appendix B: Supplemental Exhibits for Anchorage

Exhibit B.1 Percentage and Number of Students Eligible for SES, by Grade, 

in Anchorage, 2004–05 Through 2008–09 ....................................................................B–1

Exhibit B.2 Percentage and Number of Eligible Students Participating in SES, by Grade, 

in Anchorage, 2004–05 Through 2008–09 ....................................................................B–2

Exhibit B.3 Percentage and Number of Students in Grades K–12 Participating in SES, 

by Provider, in Anchorage, 2004–05 Through 2008–09..............................................B–3

Exhibit B.4 Number and Percentage of Students Participating in SES, by Provider Type 

and by Grade, in Anchorage, 2006–07 Through 2008–09 ..........................................B–4

Exhibit B.5 Percentage Distribution of SES Eligible Students in Grades K–12, 

by Demographic Characteristics, by Participation Status 

and by Provider Type, in Anchorage, 2006–07 Through 2008–09............................B–5

Exhibit B.6 Number of Eligible and Participating Students and SES Participation Rates 

in Grades K–12, by Race/Ethnicity, in Anchorage, 

2006–07 Through 2008–09 ..............................................................................................B–6

Exhibit B.7 Percentage and Number of SES Participants in Grades K–12, by Subject 

and by Provider Type, in Anchorage, 2006–07 Through 2008–09............................B–7

Exhibit B.8 Average Number of Hours of SES Received by Participants in Grades K–12, 

by Provider Type, in Anchorage, 2004–05 Through 2008–09 ...................................B–7

Exhibit B.9 Percentage Distribution of SES Participants in Grades K–12, 

by Number of Hours of Services Received and by Provider Type, 

in Anchorage, 2004–05 Through 2008–09 ....................................................................B–8
Exhibits vii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.10 Average Prior (2007–08) Achievement Scores for Students in Grades 3–10, 

by 2008–09 Eligibility and Participation Status and by Subject, in Anchorage ........B–9

Exhibit B.11 Average Prior (2007–08) Achievement Scores for Participating Students 

in Grades 3–10, by 2008–09 Provider Type and by Subject, in Anchorage .............B–9

Exhibit B.12 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, Overall and for a Single Year 

and Multiple Years, in Anchorage, 2004–05 Through 2008–09...............................B–10

Exhibit B.13 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, by Provider Type, in Anchorage, 

2004–05 Through 2008–09 ............................................................................................B–10

Exhibit B.14 Difference in Achievement Gains Associated With SES Participation 

for Students Served by District and Non-District Providers, in Grades 3–10, 

by Subject, in Anchorage, 2004–05 Through 2008–09..............................................B–11

Exhibit B.15 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Adjusted for Hours of SES, for Students in Grades 3–10, 

by Provider Type, in Anchorage, 2004–05 Through 2008–09 .................................B–12

Exhibit B.16 Difference in Achievement Gains Associated With SES Participation 

for Students in Grades 3–10 Served by District and Non-District Providers, 

Adjusted for Hours of SES, by Subject, in Anchorage, 

2004–05 Through 2008–09 ............................................................................................B–13

Exhibit B.17 Mathematics Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–10, 

by Individual Provider, in Anchorage, 2004–05 Through 2008–09.........................B–14

Exhibit B.18 Reading Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–10, 

by Individual Provider, in Anchorage, 2004–05 Through 2008–09.........................B–15

Exhibit B.19 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, by Academic Year, in Anchorage, 

2005–06 Through 2008–09 ............................................................................................B–16

Appendix C: Supplemental Exhibits for Boston

Exhibit C.1 Percentage and Number of Students Eligible for SES, by Grade, in Boston, 

2004–05 Through 2008–09 ..............................................................................................C–1

Exhibit C.2 Percentage and Number of Eligible Students Participating in SES, by Grade, 

in Boston, 2004–05 Through 2008–09...........................................................................C–2

Exhibit C.3 Percentage and Number of Students in Grades K–12 Participating in SES, 

by Provider, in Boston, 2004–05 Through 2008–09....................................................C–3

Exhibit C.4 Number and Percentage of Students Participating in SES, by Provider Type 

and by Grade, in Boston, 2004–05 Through 2008–09.................................................C–4
Exhibits viii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.5 Percentage Distribution of SES Eligible Students in Grades K–12, 

by Demographic Characteristics, by Participation Status 

and by Provider Type, in Boston, 2004–05 Through 2008–09 ..................................C–5

Exhibit C.6 Number of Eligible and Participating Students and SES Participation Rates 

in Grades K–12, by Race/Ethnicity, in Boston, 2004–05 Through 2008–09..........C–6

Exhibit C.7 Percentage and Number of SES Participants in Grades K–12, by Subject 

and by Provider Type, in Boston, 2004–05 Through 2008–09 ..................................C–7

Exhibit C.8 Average Number of Hours of SES Received by Participants in Grades K–12, 

by Provider Type, in Boston, 2004–05 Through 2008–09..........................................C–8

Exhibit C.9 Percentage Distribution of SES Participants in Grades K–12, 

by Number of Hours of Services Received and by Provider Type, in Boston, 

2004–05 Through 2008–09 ..............................................................................................C–9

Exhibit C.10 Average Prior (2007–08) Achievement Scores for Students in Grades 3–8, 

by 2008–09 Eligibility and Participation Status and by Subject, in Boston.............C–10

Exhibit C.11 Average Prior (2007–08) Achievement Score for Participating Students 

in Grades 3–8, by 2008–09 Provider Type and by Subject, in Boston....................C–10

Exhibit C.12 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, Overall and for a Single Year 

and Multiple Years, in Boston, 2002–03 Through 2008–09 .....................................C–11

Exhibit C.13 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Provider Type, in Boston, 

2002–03 Through 2008–09 ............................................................................................C–11

Exhibit C.14 Difference in Achievement Gains Associated With SES Participation 

for Students Served by District and Non-District Providers, in Grades 3–8, 

by Subject, in Boston, 2002–03 Through 2008–09 ....................................................C–12

Exhibit C.15 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Adjusted for Hours of SES, for Students in Grades 3–8, 

by Provider Type, in Boston, 2002–03 Through 2008–09........................................C–13

Exhibit C.16 Difference in Achievement Gains Associated With SES Participation 

for Students in Grades 3–8 Served by District and Non-District Providers, 

Adjusted for Hours of SES, by Subject, in Boston, 

2002–03 Through 2008–09 ............................................................................................C–14

Exhibit C.17 Mathematics Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–8, 

by Individual Provider, in Boston, 2002–03 Through 2008–09...............................C–15

Exhibit C.18 Reading Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation for Students in Grades 3–8, 

by Individual Provider, in Boston, 2002–03 Through 2008–09...............................C–16

Exhibit C.19 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Academic Year, in Boston, 

2005–06 Through 2008–09 ............................................................................................C–17
Exhibits ix Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix D: Supplemental Exhibits for Charlotte-Mecklenburg

Exhibit D.1 Percentage and Number of Students Eligible for SES, by Grade, 

in Charlotte-Mecklenburg, 2005–06 Through 2008–09..............................................D–1

Exhibit D.2 Percentage and Number of Eligible Students Participating in SES, by Grade, 

in Charlotte-Mecklenburg, 2005–06 Through 2008–09..............................................D–2

Exhibit D.3 Percentage and Number of Students in Grades K–8 Participating in SES, 

by Provider, in Charlotte-Mecklenburg, 2005–06 Through 2008–09.......................D–3

Exhibit D.4 Number and Percentage of Students Participating in SES, by Provider Type 

and by Grade, in Charlotte-Mecklenburg, 2008–09 ....................................................D–4

Exhibit D.5 Percentage Distribution of SES Eligible Students in Grades K–8, 

by Demographic Characteristics, by Participation Status 

and by Provider Type, in Charlotte-Mecklenburg, 2008–09......................................D–5

Exhibit D.6 Number of Eligible and Participating Students and SES Participation Rates 

in Grades K–8, by Race/Ethnicity, in Charlotte-Mecklenburg, 2008–09................D–6

Exhibit D.7 Percentage and Number of SES Participants in Grades K–8, by Subject 

and by Provider Type, in Charlotte-Mecklenburg, 2008–09......................................D–6

Exhibit D.8 Average Number of Hours of SES Received by Participants in Grades K–8, 

by Provider Type, in Charlotte-Mecklenburg, 2005–06 Through 2008–09.............D–7

Exhibit D.9 Percentage Distribution of SES Participants in Grades K–8, 

by Number of Hours of Services Received and by Provider Type, 

in Charlotte-Mecklenburg, 2005–06 Through 2008–09..............................................D–8

Exhibit D.10 Average Prior (2007–08) Achievement Scores for Students in Grades 3–8, 

by 2008–09 Eligibility and Participation Status and by Subject, 

in Charlotte-Mecklenburg................................................................................................D–9

Exhibit D.11 Average Prior (2007–08) Achievement Scores for Participating Students 

in Grades 3–8, by 2008–09 Provider Type and by Subject, 

in Charlotte-Mecklenburg................................................................................................D–9

Exhibit D.12 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, Overall and for a Single Year 

and Multiple Years, in Charlotte-Mecklenburg, 2005–06 Through 2008–09 ........D–10

Exhibit D.13 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Provider Type, 

in Charlotte-Mecklenburg, 2005–06 Through 2008–09............................................D–11

Exhibit D.14 Difference in Achievement Gains Associated With SES Participation 

for Students Served by District and Non-District Providers, in Grades 3–8, 

by Subject, in Charlotte-Mecklenburg, 2005–06 Through 2008–09 .......................D–12

Exhibit D.15 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Adjusted for Hours of SES, for Students in Grades 3–8, 

by Provider Type, in Charlotte-Mecklenburg, 2005–06 Through 2008–09...........D–13
Exhibits x Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.16 Difference in Achievement Gains Associated With SES Participation 

for Students in Grades 3–8 Served by District and Non-District Providers, 

Adjusted for Hours of SES, by Subject, in Charlotte-Mecklenburg, 

2005–06 Through 2008–09 ...........................................................................................D–14

Exhibit D.17 Mathematics Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–8, 

by Individual Provider, in Charlotte-Mecklenburg, 

2005–06 Through 2008–09 ...........................................................................................D–15

Exhibit D.18 Reading Student Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–8, 

by Individual Provider, in Charlotte-Mecklenburg, 

2005–06 Through 2008–09 ...........................................................................................D–16

Exhibit D.19 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Academic Year, 

in Charlotte-Mecklenburg, 2005–06 Through 2008–09............................................D–17

Appendix E: Supplemental Exhibits for Chicago

Exhibit E.1 Percentage and Number of Students Eligible for SES, by Grade, in Chicago, 

2004–05 Through 2008–09 ..............................................................................................E–1

Exhibit E.2 Percentage and Number of Eligible Students Participating in SES, 

by Grade, in Chicago, 2004–05 Through 2008–09.......................................................E–1

Exhibit E.3 Percentage and Number of Students in Grades 3–8 Participating in SES, 

by Provider, in Chicago, 2004–05 Through 2008–09 ..................................................E–2

Exhibit E.4 Number and Percentage of Students Participating in SES, by Provider Type 

and by Grade, in Chicago, 2004–05 Through 2008–09...............................................E–4

Exhibit E.5 Percentage Distribution of SES Eligible Students in Grades 3–8, 

by Demographic Characteristics, by Participation Status 

and by Provider Type, in Chicago, 2004–05 Through 2008–09.................................E–5

Exhibit E.6 Number of Eligible and Participating Students and SES Participation Rates 

in Grades 3–8, by Race/Ethnicity, in Chicago, 2004–05 Through 2008–09............E–6

Exhibit E.7 Percentage and Number of SES Participants in Grades 3–8, by Subject 

and by Provider Type, in Chicago, 2004–05 Through 2008–09.................................E–7

Exhibit E.8 Average Number of Hours of SES Received by Participants in Grades 3–8, 

by Provider Type, in Chicago, 2004–05 Through 2008–09 ........................................E–8

Exhibit E.9 Percentage Distribution of SES Participants in Grades 3–8, 

by Number of Hours of Services Received and by Provider Type, 

in Chicago, 2004–05 Through 2008–09 .........................................................................E–9

Exhibit E.10 Average Prior (2007–08) Achievement Scores for Students in Grades 3–8, 

by 2008–09 Eligibility and Participation Status and by Subject, in Chicago...........E–10

Exhibit E.11 Average Prior (2007–08) Achievement Scores for Participating Students 

in Grades 3–8, by 2008–09 Provider Type and by Subject, in Chicago ..................E–10
Exhibits xi Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.12 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, Overall and for a Single Year 

and Multiple Years, in Chicago, 2002–03 Through 2008–09....................................E–11

Exhibit E.13 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Provider Type, in Chicago, 

2002–03 Through 2008–09 ............................................................................................E–11

Exhibit E.14 Difference in Achievement Gains Associated With SES Participation 

for Students Served by District and Non-District Providers, in Grades 3–8, 

by Subject, in Chicago, 2002–03 Through 2008–09...................................................E–12

Exhibit E.15 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Adjusted for Hours of SES, for Students in Grades 3–8, 

by Provider Type, in Chicago, 2002–03 Through 2008–09 ......................................E–13

Exhibit E.16 Difference in Achievement Gains Associated With SES Participation 

for Students in Grades 3–8 Served by District and Non-District Providers, 

Adjusted for Hours of SES, by Subject, in Chicago, 

2002–03 Through 2008–09 ............................................................................................E–14

Exhibit E.17 Mathematics Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–8, 

by Individual Provider, in Chicago, 2002–03 Through 2008–09 .............................E–15

Exhibit E.18 Reading Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–8, 

by Individual Provider, in Chicago, 2002–03 Through 2008–09 .............................E–17

Exhibit E.19 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–8, by Academic Year, in Chicago, 

2005–06 Through 2008–09 ............................................................................................E–19

Appendix F: Supplemental Exhibits for Hillsborough

Exhibit F.1 Percentage and Number of Students Eligible for SES, by Grade, 

in Hillsborough, 2004–05 Through 2008–09 ................................................................F–1

Exhibit F.2 Percentage and Number of Eligible Students Participating in SES, by Grade, 

in Hillsborough, 2004–05 Through 2008–09 ................................................................F–2

Exhibit F.3 Percentage and Number of Students in Grades K–12 Participating in SES, 

by Provider, in Hillsborough, 2004–05 Through 2008–09 .........................................F–3

Exhibit F.4 Number and Percentage of Students Participating in SES, by Provider Type 

and by Grade, in Hillsborough, 2006–07 Through 2008–09......................................F–5

Exhibit F.5 Percentage Distribution of SES Eligible Students in Grades K–12, 

by Demographic Characteristics, by Participation Status and by Provider Type, 

in Hillsborough, 2006–07 Through 2008–09 ................................................................F–6

Exhibit F.6 Number of Eligible and Participating Students and SES Participation Rates 

in Grades K–12, by Race/Ethnicity, in Hillsborough, 

2006–07 Through 2008–09 ..............................................................................................F–7
Exhibits xii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.7 Percentage and Number of SES Participants in Grades K–12, by Subject 

and by Provider Type, in Hillsborough, 2006–07 Through 2008–09........................F–8

Exhibit F.8 Average Number of Hours of SES Received by Participants in Grades K–12, 

by Provider Type, in Hillsborough, 2004–05 Through 2008–09 ...............................F–9

Exhibit F.9 Percentage Distribution of SES Participants in Grades K–12, 

by Number of Hours of Services Received and by Provider Type, 

in Hillsborough, 2004–05 Through 2008–09 ..............................................................F–10

Exhibit F.10 Average Prior (2007–08) Achievement Scores for Students in Grades 3–10, 

by 2008–09 Eligibility and Participation Status and by Subject, 

in Hillsborough ................................................................................................................F–11

Exhibit F.11 Average Prior (2007–08) Achievement Scores for Participating Students 

in Grades 3–10, by 2008–09 Provider Type and by Subject, in Hillsborough .......F–11

Exhibit F.12 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, Overall and for a Single Year 

and Multiple Years, in Hillsborough, 2003–04 Through 2008–09...........................F–12

Exhibit F.13 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, by Provider Type, 

in Hillsborough, 2003–04 Through 2008–09 ..............................................................F–12

Exhibit F.14 Difference in Achievement Gains Associated With SES Participation 

for Students Served by District and Non-District Providers, in Grades 3–10, 

by Subject, in Hillsborough, 2003–04 Through 2008–09..........................................F–13

Exhibit F.15 Mathematics and Reading Achievement Gains Associated With SES 

Participation, Adjusted for Hours of SES, for Students in Grades 3–10, 

by Provider Type, in Hillsborough, 2003–04 Through 2008–09 .............................F–14

Exhibit F.16 Difference in Achievement Gains Associated With SES Participation 

for Students in Grades 3–10 Served by District and Non-District Providers, 

Adjusted for Hours of SES, by Subject, in Hillsborough, 

2003–04 Through 2008–09 ............................................................................................F–15

Exhibit F.17 Mathematics Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–10, 

by Individual Provider, in Hillsborough, 2003–04 Through 2008–09 ....................F–16

Exhibit F.18 Reading Achievement Gains Associated With SES Participation 

and Average Hours of SES Participation, for Students in Grades 3–10, 

by Individual Provider, in Hillsborough, 2003–04 Through 2008–09 ....................F–17

Exhibit F.19 Mathematics and Reading Achievement Gains Associated With SES 

Participation, for Students in Grades 3–10, by Academic Year, 

in Hillsborough, 2005–06 Through 2008–09 ..............................................................F–18

Appendix G: Comparison of Full and Partial Samples on SES Provider Type 

Achievement Gains

Exhibit G.1 Mathematics Achievement Gains Associated With SES Participation,

by School District, by Provider Type and by Sample Type .......................................G–2
Exhibits xiii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit G.2 Reading Achievement Gains Associated With SES Participation,

by School District, by Provider Type and by Sample Type .......................................G–2

Appendix H: Average Achievement Gains Associated With SES Participation Across 

Five Districts

Exhibit H.1 Mathematics and Reading Achievement Gains Associated With SES 

Participation Across Five Districts, Overall, by Provider Type and Differences 

for Students Served by District and Non-District Providers.....................................H–2

Acknowledgments xv Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Acknowledgments

We wish to thank several individuals who contributed to the completion of this study.

First, we are grateful for the guidance and support of the U.S. Department of Education. In particular, 

we thank Erica Lee, Margie Yeager, Stephanie Stullich and Collette Roney, of the Policy and Program 

Studies Service, who served as project officers on this study. We also thank Kelly Rhoads and Stacy 

Kreppel, of the Office of Innovation and Improvement. We are grateful to the staff of the American 

Institutes for Research for their assistance in producing this report. In particular, we thank Mike Garet 

and Mengli Song for their guidance on data and analysis issues and Alexis Carpenter, Marty Leff and 

Johnathan Spiegel for their production assistance. We also recognize the assistance of Ron Zimmer, of 

the RAND Corporation, and Brian Gill, of Mathematica Policy Research, for their advice on the analytic 

approach during year one. We are grateful for the efforts of Vernon Campbell and Zareena Clendaniel, 

of Anchorage Public Schools; Jeffrey Harris and Steve Desrosiers, of Boston Public Schools; 

Christopher Cobitz, of Charlotte-Mecklenburg Schools; Susan Ryan, Melva Ryan, Erick Dorris and Erica 

Harris, of Chicago Public Schools; and Jessica Swere, of Hillsborough County Public Schools, for their 

work to provide the data used for this study. 

While we appreciate the assistance and support of all the above individuals, any errors in judgment or

fact are of course the responsibility of the authors.

Executive Summary xvii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Executive Summary

Under the Elementary and Secondary Education Act of 1965 (ESEA), as amended, schools that are served 

under Title I, Part A of the ESEA and that do not make adequate yearly progress (AYP) for 

two consecutive years are identified for school improvement. If an identified Title I school does not 

make AYP while in that status, and thus enters the second year of school improvement status, its district 

must offer students from low-income families in the school the opportunity to receive free supplemental 

educational services (SES) such as tutoring, remediation or other academic instruction provided outside 

the regular school day. The district must offer SES to students from low-income families until the school 

makes AYP for two consecutive years and thereby exits improvement status. A state educational agency 

(SEA) may approve as an SES provider a school district that is not identified for improvement or 

corrective action. However, under federal regulations 34 C.F.R. § 200.47(b)(1)(iv)(B), an SEA may not 

approve as an SES provider a school district that is identified for improvement or corrective action. 

In 2005, the U.S. Department of Education began a pilot through which it granted waivers to 

five districts identified for improvement or corrective action, to allow those districts to serve as SES 

providers.1 Boston and Chicago received waivers starting in the 2005–06 school year, although both 

served as providers before the waivers. Anchorage and Hillsborough received waivers starting in the 

2006–07 school year, and Charlotte-Mecklenburg received a waiver starting in the 2008–09 school year. 

Anchorage, Charlotte-Mecklenburg and Hillsborough began serving as SES providers after receiving 

their respective waivers. 

 1. For the 2009–10 and 2010–11 school years, the pilot was replaced with a more expansive waiver opportunity. In 

those years, states were permitted to request waivers from the U.S. Department of Education to approve as SES 

providers those schools or districts in need of improvement, corrective action or restructuring. See U.S. Department of 

Education, Non-Regulatory Guidance on Title I, Part A Waivers (Washington, D.C.: U.S. Department of Education, July 

2009). Twenty-nine states were approved for waivers for the 2009–10 school year, and 22 states were approved for 

waivers for the 2010–11 school year.
Executive Summary xviii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

The central evaluation question for this report is whether SES participation for students served by the 

district providers was associated with achievement gains that were at least comparable with those of 

students served by non-district providers. This report also describes student participation in SES, the 

differences between students receiving SES from district and non-district providers, and the ways in 

which districts communicate with parents about SES. Each district provided student-level data on SES: 

whether the student was eligible and whether the student participated, with which providers, in what 

subjects, and for how many hours. Districts also supplied data on student achievement and student 

characteristics (e.g., race/ethnicity, disability status and grade level). All these data were provided for 

multiple academic years. Because districts received the waivers in different years and state assessment 

systems included different grades, this report covers different academic years and grade levels for 

each district. 

Key Findings

Participation in Supplemental Educational Services Provided by District 

and Non-District Providers

In the three districts that began to serve as SES providers after the waiver was 

granted, student participation in SES increased in the first year of the waiver. 

Districts receiving an SES waiver were expected to expand their outreach and communication to 

students and parents in an unbiased manner to encourage greater student participation in subsequent 

years. Although all five districts received waivers, Boston and Chicago served as district SES providers 

before the granting of the waiver. In Anchorage, Charlotte-Mecklenburg and Hillsborough, the waiver 

resulted in the introduction of the district into the pool of available providers, possibly increasing each 

district’s capacity to provide SES. Among the three districts that began serving as SES providers after the 

waiver was granted, SES participation grew during the first year of the waiver. For example, in CharlotteKey Findings

—In the three districts that did not serve as SES providers before the waiver (Anchorage, 

Charlotte-Mecklenburg and Hillsborough), SES participation rates increased in the first year 

of the waiver. (Boston and Chicago served as providers before receipt of the waiver.)

—There were few demographic or academic differences between students served by district 

providers and students served by non-district providers. 

—Students in three of the five districts demonstrated statistically significantly larger 

mathematics achievement gains during periods of SES participation than during periods of 

nonparticipation. In addition, in two districts, SES participation was associated with 

statistically significant reading gains. Averaged across the five districts, the overall association 

between SES participation and achievement gains was statistically significant in both 

mathematics and reading, relative to nonparticipation.

—Across the five districts, the achievement gains associated with SES participation relative to 

nonparticipation did not differ for district and non-district providers for either mathematics 

or reading. 

—All five districts reported using multiple communication strategies to reach eligible families, 

provided balanced information about SES providers, translated information into at least one 

language other than English, and provided extended enrollment periods. 
Executive Summary xix Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Mecklenburg, the total number of students who received SES increased from 2,175 in 2007–08 to 

3,972 in 2008–09 (see Exhibit S.1). Participation rates also rose by two percentage points in Anchorage 

and Charlotte-Mecklenburg and by four percentage points in Hillsborough. 

In Boston, participation rates in SES decreased from 18 percent in 2004–05 to 15 percent in 

2005–06 (see Exhibit S.1). However, since the number of eligible students markedly increased during 

this period, participation in SES more than doubled from the preceding year, increasing from 

1,521 students to more than 3,288 students. In Chicago, the number of eligible students decreased 

after the waiver was granted, from 93,890 in 2004–05 to 83,890 in 2005–06, along with the number of 

SES participants (from 31,950 students to 23,631 students) and their participation rate (from 34 percent 

to 28 percent). 

Exhibit S.1

Number and Percentage of Eligible Students Participating in SES, by School District,

Immediately Before and After the Waiver

Exhibit reads: In Anchorage, 5 percent of students eligible for SES participated in the school year before

the waiver, and 7 percent participated in the school year after the waiver. In 2005–06, 104 eligible students 

participated in SES.

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage and Hillsborough; grades K–8 in CharlotteMecklenburg; grades K–8, 2004–05, and K–12, 2005–06, in Boston; and grades 3–8 in Chicago. In Boston, for 

grades K–8 in 2005–06, the number of eligible students participating in SES was 3,102 (17 percent of the 

eligible students).

The Anchorage, Charlotte-Mecklenburg and Hillsborough school districts became providers after receiving a 

waiver. The Boston and Chicago school districts served as SES providers before receiving the waiver.

Source: Administrative data provided by the Anchorage and Hillsborough school districts, 2005–06 and 2006–07; 

the Charlotte Mecklenburg school district, 2007–08 and 2008–09; and the Boston and Chicago school districts, 

2004–05 and 2005–06.
Executive Summary xx Supplemental Educational Services and Student

Achievement in Five Waiver Districts

The introduction of the district provider did not appear to limit the number of 

non-district providers. 

One of the goals of providing waivers to districts was to expand SES to a broader eligible student 

population without detracting from the services of non-district providers. In the three districts that did 

not serve as providers before the waiver, their introduction as district providers did not appear to 

negatively influence the number of non-district providers in the long run. For example, in the first year 

after the waiver (2006–07) in Anchorage and Hillsborough, the number of non-district providers 

decreased by one and then rebounded the following year. Over time, as the number of participants 

increased in each of the districts, so did the number of non-district providers serving them. 

In the three districts that began to serve as SES providers after the waiver was 

granted, non-district providers served the majority of students participating in SES 

in the first year of the waiver. 

After receiving their respective waivers, the Anchorage, Charlotte-Mecklenburg and Hillsborough school 

districts began serving as SES providers. During the first post-waiver year, the non-district providers in 

those districts continued to serve the majority of SES participants, ranging from 55 percent to 84 percent 

(see Exhibit S.2). The Boston and Chicago school districts both served as SES providers before the 

waiver. In Boston, non-district providers served fewer than one-third of SES participants both before 

and after the waiver. Conversely, in Chicago, non-district providers served more than half of SES 

participants before the granting of the waiver (51 percent in 2004–05) and served nearly two-thirds of 

participants after the waiver (62 percent in 2005–06).
Executive Summary xxi Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit S.2

Number and Percentage of SES Participants Who Were Served by Non-District Providers, 

by School District, Immediately Before and After the Waiver

Exhibit reads: In Anchorage, where the district served as an SES provider only after the waiver, non-district 

providers served 100 percent of SES participants during the school year before the waiver and served 

55 percent of participants in the year after the waiver. In 2005–06, 104 SES participants were served by 

non-district providers in Anchorage.

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage and Hillsborough; grades K–8 in CharlotteMecklenburg; grades K–8, 2004–05, and K–12, 2005–06, in Boston; and grades 3–8 in Chicago. In Boston, for 

grades K–8 in 2005–06, non-district providers served 25 percent of SES participants.

The Anchorage, Charlotte-Mecklenburg and Hillsborough school districts became providers after receiving a 

waiver. The Boston and Chicago school districts served as SES providers before receiving the waiver.

Source: Administrative data provided by the Anchorage and Hillsborough school districts, 2005–06 and 2006–07; the 

Charlotte-Mecklenburg school district, 2007–08 and 2008–09; and the Boston and Chicago school districts, 

2004–05 and 2005–06.

In 2008–09, in Chicago and Hillsborough, participants served by district providers 

received a significantly greater number of hours of SES than participants served by 

non-district providers, while in the other three districts, participants served by the 

district providers received a similar number of hours of SES to those served by 

non-district providers. 

SES participants in Chicago and Hillsborough in 2008–09 received a statistically significantly greater 

number of hours of tutoring on average from district providers (55 and 29 hours, respectively) than from 
Executive Summary xxii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

non-district providers (37 and 21 hours, respectively). In Anchorage, Boston and Charlotte-Mecklenburg

in 2008–09, the difference in the average number of hours of tutoring received by students served by the 

district and the non-district providers was not statistically significant. 

There were few demographic or academic differences between students served by 

district providers and students served by non-district providers. 

In most districts, both district and non-district providers primarily served elementary students and 

served relatively few high school students.2 In all districts, most SES participants (87 percent or more 

in 2008–09) were of a minority race or ethnicity. In 2008–09, SES students with limited English 

proficiency were served at similar rates by both district and non-district providers in three of the 

districts, as were students with disabilities in four of the five districts. In Boston and CharlotteMecklenburg, the percentage of students with limited English proficiency participating with the district 

provider was statistically significantly larger than the percentage served by the non-district providers

(16 percent and 5 percent larger, respectively). In Anchorage, non-district providers (22 percent) served a 

statistically significantly higher percentage of students with disabilities than did the district provider 

(14 percent). 

In Boston and Chicago, students served by the district provider in 2008–09 had prior achievement 

levels (in 2007–08) that were statistically significantly lower than achievement levels of students 

served by non-district providers. In the remaining three districts, the average prior achievement 

levels did not differ statistically significantly between students served by the district provider and those 

served by non-district providers. 

SES Participation and Student Achievement for Students Served by District 

and Non-District Providers 

The following findings describe the association between SES participation and student achievement 

gains. The analyses of student achievement in this study followed the methods used in a previous 

analysis of SES and student achievement in nine large urban school districts that was conducted as part 

of the National Longitudinal Study of No Child Left Behind.3 Using a student fixed-effects approach, the 

current study examined students’ achievement gains associated with SES participation relative to 

nonparticipation. The analyses examined mathematics achievement gains for students who participated 

in mathematics SES and reading achievement gains for students who participated in reading SES.4 All 

models include an indicator for student eligibility for services and an indicator for participation, plus 

grade, year and grade-by-year terms. These models also included student fixed effects, which controlled 

for time-invariant student characteristics such as parent education and underlying student ability 

or motivation. 

The achievement gains associated with SES participation presented in the subsequent exhibits represent 

the differences in students’ annual achievement gains (where achievement is scaled in standardized 

z-scores) between periods of participation and periods of no participation. In other words, the results

indicate whether students experienced larger or smaller statistically significant achievement gains during 

periods of participation compared with periods of nonparticipation. Because statistical significance 

depends on both sample size and the size of the effect being estimated, the likelihood of finding 

 2. Charlotte-Mecklenburg and Chicago did not provide information on SES participants beyond grade 8. 

3. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental 

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007).

4. Students may have received SES in both mathematics and reading, as well as in other subjects. 
Executive Summary xxiii Supplemental Educational Services and Student

Achievement in Five Waiver Districts

statistically significant gains was lower for Anchorage, a small district, than for Chicago, a large district, 

all else being equal.

Students in three of the five districts demonstrated statistically significantly larger 

mathematics achievement gains during periods of SES participation than during 

periods of nonparticipation. In addition, in two districts, SES participation was 

associated with statistically significant reading gains. Across the five districts, SES 

participation was associated with statistically significant achievement gains in both 

mathematics and reading.

In Chicago, SES participation was associated with statistically significant overall achievement gains of

0.05 standard deviations in mathematics and 0.07 standard deviations in reading, relative to 

nonparticipation (see Exhibit S.3). In Hillsborough, SES participation was associated with statistically 

significant increases in overall achievement gains of 0.12 standard deviations for mathematics and 

0.06 standard deviations for reading. In Charlotte-Mecklenburg, SES participation was associated with 

a statistically significant increase in mathematics achievement gains (0.11 standard deviations) but not in 

reading gains (0.03 standard deviations). In Anchorage and Boston, SES participation was not associated 

with statistically significant gains in mathematics or reading achievement.5

Across the five districts, the average overall association between SES participation and achievement 

gains was statistically significant for mathematics (0.08 standard deviations) and reading (0.04 standard 

deviations), relative to nonparticipation (see Exhibit S.3). 

Across the five districts, for students served by district and non-district providers, 

SES participation was associated with statistically significant mathematics 

achievement gains, relative to nonparticipation. For students served by district 

providers, but not by non-district providers, SES participation was associated with 

statistically significant reading achievement gains.

Across the five districts, for district providers, SES participation was associated with statistically 

significant gains in mathematics (0.10 standard deviations) and reading (0.06 standard deviations), relative 

to nonparticipation (see Exhibit S.3). For non-district providers, SES participation was associated with a 

statistically significant gain in mathematics achievement (0.09 standard deviations) but not in reading 

achievement (0.03 standard deviations).

 5. Boston achievement analyses should be interpreted with caution. The findings were sensitive to sample changes. 

See Appendix G for details.
Executive Summary xxiv Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit S.3

Mathematics and Reading Achievement Gains Associated With SES Participation,

by School District and Subject, Overall and by Provider Type

Mathematics Achievement Gains Reading Achievement Gains

Overall District

Provider

Non-District

Providers Overall DistrictProvider

Non-District

Providers

Anchorage 0.08 0.05 0.10 0.05 0.10 0.00

Boston 0.03 0.04 0.05 –0.01 0.03 0.00

Charlotte-Mecklenburg 0.11* 0.23* 0.10 0.03 0.04 0.02

Chicago 0.05* 0.06* 0.04* 0.07* 0.03* 0.09*

Hillsborough 0.12* 0.11* 0.13* 0.06* 0.10 0.06*

Across district average 0.08* 0.10* 0.09* 0.04* 0.06* 0.03

Exhibit reads: In Anchorage, SES participation, relative to nonparticipation, was associated with a gain of 

0.08 standard deviations in mathematics achievement, but this gain was not statistically significant.

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 2008–09; and 

the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

Across the five districts, students served by district providers and those served by 

non-district providers did not differ statistically significantly in either the 

mathematics or reading achievement gains associated with SES participation, 

relative to nonparticipation. Separate analyses by district indicated no differences 

between district and non-district providers in four of the five districts. In one district, 

Chicago, the gains for students served by non-district providers were statistically 

significantly larger than the gains for students served by the district provider in 

reading, but not in mathematics.

Across the five districts, the achievement gains associated with SES participation, relative to 

nonparticipation, for students served by district providers did not differ statistically significantly from the 

gains for students served by non-district providers in either mathematics or reading (see Exhibit S.4). 

Within each of the five districts individually, there were no statistically significant differences between 

the two types of providers in mathematics gains associated with SES participation. The only statistically 

significant difference between the two types of providers in reading gains was in Chicago. There, 

although the reading gains associated with participation were statistically significant for both district 

(0.03 standard deviations) and non-district providers (0.09 standard deviations), these gains were 

statistically significantly larger for students served by non-district providers than for students served by 

district providers (a difference of 0.06 standard deviations). Controlling for the number of hours of SES 

received did not affect the results for these comparisons between district and non-district providers. 
Executive Summary xxv Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit S.4

Differences in Achievement Gains Associated With SES Participation, Relative to 

Nonparticipation, for Students Served by District and Non-District Providers, by Subject

Exhibit reads: In Anchorage, the mathematics achievement gains associated with SES participation for 

students served by the district provider were 0.05 standard deviations lower than the mathematics achievement 

gains associated with participation for students served by non-district providers, but this difference was not 

statistically significant.

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district providers.

The grades included in analyses vary by district: grades 3–8 in Boston, Charlotte-Mecklenburg, and Chicago, and 

grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and Boston 

due to changes in tested grades during the study years.

* Indicates that the difference in the achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

District Communication With Parents

All five districts reported using multiple communication strategies to reach families 

of eligible students, provided balanced information about SES providers, translated 

information into at least one language other than English, and provided enrollment 

periods of at least six weeks. 

After accepting an SES waiver, districts participating in the pilot were required to improve 

communication with students and families by using multiple communication strategies, extending 

enrollment windows, and providing balanced information on all providers. The goal was to ensure that 

as many eligible students as possible received SES. To improve SES participation rates, all five districts 

used a variety of strategies to communicate with parents in 2008–09. These included letters, flyers, fairs, 

and advertisements in news media. Moreover, the documentation did not appear to favor district over 
Executive Summary xxvi Supplemental Educational Services and Student

Achievement in Five Waiver Districts

non-district providers in 2008–09. However, the written communications often included jargon and 

complex language, possibly making it difficult for some parents to understand.
Introduction 1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

I. Introduction

Under the Elementary and Secondary Education Act of 1965 (ESEA), as amended, schools served under 

Title I, Part A of the ESEA that do not make adequate yearly progress (AYP) for two consecutive years 

are identified for school improvement. If an identified Title I school does not make AYP while in that 

status, and thus enters the second year of school improvement status, its district must offer that school’s 

students from low-income families the opportunity to receive free supplemental educational services 

(SES) such as tutoring, remediation or other academic instruction provided outside the regular school 

day. The district must offer SES to students from low-income families until the school makes AYP for 

two consecutive years and thereby exits improvement status. A state educational agency (SEA) may 

approve as an SES provider a school district that is not identified for improvement or corrective action. 

However, under federal regulations, 34 C.F.R. § 200.47(b)(1)(iv)(B), an SEA may not approve as an SES 

provider a school district that is identified for improvement or corrective action.

In 2005, the U.S. Department of Education began a pilot program through which it granted waivers to 

allow districts identified for improvement or corrective action to serve as SES providers. Five districts 

received the waivers in the pilot period, which ended in 2008–09; the districts and respective years in 

which the waiver began are listed as follows: 

1. Anchorage, Alaska (2006–07 school year) 

2. Boston, Massachusetts (2005–06 school year)

3. Charlotte-Mecklenburg, North Carolina (2008–09 school year)

4. Chicago, Illinois (2005–06 school year)

5. Hillsborough County, Florida (2006–07 school year)

The central evaluation question for this report is whether SES participation for students served by the 

district providers was associated with achievement gains that were at least comparable with those of 

students served by non-district providers. This report also describes student participation in SES in the 

five waiver districts, the differences between students participating with district providers and students 

served by non-district providers, the relationship between SES participation and achievement gains 

overall, as well as for district and non-district providers, and the ways in which districts communicate 

with parents about SES. 

Overview of the SES Provisions of the ESEA

Students from low-income families are eligible for SES if they attend a Title I school that is in the second 

year of school improvement, is in corrective action, or is in restructuring. Districts that are required to 

offer SES must pay for these services and spend, subject to demand, an amount equal to at least 

20 percent of their Title I, Part A allocation on SES and transportation for public school choice.1

Parents of eligible students may choose a provider from a state-approved list, which may include the 

district provider and other, non-district providers such as for-profit agencies, not-for-profit groups, as 

well as faith-based and community-based organizations. School districts must consult with providers and 

parents to establish achievement goals for students, and providers are required to measure progress 

 1. In Title I schools that have been identified for school improvement, corrective action, or restructuring, the 

district must offer students the option to transfer to another public school in the district that has not been so identified. 

All students in identified schools are eligible for this option, and the district must provide transportation for participating 

students, subject to available funding.
Introduction 2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

toward these goals and to communicate with parents regarding their children’s progress. States are 

responsible for monitoring and evaluating provider performance and may remove providers from the 

state-approved list if the provider fails, for two consecutive years, to contribute to increasing students’

academic proficiency.

Under federal regulations, school districts that have been identified for improvement or corrective action 

are not eligible to serve as SES providers. However, the U.S. Department of Education granted waivers 

to five such districts, on a pilot basis, to allow them to provide SES. The goals of the pilot are to help 

ensure that the greatest possible number of eligible students are receiving SES and to provide accurate 

and comprehensive information on the effectiveness of SES provided by districts identified for 

improvement. 

Pilot districts are eligible to serve as providers in exchange for expanding students’ ability to access SES. 

Among other conditions,2 as part of the waiver, districts agreed to provide

—early notification to parents of their children’s eligibility to participate in SES;

—extended enrollment periods so that parents can make the best choice for their children;

—use of district facilities by non-district providers for a reasonable fee; and

—academic data to an independent third party for evaluation of the effectiveness of SES.

The pilot districts—Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough—received 

waivers through the pilot in different academic years. Boston and Chicago received waivers starting in 

the 2005–06 school year; Hillsborough and Anchorage received them starting in the 2006–07 school 

year; and Charlotte-Mecklenburg received a waiver starting in the 2008–09 school year. Boston and 

Chicago served as district SES providers before receiving their respective waivers; Hillsborough, 

Anchorage and Charlotte-Mecklenburg began serving as SES providers after receiving their waivers. 

In 2009–10, the pilot was replaced with a more expansive waiver opportunity. States were permitted to 

request a waiver from the U.S. Department of Education to approve as SES providers schools or 

districts in need of improvement, corrective action or restructuring. Twenty-nine states received waivers 

for the 2009–10 school year, and 22 states received approval for waivers for the 2010–11 school year.

Overview of Recent Literature

A few recent studies, using a variety of methodologies, have examined the impact of SES participation 

on student achievement and have generally reported either effects that were not statistically significant or 

effects of small statistical significance. A study by Heinrich, Meyer and Whitten examined three separate 

academic years of achievement data (2004–05, 2005–06 and 2006–07) in Milwaukee, Wisconsin, using 

propensity score matching methods. After matching, there were no statistically significant estimated 

effects on student achievement when comparing SES participants with eligible nonparticipants in all the 

 2. The regulations governing SES at 34 C.F.R. § 200.48(d)(2)(i), as amended in October 2008, now include criteria 

that mirror some of the pilot conditions. In particular, for a local education agency (LEA) to spend less than the full 

amount needed to meet its “20 percent obligation” for SES and public school choice-related transportation, the LEA

must, among other things, provide a minimum of two enrollment windows at separate points in the school year and 

ensure that SES providers are given access to school facilities, using a fair, open and objective process, on the same basis 

and terms as those available to other groups that seek access to school facilities.
Introduction 3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

analyzed years.3 A study by Chicago Public Schools of the 2007–08 school year found that participation 

in SES was not related to reading achievement gains, but participation was associated with a small, 

statistically significant increase in mathematics achievement gains after controlling for individual and 

school-level differences between participants and nonparticipants.4 In addition, a study by Barnhart 

found a small, but statistically significant, positive effect of the 2007–08 SES program overall on student 

achievement in both mathematics (0.03 standard deviations) and in reading (0.03 standard deviations) 

when comparing SES participants with students who did not attend.5 Finally, one study by the U.S. 

Department of Education found that, on average, across seven large, urban districts, participation in SES 

had a statistically significant effect on students’ achievement in mathematics (a gain of 0.09 standard 

deviations) and reading (a gain of 0.08 standard deviations).6

Several of these studies explicitly investigated the gains for students served by district versus non-district 

providers, a key aspect of this evaluation, and here the evidence was mixed. The Chicago Public Schools 

study reported an overall statistically significant increase in mathematics and in reading for SES 

participants with the district provider, relative to nonparticipants.7 The study by the U.S. Department of 

Education reported that, when compared, achievement gains by students served by district and by

non-district providers varied and showed no clear pattern.

For an earlier part of this study, we had analyzed data in four districts: Anchorage, Boston, Chicago and 

Hillsborough. These analyses, covering the academic year 2005–06 for Boston and Chicago and 

2006–07 for Anchorage and Hillsborough, also showed mixed results for achievement gains associated 

with SES participation for students served by district and non-district providers. In Anchorage, SES 

participation in either mathematics or reading for students served by either district or non-district SES 

providers was not associated with statistically significant achievement gains relative to nonparticipation. 

In Boston, for students served by the district provider, SES participation was associated with a 

statistically significant gain in mathematics, but not in reading. For students served by non-district 

providers, SES participation was not associated with statistically significant gains in either mathematics 

or reading. In Chicago, SES participation for students served by both district and non-district providers 

was associated with statistically significant gains in both mathematics and reading. In Hillsborough, SES 

participation for students served by both district and non-district providers was not associated with 

statistically significant achievement gains. In this report, these analyses have been conducted again, using 

additional years of data and including an additional district, Charlotte-Mecklenburg. 

 3. C. J. Heinrich, R. Meyer and G. Whitten, “Supplemental Education Services Under No Child Left Behind: Who 

Signs Up, and What Do They Gain?” Evaluation and Policy Analysis 32, no. 2 (2010): 273–298.

4. Chicago Public Schools, The 2008 Supplemental Educational Services Program: Year 5 Summative Evaluation (Chicago: 

Chicago Public Schools Office of Extended Learning Opportunities, 2009). 

5. M. K. Barnhart, The Impact of Participation in Supplemental Educational Services (SES) on Student Achievement: 

2007–08 (Los Angeles: Los Angeles Unified School District Research & Planning, 2009).

6. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental 

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007). 

7. Chicago Public Schools, The 2008 Supplemental Educational Services Program. 
Introduction 4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Evaluation Questions

This report seeks to address the following sets of evaluation questions.

Eligibility for and Participation in SES

1. Did the percentage of eligible students participating in SES during the pilot years increase from 

the prior year(s)? 

2. How were participating students distributed among providers? 

3. Were there statistically significant demographic or academic differences between students served 

by different providers, specifically students served by the district providers compared with those 

served by non-district providers?

Student Achievement

1. What are the achievement gains for students participating in SES?

2. Are gains in achievement for students receiving SES from district providers comparable with

gains for students served by non-district providers? 

3. Controlling for amount of exposure to SES, are achievement gains for students served by district 

providers comparable with those for students served by non-district providers? 

District Communication With Parents

1. Are districts communicating with parents about the availability of SES in a manner that is clear, 

user friendly and accessible?

2. Is the nature of district communication unbiased with respect to particular providers, specifically 

non-district providers and district providers?

Chapter II addresses the research questions concerning eligibility for and participation in SES. Chapter 

III addresses the research questions concerning student achievement. Chapter IV addresses the research 

questions concerning district communication with parents. Appendix A includes additional exhibits that 

supplement those found in Chapters II and III.

Data Sources

In this evaluation, each district provided student-level data, including whether the student was eligible for 

SES, whether the student participated in SES, with which providers, in what subjects, and for how many 

hours. Districts also provided data related to student achievement and characteristics (e.g., race/ethnicity, 

disability status, limited English proficiency and grade level). All these data were provided for multiple 

academic years. Because districts received the waivers in different years, and because state assessment 

systems included different grades, this report covers different academic years and different grade levels 

for each district. Details on the data provided by each district follow the discussion of data challenges. 

The data request presented three challenges for the districts:

—Eligibility for SES. It was challenging for districts to provide complete and accurate data on 

student eligibility for SES. Technically, a student is eligible for SES if he or she comes from a 

low-income family and attends a Title I school that is in the second year of school improvement, 

in corrective action, or in restructuring. To determine whether a student is from a low-income 

family, the district must use the same poverty measure (usually eligibility for free or reduced-price 
Introduction 5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

lunch) used to determine its Title I schools. However, providing data on student eligibility for the 

free or reduced-price lunch program is fraught with problems because families are required to 

apply each year and districts often maintain these data separately from other student demographic 

data. Also, not all eligible students are enrolled in the free or reduced-price lunch program. Finally, 

districts may not have a final list of schools that are required to offer SES in time to notify eligible 

students of their options. In some cases, districts predict which schools and students are likely to 

be eligible. Often, districts are correct in these predictions, but occasionally, they misidentify 

schools and students. 

As a result, the research team established parameters to determine student eligibility for SES. Any 

student in a school not required (according to district designations) to offer SES was considered 

ineligible.8 In addition, if student demographic data indicated that students were not eligible for 

services by virtue of not being eligible for free or reduced-price lunch, then they were considered 

ineligible for the purpose of the analysis, even if they were, in fact, receiving services (the number 

of students removed from the analysis for this reason was very small).9

—Student participation in SES. Another challenge in reporting SES data involved student 

participation. Districts cannot ensure that enrolled students will attend tutoring services. 

Occasionally, some enrolled students do not attend any SES sessions during the entire school 

year. However, these students still hold a place in the district’s system and are, therefore, counted 

as participants by the district. In this study, “participants” who did not receive tutoring are not 

considered SES participants.10

—Multiple providers. Although the analysis model assumes that students receive services from 

only one provider a year, several districts provided information for students with multiple 

providers in a single academic year. These students represent a very small percentage of 

participants.11 In this report, these students were treated as having received services from the 

provider that supplied the most SES (i.e., the greatest number of hours) that year.

 8. There were two districts, Chicago and Hillsborough, in which schools were misidentified as being required to 

offer services. In Chicago, before the start of the school year, there was not always an official list of schools required to 

offer SES, which meant that the district had to either delay services or predict which schools would be required to offer 

SES. Chicago officials chose the latter option and, in doing so, inadvertently designated seven schools as required to 

offer services in 2004–05 when, in fact, they were not required to do so, and these schools did offer SES. From 

2006–07 to 2008–09, another eight schools were inadvertently identified as required to offer SES. In Hillsborough, one 

school in 2008–09 was inadvertently considered required to offer SES, although it was not, in fact, required to do so. 

SES participants attending misidentified schools (i.e., ineligible participants) were not included in descriptive statistics 

and were dropped from the student achievement analyses.

9. In Chicago, a district database determines which students are to be notified that they are eligible for SES. In 

schools required to offer SES, the number of students who are eligible for free or reduced-price lunch are identified 

using this database. This source is different from the one provided to the research team, which explains the few 

discrepancies between these data on eligibility for free and reduced-price lunch and district designations. According to 

the district, only low-income students are notified of their eligibility; therefore, the researchers treated all participants in 

eligible Chicago schools as eligible, which they did not do in other districts.

10. Three districts—Boston, Chicago and Hillsborough—reported SES participants who did not receive any hours 

of tutoring. In Boston, the following percentages of participants had zero hours: 23 percent in 2004–05, 26 percent in 

2006–07, 81 percent in 2007–08 and 44 percent in 2008–09. In Chicago, the following percentages of participants had 

zero hours: less than 1 percent in 2004–05 through 2007–08 and 16 percent in 2008–09. In Hillsborough, the following 

percentages of participants had zero hours: 23 percent in 2005–06, 22 percent in 2006–07, 25 percent in 2007–08 and 

21 percent in 2008–09. These students were not considered SES participants for this study.

11. Between 2004–05 and 2008–09, Hillsborough reported 1,371 students with multiple SES providers within the 

same academic year, and Charlotte-Mecklenburg reported 63 students. Between 2006–07 and 2008–09, Boston reported 

12 students with multiple SES providers.
Introduction 6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Although data on all five districts are presented together in this analysis when possible, it is important to 

note that distinct features of each district (e.g., the administration of SES and the supply of providers), 

pose challenges to a side-by-side comparison of each district’s results. Furthermore, with respect to 

achievement gains associated with SES participation, it is not possible to make direct comparisons 

among the districts because the achievement measures, the grades tested and the years of data differ 

from district to district. Although it is beyond the scope of this analysis to explore the reasons for 

possible variations in results for the districts, explanations for differences in district results might include

the following factors.

—Context: District size, demographic characteristics, average prior achievement levels, or annual 

fluctuations in the number of students eligible for and participating in SES may individually, or in 

combination, influence the gains associated with SES participation.

—Implementation: Notification practices; enrollment opportunities; and differences in the 

duration, nature and quality of the various SES programs offered in each district, including the 

hours and location of services and tutor/student ratios, may contribute individually, or in 

combination, to differences in results. District capacity for implementing and monitoring SES also 

varies in each of the districts.

Exhibit 1 provides an overview of the data analyzed from each district.
Introduction 7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 1

Data Analyzed From Anchorage, Boston, Charlotte-Mecklenburg,

Chicago and Hillsborough

Anchorage Boston CharlotteMecklenburg Chicago Hillsborough

Data on eligibility, participation, SES provider and hours of services received

Years 2004–05 

to 2008–09

2004–05

to 2008–09

2005–06 

to 2008–09

2004–05

to 2008–09

2004–05 

to 2008–09

Grades K–12

K–8 in 2004–05;

K–12 in 2005–06 to 

2008–09

K–8 3–8

K–5 in 2004–05;

K–12 in 2005–06 to 

2008–09

Data on demographic and subject of tutoring 

Years 2006–07 to 2008–09 2004–05 to 2008–09 2008–09 2004–05 to 2008–09a 2006–07 to 2008–09

Grades K–12

K–8 in 2004–05;

K–12 in 2005–06 to 

2008–09

K–8 3–8 K–12

Data on student achievement

Test 

Alaska Benchmark 

Exam in 2003–04;

Alaska 

Standards-Based 

Assessment for

2004–05 to 2008–09

Stanford 9 for 

2002–03 to 2005–06; 

Massachusetts 

Comprehensive 

Assessment System for

2005–06 to 2008–09

North Carolina 

End-of-Grade Tests for

2004–05 to 2008–09

Iowa Test of Basic 

Skills for 2001–02 to 

2004–05;

Illinois Standards 

Assessment Test for 

2005–06 to 2008–09 

Florida’s 

Comprehensive 

Assessment Test for

2002–03 to 2008–09

Tested grades

3, 6 and 8

in 2003–04;

3–9 in 2004–05;

3–10 in

2005–06 to 2008–09

3–5 in 2002–03 to 

2005–06; 4–8 

in 2006–07 to

2008–09

3–8 3–8 3–10

Exhibit reads: In terms of eligibility, participation, provider and hours of services received, Anchorage 

provided data from 2004–05 to 2008–09.

Notes: SES = Supplemental Educational Services.

In a few instances, not all data received from the districts were used in analyses for various reasons 

(e.g., gains could not be calculated because the state did not test consecutive grades at the time). 

a Chicago did not provide data on the subject of tutoring.

Anchorage

Anchorage provided detailed information on SES for students in kindergarten through grade 12. These 

data included student demographic characteristics, eligibility status, participation status, the provider 

from whom a student received services, the subject of tutoring, and the number of hours of tutoring 

participants received in school years 2004–05 through 2008–09. 

Anchorage also provided longitudinal student-level achievement data in mathematics and reading from 

its state assessments for all students in the district. In 2004–05, students in grades 3 through 9 took the 

Alaska Standards-Based Assessments, while students in grades 3 through 10 did so in 2005–06 through 

2008–09. In 2003–04, achievement data were available only for students in grades 3, 6 and 8 from the 

previous state assessment, the Alaska Benchmark Exam.
Introduction 8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Boston

For most students in the district, Boston provided data on student demographic characteristics, eligibility 

status, participation status, the provider from whom a student received services, the subject of tutoring, 

and the number of hours of tutoring participants received for kindergarten through grade 8 in

2004–05 and for kindergarten through grade 12 in 2005–06 through 2008–09.12

Boston also provided longitudinal, student-level achievement data for the entire district population. For 

the 2002–03 through 2005–06 school years, the district provided the Stanford 9 reading and mathematics 

test scores for grades 3 through 5. The district administered this assessment in the fall of each year. The 

district also provided assessment data for the Massachusetts Comprehensive Assessment System 

(MCAS), which was administered in the spring of each year. MCAS data were provided for fifth-grade 

students for 2005–06 and for grades 4 through 8 for 2006–07 through 2008–09.13

Charlotte-Mecklenburg

Charlotte-Mecklenburg provided detailed SES information on students in kindergarten through grade 

8 on student demographic characteristics, eligibility status, participation status, the provider from whom 

a student received services, the subject of tutoring, and the number of hours of tutoring that participants 

received in school years 2005–06 through 2008–09. 

Charlotte-Mecklenburg provided longitudinal, student-level achievement data on North Carolina’s 

End-of-Grade exams in mathematics and reading for all students in grades 3 through 8 from 

2004–05 through 2008–09.

Chicago

Chicago provided SES information on student eligibility, participation, the provider from whom each 

student received services, and the number of minutes or hours of tutoring participants received for 

grades 3 through 8 from 2004–05 through 2008–09.14 Chicago did not provide data on the subject of 

tutoring during this time period. Demographic data were available only for SES participants and students 

taking standardized assessments (grades 3 through 8).

For 2001–02 through 2004–05, Chicago provided student-level data for all students in grade 3 through 

grade 8 taking the Iowa Test of Basic Skills. However, the district stopped administering that test in 

2005–06; therefore, the study used student-level results from the state assessment, the Illinois Standards 

Assessment Test, for the 2005–06 through 2008–09 school years for students in grade 3 through grade 8. 

Hillsborough

Hillsborough provided detailed SES information on students in kindergarten through grade 12 on 

student demographic characteristics, eligibility status, participation status, the provider from whom a 

student received services, the subject of tutoring, and the number of hours of tutoring participants 

 12. Boston did not provide SES data for students attending charter schools not operated by the district. 

13. Boston also administered the MCAS to third-grade students from 2006–07 to 2008–09. However, in these years, 

the district did not calculate the standardized scores needed for the achievement analyses.

14. There were several implausible values in the Chicago data on number of hours of services received in 

2004–05 through 2008–09. The final dataset excluded cases in which the number of hours exceeded 120, which is the 

maximum according to the district. Approximately 2 percent of participants had more than 120 hours, with an average 

of 240 hours. 
Introduction 9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

received in 2005–06 through 2008–09. In 2004–05, detailed SES information was provided only for 

eligible students in kindergarten through grade 5.

Hillsborough provided longitudinal, student-level data for all students in grades 3 through grade 10

taking Florida’s Comprehensive Assessment Test in 2002–03 through 2008–09.

In addition to the student-level data, each district provided information about its communication efforts 

with parents. The research team examined examples of submitted documents and reviewed the methods 

of distribution, as reported by the districts. 

Analyses of Achievement Gains

Student gains on standardized assessments from one academic year to the next served as the 

achievement outcome measures. The gains are created by subtracting prior year scores from the 

subsequent year scores. Before subtracting the scores, the assessment scores were translated into 

z-scores. A z-score indicates a student’s position in the test score distribution relative to other students in 

his or her district, year and grade level. In other words, each district’s scores for each year and grade level 

are standardized so that the mean score is 0 and the standard deviation is 1. Thus, a student with a 

z-score of 0.5 is one half of a standard deviation above the district mean. Gains on this metric do not 

indicate a developmental improvement; rather, they represent a student’s movement within the test score 

distribution relative to the movement of all students in the same year and grade level in the same district.

The analytic model for achievement analyses in this study is the student fixed-effects model, which 

estimates changes in student achievement gains over time, comparing each student’s achievement gains 

during the years in which the student participated in SES with gains in the years in which the student did 

not participate. The approach followed the methods used for a previous study of SES and student 

achievement in nine large urban school districts that was conducted as part of the National Longitudinal 

Study of No Child Left Behind.15 The student fixed-effects approach assesses the changes in achievement 

gains associated with SES participation, controlling for the effects of time-invariant student 

characteristics such as parent education and underlying student ability or motivation. 

By including a fixed effect for each student in the analytic model, we guarded against potential selection 

bias that could result if students who received services were different from others in ways that could not 

be measured. This approach essentially controlled for all the time-invariant characteristics of each 

student (e.g., motivation or high level of parental involvement) that might otherwise have affected that 

student’s achievement gains. This approach does not, however, control for time-varying student 

characteristics (e.g., illness or changes to family structure). 

The model used for the analysis of the overall achievement gains associated with SES participation, 

relative to nonparticipation, was

Ait – Ait-1 = α Eligibleit + β Participationit + μi + θgt + εit

Where

Ait – Ait-1 is the achievement gain of student i from year t-1 to year t;

Eligibleit is the eligibility status of student i in year t;

 15. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental 

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007).
Introduction 10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Participationit is the participation status of student i in year t;

μi represents the fixed effect for student i;

θgt represents a set of grade-by-year effects; and 

εit represents the random error associated with student i in year t.

The coefficient β from the above model captures the overall achievement gain associated with SES 

participation relative to nonparticipation. A significant positive coefficient indicates that SES 

participation was associated with statistically significantly larger achievement gains during years when 

students participated in SES relative to years when they did not participate. 

In addition to the model for the overall gains associated with SES participation, we constructed models 

to assess the relationship between the achievement gains and the following characteristics of the SES 

that students received: 

1. Provider type: SES received from district provider or non-district provider 

2. Provider type, controlling for number of hours of participation

3. Individual non-district SES providers

4. Number of years: Students who have received SES for only a single year, compared with 

students who have received SES for multiple years

For example, the model for provider type was as follows: 

Ait – Ait-1 = α Eligibleit + λ District Providerit + ξ Non-District Providerit + μi + θgt + εit

The coefficient for the district provider variable, λ, indicates the gains associated with SES participation 

relative to nonparticipation for students served by the district provider, while the coefficient for the 

non-district provider variable, ξ, indicates the gains associated with SES participation for students served 

by non-district providers. The two coefficients test whether the achievement gains associated with SES 

participation for students served by the district provider and students served by non-district providers

are statistically significantly different from zero. We also tested whether the gains associated with SES 

participation for students served by the district provider showed statistically significant differences from 

the gains of students served by non-district providers, that is, whether λ – ξ= 0. A statistically significant 

positive value of (λ – ξ) suggests that students served by the district provider experienced larger gains in 

achievement than did students served by non-district providers. A statistically significant negative value 

of (λ – ξ) suggests otherwise.

The results of the achievement analyses are based on a subset of the student population in each 

district—students in tested grades.16 The analytic approach further requires each student to have 

achievement scores for at least three time points in either reading or mathematics to construct at least 

two gain scores for each student. Finally, students who repeated one of the tested grades (e.g., enrolled in 

third grade two years in a row) were not included in the analyses.17 The achievement analyses reported 

 16. The analyses of district providers versus non-district providers were restricted to those students who received 

SES during years when both district and non-district providers offered SES.

17. Given the fact that the outcome measure is designed to measure gains in a student’s progress in relation to the 

rest of the students in his or her grade level, the inclusion of a student who has repeated a grade inherently leads to 

comparing an on-track student with a student who is one year older and has taken the same test twice. For example, for 

the cohort of fifth-graders in 2004–05, we computed their gains as the difference between their reading score in the 

prior year (2003–04), as fourth-graders, and their reading score in 2004–05, as fifth-graders, with both scores 

standardized to the full population of test takers of each grade level in each year. Any repeating fifth-graders might 

have larger gains because they took the same test in both years.
Introduction 11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

here use this sample of students. Sample sizes vary by district based on the number of students served, 

the data available and the requirements for inclusion in this analytic method. The sample size and 

standard errors (included in each district appendix; see Appendices B–F) must be taken into account 

when examining statistically significant (or nonsignificant) findings because smaller sample sizes are less 

likely to have statistically significant gains than larger sample sizes, all else being equal.

Because the fixed-effects approach estimates the impact of SES by comparing each student’s gains 

during years when he or she participated in SES with gains during periods in which he or she did not 

participate, it would also be possible to estimate the model by restricting the sample to students with and 

without periods of SES participation. Appendix G provides estimates based on this restricted sample for 

two regression analyses in each district. 

The achievement gains associated with SES participation averaged across the five districts, based on a 

fixed-effects approach, are also reported (see Appendix H for details about the approach as well as the 

gain estimates and standard errors). 

Eligibility for and Participation in SES 13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

II. Eligibility for and Participation in

Supplemental Educational Services

This section presents an overview of SES in the five districts over time by examining the eligible and 

participating student populations and SES participation rates, the amount of services received by 

students, and the demographic and other characteristics of students participating in SES in general. In 

addition, the descriptive statistics compare the characteristics of students receiving SES from district 

providers with the characteristics of students receiving SES from non-district providers. 

The policy questions concern whether participation in SES has increased during the waiver period in 

each of the districts, whether the inclusion of a district provider affected the manner in which students 

were distributed across provider types, and whether any differences existed in the characteristics of 

students served by district versus non-district providers. Although some findings are summarized across 

five districts, it is important to note that district contexts differ in many ways, including the years 

available for analysis, the waiver year and the set of non-district providers, which makes comparing 

districts inappropriate.

Key Findings

—In all five districts, the percentage of students eligible for SES increased over time.

—Three districts became SES providers for the first time after the waiver was granted—

Anchorage, Charlotte-Mecklenburg and Hillsborough. In these districts, overall student 

participation in SES increased in the first year of the waiver.

—In all five districts, the number of non-district providers increased from the first 

post-waiver year through 2008–09; the non-district providers were primarily small, each 

serving fewer than 100 students.

—SES participants served by the district provider tended to receive a similar or greater 

number of hours of tutoring as participants served by non-district providers.

—SES participants were primarily elementary school students, and most participants were 

of a minority race or ethnicity. Eligible students with limited English proficiency and 

students with disabilities participated at similar to or higher than the rates of other 

eligible students.

—There were few demographic or academic differences between students served by district 

providers and students served by non-district providers. 
Eligibility for and Participation in SES 14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Eligibility for Supplemental Educational Services

Student eligibility for SES generally reflects the individual context in each district within each year—

including the academic and other characteristics of the student population and the number of schools 

identified for improvement. These differences across districts are demonstrated, for example, by the fact 

that, in Chicago, as many as 77 percent of district students were eligible for SES in 2008–09, whereas in 

Anchorage, the eligibility rate was 9 percent during the same period (see Exhibit 2). 

Overall, the percentage of students eligible for SES increased over time in all 

five districts.

In all five districts, the percentage and number of students eligible for SES generally increased over 

the four or five years of available data. In Boston, the number of eligible students increased each year, 

from 8,282 students (20 percent) in 2004–05 to 31,513 students (57 percent) in 2008–09 (see Exhibit 2). 

Similar continuous growth was seen in Charlotte-Mecklenburg and Hillsborough. In Anchorage and 

Chicago, the number and percentage of eligible students increased over the evaluation years, but each of 

these districts had at least one year with a decrease in the percentage of eligible students. 
Eligibility for and Participation in SES 15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 2

Number and Percentage of Students Eligible for SES, Participating in SES

and Served by Non-District Providers, by School District and by Academic Year

Academic 

Year

District 

Student 

Population

Students Eligible for SES SES Participants Students Served by Non-District 

Providers

Number Percentage of 

All Students Number Percentage of Eligible Students Number Percentage of SES Participants

Anchorage

2004–05 52,049 709 1 31 4 31 100

2005–06 52,338 2,089 4 104 5 104 100

2006–07 51,674 4,427 9 308 7 170 55

2007–08 51,163 4,110 8 540 13 193 36

2008–09 51,739 4,617 9 644 14 295 46

Boston

2004–05 41,847 8,282 20 1,521 18 459 30

2005–06 62,103 21,954 35 3,288 15 876 27

2006–07 55,616 22,429 40 2,997 13 94 3

2007–08 55,672 27,338 49 609 2 478 78

2008–09 55,710 31,513 57 1,911 6 750 39

Charlotte-Mecklenburg

2005–06 95,369 5,164 5 312 6 312 100

2006–07 98,997 6,974 7 1,081 16 1,081 100

2007–08 101,615 10,109 10 2,175 22 2,175 100

2008–09 101,712 16,865 17 3,972 24 3,028 76

Chicago

2004–05 169,217 93,890 56 31,950 34 16,370 51

2005–06 156,884 83,155 53 23,631 28 14,614 62

2006–07 164,791 118,647 72 22,678 19 15,613 69

2007–08 181,511 129,031 71 24,190 19 18,580 77

2008–09 176,906 135,338 77 34,057 25 29,552 87

Hillsborough

2004–05 94,784 1,843 2 265 14 265 100

2005–06 204,062 35,633 17 3,640 10 3,640 100

2006–07 203,468 42,048 21 6,028 14 5,084 84

2007–08 201,704 46,013 23 4,879 11 4,070 83

2008–09 200,782 49,727 25 5,786 12 4,731 82

Exhibit reads: In 2004–05, within the Anchorage district, 709 students (1 percent of all students) were 

eligible for SES, 31 students participated in SES (4 percent of eligible students), and 31 students (100 percent 

of SES participants) were served by non-district providers.

Notes: SES = Supplemental Educational Services.

Shaded boxes indicate years before waivers were granted.

The grades included vary by district: grades K–12 in Anchorage; grades K–8, 2004–05, and K–12, 

2005–06 through 2008–09, in Boston; grades K–8 in Charlotte-Mecklenburg; grades 3–8 in Chicago; and 

grades K–5, 2004–05, and K–12, 2005–06 through 2008–09, in Hillsborough.

Source: Administrative data provided by the Anchorage, Boston, Chicago and Hillsborough school districts, 

2004–05 through 2008–09, and by the Charlotte-Mecklenburg school district, 2005–06 through 2008–09.
Eligibility for and Participation in SES 16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Participation in Supplemental Educational Services

While all five districts received SES waivers, the Boston and Chicago school districts provided SES to 

students before the granting of the waivers in 2005–06. In Anchorage, Charlotte-Mecklenburg and 

Hillsborough, the waiver resulted in the introduction of the district into the pool of available providers. 

Regardless of the introduction of district providers, on receipt of the waiver, all five districts were 

required to increase communications to eligible students and their families regarding the availability 

of SES.

In the three districts that began to serve as SES providers after the waiver was 

granted, student participation in SES increased in the first post-waiver year.

Among the three districts that began serving as SES providers after the waiver was granted—Anchorage, 

Charlotte-Mecklenburg and Hillsborough—SES participation grew during the first year of the waiver. 

For example, in Charlotte-Mecklenburg, the total number of students who received SES increased from 

2,175 in 2007–08 to 3,972 in 2008–09 (see Exhibit 3). While the eligible student population increased 

after the waiver was granted in these three districts (see Exhibit 2), participation rates also rose (by 

two percentage points in Anchorage and Charlotte-Mecklenburg and four percentage points in

Hillsborough) (see Exhibit 3). 

The number of participants in Anchorage increased from 104 students before the waiver to 308 students

after the waiver (see Exhibit 3). The low number is due in part to the small size of the district and the 

small number of eligible students in Anchorage (4,427 students) (see Exhibit 2). However, Anchorage 

also had a low percentage of eligible students participating (7 percent).1

In the districts that had served as SES providers before the waiver—Boston and 

Chicago—SES participation rates decreased the year after the waiver. However, in 

Boston, the number of students participating increased after the waiver due to 

increased student eligibility.

Both Boston and Chicago were district providers before the waiver. In Boston, participation rates in SES 

decreased from 18 percent in 2004–05 to 15 percent in 2005–06 (see Exhibit 3). However, since the 

number of eligible students increased from 8,282 in 2004–05 to 21,954 in 2005–06, participation in SES 

more than doubled from the preceding year, increasing from 1,521 students to over 3,288 students (see 

Exhibit 2). In Chicago, the number of eligible students decreased after the waiver was granted, from 

93,890 in 2004–05 to 83,155 in 2005–06, along with the number of SES participants (from 

31,950 students to 23,631 students) and participation rate (from 34 percent to 28 percent). 

 1. For comparison with national rates, see data in U.S. Department of Education, Office of Planning, Evaluation 

and Policy Development, Policy and Program Studies Service, State and Local Implementation of the No Child Left Behind 

Act, Volume VII—Title I School Choice and Supplemental Educational Services: Final Report (Washington, D.C.: U.S. 

Department of Education, 2009). In 2005–06, only 5 percent of eligible students participated in Anchorage, compared 

with the 2005–06 average national participation rate of 17 percent.
Eligibility for and Participation in SES 17 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 3

Number and Percentage of Eligible Students Participating in SES, by School District, 

Immediately Before and After the Waiver

Exhibit reads: In Anchorage, 5 percent of students eligible for SES participated in the school year before

the waiver, and 7 percent participated in the school year after the waiver. In 2005–06, 104 eligible students 

participated in SES.

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage and Hillsborough; grades K–8 in CharlotteMecklenburg; grades K–8, 2004–05, and K–12, 2005–06, in Boston; and grades 3–8 in Chicago. In Boston, for 

grades K–8 in 2005–06, the number of eligible students participating in SES was 3,102 (17 percent of the 

eligible students). 

The Anchorage, Charlotte-Mecklenburg and Hillsborough school districts became providers after receiving a 

waiver. The Boston and Chicago school districts served as SES providers before receiving the waiver.

Source: Administrative data provided by the Anchorage and Hillsborough school districts, 2005–06 and 2006–07; 

the Charlotte Mecklenburg school district, 2007–08 and 2008–09; and the Boston and Chicago school districts, 

2004–05 and 2005–06.

In each district, participation rates of eligible students varied over time. 

In two districts—Anchorage and Charlotte-Mecklenburg—participation rates among eligible students 

increased consistently each year. In Anchorage, participation rates more than tripled, from 4 percent in 

2004–05 to 14 percent in 2008–09 (see Exhibit 2). In Charlotte-Mecklenburg, participation rates 

increased from 6 percent in 2005–06 to 24 percent in 2008–09. Conversely, in Boston, eligible student 

participation rates decreased annually from 18 percent in 2004–05 to 2 percent in 2007–08, with an 

increase to 6 percent in 2008–09. Participation rates of eligible students in Chicago also decreased 

between 2004–05 (34 percent) and 2006–07 (19 percent), stabilized for one year, and then returned to 
Eligibility for and Participation in SES 18 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

25 percent in 2008–09. In Hillsborough, eligible student participation rates fluctuated annually, with the 

highest rate reaching 14 percent in both 2004–05 and 2006–07 and the lowest at 10 percent in 2005–06.

Number and Size of Non-District Providers

When districts become SES providers, the goal is for the district to add to the pool of providers and 

expand services to accommodate a greater number of eligible students rather than replace non-district 

providers or shift the existing participants among more providers. 

The introduction of the district provider did not appear to limit the number

of non-district providers. Most of these providers were small, serving fewer than 

100 students.

In the three districts that did not serve as SES providers before the waiver, their introduction as district 

providers did not appear to negatively influence the number of non-district providers in the long run. In 

the first year after the waiver (2006–07) in Anchorage and Hillsborough, the number of non-district 

providers decreased by one but then rebounded by the following year (see Exhibit 4). In CharlotteMecklenburg, the number of non-district providers increased from 12 providers in 2007–08 to 

18 providers in 2008–09 (the first waiver year). The number of non-district providers increased 

immediately after the waiver in Boston and Chicago, in 2005–06, although both districts were already 

serving as SES providers.

Over time, as the number of participants increased in each of the districts, so did the number of 

non-district providers serving them (see Exhibit 2). For example, in Chicago, the number of non-district 

providers started at 19 in 2004–05 and grew to 45 in 2008–09 (see Exhibit 4). 

Most of the non-district providers served fewer than 100 students (see Appendix Exhibit A.1). Of the 

30 non-district providers in Hillsborough, for example, in 2008–09, 22 served fewer than 100 students, 

seven served 100–999 students and one served more than 1,000 students. Chicago, which is the district 

with the greatest number of participants in this evaluation, was an exception; the district had the most, 

and the largest, non-district providers. In 2008–09, Chicago had 45 non-district providers: 15 served 

fewer than 100 students, 21 served 100 to 999 students, and 9 served over 1,000 students.
Eligibility for and Participation in SES 19 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

SES Participation With District and Non-District Providers

The following section discusses participation rates with district and non-district providers, hours of SES 

provided by district and non-district providers, and any differences between characteristics of SES 

participants served by the district providers and characteristics of students served by non-district 

providers.

Participation Rates

In the three districts that began to serve as SES providers after the waiver was 

granted, non-district providers served the majority of students participating in SES 

in the first year of the waiver. Over time, participation rates by provider type varied 

by school district and by year.

After receiving their respective waivers, the Anchorage, Charlotte-Mecklenburg and Hillsborough school 

districts began serving as SES providers. During the first year after these districts received a waiver,

the non-district providers in those districts continued to serve the majority of SES participants (see

Exhibit 5). In 2006–07, non-district providers served 55 percent of participants in Anchorage and 

84 percent of participants in Hillsborough; in 2008–09, non-district providers served 76 percent of 

participants in Charlotte-Mecklenburg. Two of these districts have multiple years of post-waiver data 

available; in those additional years, Anchorage non-district providers served fewer than 50 percent of the 

participants (36 percent in 2007–08 and 46 percent in 2008–09), and Hillsborough non-district providers 

continued to serve similar percentages of participants (more than 80 percent each year) (see Exhibit 2). 

The Boston and Chicago districts both served as SES providers before the waiver. In Boston, 

non-district providers served fewer than one-third of SES participants both before and after the waiver: 

30 percent in 2004–05 and 27 percent in 2005–06 (see Exhibit 5). In the longer term, participation rates 

Exhibit 4

Number of Non-District SES Providers, by Year and by School District

Anchorage Boston CharlotteMecklenburg Chicago Hillsborough

2004–05 2 5 — 19 10

2005–06 5 15 10 35 24

2006–07 4 5 10 43 23

2007–08 5 15 12 53 28

2008–09 7 16 18 45 30

Exhibit reads: In 2004–05, the total number of non-district providers serving participating students in 

Anchorage was 2. 

Notes: SES = Supplemental Educational Services.

Charlotte-Mecklenburg did not provide data for 2004–05.

Shaded boxes indicate years before waivers were granted.

The grades included vary by district: grades K–12 in Anchorage; grades K–8, 2004–05, and K–12, 

2005–06 through 2008–09, in Boston; grades K–8 in Charlotte-Mecklenburg; grades 3–8 in Chicago; and 

grades K–5, 2004–05, and K–12, 2005–06 through 2008–09, in Hillsborough.

Source: Administrative data provided by the Anchorage, Boston, Chicago and Hillsborough school districts, 

2004–05 through 2008–09, and by the Charlotte-Mecklenburg school district, 2005–06 through 2008–09.
Eligibility for and Participation in SES 20 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

were inconsistent, with non-district providers serving 3 percent of SES participants in 2006–07, 

78 percent in 2007–08, and 39 percent in 2008–09 (see Exhibit 2). Conversely, in Chicago, non-district 

providers served more than half of SES participants before the granting of the waiver (51 percent in 

2004–05), served nearly two-thirds of participants (62 percent) in 2005–06, and increased steadily to 

serving 87 percent of participants in 2008–09.

Exhibit 5

Number and Percentage of SES Participants Who Were Served by Non-District Providers, 

by School District, Immediately Before and After the Waiver

Exhibit reads: In Anchorage, where the district served as an SES provider only after the waiver, non-district 

providers served 100 percent of SES participants during the school year before the waiver and served

55 percent of participants in the year after the waiver. In 2005–06, 104 SES participants were served by 

non-district providers in Anchorage.

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage and Hillsborough; grades K–8 in CharlotteMecklenburg; grades K–8, 2004–05, and K–12, 2005–06, in Boston; and grades 3–8 in Chicago. In Boston, for 

grades K–8 in 2005–06, non-district providers served 25 percent of SES participants.

The Anchorage, Charlotte-Mecklenburg and Hillsborough school districts became providers after receiving a 

waiver. The Boston and Chicago school districts served as SES providers before receiving the waiver.

Source: Administrative data provided by the Anchorage and Hillsborough school districts, 2005–06 and 2006–07; the 

Charlotte-Mecklenburg school district, 2007–08 and 2008–09; and the Boston and Chicago school districts, 2004–05 

and 2005–06.
Eligibility for and Participation in SES 21 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Hours of SES Provided

District providers may differ from non-district providers by offering more or fewer hours of tutoring. 

This section presents comparisons of hours of tutoring received by provider type. These comparisons 

focus on the 2008–09 data provided by districts, which is both the most recent year of data available and 

the only year in which all five districts had experienced a post-waiver year, making comparisons of the 

district provider with the non-district providers possible in every district. 

Participants served by district providers received a similar or greater number of 

hours of SES as participants served by non-district providers. 

SES participants in Chicago and Hillsborough in 2008–09 received a statistically significantly greater 

number of hours of tutoring on average from district providers (55 and 29 hours, respectively) than from 

non-district providers (37 and 21 hours, respectively) (see Exhibit 6). In Anchorage, Boston and 

Charlotte-Mecklenburg in 2008–09, the difference between the average number of hours of tutoring 

received by students served by the district and the non-district providers was not statistically significant. 
Eligibility for and Participation in SES 22 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 6

Average Number of Hours of SES Received by Participants, by School District, 

by Provider Type and by Year

2005–06 2006–07 2007–08 2008–09

Anchorage

All providers 39 28 31 31

District provider — 31 29 32

Non-district providers 39 25 36 31

Boston

All providers 44 79 46 43

District provider 45* 80* 80* 40

Non-district providers 43* 57* 36* 47

Charlotte-Mecklenburg

All providers 20 21 24 27

District provider — — — 26

Non-district providers 20 21 24 27

Chicago

All providers 81 38 37 39

District provider 80* 47* 46* 55*

Non-district providers 81* 34* 34* 37*

Hillsborough

All providers 23 22 23 22

District provider — 22 32* 29*

Non-district providers 23 23 22* 21*

Exhibit reads: In Anchorage in 2005–06, no students were served by the district provider, and students 

served by the non-district providers received an average of 39 hours of services. In Anchorage in 2006–07, all 

students on average received 28 hours of SES; students served by the district provider received an average of 

31 hours of SES, while students served by non-district providers received an average of 25 hours of services. 

This difference was not statistically significant.

Notes: SES = Supplemental Educational Services.

Shaded boxes indicate years before waivers were granted.

The grades included vary by district: grades K–12 in Anchorage, Boston and Hillsborough; grades K–8 in 

Charlotte-Mecklenburg; and grades 3–8 in Chicago. 

— Indicates that the district did not provide SES at this time.

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2005–06 through 2008–09.
Eligibility for and Participation in SES 23 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

SES Participant Characteristics

SES participants were primarily elementary school students, and in three of the 

five districts, there was no difference between district and non-district providers in 

the distribution of the grade levels of SES participants.

In all districts, most participating students in 2008–09 were in elementary school (grades K–5), ranging 

from 55 percent of participants in Boston to 86 percent in Anchorage (see Appendix Exhibit A.2).2

Middle school (grades 6–8) students constituted the next largest group, ranging from 14 percent of 

participants in Anchorage to 42 percent of participants in Chicago. Only in Boston did a sizable group of 

high school students (18 percent) participate in SES in 2008–09. 

The distribution of participant grade levels was similar for district and non-district providers within

Anchorage, Chicago and Charlotte-Mecklenburg in 2008–09 (see Appendix Exhibit A.3). In Boston and 

Hillsborough in 2008–09, there were statistically significant differences in the distribution of the grade 

levels of students served by district and non-district providers. This difference was more visible in 

Boston, where district providers primarily served participants in elementary school (60 percent) and in 

high school (29 percent) whereas non-district providers served primarily SES participants in middle 

school (50 percent) and elementary school (48 percent). 

In all five districts, most students participating in SES were of a minority race or 

ethnicity, and participation rates with district and non-district providers 

were similar. 

In 2008–09, within each district, the majority of students participating in SES (87 percent or above) were 

of a minority3 race or ethnicity (see Appendix Exhibits B.5, C.5, D.5, E.5 and F.5). Only in Hillsborough 

was there a statistically significant, but negligible, difference between the percentage of minority 

participants served by the district and non-district providers: 87 percent of students served by 

non-district providers were non-white compared with 84 percent of those served by the district provider 

(see Exhibit 7).

 2. Chicago and Charlotte-Mecklenburg did not provide information on SES participants beyond grade 8.

3. In this evaluation, minority was defined as all students other than white.
Eligibility for and Participation in SES 24 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 7

Demographic Characteristics of Students Participating in SES in 2008–09,

by School District and by Provider Type 

Percentage Minority Percentage With Limited 

English Proficiency Percentage With Disabilities

District 

Provider

Non-District 

Providers

District 

Provider

Non-District 

Providers

District 

Provider

Non-District 

Providers

Anchorage 92 84 26 26 14* 22*

Boston 92 91 38* 22* 24 26

Charlotte-Mecklenburg 98 98 30* 25* 14 14

Chicago 99 98 20 16 14 14

Hillsborough 84* 87* 35 32 29 28

Exhibit reads: In Anchorage in 2008–09, 92 percent of students who participated with the district 

provider were of a minority race or ethnicity, and 84 percent of students who participated with the 

non-district providers were of a minority race or ethnicity. This difference was not statistically significant.

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage, Boston and Hillsborough; grades K–8 in 

Charlotte-Mecklenburg; and grades 3–8 in Chicago. 

Minority includes African American, Hispanic, Asian, American Indian, Alaska Natives and multiracial 

students.

* Indicates a statistically significant difference between the percentage of district SES participants and the percentage 

of non-district SES participants in the indicated demographic group at the 0.05 level.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and

Hillsborough school districts, 2008–09.

District providers served students with limited English proficiency at similar or 

higher rates than did non-district providers. Students with limited English 

proficiency participated in SES at rates similar to or higher than those of other 

eligible students. 

In all five districts in 2008–09, one-third or fewer of the SES participants were students with limited 

English proficiency, ranging from 16 percent in Chicago to 33 percent in Hillsborough (see Appendix 

Exhibits B.5, C.5, D.5, E.5 and F.5). In 2008–09, in Boston and Charlotte-Mecklenburg, the percentage 

of students with limited English proficiency served by the district provider was statistically significantly 

higher than the percentage served by non-district providers (percentage differences were 16 and 

5 percent, respectively) (see Exhibit 7). In Anchorage, Chicago and Hillsborough in 2008–09, SES 

students with limited English proficiency participated at similar rates with district and non-district 

providers. 

With the exception of Charlotte-Mecklenburg, in all other districts in 2008–09, students with limited 

English proficiency were represented at statistically significantly higher rates among SES participants 

than among all eligible students. This difference was most notable in Anchorage, where 26 percent of 

participants in 2008–09 had limited English proficiency, compared with 14 percent in the eligible student 

population (see Appendix Exhibit B.5). 
Eligibility for and Participation in SES 25 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

In four of the five districts, there were no statistically significant differences by 

provider type in the percentage of students with disabilities served in 2008–09. 

Students with disabilities participated in SES at rates similar to or higher than those 

of eligible students. 

In 2008–09, the percentage of SES participants with disabilities ranged between 14 percent in both 

Chicago and Charlotte-Mecklenburg and 28 percent in Hillsborough (see Appendix Exhibits B.5, C.5, 

D.5, E.5 and F.5). Participation rates for students with disabilities were similar for district and 

non-district providers in Boston, Charlotte-Mecklenburg, Chicago and Hillsborough in 2008–09 (see 

Exhibit 7). In Anchorage, however, non-district providers served a statistically significantly higher 

percentage of students with disabilities (22 percent) than did the district provider (14 percent). In 

two districts—Chicago and Hillsborough—students with disabilities were represented at statistically 

significantly higher rates among SES participants than among all eligible students in 2008–09 (see 

Appendix Exhibits E.5 and F.5).

In all five districts in 2008–09, students participating in SES had statistically 

significantly lower levels of prior achievement in mathematics and reading than 

eligible nonparticipants.

The SES program is intended to serve students from low-income families who attend Title I schools that 

do not make AYP for three or more consecutive years. In each of the five districts examined, students 

eligible for SES in 2008–09 had statistically significant lower levels of 2007–08 mathematics and reading 

achievement than those of students not eligible for SES (see Appendix Exhibit A.4).4 Of these eligible 

students, SES participants had statistically significant lower levels of prior achievement in mathematics 

and reading than those of nonparticipants in all five districts (see Appendix Exhibit A.5). These findings 

are consistent with those in other studies. For example, a U.S. Department of Education study found 

that students participating in SES had statistically significant lower average achievement before the 

participation year than did eligible students who did not participate.5

In Boston and Chicago, district providers served students with lower average prior 

achievement than that of students served by non-district providers. In Anchorage, 

Charlotte-Mecklenburg and Hillsborough, district providers and non-district 

providers served students with similar prior achievement levels. 

In Boston and Chicago, district and non-district providers differed in the prior achievement levels of 

students they served. In Boston, students served by the district provider in 2008–09 had statistically

significantly lower achievement in 2007–08 than students served by non-district providers. In 

mathematics, that difference was 0.31 standard deviations, and in reading, that difference was 

0.39 standard deviations (see Exhibit 8). In Chicago, the 2007–08 achievement level of students served 

by the district provider in 2008–09 was statistically significantly lower in reading than the achievement of 

students served by non-district providers, but not in mathematics. In reading, the difference was 

0.07 standard deviations.

 4. As mentioned earlier, scale scores were standardized within each grade, year and district (to a mean of zero and a 

standard deviation of 1.0) to make comparisons across grades and years more meaningful.

5. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007).
Eligibility for and Participation in SES 26 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

In Anchorage, Charlotte-Mecklenburg and Hillsborough, average prior achievement levels did not show

statistically significant differences between students served by district providers and students served by 

non-district providers (see Exhibit 8). For example, in Charlotte-Mecklenburg, students served by the 

district provider in 2008–09 scored 0.86 standard deviations below the overall district mean on the prior 

year’s mathematics assessment while students served by non-district providers scored 0.82 standard 

deviations below the district mean (see Exhibit 8). This difference was not statistically significant.

Exhibit 8

Average Prior (2007–08) Achievement Scores of Students Participating

in SES in 2008–09 With District and Non-District Providers,

and the Difference Between Provider Types, by School District and by Subject

Mathematics Reading

District 

Provider

Non-District 

Providers Difference District 

Provider

Non-District 

Providers Difference

Anchorage –0.90 –0.98 0.08 –0.80 –1.02 0.22

Boston –0.58 –0.27 –0.31* –0.64 –0.25 –0.39*

Charlotte-Mecklenburg –0.86 –0.82 –0.04 –0.87 –0.83 –0.04

Chicago –0.37 –0.33 –0.04 –0.39 –0.32 –0.07*

Hillsborough –0.82 –0.80 –0.02 –0.87 –0.83 –0.04

Exhibit reads: In Anchorage, among students who participated in SES with the district provider in 

2008–09, the average mathematics achievement score in 2007–08 was 0.90 standard deviations below the 

district average, while students who participated in SES with the non-district providers had an average prior 

mathematics achievement score that was 0.98 standard deviations below the district average. Among students 

who participated in SES with the district provider in 2008–09, the average mathematics achievement score in 

2007–08 was 0.08 standard deviations above the average score of students who participated with non-district 

providers. This difference was not statistically significant.

Notes: SES = Supplemental Educational Services.

Negative values in the difference columns occur when the district provider served students with lower prior 

achievement than those served by the non-district providers. 

Achievement scores are standardized within year, grade level and district.

The grades included vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg; and grades 

3–10 in Anchorage and Hillsborough.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2007–08 and 2008–09.

Summary

Overall, all five districts that received SES waivers enrolled more students in SES in 2008–09 than in 

each district’s pre-waiver year, although the participation rates varied. Non-district providers did not 

appear to be adversely affected by the presence of the district as an SES provider; the number of 

non-district providers continued to grow in all five districts. In addition, both district and non-district 

providers served similar student populations in terms of their grade, race/ethnicity, special needs and 

prior academic achievement. There are two key exceptions: in Boston and Chicago, district providers 

served students with lower prior achievement than students served by non-district providers. 
Student Achievement 27 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

III. Student Achievement

This chapter focuses on the relationship between student academic achievement, as measured by 

annual assessments in mathematics and reading, and participation in SES. All five districts provided 

achievement data, but for different grades, different assessments and different academic years (see 

Exhibit 1 for a description of the data analyzed in this study). Despite the differences, all districts were 

analyzed with the same analytic approaches. These approaches are summarized in Chapter 1 and in the 

relevant sections that follow. 

SES Participation and Student Achievement Gains

The following sections describe the association between SES participation and student achievement 

gains. The analyses of student achievement in this study followed the methods used in a previous 

analysis of SES and student achievement in nine large urban school districts that was conducted as part 

of the National Longitudinal Study of No Child Left Behind.1 Using a student fixed-effects approach, the 

current study examined students’ achievement gains associated with SES participation relative to 

nonparticipation. The analyses examined mathematics achievement gains for students who participated 

in mathematics SES and reading achievement gains for students who participated in reading SES.2 All 

models include an indicator for student eligibility for services and an indicator for participation, plus 

grade, year and grade-by-year terms. These models also included student fixed effects, which controlled 

for time-invariant student characteristics such as parent education and underlying student ability or 

motivation (see Chapter 1 for details). The sample sizes for the analyses in the current chapter are 

 1. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental 

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007).

2. Students may have received SES in both mathematics and reading, as well as in other subjects. 

Key Findings

—Students in three of the five districts demonstrated statistically significantly larger 

mathematics achievement gains during periods of SES participation than during periods 

of nonparticipation. In addition, in two districts, SES participation was associated with 

statistically significant reading gains. Averaged across the five districts, the overall 

association between SES participation and achievement gains was statistically significant 

in both mathematics and reading, relative to nonparticipation.

—Across the five districts, SES participation was associated with statistically significant 

mathematics achievement gains, relative to nonparticipation, for students served by 

district providers and also for students served by non-district providers. For students 

served by district providers, but not for students served by non-district providers, SES 

participation was associated with statistically significant reading achievement gains.

—Across the five districts, the achievement gains associated with SES participation relative 

to nonparticipation did not differ for district and non-district providers for either 

mathematics or reading. When examined within each of the five districts individually, 

one district, Chicago, showed statistically significantly larger reading gains for students 

served by non-district providers than for students served by the district provider. 
Student Achievement 28 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

presented in the individual district appendixes (Appendices B–F). Details on the calculation of average 

achievement gains across the five districts are provided in Appendix H.

The achievement gains associated with SES participation presented in the subsequent exhibits represent 

the differences in students’ annual achievement gains (where achievement is scaled in standardized 

z-scores) between periods of participation and periods with no participation. In other words, the results

indicate whether students experienced larger or smaller achievement gains during periods of participation 

compared with periods of nonparticipation. Because statistical significance depends on both sample size 

and the size of the effect being estimated, the likelihood of finding statistically significant gains was lower 

for Anchorage, a small district, than for Chicago, a large district, all else being equal. 

For students in both Chicago and Hillsborough, SES participation was associated 

with statistically significantly larger achievement gains in both mathematics and 

reading during periods of SES participation, relative to periods of nonparticipation. 

In Charlotte-Mecklenburg, SES participation was associated with a statistically 

significant achievement gain in mathematics. Across the five districts, SES 

participation was associated with statistically significant achievement gains in both 

mathematics and reading.

For students in both Chicago and Hillsborough, SES participation was associated with statistically 

significantly larger achievement gains in both subject areas during periods of SES participation than they 

did during periods of nonparticipation. In Chicago, SES participation was associated with a statistically 

significant increase in overall achievement gains of 0.05 standard deviations in mathematics and 

0.07 standard deviations in reading (see Exhibit 9). Both the gains associated with a single year of SES 

participation and the gains associated with of multiple years of SES participation were statistically 

significant for both mathematics and reading (see Appendix Exhibit A.6). 

In Hillsborough, SES participation was associated with statistically significantly larger achievement gains 

overall of 0.12 standard deviations for mathematics and 0.06 standard deviations for reading, relative to 

nonparticipation (see Exhibit 9). For students with one year of SES participation, the gains associated 

with participation were statistically significant for both mathematics and reading. For students with two 

or more years of SES participation, the gains associated with participation were not statistically 

significant, perhaps in part due to the small number of multiple-year participants in Hillsborough (see 

Appendix Exhibits A.6 and F.12).

In Charlotte-Mecklenburg, SES participation was associated with a statistically significant increase in 

overall achievement gains for mathematics (0.11 standard deviations) but not for reading (0.03 standard 

deviations), relative to nonparticipation (see Exhibit 9). For students with one year of SES participation, 

the gains associated with participation were statistically significant for mathematics, but were not

statistically significant for students with multiple years of SES participation in either subject area (see 

Appendix Exhibit A.6). As was the case in Hillsborough, few students had participated in SES for 

multiple years in Charlotte-Mecklenburg (see Appendix Exhibit D.12).

In Anchorage and Boston, the association between SES participation and student achievement gains was 

not statistically significant for mathematics or reading, relative to nonparticipation (see Exhibit 9).3

 3. Boston achievement analyses should be interpreted with caution. The findings were sensitive to sample changes. 

See Appendix G for details.
Student Achievement 29 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Across the five districts, SES participation was associated with statistically significant achievement gains 

in mathematics (0.08 standard deviations) and in reading (0.04 standard deviations) relative to periods of 

nonparticipation (see Exhibit 9). 

Exhibit 9

Overall Mathematics and Reading Achievement Gains Associated With SES Participation, 

Relative to Nonparticipation

Exhibit reads: In Anchorage, SES participation was associated with a gain of 0.08 standard deviations in 

mathematics achievement, relative to nonparticipation, but this gain was not statistically significant.

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 2008–09; and 

the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

SES Providers and Student Achievement Gains

The central question in this study focuses on how district providers performed compared with 

non-district providers on improving student achievement. Two types of analyses were conducted to 

address this question. The first type examined whether SES participation was associated with statistically 

significant achievement gains relative to nonparticipation, separately for district and non-district 

providers (see Exhibit 10).4 The second type examined whether there were statistically significant 

differences between the achievement gains associated with SES participation for students served by the 

 4. In analyses of provider type (i.e., district compared with non-district), records were removed for students 

receiving services from non-district providers for years before the district started providing services (the pre-waiver 

years). The purpose was to compare district and non-district providers over the same period of time. 
Student Achievement 30 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

district provider and the achievement gains associated with SES participation for students served by 

non-district providers (see Exhibit 11). 

The remainder of this chapter provides results for these two types of analyses. The results for the first 

type, where each provider type is analyzed separately, are followed by the results for the second type,

where the two provider types are compared to each other. Additionally, the findings for the second type 

of analyses include a separate set of analyses that control for the number of hours of tutoring provided 

by the two types of providers. Finally, results for individual non-district providers are discussed. 

In Hillsborough, SES participation was associated with statistically significant gains 

in reading achievement, relative to nonparticipation, for students served by 

non-district providers, but not the district provider; SES participation for students 

served by both types of providers was associated with statistically significant

mathematics gains. In Charlotte-Mecklenburg, only SES participation for students 

served by the district provider was associated with statistically significant 

mathematics gains. SES participation in Chicago was associated with statistically 

significant gains in both subject areas for students served by both district and 

non-district providers. There were no statistically significant gains associated with 

SES participation in Anchorage and Boston for either provider type.

In Hillsborough, SES participation was associated with statistically significant mathematics gains for both 

types of providers (0.11 and 0.13 standard deviations, respectively), relative to nonparticipation (see 

Exhibit 10). However, only for students served by non-district providers was SES participation 

associated with statistically significant reading gains (0.06 standard deviations); students served by the 

district provider did not demonstrate statistically significant gains (0.10 standard deviations). In 

Charlotte-Mecklenburg, SES participation was associated with statistically significant mathematics gains 

for students served by the district provider (0.23 standard deviations), but not for students served by 

non-district providers (0.10 standard deviations). There were no statistically significant gains in reading 

associated with either provider type.

In Chicago, SES participation was associated with statistically significant achievement gains for both 

mathematics and reading, relative to nonparticipation. This finding carried through for both district and 

non-district providers (see Exhibit 10). The mathematics gains associated with SES participation were 

0.06 and 0.04 standard deviations for district and non-district providers, respectively. The reading gains 

associated with SES participation were 0.03 and 0.09 standard deviations for district and non-district 

providers, respectively. 

In Boston and Anchorage, the achievement gains associated with SES participation were not statistically 

significant for either mathematics or reading. There were no statistically significant gains associated with 

SES participation in Anchorage and Boston for either provider type (see Exhibit 10). 
Student Achievement 31 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Across the five districts, SES participation was associated with statistically 

significant mathematics achievement gains, relative to nonparticipation, for 

students served by district providers and also for students served by non-district 

providers. For students served by district providers, but not non-district providers, 

SES participation was associated with statistically significant reading achievement 

gains.

Across the five districts, for district providers, SES participation was associated with statistically 

significant gains in mathematics (0.10 standard deviations) and reading (0.06 standard deviations) relative 

to nonparticipation (see Exhibit 10). For non-district providers, SES participation was associated with a 

statistically significant gain in mathematics (0.09 standard deviations), but not in reading (0.03 standard 

deviations). 
Student Achievement 32 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 10

Mathematics and Reading Achievement Gains Associated With SES Participation,

Relative to Nonparticipation, by Provider Type

Exhibit reads: In Anchorage, for district providers, SES participation was associated with a gain of 

0.05 standard deviations in mathematics achievement, relative to nonparticipation, but this gain was not 

statistically significant. 

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, and 

grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and Boston 

due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.
Student Achievement 33 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Across the five districts, there were no statistically significant differences between

district and non-district providers in the achievement gains associated with SES 

participation, relative to nonparticipation, for either mathematics or reading. 

Separate analyses by district indicated no differences between district and 

non-district providers in four of the five districts. In one district, Chicago, the gains 

for students served by non-district providers were statistically significantly larger 

than the gains for students served by the district provider in reading, but not in 

mathematics. 

On average, the gains associated with SES participation for students served by district providers did not 

differ statistically significantly from the gains associated with participation for students served by 

non-district providers for either mathematics or reading (see Exhibit 11). Within each of the five districts 

individually, there were no statistically significant differences between the two types of providers in the 

mathematics gains or the reading gains associated with SES participation except for in Chicago, where 

there was a statistically significant difference between the two types of providers in reading gains.
Student Achievement 34 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit 11

Differences in Achievement Gains Associated With SES Participation, Relative to 

Nonparticipation, for Students Served by District and Non-District Providers, by Subject

Exhibit reads: In Anchorage, the mathematics achievement gains associated with SES participation for 

students served by the district provider were 0.05 standard deviations lower than the mathematics achievement 

gains associated with participation for students served by non-district providers, but this difference was not 

statistically significant.

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district providers.

The grades included in analyses vary by district: grades 3–8 in Boston, Charlotte-Mecklenburg, and Chicago, and 

grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and Boston 

due to changes in tested grades during the study years.

* Indicates that the difference in the achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

In Chicago, although Exhibit 10 shows that the reading gains associated with participation, relative to 

nonparticipation, were statistically significant for both the district provider (0.03 standard deviations) and 

non-district providers (0.09 standard deviations), these gains were statistically significantly larger for 

non-district providers than for the district provider (a difference of 0.06 standard deviations) (see Exhibit 

11). This finding may be related to the student population served by the district provider: For example, 

the district provider served students with statistically significantly lower levels of prior reading 

achievement in 2008–09 than the students served by the non-district providers, and this circumstance

may have made it more challenging to produce improvements (see Exhibit 8). 

In Hillsborough, although SES participation for students served by non-district providers was associated 

with a statistically significant gain in reading, relative to nonparticipation, and participation for students 

served by the district provider was not (see Exhibit 10), the difference between the gains associated with 

the two types of providers was not statistically significant (see Exhibit 11). Similarly, in Charlotte-
Student Achievement 35 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Mecklenburg, SES participation was associated with statistically significant gains in mathematics for 

students served by the district provider, but not for students served by non-district providers; however, 

the difference in the gains associated with SES participation between the two types of providers was not 

statistically significant. This result in Charlotte-Mecklenburg may reflect the small sample size (only 

415 participants) making it more difficult to detect statistically significant differences in gains between 

provider types (see Appendix Exhibit D.14).

Controlling for the number of hours of SES received did not affect the results for 

these comparisons between district and non-district providers. 

One potential explanation for differences in gains associated with participation in SES offered by district 

versus non-district providers could be the number of hours of SES they provided. However, even after 

controlling for the number of hours of tutoring, the reading achievement gains associated with SES 

participation in Chicago, relative to nonparticipation, remained statistically significantly larger for 

students served by non-district providers than for students served by the district provider (a difference 

of 0.08 standard deviations) (see Appendix Exhibit E.16). In the other four districts, after controlling for 

the number of hours of tutoring received, there were no statistically significant differences between 

district and non-district providers in the achievement gains associated with SES participation.

Achievement gains associated with SES participation, relative to nonparticipation, 

for individual non-district providers varied. Most individual non-district providers 

did not demonstrate statistically significant gains. However, many non-district 

providers may have had too few participants to show statistically significant gains. 

The non-district providers varied in the achievement gains associated with SES participation, relative to 

nonparticipation (see Exhibits 17 and 18 in Appendices B–F).5 For students served by most individual 

non-district providers, SES was not associated with statistically significant gains during periods of SES 

participation relative to periods of nonparticipation. Although there was at least one individual 

non-district provider in each district (with the exception of Anchorage) that demonstrated statistically 

significant student achievement gains in at least one subject area, the gains associated with SES 

participation for most of the individual providers were not statistically significant. Even in districts where 

SES participation across non-district providers was associated with statistically significant increases in 

achievement gains, most individual non-district providers did not demonstrate significant gains. For 

example, in Chicago, the gains for SES participation across all non-district providers on mathematics 

achievement were statistically significant (0.04 standard deviations) (see Exhibit 10); yet SES participation 

for students served by most of the individual non-district providers was not associated with statistically 

significant gains (see Appendix Exhibit E.17).

When gains were examined at the individual provider level, as opposed to aggregating all non-district 

providers, the power to detect statistically significant achievement gains was limited by the number of 

students served. For example (as shown in Appendix Exhibit E.17), Non-District Provider BR in 

Chicago had an estimated gain of 0.13 standard deviations in mathematics. But with the small number of 

observations (95), the estimated gain would need to be at least 0.16 standard deviations to be statistically 

significant. In contrast, Non-District Provider AY had 8,029 observations, and the estimated gain of 

0.05 standard deviations was statistically significant, although it was less than half of Non-District 

Provider BR’s estimated gain. Therefore, in examining achievement gains associated with SES 

participation for an individual provider it is important to consider that, when providers serve fewer 

students, they need larger gains for statistical significance. 

 5. The providers are referenced by randomly assigned letters.
Student Achievement 36 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Summary

In summary, across the five districts, SES participation was associated with statistically significant 

mathematics and reading achievement gains relative to nonparticipation. SES participation was 

associated with similar achievement gains for both district and non-district providers. Only in Chicago 

did reading gains differ statistically significantly between students served by the district provider and 

those served by non-district providers, despite SES participation with each being associated with 

statistically significant gains.
District Communication With Parents 37 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

IV. District Communication With Parents

As a condition of receiving the SES waivers for the pilot, districts were required to improve 

communication with parents by using multiple communication strategies, extending enrollment 

windows, and providing balanced information on all providers. Through these strategies, districts were 

expected to achieve one of the objectives of the waivers: to ensure that more eligible students received 

SES. 

The U.S. Department of Education sent guidance to the districts on the expectations for SES 

implementation and practice. The guidance included recommendations for sending parent notification 

letters written in simple and accurate language, offering either an extended enrollment window or more 

than one enrollment window, broadly circulating information about SES throughout the district, and 

enacting other administrative details. The guidance served as a lens for evaluating the quality of parent 

communication. 

As evidence of meeting the parent communication requirements of the waiver, the five districts 

submitted to the U.S. Department of Education examples of their communication with parents about 

SES eligibility and provider options. For 2008–09, Boston, Chicago, Anchorage, Hillsborough and 

Charlotte-Mecklenburg all submitted documents.1 The contents of the documents varied, but examples 

included letters, flyers, advertisements and handbooks. Each district also reported on their outreach and 

recruitment strategies for 2008–09. 

Analysts coded the provided documents and reported outreach practices for several features: 

1. General readability—After examining the clarity of the writing, as assessed by the Flesch-Kincaid 

score,2 and the availability of documents in other languages, analysts concluded that 

communications provided at the 10th-grade reading level and lower were considered accessible 

to parents.

 1. All districts except Charlotte-Mecklenburg also submitted documents from two to three previous years. 

2. The Flesch-Kincaid test measures word length and sentence length to generate a score, which is translated into a 

U.S. grade-level indicator, representative of the approximate educational level at which the text is written.

Key Findings

—All five districts created information documents for parents that might be difficult for 

some parents to understand. 

—All five districts translated informational materials into at least one language other 

than English.

—All five districts presented balanced information about providers.

—All five districts provided an enrollment window of at least six weeks for parents to 

enroll their children in SES; however, only Chicago and Anchorage reported offering 

more than one enrollment window.

—Districts have demonstrated minor improvements in communication with parents 

over time.
District Communication With Parents 38 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

2. Balanced information—Analysts examined promotional documents to see whether district 

providers are given extra attention or whether the documents showed bias, which was 

determined to exist if the district was either highlighted over other providers or advertized as 

better than other providers. 

3. Recruitment activities—Analysts examined whether districts used multiple outreach strategies to 

distribute information about SES to parents and whether district materials included key details 

such as contact information and enrollment dates. 

4. Enrollment opportunities—Analysts examined the length and number of enrollment opportunities 

available to parents. 

Four districts provided documentation from earlier years. Boston and Chicago provided documents 

from 2005–06 to 2007–08; Anchorage and Hillsborough provided documents from 2006–07 and 

2007–08. These documents varied by district and by year, but all were coded for readability, balance and 

recruitment activities. Changes over time are included in the final section of this chapter. 

General Readability

All five districts created information documents that may be difficult for some 

parents to understand. 

The documents, such as letters, handbooks and flyers, that the districts distributed to parents 

demonstrated an effort to communicate necessary information about SES to parents. However, the 

documents contained language that might be difficult for some parents to understand. For example, a 

three-page letter to parents in Charlotte-Mecklenburg included the following:

According to 2007–08 assessment results in math, [X] school is in the corrective action 

phase of Title I School Improvement for the 2008–09 school year under the federal No 

Child Left Behind (NCLB) law. Schools enter this phase after not making Adequate 

Yearly Progress for four years. ...

This letter scored 12.5 on the Flesch-Kincaid readability test, indicating that the letter was written at 

higher than the 12th-grade level. In a similar manner, a difficult-to-read letter from Chicago stated, “The 

No Child Left Behind Act provides selected students with the opportunity to receive additional tutorial 

assistance known as Supplemental Educational Services.” Although this letter did use the phrase “free 

tutoring,” a clear description of what the district is offering, it scored a 13.6 on the Flesch-Kincaid 

readability test, suggesting that a parent who had not attended college would find it challenging to 

understand. 

All five districts used jargon in most of their documents. Jargon includes technical terms from the law 

that are not often used in daily conversation, for example, “Adequate Yearly Progress” or “schools 

identified for corrective action.” Despite the use of jargon, districts, except for Charlotte-Mecklenburg, 

did use the clear phrase “free tutoring” in many of their documents. As an example, a flyer that Boston 

distributed to parents asked, “Does your child need extra help in school?” and showed “free tutoring” in 

a visible bubble. Chicago distributed a flyer that showed “Free Tutoring!” in large letters across the top, 

provided registration beginning and ending dates, and encouraged readers to contact their school for 

more information. A flyer from Anchorage announcing a Family Learning Fair began with, “Did you 

know your child could get FREE tutoring?” This flyer also included eligibility requirements. 
District Communication With Parents 39 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

All five districts translated informational materials into at least one language other 

than English.

Each district translated documents into the languages most commonly spoken by parents. CharlotteMecklenburg sent parents letters and information booklets translated into Spanish and Vietnamese. 

Boston translated documents into seven languages. For example, an informational booklet and a 

question and answer sheet were translated into Spanish, Somali, Portuguese, Haitian Creole, Chinese, 

Cape Verdean and Vietnamese. Anchorage translated an SES informational packet into Hmong, Samoan 

and Spanish. In Chicago, letters to parents, a flyer and the parent handbooks were all translated into 

Spanish. In a similar manner, in Hillsborough, all submitted documents were translated into Spanish, and 

interpreters were made available for parents who spoke other languages. 

Balanced Information

In 2008–09, all five districts presented balanced information about provider options.

Districts receiving the waiver needed to demonstrate that they provided eligible families with balanced 

information on all providers. All districts presented the information about the various provider options 

without bias. All the Anchorage documents listed the district provider along with the non-district 

providers and did not place emphasis on the fact that the district was an option. The sample 

documentation provided by Charlotte-Mecklenburg noted that students had access to “several

providers” but did not list any by name. Hillsborough documents either did not mention that the district 

was a provider or simply listed all providers in alphabetical order. In the documents Boston shared, all 

spoke about the SES program in general, and none mentioned the district specifically as a provider. 

Chicago presented balanced information on all providers, although its SES program often appeared first 

on alphabetical lists of providers because it was called “AIM High.”

Recruitment Activities

In addition to sending letters, districts reported using multiple communication 

strategies to reach eligible families, the most common of which was to advertise in 

newspapers, as well as on radio and television. 

Districts could meet the waiver requirements by providing information to parents in letters and through 

at least two other means. For example, they could post flyers in locations parents were likely to visit such 

as libraries, community centers, health centers and religious organizations. Other communication 

strategies might include door-to-door outreach campaigns, an established multilingual “SES hotline” for 

parents, public service announcements related to SES, open houses and back-to-school fairs at schools 

required to offer SES. 

All five districts mailed SES informational letters to parents. In addition, all the districts reported at least 

two additional strategies to inform parents about SES. For example, all five districts posted information 

in newsletters and parent association publications. 

One common strategy for parent outreach was advertising in newspapers and other print media, and all 

five districts reported doing so. Anchorage placed two full pages of advertisements covering AYP results 

and SES information in the Anchorage Daily News. Boston secured advertising space for four consecutive 

weeks in the Boston Metro newspaper. Charlotte-Mecklenburg advertized in a local Spanish newspaper. 
District Communication With Parents 40 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Chicago purchased an advertisement in five local newspapers for five days in early September, and 

Hillsborough advertised events such as provider fairs in five newspapers. 

Because not all parents read the local newspaper, all five districts also provided information about SES 

by means of radio, television or telephone. For example, Boston used its local access cable network to 

broadcast a program on SES, and Charlotte-Mecklenburg placed an advertisement on a local Hispanic 

radio station and a special interest spot on a local television channel. Anchorage, Boston and 

Hillsborough used automated telephone call technology to leave phone messages at home for parents of 

students eligible for SES. 

Four of the five districts reported holding provider fairs or open houses. These were opportunities for 

parents to learn about the different SES providers offering services and to select one they thought would 

best meet their child’s learning needs. Anchorage reported holding one provider fair, and Boston hosted 

three. Charlotte-Mecklenburg held four regional provider fairs with language interpreters available. 

Chicago offered SES information at each school’s Open House. Hillsborough reported seven provider 

fairs throughout the county.

Finally, three districts reported communicating with parents through community organizations. 

Anchorage worked with the Alaska Parent Information Resource Center and other community- and 

faith-based organizations to provide SES information to parents. All six parent resource centers of the 

Boston Public Schools had copies of the enrollment workbook and applications in the seven major 

languages represented in the schools (languages listed above). Charlotte-Mecklenburg publicized SES 

through a grassroots community organization.

Three districts used techniques not used in the other districts. Charlotte-Mecklenburg established an 

information hotline with a Spanish script available. This district also advertized in schools and 

community sites such as libraries, local grocery stores and social services offices. Hillsborough provided 

literature on SES to seven family support and resource centers, three after-hours pediatric locations, city 

and county parks, YMCA locations, libraries, Boys and Girls Clubs and the Hispanic Services Council. 

Chicago distributed 1,000 SES informational flyers at a parade.

Enrollment Opportunities

All five districts provided an enrollment window of at least six weeks for parents to 

enroll their children in SES; however, only Chicago and Anchorage reported offering 

more than one enrollment window.

Another stipulation of the waiver was that districts should offer an extended enrollment period or more 

than one enrollment window to give eligible students more opportunity to sign up for and participate in 

SES. Although the department did not define “extended,” all five districts provided an enrollment 

window of at least six weeks. Anchorage reported that it began sending letters in August for what 

appeared to be a November enrollment deadline, indicating an enrollment window of more than 

six weeks. Hillsborough reported sending letters to parents in April, giving parents advance notice about 

SES, and then sent letters again before school started in August.3 Charlotte-Mecklenburg reported an 

enrollment period of August 19, 2008, through September 26, 2008. Chicago reported having an SES 

information dissemination and enrollment period of approximately six weeks during August and 

September. In Boston, an informational booklet and questions and answers sheet dated September 24

 3. Although all slots were filled by September, Hillsborough reported they continued to accept application forms for 

a waiting list until December. In December, additional students were placed with SES providers as a result of attrition.
District Communication With Parents 41 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

were distributed to parents, with an enrollment deadline of November 9, providing evidence of a 

six-week window for SES registration. 

Two districts—Chicago and Anchorage—provided evidence of an additional enrollment window. 

Charlotte-Mecklenburg planned for a second enrollment window, but canceled it because the district had 

spent all allocated funds during the first round. Hillsborough and Boston did not provide evidence of 

additional enrollment windows. 

Changes in District Communication Over Time

Districts have demonstrated minor improvements in communication with parents 

over time.

Of the four districts with multiple post-waiver years, Hillsborough was the only district to demonstrate 

improvements in the readability of their communications over time. In 2006–07, the submitted 

documents all required at least a 12th-grade reading level. By 2008–09, the documents were written at a 

reading level of 9th grade or lower. In the documentation provided by Anchorage across three academic 

years, none of the documents required a reading level higher than 10th grade. The level of readability in 

district communication with parents has changed little over time in the two districts that have had the 

waiver the longest—Boston and Chicago. Documentation between 2005–06 and 2008–09 consistently 

required at least a 10th-grade reading level, as measured by the Fleisch-Kincaid scores, and some 

documents continued to score at the 12th-grade or higher reading level. All four districts consistently 

translated documentation into other languages and mentioned “free tutoring” in at least some 

communications. 

By 2008–09, all districts produced balanced materials, including Boston and Chicago, which had not 

done so in the past. In 2005–06, Boston advised parents to “choose wisely” and went on to list why the 

district was a better SES provider than other providers. It produced similar messages in 2006–07 and 

2007–08, when it highlighted Boston in the provider list, mentioned that only the district provided 

services at the school (requiring no additional travel or transportation), and listed the number of hours of 

tutoring a student would receive from the district (which was greater than the number of hours provided 

by non-district providers). The documents Boston submitted for 2008–09 were balanced.4 Chicago 

documentation was also biased in 2005–06, mentioning transportation and computer access issues for 

the non-district providers, but Chicago then offered balanced information in subsequent years. Materials 

from Anchorage and Hillsborough did not demonstrate a bias in any year. 

All four districts used multiple communication strategies for parent outreach and student recruitment in 

each year. Before 2008–09, districts did not provide information on enrollment opportunities 

consistently. As a result, no analysis over time can be presented. 

Summary

In an effort to increase the number of eligible students receiving SES, districts used multiple strategies to 

improve communication with parents. Districts communicated with parents in a variety of ways, 

including letters, flyers, fairs and advertisements in news media. Furthermore, they translated 

information into common languages spoken in their communities. When communicating about provider 

 4. Boston’s waiver approval letter for 2008-–09 included the condition that Boston “[eliminate] apparent bias in its 

parent outreach materials so that the materials do not favor the LEA’s program over private providers’ programs.” See 

the approval letter at http://www2.ed.gov/nclb/choice/help/ses/08-0804boston.pdf. 
District Communication With Parents 42 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

options, all districts presented balanced information on all providers and did not show bias favoring the 

district provider. All five districts also provided families with enrollment windows of at least six weeks, 

thereby increasing the opportunity for parents to learn about SES, select an appropriate provider and 

enroll their children.
Summary 43 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

V. Summary

This examination of the implementation of SES and the achievement gains of SES participants in the 

five pilot waiver districts (Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough) 

resulted in eight key findings, which are summarized below. 

In the three districts that did not serve as SES providers before the waiver, SES 

participation rates increased in the first post-waiver year.

In Anchorage, Charlotte-Mecklenburg and Hillsborough, both the number of students eligible for SES 

and SES participation rates rose during the first year of the waiver. In Boston and Chicago, where the 

districts had provided SES before the waiver, SES participation rates dropped during the first 

post-waiver year. By 2008–09, all five districts enrolled more students in SES than in the pre-waiver year.

In all five districts, the number of non-district SES providers increased from the first 

post-waiver year through the 2008–09 school year.

In all five districts, the number of non-district providers increased after the districts received the waivers. 

Most of the non-district providers were small, however, serving fewer than 100 students.

In the three districts that began to serve as SES providers after the waiver was 

granted, non-district providers served the majority of SES participants in the first 

year of the waiver.

During the year that the Anchorage, Charlotte-Mecklenburg and Hillsborough school districts started 

serving as SES providers, the non-district providers in those districts continued to serve the majority of 

the SES participants. In the four districts with subsequent SES years after the waiver, the distribution 

varied. For several years, in Anchorage and Boston, the district provider served more SES participants 

than did the non-district providers. In Chicago and Hillsborough, the non-district providers consistently 

served more SES participants than did the district provider. 

There were few demographic or academic differences between students served by 

district providers and students served by non-district providers.

In all five districts, participation rates by students from racial and ethnic minorities were similar for 

district and non-district providers. District providers served students with limited English proficiency at 

similar rates to or higher rates than non-district providers. In four of the five districts, there were no 

differences by provider type in the participation rate of students with disabilities. In Anchorage, 

however, the non-district providers served a higher percentage of students with disabilities than the 

district provider did. In Boston and Chicago, district providers served students with lower prior 

achievement than that of students served by non-district providers.

Averaged across the five districts, the overall association between SES participation 

and achievement gains was statistically significant in both mathematics and 

reading, relative to nonparticipation.

Averaged across the five districts, the overall gain associated with SES participation, relative to 

nonparticipation, was statistically significant in both mathematics and reading. In Chicago and 

Hillsborough, SES participation was associated with a statistically significant increase in achievement 
Summary 44 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

gains in both mathematics and reading. In Charlotte-Mecklenburg, SES participation was associated with 

statistically significant gains in mathematics. In Anchorage and Boston, there were no statistically 

significant gains associated with SES participation in mathematics or reading. 

Averaged across five districts, the achievement gains associated with SES 

participation relative to nonparticipation did not differ for district and non-district 

providers for either mathematics or reading. When examined within each of the five 

districts individually, one district, Chicago, showed statistically significantly larger 

gains for students served by non-district providers than for students served by the 

district provider in reading, but not in mathematics. 

Across the five districts, the gains associated with SES participation for students served by district 

providers did not differ statistically significantly from the gains associated with participation for students 

served by non-district providers. There were no statistically significant differences in mathematics gains 

between the students served by the two types of providers in any of the five individual districts. The only 

statistically significant difference between students served by the two types of providers in reading gains 

was in Chicago. There, although the reading gains associated with SES participation, relative to 

nonparticipation, were statistically significant for both the district and non-district providers, these gains 

were statistically significantly larger for non-district providers. Controlling for the number of hours of 

SES students received did not change these findings.

All five districts reported using multiple communication strategies to reach eligible 

families, provided balanced information about SES providers, translated information 

into at least one language other than English and provided enrollment periods of at 

least six weeks.

To improve SES participation rates, all five districts used a variety of strategies to communicate with 

parents; these strategies included letters, flyers, fairs and advertisements in news media. Moreover, the 

documentation did not appear to favor district over non-district providers. However, the written 

communication often included jargon and complex language, possibly making it difficult for some 

parents to understand. 

Additional research and expanded data collection are needed for better 

understanding of the practices that underlie successful SES programs.

To fully identify the features and practices that characterize successful SES programs and student 

achievement, we will need more data and research. Additional analyses of the data on programmatic 

elements such as the frequency of sessions, characteristics of the instructional staff, and the curricula and 

instruction the staff members provide could demonstrate the characteristics associated with more 

successful programs. In combination with more in-depth research on the expenditures associated with 

SES programs, the findings could provide useful information for policymakers.
Bibliography 45 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Bibliography

Barnhart, M. K. The Impact of Participation in Supplemental Educational Services (SES) on Student Achievement: 

2007–08. Los Angeles: Los Angeles Unified School District Research & Planning, 2009. 

Bloom, H. S. “Using ‘Short’ Interrupted Time-Series Analysis to Measure the Impacts of Whole-School 

Reforms: With Applications to a Study of Accelerated Schools.” Evaluation Review 27 (2003):

3–49.

Chicago Public Schools, The 2008 Supplemental Educational Services Program: Year 5 Summative Evaluation.

Chicago: Chicago Public Schools Office of Extended Learning Opportunities, 2009.

Elementary and Secondary Education Act of 1965, Public Law 89-313, 20 U.S.C. 2701 et seq.

Heinrich, C. J., Meyer, R., and Whitten, G. “Supplemental Education Services Under No Child Left 

Behind: Who Signs Up, and What Do They Gain?” Evaluation and Policy Analysis 32, no. 2 (2010): 

273–98.

Schochet, P. Z. “Statistical Power for Random Assignment Evaluations of Education Programs. Journal 

of Educational and Behavioral Statistics 33, no. 1) (2008): 62–87.

U.S. Department of Education. Non-Regulatory Guidance on Title I, Part A Waivers. Washington, D.C.: U.S. 

Department of Education, July 2009.

U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and 

Program Studies Service. State and Local Implementation of the No Child Left Behind Act: Volume 

I—Title I School Choice, Supplemental Educational Services, and Student Achievement. Washington, D.C.: 

U.S. Department of Education, 2007. 

U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and 

Program Studies Service. State and Local Implementation of the No Child Left Behind Act, Volume 

VII—Title I School Choice and Supplemental Educational Services: Final Report. Washington, D.C.: U.S.

Department of Education, 2009.

Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX A:

ADDITIONAL EXHIBITS

Appendix A A–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix A: Additional Exhibits

Exhibit A.1

Number of Non-District SES Providers, by School District and 

by Number of Students Served, Post-Waiver Year and Most Recent Year

Notes: SES = Supplemental Educational Services.

The grades included vary by district: grades K–12 in Anchorage, Boston and Hillsborough; grades K–8 in 

Charlotte-Mecklenburg; and grades 3–8 in Chicago. 

For Charlotte-Mecklenburg, the first year after the waiver, 2008–09, is also the most recent year of data. 

Source: Administrative data provided by the Anchorage and Hillsborough school districts, 2006–07 and 2008–09; the 

Boston and Chicago school districts, 2005–06 and 2008–09; and the Charlotte-Mecklenburg school district, 2008–09.
Appendix A A–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit A.2

Distribution of Students Participating in SES, by School District and

by Grade Level, 2008–09

Notes: SES = Supplemental Educational Services.

Elementary includes grades K–5, Middle includes grades 6–8, and High includes grades 9–12.

The grades included vary by district: grades K–12 in Anchorage, Boston and Hillsborough; grades K–8 in 

Charlotte-Mecklenburg; and grades 3–8 in Chicago.

Percentages may not total 100 due to rounding.

a In Anchorage, less than 1 percent of SES participants were in high school grades.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2008–09.
Appendix A A–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit A.3

Percentage of Students Participating in SES, by School District,

by Provider Type and by Grade Level, 2008–09

Notes: SES = Supplemental Educational Services.

Elementary includes grades K–5, Middle includes grades 6–8, and High includes grades 9–12.

Percentages may not total 100 due to rounding.

The grades included vary by district: grades K–12 in Anchorage, Boston and Hillsborough; grades K–8 in 

Charlotte-Mecklenburg; and grades 3–8 in Chicago.

* Indicates a statistically significant difference between the district and non-district providers at the 0.05 level.

a In Anchorage, less than 1 percent of SES participants with district providers were in high school grades.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2008–09.
Appendix A A–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit A.4

Average Prior (2007–08) Achievement Scores, by School District,

by 2008–09 Eligibility and by Subject

Average 2007–08 Mathematics

Achievement Scores

Average 2007–08 Reading

Achievement Scores

Eligible Students

(2008–09)

Ineligible 

Students

(2008–09)

Eligible Students

(2008–09)

Ineligible 

Students

(2008–09)

Anchorage –0.56* 0.04* –0.59* 0.04*

Boston –0.26* 0.42* –0.26* 0.40*

Charlotte-Mecklenburg –0.60* 0.11* –0.61* 0.11*

Chicago –0.10* 0.42* –0.11* 0.43*

Hillsborough –0.40* 0.13* –0.44* 0.13*

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The grades included vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, and grades 

3–10 in Anchorage and Hillsborough.

* Indicates a statistically significant difference between eligible and ineligible students at the 0.05 level.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2007–08 and 2008–09.

Exhibit A.5

Average Prior (2007–08) Achievement Scores, by School District,

by 2008–09 Participation Status and by Subject

Average 2007–08 Mathematics

Achievement Scores

Average 2007–08 Reading

Achievement Scores

Participants

(2008–09)

Eligible 

Nonparticipants

(2008–09)

Participants

(2008–09)

Eligible 

Nonparticipants

(2008–09)

Anchorage –0.94* –0.47* –0.91* –0.52*

Boston –0.42* –0.25* –0.43* –0.25*

Charlotte-Mecklenburg –0.83* –0.55* –0.84* –0.55*

Chicago –0.33* –0.03* –0.33* –0.04*

Hillsborough –0.80* –0.35* –0.84* –0.38*

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The grades included vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, and grades 

3–10 in Anchorage and Hillsborough.

* Indicates a statistically significant difference between participants and eligible nonparticipants at the 0.05 level.

Source: Administrative data provided by the Anchorage, Boston, Charlotte-Mecklenburg, Chicago and Hillsborough 

school districts, 2007–08 and 2008–09.
Appendix A A–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit A.6

Mathematics and Reading Achievement Gains Associated With Single-Year or 

Multiple-Year SES Participation

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg, 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 2008–09; and 

the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX B:

SUPPLEMENTAL EXHIBITS

FOR ANCHORAGE

Appendix B B–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix B: Supplemental Exhibits

for Anchorage

Exhibit B.1

Percentage and Number of Students Eligible for SES,

by Grade, in Anchorage, 2004–05 Through 2008–09

Grade

Students Eligible for SES

2004–05

(n = 52,049)

2005–06

(n = 52,338)

2006–07

(n = 51,674)

2007–08

(n = 51,163)

2008–09

(n = 51,739)

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 1 47 3 138 11 458 14 568 15 612

1 2 59 4 146 11 428 13 502 16 619

2 1 50 4 142 12 463 13 503 14 557

3 1 52 4 145 10 377 13 525 15 590

4 2 72 4 136 11 415 12 460 15 610

5 2 66 4 156 10 372 12 448 15 569

6 2 82 4 162 10 368 11 403 13 509

7 1 28 10 424 13 508 2 71 1 45

8 1 30 10 409 13 511 2 85 1 43

9 1 44 1 34 3 111 3 118 2 84

10 1 39 1 45 3 113 3 117 2 86

11 1 32 1 27 3 117 3 110 2 75

12 3 108 3 125 5 186 5 200 5 218

Total 1 709 4 2,089 9 4,427 8 4,110 9 4,617

Note: SES = Supplemental Educational Services.

Source: Anchorage School District administrative data, 2004–05 through 2008–09.
Appendix B B–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.2

Percentage and Number of Eligible Students Participating in SES,

by Grade, in Anchorage, 2004–05 Through 2008–09

Grade

Eligible Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 2 1 4 6 5 25 8 48 11 69

1 8 5 7 10 4 18 15 75 14 88

2 6 3 8 12 6 27 16 80 17 94

3 13 7 5 7 12 45 18 93 17 99

4 4 3 8 11 8 35 14 63 18 109

5 6 4 7 11 12 44 23 101 16 92

6 10 8 7 11 7 24 18 73 17 87

7 0 0 4 15 10 53 0 0 4 2

8 0 0 5 21 6 31 4 3 2 1

9 0 0 0 0 3 3 1 1 2 2

10 0 0 0 0 1 1 0 0 0 0

11 0 0 0 0 2 2 1 1 1 1

12 0 0 0 0 0 0 1 2 0 0

Total 4 31 5 104 7 308 13 540 14 644

Note: SES = Supplemental Educational Services.

Source: Anchorage School District administrative data, 2004–05 through 2008–09.
Appendix B B–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.3

Percentage and Number of Students in Grades K–12 Participating in SES,

by Provider, in Anchorage, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

District provider — — — — 45 138 64 347 54 349

Non-district provider A — — — — 42 128 24 130 23 148

Non-district provider AR — — — — — — 1 5 — —

Non-district provider B — — 29 30 6 19 3 18 5 34

Non-district provider BU — — — — — — — — <1 2

Non-district provider C — — — — 5 16 — — — —

Non-district provider CL — — — — — — 6 35 6 37

Non-district provider CO — — — — — — — — <1 1

Non-district provider D — — 10 10 2 7 1 5 1 8

Non-district provider E 52 16 59 61 — — — — — —

Non-district provider F 48 15 2 2 — — — — — —

Non-district provider G — — 1 1 — — — — — —

Non-district provider L — — — — — — — — 10 65

Total 100 31 100 104 100 308 100 540 100 644

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Providers are referenced by randomly assigned letters.

— Indicates that the provider did not serve SES participants. 

Source: Anchorage School District administrative data, 2004–05 through 2008–09.
Appendix B B–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.4

Number and Percentage of Students Participating in SES, by Provider Type

and by Grade, in Anchorage, 2006–07 Through 2008–09

Grade

2006–07 2007–08 2008–09

Total 

Number of 

Students

Participating

in SES

Provider Type Total 

Number of 

Students

Participating

in SES

Provider Type Total 

Number of 

Students

Participating

in SES

Provider Type

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

K 25 56 44 48 63 38 69 43 57

1 18 33 67 75 76 24 88 57 43

2 27 59 41 80 63 38 94 62 38

3 45 60 40 93 66 34 99 56 44

4 35 66 34 63 57 43 109 56 44

5 44 68 32 101 62 38 92 50 50

6 24 54 46 73 63 37 87 55 45

7 53 6 94 0 0 0 2 0 100

8 31 3 97 3 67 33 1 0 100

9 3 100 0 1 100 0 2 50 50

10 1 0 100 0 0 0 0 0 0

11 2 100 0 1 0 100 1 0 100

12 0 0 0 2 50 50 0 0 0

Total (n) 308 138 170 540 347 193 644 349 295

Total (%) 100 45 55 100 64 36 100 54 46

Note: SES = Supplemental Educational Services.

Source: Anchorage School District administrative data, 2006–07 through 2008–09.
Appendix B B–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.5

Percentage Distribution of SES Eligible Students in Grades K–12, by Demographic Characteristics,

by Participation Status and by Provider Type, in Anchorage, 2006–07 Through 2008–09

Demographic Characteristics

2006–07 2007–08 2008–09

All

Eligible

All 

Participants

Served by 

District 

Provider

Served by 

Non-District 

Providers

All 

Eligible

All 

Participants

Served by 

District 

Provider

Served by 

Non-District 

Providers

All 

Eligible

All 

Participants

Served by 

District 

Provider

Served by 

Non-District 

Providers

Total n 4,427 308 138 170 4,110 540 347 193 4,617 644 349 295

Minority students (%) 80 83 84 83 79* 89* 88 91 81* 89* 92 84

Percentage distribution by race/ethnicitya

African American 13 21 15 25 10 11 9 13 10 9 10 8

Hispanic 8 12 20 6 13 20 23 13 13 20 21 19

White 20 17 16 17 21 11 12 9 19 11 8 16

Asian 20 16 11 19 21 34 35 33 25 32 32 33

American Indian 1 1 1 1 1 0 0 0 <1 <1 <1 0

Alaska Native 27 20 22 18 17 12 12 14 16 11 13 9

Multiracial 11 14 16 13 17 12 10 17 16 15 16 15

Total 100 100 100 100 100 100 100 100 100 100 100 100

Percentage of students with special needs

Students with limited English 

proficiency — — — — 14* 28* 32 20 14* 26* 26 26

Students with disabilities — — — — 19 23 21 27 15 18 14+ 22+

Notes: SES = Supplemental Educational Services.

Minority students include African American, Hispanic, Asian, American Indian, Alaska Native and multiracial students.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but greater than zero.

— Indicates that this demographic group was not identified by the district.

* Indicates a statistically significant difference between the percentage of eligible students and the percentage of participating students in the given demographic group

at the 0.05 level.

+ Indicates a statistically significant difference between the percentage of district SES participants and the percentage of non-district SES participants in the given 

demographic group at the 0.05 level.

a The difference in the distribution of students by race/ethnicity was not tested for significance. 

Source: Anchorage School District administrative data, 2006–07 through 2008–09.
Appendix B B–6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.6

Number of Eligible and Participating Students and SES Participation Rates

in Grades K–12, by Race/Ethnicity, in Anchorage, 2006–07 Through 2008–09

Race/Ethnicity

Number of Eligible 

Students Number of Participants Participation Rate (%)

2006–07

African American 561 64 11

Hispanic 354 37 10

White 907 51 6

Asian 899 48 5

American Indian 49 3 6

Alaska Native 1,183 61 5

Multiracial 474 44 9

Total 4,427 308 7

2007–08

African American 395 57 14

Hispanic 542 106 20

White 859 58 7

Asian 881 186 21

American Indian 26 0 0

Alaska Native 710 67 9

Multiracial 697 66 9

Total 4,110 540 13

2008–09

African American 461 58 13

Hispanic 589 130 22

White 892 74 8

Asian 1,145 209 18

American Indian 23 1 4

Alaska Native 759 73 10

Multiracial 748 99 13

Total 4,617 644 14

Note: SES = Supplemental Educational Services.

Source: Anchorage School District administrative data, 2006–07 through 2008–09.
Appendix B B–7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.7

Percentage and Number of SES Participants in Grades K–12,

by Subject and by Provider Type, in Anchorage, 2006–07 Through 2008–09

Subject of Services Received

Students Served by

All Providers

Students Served by 

District Provider

Students Served by 

Non-District Providers

Percentage Number Percentage Number Percentage Number

2006–07

Mathematics and reading 23 72 39 54 11 18

Mathematics only 17 51 8 11 24 40

Reading only 44 135 38 52 49 83

No information 16 50 15 21 17 29

Total 100 308 100 138 100 170

2007–08

Mathematics and reading 37 201 42 146 28 55

Mathematics only 18 97 18 61 19 36

Reading only 37 201 34 119 42 82

No information 8 41 6 21 10 20

Total 100 540 100 347 100 193

2008–09

Mathematics and reading 46 296 64 223 25 73

Mathematics only 19 121 11 40 27 81

Reading only 33 211 23 79 45 132

No information 2 16 2 7 3 9

Total 100 644 100 349 100 295

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding.

Source: Anchorage School District administrative data, 2006–07 through 2008–09.

Exhibit B.8

Average Number of Hours of SES Received by Participants in Grades K–12,

by Provider Type, in Anchorage, 2004–05 Through 2008–09

Year All Providers District Provider Non-District Providers

2004–05 46 — 46

2005–06 39 — 39

2006–07 28 31 25

2007–08 31 29 36

2008–09 31 32 31

Note: SES = Supplemental Educational Services.

— Indicates that data were not available because the district did not provide SES at this time. 

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Anchorage School District administrative data, 2004–05 through 2008–09.
Appendix B B–8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.9

Percentage Distribution of SES Participants in Grades K–12, by Number of Hours

of Services Received and by Provider Type, in Anchorage, 2004–05 Through 2008–09

Hours of Services 

Received

Students Served by All Providers Students Served by District Provider Students Served by Non-District Providers

Percentage Number Percentage Number Percentage Number

2004–05

Fewer than 20 hours 0 0 — — 0 0

20–39 hours 48 15 — — 48 15

40–59 hours 52 16 — — 52 16

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 31 — — 100 31

2005–06

Fewer than 20 hours 24 25 — — 24 25

20–39 hours 18 19 — — 18 19

40–59 hours 58 60 — — 58 60

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 104 — — 100 104

2006–07

Fewer than 20 hours 32 98 33 45 31 53

20–39 hours 51 157 35 48 64 109

40–59 hours 14 43 25 35 5 8

60–79 hours 3 10 7 10 0 0

80 or more hours 0 0 0 0 0 0

Total 100 308 100 138 100 170

2007–08

Fewer than 20 hours 18 97 21 74 12 23

20–39 hours 64 346 61 211 70 135

40–59 hours 11 62 17 59 2 3

60–79 hours 6 34 1 2 17 32

80 or more hours < 1 1 < 1 1 0 0

Total 100 540 100 347 100 193

2008–09

Fewer than 20 hours 19 120 19 68 18 52

20–39 hours 64 412 57 198 73 214

40–59 hours 14 88 19 67 7 21

60–79 hours 4 24 5 16 3 8

80 or more hours 0 0 0 0 0 0

Total 100 644 100 349 100 295

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

— Indicates that data were not available because the district did not provide SES at this time. 

* Indicates a statistically significant difference between the percentage of students receiving the specified hours of SES 

with the district provider and the percentage of students receiving the specified hours of SES with non-district 

providers at the 0.05 level.

Source: Anchorage School District administrative data, 2004–05 through 2008–09.
Appendix B B–9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.10

Average Prior (2007–08) Achievement Scores for Students in Grades 3–10,

by 2008–09 Eligibility and Participation Status and by Subject, in Anchorage

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Students

Average Prior 

Achievement 

Score

Number of 

Students

Eligibility

Ineligible students 0.04* 25,176 0.04* 25,185

Eligible students –0.56* 1,433 –0.59* 1,415

Participation status

Participants –0.94+ 266 –0.91+ 261

Eligible nonparticipants –0.47+ 1,167 –0.52+ 1,154

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between ineligible students and eligible students at the 0.05 level.

+ Indicates a statistically significant difference between participants and eligible nonparticipants at the 

0.05 level.

Source: Anchorage School District administrative data, 2007–08 and 2008–09.

Exhibit B.11

Average Prior (2007–08) Achievement Scores for Participating Students in Grades 3–10,

by 2008–09 Provider Type and by Subject, in Anchorage

Provider Type

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Participants

Average Prior 

Achievement 

Score

Number of 

Participants

District provider –0.90 137 –0.80 133

Non-district providers –0.98 129 –1.02 128

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Anchorage School District administrative data, 2007–08 and 2008–09.
Appendix B B–10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.12

Mathematics and Reading Achievement Gains Associated With SES Participation, for 

Students in Grades 3–10, Overall and for a Single Year and Multiple Years,

in Anchorage, 2004–05 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Participation

Overall 0.08

(0.05) 336 0.05(0.05) 345

Years of participation

Students with one year of SES 

participation

0.08

(0.05) 298 0.05(0.04) 307

Students with multiple years of SES 

participation

0.01

(0.17) 38 0.11(0.11) 38

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,346 for mathematics and 82,340 for 

reading.

All models controlled for student eligibility for services, grade, year and grade-by-year interactions.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.

Exhibit B.13

Mathematics and Reading Achievement Gains Associated With SES Participation, for 

Students in Grades 3–10, by Provider Type, in Anchorage, 2004–05 Through 2008–09

Provider Type

Mathematics Reading

Gain

(SE) Number Gain(SE) Number 

District provider 0.05

(0.08) 164 0.10(0.05) 159

Non-district providers 0.10

(0.07) 169 0.00(0.06) 183

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,343 for mathematics and 82,337 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.
Appendix B B–11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.14

Difference in Achievement Gains Associated With SES Participation for Students

Served by District and Non-District Providers, in Grades 3–10, by Subject,

in Anchorage, 2004–05 Through 2008–09

Subject

Difference in Gain: District 

Minus Non-District

(SE)

Number 

Mathematics –0.05

(0.12) 333

Reading 0.10

(0.07) 342

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,343 for mathematics and 82,337 for 

reading.

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.
Appendix B B–12 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.15

Mathematics and Reading Achievement Gains Associated With SES Participation,

Adjusted for Hours of SES, for Students in Grades 3–10, by Provider Type,

in Anchorage, 2004–05 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number 

Provider type

District provider –0.12

(0.17) 164 –0.01(0.10) 159

Non-district providers –0.10

(0.11) 169 –0.12(0.13) 183

Hours

Gain adjusted for hours

0.06

(0.04) 333 0.04(0.04) 342

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,343 for mathematics and 82,337 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

The measure for hours was coded in units of 10 hours. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage Public Schools administrative data, 2003–04 through 2008–09.
Appendix B B–13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.16

Difference in Achievement Gains Associated With SES Participation for Students in 

Grades 3–10 Served by District and Non-District Providers, Adjusted for Hours of SES,

by Subject, in Anchorage, 2004–05 Through 2008–09

Subject

Difference in Gain:

District Minus Non-District

(SE)

Number 

Mathematics –0.02

(0.10) 333

Reading 0.12

(0.07) 342

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,343 for mathematics and 82,337 for 

reading.

The measure for hours was coded in units of 10 hours. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Anchorage Public Schools administrative data, 2003–04 through 2008–09.
Appendix B B–14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.17

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–10,

by Individual Provider, in Anchorage, 2004–05 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number 

District provider 0.05

(0.08) 28 164

Non-district provider A –0.06

(0.12) 29 75

Non-district provider B 0.04

(0.14) 22 31

Other providers 0.31*

(0.09) 41 66

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in mathematics. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,346.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.
Appendix B B–15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.18

Reading Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–10,

by Individual Provider, in Anchorage, 2004–05 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number 

District provider 0.10

(0.05) 27 159

Non-district provider A 0.06

(0.07) 27 92

Non-district provider B –0.03

(0.16) 22 31

Other providers –0.04

(0.06) 40 63

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district. 

The “other providers” category includes non-district providers serving fewer than 30 students in reading. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,340. 

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.
Appendix B B–16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit B.19

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–10, by Academic Year,

in Anchorage, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number 

2005–06 gain 0.24*

(0.08) 54 –0.11

(0.08) 54

2006–07 gain –0.02

(0.14) 86 0.05

(0.06) 116

2007–08 gain 0.01

(0.09) 106 0.07(0.08) 108

2008–09 gain 0.16

(0.08) 87 0.18(0.11) 64

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 82,346 for mathematics and 82,340 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Anchorage School District administrative data, 2003–04 through 2008–09.
Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX C:

SUPPLEMENTAL EXHIBITS

FOR BOSTON

Appendix C C–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix C: Supplemental Exhibits

for Boston

Exhibit C.1

Percentage and Number of Students Eligible for SES,

by Grade, in Boston, 2004–05 Through 2008–09

Grade

Students Eligible for SES

2004–05

(n = 41,847)

2005–06

(n = 62,103)

2006–07

(n = 55,616)

2007–08

(n = 55,672)

2008–09

(n = 55,710)

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 18 988 33 1,946 39 2,267 49 3,065 52 3,441

1 21 966 35 1,612 43 1,820 58 2,447 61 2,558

2 21 911 38 1,623 45 1,792 59 2,415 67 2,785

3 22 952 38 1,578 44 1,779 59 2,387 68 2,759

4 21 974 38 1,619 44 1,667 60 2,381 68 2,756

5 22 970 38 1,622 45 1,750 61 2,204 70 2,737

6 20 857 64 2,654 70 2,579 74 2,709 81 2,739

7 17 809 56 2,629 57 2,372 61 2,486 66 2,608

8 17 849 59 2,785 58 2,495 58 2,389 66 2,710

9 — — 19 1,221 26 1,430 29 1,472 40 2,041

10 — — 18 930 22 945 30 1,250 36 1,512

11 — — 19 897 20 809 26 1,093 37 1,456

12 — — 18 838 19 724 25 1,040 33 1,411

Total 20 8,282* 35 21,954 40 22,429 49 27,338 57 31,513

Note: SES = Supplemental Educational Services.

— Indicates that the district did not provide eligibility information.

* There were six eligible students missing grade information in 2004–05.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.2

Percentage and Number of Eligible Students Participating in SES,

by Grade, in Boston, 2004–05 Through 2008–09

Grade

Students Eligible Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 5 50 3 61 4 96 <1 15 1 45

1 16 158 14 226 15 281 1 23 2 61

2 23 213 19 303 15 260 1 31 3 94

3 27 255 27 424 18 323 3 74 8 216

4 25 241 30 485 19 319 4 97 13 355

5 21 202 21 334 18 318 4 78 10 282

6 23 198 19 513 15 391 3 75 9 246

7 13 106 16 426 15 346 3 76 6 144

8 12 98 12 330 10 251 3 70 4 115

9 — — 4 54 9 132 1 22 4 84

10 — — 9 83 15 138 1 17 13 190

11 — — 4 36 7 59 3 29 4 55

12 — — 2 13 11 83 <1 2 2 24

Total 18 1,521 15 3,288 13 2,997 2 609 6 1,911

Notes: SES = Supplemental Educational Services.

Percentages less than one (<1) are less than 0.5 percent but greater than zero.

— Indicates that the district did not provide participation information.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.3

Percentage and Number of Students in Grades K–12 Participating in SES,

by Provider, in Boston, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

District provider 70 1,062 73 2,412 97 2,903 22 131 61 1,161

Non-district provider AD 12 187 7 229 — — 12 73 — —

Non-district provider AM 5 73 6 188 3 76 3 19 10 200

Non-district provider AN 3 42 5 159 — — 18 108 3 48

Non-district provider AO 9 134 5 155 — — 11 68 5 101

Non-district provider AP 2 23 1 43 <1 1 7 44 2 44

Non-district provider AQ — — <1 13 — — — — — —

Non-district provider AR — — <1 9 <1 14 10 63 1 25

Non-district provider AS — — <1 5 <1 2 2 12 <1 9

Non-district provider AT — — <1 4 — — — — — —

Non-district provider AU — — <1 3 <1 1 1 8 <1 6

Non-district provider AV — — <1 3 — — 1 8 1 16

Non-district provider AW — — <1 2 — — — — — —

Non-district provider AX — — <1 1 — — <1 2 2 29

Non-district provider AZ — — — — — — <1 2 <1 7

Non-district provider BU — — — — — — — — 2 40

Non-district provider BV — — — — — — — — 2 41

Non-district provider BW — — — — — — 8 50 4 75

Non-district provider BX — — — — — — — — 2 45

Non-district provider EQ — — — — — — 2 15 — —

Non-district provider ER — — — — — — <1 1 — —

Non-district provider ES — — — — — — — — <1 2

Non-district provider L — — 2 58 — — 1 5 3 62

Non-district provider P — — <1 4 — — — — — —

Total 100 1,521 100 3,288 100 2,997 100 609 100 1,911

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Providers are referenced by randomly assigned letters.

2004–05 data included only grades K–8.

— Indicates that the provider did not serve SES participants. 

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.4

Number and Percentage of Students Participating in SES,

by Provider Type and by Grade, in Boston, 2004–05 Through 2008–09

Grade

2004–05 2005–06 2006–07 2007–08 2008–09

Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

K 50 62 38 61 85 15 96 100 0 15 13 87 45 51 49

1 158 68 32 226 71 29 281 100 0 23 9 91 61 41 59

2 213 64 36 303 74 26 260 100 0 31 23 77 94 40 60

3 255 61 39 424 82 18 323 100 0 74 18 82 216 72 28

4 241 73 27 485 73 27 319 100 0 97 6 94 355 72 28

5 202 64 36 334 64 36 318 99 <1 78 8 92 282 69 31

6 198 89 11 513 74 26 391 86 14 75 17 83 246 17 83

7 106 68 32 426 81 19 346 92 8 76 16 84 144 34 66

8 98 81 19 330 72 28 251 98 2 70 31 69 115 35 65

9 — — — 54 19 81 132 99 1 22 59 41 84 88 12

10 — — — 83 81 19 138 100 0 17 71 29 190 100 0

11 — — — 36 33 67 59 100 0 29 72 28 55 91 9

12 — — — 13 46 54 83 99 1 2 100 0 24 92 8

Total (n) 1,521 1,062 459 3,288 2,412 876 2,997 2,903 94 609 131 478 1,911 1,161 750

Total (%) 100 70 30 100 73 27 100 97 3 100 22 78 100 61 39

Note: SES = Supplemental Educational Services.

— Indicates that the district did not provide participation information.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.5

Percentage Distribution of SES Eligible Students in Grades K–12, by Demographic Characteristics,

by Participation Status and by Provider Type, in Boston, 2004–05 Through 2008–09

Demographic 

Characteristics

2004–05 2005–06 2006–07 2007–08 2008–09

All 

Eligible

All 

Partici

-pants

Served 

by 

District 

Provider

Served 

by

NonDistrict 

Providers

All 

Eligible

All 

Partici

-pants

Served 

by

District 

Provider

Served 

by

NonDistrict 

Providers

All 

Eligible

All 

Partici

-pants

Served 

by

District 

Provider

Served 

by

NonDistrict 

Providers

All 

Eligible

All 

Participants

Served 

by

District 

Provider

Served 

by

NonDistrict 

Providers

All 

Eligible

All 

Partici

-pants

Served 

by

District 

Provider

Served 

by

NonDistrict 

Providers

Total n 8,276 1,521 1,062 459 21,954 3,288 2,412 876 22,429 2,997 2,903 94 27,338 609 131 478 31,513 1,911 1,161 750

Minority students (%) 93 93 92 95 92* 94* 94 95 93 93 93 86 93 94 95 93 93* 92* 92 91

Percentage distribution by race/ethnicitya

African American 50 53 46 69 46 47 45 53 43 44 44 35 42 47 40 49 40 41 40 41

Hispanic 39 36 40 24 39 41 42 38 42 38 38 34 41 38 39 38 44 43 43 43

White 7 7 8 5 8 6 6 5 7 7 7 14 7 6 5 7 7 8 8 9

Asian 3 4 5 1 6 5 6 4 6 11 11 16 9 8 16 5 8 6 7 6

American Indian <1 <1 <1 <1 <1 1 1 <1 <1 <1 <1 1 <1 <1 0 <1 <1 1 1 <1

Alaska Native — — — — — — — — — — — — — — — — — — — —

Multiracial <1 0 0 0 1 <1 <1 <1 1 <1 0 <1 1 0 0 0 1 1 1 1

Total 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100

Percentage of students with special needs

Students with limited 

English proficiency 16 17 19 12 19* 23* 26+ 16+ 20* 25* 26+ 5+ 23 19 25 17 26* 32* 38+ 22+

Students with 

disabilities 24 25 26 22 25 27 28 26 26 27 27 35 25 28 21 31 24 25 24 26

Notes: SES = Supplemental Educational Services.

Minority students include African American, Hispanic, Asian, American Indian and multiracial students.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but greater than zero.

2004–05 data included only grades K–8.

— Indicates that this demographic group was not identified by the district.

* Indicates a statistically significant difference between the percentage of eligible students and the percentage of participating students in the given demographic group 

at the 0.05 level.

+ Indicates a statistically significant difference between the percentage of district SES participants and the percentage of non-district SES participants in the given 

demographic group at the 0.05 level.

a The difference in the distribution of students by race/ethnicity was not tested for significance.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.6

Number of Eligible and Participating Students and SES Participation Rates

in Grades K–12, by Race/Ethnicity, in Boston, 2004–05 Through 2008–09

Race/Ethnicity

Number of

Eligible 

Students

Number of

Participants

Participation 

Rate (%) Race/Ethnicity

Number of

Eligible 

Students

Number of

Participants

Participation 

Rate (%)

2004–05 2005–06

African American 4,110 807 20 African American 10,177 1,545 15

Hispanic 3,243 541 17 Hispanic 8,512 1,348 16

White 613 107 23 White 1,773 192 11

Asian 267 61 17 Asian 1,288 169 13

American Indian 40 5 13 American Indian 94 21 22

Alaska Native — — — Alaska Native — — —

Multiracial 3 0 0 Multiracial 110 13 12

Total 8,276 1,521 18 Total 21,954 3,288 15

2006–07 2007–08

African American 9,643 1,304 14 African American 11,349 287 3

Hispanic 9,509 1,133 12 Hispanic 11,213 234 2

White 1,624 207 13 White 1,964 39 2

Asian 1,377 322 23 Asian 2,404 46 2

American Indian 99 13 13 American Indian 110 1 1

Alaska Native — — — Alaska Native — — —

Multiracial 177 18 10 Multiracial 298 2 1

Total 22,429 2,997 13 Total 27,338 609 2

2008–09

African American 12,601 776 6

Hispanic 13,755 822 6

White 2,160 161 7

Asian 2,482 120 5

American Indian 122 13 11

Alaska Native — — —

Multiracial 393 19 5

Total 31,513 1,911 6

Notes: SES = Supplemental Educational Services.

2004–05 data included only grades K–8.

— Indicates that this demographic group was not identified by the district. 

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.7

Percentage and Number of SES Participants in Grades K–12,

by Subject and by Provider Type, in Boston, 2004–05 Through 2008–09

Subject of Services 

Received

Student Served by

All Providers 

Students Served by 

District Provider 

Students Served by 

Non-District Providers

Percentage Number Percentage Number Percentage Number

2004–05

Mathematics and reading 61 929 66 703 49 226

Mathematics only 4 67 4 43 5 24

Reading only 6 96 3 34 14 62

No information 28 429 27 282 32 147

Total 100 1,521 100 1,062 100 459

2005–06

Mathematics and reading 43 1,422 45 1,082 39 340

Mathematics only 10 340 10 230 13 110

Reading only 16 519 15 367 17 152

No information 31 1,007 30 733 31 274

Total 100 3,288 100 2,412 100 876

2006–07

Mathematics and reading 100 2,992 100 2,898 100 94

Mathematics only <1 3 <1 3 0 0

Reading only <1 2 <1 2 0 0

No information 0 0 0 0 0 0

Total 100 2,997 100 2,903 100 94

2007–08

Mathematics and reading 25 151 1 1 31 150

Mathematics only 41 247 99 130 24 117

Reading only 34 210 0 0 44 210

No information <1 1 0 0 <1 1

Total 100 609 100 131 100 478

2008–09

Mathematics and reading 64 1,232 75 868 49 364

Mathematics only 18 352 12 140 28 212

Reading only 17 317 12 143 23 174

No information 1 10 1 10 0 0

Total 100 1,911 100 1,161 100 750

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent

but greater than zero.

2004–05 data included only grades K–8.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.8

Average Number of Hours of SES Received by Participants in Grades K–12,

by Provider Type, in Boston, 2004–05 Through 2008–09

Year All Providers District Provider Non-District Providers

2004–05 44 48* 34*

2005–06 44 45 43

2006–07 79 80* 57*

2007–08 46 80* 36*

2008–09 43 40 47

Notes: SES = Supplemental Educational Services.

2004–05 data included only grades K–8.

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.9

Percentage Distribution of SES Participants in Grades K–12, by Number of Hours

of Services Received and by Provider Type, in Boston, 2004–05 Through 2008–09

Hours of Services Received

Students Served by

All Providers

Students Served by

District Provider 

Students Served by

Non-District Providers

Percentage Number Percentage Number Percentage Number

2004–05

Fewer than 20 hours 26 389 25* 261 28* 128

20–39 hours 18 277 14* 151 27* 126

40–59 hours 21 315 17* 176 30* 139

60–79 hours 29 442 35* 376 14* 66

80 or more hours 6 98 9* 98 0* 0

Total 100 1,521 100 1,062 100 459

2005–06

Fewer than 20 hours 22 714 19* 451 30* 263

20–39 hours 21 694 20* 474 25* 220

40–59 hours 26 854 29* 693 18* 161

60–79 hours 26 841 30* 735 12* 106

80 or more hours 6 185 2* 59 14* 126

Total 100 3,288 100 2,412 100 876

2006–07

Fewer than 20 hours 0 0 0 0 0 0

20–39 hours 0 0 0 0 0 0

40–59 hours 1 21 <1* 6 16* 15

60–79 hours 3 95 1* 18 82* 77

80 or more hours 96 2,881 99* 2,879 2* 2

Total 100 2,997 100 2,903 100 94

2007–08

Fewer than 20 hours 10 63 0* 0 13* 63

20–39 hours 34 205 1* 1 43* 204

40–59 hours 23 141 0* 0 29* 141

60–79 hours 9 57 0* 0 12* 57

80 or more hours 23 143 99* 130 3* 13

Total 100 609 100 131 100 478

2008–09

Fewer than 20 hours 15 291 16 190 13 101

20–39 hours 29 562 29 341 29 221

40–59 hours 28 544 33 388 21 156

60–79 hours 26 494 21 242 34 252

80 or more hours 1 20 0 0 3 20

Total 100 1,911 100 1,161 100 750

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

2004–05 data included only grades K–8.

* Indicates a statistically significant difference between the percentage of students receiving the specified hours of SES 

with the district provider and the percentage of students receiving the specified hours of SES with non-district 

providers at the 0.05 level.

Source: Boston Public Schools administrative data, 2004–05 through 2008–09.
Appendix C C–10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.10

Average Prior (2007–08) Achievement Scores for Students in Grades 3–8,

by 2008–09 Eligibility and Participation Status and by Subject, in Boston

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Students

Average Prior 

Achievement 

Score

Number of 

Students

Eligibility

Ineligible students 0.42* 8,097 0.40* 8,077

Eligible students –0.26* 11,832 –0.26* 11,702

Participation status

Participants –0.42+ 832 –0.43+ 815

Eligible nonparticipants –0.25+ 11,000 –0.25+ 10,887

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between ineligible students and eligible students at the 0.05 level.

+ Indicates a statistically significant difference between participants and eligible nonparticipants at the 0.05 level.

Source: Boston Public Schools administrative data, 2007–08 and 2008–09.

Exhibit C.11

Average Prior (2007–08) Achievement Score for Participating Students in Grades 3–8,

by 2008–09 Provider Type and by Subject, in Boston

Provider Type

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Participants 

Average Prior 

Achievement 

Score

Number of 

Participants

District provider –0.58* 384 –0.64* 374

Non-district providers –0.27* 448 –0.25* 441

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Boston Public Schools administrative data, 2007–08 and 2008–09.
Appendix C C–11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.12

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, Overall and for a Single Year and Multiple Years,

in Boston, 2002–03 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Participation

Overall 0.03

(0.02) 2,418 –0.01(0.02) 2,397

Years of participation

Students with one year of SES 

participation

0.04

(0.03) 1,822 –0.02(0.03) 1,797

Students with multiple years of SES 

participation

0.01

(0.03) 596 0.03(0.03) 600

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,882 for mathematics and 43,306 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.

Exhibit C.13

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Provider Type, in Boston, 2002–03 Through 2008–09

Provider Type

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

District provider 0.04

(0.03) 1,370 0.03(0.03) 1,369

Non-district providers 0.05

(0.04) 565 0.00(0.03) 573

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,880 for mathematics and 43,303 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–12 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.14

Difference in Achievement Gains Associated With SES Participation for Students Served 

by District and Non-District Providers, in Grades 3–8, by Subject,

in Boston, 2002–03 Through 2008–09

Subject

Difference in Gain: District Minus Non-District

(SE) Number

Mathematics –0.01

(0.05) 1,935

Reading 0.03

(0.04) 1,942

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,880 for mathematics and 43,303 for 

reading. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.15

Mathematics and Reading Achievement Gains Associated With SES Participation, 

Adjusted for Hours of SES, for Students in Grades 3–8, by Provider Type,

in Boston, 2002–03 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Provider type

District provider 0.06

(0.08) 1,370 0.06(0.07) 1,369

Non-district providers 0.06

(0.08) 565 0.02(0.07) 573

Hours

Gain adjusted for hours

0.00

(0.01) 1,935 –0.01(0.01) 1,942

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,392 for mathematics and 42,843 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

The measure for hours was coded in units of 10 hours.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.16

Difference in Achievement Gains Associated With SES Participation for Students

in Grades 3–8 Served by District and Non-District Providers, Adjusted for Hours of SES,

by Subject, in Boston, 2002–03 Through 2008–09

Subject

Difference in Gain: District Minus Non-District

(SE) Number

Mathematics –0.01

(0.05) 1,935

Reading 0.04

(0.04) 1,942

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with nondistrict providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,392 for mathematics and 44,843 for 

reading. 

The measure for hours was coded in units of 10 hours. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.17

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Boston, 2002–03 Through 2008–09

SES Provider

Gain

(SE)

Average Hours per 

Student Number

District provider 0.04

(0.03) 65 1,370

Non-district provider AM 0.07

(0.05) 67 214

Non-district provider AO 0.10

(0.10) 63 115

Non-district provider AD –0.07

(0.14) 33 101

Non-district provider AP 0.15*

(0.06) 50 55

Other providers 0.00

(0.11) 26 80

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in mathematics. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,882. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.18

Reading Achievement Gains Associated With SES Participation

and Average Hours of SES Participation for Students in Grades 3–8,

by Individual Provider, in Boston, 2002–03 Through 2008–09

SES Provider

Gain

(SE)

Average Hours per 

Student Number

District provider 0.03

(0.03) 65 1,369

Non-district provider AM –0.01

(0.04) 67 156

Non-district provider AO –0.09

(0.09) 62 115

Non-district provider AD –0.02

(0.03) 26 82

Non-district provider AP 0.15*

(0.06) 50 55

Other providers 0.02

(0.06) 37 165

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in reading. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 43,306.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.
Appendix C C–17 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit C.19

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Academic Year,

in Boston, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

2005–06 gain 0.13*

(0.06) 612 0.08(0.05) 671

2006–07 gain –0.06

(0.05) 672 –0.02(0.04) 660

2007–08 gain 0.05

(0.06) 120 0.07(0.07) 123

2008–09 gain 0.06

(0.05) 257 0.03(0.05) 227

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 44,882 for mathematics and 43,306 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Boston Public Schools administrative data, 2002–03 through 2008–09.

Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX D:

SUPPLEMENTAL EXHIBITS

FOR CHARLOTTE-MECKLENBURG

Appendix D D–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix D: Supplemental Exhibits

for Charlotte-Mecklenburg

Exhibit D.1

Percentage and Number of Students Eligible for SES,

by Grade, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Grade

Students Eligible for SES

2005–06

(n = 95,369)

2006–07

(n = 98,997)

2007–08

(n = 101,615)

2008–09

(n = 101,712)

Percentage Number Percentage Number Percentage Number Percentage Number

K 2 270 5 628 8 966 16 1,951

1 2 266 5 574 7 860 16 1,941

2 2 234 5 566 7 829 16 1,933

3 2 231 5 553 7 804 16 1,885

4 2 229 5 522 7 763 15 1,687

5 2 228 5 484 7 705 15 1,603

6 13 1,265 12 1,230 17 1,738 18 1,921

7 12 1,197 12 1,231 16 1,663 19 1,955

8 12 1,244 11 1,186 17 1,781 19 1,989

Total 5 5,164 7 6,974 10 10,109 17 16,865

Note: SES = Supplemental Educational Services.

Source: Charlotte-Mecklenburg School District administrative data, 2005–06 through 2008–09.
Appendix D D–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.2

Percentage and Number of Eligible Students Participating in SES,

by Grade, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Grade

Eligible Students Participating in SES

2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number

K 4 11 19 119 25 238 25 490

1 5 14 20 112 33 282 31 607

2 4 10 24 135 32 262 31 606

3 7 17 24 131 38 307 35 659

4 6 14 26 134 33 248 32 535

5 11 26 25 119 30 212 31 497

6 8 96 11 139 14 251 13 250

7 5 54 9 105 12 203 9 172

8 6 70 7 87 10 172 8 156

Total 6 312 16 1,081 22 2,175 24 3,972

Note: SES = Supplemental Educational Services.

Source: Charlotte-Mecklenburg School District administrative data, 2005–06 through 2008–09.
Appendix D D–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.3

Percentage and Number of Students in Grades K–8 Participating in SES,

by Provider, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

SES Provider

Students Participating in SES

2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number

District provider — — — — — — 24 944

Non-district provider A 40 126 26 285 28 612 22 872

Non-district provider AD — — 40 437 16 354 — —

Non-district provider AI 34 105 — — — — — —

Non-district provider AL 2 6 — — — — — —

Non-district provider AU <1 1 <1 2 — — <1 4

Non-district provider BJ 8 26 8 87 3 74 1 57

Non-district provider BV — — — — — — 7 290

Non-district provider BY — — — — 1 16 — —

Non-district provider BZ — — — — — — 2 65

Non-district provider CA — — — — — — 1 48

Non-district provider CB — — — — — — 2 73

Non-district provider CC — — — — 2 39 — —

Non-district provider CD — — — — — — 1 56

Non-district provider CE — — — — — — 2 93

Non-district provider CF — — — — — — 10 406

Non-district provider CG — — 3 30 10 218 3 139

Non-district provider CH — — — — 3 74 11 422

Non-district provider CI <1 1 — — 6 133 — —

Non-district provider CJ 7 21 4 47 5 105 1 49

Non-district provider CN — — 5 59 15 334 6 233

Non-district provider DP <1 1 — — — — — —

Non-district provider DW — — — — — — <1 5

Non-district provider L — — <1 4 — — — —

Non-district provider M 6 18 10 105 5 118 5 181

Non-district provider P 2 7 2 25 — — 1 20

Non-district provider R — — — — 5 98 <1 15

Total 100 312 100 1,081 100 2,175 100 3,972

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Providers are referenced by randomly assigned letters.

— Indicates that the provider did not serve SES participants.

Source: Charlotte-Mecklenburg School District administrative data, 2005–06 through 2008–09.
Appendix D D–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.4

Number and Percentage of Students Participating in SES,

by Provider Type and by Grade, in Charlotte-Mecklenburg, 2008–09

Grade

Total Number of Students

Participating in SES

Provider Type

Percentage Served by

District Provider

Percentage Served by

Non-District Providers

K 490 33 67

1 607 21 79

2 606 25 75

3 659 22 78

4 535 21 79

5 497 20 80

6 250 31 69

7 172 22 78

8 156 21 79

Total (n) 3,972 944 3,028

Total (%) 100 24 76

Note: SES = Supplemental Educational Services.

Source: Charlotte-Mecklenburg School District administrative data, 2008–09.
Appendix D D–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.5

Percentage Distribution of SES Eligible Students in Grades K–8,

by Demographic Characteristics, by Participation Status and by Provider Type,

in Charlotte-Mecklenburg, 2008–09

Demographic 

Characteristics All Eligible All Participants Served byDistrict Provider

Served by

Non-District Providers

Total n 16,865 3,972 944 3,028

Minority students (%) 96* 98* 98 98

Percentage distribution by race/ethnicitya

African American 61 64 59 66

Hispanic 27 28 33 27

White 4 2 2 2

Asian 5 3 3 3

American Indian <1 <1 1 <1

Alaska Native — — — —

Multiracial 3 3 2 3

Total 100 100 100 100

Percentage of students with special needs

Students with limited 

English proficiency 25 26 30+ 25+

Students with disabilities 12 14 14 14

Notes: SES = Supplemental Educational Services.

Minority students include African American, Hispanic, Asian, American Indian and multiracial students.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

— Indicates that this demographic group was not identified by the district.

* Indicates a statistically significant difference between the percentage of eligible students and the percentage of

participating students in the given demographic group at the 0.05 level.

+ Indicates a statistically significant difference between the percentage of district SES participants and the percentage of

non-district SES participants in the given demographic group at the 0.05 level.

a The difference in distribution of students by race/ethnicity was not tested for significance.

Source: Charlotte-Mecklenburg School District administrative data, 2008–09.
Appendix D D–6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.6

Number of Eligible and Participating Students and SES Participation Rates

in Grades K–8, by Race/Ethnicity, in Charlotte-Mecklenburg, 2008–09

Race/Ethnicity Number of Eligible Students Number of Participants Participation Rate (%)

African American 10,248 2,559 25

Hispanic 4,474 1,113 25

White 693 69 10

Asian 850 113 13

American Indian 87 17 20

Alaska Native — — —

Multiracial 513 101 20

Total 16,865 3,972 24

Note: SES = Supplemental Educational Services.

— Indicates that this demographic group was not identified by the district.

Source: Charlotte-Mecklenburg School District administrative data, 2008–09.

Exhibit D.7

Percentage and Number of SES Participants in Grades K–8,

by Subject and by Provider Type, in Charlotte-Mecklenburg, 2008–09

Subject of Services 

Received

Students Served by

All Providers

Students Served by

District Provider

Students Served by 

Non-District Providers

Percentage Number Percentage Number Percentage Number

Mathematics and reading 53 2,088 100 944 38 1,144

Mathematics only 10 415 0 0 14 415

Reading only 37 1,469 0 0 49 1,469

No information 0 0 0 0 0 0

Total 100 3,972 100 944 100 3,028

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding.

Source: Charlotte-Mecklenburg School District administrative data, 2008–09.
Appendix D D–7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.8

Average Number of Hours of SES Received by Participants in Grades K–8,

by Provider Type, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Year All Providers District Provider Non-District Providers

2005–06 20 — 20

2006–07 21 — 21

2007–08 24 — 24

2008–09 27 26 27

Note: SES = Supplemental Educational Services.

— Indicates that data were not available because the district did not provide SES at this time. 

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2005–06 through 2008–09.
Appendix D D–8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.9

Percentage Distribution of SES Participants in Grades K–8,

by Number of Hours of Services Received and by Provider Type,

in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Hours of Services 

Received

Students Served by

All Providers

Students Served by 

District Provider

Students Served by 

Non-District Providers

Percentage Number Percentage Number Percentage Number

2005–06

Fewer than 20 hours 56 176 — — 56 176

20–39 hours 43 135 — — 43 135

40–59 hours <1 1 — — <1 1

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 312 — — 100 312

2006–07

Fewer than 20 hours 39 421 — — 39 421

20–39 hours 58 624 — — 58 624

40–59 hours 3 36 — — 3 36

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 1,081 — — 100 1,081

2007–08

Fewer than 20 hours 28 612 — — 28 612

20–39 hours 70 1,521 — — 70 1,521

40–59 hours 2 42 — — 2 42

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 2,175 — — 100 2,175

2008–09

Fewer than 20 hours 23 902 23* 213 23* 689

20–39 hours 71 2,820 77* 729 69* 2,091

40–59 hours 6 250 <1* 2 8* 248

60–79 hours 0 0 0 0 0 0

80 or more hours 0 0 0 0 0 0

Total 100 3,972 100 944 100 3,028

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

— Indicates that data were not available because the district did not provide SES at this time.

* Indicates a statistically significant difference between the percentage of students receiving the specified hours of SES 

with the district provider and the percentage of students receiving the specified hours of SES with non-district 

providers at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2005–06 through 2008–09.
Appendix D D–9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.10

Average Prior (2007–08) Achievement Scores for Students in Grades 3–8,

by 2008–09 Eligibility and Participation Status and by Subject, in Charlotte-Mecklenburg

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Students

Average Prior 

Achievement 

Score

Number of 

Students

Eligibility

Ineligible students 0.11* 38,847 0.11* 38,673

Eligible students –0.60* 7,613 –0.61* 7,557

Participation status

Participants –0.83+ 1,424 –0.84+ 1,419

Eligible nonparticipants –0.55+ 6,189 –0.55+ 6,138

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between ineligible students and eligible students at the 0.05 level.

+ Indicates a statistically significant difference between participants and eligible nonparticipants at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2007–08 and 2008–09.

Exhibit D.11

Average Prior (2007–08) Achievement Scores for Participating Students in Grades 3–8,

by 2008–09 Provider Type and by Subject, in Charlotte-Mecklenburg

Provider Type

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Participants

Average Prior 

Achievement 

Score

Number of 

Participants

District provider –0.86 317 –0.87 315

Non-district providers –0.82 1,107 –0.83 1,104

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2007–08 and 2008–09.
Appendix D D–10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.12

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, Overall and for a Single Year and Multiple Years,

in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Participation

Overall 0.11*

(0.06) 724 0.03(0.03) 1,456

Years of participation

Students with one year of SES 

participation

0.12*

(0.06) 640 0.03(0.03) 1,274

Students with multiple years of SES 

participation

0.04

(0.10) 84 0.04(0.07) 182

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,602 for mathematics and 133,135 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2004–05 through 2008–09.
Appendix D D–11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.13

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Provider Type, in Charlotte-Mecklenburg,

2005–06 Through 2008–09

Provider Type

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

District provider 0.23*

(0.08) 152 0.04(0.06) 151

Non-district providers 0.10

(0.08) 263 0.02(0.07) 319

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,269 for mathematics and 132,034 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2004–05 through 2008–09.
Appendix D D–12 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.14

Difference in Achievement Gains Associated With SES Participation

for Students Served by District and Non-District Providers, in Grades 3–8, by Subject,

in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Subject

Difference in Gain: District Minus Non-District

(SE) Number

Mathematics 0.12

(0.07) 415

Reading 0.03

(0.08) 470

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,269 for mathematics and 132,034 for 

reading.

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Charlotte-Mecklenburg Public Schools administrative data, 2004–05 through 2008–09.
Appendix D D–13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.15

Mathematics and Reading Achievement Gains Associated With SES Participation,

Adjusted for Hours of SES, for Students in Grades 3–8, by Provider Type,

in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Provider type

District provider 0.21*

(0.09) 152 0.12(0.10) 151

Non-district providers 0.09

(0.11) 263 0.10(0.10) 319

Hours

Gain adjusted for hours

0.01

(0.04) 415 –0.04(0.03) 470

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,269 for mathematics and 132,034 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

The measure for hours was coded in units of 10 hours.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg Public Schools administrative data, 2004–05 through 2008–09.
Appendix D D–14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.16

Difference in Achievement Gains Associated With SES Participation for Students in 

Grades 3–8 Served by District and Non-District Providers, Adjusted for Hours of SES, by 

Subject, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Subject

Difference in Gain: District Minus Non-District

(SE) Number

Mathematics 0.13

(0.07) 415

Reading 0.02

(0.08) 470

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,269 for mathematics and 132,034 for 

reading.

The measure for hours was coded in units of 10 hours. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Charlotte-Mecklenburg Public Schools administrative data, 2004–05 through 2008–09.
Appendix D D–15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.17

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number

District provider 0.22*

(0.08) 22 152

Non-district provider A 0.08

(0.10) 27 149

Non-district provider CN 0.05

(0.11) 22 101

Non-district provider CH 0.03

(0.10) 21 74

Non-district provider CF 0.01

(0.12) 30 53

Non-district provider AI 0.07

(0.11) 14 47

Non-district provider CG 0.11

(0.19) 22 44

Non-district provider CJ 0.34*

(0.10) 20 30

Other providers 0.13

(0.08) 20 74

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in mathematics. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,602.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are reference by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2004–05 through 2008–09.
Appendix D D–16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.18

Reading Student Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Charlotte-Mecklenburg, 2005–06 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number

Non-district provider A 0.00

(0.06) 23 381

District provider 0.01

(0.06) 22 151

Non-district provider CN –0.02

(0.09) 22 99

Non-district provider BJ 0.09

(0.09) 21 77

Non-district provider M –0.06

(0.09) 20 70

Non-district provider CH 0.17

(0.11) 20 60

Non-district provider CG –0.01

(0.12) 21 55

Non-district provider CF –0.16

(0.18) 30 52

Non-district provider AI 0.11

(0.06) 14 47

Non-district provider R 0.27

(0.20) 16 37

Non-district provider CJ 0.12

(0.16) 22 31

Other providers 0.06

(0.05) 19 396

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in reading. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 133,135. 

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2004–05 through 2008–09.
Appendix D D–17 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit D.19

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Academic Year,

in Charlotte-Mecklenburg, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

2005–06 gain 0.10

(0.12) 53 0.07(0.07) 113

2006–07 gain 0.29*

(0.13) 31 0.03(0.05) 385

2007–08 gain 0.04

(0.08) 225 0.04(0.05) 488

2008–09 gain 0.14

(0.07) 415 0.02(0.05) 470

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 132,602 for mathematics and 133,135 reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Charlotte-Mecklenburg School District administrative data, 2004–05 through 2008–09.

Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX E:

SUPPLEMENTAL EXHIBITS

FOR CHICAGO

Appendix E E–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix E: Supplemental Exhibits

for Chicago

Exhibit E.1

Percentage and Number of Students Eligible for SES,

by Grade, in Chicago, 2004–05 Through 2008–09

Grade

Students Eligible for SES

2004–05

(n = 169,217)

2005–06

(n = 156,884)

2006–07

(n = 164,791)

2007–08

(n = 181,511)

2008–09

(n = 176,906)

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

3 55 15,445 56 13,921 73 18,276 72 22,738 77 24,405

4 57 15,859 54 13,379 72 17,124 71 20,509 77 22,128

5 56 15,751 53 13,487 72 19,009 71 20,870 77 21,822

6 56 15,548 53 14,249 73 22,114 71 21,830 77 22,803

7 57 15,740 52 13,795 72 22,168 71 21,148 76 21,956

8 57 15,547 53 14,324 70 19,956 71 21,936 75 22,224

Total 56 93,890 53 83,155 72 118,647 71 129,031 77 135,338

Note: SES = Supplemental Educational Services.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.

Exhibit E.2

Percentage and Number of Eligible Students Participating in SES,

by Grade, in Chicago, 2004–05 Through 2008–09

Grade

Eligible Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

3 42 6,499 53 7,426 27 4,997 25 5,694 32 7,789

4 38 5,996 32 4,243 23 3,988 22 4,467 28 6,305

5 36 5,647 25 3,314 20 3,773 19 4,038 26 5,616

6 31 4,880 21 2,945 18 4,048 18 3,947 25 5,706

7 27 4,296 20 2,804 15 3,254 14 2,879 20 4,324

8 30 4,632 20 2,899 13 2,618 14 3,165 19 4,317

Total 34 31,950 28 23,631 19 22,678 19 24,190 25 34,057

Note: SES = Supplemental Educational Services.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.3

Percentage and Number of Students in Grades 3–8 Participating in SES,

by Provider, in Chicago, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

District provider 49 15,580 38 9,017 31 7,065 23 5,610 13 4,505

Non-district provider AD 6 1,933 7 1,759 9 1,983 5 1,279 — —

Non-district provider AI <1 137 1 339 1 259 1 265 2 807

Non-district provider AJ — — — — — — <1 12 — —

Non-district provider AL — — <1 75 — — — — — —

Non-district provider AR — — <1 76 4 939 2 563 4 1,364

Non-district provider AY 15 4,664 21 5,061 11 2,408 10 2,494 11 3,743

Non-district provider AZ — — 9 2,121 6 1,380 15 3,606 9 3,049

Non-district provider BA 1 473 <1 85 — — — — — —

Non-district provider BB 1 333 3 768 6 1,394 6 1,367 7 2,403

Non-district provider BC — — 2 441 2 527 4 942 4 1,346

Non-district provider BD 1 290 2 405 2 503 2 570 2 595

Non-district provider BE <1 97 1 328 2 359 1 198 1 330

Non-district provider BF 1 262 1 308 1 148 1 164 <1 146

Non-district provider BG 3 880 1 209 — — — — — —

Non-district provider BH — — 1 193 3 712 1 289 <1 160

Non-district provider BI — — <1 117 — — — — — —

Non-district provider BJ <1 117 <1 86 1 206 <1 79 1 196

Non-district provider BK — — <1 92 1 154 <1 43 — —

Non-district provider BL 17 5,582 5 1,234 5 1,076 4 853 2 630

Non-district provider BM <1 121 <1 69 1 154 <1 89 <1 155

Non-district provider BN 1 247 1 167 2 452 1 239 1 507

Non-district provider BO <1 10 <1 23 1 199 <1 48 — —

Non-district provider BP 1 466 — — — — — — — —

Non-district provider BQ <1 51 — — — — — — — —

Non-district provider BR <1 80 <1 26 <1 88 <1 29 — —

Non-district provider BS 1 220 — — — — — — — —

Non-district provider BT 1 407 — — — — — — — —

Non-district provider BY — — — — — — <1 4 <1 161

Non-district provider CV — — <1 2 <1 8 — — — —

Non-district provider CZ — — — — <1 70 <1 56 — —

Non-district provider DI — — <1 14 <1 31 <1 21 <1 43

Non-district provider DX — — — — <1 22 <1 18 <1 102

Non-district provider DY — — — — <1 82 <1 41 <1 34

Non-district provider DZ — — <1 12 — — <1 23 1 396

Non-district provider EA — — — — — — <1 27 1 194

Non-district provider EB — — <1 65 1 188 2 475 2 743

Non-district provider EC — — <1 61 <1 33 1 183 1 190

Non-district provider ED — — — — <1 35 <1 61 <1 5

Non-district provider EE — — — — <1 112 <1 106 1 404

Non-district provider EF — — — — — — <1 35 <1 6

Non-district provider EG — — — — — — — — <1 102

continued next page
Appendix E E–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.3 (continued)

Percentage and Number of Students in Grades 3–8 Participating in SES,

by Provider, in Chicago, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

Non-district provider EH — — — — <1 78 1 286 2 610

Non-district provider EI — — — — — — <1 117 1 198

Non-district provider EJ — — — — <1 86 7 1,579 18 6,047

Non-district provider EK — — <1 69 <1 42 <1 33 — —

Non-district provider EL — — — — — — — — 3 1,013

Non-district provider EM — — — — — — 2 541 5 1,615

Non-district provider EN — — <1 7 <1 67 <1 45 <1 37

Non-district provider EO — — — — — — — — <1 98

Non-district provider EP — — — — <1 61 <1 42 <1 73

Non-district provider ET — — <1 5 <1 22 — — — —

Non-district provider EU — — — — — — <1 2 — —

Non-district provider EV — — — — — — — — <1 19

Non-district provider EW — — — — — — <1 9 <1 15

Non-district provider EX — — — — — — <1 6 <1 8

Non-district provider EY — — — — <1 13 <1 30 — —

Non-district provider EZ — — — — <1 4 — — — —

Non-district provider FA — — — — <1 13 <1 1 — —

Non-district provider FB — — — — — — — — <1 3

Non-district provider FC — — — — — — <1 43 — —

Non-district provider FD — — <1 22 <1 14 — — — —

Non-district provider FE — — — — — — — — <1 41

Non-district provider FF — — <1 23 — — <1 3 — —

Non-district provider FG — — — — <1 8 <1 13 <1 8

Non-district provider FH — — — — — — <1 4 — —

Non-district provider FI — — — — — — — — <1 14

Non-district provider FJ — — <1 12 <1 27 <1 5 — —

Non-district provider L — — 1 268 3 732 1 211 1 448

Non-district provider N — — — — 3 786 4 931 — —

Non-district provider O — — — — <1 14 <1 61 <1 121

Non-district provider P — — <1 72 1 124 2 435 4 1,366

Non-district provider T — — — — — — <1 4 <1 7

Total 100 31,950 100 23,631 100 22,678 100 24,190 100 34,057

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Providers are referenced by randomly assigned letters.

— Indicates that the provider did not serve SES participants.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.4

Number and Percentage of Students Participating in SES, by Provider Type and by Grade,

in Chicago, 2004–05 Through 2008–09

Grade

2004–05 2005–06 2006–07 2007–08 2008–09

Total 

Number of

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type Total 

Number of 

Students

Participating in 

SES

Provider Type

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

3 6,499 53 47 7,426 25 75 4,997 31 69 5,694 24 76 7,789 14 86

4 5,996 48 52 4,243 41 59 3,988 30 70 4,467 23 77 6,305 13 87

5 5,647 48 52 3,314 42 58 3,773 30 70 4,038 22 78 5,616 13 87

6 4,880 48 52 2,945 47 53 4,048 30 70 3,947 25 75 5,706 13 87

7 4,296 46 54 2,804 42 58 3,254 33 67 2,879 21 79 4,324 12 88

8 4,632 50 50 2,899 50 50 2,618 34 66 3,165 23 77 4,317 14 86

Total (n) 31,950 15,580 16,370 23,631 9,017 14,614 22,678 7,065 15,613 24,190 5,610 18,580 34,057 4,505 29,552

Total (%) 100 49 51 100 38 62 100 31 69 100 23 77 100 13 87

Note: SES = Supplemental Educational Services.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.5

Percentage Distribution of SES Eligible Students in Grades 3–8, by Demographic Characteristics,

by Participation Status and by Provider Type, in Chicago, 2004–05 Through 2008–09

Demographic 

Characteristics

2004–05 2005–06 2006–07 2007–08 2008–09

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

NonDistrict 

Providers

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

NonDistrict 

Providers

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

NonDistrict 

Providers

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

NonDistrict 

Providers

All 

Eligible

All 

Participants

Served 

by

District 

Provider

Served by

NonDistrict 

Providers

Total n 63,950 22,270 10,445 11,825 75,283 17,609 7,265 10,344 118,646 22,677 7,065 15,612 129,031 24,190 5,610 18,580 135,338 34,057 4,505 29,552

Minority students (%) 98* 99* 99+ 99+ 99* 99* 99 99 97* 99* 99 99 96* 99* 99 99 96* 98* 99 98

Percentage distribution by race/ethnicitya

African American 61 65 52 77 65 72 71 73 60* 78 79 78 49 62 58 63 46 55 48 56

Hispanic 35 32 44 21 31 24 26 23 33 19 19 19 42 35 39 34 44 40 48 39

White 2 1 1 1 1 1 1 1 3 1 1 1 4 1 1 1 4 2 1 2

Asian <1 <1 <1 <1 <1 <1 <1 <1 2 <1 <1 <1 2 <1 <1 <1 3 1 <1 1

American Indian <1 <1 0 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1

Alaska Native — — — — — — — — — — — — — — — — — — — —

Multiracial 2 2 2 1 2 2 2 2 2 2 2 2 3 2 2 2 3 2 2 2

Total 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100

Percentage of students with special needs

Students with limited 

English proficiency 12* 14* 20+ 8+ 4 11 8 13 8* 5* 4 5 14 14 16 13 14* 16* 20 16

Students with disabilities 14* 13* 13 13 13* 16* 13+ 18+ 14 14 14 14 12* 13* 13 13 13* 14* 14 14

Notes: SES = Supplemental Educational Services.

In 2004–05, the data provided by Chicago Public Schools did not have the race/ethnicity information for 29,940 eligible students, of whom 

9,680 (32 percent) were SES participants. In 2005–06, the data provided did not have the race/ethnicity information for 7,872 eligible students, of whom 

6,022 (76 percent) were SES participants. In 2006–07, the data provided did not have the race/ethnicity information for one eligible student, who also 

participated in SES.

Minority students include African American, Hispanic, Asian, American Indian and multiracial students.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but greater than zero.

— Indicates that this demographic group was not identified by the district.

* Indicates a statistically significant difference between the percentage of eligible students and the percentage of participating students in the given demographic group 

at the 0.05 level.

+ Indicates a statistically significant difference between the percentage of district SES participants and the percentage of non-district SES participants in the given 

demographic group at the 0.05 level.

a The distribution of students by race/ethnicity was not tested for significance.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.6

Number of Eligible and Participating Students and SES Participation Rates

in Grades 3–8, by Race/Ethnicity, in Chicago, 2004–05 Through 2008–09

Race/

Ethnicity

Number of

Eligible 

Students

Number of

Participants

Participation 

Rate (%) Race/Ethnicity

Number of

Eligible 

Students

Number of

Participants

Participation 

Rate (%)

2004–05 2005–06

African American 38,818 14,485 37 African American 49,185 12,741 26

Hispanic 22,611 7,090 31 Hispanic 23,017 4,288 19

White 982 247 25 White 1,129 175 16

Asian 253 57 23 Asian 409 45 11

American Indian 33 11 33 American Indian 44 15 34

Alaska Native — — — Alaska Native — — —

Multiracial 1,253 380 30 Multiracial 1,499 345 23

Total 63,950 22,270 35 Total 75,283 17,609 23

2006–07 2007–08

African American 71,028 17,728 25 African American 63,821 14,926 23

Hispanic 39,018 4,354 11 Hispanic 53,946 8,483 16

White 3,598 171 5 White 4,761 214 4

Asian 2,116 61 3 Asian 2,843 70 2

American Indian 76 7 9 American Indian 97 8 8

Alaska Native — — — Alaska Native — — —

Multiracial 2,810 356 13 Multiracial 3,563 489 14

Total 118,646 22,677 19 Total 129,031 24,190 19

2008–09

African American 62,117 18,774 30

Hispanic 60,188 13,765 23

White 5,518 526 10

Asian 3,685 318 9

American Indian 103 12 12

Alaska Native — — —

Multiracial 3,727 662 18

Total 135,338 34,057 25

Notes: SES = Supplemental Educational Services.

In 2004–05, the data provided by Chicago Public Schools did not have the race/ethnicity information for 

29,940 eligible students, and 9,680 (32 percent) of those eligible students were SES participants. In 2005–06, 

the data provided did not have the race/ethnicity information for 7,872 eligible students, and 6,022 

(76 percent) of those eligible students were SES participants. In 2006–07, the data provided did not have the 

race/ethnicity information for one eligible student, who also participated in SES.

— Indicates that this demographic group was not identified by the district.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.7

Percentage and Number of SES Participants in Grades 3–8,

by Subject and by Provider Type, in Chicago, 2004–05 Through 2008–09

Subject of Services 

Received

Students Served by

All Providers 

Students Served by 

District Provider 

Students Served by 

Non-District Providers 

Percentage Number Percentage Number Percentage Number

2004–05

Mathematics and reading — — — — — —

Mathematics only — — — — — —

Reading only — — — — — —

No information 100 31,950 100 15,580 100 16,370

Total 100 31,950 100 15,580 100 16,370

2005–06

Mathematics and reading — — — — — —

Mathematics only — — — — — —

Reading only — — — — — —

No information 100 23,631 100 9,017 100 14,614

Total 100 23,631 100 9,017 100 14,614

2006–07

Mathematics and reading — — — — — —

Mathematics only — — — — — —

Reading only — — — — — —

No information 100 22,678 100 7,065 100 15,613

Total 100 22,678 100 7,065 100 15,613

2007–08

Mathematics and reading — — — — — —

Mathematics only — — — — — —

Reading only — — — — — —

No information 100 24,190 100 5,610 100 18,580

Total 100 24,190 100 5,610 100 18,580

2008–09

Mathematics and reading — — — — — —

Mathematics only — — — — — —

Reading only — — — — — —

No information 100 34,057 100 4,505 100 29,552

Total 100 34,057 100 4,505 100 29,552

Note: SES = Supplemental Educational Services.

— Indicates that the district did not provide data on the subject of tutoring. 

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.8

Average Number of Hours of SES Received by Participants in Grades 3–8,

by Provider Type, in Chicago, 2004–05 Through 2008–09

Year All Providers District Provider Non-District Providers

2004–05 55 58 53

2005–06 81 80* 81*

2006–07 38 47* 34*

2007–08 37 46* 34*

2008–09 39 55* 37*

Note: SES = Supplemental Educational Services.

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.9

Percentage Distribution of SES Participants in Grades 3–8, by Number of Hours of 

Services Received and by Provider Type, in Chicago, 2004–05 Through 2008–09

Hours of Services Received

Students Served by

All Providers

Students Served by

District Provider

Students Served by

Non-District Providers

Percentage Number Percentage Number Percentage Number

2004–05

Fewer than 20 hours 17 5,391 14 2,164 20 3,227

20–39 hours 19 6,114 18 2,877 20 3,237

40–59 hours 22 6,874 19 3,021 24 3,853

60–79 hours 27 8,785 31 4,821 24 3,964

80 or more hours 15 4,786 17 2,697 13 2,089

Total 100 31,950 100 15,580 100 16,370

2005–06

Fewer than 20 hours 6 1,345 3* 248 8* 1,097

20–39 hours 5 1,067 6* 549 4* 518

40–59 hours 19 4,439 5* 458 27* 3,981

60–79 hours 14 3,252 8* 755 17* 2,497

80 or more hours 57 13,528 78* 7,007 45* 6,521

Total 100 23,631 100 9,017 100 14,614

2006–07

Fewer than 20 hours 22 4,939 20* 1,388 23* 3,551

20–39 hours 31 6,947 16* 1,139 37* 5,808

40–59 hours 30 6,858 22* 1,520 34* 5,338

60–79 hours 16 3,740 40* 2,841 6* 899

80 or more hours 1 194 3* 177 <1* 17

Total 100 22,678 100 7,065 100 15,613

2007–08

Fewer than 20 hours 21 4,980 17* 959 22* 4,021

20–39 hours 39 9,384 22* 1,248 44* 8,136

40–59 hours 27 6,546 26* 1,475 27* 5,071

60–79 hours 13 3,177 33* 1,840 7* 1,337

80 or more hours 0 103 2* 88 <1* 15

Total 100 24,190 100 5,610 100 18,580

2008–09

Fewer than 20 hours 16 5,581 15* 667 17* 4,914

20–39 hours 31 10,547 12* 552 34* 9,995

40–59 hours 37 12,484 16* 728 40* 11,756

60–79 hours 14 4,876 45* 2,034 10* 2,842

80 or more hours 2 569 12* 524 <1* 45

Total 100 34,057 100 4,505 100 29,552

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

* Indicates a statistically significant difference between the percentage of students receiving the specified hours of SES 

with the district provider and the percentage of students receiving the specified hours of SES with the non-district 

providers at the 0.05 level.

Source: Chicago Public Schools administrative data, 2004–05 through 2008–09.
Appendix E E–10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.10

Average Prior (2007–08) Achievement Scores for Students in Grades 3–8,

by 2008–09 Eligibility and Participation Status and by Subject, in Chicago

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Students

Average Prior 

Achievement 

Score

Number of 

Students

Eligibility

Ineligible students 0.42* 31,742 0.43* 31,571

Eligible students –0.10* 106,276 –0.11* 105,364

Participation status

Participants –0.33+ 25,827 –0.33+ 25,592

Eligible nonparticipants –0.03+ 80,449 –0.04+ 79,772

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between ineligible students and eligible students at the 0.05 level.

+ Indicates a statistically significant difference between participants and eligible nonparticipants at the 0.05 level.

Source: Chicago Public Schools administrative data, 2007–08 and 2008–09.

Exhibit E.11

Average Prior (2007–08) Achievement Scores for Participating Students in Grades 3–8,

by 2008–09 Provider Type and by Subject, in Chicago

Provider Type

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Participants

Average Prior 

Achievement 

Score

Number of 

Participants

District provider –0.37 3,389 –0.39* 3,356

Non-district providers –0.33 22,438 –0.32* 22,236

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Chicago Public Schools administrative data, 2007–08 and 2008–09.
Appendix E E–11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.12

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, Overall and for a Single Year and Multiple Years,

in Chicago, 2002–03 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Participation

Overall 0.05*

(0.01) 61,843 0.07*(0.01) 62,011

Years of participation

Students with one year of SES 

participation

0.05*

(0.01) 40,898 0.07*(0.01) 41,017

Students with multiple years of SES 

participation

0.06*

(0.01) 20,945 0.08*(0.01) 20,994

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.

Exhibit E.13

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Provider Type, in Chicago, 2002–03 Through 2008–09

Provider Type

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

District provider 0.06*

(0.01) 21,909 0.03*(0.01) 21,974

Non-district providers 0.04*

(0.01) 39,934 0.09*(0.01) 40,037

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–12 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.14

Difference in Achievement Gains Associated With SES Participation for Students Served 

by District and Non-District Providers, in Grades 3–8,

by Subject, in Chicago, 2002–03 Through 2008–09

Subject

Difference in Gain:

District Minus Non-District

(SE)

Number

Mathematics 0.01

(0.01) 61,843

Reading –0.06*

(0.01) 62,011

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with nondistrict providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.15

Mathematics and Reading Achievement Gains Associated With SES Participation,

Adjusted for Hours of SES, for Students in Grades 3–8,

by Provider Type, in Chicago, 2002–03 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Provider type

District provider 0.02

(0.01) 21,909 –0.05*(0.01) 21,974

Non-district providers 0.02*

(0.01) 39,934 0.03*(0.01) 40,037

Hours

Gain adjusted for hours

0.01*

(0.00) 61,843 0.02*(0.00) 62,011

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The measure for hours was coded in units of 10 hours. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.16

Difference in Achievement Gains Associated With SES Participation

for Students in Grades 3–8 Served by District and Non-District Providers,

Adjusted for Hours of SES, by Subject, in Chicago, 2002–03 Through 2008–09

Subject

Difference in Gain:

District Minus Non-District

(SE)

Number

Mathematics 0.01

(0.01) 61,843

Reading –0.08*

(0.01) 62,011

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

The measure for hours was coded in units of 10 hours. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.17

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Chicago, 2002–03 Through 2008–09

SES Provider 

Gain

(SE) Average Hours per Student Number

District provider 0.06*

(0.01) 55 21,909

Non-district provider AY 0.05*

(0.01) 56 8,029

Non-district provider BL 0.02

(0.02) 44 4,933

Non-district provider AZ 0.07*

(0.02) 40 3,925

Non-district provider AD 0.00

(0.02) 37 3,233

Non-district provider BB 0.04

(0.02) 31 2,697

Non-district provider EJ 0.07*

(0.02) 31 2,391

Non-district provider BC 0.06*

(0.03) 59 1,446

Non-district provider AR 0.04

(0.02) 21 1,402

Non-district provider BD 0.08*

(0.03) 57 1,049

Non-district provider BN 0.02

(0.05) 23 767

Non-district provider N 0.04

(0.04) 31 723

Non-district provider AI 0.12*

(0.04) 31 718

Non-district provider EM 0.10*

(0.05) 32 717

Non-district provider L 0.03

(0.03) 30 649

Non-district provider P 0.01

(0.04) 27 584

Non-district provider BG 0.00

(0.04) 41 567

Non-district provider BH –0.01

(0.04) 34 561

Non-district provider BE 0.11*

(0.04) 38 485

Non-district provider EB 0.08*

(0.03) 25 474

Non-district provider BP 0.12

(0.08) 49 421

Non-district provider BA 0.02

(0.03) 35 402

Non-district provider BF 0.08

(0.05) 33 371

Non-district provider EL –0.02

(0.06) 28 342

Non-district provider BM 0.09

(0.05) 48 274

Non-district provider EH –0.05

(0.06) 37 253

Non-district provider BJ –0.02

(0.06) 30 235

Non-district provider BT –0.03

(0.04) 46 234

continued next page
Appendix E E–16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.17 (continued)

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Chicago, 2002–03 Through 2008–09

SES Provider 

Gain

(SE) Average Hours per Student Number

Non-district provider EE 0.06

(0.09) 45 226

Non-district provider BS 0.14*

(0.04) 48 171

Non-district provider DZ 0.12*

(0.05) 39 160

Non-district provider BO 0.05

(0.08) 35 137

Non-district provider BK 0.06

(0.08) 27 127

Non-district provider EI 0.10

(0.07) 38 119

Non-district provider BR 0.13

(0.08) 37 95

Non-district provider DX –0.05

(0.11) 38 95

Non-district provider EC 0.08

(0.10) 37 95

Non-district provider EA 0.01

(0.07) 39 83

Non-district provider DY 0.01

(0.09) 22 61

Non-district provider EN 0.09

(0.08) 36 59

Non-district provider EO –0.02

(0.07) 36 59

Non-district provider EP 0.06

(0.09) 42 57

Non-district provider CZ 0.05

(0.10) 36 55

Non-district provider ED 0.05

(0.05) 27 54

Non-district provider DI –0.01

(0.10) 38 48

Non-district provider O –0.15*

(0.05) 38 38

Non-district provider BQ –0.11*

(0.02) 16 38

Non-district provider EG –0.02

(0.04) 47 32

Other providers 0.00

(0.06) 35 243

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in mathematics. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–17 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.18

Reading Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Chicago, 2002–03 Through 2008–09

SES Provider 

Gain

(SE) Average Hours per Student Number

District provider 0.03*

(0.01) 55 21,974

Non-district provider AY 0.13*

(0.02) 56 8,040

Non-district provider BL 0.06*

(0.02) 44 4,953

Non-district provider AZ 0.13*

(0.02) 40 3,928

Non-district provider AD 0.07*

(0.02) 37 3,246

Non-district provider BB 0.13*

(0.02) 31 2,698

Non-district provider EJ 0.06*

(0.02) 31 2,397

Non-district provider BC 0.12*

(0.03) 59 1,452

Non-district provider AR 0.03

(0.02) 21 1,408

Non-district provider BD 0.13*

(0.03) 57 1,050

Non-district provider BN 0.00

(0.03) 23 767

Non-district provider N 0.05

(0.03) 30 726

Non-district provider AI 0.10*

(0.04) 31 725

Non-district provider EM 0.06

(0.04) 32 718

Non-district provider L 0.06

(0.04) 30 656

Non-district provider P 0.05

(0.03) 27 584

Non-district provider BG 0.10*

(0.05) 41 569

Non-district provider BH 0.00

(0.03) 34 561

Non-district provider BE 0.13*

(0.05) 38 493

Non-district provider EB 0.08

(0.05) 25 472

Non-district provider BP 0.00

(0.06) 48 422

Non-district provider BA 0.04

(0.04) 35 404

Non-district provider BF 0.11

(0.07) 33 372

Non-district provider EL 0.01

(0.06) 28 340

Non-district provider BM 0.12*

(0.05) 47 276

Non-district provider EH 0.09

(0.05) 37 253

Non-district provider BJ 0.13*

(0.06) 30 237

Non-district provider BT –0.02

(0.06) 46 237

continued next page
Appendix E E–18 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.18 (continued)

Reading Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–8,

by Individual Provider, in Chicago, 2002–03 Through 2008–09

SES Provider 

Gain

(SE) Average Hours per Student Number

Non-district provider EE 0.06

(0.05) 45 225

Non-district provider BS –0.06

(0.08) 48 171

Non-district provider DZ 0.08

(0.10) 39 160

Non-district provider BO 0.12

(0.08) 35 138

Non-district provider BK 0.18*

(0.06) 27 127

Non-district provider EI 0.10

(0.07) 38 119

Non-district provider BR 0.20

(0.12) 37 96

Non-district provider DX –0.04

(0.05) 38 95

Non-district provider EC 0.14

(0.07) 37 94

Non-district provider EA 0.13

(0.11) 39 85

Non-district provider EN 0.07

(0.09) 35 61

Non-district provider DY –0.05

(0.12) 22 60

Non-district provider EO 0.01

(0.09) 36 59

Non-district provider EP 0.15

(0.09) 42 57

Non-district provider CZ 0.01

(0.15) 36 55

Non-district provider ED 0.05

(0.12) 27 53

Non-district provider DI 0.16

(0.12) 38 48

Non-district provider O 0.19

(0.10) 38 38

Non-district provider BQ 0.10*

(0.03) 16 38

Non-district provider EG 0.09

(0.09) 47 32

Other providers 0.09

(0.06) 35 242

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in reading. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 744,695.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.
Appendix E E–19 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit E.19

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–8, by Academic Year,

in Chicago, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

2005–06 gain 0.06*

(0.01) 9,856 0.17*(0.02) 9,890

2006–07 gain 0.04*

(0.01) 11,200 0.02*(0.01) 11,213

2007–08 gain 0.06*

(0.01) 9,707 0.06*(0.01) 9,730

2008–09 gain 0.07*

(0.01) 10,327 0.07*(0.01) 10,353

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 742,696 for mathematics and 744,695 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Chicago Public Schools administrative data, 2001–02 through 2008–09.

Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX F:

SUPPLEMENTAL EXHIBITS

FOR HILLSBOROUGH

Appendix F F–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix F: Supplemental Exhibits

for Hillsborough

Exhibit F.1

Percentage and Number of Students Eligible for SES,

by Grade, in Hillsborough, 2004–05 Through 2008–09

Grade

Students Eligible for SES

2004–05

(n = 94,784)

2005–06

(n = 204,062)

2006–07

(n = 203,468)

2007–08

(n = 201,704)

2008–09

(n = 200,782)

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 2 337 25 4,032 30 4,881 34 5,310 35 5,388

1 2 356 24 4,191 31 5,329 35 5,858 36 5,976

2 2 296 24 3,832 29 4,726 33 5,409 35 5,719

3 2 309 24 3,829 29 4,705 32 5,171 34 5,647

4 2 279 23 3,562 28 4,415 31 4,796 34 5,323

5 2 266 22 3,523 27 4,192 31 4,865 32 5,033

6 — — 20 3,112 23 3,608 26 4,073 27 4,273

7 — — 20 3,306 22 3,501 25 3,942 26 3,993

8 — — 19 3,169 21 3,500 24 3,761 25 3,944

9 — — 6 1,020 6 988 6 945 9 1,489

10 — — 6 864 6 903 6 821 8 1,236

11 — — 5 712 5 702 5 757 8 1,061

12 — — 4 481 5 598 2 305 5 645

Total 2 1,843 17 35,633 21 42,048 23 46,013 25 49,727

Note: SES = Supplemental Educational Services.

— Indicates that the district did not provide eligibility information.

Source: Hillsborough School District administrative data, 2004–05 through 2008–09.
Appendix F F–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.2

Percentage and Number of Eligible Students Participating in SES,

by Grade, in Hillsborough, 2004–05 Through 2008–09

Grade

Eligible Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

K 17 57 9 370 16 788 7 354 3 159

1 14 50 16 680 21 1,136 13 744 9 537

2 14 41 17 637 24 1,119 16 844 18 1,032

3 16 48 16 595 21 980 18 924 21 1,178

4 8 23 14 498 16 723 15 734 19 1,032

5 17 46 11 402 15 640 14 688 17 869

6 — — 5 158 7 240 6 261 10 408

7 — — 4 139 4 153 4 146 7 288

8 — — 3 105 5 188 3 130 4 164

9 — — 2 19 2 23 2 18 2 28

10 — — 2 17 2 19 2 15 3 38

11 — — 2 15 1 9 2 14 3 37

12 — — 1 5 2 10 2 7 2 16

Total 14 265 10 3,640 14 6,028 11 4,879 12 5,786

Note: SES = Supplemental Educational Services.

— Indicates that the district did not provide participation information.

Source: Hillsborough School District administrative data, 2004–05 through 2008–09.
Appendix F F–3 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.3

Percentage and Number of Students in Grades K–12 Participating in SES,

by Provider, in Hillsborough, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

District provider — — — — 16 944 17 809 18 1,055

Non-district provider A — — — — 1 69 — — — —

Non-district provider B — — — — 4 262 4 188 2 133

Non-district provider C — — — — — — — — 1 41

Non-district provider H — — 17 608 20 1,191 18 877 20 1,165

Non-district provider I 14 36 20 721 13 797 — — — —

Non-district provider J — — 3 117 13 765 11 554 17 977

Non-district provider K — — 8 292 5 320 4 198 2 130

Non-district provider L 18 48 8 303 5 294 4 183 4 219

Non-district provider M — — — — 5 293 6 284 — —

Non-district provider N — — 8 293 3 206 — — — —

Non-district provider O 5 14 5 178 2 148 1 31 — —

Non-district provider P — — 2 68 2 142 2 104 1 72

Non-district provider Q — — 1 39 2 118 1 39 <1 26

Non-district provider R — — 8 292 2 105 1 31 1 29

Non-district provider S — — — — 1 82 — — <1 10

Non-district provider T 3 7 2 65 1 64 2 76 1 73

Non-district provider U — — 2 67 1 66 1 35 — —

Non-district provider V — — 1 27 1 57 — — — —

Non-district provider W — — — — 1 42 — — — —

Non-district provider Y — — 3 95 1 31 — — — —

Non-district provider Z — — — — <1 21 1 27 <1 22

Non-district provider AA — — — — <1 5 — — — —

Non-district provider AB — — — — <1 4 — — — —

Non-district provider AC — — — — <1 2 <1 17 — —

Non-district provider AD 20 54 4 134 — — 1 35 — —

Non-district provider AE 10 27 2 61 — — — — — —

Non-district provider AF — — 2 55 — — — — — —

Non-district provider AG — — 1 51 — — — — — —

Non-district provider AH 18 48 1 53 — — — — — —

Non-district provider AI — — 1 37 — — — — — —

Non-district provider AJ 3 9 1 39 — — — — — —

Non-district provider AK — — 1 26 — — — — — —

Non-district provider AL — — <1 17 — — — — — —

Non-district provider AU 6 16 — — — — 1 56 — —

Non-district provider AZ — — <1 2 — — — — — —

Non-district provider CP — — — — — — — — 1 63

Non-district provider CQ — — — — — — — — 8 449

Non-district provider CR — — — — — — 21 1,006 9 543

Non-district provider CT — — — — — — — — <1 15

Non-district provider CU — — — — — — — — 1 72

Non-district provider CW — — — — — — — — <1 11

continued next page
Appendix F F–4 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.3 (continued)

Percentage and Number of Students in Grades K–12 Participating in SES,

by Provider, in Hillsborough, 2004–05 Through 2008–09

SES Provider

Students Participating in SES

2004–05 2005–06 2006–07 2007–08 2008–09

Percentage Number Percentage Number Percentage Number Percentage Number Percentage Number

Non-district provider CX — — — — — — 1 30 — —

Non-district provider CY — — — — — — — — 1 53

Non-district provider CZ — — — — — — <1 15 <1 15

Non-district provider DA — — — — — — <1 4 <1 26

Non-district provider DB — — — — — — — — 1 34

Non-district provider DC — — — — — — — — <1 6

Non-district provider DE — — — — — — — — 1 83

Non-district provider DG — — — — — — 2 98 5 289

Non-district provider DH — — — — — — 1 33 — —

Non-district provider DI — — — — — — 1 50 1 52

Non-district provider DJ — — — — — — — — 1 66

Non-district provider DK — — — — — — <1 10 — —

Non-district provider DL — — — — — — <1 23 — —

Non-district provider DM — — — — — — <1 8 <1 4

Non-district provider DN — — — — — — 1 36 1 48

Non-district provider DQ — — — — — — <1 22 — —

Non-district provider DR — — — — — — — — <1 5

Non-district provider FK 2 6 — — — — — — — —

Total 100 265 100 3,640 100 6,028 100 4,879 100 5,786

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Providers are referenced randomly by letters.

2004–05 data included only grades K–5. 

— Indicates that the provider did not serve SES participants.

Source: Hillsborough School District administrative data, 2004–05 through 2008–09.
Appendix F F–5 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.4

Number and Percentage of Students Participating in SES,

by Provider Type and by Grade, in Hillsborough, 2006–07 Through 2008–09

Grade

2006–07 2007–08 2008–09

Total 

Number of 

Students

Participating

in SES

Provider Type Total 

Number of 

Students

Participating

in SES

Provider Type Total 

Number of 

Students

Participating

in SES

Provider Type

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

Percentage 

Served by 

District 

Provider

Percentage 

Served by 

Non-District 

Providers

K 788 14 86 354 16 84 159 20 80

1 1,136 16 84 744 17 83 537 20 80

2 1,119 16 84 844 19 81 1,032 18 82

3 980 17 83 924 16 84 1,178 17 83

4 723 17 83 734 16 84 1,032 16 84

5 640 15 85 688 15 85 869 16 84

6 240 15 85 261 15 85 408 24 76

7 153 12 88 146 18 82 288 22 78

8 188 16 84 130 15 85 164 22 78

9 23 4 96 18 6 94 28 7 93

10 19 11 89 15 20 80 38 28 82

11 9 0 100 14 14 86 37 24 76

12 10 0 100 7 14 86 16 13 88

Total (n) 6,028 944 5,084 4,879 809 4,070 5,786 1,055 4,731

Total (%) 100 16 84 100 17 83 100 18 82

Note: SES = Supplemental Educational Services.

Source: Hillsborough School District administrative data, 2006–07 through 2008–09.
Appendix F F–6 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.5

Percentage Distribution of SES Eligible Students in Grades K–12, by Demographic Characteristics,

by Participation Status and by Provider Type, in Hillsborough, 2006–07 Through 2008–09

Demographic Characteristics

2006–07 2007–08 2008–09

All

Eligible

All 

Participants

Served by

District 

Provider

Served by 

Non-District 

Providers

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

Non-District 

Providers

All 

Eligible

All 

Participants

Served by

District 

Provider

Served by

Non-District 

Providers

Total n 42,048 6,028 944 5,084 46,013 4,879 809 4,070 49,727 5,786 1,055 4,731

Minority students (%) 83* 87* 85 87 84* 87* 84+ 88+ 84* 87* 84+ 87+

Percentage distribution by race/ethnicitya 

African American 35 33 36 32 35 38 35 38 34 38 35 39

Hispanic 43 48 45 49 43 44 42 44 43 43 43 43

White 17 13 15 13 16 13 16 12 1 13 16 13

Asian 1 1 1 1 1 1 1 1 1 1 1 1

American Indian <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1

Alaska Native — — — — — — — — — — — —

Multiracial 4 4 3 4 5 5 5 5 5 4 5 4

Total 100 100 100 100 100 100 100 100 100 100 100 100

Percentage of students with special needs

Students with limited English 

proficiency 27* 37* 34 38 27* 33* 35 33 26* 33* 35 32

Students with disabilities 18* 21* 23 21 18* 24* 24 24 18* 28* 29 28

Notes: SES = Supplemental Educational Services.

Minority students include African American, Hispanic, Asian, American Indian and multiracial students.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but greater than zero.

— Indicates that this demographic group was not identified by the district.

* Indicates a statistically significant difference between the percentage of eligible students and the percentage of participating students in the given demographic group

at the 0.05 level.

+ Indicates a statistically significant difference between the percentage of district SES participants and the percentage of non-district SES participants in the given 

demographic group at the 0.05 level. 

a The difference in the distribution of students by race/ethnicity was not tested for significance.

Source: Hillsborough School District administrative data, 2006–07 through 2008–09.
Appendix F F–7 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.6

Number of Eligible and Participating Students and SES Participation Rates

in Grades K–12, by Race/Ethnicity, in Hillsborough, 2006–07 Through 2008–09

Race/Ethnicity

Number of Eligible 

Students Number of Participants Participation Rate (%)

2006–07

African American 14,652 1,979 14

Hispanic 17,921 2,911 16

White 7,047 810 11

Asian 563 70 12

American Indian 78 9 12

Alaska Native — — —

Multiracial 1,787 249 14

Total 42,048 6,028 14

2007–08

African American 16,220 1,842 11

Hispanic 19,706 2,129 11

White 7,224 614 8

Asian 651 66 10

American Indian 70 6 9

Alaska Native — — —

Multiracial 2,142 222 10

Total 46,013 4,879 11

2008–09

African American 16,876 2,192 13

Hispanic 21,557 2,502 12

White 8,137 761 9

Asian 734 67 9

American Indian 69 10 14

Alaska Native — — —

Multiracial 2,354 254 11

Total 49,727 5,786 12

Note: SES = Supplemental Educational Services.

— Indicates that this demographic group was not identified by the district.

Source: Hillsborough School District administrative data, 2006–07 through 2008–09.
Appendix F F–8 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.7

Percentage and Number of SES Participants in Grades K–12,

by Subject and by Provider Type, in Hillsborough, 2006–07 Through 2008–09

Subject of Services 

Received

Students Served by

All Providers

Students Served by 

District Provider

Students Served by 

Non-District Providers

Percentage Number Percentage Number Percentage Number

2006–07

Mathematics and reading <1 27 1 5 <1 22

Mathematics only 21 1,252 30 287 19 965

Reading only 79 4,749 69 652 81 4,097

No information 0 0 0 0 0 0

Total 100 6,028 100 944 100 5,084

2007–08

Mathematics and reading 2 74 1 9 2 65

Mathematics only 24 1,169 23 188 24 981

Reading only 75 3,636 76 612 74 3,024

No information 0 0 0 0 0 0

Total 100 4,879 100 809 100 4,070

2008–09

Mathematics and reading 1 39 1 7 1 32

Mathematics only 29 1,659 26 275 29 1,384

Reading only 71 4,088 73 773 70 3,315

No information 0 0 0 0 0 0

Total 100 5,786 100 1,055 100 4,731

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

Source: Hillsborough School District administrative data, 2006–07 through 2008–09.
Appendix F F–9 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.8

Average Number of Hours of SES Received by Participants in Grades K–12,

by Provider Type, in Hillsborough, 2004–05 Through 2008–09

Year All Providers District Provider Non-District Providers

2004–05 25 — 25

2005–06 23 — 23

2006–07 22 22 23

2007–08 23 32* 22*

2008–09 22 29* 21*

Notes: SES = Supplemental Educational Services.

2004–05 data included only grades K–5. 

— Indicates that data were not available because the district did not provide SES at this time.

* Indicates a statistically significant difference in average hours between district and non-district providers at the 

0.05 level.

Source: Hillsborough School District administrative data, 2004–05 through 2008–09.
Appendix F F–10 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.9

Percentage Distribution of SES Participants in Grades K–12, by Number of Hours of 

Services Received and by Provider Type, in Hillsborough, 2004–05 Through 2008–09

Hours of Services 

Received

Students Served by All Providers Students Served by District Provider Students Served by Non-District Providers

Percentage Number Percentage Number Percentage Number

2004–05

Fewer than 20 hours 21 55 — — 21 55

20–39 hours 79 210 — — 79 210

40–59 hours 0 0 — — 0 0

60–79 hours 0 0 — — 0 0

80 or more hours 0 0 — — 0 0

Total 100 265 — — 100 265

2005–06

Fewer than 20 hours 38 1,387 — — 38 1,387

20–39 hours 55 2,009 — — 55 2,009

40–59 hours 6 232 — — 6 232

60–79 hours <1 12 — — <1 12

80 or more hours 0 0 — — 0 0

Total 100 3,640 — — 100 3,640

2006–07

Fewer than 20 hours 32 1,952 45* 422 30* 1,530

20–39 hours 63 3,824 46* 434 67* 3,390

40–59 hours 4 251 9* 87 3* 164

60–79 hours <1 1 <1* 1 0* 0

80 or more hours 0 0 0 0 0 0

Total 100 6,028 100 944 100 5,084

2007–08

Fewer than 20 hours 30 1,487 24* 197 32* 1,290

20–39 hours 64 3,139 47* 381 68* 2,758

40–59 hours 4 190 21* 168 1* 22

60–79 hours 1 63 8* 63 0* 0

80 or more hours 0 0 0 0 0 0

Total 100 4,879 100 809 100 4,070

2008–09

Fewer than 20 hours 32 1,861 35* 372 31* 1,489

20–39 hours 62 3,616 39* 410 68* 3,206

40–59 hours 4 259 21* 223 1* 36

60–79 hours 1 48 5* 48 0* 0

80 or more hours <1 2 <1* 2 0* 0

Total 100 5,786 100 1,055 100 4,731

Notes: SES = Supplemental Educational Services.

Percentages may not total 100 due to rounding. Percentages less than one (<1) are less than 0.5 percent but 

greater than zero.

2004–05 data included only grades K–5. 

— Indicates that data were not available because the district did not provide SES at this time.

* Indicates a statistically significant difference between the percentage of students receiving the specified hours of SES 

with the district provider and the percentage of students receiving the specified hours of SES with the non-district 

providers at the 0.05 level.

Source: Hillsborough School District administrative data, 2004–05 through 2008–09.
Appendix F F–11 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.10

Average Prior (2007–08) Achievement Scores for Students in Grades 3–10,

by 2008–09 Eligibility and Participation Status and by Subject, in Hillsborough

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Students

Average Prior 

Achievement 

Score

Number of 

Students

Eligibility

Ineligible students 0.13* 83,044 0.13* 83,197

Eligible students –0.40* 23,299 –0.44* 23,317

Participation status

Participants –0.80+ 2,700 –0.84+ 2,706

Eligible nonparticipants –0.35+ 20,599 –0.38+ 20,611

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between ineligible students and eligible students at the 0.05 level.

+ Indicates a statistically significant difference between participants and eligible nonparticipants at the 0.05 level.

Source: Hillsborough School District administrative data, 2007–08 and 2008–09.

Exhibit F.11

Average Prior (2007–08) Achievement Scores for Participating Students in Grades 3–10,

by 2008–09 Provider Type and by Subject, in Hillsborough

Provider Type

Mathematics Reading

Average Prior 

Achievement 

Score

Number of 

Participants

Average Prior 

Achievement 

Score

Number of 

Participants

District provider –0.82 477 –0.87 480

Non-district providers –0.80 2,223 –0.83 2,226

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

* Indicates a statistically significant difference between district and non-district providers at the 0.05 level.

Source: Hillsborough School District administrative data, 2007–08 and 2008–09.
Appendix F F–12 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.12

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–10, Overall and for a Single Year and Multiple Years,

in Hillsborough, 2003–04 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Participation

Overall 0.12*

(0.02) 1,658 0.06*(0.02) 2,441

Years of participation

Students with one year of SES 

participation

0.13*

(0.03) 1,283 0.07*(0.02) 1,870

Students with multiple years of SES 

participation

0.08

(0.05) 375 0.06(0.04) 571

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,595 for mathematics and 405,995 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.

Exhibit F.13

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–10, by Provider Type,

in Hillsborough, 2003–04 Through 2008–09

Provider Type

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

District provider 0.11*

(0.05) 293 0.10(0.07) 281

Non-district providers 0.13*

(0.03) 1,342 0.06*(0.02) 2,129

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,571 for mathematics and 405,963 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Appendix F F–13 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.14

Difference in Achievement Gains Associated With SES Participation

for Students Served by District and Non-District Providers,

in Grades 3–10, by Subject, in Hillsborough, 2003–04 Through 2008–09

Subject

Difference in Gain:

District Minus Non-District

(SE)

Number

Mathematics –0.01

(0.06) 1,635

Reading 0.04

(0.07) 2,410

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,571 for mathematics and 405,963 for 

reading.

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Appendix F F–14 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.15

Mathematics and Reading Achievement Gains Associated With SES Participation, 

Adjusted for Hours of SES, for Students in Grades 3–10,

by Provider Type, in Hillsborough, 2003–04 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

Provider type

District provider –0.01

(0.07) 293 0.08(0.08) 281

Non-district providers 0.03

(0.05) 1,342 0.04(0.04) 2,129

Hours

Gain adjusted for hours

0.05*

(0.02) 1,635 0.01(0.02) 2,410

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,571 for mathematics and 405,963 for 

reading.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The measure for hours was coded in units of 10 hours. 

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough Public Schools administrative data, 2002–03 through 2008–09.
Appendix F F–15 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.16

Difference in Achievement Gains Associated With SES Participation

for Students in Grades 3–10 Served by District and Non-District Providers,

Adjusted for Hours of SES, by Subject, in Hillsborough, 2003–04 Through 2008–09

Subject

Difference in Gain: 

District Minus Non-District

(SE)

Number

Mathematics –0.04

(0.06) 1,635

Reading 0.04

(0.07) 2,410

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider.

The number column indicates the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,571 for mathematics and 405,963 for 

reading.

The measure for hours was coded in units of 10 hours. 

* Indicates that the difference in achievement gains associated with district and non-district providers was statistically 

significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Appendix F F–16 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.17

Mathematics Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–10,

by Individual Provider, in Hillsborough, 2003–04 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number

District provider 0.11*

(0.05) 26 293

Non-district provider H 0.18*

(0.07) 23 224

Non-district provider CR 0.24*

(0.08) 19 178

Non-district provider L 0.07

(0.08) 18 116

Non-district provider K 0.23*

(0.09) 21 106

Non-district provider J 0.11

(0.11) 20 103

Non-district provider I 0.13

(0.09) 19 102

Non-district provider Q 0.10

(0.12) 23 52

Non-district provider N 0.06

(0.11) 31 37

Other providers 0.06

(0.04) 20 447

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in mathematics. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,595.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are reference by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Appendix F F–17 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.18

Reading Achievement Gains Associated With SES Participation

and Average Hours of SES Participation, for Students in Grades 3–10,

by Individual Provider, in Hillsborough, 2003–04 Through 2008–09

SES Provider 

Gain

(SE)

Average Hours per 

Student Number

Non-district provider H 0.03

(0.06) 25 336

Non-district provider J 0.08

(0.06) 22 328

District provider 0.10

(0.07) 25 281

Non-district provider K 0.03

(0.07) 23 170

Non-district provider I 0.08

(0.08) 18 166

Non-district provider CR 0.01

(0.08) 19 154

Non-district provider M –0.05

(0.09) 18 146

Non-district provider L –0.01

(0.07) 17 115

Non-district provider N 0.11

(0.19) 31 59

Non-district provider Q 0.15

(0.21) 23 50

Other providers 0.10*

(0.04) 22 636

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The “other providers” category includes non-district providers serving fewer than 30 students in reading. 

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 405,995.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Providers are referenced by randomly assigned letters.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Appendix F F–18 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit F.19

Mathematics and Reading Achievement Gains Associated With SES Participation,

for Students in Grades 3–10, by Academic Year,

in Hillsborough, 2005–06 Through 2008–09

Mathematics Reading

Gain

(SE) Number Gain(SE) Number

2005–06 gain 0.12

(0.06) 182 0.06(0.04) 599

2006–07 gain 0.10*

(0.04) 502 0.06(0.04) 713

2007–08 gain 0.11*

(0.05) 483 0.04(0.05) 680

2008–09 gain 0.18*

(0.05) 468 0.12*(0.05) 418

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

The number column includes the number of achievement gain scores that contributed to the estimate.

The total number of achievement gain scores in the analysis was 404,595 for mathematics and 405,995 for 

reading. 

All models controlled for student eligibility, grade, year and grade-by-year interactions.

* Indicates that the gain was statistically significant at the 0.05 level.

Source: Hillsborough School District administrative data, 2002–03 through 2008–09.
Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX G:

COMPARISON OF FULL AND PARTIAL SAMPLES 

ON SES PROVIDER TYPE ACHIEVEMENT GAINS

Appendix G G–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix G: Comparison of Full and Partial Samples

on SES Provider Type Achievement Gains

This appendix describes sensitivity tests conducted to examine how different choices of the analysis 

sample would affect the achievement gains associated with SES participation, relative to 

nonparticipation. The gains reported in the body of the report and appendices are based on all students 

in the district with at least two achievement gain scores. Although this approach makes maximal use of 

the available data, many of the students in the full sample either never participated in SES or participated 

in all years for which data are available. Although these students contribute to the estimation of many 

variables in the model, they do not contribute to the estimated impact of SES participation. Thus, a 

reasonable alternative approach would be to estimate the achievement gains associated with SES 

participation by restricting the sample to only students who participated in SES in some years but not 

others. 

To examine how the sample choice affects the achievement gains associated with SES participation, we

assessed the gains by provider type based on both the full sample and the restricted sample. The two 

samples included the following students:

1. Full Sample—All students in each district with at least two achievement gain scores. 

2. Restricted Sample—Only students who participated in SES in some years and not in other 

years who also had at least two achievement gain scores.

In a simple model with SES participation as the only independent variable, achievement gains associated 

with SES participation, relative to nonparticipation, will be the same for both full-sample and restrictedsample analyses. This situation is the case because in both models only students with at least one period 

of participation and one period of nonparticipation will contribute to the estimates. However, the 

models used for this evaluation included additional variables: eligibility, grade, year and grade-by-year 

interactions. The estimated coefficients of these variables may differ for the full and restricted samples, 

and these differences may influence the estimated gains associated with SES participation. 

Specifically, the estimated achievement gains from full-sample analyses and restricted-sample analyses are 

similar in most cases, although the analyses based on the restricted sample did not generate as many 

significant findings (see Appendix Exhibits G.1 and G.2). The one exception was Boston, where findings 

based on the two samples differed. On the basis of the full sample, SES participation was not associated 

with statistically significant achievement gains in Boston for either provider type. On the basis of the 

restricted sample, however, for students served by the district provider, SES participation was associated 

with statistically significant achievement gains in mathematics and all other gains remained not 

statistically significant. Therefore, the results for Boston should be interpreted with caution because they 

are sensitive to the choice of the analytic sample on which the estimates are based.
Appendix G G–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit G.1

Mathematics Achievement Gains Associated With SES Participation,

by School District, by Provider Type and by Sample Type

District Provider Non-District Providers Number of Students

Full 

Sample

Restricted 

Sample

Full 

Sample

Restricted 

Sample

Full 

Sample

Restricted 

Sample

Anchorage 0.05 0.09 0.10 0.17 28,155 312

Boston 0.04 0.08* 0.05 0.02 12,022 2,031

Charlotte-Mecklenburg 0.23* 0.23 0.10 0.10 47,005 701

Chicago 0.06* 0.06* 0.04* 0.05* 220,419 48,403

Hillsborough 0.11* 0.12* 0.13* 0.14* 113,169 1,545

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg; 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.

Exhibit G.2

Reading Achievement Gains Associated With SES Participation,

by School District, by Provider Type and by Sample Type

District Provider Non-District Providers Number of Students

Full 

Sample

Restricted 

Sample

Full 

Sample

Restricted 

Sample

Full 

Sample

Restricted 

Sample

Anchorage 0.10 0.07 0.00 –0.02 28,141 321

Boston 0.03 0.03 0.00 –0.03 11,774 2,002

Charlotte-Mecklenburg 0.04 0.07 0.02 0.02 47,112 1,333

Chicago 0.03* 0.04* 0.09* 0.09* 220,786 48,517

Hillsborough 0.10 0.07 0.06* 0.05 113,481 2,201

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg; 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.
Supplemental Educational Services and Student

Achievement in Five Waiver Districts

APPENDIX H:

AVERAGE ACHIEVEMENT GAINS ASSOCIATED 

WITH SES PARTICIPATION

ACROSS FIVE DISTRICTS

Appendix H H–1 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Appendix H: Average Achievement Gains Associated With 

SES Participation Across Five Districts

This evaluation is a multisite study in which each of the five districts constitutes a study site and 

achievement gains associated with SES participation were assessed within each individual site. Once the 

site-specific gains were obtained, it was possible to calculate average achievement gains across the 

five districts. Two alternative approaches for combining effects across study sites are the fixed-effects 

approach and the random-effects approach. These two approaches are based on different assumptions 

about the nature of the study sample and the variation in gains across sites and, therefore, have different 

implications for the analytic method used for combining gains. 

Under the fixed-effects approach, each study site is regarded as a unique entity (for example, as in a 

purposive sample), and the goal of the study is to understand the treatment effect just for the sites 

included in the particular study sample rather than to generalize study findings to a larger identifiable 

population of sites. Under the assumption that each unique site has its own unique treatment effect, an 

appropriate estimator of the overall gain in the fixed-effects approach is the simple mean of the sitespecific gains, with equal weight for each site.1 In contrast, under the random-effect approach, the sites 

in a study are considered a sample of sites randomly drawn from a larger identifiable population, and the 

goal of the study is to generalize the treatment effect observed based on the study sites to the larger 

population of all possible sites. Viewing each site as a random draw from a certain population, an 

appropriate estimator of the overall gain (that is, the true population gain) under the random-effects 

approach is the precision-weighted average of site-specific gains, with larger weights given to sites with 

more precise estimates. 

Given that this evaluation included only five study sites (districts), and given the uniqueness of each 

district, it is not appropriate to construe these particular five districts as a representative sample of a 

larger, clearly defined population of districts. Therefore, the reported average achievement gains across 

the five districts are based on the fixed-effects approach rather than on the random-effects approach (see 

Exhibit H.1). This analytic decision is consistent with Schochet’s argument that fixed-effects models are 

usually more realistic for evaluations of education interventions because the sites in most multisite 

studies are limited in number, purposively selected, and not representative of a larger, well-defined 

population.2 A previous U.S. Department of Education study used the random-effects approach to 

combine district-specific effects of SES across districts, but that study included more districts and used 

different site selection criteria.3

 1. H. S. Bloom, “Using ‘Short’ Interrupted Time-Series Analysis to Measure the Impacts of Whole-School Reforms: 

With Applications to a Study of Accelerated Schools,” Evaluation Review 27 (2003): 3–49.

2. P. Z. Schochet, “Statistical Power for Random Assignment Evaluations of Education Programs. Journal of 

Educational and Behavioral Statistics 33, no. 1 (2008): 62–87.

3. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, State and Local Implementation of the No Child Left Behind Act: Volume I—Title I School Choice, Supplemental 

Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, 2007).
Appendix H H–2 Supplemental Educational Services and Student

Achievement in Five Waiver Districts

Exhibit H.1

Mathematics and Reading Achievement Gains Associated With SES Participation

Across Five Districts, Overall, by Provider Type and Differences for Students Served

by District and Non-District Providers

Gains Associated With SES Participation 

Average Gain Across Districts

(SE) 

Mathematics Reading

Overall 0.08*

(0.02)

0.04*

(0.01)

District provider 0.10*

(0.03)

0.06*

(0.02)

Non-district provider 0.09*

(0.02)

0.03

(0.02)

Differences between district and non-district provider gains 0.01

(0.04)

0.03

(0.03)

Notes: SES = Supplemental Educational Services.

Achievement scores are z-scores standardized within year, grade level and district.

All models controlled for student eligibility, grade, year and grade-by-year interactions.

Differences in achievement gains were calculated by subtracting the achievement gains associated with 

non-district providers from the achievement gains associated with the district provider. 

The grades included in analyses vary by district: grades 3–8 in Boston, Chicago and Charlotte-Mecklenburg; 

and grades 3–10 in Anchorage and Hillsborough. Not all grades are included in all years for Anchorage and 

Boston due to changes in tested grades during the study years.

* Indicates that the gain was statistically significant at the 0.05 level. 

Source: Administrative data provided by the Chicago school district, 2001–02 through 2008–09; the Boston and 

Hillsborough school districts, 2002–03 through 2008–09; the Anchorage school district, 2003–04 through 

2008–09; and the Charlotte-Mecklenburg school district, 2004–05 through 2008–09.