1 

Breakthrough_17AC189783

Breakthrough 

Impact Study for Evidence Base 

Grant Application ID: 17AC189783 

BREAKTHROUGH AUSTIN 

IMPACT STUDY 

Catherine A. Malerba and Allison G. Caplovitz 

April 30, 2011 
   

   

Breakthrough_17AC189783

Table of Contents 

Executive Summary 3
 

Introduction 5
 

........................................................................................................................... 

........................................................................................................................................ 

................................................................................................................................................ 

................................................................................................. 

........................................................................................................................................ 

........................................................................................................................................... 

................................................................................................................ 

............................................................................................................. 

............................................................................................................................ 

.............................................................................................................................................. 

......................................................................................................................................... 

...................................................................................................................... 

....................................................................................................................................... 

Breakthrough Austin Impact Study 7
 

Method 8
 

Participants 8
 

Measures 13
 

Dependent Variables 13
 

Independent Variables 14
 

Analysis Plan 16
 

Results 17
 

Discussion 30
 

Program Implications 30
 

Limitations 31
 

References......................................................................................................................................... 

.................................................................................................... 

........................................................................................................................................................... 

34
 

Appendix A.  Correlation Matrix 37
 

38
 

Appendix B. Incremental Impact of Breakthrough (BT) Participation by Magnet Participation
 

2
   

       

Breakthrough_17AC189783

EXECUTIVE SUMMARY
 

Breakthrough Austin is a college preparation program for low­income students who will 

be the first in their families to graduate from college. Sixth­grade students are selected for 

participation based on their grades, state assessment performance, regular school attendance, 

and teacher recommendations. The program begins serving students during the summer before 

7th grade through a rigorous academic bridge program and continues a multi­year commitment 

to students by providing them the academic and social support they need to graduate from high 

school and enroll in college.  

The present evaluation study examined three cohorts of Breakthrough program 

students to determine, (a) if there is evidence of program selectivity when comparing 

Breakthrough participants to other students who attended the same 6th grade campuses, (b) 

Breakthrough students’ likelihood of graduating from high school and enrolling in college after 

accounting for their demographic, academic, and behavioral characteristics, and (c) the 

incremental impact of Breakthrough participation after accounting for these same 

characteristics. 

The main results are as follows: 

Program selectivity 

>

>	 

>	 

Breakthrough participants were similar to other students from the same 6th grade 

campuses according to their ethnicity and economic disadvantage. 

Breakthrough students were more likely to be female, to be in magnet programs and to 

pass the reading and math state assessments than other students from the same 6th

grade campuses. Breakthrough students also had higher average grades in reading and

math and better daily attendance than other students.

Breakthrough students were less likely to participate in special education programs 

and less likely to have a disciplinary referral than other students from the same 6th

grade campuses. 

3
   

   

   

  

>	 

>	 

>	 

Breakthrough_17AC189783

Likelihood of high school graduation and college enrollment

>	 

>	 

After accounting for student characteristics, Breakthrough students were 3.5 times 

more likely to graduate from high school, rather than drop out, in comparison to other

students from the same 6th grade campuses.

After accounting for student characteristics, Breakthrough participants were 1.98 

times more likely to enroll in college, rather than not be enrolled, in comparison to

other students from the same 6th grade campuses.

Incremental impact on high school graduation and college enrollment

Among students with a propensity to be served by the program, Breakthrough has the 

largest incremental impact on high school graduation (+18%) among students with

slightly lower academic achievement and attendance rates, and slightly higher rates of

disciplinary referrals than other students.

Among students with a propensity to be served by the program, Breakthrough has the 

largest incremental impact on college enrollment (+63%) among students who met 6th

grade academic achievement standards, who had good attendance and good behavior, 

but who were not served by district magnet programs.

Among students with a propensity to be served by the program who also were served by 

district magnet programs, Breakthrough program participation had a little impact on

high school graduation (+3%); however, participation had a modest positive impact on

college enrollment (+16%).

4
Breakthrough_17AC189783

INTRODUCTION

Regardless of socioeconomic status (SES), children who have strong academic 

backgrounds are more likely to graduate from high school and enroll in and succeed in college 

then those that do not (National Center for Educational Statistics, 2001). It is also the case 

that students from low­income families often lack the social capital needed to navigate the 

complicated road of seeking academic support, selecting rigorous high school courses, applying 

to college, seeking and obtaining financial aid, and other aspects of preparing for postsecondary 

education (Coleman, 1988; Cabrera & La Nasa, 2000; Cabrera & La Nasa, 2001). 

 Breakthrough Austin’s (Breakthrough) mission is to provide a path to college for lowincome students who will be the first in their families to graduate from college. Breakthrough 

provides students the extensive personal and academic supports they need to strive for 

academic excellence, to graduate from high school, and to apply for and enroll in college.  The 

program model comprises a suite of research­based services that promote student success 

including early intervention, case management, and academic supports outside of school. 

Early intervention. In contrast to college­preparation programs that begin in high 

school (Dynarski, Gleason, Rangarajan, & Wood, 1998) Breakthrough begins program services 

in middle school. Early intervention often is necessary for low­income students, because even 

those who appear to be making adequate academic progress in elementary and middle school 

according to their grades and meeting minimum standards on state assessments may, in fact, 

not be on track to acquire the skills needed to be successful in rigorous high school coursework.  

There is considerable evidence that middle school intervention programs that provide 

extensive academic supports, particularly in reading and writing, are highly effective at 

reducing the likelihood of truancy, grade level retention, and dropout (Schargel & Smink, 

2001). In contrast, intensive high school programs often do not show the same effects (Dynarski 

et al, 1998). Breakthrough programs begin immediately after the 6th grade with a rigorous 

5
Breakthrough_17AC189783

summer academic enrichment program designed to address any skill deficits that students may 

have accumulated in the core academic areas of English, math, science, and social studies.  

Case management. Upon entering Breakthrough, each student is assigned a case 

manager, another best practice to help low­income students succeed academically and to 

graduate from high school (Dynarshi, Clark, Cobb, Finn, Rumberger & Smink, 2008). 

Breakthrough case managers support students in several important ways. First, they serve as 

an adult advocate within the school system, for example, by advising them to register for 

rigorous coursework. They also play an important role in monitoring students’ daily school 

attendance and grades, and holding students accountable for their schoolwork.  

Through their frequent interaction with students, case managers have many 

opportunities to model positive behavior and long­term, goal­oriented decision­making skills 

that are often in contrast to those modeled by peer groups at school. Case management support 

has been consistently associated with increased school attendance rates and improved course 

grades among low­income and ethnic minority students (Dynarshi, Clark, Cobb, Finn, 

Rumberger & Smink, 2008; Larson & Rumberger, 1995; Sinclair, Christianson, Evelo, & 

Hurley, 1998). 

Academic support out of school. Through Saturday and after school tutoring, 

Breakthrough directly addresses students’ academic skill deficits and supports ongoing 

academic success. There is substantial evidence that when tutoring programs are aligned 

academically with the school day and when they provide a sufficient time investment, i.e., 45 

hours or more of tutoring each academic year, participating students are more likely to 

graduate from high school and enroll in college than students who do not receive the out of 

school tutoring support (Beckett, Borman, Capizzano, Parsley, Ross, Schirm, & Taylor, 2009; 

Constantine, Seftor, Martin, Silva & Myers, 2006; Lauer, Akiba, Wilkerson, Apthorp, Snow, & 

Martin­Glenn, 1995). 

6
   

Breakthrough_17AC189783

BREAKTHROUGH AUSTIN IMPACT STUDY

Breakthrough is a college preparation program for low­income students who will be 

first generation college graduates. Sixth­grade students apply to Breakthrough and are 

selected for participation based on their grades, state assessment performance, regular school 

attendance, and teacher recommendations. The program begins serving students during the 

summer before 7th grade through a rigorous academic bridge program and continues a multiyear commitment to students by providing them the academic and social support they need to 

graduate from high school and enroll in college. Breakthrough Austin admitted its first class in 

the summer of 2002, and currently serves over 350 students in grades seven through twelve, 

across several Central Texas school districts, in addition to the three classes of high school 

graduates. Breakthrough Austin’s program goals are for at least 90% of participating students 

to graduate from high school and for at least 80% of them to enroll directly in college. 

The purpose of the present evaluation study is to estimate the impact of Breakthrough 

program participation on high school graduation rates and college enrollment after accounting 

for students’ pre­participation demographic, academic, and behavioral characteristics. Because 

of the logistical challenges associated with obtaining approval to conduct research and 

evaluation projects using school district data and because the majority of students served by 

Breakthrough during the 2001­2002, 2002­2003, and 2003­2004 school years were enrolled in 

the Austin Independent School District (AISD), the present study is limited to program and 

comparison students who were enrolled in AISD.  

The three research questions that this report seeks to answer are:

Phase I. Selectivity Analyses: 

Are Breakthrough students significantly different from other students at the same 6th

grade campuses before they begin the program?

7
Breakthrough_17AC189783

Phase II.  Likelihood and Incremental Impact Analyses: 

A. What is the likelihood of high school graduation and college enrollment among

Breakthrough students in comparison to other students after controlling for 6th grade

demographic, academic, and behavioral characteristics?

B. What is the incremental impact of Breakthrough participation on high school

graduation and college enrollment? 

METHOD

Because it was not possible to retroactively design a random assignment experiment to 

determine the impact of Breakthrough participation on students’ likelihood of graduating from 

high school and enrolling in college, and because of the logistical and ethical challenges 

associated with this research method, several non­experimental methods were used to estimate 

program effects after accounting for as many 6th grade student characteristics as possible. 

Participants

The participants in this study were the 3487 students who were enrolled in 6th grade in 

the Austin Independent School District (AISD) across the 2001­2002 (Cohort 1), 2002­2003 

(Cohort 2) and 2003­2004 (Cohort 3) school years (Table 1). Within the study population, 131 

students were selected to participate in Breakthrough programs either at the end of 6th grade 

(n = 122) or the end of 7th grade (n = 9). The remaining students (n = 3356) comprise all other 

students who were enrolled in 6th grade at the same campuses, during the same cohort years, 

as the Breakthrough participants. Across the three cohorts, 718 students (21%) attended 6th

grade on an elementary campus and the remainder, 2769 students (79%), attended 6th grade on 

a middle school campus. 

Table 1 shows the number of Breakthrough participants and comparison students by 

campus and 6th grade cohort year. The full population was used in the selectivity analyses 

(Phase I); however, only the subsample of students who either graduated from or dropped out 

of Austin ISD was used to address the evaluation questions concerning the likelihood of 

8
Breakthrough_17AC189783

graduation and college enrollment and the incremental impact of program participation. The

analysis subsample is described in the next section.

9
 
Cohort 1:  Cohort 2:  Cohort 3: 

2001­2002  2002­2003  2003­2004  Total 

Campus  BT  Other  BT  Other  BT  Other 

Allison

Elementary

6 43 3 46 5 36 139

Blackshear

Elementary

3 46 49

Brown

Elementary

2 14 16

Campbell

Elementary

2 82 1 60 145

Dobie

Middle School

11 290 301

Fulmore

Middle School

10 346 20 339 10 310 1035

Govalle

Elementary

3 36 39

Lee

Elementary

1 50 51

Maplewood 

Elementary

2 41 1 41 85

Martin

Middle School

5 117 8 219 349

Mendez 

Middle School

4 303 307

Metz 

Elementary

9 69 8 65 151

Pearce

Middle School

6 266 272

Webb

Middle School

2 247 1 255 505

Zavala

Elementary

8 35 43

Grand 

Total

42 1298 45 1031 44 1027 3487

Breakthrough_17AC189783

Table 1. Number of Breakthrough (BT) and Comparison (Other) students who 

were enrolled in Austin Independent School District, by 6th grade cohort year.

Note. Sixth grade students from Cohorts 1, 2, and 3 were expected to graduate from high school in

the Spring of 2008, 2009, and 2010 respectively.  

10
 
Breakthrough_17AC189783

Records used in the likelihood of high school graduation, college enrollment, 

and incremental program impact analyses (Phase II). To answer these evaluation 

questions, the number of students available for analysis was reduced; only students who 

remained in AISD until they graduated or dropped out were included in these analyses (n = 

1991). Students who left the district for any other reason, e.g., transferred to another school 

district, returned to their home country, etc., or for whom there was no clear leave reason 

specified in district records were excluded from these analyses.  

Also for this set of analyses, Breakthrough participation was refined to include only 

students who remained enrolled in AISD and continued to participate in the program until 

they graduated or dropped out of AISD, i.e., they received full program treatment. Students 

who remained in AISD but who withdrew from Breakthrough for any reason were considered 

non­program participants in the high school graduation, college enrollment, and incremental 

program impact analyses. Although Breakthrough staff continued to provide services to 

program students if they left AISD, the analysis sample necessarily was limited to 

Breakthrough and comparison students for whom there were AISD administrative records; the 

final outcomes for all non­AISD students are censored within the context of this evaluation. 

Redacted records. As of the 2008­2009 school year, AISD has offered families the 

option to opt­out of sharing deidentified student records for external research and evaluation 

projects. If parents or guardians desire to opt­out of data sharing, they complete the 

appropriate form and return it to the district along with other student registration forms. This 

preference is then entered into the student information system, so district staff can identify 

which records to withhold from all data requests. These opt­out requests are tied to records 

created during the school year for which the student is enrolled and therefore may change from 

year to year. All Breakthrough student families provide active consent to share education 

11
   

   

Breakthrough_17AC189783

records upon enrollment in the program, so no Breakthrough student records were redacted 

because of the external research opt­out provision. 

As shown in row 3 of Table 2, there were 167 comparison students whose final outcome 

data was redated due to the external research opt­out provision, which accounts for about 9% of 

the Phase II comparison group. The vast majority of the redacted records (n = 144) were for 

students who were 12th graders in 2009­2010; therefore, it is very likely that their final district 

leave reason was graduation. This systematic pattern of missing data must be taken into 

account when interpreting the results of this evaluation, the most likely result being a slight 

overestimation of Breakthrough program effects on high school graduation.  

Table 2. Number of Breakthrough and comparison students in the high school 

graduation, college enrollment, and program impact analyses by final leave reason 

from Austin Independent School District (AISD) 

Final outcome 

Breakthrough 

Active 

Deactivated 

Other 

students  Total 

with 

Breakthrough 

Participants  Comparison students 

Graduation 85 4* 1258 1347

Dropout 3 2* 472 477

Leaver reason

redacted

0 0 167 167

Total Phase II

Analysis sample

88 1903 1991

Withdrew from

AISD

21 16 1231 1268

Leave reason

unclear

0 0 228 228

Total censored

cases

21 1475 1496

Grand total 109 22 3356 3487

Note. *Students who deactivated with Breakthrough but remained in AISD were considered

non­participants in the high school graduation, college enrollment, and incremental impact

analyses.

12
Breakthrough_17AC189783

Measures 

Dependent Variables 

High school graduation. Students were identified as high school graduates in AISD 

records if they met the Texas Education Agency (TEA) criteria for graduation at any time 

during the requested school years, i.e., August 2001 through August 2010. Evaluation 

questions regarding “on­time” graduation were not addressed in this study because of different 

definitions of on­time that are used by TEA and Breakthrough Austin. Texas public school 

students who meet graduation requirements by August following their 12th grade year are 

considered part of their 4­year graduation cohort, whereas Breakthrough considers on­time 

graduation May or June of the senior year. Of the 109 Breakthrough students included in the 

graduation v. dropout analyses, there was 100% correspondence between district and program 

records with regard to student graduation. For the Phase II analyses, students who met state 

graduation requirements were coded as graduate = 1. If the final leave reason was redacted, 

graduate was coded as missing. 

High school dropout. Students were identified dropouts in the analysis data set if the 

last leaver reason in AISD records was dropout and the district records included a dropout date 

(month and year). If district records had a dropout indicator, but no associated date, the final 

status for the purpose of this study was considered unclear and the student records were 

removed (n = 228); the result of which may be an underestimate of Breakthrough program 

impact on graduation in comparison to dropout. Of the 109 Breakthrough students included in 

the graduation v. dropout analyses, there was 100% correspondence between district and 

program records with regard to student dropout. For the Phase II analyses, these students who 

dropped out were coded as graduate = 0.  

College enrollment. Students were identified as college enrollees in the Phase II data 

set according to AISD college enrollment records through December 2010. The source of AISD

college enrollment information is the National Student Clearinghouse (NSC). For the Phase II

13
   

   

Breakthrough_17AC189783

analyses, students were coded as enrolled (1) or not enrolled (0); students with missing data on

college enrollment were excluded from these analyses.

Of the 109 students Breakthrough students included in these analyses, there was 57% 

correspondence between district and program records with regard to college enrollment. This

disparity is due, in part, to the fact that Breakthrough Austin remains in direct contact with

students after graduation and therefore has more up­to­date information on student

enrollment status than the district. Although the NSC database is fairly inclusive, it does not

include data from every college, university, and technical school in the US, including several

large universities in Texas. Both AISD and Breakthrough records indicate the type of college in

which students enrolled, i.e., 2­year, 4­year; however, no analyses were conducted regarding

the type of college enrollment due to concerns about data quality and completeness.

Independent Variables

Breakthrough program participation (Phase I). Students who were accepted as 

Breakthrough participants (1) during their 6th or 7th grade school year were coded as ‘1’ in the 

selectivity analysis. All other students were coded as comparison students (0). 

Full Breakthrough program participation (Phase II). Students who remained 

active Breakthrough participants (1) until they graduated or dropped out of AISD were coded 

‘1’ in the likelihood and incremental impact analyses. All other AISD graduates and dropouts, 

including deactivated Breakthrough students were coded as comparison students (0).

Gender. Students were identified as male or female by their parent or guardian upon 

enrollment in the district. In the analysis data sets, male = 1 and female = 0 

Ethnicity. Parents or guardians specified student ethnicity upon enrollment in the 

district. The analysis data sets contains three dummy codes based on TEA race/ethnicity 

variables in use during these students’ 6th grade year: Hispanic, African American, or other 

(the aggregation of Native American, Asian, and White)

14
Breakthrough_17AC189783

Economic disadvantage. Students were classified as economically disadvantaged if 

their parent or guardian applied for and the student was approved to receive free or reducedpriced lunch according to criteria set by the US Department of Agriculture during the students 

6th grade year. In the analysis data sets, economic disadvantage = 1 and not economically 

disadvantaged = 0. 

Magnet program participation. Magnet participation is the only independent 

variable that was collected as of the students’ 7th grade year, i.e., after Breakthrough program 

admission, because 6th grade magnet participation data was not available for these cohort 

school years. All magnet students from these analysis cohorts were enrolled in the specialized 

academic program at Fulmore Middle School. In the analysis data sets, magnet student = 1 and 

non­magnet student = 0. 

Special education. Students were identified as special education students if they were 

enrolled in special education programs or in general education programs that used special 

education support services during their 6th grade year. In the analysis data sets, special 

education = 1 and general education = 0.  

Texas state academic proficiency assessments in reading and math. Students 

from the first two cohorts included in this study were tested with the Texas Assessment of 

Academic Skills (TAAS) as 6th graders; however, students from the third cohort were tested 

using the Texas Assessment of Knowledge and Skills (TAKS). Because scale scores on the two 

assessments are not directly comparable, we examined student academic performance in 

reading and math according to whether the student met the passing standard the school year 

they were tested. In the analysis data sets, met passing standard = 1 and did not meet the 

passing standard = 0. 

Course grades in reading and math. The district provided the requested final course 

grades records, on a 0­100% scale, in reading and math during the 6th grade year; however, at 

the time these three cohorts of students were enrolled, AISD elementary campuses did not 

15
   

Breakthrough_17AC189783

maintain electronic grade records. Final reading and math grades are included in the 

selectivity analyses (Phase I), but because grade reports were missing for all students enrolled 

in 6th grade on elementary campuses, student grades are not included in the likelihood 

analyses or incremental impact analyses to avoid the systematic loss of a large number of 

cases. 

Disciplinary event. Students have an indicator for whether they had one or more TEA 

required disciplinary referrals during their 6th grade year. In the analysis data sets, student 

had one or more discipline referrals = 1 and no disciplinary referrals = 0. Campus discretionary 

discipline referrals are not included in these analyses because referral rates are known to vary 

from campus to campus. 

Annual attendance rate. The 6th grade annual daily attendance rate was calculated 

by dividing the number of days each students attended school by the number of days each 

student was enrolled in the district. 

Analysis Plan 

Phase I – Selectivity.  To address the evaluation question regarding Breakthrough 

program selectivity, we used analysis of variance (ANOVA) for continuous variables and 

Pearson chi­square for dichotomous variables to determine if Breakthrough students differed 

from other 6th grade students on the same campuses before they enrolled in the program. The 

analysis included a range of 6th grade demographic, academic achievement, and behavioral 

indicators known to be associated with later high school graduation and college enrollment.  

Phase IIA ­ Likelihood of high school graduation and college enrollment.

Logistic regression was used to estimate the likelihood that Breakthrough students would 

graduate from high school rather than drop out and the likelihood that they would enroll in 

college in comparison to not enrolling in college, after controlling for a range of 6th grade 

demographic, academic achievement, and behavioral characteristics, including attendance and 

discipline, that are known to be associated with later high school graduation and college 

enrollment. 

16
Breakthrough_17AC189783

Phase IIB­ Program incremental impact. In order to estimate the incremental 

impact of Breakthrough program participation on high school graduation and college 

enrollment, we used a multistep procedure in STATA 11 (Leuvern & Sianesi, 2003). First, 

students’ 6th grade demographic, academic achievement, and behavioral characteristics were 

used to estimate the conditional probability (propensity) that students would participate in 

Breakthrough, whether they actually participated or not. Second, we used the propensity block 

procedure to assign participants and non­participants to groups in which they were not 

significantly different from each other according to their propensity to be served by the 

program, and their demographic, achievement, and behavioral characteristics. Third, within 

propensity block group, the percentage of Breakthrough students who graduated from high 

school or enrolled in college was compared to the percentage of non­participants who graduated 

from high school or enrolled in college. 

RESULTS

Results are grouped according to the evaluation question they were designed to answer. 

The entire Phase I analysis population was used to answer the question about selectivity, but 

the reduced Phase II sample that included only students who graduated or dropped out of 

AISD was used to answer the second and third questions. 

Phase I: Are Breakthrough students significantly different from other 

students at the same schools before they begin the program? 

The selectivity analysis was conducted for all students in the study sample and then 

repeated for each cohort sub­sample separately to determine if there were differences across 

cohorts. Results for the entire Phase I sample are presented in Table 3; if the comparison also 

was significant within a cohort group, the cohort for which the comparison was statistically 

significant is indicated in the final column of the table.  

Across the entire study sample and the three cohorts, Breakthrough students were not 

significantly different from other students on the same campuses with regard to their ethnicity 

17
   

   

Breakthrough_17AC189783

or economic disadvantage. The majority of students in both groups were Hispanic or African

American and the majority of both groups were economically disadvantaged, which confirms

that the program effectively targeted students from populations that are less likely to graduate

from high school and enroll in college. In only one case, this pattern did not hold; Breakthrough

students from Cohort 1 were more likely to be African American than other students from the

same campuses.

Although the program effectively targeted students from demographic groups who were

likely to benefit from an intensive program designed to help them graduate from high school

and enroll in college, there is evidence of selectivity with regard to key academic and

behavioral characteristics. In fact, across the full sample, Breakthrough students were

significantly different from other 6th grade students on the same campuses with regard to all

remaining demographic, academic, and behavioral characteristics. Breakthrough students were

more likely to be female, to participate in magnet programs, and to pass the reading and math

state assessments than would be expected within these schools. Breakthrough students were

less likely to participate in special education programs and were less likely to have a

disciplinary referral than would be expected on these campuses. Breakthrough students also

had higher average grades in reading and math and had better daily attendance than other

students on the same campuses. There were a few exceptions within Cohort 1; Breakthrough

students who were in 6th grade in 2001­2002 did not differ from other students with regard to

their final math course grades, the proportion of students with a disciplinary referral, or

average daily attendance rate.

These results are not surprising given the criteria for Breakthrough program

acceptance, but they strongly confirm the need to include cohort year and pre­participation

demographic, academic, and behavioral characteristics as covariates in the analyses designed

to estimate the associations between program participation and high school graduation or

college enrollment.

18
=  = 

         

         

     

         

     

     

     

         

         

       

       

         

Breakthrough_17AC189783

Table 3. Breakthrough students’ pre­participation demographic, academic 

achievement, and behavioral characteristics in comparison to all other 6th

grade students at the same campuses 

Student 

Characteristic 

Breakthrough (BT) 

(n  131) 

Other students (O) 

(n  3356) 

Significance 

tests 

Sig. Tests 

within 

cohort 

Demographics  N  Percentage  N  Percentage 

Female 87 66.4% 1691 49.6% BT > O*** 2001,

2002, 2003

Male 44 33.6% 1665 50.4% BT < O*** 2001,

2002, 2003

Other Ethnicity 12 9.2% 342 10.2% ns

African

American

27 20.6% 627 18.7% ns 2001

BT > O*

Hispanic 92 70.2% 2387 71.1% ns

Economic

Disadvantage

99 76.7% 2510 79.4% ns

Academics  N  Pct./Average  N Pct./Average 

Magnet student

(7th grade)

53 40.5% 2844 15.3% BT > O***

2001,

2002,

2003

Special

Education

4 3.1% 665 19.8% BT < O***

2001,

2002,

2003

Passed State

Reading

Assessment

123 94.6% 1867 71.6% BT > O***

2001,

2002,

2003

Passed State

Math

Assessment

117 92.9% 1788 68.6% BT > O***

2001,

2002,

2003

Final Reading

Course Grade

77 89.8% 2598 85.5% BT > O***

2001,

2002,

2003

Final Math

Course Grade

77 87.6% 2615 81.5% BT > O*** 2002,

2003

Behavior  N Pct./Average  N  Pct./Average 

One or More

Disciplinary

Referrals

16 12.2% 936 27.9% BT < O*** 2002, 2003

Attendance

Rate

131 97.3% 3356 95.2% BT > O*** 2002, 2003

Note. These n’s represent the number of students with each characteristic; although the denominator

used to calculate the frequencies varied slightly across indicators as a result of missing data. The n

associated with averages is the total number of students with data for that indicator.

* p < .05, ** p <.10, ***p < .001

19
   

Breakthrough_17AC189783

Phase IIA: What is the likelihood of high school graduation and college 

enrollment among Breakthrough students in comparison to other 

students after controlling for 6th grade demographic, academic and 

behavioral characteristics? 

Before conducting logistic regressions to estimate the Breakthrough students’ 

likelihood to graduate from high school and enroll in college, we examined the correlation 

matrix of all independent and dependent variables to determine if there might be problems of 

multicolinearity among some of the independent variables. Among the Phase II independent 

and dependent variables, the highest correlation was between 6th grade course grades in math 

and reading (r = .554); however, because course grades were not to be included in the analyses 

due to systematically missing data, this was not a cause for concern. The correlation between 

passing the reading state assessment and the math state assessment was also moderately 

high, but the logistic procedure in STATA 11 did not detect multicolinearity between these two 

variables; both were retained in the final model. The full correlation matrix is included in 

Appendix A. 

The results of logistic regression analyses predicting high school graduation are 

presented in Table 4. After accounting for pre­participation demographic, academic, and 

behavioral characteristics, students who remained active Breakthrough participants were 3.5 

times more likely to graduate from high school than to drop out, in comparison to students who 

did not participate. It is important to note the wide confidence interval associated with this 

After accounting for pre­participation 

demographic, academic, and 

behavioral characteristics, students

who remained active Breakthrough 

participants were 3.5 times more likely 

to graduate from high school than to

drop out, in comparison to students

who did not participate.

estimate, i.e., 1 to 11 times more likely

to graduate.  An interval this large

indicates that it was difficult for the

procedure to calculate an accurate

odds ratio, most likely due to the small

sample of Breakthrough participants

in the analysis (n = 89).  The most

appropriate way to think about these

20
   

   

Breakthrough_17AC189783

results is that there is an increased likelihood of graduation even after accounting for program

selectivity and to consider the odds ratio of 3.5 as a rough estimate of program effectiveness. To

obtain a firmer estimate, we strongly recommend replicating these analyses with a larger

participant group. Students who participated in the district magnet program, those who passed

reading or math state assessments, and those with regular attendance also were more likely to

graduate from high school than students without these characteristics. Students with a

disciplinary referral in 6th grade were only one third as likely to graduate from high school

compared to those who did not. 

The results of logistic

regression analyses predicting the

likelihood of college enrollment are

presented in Table 5. After

accounting for pre­participation

demographic, academic, and

behavioral characteristics, students

who remained active Breakthrough

participants were 1.98 times more

After accounting for pre­participation 

demographic, academic, and 

behavioral characteristics, students

who remained active Breakthrough 

participants were 1.98 times more

likely to enroll in college than 

comparison students from the same 6th

grade campuses.

likely to enroll in college than other students from the same 6th grade campuses. The

confidence interval for the college enrollment is much narrower, i.e. 1 to 4, than for the high

school graduation estimate; however it must also be interpreted with caution. We know that

the district college enrollment data had only a 57% correspondence rate with Breakthrough

college enrollment records and that the NSC does not collect data from all postsecondary

institutions in the US. It is best to consider this college enrollment estimate accurate within

the context of data available to AISD from the NSC and not to extrapolate the accuracy of the

odds ratio beyond this data source. 

Magnet program participation, passing the state assessments in reading or math, and

being a member of the 6th grade class of 2002­2003 (Cohort 2) also were associated with a

higher likelihood of enrolling in college; however, the increased likelihood of college enrollment

21
   

Breakthrough_17AC189783

for students in Cohort 2 likely is an artifact of the missing outcome data among the students in

Cohort 3 due to the AISD external research opt­out provision, rather than a true performance

difference among the students in Cohort 2.  Student gender also was significantly associated

with college enrollment; males were half as likely to enroll in college as females after

accounting for the other demographic, achievement, and behavioral indicators in the model. 

This reflects state and national trends of female students comprising an increasingly large

percentage of college enrollees (Mather & Adams, 2007).

22
   

   

Breakthrough_17AC189783

Table 4. Logistic regression analysis of graduation from high school rather 

than dropping out 

Odds Standard

z  p 

95% Confidence 

Ratio  Error  Interval 

Breakthrough

Participation 3.47* 2.10 2.06 0.04 1.06 11.36

Cohort 1 0.72 0.14 ­1.74 0.08 0.49 1.04

Cohort 2 0.96 0.19 ­0.22 0.82 0.64 1.42

Male 1.08 0.16 0.51 0.61 0.80 1.45

Hispanic 0.58 0.22 ­1.40 0.16 0.27 1.24

African

American 0.83 0.34 ­0.46 0.65 0.37 1.86

Economic

Disadvantage 0.72 0.16 ­1.46 0.14 0.47 1.11

Magnet 2.74*** 0.77 3.58 0.00 1.58 4.75

Special

Education 1.59 0.71 1.04 0.30 0.66 3.83

Passed State

Reading

Sssessment 1.72*** 0.30 3.07 0.00 1.22 2.43

Pass State

Math

Assessment 2.87*** 0.51 5.96 0.00 2.03 4.05

One or More

Disciplinary

Referrals 0.34*** 0.06 ­6.29 0.00 0.24 0.48

Daily

Attendance

Rate 1.10*** 0.02 5.38 0.00 1.06 1.14

Note. Overall model evaluation using the likelihood ratio test (13, 1411) Χ2 = 344.18, p < .001.

*p < .05, **p < .01, ***p < .001

23
   

   

   

   

Breakthrough_17AC189783

Table 5. Logistic regression analysis of enrolling in college rather than not enrolling 

Odds  Standard 

z  p

95% Confidence 

Ratio  Error Interval 

Breakthrough

Participation 1.98* 0.65 2.10 0.04 1.05 3.75

Cohort 1 0.53 0.23 ­1.45 0.15 0.23 1.25

Cohort 2 1.72*** 0.28 3.31 0.00 1.25 2.38

Male 0.56*** 0.09 ­3.53 0.00 0.41 0.77

Hispanic 0.68 0.18 ­1.50 0.13 0.41 1.13

African

American 1.66 0.51 1.65 0.10 0.91 3.04

Economic

Disadvantage 1.02 0.21 0.10 0.92 0.69 1.52

Magnet 2.43*** 0.48 4.50 0.00 1.65 3.58

Special

Education 1.42 0.85 0.58 0.56 0.44 4.58

Passed State

Reading

Assessment 2.27*** 0.66 2.84 0.00 1.29 4.01

Passed State

Math

Assessment 1.55* 0.35 1.92 0.05 0.99 2.41

One or More

Disciplinary

Referrals 1.32 0.30 1.22 0.22 0.85 2.06

Daily

Attendance

Rate 1.02 0.03 0.76 0.45 0.97 1.07

Note. Overall model evaluation using the likelihood ratio test (13, 786) Χ2 = 143.51, p < .001.

*p < .05, **p < .01, ***p < .001

24
Breakthrough_17AC189783

Phase IIB: What is the incremental impact of Breakthrough 

participation on high school graduation and college enrollment? 

A multistep procedure was used, first, to determine students’ propensity to have 

participated in Breakthrough according to their 6th grade demographic, academic, and 

behavioral characteristics. A second step grouped students into “blocks” within which the 

program students (Breakthrough) and the non­program students (comparison) had similar 

propensity to have been served by the program and were not statistically different from each 

other according to their demographic, achievement, and behavioral characteristics. The 

procedures resulted in three blocks of students whose 6th grade characteristics are summarized 

in Table 6. There were 4 Breakthrough students and 815 comparison students whose 

propensity to be served by the program and other demographic characteristics made them 

significantly different from the students retained in the procedure; their characteristics are 

tabled in the final column of Table 6. 

The students in Block 3 had the highest percentage of magnet students, the highest 

daily attendance rates, 100% passing rates on the state assessments in reading and math, and 

almost no students who were involved in a disciplinary incident. The students in Block 2 

differed from those in Block 3 by having a higher percentage of economically disadvantaged 

students and a lower percentage of magnet students, but were similar to those in Block 3 

according to their very high pass rates on the state reading and math assessments and high 

average daily attendance.  

The students in Block 1 differed somewhat from the other two groups according to their 

lower passing rates on the state assessments and their higher percentage of students with a 

disciplinary referral in 6th grade. The remaining students, who were not assigned to a 

propensity block because of their low probability of participating in Breakthrough, had higher 

percentages of special education students, lower percentages of magnet students, lower state 

assessment passing rates, higher percentages of students with a disciplinary incident, and 

25
Breakthrough_17AC189783

lower daily attendance rates than any of the groups who were assigned to a propensity score

block group. 

Table 6. Sixth­grade demographic, achievement, and behavioral characteristics 

by propensity block 

Block 1  Block 2  Block 3 

Not included 

in a propensity 

block 

Breakthrough

Participants

18 39 27 4

Comparison

Students 440 449 199 815

Cohort 1 176 219 58 300

Cohort 2 136 146 92 249

Cohort 3 146 123 76 270

Other Ethnicity 31.0% 8.2% 0.0% 5.6%

Hispanic 56.8% 81.4% 57.5% 73.9%

African

American 12.2% 10.5% 42.5% 20.5%

Economic

Disadvantage 64.1% 76.9% 67.1% 86.2%

Magnet 31.0% 19.7% 81.9% 1.7%

Special

Education 1.5% 1.4% 0.9% 43.7%

Passed State

Reading

Assessment

84.5% 98.8% 100.0% 31.9%

Passed State

Math

Assessment

82.3% 99.8% 100.0% 21.5%

One or More

Disciplinary

Referrals

24.5% 4.1% 0.9% 41.4%

Average

Attendance Rate 96.3% 97.4% 98.4% 93.3%

26
   

Breakthrough_17AC189783

To answer the question regarding the incremental impact of Breakthrough

participation on high school graduation and college enrollment after accounting for students’

likelihood of participating in the program, we examined the percentages of students with each

outcome within each propensity block (see Table 7). Overall, without accounting for student

pre­participation characteristics and propensity to participate in the program, Breakthrough

appears to have a profound effect on student outcomes; 33% more Breakthrough students

graduate from high school and 69% more students enroll in college than other AISD students. 

However, after accounting for students’ probability of participating in the program, the impact

is more nuanced. Among the highest achieving students, many of whom were magnet students

(Block 3), Breakthrough participation produced 3% more graduates and 16% more college

enrollees than attending AISD alone. This suggests that high performing, low­income, minority

students, particularly those who enrolled in district magnet programs, needed little additional

assistance to graduate from high school, but gain some benefit from program participation with

regard to college enrollment. Follow up analyses designed to isolate the impact of magnet

participation were conducted; however, the results did not change the overall conclusions

presented here (see Appendix B). 

27
   

Breakthrough_17AC189783

Among the students who met 6th grade academic standards, but who were less likely to

be in the district magnet

program (Block 2), Breakthrough

Breakthrough had the largest impact

on high school graduation rates among

the students ... (with) somewhat lower

6th grade state assessment passing

rates and somewhat lower rates of

daily attendance.

program participation resulted in

approximately 10% more

graduates and 63% more college

enrollees than district enrollment

alone. This suggests that

students who passed their 6th

grade state reading and math assessments and who regularly attended school, but who weren’t

served by the district magnet program, were somewhat more likely to graduate from high

school and much more likely to enroll in college if they had the support of Breakthrough. 

Breakthrough had the largest impact on high school graduation rates among the

students in Block 1. These students had a somewhat lower 6th grade state assessment passing

rates and somewhat lower rates of daily attendance than the other two groups of students, 

although both rates far exceeded

those with a low probability of

participating in the program. 

Within this group, Breakthrough

had an incremental impact of

18% on high school graduation

and a similar level of impact on

college enrollment; 20% more of

these students enrolled in college

than similar students in the

district.

Among the students who met 6th grade

academic standards, but who were

less likely to be in the district magnet

program...Breakthrough program

participation resulted in 

approximately 10% more graduates

and 63% more college enrollees than 

district enrollment alone.

28
Breakthrough_17AC189783Table 7. Incremental impact of Breakthrough (BT) participation by propensity block gro

Block 1  Block 2  Not included in a  Block 3  block  

Students with

Graduation or BT Other BT Other BT Other BT Other B

DropoutRecords (n =18) (n

= 397) (n

= 39) (n

= 419) (n

= 27) (n

= 186) (n

= 4) (n

= 734) (n

Graduated 94.4% 79.6% 94.9% 86.4% 100% 97.3% 100% 54.1% 96

Incrementalimpact on +18% +10% +3% n/a +3

Graduation

Block 1  Block 2  Block 3  Not included in a  block  

Students withAISD college BT Other BT Other BT Other BT Other B

enrollmentindicators (n

= 17) (n

= 318) (n

= 37) (n

= 363) (n

= 27) (n

= 182) (n

= 4) (n

= 399) (n

EnrolledCollege

in

41.2% 34.3% 45.9% 28.1% 55.6% 47.8% 75.0% 17.8% 49

Incrementalimpact onCollege +20% +63% +16% n/a +6

Enrollment

Note.

Four (4) program students and 815 non­program students were not assigned to

a propensity block based on their

characteristics.
Breakthrough_17AC189783

DISCUSSION

The purpose of this evaluation study was to estimate the impact of Breakthrough 

Austin program participation on high school graduation rates and college enrollment after 

accounting for student characteristics. Phase I of the study confirmed that the Breakthrough 

admissions process was selective with regard to academic achievement and behavioral 

characteristics for students who were in 6th grade during the 2001­2001, 2002­2003, and 2003­

2004 school years; however, the students who were selected to participate in the program were 

ethnically and economically similar to other students on their campuses. These findings 

confirm that the program selects the students they intend to select, i.e., students who show 

grade level academic proficiency but who are likely to benefit from additional academic and 

social support to be the first in their families to enroll in and graduate from college.  

The Phase II analyses indicate that the program is successful in its mission to serve 

these students. The results of logistic regression showed that Breakthrough students were 

more likely than other district students to graduate from high school and to enroll in college 

after controlling for student demographic, academic, and behavioral characteristics. The 

program incremental impact analyses show that Breakthrough participation had the biggest 

impact on college enrollment among students who met 6th­grade academic achievement 

standards but who were not served by district magnet programs. Breakthrough also had a 

notable impact on both high school graduation and college enrollment among students with 

slightly lower levels of academic achievement but who, on average, were doing grade level work 

in middle school and who were not served by district magnet programs. 

Program Implications 

These results have several important program implications. First, it is clear that 

Breakthrough has the lowest incremental impact of students who have academic achievement 

levels, motivation, and family support to have applied to and enrolled in district magnet 

programs. Program leaders and staff should consider whether or not to continue enrolling 

30
Breakthrough_17AC189783

students who have access to other rigorous academic enrichment programs like district magnet 

programs. 

Breakthrough participation has its greatest impact on the college enrollment rates of 

students who met 6th­grade academic standards in middle school, but who were not enrolled in 

a magnet program. This group appears to be the appropriate target for the program. Within 

the context of this evaluation study, middle school academic achievement appears to be a 

baseline for later college enrollment, but it is well­known that there are many obstacles along 

the road to college for low­income, minority students and Breakthrough programs appear to 

give students the support they need to reach this important life benchmark. 

Breakthrough also appears to provide an important resource for students who are close 

to grade level achievement but who have slightly lower average levels of attendance (though 

still over 95%) or a disciplinary referral. This group of students was most likely to show notable 

incremental gains on high school graduation as a result of Breakthrough participation. 

Breakthrough participants showed similar incremental gains on college enrollment within this 

group; however the impact was not as high as among those with slightly higher levels of 

achievement. 

Limitations 

Although every effort was made to address the evaluation questions with the 

appropriate research methods, there are several important limitations to this study. 

Sample size. The size of the Breakthrough participant group was quite small in 

comparison to the non­participant group, which had an impact on the estimation of firm odds 

ratios. Ideally, evaluations with future cohorts will have participant samples that are large 

enough for the estimation of independent effects and of interaction effects among Breakthrough 

participation and participation in other programs like district magnet programs, which also are 

strongly associated with high school graduation and college enrollment. 

Data quality. One of the most important Breakthrough program entry criteria, 6th

grade course grades, was not included in the Phase II analyses because of systematically 

31
  

Breakthrough_17AC189783

missing electronic grade records for all students who were enrolled on AISD elementary 

campuses. If these data are missing among future evaluation cohorts, it is recommended that a 

multiple imputation procedure is used to estimate these missing values.  Missing data on this 

important indicator of grade level academic achievement has implications for the calculation of 

accurate likelihood estimates and propensity block groups. However, of the two available 

academic achievement indicators that could have been included here (state assessment 

performance and grade reports), the state assessment results could be considered more 

valuable because they are relatively objective, whereas course grades are subject to inflation 

and other non­random effects. 

There also were notable problems with final graduation v. dropout data. Over 200 

records were eliminated from the Phase II analyses because there was no dropout date 

provided in the district records. Another 167 final outcomes were systematically redacted 

because of the external research opt­out provision in AISD. To a certain extent, these two 

sources of missing data may “cancel each other out” because one set of records would have led 

to higher estimates of program impact and the other to lower estimates of program impact. 

The most significant problem with data quality was among the college enrollment 

records. Because of the infrequency with which districts receive these records and because of 

limitations to the number of postsecondary institutions that participate in the NSC, it is very 

likely that a large number of postsecondary college enrollment records were missing for both 

Breakthrough participants and other students. Interpretations of the Breakthrough impact on 

college enrollment should be limited to the context of schools that participate in NSC and for 

students who were enrolled in a college or university on or about December 2010.  

Unmeasured student characteristics. Although a wide range of student 

characteristics were included in both phases of this evaluation study, there are likely 

important unmeasured family support and motivational characteristics that led students to 

seek out the support of a program like Breakthrough. The inclusion of support and motivation 

32
Breakthrough_17AC189783

factors would increase the accuracy of estimates of program impact in future evaluation 

studies. 

No direct examination of program characteristics. Although the Breakthrough 

program model uses research­supported best practices for helping low­income and minority 

students graduate from high school and enroll in college, this study did not directly examine 

the effectiveness of particular program features. Future evaluation studies could include 

qualitative and quantitative measures of program implementation and effectiveness to answer 

formative evaluation questions for the purposes of program improvement. Similarly, program 

features could be examined in a cost­utility or cost­benefit framework to answer questions 

about program scalability and to determine which program features provide the maximum 

benefit for the minimum cost. 

33
   

   

Breakthrough_17AC189783

REFERENCES
 

Beckett, M., Borman, G., Capizzano, J., Parsley, D., Ross, S., Schirm, A., & Taylor, J. (2009).  

Structuring out­of­school time to improve academic achievement: IES practice guide.  

Guide prepared by the National Center for Education Evaluation and Regional

Assistance for the U.S. Department of Education What Works Clearinghouse.  

Washington, DC: U.S. Department of Education.

Cabrera, A. F. & La Nasa, S. M. (2000). Understanding the college choice of disadvantaged

students. New Directions for Institutional Research. San Francisco: Jossey­Bass

Cabrera, A. F. & La Nasa, S. M. (2001). On the path to college: Three critical tasks facing

America’s disadvantaged. Research in Higher Education, 42(2), 119­150.

Coleman, J. S. (1988). Social capital in the creation of human capital. American Journal

of Sociology, 94 (Suppl.), 95–120.

Constantine, J. M., Seftor, N. S., Martin, E. S., Silva, T., & Myers, D. (2006). A study of the

effect of the Talent Search program on secondary and postsecondary outcomes in

Florida, Indiana, and Texas: Final report from phase II of the national evaluation.

Report prepared by Mathematica Policy Research for the U.S. Department of

Education, Office of Planning, Evaluation, and Policy Development, Policy and

Program Studies Service. Washington, DC: U.S. Department of Education.

Dynarski, M., Clarke, L., Cobb, B., Finn, J., Rumberger, R., & Smink J. (2008).  Dropout

prevention: IES practice guide.  Guide prepared by the National Center for Education

Evaluation and Regional Assistance for the U.S. Department of Education What Works

Clearinghouse.  Washington, DC: U.S. Department of Education. 

34 
   

  

  

Breakthrough_17AC189783

Dynarski, M., Gleason, P., Rangarajan, A., & Wood, R. (1998).  Impacts of Dropout Prevention

Programs: Final Report.  Report prepared by Mathematica Policy Research for the U.S. 

Department of Education, Office of Planning, Evaluation, and Policy Development, 

Policy and Program Studies Service. Washington, DC: U.S. Department of Education.

Larson, K.A. & Rumberger, R. W. (1995).  ALAS:  Achievement for Latinos through Academic

Success.  In H. Thornton (Ed.), Staying in school: A technical report of the dropout

prevention projects for junior high school students with learning and emotional

disabilities.  Minneapolis, MN: University of Minnesota, Institute on community

Integration. 

Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin­Glenn, M. (2004). 

The effectiveness of out­of­school­time strategies in assisting low­achieving students in

reading and mathematics: A research synthesis (Updated ed.). Aurora, CO: Midcontinent Research for Education and Learning.

Leuvern, E. & Sianesi, B. (2003). “PSMATCH2”: Stat module to perform full Mahalanobis and

propensity score matching, common support graphic, and covariate imbalance testing. 

Retrieved from http://ideas.repec.org/c/boc/bocode/s432001.html.

Mather, M. & Adams, D. (Feb, 2007). The crossover in female­male college enrollment rates. 

Population reference bureau. Retrieved from

http://www.prb.org/articles/2007/crossoverinfemalemalecollegeenrollmentrates.aspxs

National Center for Education Statistics (2001). High school academic curriculum and the

persistence path through college.  Retrieved from

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2001163.

Schargel, F. P., & Smink, J. (2001). Strategies to help solve our school dropout problem. 

Larchmont, NY: Eye on Education.

35 
Breakthrough_17AC189783

Sinclair, M.F., Christenson, S.L., Evelo, D.L., & Hurley, C.M. (1998).  Dropout prevention for

youth with disabilities: efficacy of a sustained school engagement procedure.  

Exceptional Children, 65(1), 7­21.

36 
Breakthrough_17AC189783 APPENDIX A. CORRELATION MATRIX Graduated Enrolled in College Male Other Ethnicity African American Hispanic Econ. Dis Magnet Special Education Pass State Reading. Pass State Math Fin

Read

Grade 

Graduated 1.000

Enrolled in College ­0.028 1.000

p (0.299)

Male ­0.100 ­0.102 1.000

p (0.000) (0.000)

Other Ethnicity 0.152 0.111 ­0.012 1.000

p (0.000) (0.000) (0.491)

African American ­0.009 0.104 ­0.012 ­0.162 1.000

p (0.691) (0.000) (0.466) (0.000)

Hispanic ­0.099 ­0.169 0.018 ­0.527 ­0.754 1.000

p (0.000) (0.000) (0.277) (0.000) (0.000)

Economic

Disadvantage ­0.171 ­0.109 0.005 ­0.382 ­0.014 0.268 1.000

p (0.000) (0.000) (0.755) (0.000) (0.412) (0.000)

Magnet 0.267 0.238 ­0.101 0.396 ­0.026 ­0.242 ­0.331 1.000

p (0.000) (0.000) (0.000) (0.000) (0.127) (0.000) (0.000)

Special Education ­0.156 ­0.128 0.160 ­0.043 0.119 ­0.073 0.061 ­0.189 1.000

p (0.000) (0.000) (0.000) (0.011) (0.000) (0.000) (0.001) (0.000)

Passed State

Reading Assessment 0.318 0.193 ­0.064 0.187 0.011 ­0.140 ­0.162 0.281 ­0.103 1.000

p (0.000) (0.000) (0.001) (0.000) (0.562) (0.000) (0.000) (0.000) (0.000)

Pass State Math

Assessment 0.347 0.076 ­0.025 0.193 ­0.043 ­0.099 ­0.168 0.274 ­0.074 0.502 1.000

p (0.000) (0.008) (0.195) (0.000) (0.024) (0.000) (0.000) (0.000) (0.000) (0.000)

Final Reading

Course Grade 0.324 0.157 ­0.222 0.218 ­0.126 ­0.046 ­0.167 0.214 ­0.133 0.339 0.335 1.00

p (0.000) (0.000) (0.000) (0.000) (0.000) (0.019) (0.000) (0.000) (0.000) (0.000) (0.000)

Final Math Course

Grade 0.356 0.113 ­0.204 0.219 ­0.072 ­0.092 ­0.166 0.273 ­0.066 0.337 0.445 0.55

p (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.001) (0.000) (0.000) (0.00

Disciplinary Referral ­0.307 0.010 0.205 ­0.102 0.151 ­0.062 0.116 ­0.201 0.154 ­0.153 ­0.159 ­0.3

p (0.000) (0.700) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.00

Attendance Rate 0.327 0.085 ­0.054 0.044 ­0.038 0.003 ­0.077 0.153 ­0.139 0.126 0.136 0.29

p (0.000) (0.001) (0.001) (0.010) (0.027) (0.849) (0.000) (0.000) (0.000) (0.000) (0.000) (0.00
  

Breakthrough_17AC189783

APPENDIX B. INCREMENTAL IMPACT OF BREAKTHROUGH (BT)
 

PARTICIPATION BY MAGNET PARTICIPATION
 

Magnet Non­magnet Overall

Students

with

Graduation

or Dropout

Records

BT

(n = 40)

Other

(n = 370 )

BT

(n =  48)

Other

(n = 1366 )

BT

(n = 88 )

Other

(n = 1736)

Graduated 95.0% 95.4% 97.9% 66.1% 96.6% 72.4%

Incremental

impact on

Graduation

­.04 % + 48.1% + 33.5%

Students

with AISD

college

enrollment

indicators

BT

(n = 38)

Other

(n = 393)

BT

(n = 47)

Other

(n = 907)

BT

(n = 85 )

Other

(n = 1262)

Enrolled in

College 60.5% 46.2% 40.4% 22.6% 29.2% 49.4%

Incremental

impact on

College

Enrollment

+ 31% + 79% + 69%

Note. As found in the propensity block analyses, Breakthrough participation had little impact on

the graduation rates of magnet students. There was an impact on college enrollment among

magnet and non­magnet students; however the impact was higher among non­magnet students.

38