Contract No.: LC-92001001

MPR Reference No.: 8046-515

The Impacts of Upward

Bound: Final Report for

Phase I of the National

Evaluation 

Final Report

April 1999

David Myers

Allen Schirm

Submitted to: Submitted by:

U.S. Department of Education Mathematica Policy Research, Inc.

Planning and Evaluation Services 600 Maryland Avenue, SW

400 Maryland Avenue, SW Suite 550

Room 6W306 Washington, DC 20024-2512

Washington, DC 20202 (202) 484-9220

Project Officer: Project Director:

David Goodwin David Myers
iii

ACKNOWLEDGMENTS

This report on the impacts of Upward Bound on student outcomes reflects the contributions of

many people. The authors would like to thank David Goodwin of the Planning and Evaluation

Service of the U.S. Department of Education, for his continued support, substantive guidance, and

encouragement throughout the study. Mary Moore, a principal investigator for the national

evaluation of Upward Bound, throughout the study helped shape the study design and interpret our

findings. We are also grateful for the contributions of several others who played important roles as

we collected and analyzed our data, and prepared this report. These people include Anne Ciemnecki,

Jill Corcoran, Paul Decker, Lance Freeman, Darnecia Morris, Tim Silva, and Steve Sweetland.
v

CONTENTS

Chapter Page

EXECUTIVE SUMMARY ....................................... xv

I INTRODUCTION ............................................... 1

A. THE CONTEXT AND PURPOSE OF UPWARD BOUND .......... 2

B. MAJOR FINDINGS FROM THE NATIONAL EVALUATION ...... 3

1. How Upward Bound Operates Today ....................... 3

2. Previous Results Based on Students' Early Involvement in 

Upward Bound ........................................ 4

C. KEY TOPICS ADDRESSED IN THIS REPORT .................. 5

D. RESEARCH DESIGN ...................................... 6

1. Random Assignment .................................... 6

2. Data Collection ........................................ 8

E. PHASE II OF THE NATIONAL EVALUATION OF UPWARD 

BOUND ................................................. 9

F. ORGANIZATION OF THIS REPORT .......................... 9

II THE UPWARD BOUND EXPERIENCE: PARTICIPATION, 

COMPLETION, PERSISTENCE, AND INTENSITY .................. 11

A. PROGRAM PARTICIPATION .............................. 13

B. PROGRAM COMPLETION ................................ 21

C. PROGRAM PERSISTENCE ................................ 26

D. PROGRAM INTENSITY ................................... 32
Contents (continued)

Chapter Page

vi

III THE IMPACT OF UPWARD BOUND .............................. 39

A. COMPUTING PROGRAM IMPACTS ......................... 41

B. CONTEXT FOR INTERPRETING PROGRAM IMPACTS ........ 42

1. Many Students in the Study Are Too Young to Have 

Attended College ...................................... 43

2. Estimates of Program Impact Show Upward Bound’s 

Value Added ......................................... 43

C. PROGRAM IMPACTS .................................... 45

1. High School Graduation, Educational Expectations, and 

High School Course Taking ............................. 46

2. Going to College ...................................... 50

D. IMPACTS FOR SUBGROUPS OF STUDENTS ................. 57

1. Upward Bound Has Large Impacts on Students with 

Lower Initial Educational Expectations ..................... 59

2. Upward Bound Has Substantial Impacts on Boys .............. 62

3. Hispanic and White Youth Benefit More from 

Upward Bound than African American Youth ................ 66

4. Giving Students the Opportunity to Participate in 

Upward Bound Had Substantial Impacts on LowIncome/First-Generation Students ......................... 70

5. Upward Bound Has Large Impacts on Students 

Who Have Lower Academic Performance as 

High School Freshman .................................. 75

6. The Impact of Upward Bound Is Similar for 

9th- and 10th-Grade Applicants ........................... 81

IV THE IMPACT OF DURATION IN UPWARD BOUND ON 

EDUCATIONAL OUTCOMES .................................... 85

A. ESTIMATING THE EFFECTS OF DURATION ON 

PARTICIPANTS’ OUTCOMES ............................. 86

B. EFFECTS OF DURATION IN UPWARD BOUND ON 

STUDENT OUTCOMES ................................... 88

1. Participants Who Remained in Upward Bound for Longer 
Contents (continued)

Chapter Page

vii

Periods Had Higher Educational Expectations and Earned 

More Credits in High School ............................. 88

2. Longer Exposure to Upward Bound Led to Higher 

Postsecondary Attendance Rates, Attendance at More 

Selective Schools, and Earning More Postsecondary Credits ..... 91

C. COMPUTING THE EFFECTS OF COMPLETING 

UPWARD BOUND ....................................... 92

D. EFFECT OF PROGRAM COMPLETION OF STUDENT 

OUTCOMES ............................................ 93

V SUMMARY AND CONCLUSIONS ................................ 95

A. SUMMARY OF FINDINGS ................................ 95

B. IMPLICATIONS OF FINDINGS FOR PROGRAM 

IMPROVEMENT ........................................ 102

1. Retaining Upward Bound Participants ..................... 102

2. Serving More Students Who Are At Greater Risk of 

Educational Failure ................................... 103

C. ISSUES TO BE CONSIDERED IN PHASE II OF THE 

NATIONAL EVALUATION OF UPWARD BOUND ............ 104

REFERENCES ............................................... 107

APPENDIX A: SAMPLE DESIGN, WEIGHTING, AND 

ERROR ESTIMATION

APPENDIX B: ESTIMATION OF PROGRAM IMPACTS

APPENDIX C: ESTIMATION OF THE EFFECTS OF DURATION 

ON STUDENT OUTCOMES

 APPENDIX D: USE OF SUPPLEMENTAL SERVICES
ix

TABLES

Table Page

1 SUMMARY OF FINDINGS ON THE IMPACTS OF UPWARD 

BOUND ON SELECTED STUDENT OUTCOMES ................... xvii

II.1 MOST COMMON REASONS FOR NOT PARTICIPATING IN 

UPWARD BOUND ............................................. 15

II.2 ADJUSTED PARTICIPATION RATES ............................. 17

II.3 ADJUSTED COMPLETION RATES ............................... 23

II.4 DURATION IN PROGRAM ...................................... 28

II.5 MOST COMMON REASONS FOR LEAVING UPWARD BOUND 

BEFORE COMPLETING THE PROGRAM .......................... 28

II.6 ADJUSTED DURATION ESTIMATES ............................. 30

II.7 STUDENT PARTICIPATION IN UPWARD BOUND ACADEMIC 

COURSES AND NONACADEMIC ACTIVITIES DURING THE 

SUMMER .................................................... 33

II.8 STUDENT PARTICIPATION IN UPWARD BOUND ACADEMIC 

COURSES AND NONACADEMIC ACTIVITIES DURING THE 

ACADEMIC YEAR ............................................. 35

III.1 USE OF SUPPLEMENTAL SERVICES OTHER THAN THOSE 

PROVIDED BY UPWARD BOUND ................................ 45

III.2 IMPACT OF UPWARD BOUND ON STUDENTS’ EXPECTATIONS 

AND HIGH SCHOOL COURSE TAKING ........................... 47

III.3 IMPACT OF UPWARD BOUND ON STUDENTS’ HIGH SCHOOL

EXPERIENCES ................................................ 51

III.4 IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT 

AND RELATED OUTCOMES .................................... 54
TABLE (continued)

Page

x

III.5 IMPACT OF UPWARD BOUND ON STUDENTS’ RECEIPT OF 

FINANCIAL AID AND SOURCES OF INFORMATION 

CONCERNING FINANCIAL AID ................................. 56

III.6 IMPACT OF UPWARD BOUND ON COLLEGE ACTIVITIES ........... 58

III.7 IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS 

AND HIGH SCHOOL COURSE TAKING FOR STUDENTS WITH 

HIGHER AND LOWER INITIAL EDUCATIONAL EXPECTATIONS ..... 60

III.8 IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND 

CREDITS EARNED IN COLLEGE FOR STUDENTS WITH HIGHER 

AND LOWER EDUCATIONAL EXPECTATIONS .................... 63

III.9 IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS 

AND HIGH SCHOOL COURSE TAKING FOR BOYS AND GIRLS ....... 64

III.10 IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND 

CREDITS EARNED IN COLLEGE FOR BOYS AND GIRLS ............ 67

III.11 IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS 

AND HIGH SCHOOL COURSE TAKING FOR AFRICAN AMERICAN, 

WHITE, AND HISPANIC STUDENTS ............................. 68

III.12 IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND 

CREDITS EARNED IN COLLEGE FOR AFRICAN AMERICAN, 

WHITE, AND HISPANIC STUDENTS ............................. 71

III.13 IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT 

AND RELATED OUTCOMES FOR LOW-INCOME AND POTENTIAL 

FIRST-GENERATION COLLEGE STUDENTS ....................... 73

III.14 IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT 

AND RELATED OUTCOMES FOR LOW-INCOME AND POTENTIAL 

FIRST-GENERATION COLLEGE STUDENTS ....................... 76

III.15 IMPACT OF UPWARD BOUND ON STUDENTS’ EDUCATIONAL

EXPECTATIONS AND HIGH SCHOOL COURSE TAKING, BY 

STUDENTS’ AT-RISK STATUS .................................. 78
TABLE (continued)

Page

xi

III.16 IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE 

AND CREDITS EARNED IN COLLEGE BY STUDENTS’ 

AT-RISK STATUS ............................................. 80

III.17 IMPACT OF UPWARD BOUND ON HIGH SCHOOL OUTCOMES 

FOR 9TH- AND 10TH-GRADE APPLICANTS ....................... 82

IV.1 EFFECTS OF DURATION ON PARTICIPANTS’ HIGH SCHOOL 

OUTCOMES .................................................. 89

IV.2 EFFECTS OF DURATION ON PARTICIPANTS’ COLLEGE 

OUTCOMES: 10TH-GRADE COHORT ............................. 92

IV.3 EFFECTS OF COMPLETING UPWARD BOUND FOR 

10TH-GRADE APPLICANTS ..................................... 94

V.1 SUMMARY OF FINDINGS AFTER SECOND FOLLOW-UP 

OF STUDENTS ................................................ 96
xiii

FIGURES

 Figure Page

II.1 A COHORT’S PATH TO COMPLETING UPWARD BOUND ............ 12

III.1 SCHOOL STATUS IN 1996 ...................................... 43

III.2 IMPACT OF UPWARD BOUND ON STUDENTS’ EDUCATIONAL

EXPECTATIONS .............................................. 48

IV.1 DURATION IN UPWARD BOUND AFTER SECOND 

FOLLOW-UP OF STUDENTS .................................... 88
This does not include Math/Science centers or Veterans projects. 1

During the course of the overall evaluation, MPR worked with several subcontractors: Westat, 2

Decision Information Resources, Educational Testing Services, and Public/Private Ventures.

A previous report (Myers and Schirm 1997) addressed only short-term program impacts on

3

students while they were primarily high school freshman and sophomores.

xv

EXECUTIVE SUMMARY

Upward Bound is a federal program designed to help disadvantaged students prepare to enter

and succeed in college. Established in 1965, it is the largest federal program other than student

financial aid programs, to help American high school students attain a postsecondary education.

Currently, about 44,000 students participate in 563 regular Upward Bound projects around the

country. At least two-thirds of each project’s participants must be both low-income and potential 1

first-generation college students. Students typically enter the program in their freshman or

sophomore year of high school and can remain in it through the summer following high school

graduation. Projects provide students with a variety of services, including instruction, tutoring, and

counseling. In addition to regularly scheduled meetings throughout the academic year, projects also

offer an intensive instructional program that meets daily for about six weeks during the summer.

The vast majority of Upward Bound projects are “hosted” by two- or four-year colleges.

 

This report presents findings from the national evaluation of Upward Bound, conducted by

Mathematica Policy Research (MPR) for the U.S. Department of Education (ED), Planning and

Evaluation Service. This report focuses on the effect of Upward Bound on students during their

2

high school years and the first year or two of college. Specifically, the report addresses the

3

following major research questions:

C To what extent does Upward Bound further the academic and personal development of

students during high school?

C What impact does Upward Bound have on students during their early years in

postsecondary school?

C Who benefits most from participating in Upward Bound?

C What are the typical experiences of students in Upward Bound, for example, in terms

of how long they participate? Does the amount of time students spend in the program

influence outcomes?
xvi

SCOPE AND METHODOLOGY

This report draws on several data sources collected as part of a longitudinal study of Upward

Bound program effectiveness. The evaluation design initially involved the selection of a nationally

representative sample of 67 regular Upward Bound projects hosted by two- and four-year colleges.

Excluded from the study were projects that serve veterans, projects funded through the Upward

Bound Math/Science Initiative, and projects hosted by high schools and community-based

organizations. For information on various project characteristics and practices, we surveyed the 67

project directors in 1993, as part of a larger survey of Upward Bound grantees.

From 1992 to 1994, eligible applicants at these 67 projects completed a baseline questionnaire

that gathered information about their family backgrounds, attitudes and expectations, and school

experiences. During the same time period, we also randomly assigned the eligible applicants at each

project to either a treatment group (Upward Bound) or a control group. Altogether, about 1,500

students nationwide were assigned to the treatment group and about 1,300 were assigned to the

control group. We conducted follow-up surveys of both groups in 1994 and 1996 to collect updated

information on their attitudes, school experiences, and other outcomes. The response rates for these

surveys were about 97 percent and 85 percent, respectively. We also collected the students’ school

transcripts in 1994 and 1996 to assess their academic experiences and performance in high school

and, for those old enough, in college. Finally, project staff reported annually on the participation of

students in the program.

To assess most of the program impacts discussed in this report, we compared the average

outcomes of students in the treatment and control groups. Because the two groups of students were

essentially indistinguishable at the time of random assignment, any statistically significant difference

between them in outcomes, such as courses taken in high school or enrollment rates at postsecondary

institutions, can be attributed to the fact that the one group had the opportunity to participate in

Upward Bound and the other did not. To analyze the impact of duration in Upward Bound, we

matched students in the control group with participants who stayed in the program for varying

lengths of time. 

OVERVIEW OF MAJOR FINDINGS

When we last collected information from students, many were in their last year of high school

or had just finished high school and not yet attended college. Because many of the students had not

had an opportunity to attend college, our most credible findings pertain to students’ high school

experience. We present findings concerning the college experience because they are suggestive of

what Upward Bound may achieve in the long run. However, more definitive results concerning

college access and retention need to wait until data are collected again in late 1998 and 1999. At this

stage of the national evaluation, several key findings stand out (see Table 1 for a summary):
xvii

TABLE 1

SUMMARY OF FINDINGS ON THE IMPACTS OF UPWARD BOUND 

ON SELECTED STUDENT OUTCOMES

Expectations Sex Race/Ethenicity LIFG Status At-Risk Status Application a

Grade At

Overall Higher Lower Girls Boys American White Hispanic LIFG FG LI Lower Higher Grade Grade

African 9th 10th

Educational Expectations

(years of schooling)

 Students’ T T T T T T T T T T T T

 Fathers’ T T T

 Mothers’ T T T T

b

High School Credits

 Non-remedial English T T T T

 Non-remedial social T T T T T T T

studies T T T T T T T T T T T

 Non-remedial math T T T T T T

 Non-remedial science T T

 Non-remedial foreign T T T T

language

 Non-remedial total for 5 T T T T T T

major subjects

 Non-remedial vocational T

education

 Non-remedial computer T T T T

science T T T T T

 Non-remedial other T T T T T T T

 Total Non-remedial T T T T

 Total AP/Honors, all

subjects T T T T T T T

 Total Credits, includes

remedial

Satisfied New Basics

Curriculum T T T T T T

Cumulative GPA T T

High School Status

 Still in high school T T T T T T T ___ ___

 Dropped out T T T T T T T T ___ ___

 Graduated T T T ___ ___
TABLE 1 (continued)

Expectations Sex Race/Ethenicity LIFG Status At-Risk Status Application a

Grade At

Overall Higher Lower Girls Boys American White Hispanic LIFG FG LI Lower Higher Grade Grade

African 9th 10th

xviii

School Status

Attend College T T ___ ___

Attend four-year college T T T T ___ ___

Attend two-year college ___ ___

Attend vocational school ___ ___

c

Credits Earned

Four-Year College

Total non-remedial T T T T T ___ ___

Total remedial T ___ ___

Two-Year College

Total non-remedial ___ ___

Total remedial T T T ___ ___

College Selectivity T T T ___ ___

LIGF refers to students’ low-income /first generation status when they apply to participate in Upward Bound. LIFG means both criteria were met; FG indicates student did not meet the low-income

a

criteria but they were potential first generation college students; LI indicates that they were from a low-income family, but they were not potential first generation college students.

Indicates a significant positive impact was found for the outcome. b

Not computed for 9th and 10th grade applicants when analyzed separately. c
xix

C Many students remain in Upward Bound for only a short time. Although most

participants had the opportunity to remain in Upward Bound for up to three or four

years, about 35 percent left the program during the first year, and we estimate that at

least an additional 20 percent will drop out of the program before the end of their senior

year in high school. The typical participant was exposed to Upward Bound for only

about 19 months, and remained commonly in the program for one summer and parts of

two academic years.

C Upward Bound has limited impacts on students during high school. When we looked

across a variety of outcomes, we found that Upward Bound had impacts on only a few

outcomes. Roughly two to three years after being selected for Upward Bound, students

in the treatment group expected to complete slightly more years of education and had

earned more credits in math and social studies than students in the control group.

Upward Bound generally had no impacts on students’ in-schools behavior, participation

in extra curricular activities, grade point average, or credits earned in subjects such as

English or science. Furthermore, Upward Bound had no impact on high school

graduation.

C Upward Bound may have some impact on participants’ postsecondary education

experiences. Although, Upward Bound had no impact on the chances students attended

college, students in the treatment group earned more credits from four-year colleges,

were more likely to receive financial aid, and were more actively engaged in some

college activities. But since relatively few study participants had reached college age

and those who did entered Upward Bound during grades 10-12, it is too soon to make

definitive statements about Upward Bound’s impact on postsecondary outcomes.

C Upward Bound has substantial impacts on some groups of students and not others.

Although Upward Bound had small impacts on students as a whole, some groups of

students received greater benefits. Focusing on the key outcomes of educational

expectations, courses taken in high school, and high school completion we found that

(1) students with lower initial educational expectations benefitted substantially more

than those with higher expectations, (2) boys showed substantially larger impacts than

girls, (3) Hispanic and white students benefitted more than African American students,

(4) students who were low-income only or low-income and potential first-generation

college students showed larger impacts than those who qualified for the program only

as potential first-generation students, and (5) poorer performing students benefitted

substantially more than their better performing peers.

DETAILED FINDINGS

Program Participation, Completion, Persistence, and Intensity

About 82 percent of eligible applicants who had the opportunity to participate in Upward Bound

chose to do so. Among the more common reasons students give for not participating are

transportation problems, taking a job, not being notified by the project, and time conflicts. Holding
This is an upper bound and assumes all current participants will stay in Upward Bound until

4

the end of their senior year in high school.

xx

other factors constant, Hispanics and Asian/Pacific Islanders are more likely to participate than

whites and African Americans.

Less than 45 percent of participants continue with the program through the senior year of high

school. Controlling for other factors, students from low-income families are more likely to

4

complete the program than their peers who qualify solely by being a potential first-generation college

student. Also, students who initially expect to obtain at least a bachelor’s degree are more likely to

complete Upward Bound than those who expect to accumulate less education.

Focusing on duration rather than completion, we found that over one-third of participants leave

Upward Bound within the first year and nearly two-thirds leave within two years. The median length

of participation in the program is 19 months. By far the most common reason students give for

leaving Upward Bound before completing it is to take a job. Holding other factors constant, we

found, not surprisingly, that the earlier in high school students join, the longer they tend to stay in

the program.

Upward Bound is intensive as measured by the number of activities undertaken by students.

During the academic year, the average student attends 36 academic sessions and 43 nonacademic

activities; in the summer, the typical participant attends 103 academic sessions and 72 nonacademic

activities. Over the course of his or her participation in Upward Bound, the average student attends

a total of 375 sessions, 55 percent of which are academic.

The Impacts of Upward Bound

High School Outcomes. Many outcomes concerning students’ experiences were analyzed as

part of the evaluation; however, Upward Bound had an impact on only a few of them when we

looked at all Upward Bound students. Upward Bound students had sightly higher educational

expectations than the control group. Also, the treatment group students earned, on average, slightly

more nonremedial high school credits in social studies and math--0.1 and 0.2 additional credits,

respectively. We found no impacts on a host of other outcomes related to high school, including

misbehavior, participation in school activities, talking with parents about school, or parental

involvement at school.

Postsecondary Outcomes. Compared with students in the control group, students in Upward

Bound where just as likely to enroll in a two- or four-year college; however, Upward Bound students

earned more nonremedial credits at four-year colleges and fewer remedial credits at two-year

colleges. In addition, the treatment group students were more likely (by margins of 3-4 percentage

points) to receive various types of financial aid in college, including loans, grants, and work-study

appointments. Finally, the treatment group participated more often in various college activities,

including informal contacts with an advisor or faculty member, talking with faculty in their offices,

participating in a study group with fellow students, and participating in school clubs. These findings
xxi

should be interpreted with caution, however, because only about one-fourth of the students in our

study had entered college at the time we last contacted them; about one-third were still in high

school. A more comprehensive analysis of how Upward Bound affects students’ postsecondary

education pursuits will be possible in 1999, after we next survey the study participants and again

collect college transcripts.

Subgroup Differences. Upward Bound had large impacts on students specific characteristics,

such as students with lower educational expectations, boys, and white and Hispanic youth. Some

highlights of our analysis of subgroup differences include the following:

C Among students with initially lower educational expectations (that is, students who

expect to complete less than a bachelor’s degree), Upward Bound students earned an

average of almost three high school credits more than those in the control group and

were less likely to drop out of high school than those who did not participate in Upward

Bound. We found no such difference among students who expected to complete at least

a bachelor’s degree. Similar results were found for college enrollment: students with

lower expectations who were in the treatment group were more likely to attend college

than students in the control group and we found no impacts for those with higher

educational expectations.

C Boys in Upward Bound earned almost two more high school credits than did boys in the

control group; girls selected for the program did not earn more total high school credits

than girls in the control group. Boys in Upward Bound also were less likely to drop out

of high school and were more likely to attend four-year colleges than boys in the control

group. Upward Bound had no impact on college enrollment for girls.

C Whites and Hispanics in the treatment group earned almost two more credits in high

school than similar students in the control group, but there was no such difference

among African American students. Furthermore, white and Hispanic Upward Bound

students were less likely to drop out of high school than similar students in the control

group; there was no impact on dropping out for African American students. In addition,

whites who were in Upward Bound were more likely to enroll in college (two- and fouryear colleges combined) than similar students in the control group and Hispanics in

Upward Bound attended four-year colleges at a higher rate than those who were in the

control group.

C Upward Bound students who qualified for the program solely because of low-income

earned an average of about two credits more in high school than similar students not

selected for the program and were less likely to drop out of high school. Those who met

both the low-income and first-generation criteria and were selected earned about one

credit more and were somewhat less likely to drop out of school than those not selected.

We found no difference among those who were eligible only as potential first-generation

college students.
xxii

C Poorer-performing students (a categorization based on both grade point average and

core academic credits earned in 9th grade) who were in Upward Bound earned, on

average, almost two more credits in high school than those in the control group. We

found no similar difference among better-performing students. In addition, Upward

Bound reduced the chances that poorer-performing students would drop out of high

school; the program had a small impact for those less at risk. Also among the poorerperforming students, those who were in Upward Bound attended four-year colleges at

a higher rate than similar students in the control group. Upward Bound had no impact

on college attendance for better-performing students.

The Impact of Duration in Upward Bound on Educational Outcomes

Longer exposure to Upward Bound was associated with greater program impacts. Students who

entered Upward Bound in the 9th grade and remained in for more than two years had higher

educational expectations and earned about three more credits in core academic subjects than similar

nonparticipants. Similarly, students who began the program as 10th graders and persisted for more

than two years earned about two more core academic credits than their counterparts. As for

postsecondary outcomes, students who started Upward Bound as 10th graders and remained in

Upward Bound for 13-24 months were more likely (by a margin of 14 points) to go on to a four-year

college and also earned almost six more nonremedial credits at four-year schools. Comparison of

the experiences of students who completed Upward Bound and noncompleters suggests that

continuing in the program until the end of the senior year of high school generally has no impact on

students’ outcomes.

CONCLUSIONS AND IMPLICATIONS

Although Upward Bound generally has few and small impacts on students during their high

school years, and it is too soon to tell for certain how the program affects students in terms of college

attendance and completion, our findings demonstrate that Upward Bound can have large impacts for

some groups of students. Most notably, the program appears more helpful to students with lower

initial educational expectations, students with poorer academic performance as high school

freshmen, and those who remain in the program for at least two years. These findings suggest that

if the U.S. Department of Education (ED) or individual projects want to increase the impact of the

program, they may want to consider action in two areas: retention and student selection.

Improve Student Retention

As mentioned above, only about one-third of the Upward Bound participants in our study had

been in the program for more than two years at the time of our last data collection, and about 40

percent had stopped participating within one year. Yet, the longer students remain in the program,

the more they appear to benefit in terms of both high school and college outcomes. Therefore, one

obvious way to increase overall program effectiveness is to improve student retention.
xxiii

While a variety of retention improvement strategies may be worthy of consideration, a logical

approach would be to address a common reason students leave the program before high school

graduation--to get a job. One way to retain participants longer would be for projects to offer students

more job opportunities in the summer. Such a strategy has a number of potential drawbacks,

including a reduction in academic instruction. In addition, to maintain the academic character and

focus of Upward Bound, an important goal would be to arrange work experiences that complement

a project’s design and curriculum. Nonetheless, although not easy to accomplish, improving

program retention rates through greater work opportunities could result in greater impacts on student

outcomes.

Serve More At-Risk Students

Students who enter Upward Bound expecting to complete less than a bachelor’s degree benefit

more from the program than their peers with higher expectations, but they account for only about

one-fifth of all students served. Furthermore, students who are more at risk of academic failure as

freshman benefit more from Upward Bound than students less at risk. Increasing the proportion of

9th graders served who have lower educational expectations or have poorer records of academic

performance could substantially boost the average impact of Upward Bound on a variety of

important outcomes.

How might project staff attract such students? Three possible strategies include (1) recruiting

more students with grade point averages in the C or D range as high school freshmen; (2) recruiting

more students who have earned relatively few credits in the core academic subjects of English, math,

science, social studies, and foreign language; and (3) working with target school personnel to identify

students who have the potential to complete a four-year degree, but who plan to pursue some other

postsecondary objective or not to attend a postsecondary school at all.

Trying to change the mix of students served may be controversial among some project staff and

may have an impact on projects’ performance measures because they would be serving a group of

students whom we would expect to be less likely to attend college. Efforts to do so should be

monitored carefully to guard against possible adverse effects and current accountability requirements

should be aligned with this objective. If ED is interested in exploring this idea, one option would

be to set up demonstration projects that enroll different proportions of students with lower

expectations, and compare impacts over time.
Previous reports have addressed the short-term impact of Upward Bound (Myers and Schirm 1

1997) and programs offered, students served, and operational issues (Moore 1997). The second fiveyear phase of the evaluation, which will end in 2002, will result in reports addressing the longer-term

(continued...)

1

I. INTRODUCTION

A college degree is becoming increasingly critical to success in the workforce. College

graduates have substantially lower unemployment rates and substantially higher incomes than

individuals with less education. Students from disadvantaged backgrounds have historically been

much less likely than their peers to complete a college education.

A major federal effort to address this concern is Upward Bound, a program designed to help

disadvantaged students prepare for college by providing supplemental academic and support

services. In FY 1998, with federal funds of more than $170 million, the program served about

42,000 students in 566 projects nationwide. Learning more about Upward Bound program

effectiveness will not only help to improve Upward Bound, but also potentially hundreds of other

precollege programs operating throughout the country. Ultimately, policy makers and program

operators must know the extent to which their efforts actually improve the educational outcomes

of economically disadvantaged students.

This report summarizes the impacts that Upward Bound has on students through their high

school years and, for some students, their first one or two years in college. It represents one piece

of an ongoing, long-term study that will track students through the college years. This is the final

report from the first five-year phase of a national evaluation of Upward Bound, sponsored by the

U.S. Department of Education (ED) and conducted by Mathematica Policy Research, Inc. (MPR)

and its subcontractors: Westat, Decision Information Resources, Educational Testing Service, and

Public/Private Ventures.1
(...continued) 1

impacts of the regular Upward Bound program, common practices of highly effective Upward Bound

projects, and the impacts and benefits of Upward Bound Math/Science projects.

U.S. Department of Education, National Center for Education Statistics. The Condition of

2

Education, 1997. Indicator 8. NCES 97-388. Washington, DC: U.S. Government Printing Office,

1997.

2

A. THE CONTEXT AND PURPOSE OF UPWARD BOUND

In the mid-1960s and early 1970s, several well-publicized studies called attention to the poor

academic preparation and low educational attainment of low-income youth (see, for example,

Coleman et al. 1966, Jencks et al. 1972, Mosteller and Moynihan 1972). Concern about major

disparities in the educational outcomes of children from different income groups resulted in the

creation of some major federal education programs as part of the War on Poverty, including Head

Start for young children and the Upward Bound program for high school-age youth. The purpose

of Upward Bound is to help prepare disadvantaged high school students to enroll and succeed in

college.

Despite the recent improved academic achievements of low-income and minority students,

substantial gaps still exist between low-income and better-off students. For example, between 1972

and 1995, the percentage of high school graduates who were enrolled in college during the October

or after graduation rose from 26 to 34 percent among students from low-income families, but at the

same time the enrollment rate among middle income students rose from 45 to 56 percent, and among

high income students it went from 64 to 83 percent. Even taking a longer view of postsecondary 2

educational attainment, students of lower socioeconomic status do not fair as well as their more

advantaged peers. Among high school sophomores expected to graduate from high school in 1982,

only seven percent of those from the lowest socioeconomic quartile had earned a bachelor's degree

or higher by 1992, compared with 22 percent of those in the middle two quartiles, and 51 percent
U.S. Department of Education, National Center for Education Statistics. The Condition of

3

Education, 1996. Table 26-2. NCES 96-304. Washington, DC: U.S. Government Printing Office,

1996.

3

of those from the highest quartile. Thus, more than 30 years after its creation, the Upward Bound 3

program seems to be needed as much as ever.

B. MAJOR FINDINGS FROM THE NATIONAL EVALUATION

1. How Upward Bound Operates Today

There are more than 550 Upward Bound projects currently operating around the nation. The key

elements of these projects are the students, the services, and the staff:

C Students. At least two-thirds of the students served by each Upward Bound project

must be both low-income and potential first-generation college students; the remainder

must meet one of these criteria. Most participants join the program in their early years

of high school. Furthermore, most of the students who currently apply for Upward

Bound are educationally motivated, have few behavioral or disciplinary problems, and

earn mostly B's or C's in school.

C Services. Upward Bound, unlike many other pre-college programs, provides students

with an opportunity to generally participate in the program for up to four years. Upward

Bound offers students an intensive program that meets regularly during the academic

year and the summer. Projects typically provide a wide variety of services, including

traditional academic instruction, tutoring, mentoring, counseling, career planning,

cultural programs, and college planning activities. In addition to regular meetings

during the academic year, programs also offer an intensive instructional program in the

summer, typically lasting about six weeks, and at almost 90 percent of projects the

summer session includes a residential component during which students live on a

college campus to help simulate the college experience. And in many (85 percent)

Upward Bound projects after participants graduate from high school, they can participate

in a summer “bridge” program that is intended to smooth the transition to college.

C Staff. Project staff generally hold a bachelor’s or graduate degree. Many staff are from

the same racial/ethnic group as the majority of their participants, which was often noted

by project staff and others as important since staff members become role models for the

students.
4

To provide a fuller picture of program operations, following are some of the major findings from

our comprehensive assessment of the Upward Bound program (Moore 1997), which drew on

national surveys of grantees and target schools as well as case studies of 20 projects:

C Upward Bound projects typically offer a rich and challenging academic program.

Projects’ heavy emphasis on academic preparation for college is evident in three areas:

(1) the number of courses offered--50 percent offer more than 17 academic courses in

the summer session and more than 10 academic courses during the school year; (2) the

nature and content of the courses--fewer than one-third of projects focus on remedial

instruction and most projects offer courses that reflect a fairly traditional precollege

prepatory curriculum; and (3) course requirements--80 percent of projects require

students to complete at least six courses, typically comprising either a traditional

curriculum focused on reading, writing, algebra and geometry, or one with a heavy focus

on science and more advanced math.

C Upward Bound is primarily student-centered; parent- and school-centered activities

are secondary. Project operations are primarily intended to influence students directly,

not to change the student’s family or school. While virtually all projects provide some

opportunities for parental involvement, the emphasis is sometimes limited. In fact,

parents may desire a closer connection to the programs.

C Projects reported high college enrollment rates for seniors. Of the students who

remain in the program through their senior year of high school, about 85 percent enroll

in college the next fall according to project reports. Furthermore, projects reported that

about two-thirds of their graduates enroll at a four-year college.

2. Previous Results Based on Students' Early Involvement in Upward Bound

Because of the timing of data collection, our previous report (Myers and Schirm 1997) was only

able to describe the short-term impact of Upward Bound on students, while most students were still

in the early high school years. The major findings of that report include the following:

C Upward Bound had early positive impacts on students' academic course taking and

educational expectations. Students in Upward Bound who entered the program while

high school sophomores or later earned, on average, about one credit (Carnegie unit)

more in core academic subjects--English, science, math, social studies, and foreign

languages--than did those who were in the control group. In addition, in the two to three

years after they applied for the program, both students in Upward Bound and their

counterparts in the control group typically lowered their expectations for how many
5

years of schooling they would complete; however, the decline was smaller for those who

were in Upward Bound than for those who were not.

C Students with lower educational expectations benefit most from Upward Bound.

Among students who initially did not expect to complete a college degree, those who

were selected for Upward Bound earned about three more academic credits than those

who did not participate. In comparison, among students with higher initial educational

expectations, students in Upward Bound earned only about .5 more academic credits

than those in the control group. We also found a positive impact among the lower

expectations group, but not the higher expectations group, on parents' expectations for

the amount of schooling their children would complete, as reported by the students.

C Hispanic students initially benefit more from Upward Bound than do students from

other racial/ethnic groups. Among the three largest racial/ethnic groups in Upward

Bound--African Americans, Hispanics, and whites--Hispanics consistently experienced

the largest benefits from participating in the program. For example, Hispanic Upward

Bound students earned about two credits more in core academic subjects than did

Hispanic students in the control group. In comparison, African American and white

participants earned less than .5 credits more than nonparticipants from those

racial/ethnic groups (differences that were not statistically significant).

The earlier report also found that Upward Bound participants have a high early attrition rate.

An estimated 37 percent of students who entered Upward Bound left the program within the first 12

months, especially to take jobs. Moreover, longer participation in Upward Bound was associated

with earning more high school credits than those who left the program sooner.

C. KEY TOPICS ADDRESSED IN THIS REPORT

This report builds upon the results of our earlier work; it addresses many of the same issues, and

is based on the same students, but adds data collected two to three years later. The general question

underlying this report is how Upward Bound affects students as they continue to progress in their

schooling and other areas. Specifically, the report addresses the following major research questions:

C To what extent does Upward Bound further the academic and personal development of

students during high school?

C Does Upward Bound have an impact on college access and retention?
The sample represents Upward Bound projects that are (1) located in the 50 states or the

4

District of Columbia; (2) hosted by a postsecondary institution; (3) mature, having operated for at

least three years by October 1992; and (4) not dedicated to serving only students with physical

disabilities.

6

C Do some groups of students benefit more from Upward Bound than other students?

C What are the typical experiences of students in Upward Bound, for example, in terms

of how long they participate? Does the amount of time students spend in the program

have an influence on various outcomes?

D. RESEARCH DESIGN

To address the questions listed above, MPR has implemented the first rigorous evaluation of

Upward Bound since the late 1970s. The key feature of the research design is the use of random

assignment to select students for Upward Bound in a nationally representative sample of Upward

Bound projects. The random assignment design allows us to compare education-related outcomes

of students in the program to outcomes of students without access to the program.

1. Random Assignment

From the list of regular Upward Bound projects, we randomly selected 70 for the program

impactsstudy. However, 11 of the selected projects could not participate or had to be ruled out for 4

various reasons. For example, some were defunded by ED during the 1991-92 grants competition;

others did not plan to recruit new students for the 1992-93 school year; and some had too few

applicants to accommodate random assignment. We replaced eight of these 11 projects with similar,

randomly selected projects, and ended up with 67 Upward Bound projects.

During the 1992-93 and 1993-94 school years, we randomly assigned eligible applicants from

each site to either a treatment group, which was invited to participate in the program, or a control
We later learned that project directors had allowed a total of 29 control group members, out of 5

more than 1,300 students in the control group, to participate in Upward Bound. Although these

students continued to receive program services, we maintained their original status as members of

the control group. While this results in a trivially small underestimate of program impacts, to

consider these students members of the treatment group or remove them from the analysis sample

would violate the random assignment design and reduce the study's internal validity.

To accommodate project wishes concerning the composition of the participants--such as sex

6

or racial/ethnic balances--we used stratified random sampling when selecting the treatment and

control groups. In addition, before random assignment, we asked project directors to rate each

applicant as either most likely to be selected under normal selection procedures, somewhat likely to

be selected, or least likely to be selected. Later analyses of these three groups showed that they had

few differences in their observable characteristics and few differences in who benefitted most from

the program. These findings suggest that random selection of students meeting the federal and

project-specific selection criteria did not substantively alter the kinds of students projects normally

serve.

7

group, which was not invited to participate. Eligible applicants were defined as students the

5

projects had recruited who met both the federal requirements (concerning income or first-generation

status), as well as any project-specific criteria for participation. All the projects had more applicants

than openings, and all ended up serving the same number of students they would have served under

their usual selection procedures. We implemented the random assignment slowly over 14 months 6

so that projects could use their normal recruiting procedures and enroll students following their

normal enrollment schedules. Nationwide, the random assignment process resulted in a treatment

group of about 1,500 students and a control group of about 1,300. A more detailed description of

our random assignment procedures is presented in Myers et al. (1993).

The random assignment design allows us to calculate estimates of program impacts that are free

of selection biases. The only difference between the treatment and control groups is that the former

was offered the opportunity to participate in Upward Bound; otherwise, the two groups are

statistically equivalent (see Myers and Schirm 1997). Comparing the outcomes for the two student

groups indicates, for all applicants, the impact of being offered a chance to participate in Upward

Bound. However, not all students randomly selected for the treatment group chose to enroll in the
The “no-show” rate for the entire treatment group and for various subgroups (for example, sex 7

and grade level) are shown in Chapter III.

8

program. By making statistical adjustments, we can estimate the impacts of program participation. 7

In subsequent chapters, we provide both types of estimates. The sizes of the two types of estimates

are generally close, but in our summary of findings we focus on impacts for all students.

2. Data Collection

The analyses in this report are based on information from students, their schools, and the

Upward Bound projects they applied to or participated in. Virtually all sample students completed

a baseline written questionnaire at the time of application, prior to random assignment. In the spring

of 1994 we conducted a first follow-up survey of the students, by telephone, achieving a response

rate of 97 percent. In 1996 we conducted a second follow-up telephone survey, achieving a response

rate of about 85 percent. Each of these surveys addressed a wide variety of topics, including the

students' background; experiences related to school, supplemental services, and employment; and

their plans for the future. With the students' permission, we also collected their official school

transcripts in 1994 and 1996, to determine the number and type of courses they had taken. At the

start of our study, the directors of the 67 participating projects completed a written questionnaire

about various aspects of program operations.
9

E. PHASE II OF THE NATIONAL EVALUATION OF UPWARD BOUND

The U.S. Department of Education (ED) has awarded a contract to MPR to follow the current

sample of students for five more years to assess the program’s impact on a series of college related

outcomes, such as access to, retention in, and completion of college; selectivity of the colleges

attended; college major; and employment related outcomes. Reports describing program impacts

will be available in 1999 and 2001.

ED has also funded two studies linked to the ongoing evaluation. The first is an assessment of

the practices used by nine Upward Bound projects that have potentially large impacts on key

outcomes, such as educational expectations, and credits earned in core high school subjects. MPR

staff and its subcontractors will visit these projects twice: once during the school year and another

time during the summer program in 1998. The final report describing the findings from the case

study visits will be available in early 1999. The second new study is an evaluation of the effects of

Math/Science centers (MSCs), funded under the Upward Bound Math/Science Initiative, on

students’ outcomes, such as college major. This evaluation will compare the experiences of a sample

of MSC participants with those of similar students who are already part of the national evaluation.

Reports describing the effects of MSC participation on student outcomes will be available in late

1999 and 2001.

F. ORGANIZATION OF THIS REPORT

The remainder of this report is organized into four chapters. Chapter II describes students’

participation and persistence in Upward Bound. Chapter III addresses the impact of Upward Bound

on students in terms of educational expectations, courses in high school, college attendance, and

other important outcomes. We also analyze outcomes for different groups of students, such as

students with relatively lower and higher initial educational expectations, boys and girls, and various
10

racial/ethnic groups. Chapter IV discusses how the length of time students spend in Upward Bound

affects various secondary and postsecondary outcomes. Chapter V presents our conclusions and a

discussion of potential policy implications. In addition, several appendices present details

concerning the analyses conducted for this report.
11

II. THE UPWARD BOUND EXPERIENCE: PARTICIPATION,

COMPLETION, PERSISTENCE, AND INTENSITY

In this chapter, we examine the Upward Bound experiences of the roughly 1,500 eligible

applicants randomly selected between late 1992 and early 1994 for participation in the program.

Specifically, we answer the following questions:

C What fraction of eligible applicants choose to participate in the program when given the

opportunity? Which ones participate? Why do some not participate?

C What fraction of participants remain in the program until they graduate from high

school? Which ones are more likely to remain in the program that long?

C How long do participants typically remain in the program? Do some remain in the

program longer than others? Why do some participants leave early?

C In what activities do participants engage while in the program?

C Are some projects better able to retain students than other projects?

The answers to these questions are important for gauging the effectiveness of the program in

attracting and retaining students. They also provide a context for interpreting the impact results

presented in this report, indicating how much of a “dose” of Upward Bound the program’s

participants typically get. Finally, the findings presented in this chapter are a critical input to our

analysis of the effects of duration on student outcomes. The key results from our analysis of

students’ Upward Bound experiences are:

C Out of the roughly 20,000 eligible applicants offered the opportunity to participate in

Upward Bound each year, about 16,400 enter the program, and 10,600 remain in

Upward Bound for at least one year, as depicted in Figure II.1. About 7,200, at most,

ever complete the program, remaining in Upward Bound until high school graduation.

The typical Upward Bound student participates in the program for 19 months and

attends nearly 400 sessions of Upward Bound academic and nonacademic activities.
FIGURE II.1

A COHORT’S PATH TO COMPLETING UPWARD BOUND

20 Eligible Applicants

Admitted to Upward

Bound

7 Participate until

High School

Graduation

11 Participate for

at least 1 Year

16 Enter the Program

12
Even though it does not matter for our results how slots are allocated to eligible applicants, it 1

may matter substantially how the pool of eligible applicants is formed. Our results pertain to the

types of eligible students who are recruited for and apply to the program and whose characteristics

were described earlier in this report. If Upward Bound projects began recruiting a substantially

different type of eligible student, such as students from the bottom of the grade distribution, the new

recruits who apply and are offered admission to the program may have participation, completion, and

persistence rates that are very different from the rates of current Upward Bound students.

13

C Some groups of students participate and persist in Upward Bound longer or complete

the program at higher rates than other groups. For example, younger participants stay

in longer but are less likely to complete the program than older participants. Students

with lower educational expectations are as likely to enter Upward Bound but have lower

persistence and are less likely to complete the program than students with higher

expectations. First generation only students tend to enter and complete Upward Bound

at lower rates than low-income students. Boys and girls have similar patterns of

participation.

C There are no consistent differences among the participation patterns of students from

different types of projects, such as small and large projects.

The principal data analyzed in this chapter were collected in the late summer and fall of 1994,

1995, and 1996. For each Upward Bound student in the study, program staff provided information

on the periods and activities in which the student participated, indicating when the student started

and permanently left the program, as well as any temporary withdrawals. Although the Upward

Bound students in the study were randomly selected, comparisons of students who would normally

be selected for Upward Bound with students who would not normally be selected (as identified by

project directors) demonstrate that our results reflect the typical experiences of Upward Bound

students. Our results are not affected by random assignment having replaced the program’s usual

selection procedures.1

A. PROGRAM PARTICIPATION

More than one in six eligible applicants (18 percent) offered the opportunity to participate in

Upward Bound never do so. To place this in perspective, the Job Corps program which serves a
The results presented in this section are essentially the same as the results presented in Myers 2

and Schirm (1997), except here we consider the effects on participation of a broader range of student

and project characteristics. In the next two sections, we are able to substantially refine our earlier

analyses because much additional data on participation have become available.

14

more disadvantaged group of youth, has a no-show rate that is somewhat higher than Upward

Bound--about 25 percent. Students give a wide range of reasons for not participating, with no one

reason given very frequently, as shown in Table II.1. The nonparticipation rate is 18 percent in spite 2

of the lengthy and rigorous recruiting and application process during which program staff and each

student can assess the student’s interests and motivation. This process gives a student ample

opportunity to determine that Upward Bound may not be the program for him or her and to be

removed from consideration for the program. Nevertheless, a substantial fraction of students

completing the process choose not to take advantage of Upward Bound when it is offered.

The chances of participating in Upward Bound are associated with only a few observed student

and project characteristics: race, scheduled time of entry into the program, and being in the sexual

minority within a project. The relationships between other characteristics and participation rates are

weak or, if seemingly strong, not statistically significant. For some key subgroups of students, the

estimated participation rates are: 81 percent for females, 84 percent for males, 78 percent for African

Americans, 83 percent for whites, 91 percent for Hispanics, 82 percent for students who expected

to complete a four-year college degree, and 81 percent for students who did not expect to complete

four-year college degree.

These subgroup participation rates are “unadjusted,” reflecting differences between subgroups

in many characteristics. Thus, the three percentage point difference between the participation rates

for females and males is attributable to not only the difference in and influence of sex on
15

TABLE II.1

MOST COMMON REASONS FOR NOT PARTICIPATING IN UPWARD BOUND

Reason Percentage Giving Reason

Transportation problems 20

Took a job 18

Program did not contact me 18

Time conflict 15

Family issues 11
For each characteristic, we have identified by a “**” a reference category, which is the most

3

common attribute among Upward Bound students. Considering the relationship between sex and

participation rates, for example, we see in Table II.2 that the reference category is female because

more Upward Bound students are female. The adjusted participation rate of 72 percent for males

shows the frequency of participation that we predict would occur among males if they had the same

distribution of other characteristics (e.g., race and educational expectations) as females. Because

females already have that distribution of other characteristics without any adjustment, the adjusted

and unadjusted participation rates for females are the same--81 percent, or 9 percentage points higher

than the adjusted rate for males. As we noted earlier, males actually have a higher unadjusted

participation rate than females (86 versus 81 percent), indicating that males are more likely than

females to have other characteristics associated with high participation rates, in particular, males are

much more often in the (sexual) minority in their projects. Imposing a common distribution of all

the other characteristics listed in Table II.2, we have “adjusted” for the effects on participation of

extraneous differences between males and females to isolate the relationship between sex and the

Upward Bound participation rate. The adjusted participation rates in Table II.2 were obtained using

a logistic regression model, which is an appropriate specification when the phenomenon of interest

has two outcomes (e.g., participate/did not participate). Each “p-value” in Table II.2 indicates the

lowest significance level at which we would reject the hypothesis that there are no differences among

categories of a characteristic. Generally, we are regarding a relationship as significant if the

associated p-value is under 0.10.

16

participation, but also the differences between males and females and the influences of race,

educational expectations, and many other characteristics.

Table II.2 displays “adjusted” participation rates. These rates allow us to contrast the

participation rates of students who have all the same characteristics except for one, such as sex or

race.3

Among racial groups, Asians and Hispanics have the highest participation rates at 90 percent

or higher according to Table II.2. The participation rates for African American, white, and Native
17

TABLE II.2

ADJUSTED PARTICIPATION RATES

Characteristic Participation Rate (%) p-value for P *

2

Sex 0.15

Female** 81

Male 72

Race 0.09

African American** 78

Hispanic 90

Asian 93

White 79

Native American 78

Scheduled entry into Upward Bound 0.09

During 8th grade 90

Summer after 8th grade 86

During 9th grade 71

Summer after 9th grade** 83

During 10th grade 88

Summer after 10th grade 72

During 11th grade 82

Summer after 11th grade 63

Low-income/first generation 0.11

Both low-income and first generation** 82

Low-income only 95

First generation only 77

Project director’s rating 0.25

Most likely to serve** 85

Somewhat likely to serve 78

Least likely to serve 87

College plans 0.89

Plan to complete a degree** 82

Do not plan to complete a degree 82

Student in racial minority within project 0.13

yes 87

no** 80

Student in sexual minority within project 0.08

yes 89

no** 80

Project host institution type and control 0.43

Four-year public institution** 79

Four-year private institution 84

Two-year institution 78
TABLE II.2 (continued)

Characteristic Participation Rate (%) p-value for P *

2

18

Project is racially diverse (dominant race has less than 0.11

60 percent of students)

yes 87

no** 80

Project is racially homogeneous (dominant race has 100 0.91

percent of students)

yes 82

no** 82

Project is sexually diverse (dominant sex has less than 0.26

60 percent of students)

yes 83

no** 78

Academic requirements 0.49

Math/Science** 77

Foundational 76

Structured 75

Unstructured 82

Uncategorized 88

 *p-values corresponds to the probability of rejecting the hypothesis that one or more of the participation rates differ

 from the others in the group by chance alone.

**The reference attribute.

SU0118 (5/11/98, 15:44) and PRED9A.WK4.
The Hispanic-Native American difference is significant at the 0.11 level. For multiple category 4

characteristics like race, the results of pairwise comparisons such as Hispanic-Native American or

Asian-African American are not displayed in Table II.2, although they are discussed when

interesting.

If we set aside the possibly anomalous result for the ninth grade academic year, we see that

5

academic year participation rates are fairly stable, while summer participation rates are lower and

decline with age. Sample sizes are not sufficiently large for even some of the bigger differences to

be significant. However, students scheduled to enter before ninth grade or during tenth grade do

have significantly different participation rates from students scheduled to enter during ninth grade

or during the summers after tenth and eleventh grades.

19

American students, who together constitute about three quarters of Upward Bound students, are all

just under 80 percent, significantly different from the rates for Asians and Hispanics.4

Perhaps the most puzzling result from our analysis of participation is the relatively low

participation rate for students selected to enter the program during ninth grade. Furthermore, for

each of the other three grades, there is a lower participation rate--sometimes a much lower rate--for

students scheduled to start in the summer than for students scheduled to start in the academic year.

This pattern probably reflects the coincidence of two phenomena: (1) the Upward Bound program

is more intensive and more demanding during the summer and (2) there are more conflicting

activities, such as jobs and vacations, confronting the student during the summer. As expected, the

summer “drop off” becomes more pronounced as students age. Because ninth grade is the first year

of high school for many students, perhaps it is the many academic and social pressures associated

with entering high school that explain why students who are supposed to enter the program during

the ninth grade academic year participate at a lower rate than students who are supposed to enter the

summer after ninth grade and, indeed, at a lower rate than any group except students scheduled to

start the summer after eleventh grade.5

In addition to the differences in participation rates by race and scheduled time of entry into the

program, we find other significant differences:
The difference in participation rates between students who will be in the racial minority and

6

students who will not is not statistically significant at the level that we are using for most

comparisons (0.10), although it is significant at a slightly higher level (0.13). The larger estimated

difference in participation rates between students who will be in the sexual minority and students

who will not is significant at the 0.08 level.

In preliminary analyses, we also found no significant association between student participation, 7

completion, or persistence rates and the size of projects or their location (urban/rural). Therefore,

we excluded these two project characteristics from the analyses presented in this chapter.

20

C Students who are potential first generation college, whether low-income or not, are

much less likely to participate than students who are low-income only.

C Students who will be in the minority in their projects--by either race or sex--are more

likely to participate than students who will not. That such students are even willing to 6

apply under such circumstances may suggest that they are more motivated to overcome

obstacles like the prospect of “not fitting in.” Alternatively, they may be more highly

recruited by project staff seeking to diversify their programs.

We found that a wide range of other observable student and project characteristics such as

educational expectations and the type and control of the program’s host institution do not seem to

be associated with the likelihood of entering the program.7

The most important finding of no significant differences among participation rates is for project

director’s rating. Prior to the random assignment of students, each Upward Bound project director

rated eligible students as “most likely to serve,” “somewhat likely to serve,” or “least likely to

serve.” The “most likely to serve” students are essentially the students the project directors would

have selected in the absence of random assignment and are, therefore, the kinds of students who

would typically be given the chance to participate in Upward Bound. We found that the participation

rate for these students is not significantly different from the participation rates for students in the

other two groups. Thus, the estimated participation rate for the students who were selected at

random for Upward Bound is probably typical for the program and not a consequence of offering

Upward Bound to students who would not normally be selected.
For the small number of students graduating between terms of the academic year, a program

8

completer was still participating in the month of graduation.

This definition ignores participation in projects’ summer bridge programs. Hence, it is a liberal 9

definition, overstating completion rates for students in projects where the bridge component is an

important element of the Upward Bound program.

If all of the students who were still participating in Upward Bound in August 1996 complete 10

the program, which is unlikely, the final completion rate will be 44 percent. If none of those students

complete the program, the completion rate will drop to about 31 percent. Nearly all of the students

who had not graduated before August 1996 and were still in Upward Bound had entered the program

relatively “early” (before the end of ninth grade). Our calculated minimum and maximum overall

completion rates suggest that the completion rates for these early entrants will likely fall below the

rates for the later entrants. Such an outcome would imply that the effect of getting a student started

early is not strong enough to overcome the effect of being exposed longer to competing

opportunities--for example, jobs and extracurricular activities--and other obstacles to completion.

21

B. PROGRAM COMPLETION

Fewer than half of the students (44 percent) who start Upward Bound complete the program.

By “complete,” we mean that a student was still participating in May of the year that the student

graduated from high school. To calculate the fraction of students completing the program, we only 8,9

counted students who had graduated from high school before August 1996, the last month covered

by our data on participation. Because many (nearly 60 percent) of the students who will graduate

later had already permanently dropped out of Upward Bound before August 1996, the “final”

completion rate that will be obtained after future rounds of data collection will fall within a range

of 31 percent to 44 percent.10

Although program completion rates do not seem to vary by most student and project

characteristics, there are large differences associated with time of entry into Upward Bound, lowincome/first generation status, and college plans. For some key subgroups of students, the estimated

completion rates are: 47 percent for females, 42 percent for males, 46 percent for African

Americans, 39 percent for whites, 48 percent for Hispanics, 49 percent for students who expected

to complete a four-year college degree, and 34 percent for students who did not. These rates are
22

unadjusted. Thus, the difference in completion rates between females and males reflects all the ways

in which female and male Upward Bound applicants are different, including any differences that may

exist in, for example, their ages when they enter Upward Bound or their college plans.

Table II.3 presents adjusted program completion rates that were calculated the same way as the

adjusted participation rates presented earlier (in Table II.2) to control for differences among

subgroups. We found a strong association between when a student enters Upward Bound and the

student’s chances of completing the program. Completion rates rise fairly steadily as entry is

delayed, with the highest rate obtained for students entering the program after eleventh grade. This

pattern could have several explanations. The simplest is that students entering late do not have to

persist very long to complete the program. The estimates suggest that although there may be
23

TABLE II.3

ADJUSTED COMPLETION RATES

Characteristic Completion Rate (%) p-value for P

2

Sex 0.42

Female* 47

Male 42

Race 0.19

African American* 46

Hispanic 46

Asian 59

White 38

Native American 34

Entry into Upward Bound 0.03

During 8th grade --

Summer after 8th grade --

During 9th grade 25

Summer after 9th grade* 36

During 10th grade 43

Summer after 10th grade 49

During 11th grade 50

Summer after 11th grade 62

Low-income/first generation 0.02

Both low-income and first generation* 47

Low-income only 45

First generation only 31

Project director’s rating 0.75

Most likely to serve* 46

Somewhat likely to serve 43

Least likely to serve 48

College plans 0.01

Plan to complete a degree* 49

Do not plan to complete a degree 33

Student in racial minority within project 0.88

yes 45

no* 44

Student in sexual minority within project 0.55

yes 43

no* 47

Project host institution type and control 0.14

Four-year public institution* 41

Four-year private institution 53

Two-year institution 49
TABLE II.3 (continued)

Characteristic Completion Rate (%) p-value for P

2

24

Project is racially diverse (dominant race has less than 60 0.68

percent of students)

yes 42

no* 45

Project is racially homogeneous (dominant race has 100 0.33

percent of students)

yes 54

no* 43

Project is sexually diverse (dominant sex has less than 60 0.34

percent of students)

yes 50

no* 44

Academic requirements 0.18

Math/Science* 45

Foundational 36

Structured 53

Unstructured 36

Uncategorized 37

*The reference attribute.

SU0118 (5/11/98, 15:44) and PRED9C.WK4.
Completion rates are not presented for students entering before ninth grade because those

11

students had not graduated before August 1996. The reported rate for students entering during ninth

grade is based on the subset of those students (about one-fifth of the total) who had graduated before

that date. The highest possible completion rate for students graduating after August 1996 is 43

percent. This assumes that all the students still in Upward Bound on that date complete the program.

Not all of the estimated differences are statistically significant. The lowest completion rate-- 12

for students entering during ninth grade--is significantly different from the rates for students entering

during the tenth grade or later. The next lowest completion rate--for students entering the summer

after the ninth grade--is significantly different from the rates for students entering the summers after

tenth and eleventh grades and, at the 0.11 level, from the rate for students entering during eleventh

grade. The only other significant difference is between the rates for students entering during the

summers after tenth and eleventh grades.

The rates for the low-income only and first generation only students are not significantly

13

different because these groups are fairly small. In fact, under program regulations, they cannot

exceed one-third of any project’s students.

“Math/science” programs require calculus or pre-calculus, at least three science courses, and 14

at least six courses in total. “Foundational” programs do not require the math/science curriculum,

but they do require six courses in total, including reading, writing, Algebra I, Algebra II, and

geometry. “Structured” programs require six courses in total, but neither the math/science nor the

foundational curriculum. “Unstructured” programs require fewer than six courses. This program

(continued...)

25

important benefits to early entry, a higher completion rate is not one of them. As we will see in the

next section, however, longer duration in the program is a potential benefit.11,12

We also found large differences in completion rates associated with other characteristics:

C Asian students are more likely to complete the program than white and Native American

students. Other differences in participation rates by race are not statistically significant,

even though some are fairly large.

C Students who are from low-income families are more likely to complete the program

than students who are not.13

C Students who, before entering the program, expected to earn a college degree are much

more likely to complete the program than students who did not.

C Students in “math/science” and “structured” programs are more likely to complete the

program than other students. However, if it is the high degree of structure that drives

students to complete the program, it is not clear why students in “foundational”

programs have relatively low completion rates.14
(...continued) 14

typology is described in greater detail in Fasciano and Jacobson (1995). None of the differences

involving math/science programs is significant, but the differences between structured programs and

unstructured and foundational programs are significant.

26

C Students in projects hosted by four-year private institutions are more likely to complete

the program than students in projects hosted by four-year public institutions. There is

no obvious explanation for this.

We have discovered that many student and project characteristics are associated with differences

in completion rates. That many are not is equally important. For example, the completion rates for

males and females are not substantially different. Also, being in the minority (by race or sex) in

one’s project does not seem to make it either easier or harder to “stick it out” and complete the

program. Perhaps project staff work hard to retain minority students and successfully address any

problems such students may encounter. Finally, there is no significant association between the

project director’s rating of the likelihood of accepting a student and the student’s chances of

completing the program. Therefore, our inferences about Upward Bound completion rates are not

affected by our having replaced the usual selection procedures by random assignment; that is, the

experiences of our study sample reflect the typical experiences of Upward Bound participants.

C. PROGRAM PERSISTENCE

The previous section examined the fraction and type of students who complete Upward Bound.

This section considers how long students are in the program, whether they complete it or not.

Duration merits examination separate from completion for two reasons. First, noncompleters who

are in Upward Bound for a long time may derive substantial benefit from the program even though

they did not finish it. Second, the experience and benefits of a completer who entered the program
These estimates of students leaving Upward Bound include students who leave because they 15

have completed the program and students who leave early.

These and the other estimates in Table II.4 are from Kaplan-Meier survival functions, which 16

properly account for the fact that some students were “censored,” that is, they were still participating

in the program at the end of the observation period (August 1996).

If all students still participating in August 1996 remain in the program until completing it, the 17

three year survival rate will end up being about one in six rather than one in seven. However, the

estimates of the one year and two year survival rates and the median duration will not change.

This figure is lower than the completion rates presented in the previous section because we

18

have included in the base of the percentage the program dropouts who had not yet graduated from

high school.

27

in the ninth grade may be very different from the experience and benefits of a completer who entered

in the eleventh grade.

Just over one-third of the students entering Upward Bound leave the program within the first

year, and nearly two-thirds leave within the first two years, as shown in Table II.4. About one in 15

seven remain in the program for three years. The median length of participation in Upward Bound

is 19 months. Among students participating roughly this long (16 to 21 months, in fact), about 16,17

one-third are in Upward Bound for one summer and parts of two academic years. Just over 40

percent are in for two summers, with such students splitting almost evenly between those in for just

one academic year and those in for parts of two academic years.

Students leave Upward Bound either when they complete the program or when they choose to

leave (or are asked to leave) before completing the program. About 36 percent of the students whom

we have observed leaving the program left upon completing Upward Bound. The remaining

18

students left early. We found that the most common reason for leaving Upward Bound before

completing the program is to take a job (see Table II.5). That nearly half of the noncompleters drop

out of the program in May, June, or July suggests, perhaps, that students feel pressure to take a
28

TABLE II.4

DURATION IN PROGRAM

Percentage in Program for

Group Median Duration (months) 1 year 2 years 3 years

Overall 19 65 36 14

Entry into Upward Bound

During 8th grade 19 67 42 42

Summer after 8th grade 32 75 58 48

During 9th grade 16 55 42 15

Summer after 9th grade 24 70 49 15

During 10th grade 20 67 44 --

Summer after 10th 15 63 6 --

During 11th grade 17 59 -- --

Summer after 11th grade 13 53 -- --

ST0109 (12/31/97, 14:27)

TABLE II.5

MOST COMMON REASONS FOR LEAVING UPWARD BOUND

BEFORE COMPLETING THE PROGRAM

Reason Percentage Giving Reason

Took a job 30

Too busy 12

Moved 11

Asked to leave 11
A student who does not graduate from high school on time could participate longer. 19

The median duration is the duration at which the percentage remaining in the program drops 20

below 50 percent. The 12 and 18 month survival rates are the estimated percentages of participants

remaining in the program for at least 12 and 18 months, respectively. Estimates were calculated

from a discrete-time proportional hazards model.

29

summer job. This implies that program persistence and completion rates might improve if Upward

Bound could be made more compatible with work, especially during the summer.

Not surprisingly, how long a student participates is strongly influenced by when the student

starts. According to Table II.4, which presents duration estimates that are not adjusted for student

and project characteristics, students entering Upward Bound after the eighth or ninth grade persist

longer than other students. Of course, this effect arises partly because students eventually become

ineligible. Students entering after eleventh grade, for example, can only be in the program for 12

to 14 months, depending on whether they attend a summer bridge program. Students entering at 19

the end of eighth grade have the opportunity to remain in the program much longer. For some key

subgroups defined by characteristics other than time of entry into the program, unadjusted median

durations are--in months--20 for females, 17 for males, 20 for African Americans and whites, 15 for

Hispanics, 21 for students who expected to complete a four-year college degree, and 15 for students

who did not.

In Table II.6, we present median durations and 12- and 18- month survival rates. These

20

estimates are adjusted for a wide range of student and project characteristics, like the participation

and completion rates presented earlier.

According to Table II.6, the largest differences in persistence are associated with when a student

entered the program. As noted before, students entering Upward Bound before the tenth grade tend
30

TABLE II.6

ADJUSTED DURATION ESTIMATES

Survival Rate (%)

Characteristic Median Duration 12 month 18 Month p-value for P

2

Sex 0.35

Female* 20 65 52

Male 19 63 50

Race 0.26

African American* 20 63 52

Hispanic 22 68 58

Asian 24 70 61

White 22 67 57

Native American 22 68 58

Entry into Upward Bound 0.00

During 8th grade 25 70 58

Summer after 8th grade 35 82 74

During 9th grade 16 60 46

Summer after 9th grade* 24 69 56

During 10th grade 13 53 37

Summer after 10th grade 11 43 27

During 11th grade 6 29 15

Summer after 11th grade 5 22 10

Low-income/first generation 0.45

Both low-income and first generation* 19 65 51

Low-income only 22 69 56

First generation only 19 64 51

Project director’s rating 0.94

Most likely to serve* 20 65 53

Somewhat likely to serve 20 65 53

Least likely to serve 21 66 54

College plans 0.19

Plan to complete a degree* 21 65 54

Do not plan to complete a degree 18 61 50

Student in racial minority within project 0.46

yes 18 64 50

no* 20 66 53

Student in sexual minority within project 0.04

yes 17 61 48

no* 20 66 53

Project host institution type and control 0.03

Four-year public institution* 16 59 46

Four-year private institution 22 66 55

Two-year institution 21 65 53
TABLE II.6 (continued)

Survival Rate (%)

Characteristic Median Duration 12 month 18 Month p-value for P

2

31

Project is racially diverse (dominant race has 0.47

less than 60 percent of students)

yes 17 60 48

no* 18 62 50

Project is racially homogeneous (dominant 0.31

race has 100 percent of students)

yes 20 65 52

no* 17 62 48

Project is sexually diverse (dominant sex has 0.44

less than 60 percent of students)

yes 18 62 49

no* 16 60 46

Academic requirements 0.48

Math/Science* 16 56 45

Foundational 15 53 42

Structured 17 59 48

Unstructured 19 61 50

Uncategorized 15 54 43

*The reference attribute.

SU0118 (5/11/98, 15:44) and PRED9B.WK4.
32

to remain in the program longer than students entering later. Students entering during the summer

after eighth grade persist the longest at a rate significantly different from the rate for any other group.

The shortest durations are observed for students entering Upward Bound during or after the eleventh

grade, with all observed differences in rates being significant.

Other characteristics that are associated with persistence in Upward Bound are: race, whether

the student is in the sexual minority within a project, and type and control of the project’s host

institution. Native American, Asian, Hispanic, and white students have similar patterns of program

persistence, and they remain in the program longer than African American students, although the

Native American-African American difference is not statistically significant. Large and statistically

significant differences are also associated with the type and control of the host institution. For

reasons that are not clear, the lowest persistence rates are found for students whose Upward Bound

programs are hosted by public four-year institutions, rather than private four-year institutions or twoyear institutions.

D. PROGRAM INTENSITY

The preceding sections in this chapter examined whether students participate in Upward Bound,

whether they complete the program, and how long they are in it. This sections looks at what students

do while they are in Upward Bound.

Upward Bound is an intensive program in which participants engage in many academic and

nonacademic activities. Table II.7 shows that during a typical Upward Bound summer program, the

average student attends just over 100 sessions of academic courses and about 70 sessions of
33

TABLE II.7

STUDENT PARTICIPATION IN UPWARD BOUND ACADEMIC COURSES

AND NONACADEMIC ACTIVITIES DURING THE SUMMER

Typical Number of Sessions Attendeda

Percentage Enrolled All Students Enrolled Students b b

All courses: 99 103 104

English 88 30 30

ESL 0 0 24

Foreign languages 22 0 23

Math 95 24 24

Computers 24 0 24

Science 72 20 24

Social science 14 0 24

Elective 77 20 24

Other 6 0 24

All activities: 100 72 72

College preparation 93 13 14

Career exploration 45 0 3

Self-awareness 58 1 2

Field trips 91 4 4

Cultural awareness 58 1 2

Counseling 92 13 17

Skill development 87 21 24

Other 6 0 1

All courses and activities 100 181 181

“Typical” refers to the median number of sessions attended and number of sessions is defined as the number a

of times a student attended a course, for example.

“All students” refers to all Upward Bound participants, regardless of whether they actually enrolled in a

b

course. “Enrolled students” refers to only those who were actually enrolled.
The estimates in Table II.7 pertain to the first summer of program participation by students

21

who participated at least one summer. These figures reflect a typical summer experience because

we find no evidence suggesting that a student’s level of activity rises or falls substantially over time

so long as the student remains in the program. Two related findings are (1) the level of activity does

not vary much according to the student’s grade in school and (2) students who remain in the program

only a short time appear to participate as fully while they are in the program as the seemingly more

dedicated students who are in the program much longer. We obtain similar findings for the academic

year. Hence, the estimates presented in Table II.8 later in this section pertain to the first academic

year of program participation by students who participated at least one academic year. 

34

nonacademic activities. Nearly all students take at least one math course, and almost 90 percent 21

take at least one English course. Nearly three-quarters of the students take a science course and an

elective. Other types of courses are taken by relatively few students. Students taking a course in a

particular subject attend about 24 sessions, except in English for which the average number of

sessions attended is 30. These figures imply about 3 to 5 meetings per week for an Upward Bound

summer course.

During the summer, participation rates are high--sometimes exceeding 90 percent--for most of

the nonacademic activities listed in Table II.7. The activities involving the most sessions are college

preparation, counseling, and skill development, with 14 to 24 sessions--2 to 3 per week--attended

by students engaged in those activities.

Not surprisingly, students are much less active in Upward Bound during the academic year.

According to Table II.8, the average student attends 36 sessions of academic courses and 43 sessions

of nonacademic activities. About one-quarter of the students take no academic courses through the

Upward Bound program. Participation rates in science, math, and English range from one-half to

two-thirds. Students taking a course in one of those subjects typically attend 15 to 20 sessions during

the academic year. Although participation rates for most nonacademic activities are high during the

academic year, attendance tends to be infrequent.
35

TABLE II.8

STUDENT PARTICIPATION IN UPWARD BOUND ACADEMIC COURSES

AND NONACADEMIC ACTIVITIES DURING THE ACADEMIC YEAR

Typical Number of Sessions Attendeda

Percentage Enrolled All Students Enrolled Students b b

All courses: 75 36 60

English 67 12 20

ESL 0 0 10

Foreign languages 7 0 14

Math 64 8 15

Computers 7 0 17

Science 50 0 15

Social science 7 0 15

Elective 42 0 15

Other 2 0 6

All activities: 99 43 45

College preparation 81 5 9

Career exploration 20 0 2

Self-awareness 61 1 3

Field trips 73 2 3

Cultural awareness 58 1 2

Counseling 92 11 12

Skill development 87 12 15

Other 4 0 1

All courses and activities 100 81 81

“Typical” refers to the median number of sessions attended. a

“All students” refers to all Upward Bound participants regardless of whether they actually enrolled in

b

a course. “Enrolled students” refers to only those who were actually enrolled.
Across different measures of activity levels (numbers of sessions attended), we found that 65 22

to 85 percent of the variability among all students is attributable to variability among students in

different projects, and only 15 to 35 percent is attributable to variability among students in the same

project. Thus, while the activity levels of different students in a given project vary only narrowly

about the project average, the average activity levels in different projects vary widely about the

average for the whole Upward Bound program.

This total career sessions figure surely understates somewhat the typical dose of Upward

23

Bound because as noted earlier, about 13 percent of the students in the evaluation sample were still

active and had up to one more year in the program when their activities were last reported to us.

36

As noted earlier, the total number of academic or nonacademic activities in which a student is

engaged during a summer or an academic year does not seem to rise or fall as the student remains

in the program longer and does not seem to vary according to the student’s grade in school.

Furthermore, all of the students in a given Upward Bound project are attending roughly the same

number of academic and nonacademic sessions. In other words, most of the observed differences

in activity levels among students are attributable to differences among projects in program offerings

and staff expectations. However, after examining relationships between average attendance figures 22

and various project characteristics, we found no consistent evidence suggesting that students in some

types of projects are more active than students in other types of projects.

The measures of intensity examined so far pertain to periods of well-defined and limited

duration--a summer or an academic year. For a final measure of intensity, we will consider students’

entire Upward Bound careers, which can span just part of one summer or academic year or several

summers and academic years. Whereas, earlier, we measured the dose of Upward Bound received

by a student by the number of months the student participated, we will measure the dose here by the

number of activities in which the student participated. We find that the typical student attends

roughly 375 Upward Bound sessions, about 55 percent of which are academic courses and the

remainder are nonacademic activities. Over the course of an average student’s Upward Bound

23
37

career, about 30 percent of the academic sessions attended are in English, 20 percent are in math, and

15 percent are in science. Thus, there is a strong focus on core academic subjects.
39

III. THE IMPACT OF UPWARD BOUND

Increasing the skills and motivation of students to prepare them to succeed in college is an

important goal of the Upward Bound program. This evaluation examines many outcomes related

to this goal, including students’ expectations concerning how far they will go in school, the range

of courses they completed in high school, and their performance in these courses. Furthermore, since

the students in our study sample who entered Upward Bound in 10th or 11th grade had an

opportunity to attend college for one or two years, we can begin to examine the program’s impact

on the college experience.

In this chapter, we examine Upward Bound’s impact on student outcomes, such as educational

expectations, high school courses, grades, misbehavior in school, parent involvement, college

enrollment and selectivity of the college attended, access to financial aid, credits earned in college

courses, and participation in college activities. Specifically, we address the following questions:

C What is the impact of Upward bound on all students who were randomly selected for the

program and what are the effects of participating in Upward Bound on students’

outcomes?

C Do some groups of students benefit from Upward Bound more than others?

Our findings include the following:

C Upward Bound has limited impacts on students during high school. When we looked

across a variety of outcomes, we found that Upward Bound had impacts on only a few.

Roughly two to three years after being selected for Upward Bound, students in the

treatment group expected to complete slightly more years of education and had earned

slightly more credits in math and social studies than students in the control group.

Upward Bound generally had no impacts on students’ in-schools behavior, participation

in extra curricular activities, grade point average, or credits earned in subjects such as

English or science. Furthermore, Upward Bound had no impact on high school

graduation.
Generally, less than five percent of the total variation in student outcomes could be attributed 1

to differences among project impacts.

40

C Upward Bound may have some impact on participants’ postsecondary education

experiences. Although, Upward Bound had no impact on the chances students attended

college, students in the treatment group earned more credits from four-year colleges,

were more likely to receive financial aid, and were more actively engaged in some

college activities. But since relatively few study participants had reached college age

and those who did entered Upward Bound during grades 10-12, it is too soon to make

definitive statements about Upward Bound’s impact on postsecondary outcomes.

C Upward Bound has substantial impacts on some groups of students and not others.

Although Upward Bound had small impacts on students as a whole, some groups of

students received greater benefits. Focusing on the key outcomes of educational

expectations and courses taken in high school, we found that (1) students with lower

initial educational expectations benefitted substantially more than those with higher

expectations, (2) boys showed substantially larger impacts than girls, (3) Hispanic and

white students benefitted more than African American students, (4) students who were

low-income only or low-income and potential first-generation college students showed

larger impacts than those who qualified for the program only as potential first-generation

students, and (5) poorer performing students benefitted substantially more than their

better performing peers.

As part of this evaluation, we also explored whether there was an association between projectlevel characteristics and project-level impacts. This analysis showed that there were only small

differences in impacts among projects. Furthermore, no consistent relationships existed between

1

project characteristics, such as number of students served, composition of the student population,

or average duration in Upward bound by project participants and the size of project impacts.

This chapter describes our method of measuring the impact of Upward Bound on students’

outcomes. Next, we present a context for interpreting the findings. Finally, we describe impact of

the program for all students and some subgroups of students.
41

A. COMPUTING PROGRAM IMPACTS

An important feature of the evaluation was the application of random assignment to construct

two statistically equivalent groups of students: (1) a treatment group and (2) a control group.

Because students were randomly selected for the treatment group and the control group, the only

difference between them should be the offer of Upward Bound services to the treatment group and

those students’ participation in the program. The difference between the two groups on a given

outcome--for example, the average number of credits earned in high school--provides an estimate

of the program’s impact. 

Two sets of program impacts are presented in this chapter. One set of estimates shows the

impact of being offered the opportunity to participate in the Upward Bound program; that is, the

impact of Upward Bound on all students randomly selected to participate in Upward Bound

(assigned to the treatment group). These impact estimates combine the effects of two processes: (1)

students showing up (or not showing up) for Upward Bound services and (2) students actually

participating in the program. By focusing the analyses on the impact of students being given the

opportunity to participate in Upward Bound, we define the treatment more broadly than simply what

the Upward Bound staff accomplishes once students participate in various Upward Bound activities

and courses; it also includes the projects’ efforts in getting students to show up for services and their

exposure to these services. About 18 percent of the students who were eligible for the program did

not participate at all and about 65 percent of all those selected for Upward Bound had stopped

participating before the end of 12th grade. Furthermore, many students who showed up for services

decided to leave the program rather early on--almost 40 percent left within the first 12 months.

The second set of estimates we present show the impact of actually participating in Upward

Bound; these estimates adjust for students who never showed up for services and indicate the impacts
The estimates, however, do not take account of the fact that some students only participate for 2

a few months while other students participate for three or four years. Chapter IV examines the effect

of duration in Upward Bound on students’ outcomes.

Generally, impacts adjusted for no-shows are larger than those we compute for all students;

3

however, because we used different analytic approaches for computing program impacts for all

students and for participants we may find impacts for participants that are smaller than for all

students. When computing the impact for all students we used a procedure referred to as

subclassification analysis and when adjusting for no-shows we used an analytic model to compute

program impacts. A complete discussion of these approaches is described in Appendix 2.A.

42

for those who attended at least one session. For this study, we assumed that (1) the same proportion 2

of students in the control group and the treatment group, when given the opportunity to participate

in Upward Bound, would decide to participate and (2) Upward Bound has no impact on students who

never attended a course or activity. Since we had to make these untestable assumptions when

3

computing impacts for program participants and these estimates only tell part of the story concerning

the program’s overall impact on students, we focus our discussion on the findings pertaining to all

students who were offered the opportunity to participate in Upward Bound; however, we present

both sets of estimates in tables so that readers may compare findings for all students and for

participants. In general, the conclusions drawn from the two sets of estimates are similar. 

B. CONTEXT FOR INTERPRETING PROGRAM IMPACTS

When interpreting the findings concerning program impacts, it is useful to keep two contextual

features in mind: (1) many students in the study were too young to have attended college by the time

we last collected information in 1996, and (2) students in both the treatment and control groups

received other services.
FIGURE III.1

SCHOOL STATUS IN 1996

32

7

24

37

Still in high school

Dropped out of high school

Attended College

High School Graduate 

Not Yet Attended College

0 10 20 30 40 50

Percent

43

1. Many Students in the Study Are Too Young to Have Attended College

Most students who applied for Upward Bound in late 1992 through early 1994 were in 9th or

10th grade when they applied; a few were in 8th grade or 11th grade. This has important

implications when examining students’

college enrollment. When we last collected

information from students, many were still

in high school (see Figure III.1) and findings

concerning college enrollment reflect only

the experiences of about one-quarter of all

students in the study. Moreover, these

findings relate to the experiences of students who entered Upward Bound later in high school and

had comparatively few years of exposure to the program. A more comprehensive picture of the

experiences of all students in the study will have to wait until after late 1998 when we again collect

data from the students.

2. Estimates of Program Impact Show Upward Bound’s Value Added

The experiences of students in the control group show the educational outcomes we would

expect in the absence of the Upward Bound program. These outcomes are the product of many

factors, such as high school attendance, parents’ efforts to motivate students and help them with their

homework, peer interaction, tutoring, counseling, and other pre-college programs. The results from

the most recent data collection show that almost half the students who were in the control group

received some kind of supplemental service between the time they applied and 1996: almost 60

percent of all students reported receiving services during at least one school year and more than 20
About 41 percent of students in the treatment group reported that they too had participated in 4

supplemental service programs other than Upward Bound. Some Upward Bound students may have

participated in these programs after leaving Upward Bound.

Up to 11 percent of the control group, for example, participated in programs referred to as

5

Talent Search/Educational Talent Search. Because students often were not specific when they

reported this information, it is impossible to identify all students who participated in the TRIO

funded program.

A few students selected for the control group were given Upward Bound services by projects 6

that participated in the evaluation. Also, a few students obtained Upward Bound services from

Upward Bound projects that were not in the evaluation. To maintain the integrity of the random

assignment design, we treated all students selected for the control group as controls, regardless of

whether they participated in Upward Bound.

Appendix D shows the extent to which various subgroups of students participated in

7

supplemental services.

44

percent reported participating in a program during at least one summer (see Table III.1). These

4

students received a broad range of services that included tutoring and counseling, and some

participated in more formal programs such as the TRIO Talent Search program. Upward Bound

5

students also reported receiving supplemental services; however, they generally reported

participating less in these services than members of our control group.

Since we compute the impact of Upward Bound by comparing the outcomes of the Upward

Bound group with the control group, the impacts show how much the program leads to changes

beyond the experiences of similar students who were not in Upward Bound. This difference is the 6

value added by Upward Bound over these outside factors. If Upward Bound is only as effective as

the services that students already have available, we would expect the Upward Bound students and

the students in the control group to have similar outcomes.7
45

TABLE III.1

USE OF SUPPLEMENTAL SERVICES OTHER THAN

THOSE PROVIDED BY UPWARD BOUND

Upward Control

Bound Group

Any Supplemental Service

Ever 41% 58%

Summer 15 21

Academic Year 39 54

Program Attended Most

Regular Upward Bound (other than project applied to) 0 1

Upward Bound Math/Science 12 13

Talent Search 7 11

Other pre-college 2 4

Tutoring 28 41

UBP0106D.WK4

C. PROGRAM IMPACTS

This section describes Upward Bound’s impacts on students’ outcomes. These findings apply

to the eligible students who applied for Upward Bound during 1992-94. A broad range of outcomes

were analyzed, such as college attendance, postsecondary credits earned, receipt of financial aid, high

school completion, high school course taking, participation in high school activities, school-related

behavior, and parent involvement.
46

1. High School Graduation, Educational Expectations, and High School Course Taking

a. Upward Bound Had No Impact on High School Graduation Status

When we last collected data, there was a small, but statistically insignificant difference in the

percent of Upward Bound students who had graduated from high school and the percent of students

in the control group who had graduated; 59 percent of the Upward Bound students had graduated

from high school and 63 percent of those in the control group had graduated. While Upward Bound

had no impact on high school graduation, it did appear that relatively more Upward Bound students

were still in high school when we last collected data than students in the control group (35 percent

of the treatment group were in high school and 28 percent of the control group were in high school).

b. Being Selected for Upward Bound Led to Higher Educational Expectations

Each time we collected information from students, we asked them about their educational

expectations--how many years of schooling they think they would complete and how far in school

their parents think they will go. Upward Bound students reported that they expected to complete

about 0.3 more years of education than similar students not selected for the program (see Table III.2).

On average, students in Upward Bound reported that they expected to complete 16.4 years of

schooling. On the other hand, those in the control group said they expected to complete 16.1 years.

Further analysis suggests that this difference in expectations arises in part because more students in

Upward Bound expected to complete post-baccalaureate work than those in the control 
47

TABLE III.2

IMPACT OF UPWARD BOUND ON STUDENTS’ EXPECTATIONS 

AND HIGH SCHOOL COURSE TAKING

All Students Participants

Treatment Control

Mean Mean Impact Impact

Educational Expectations (years of schooling)

 Students’ 16.4 16.1 0.3* 0.4*

 Fathers’ 16.9 16.8 0.0 0.1

 Mothers’ 17.1 17.0 0.1 0.1

High School Creditsa

 Non-remedial English 3.8 3.7 0.1 0.1

 Non-remedial social studies 2.8 2.6 0.1* 0.2

 Non-remedial math 2.8 2.6 0.2* 0.3*

 Non-remedial science 2.6 2.5 0.1 0.1

 Non-remedial foreign language 1.4 1.4 0.0 0.0

 Non-remedial total for 5 major subjects 13.5 12.9 0.6 0.6

 Non-remedial vocational education 1.6 1.7 -0.1 -0.1

 Non-remedial computer science 0.8 0.9 -0.0 0.0

 Non-remedial other 4.1 4.0 0.1 0.1

 Total non-remedial 20.0 19.5 0.6 0.7

 Total AP/honors, all subjects 1.9 2.0 -0.1 -0.0

 Total credits, includes remedial 20.4 19.9 0.6 0.7

Satisfied New Basics Curriculum 30% 29% 1 1 a

Cumulative GPA 2.3 2.3 0.0 0.0

School Status

 Still in High School 35% 28% 7* 8*

 Dropped out 6 9 -3 -4

 Graduated 59 63 -3 -4

b

*Significant at the .10 level using a one-tailed t-test.

High school credits are defined in terms of Carnegie units. A Carnegie unit corresponds to a course that meets

a

for 45-60 minutes, five days a week, for an entire academic year.

Impact expressed as percentage points. b
FIGURE III.2

IMPACT OF UPWARD BOUND ON STUDENTS' 

EDUCATIONAL EXPECTATIONS

High school or less

Some college

Four-year college degree

More than four-year college degree

0 10 20 30 40

Percent

Upward Bound

Control Group

The analyses described by Myers and Schirm (1997), where larger impacts were found, focused 8

on the course taking of students who began participating in Upward Bound as a high school

sophomore or later and examined gains in credits earned between their freshman year and the end

of the 1993/94 school year. This approach was used to adjust for possible differences in the initial

number of credits earned for treatments and controls. A different approach was used here (see

Appendix B) and all students were included in the analyses. We employed this new approach

because many of the outcomes used in this analysis did not have baseline measures. For example,

without limiting the sample, high school credits could not be measured before students entered

Upward Bound. Also, by using this approach we were able to include all students in the analysis of

high school credits and not just those who entered Upward Bound in grade 10 or later. In part, the

larger impacts found in the earlier report may have reflected differences in approaches or because

we only took a snapshot of the students’ experiences after they had only completed a few years of

high school.

48

group (see Figure III.2). We found

no impacts for parents’

expectations concerning years of

schooling, as reported by the

students. We found no impacts for

parents’ expectations concerning

years of schooling, as reported by

the students.

c. Upward Bound Had Small Impacts on Credits Earned in High School

Using data from students’ high school transcripts, we found that among all students who were

selected for Upward Bound, the program had only small impacts on the number of credits students

had earned (see Table III.2). Upward Bound students earned about 0.2 more credits in math than 8

students in the control group and about 0.1 more credits in social studies by the time most of them

had reached their senior year in high school or graduated. These impacts correspond to about an 8

percent increase in math credits and about a 5 percent increase in social studies credits over what
The authors of a Nation At Risk also included one-half credit of computer science. NCES does 9

not include this last element in its definition of the New Basics curriculum and we have followed

the practice in this report. Nationwide, about 50 percent of all 1994 high school graduates had

completed the New Basics curriculum (U.S. Department of Education, 1996) and 44 percent of the

Upward Bound high school graduates had completed this core curriculum. The most common

subject requirement that these students had not completed was earning three credits in science and

20 percent of the students had not completed any of the requirements. (Source: UBSC2201.)

49

students would have earned in the absence of Upward Bound. We found positive, but statistically

insignificant impacts for the other core subject areas--English, science, and foreign language. 

In the past decade, some states have initiated tougher standards concerning the core curriculum

that students must complete to obtain a high school diploma; often this curriculum is referred to as

the New Basics (A Nation at Risk 1983). The requirements of the New Basics Curriculum include

earning four credits in English, three credits in mathematics, three credits in science, and three

credits in social studies. Although many Upward Bound students completed the core curriculum, 9

we found no significant difference between the treatment and control groups: 30 percent of all

Upward Bound students had completed the core curriculum and 29 percent of the control group had

completed it (see Table III.2).

To compute the impact of Upward Bound, we use all students in the treatment and control

groups regardless of whether or not they had completed high school. As a consequence, the

percentage of Upward Bound students and students in the control group who completed the New

Basics Curriculum is lower than would be expected if we just looked at high school graduates. If

we compare the experiences of high school graduates in Upward Bound and the control group, we

find that about 44 percent of the Upward Bound high school graduates and the control group

graduates had completed the curriculum.
In almost all cases we used a one-tailed test of statistical significance to assess the chances of 10

a positive impact being as large or larger than that observed in the sample, by chance alone. For

some analyses where the direction of the expected impact was unclear, we used a two-tailed test.

For example, analysis of impacts of participating in extracurricular activities relied on two-tailed

tests.

50

d. Upward Bound Had No Impact on Extra Curricular Activities or Parent Involvement

We found no impacts for a wide range of the outcomes dealing with participation in

extracurricular and community activities, school misbehavior and parent involvement; and for some

outcomes we found that students in the control group had better outcomes than Upward Bound

students. For example, students who were selected for Upward Bound were just as likely as those 10

in the control group to report that they were late for school, skipped classes, or had been in trouble

at school for not following rules (see Table III.3). Students in Upward Bound were less likely than

students in the control group to participate in high school activities such as student government, the

school newspaper, or yearbook. Finally, Upward Bound students reported that they were less likely

to participate in church or community groups. Limited participation in these activities should not

be particularly surprising given the time commitment that Upward Bound projects often demand of

students. During a typical summer and academic year program, students attend about 250 sessions

concerning formal instruction in academic subjects, tutoring, and counseling.

2. Going to College

The most important goal for Upward Bound is increasing students’ access to college. We used

two approaches to collect data on college access and related outcomes. First, we conducted a survey

of students and asked them about college attendance. Second, we collected students’ transcripts
51

TABLE III.3

IMPACT OF UPWARD BOUND ON STUDENTS’ HIGH SCHOOL EXPERIENCES

All Students Participants

 Treatment Control

Mean Mean Impact Impact

Participated in Activities in High School (Percent Yes):

 Band, orchestra 25% 29% -4 -5%#

 Drama club, school play 16 22 -6# -6 

 Student government 17 23 -6# -7 

 Academic honor society 21 25 -4 -4 

 School newspaper or yearbook 18 25 -7# -9# 

 Service club 20 25 -5# -6 

 Academic club 41 42 -1 0 

 Hobby club 12 10 2 3*#

 Vocational education club 30 37 -7# -8 #

 Team sports 26 26 0 -1 

 Individual sports 17 17 -1 -2 

 Cheerleading, pom pom 11 11 0 0 

 Church activities, youth groups 37 46 -10# -12 

 Community groups 10 15 -5# -6 # 

Misbehavior (Number of times):

 Late for school 4.4 4.3 0.1 0.1

 Skipped classes 2.0 1.8 0.2 0.2

 Missed a day of school 5.0 4.9 0.1 0.1

 In trouble for not following school rules 0.9 0.9 0.0 0.0

 Put on in-school suspension 0.3 0.4 -0.1 -0.1 

 Suspended 0.2 0.2 -0.0 -0.1 

 Transferred to another school 0.0 0.0 -0.0 -0.0 

 Arrested 0.1 0.1 -0.0 -0.0 

 Spent time in juvenile home 0.0 0.0 -0.0 -0.0 

 Tried and convicted as an adult 0.0 0.0 0.0 0.0

Discuss with Parent Sometimes/Often (Percent Yes):

 About selecting courses 67% 73% -6%# -8%

 About school activities 72 76 -4 # -5

 About studies 65 75 -10 # -12 

 About grades 84 87 -3 # -3

 About transferring to another school 14 15 -1 -2

 About taking, preparing for ACT/SAT 67 68 -1 -2

 About going to college 89 92 -3 -4
TABLE III.3 (continued)

All Students Participants

 Treatment Control

Mean Mean Impact Impact

52

Parent Sometimes/Often (Percent Yes):

 Checked on homework 64% 63% 1% 1% 

 Helped with homework 48 48 0 -0 

 Gave special privileges for good grades 61 61 1 1

 Limited privileges because of poor grades 44 42 1 2

 Required chores around the house 84 87 -3 -3 

 Limited time watching TV, playing video games 37 34 3 4*

 Limited time with friends 59 57 2 2

Did parent... (Percent Yes):

 Attend a school meeting? 47% 57% -10%# -12%#

 Speak with teachers? 69 71 -2 -3 

 Visit classes? 37 43 -7 # -7 # 

 Attend a school event? 53 62 -8 -11 

*Significant at the .10 level using a one-tailed t-test.

#Significant at the .10 level using a two-tailed test.
Selectivity of the college was based on Peterson’s 1998 Four-Year college Selectivity Index. 11

Scores ranged from 1 (non-competitive) to 5 (most difficult). Scores were based on the percentage

of 1996 applicants who were accepted, and on the high school rank and standardized test scores of

the accepted freshman. Two-year colleges and vocational schools were given a score of zero. 

Postsecondary credits are defined in terms of semester hours. 12

53

from all postsecondary institutions they told us about in the student survey. Using both types of data

we compiled a picture of students’ college attendance, credits earned in college, type of school

attended (four-year, two-year, or vocational), selectivity of the college attended, engagement in

college life, and receipt of financial aid.

While we must be cautious not to place too much emphasis on the findings concerning college

outcomes because our data only reflect the experiences of students who entered Upward Bound later

in high school, the results show that in the short-term, Upward Bound students were no more likely

than control group students to have ever attended college (either a two- or four-year college), and

the four-year colleges they attended were no more selective than those attended by the control

group. Upward Bound students, however, earned more postsecondary credits than students in the 11

control group, particularly in four-year colleges. Students in Upward Bound, were also more likely 12

to receive financial aid and were often more involved in college activities.

a. Students Selected for Upward Bound Earn More Postsecondary Credits

Although students in Upward Bound and the control group were equally likely to have ever

attended a postsecondary school, Upward Bound students earned more non-remedial postsecondary

credits (see Table III.4). Students who were selected for Upward Bound earned about
54

TABLE III.4

IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT 

AND RELATED OUTCOMES

All Students Participants

Treatment Control

Mean Mean Impact Impact

School Status Indicator

 Attend college 23% 25% -3 -2 

 Attend four-year college 15 16 -1 0

 Attend two-year college 5 8 -3 -4 

 Attend vocational school 2 1 1 1

a

Postsecondary Creditsb

 Any Type School

 Total non-remedial 6.8 5.7 1.1* 1.7*

 Total remedial 0.2 0.4 -0.3* -0.4*

 Four-Year College

 Total non-remedial 5.7 4.4 1.3* 2.0

 Total remedial 0.1 0.1 -0.0 -0.0

 Two-Year College

 Total non-remedial 1.0 1.2 -0.2 -0.3

 Total remedial 0.0 0.2 -0.2* -0.2*

 Vocational/Technical School

 Total non-remedial 0.2 0.1 0.0 0.0

 Total remedial NA NA NA NA

 College Selectivity 0.8 0.9 -0.12 -0.03 

*Significant at the .10 level using a one-tailed t-test.

NA = unable to estimate.

Impacts on outcomes concerning school status expressed as percentage points. a

Postsecondary credits are defined in terms of semester hours. b
The values for the average credits earned while attending a postsecondary institution are lower 13

than one might expect because they refer to all students selected for Upward Bound and the control

group, and not just those who attended college. We cannot make a direct comparison of college

goers in the treatment group and the control group to compute the program’s impact on those who

attended college because the two groups will be no longer statistically equivalent and biased

estimates of project impacts will be computed. To provide some perspective, however, we note that

among those who attended a four-year college, Upward Bound students earned 34 credits and

students in the control group earned 27 credits.

Almost all students in the study who attended college received some type of financial aid. This 14

should not be surprising given that more than 80 percent of the Upward Bound applicants were from

low-income families.

55

seven credits and students in the control group earned about six credits. When we compared

13

students’ experiences earning credits in remedial courses, we found that Upward Bound students

earned slightly fewer remedial credits than students in the control group (0.3 credits less).

Separating the findings for each of these school types showed that Upward Bound led to an

increase of 1.3 credits in the number of non-remedial credits earned in four-year colleges and to

reduction of 0.2 credits earned in remedial courses while attending two-year colleges. This suggests

that participation in Upward Bound helps to better prepare students for college course work. Upward

Bound did not have a substantial impact on credits earned in vocational schools.

b. Upward Bound Students are More Likely to Receive Financial Aid

We found that the Upward Bound group had a somewhat greater chance of receiving financial

aid than the control group: 33 percent of those in Upward Bound report receiving aid, compared to

30 percent of the control group (see Table III.5). The most common form of financial aid for both 14

groups was a Pell grant (24 percent of the treatment group and 21 percent of the control group

received a Pell grant). The next most common form of aid was a loan, and Upward Bound students

were more likely to receive loans as well. Compared to students in the control group, students in
56

TABLE III.5

IMPACT OF UPWARD BOUND ON STUDENTS’ RECEIPT OF FINANCIAL 

AID AND SOURCES OF INFORMATION CONCERNING FINANCIAL AID

All Students Participants

Treatment Control

Mean Mean Impact Impact

Received any type of financial aid 33% 30% 3*# 4*# 

 Received loans 22 20 3* 3* 

 Received tuition waiver 8 7 1 1 

 Received Pell Grant 24 21 3*# 3 

 Received another type of grant or fellowship 19 16 3*# 3*# 

 Received a work/study appointment 11 7 4*# 4*# 

a

Did to learn about financial aid

 Talk with teacher/counselor 78% 81% -2 -3 

 Talk with college representative 61 62 -1 -2 

 Talk with loan officer at a bank 3 6 -3 # -3 # 

 Read Dept. of Education information 40 42 -2 -2 

 Read information from a college 73 67 5*# 6*# 

 Read about aid available through military 35 43 -8 # -10 

 Talk to a knowledgeable adult 83 80 3* 3 

 Talk with friends 75 76 -1 -3 

*Significant at the .10 level using a one-tailed t-test.

#Significant at the .10 level using a two-tailed t-test.

Impacts expressed as percentage points. a

Upward Bound were also more likely to obtain financial aid from a grant/fellowship, or a work study

appointment. When we asked students how they had learned about the various sources of financial

aid, we learned that Upward Bound students and those in the control group often obtained their

information from different sources. Upward Bound increased the likelihood of students receiving

financial aid information from colleges and reduced their reliance on information provided by the

military.
57

c. Upward Bound Students are More Actively Engaged in College Activities

Many educators believe that retention in college is associated with students’ successful

integration into the college experience. When we asked students about the number of times they

talked with faculty, met with an academic advisor, had informal contacts with faculty, or participated

in study groups or school clubs we found that Upward Bound students were more likely to be

engaged in such activities (see Table III.6). For example, Upward Bound led to a 30 percent increase

in informal contacts with an advisor or faculty. Furthermore, Upward Bound students were 20

percent more likely to have talked with faculty in their offices and participation in study groups or

school clubs with other students.

D. IMPACTS FOR SUBGROUPS OF STUDENTS

In this section we focus on specific groups of students to assess who benefits most from Upward

Bound. The groups we examined were students with lower/higher educational expectations; boys

and girls; African American, Hispanic, and white youth; students who were from low-income and

first generation college families, low-income only, and first generation only families; students whom

we identified as having strong/weak academic preparation as high school freshmen; and students

who were 9th and 10th grade Upward Bound applicants. Because we make a large number of

comparisons in the subgroup analysis, we limited the outcomes to include students’ educational

expectations, high school course taking, high school enrollment status, college attendance, credits

earned in postsecondary schools, and college selectivity.

Our exploration of subgroup impacts shows that Upward Bound had large impacts on some

groups of students, particularly students with lower educational expectations, Hispanic and white

students, boys, students from low-income/first generation and low-income only families, and 
58

TABLE III.6

IMPACT OF UPWARD BOUND ON COLLEGE ACTIVITIES

All Students Participants

Treatment Control

Mean Mean Impact Impact

College activities: (Number of times in the last

term attended college)

 Talk with faculty in their offices 0.6 0.5 0.1* 0.1*

 Meet with advisor about academic plans 0.6 0.5 0.1 0.1*

 Have informal contacts with advisor or faculty 0.6 0.4 0.2* 0.2*

 Participate in study groups with other students 0.7 0.6 0.1* 0.2*

 Go places with friends from school 0.9 0.8 0.1 0.1*

 Participate in student assistance centers 0.4 0.3 0.1 0.1

 Participate in school clubs 0.3 0.2 0.1* 0.1*

 Attend career-related lectures with friends 0.3 0.2 0.0 0.0

 Participate in sports, music, drama 0.2 0.2 0.0 0.0

 Cut classes 0.2 0.2 0.1 0.1

*Significant at the .10 level using a one-tailed t-test.
The core subjects refer to English, math, science, social studies, and foreign language. 15

We defined students as having higher expectations if they expected to complete at least a

16

bachelor’s degree; students who expected to complete less than a four-year college degree were

defined as having lower educational expectations.

59

students who had earned fewer credits in core subjects in 9th grade and had a lower grade point

average (more academically at-risk students).15

1. Upward Bound Has Large Impacts on Students with Lower Initial Educational

Expectations

Before they applied for Upward Bound, almost 80 percent of the students in the study expected

to complete at least a four-year college degree. However, the students who appeared to benefit

16

most from Upward Bound were the 20 percent with lower educational expectations (see Table III.7),

particularly when we consider their high school course taking behavior and high school completion.

Upward Bound had a large impact on high school course taking for students who had lower

expectations. When we looked across all high school subjects, we found that Upward Bound led to

an increase of almost three credits for these students; this was a 17 percent increase over what similar

students would achieve without Upward Bound. We found no impact on total credits earned for

students with higher expectations. The impact estimates also show that Upward Bound increased

the chances of low expectation students graduating from high school--65 percent of the students in

Upward Bound had graduated from high school and 52 percent of the control group had graduated.

Upward Bound had no impact on high school graduation or the chances of dropping out of school

for students with higher expectations.

The program’s impact on credits earned in English, math, science, social studies, and foreign

language for lower expectation students ranged from a 19 percent increase in science to a 26 percent

increase in social studies. Only for math credits did Upward Bound have a significant impact on
60

TABLE III.7

IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS AND HIGH SCHOOL COURSE

TAKING FOR STUDENTS WITH HIGHER AND LOWER INITIAL EDUCATIONAL EXPECTATIONS

All Students Participants

Higher Expectations Lower Expectations

Higher

Expectations

Lower

Expectations

Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact

Educational Expectations (years of schooling) Students’ Fathers’ Mothers’

16.517.117.2

 0.3*

0.00.1

15.015.916.6

 0.5* 

0.2 0.0 

 0.3*

0.00.1

0.6*

.3 

-0.0 

High School Credits Non-remedial English Non-remedial social studies Non-remedial math Non-remedial science Non-remedial foreign language Non-remedial total for 5 major subjects Non-remedial vocational education Non-remedial computer science Non-remedial other Total Non-remedial Total AP/Honors, all subjects Total Credits, includes remedial

3.82.72.72.71.5

13.5

1.7

.9

4.0

20.1

2.1

20.4

0.00.1

 0.3*

0.00.00.4

-0.2-0.0

 0.1*

0.2

-0.2

0.2

3.32.22.21.91.1

10.6

1.7

.7

3.9

16.8

.9

17.5

.7*.6*

0.4*

.4*.3*

2.3*

0.1 

0.3*

0.2 

2.9*1.6*2.9*

******

0.00.1

 0.3*

0.00.00.5

-0.2

0.1

 0.1*

0.3

-0.3

0.2

0.9*0.7*0.5*0.5*0.3*2.9*

0.2 

0.3*

0.3 

3.7*1.9*3.7*

Satisfied New Basics Curriculum 35% 0a 11% 2* * 0 2* 
61

TABLE III.7 (continued)

All Students Participants

Higher Expectations Lower Expectations

Higher

Expectations

Lower

Expectations

Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact

Cumulative GPA 2.4 -0.0 2.0 0.1 0.0 .2High School Status Still in high school Dropped out Graduated

 29% 6 65

 8* -2 -6

 2% 23 52

-3

 -10* 13*

**

10*

-2-7

-3

-12*

15

Show-Up Rate 83% 81% 

UBDGT06, 1407*Significant at the .10 level using a one-tailed t-test for impacts and an F-test for differences in impacts between subgroups.

Impact on New Basics Curriculum expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting th e b

hypothesis of no difference in impacts by chance was .10 or less. 
In part, the lack of statistical significance concerning differences in impacts may reflect the

17

small sample of students with lower expectations--the number of treatments and controls with lower

expectations was 296 and 203, respectively. The number of students with higher initial expectations

in the treatment and control groups was 1,097 and 1,002, respectively.

62

course taking for students with higher expectations. Looking at course taking from the perspective

of whether students were completing the New Basics Curriculum, we found that Upward Bound led

to an increase in the percentage of lower expectations students who completed this curriculum.

Although the impacts for students with lower and higher expectations were not always significantly

different, the findings suggest that there may be substantial differences between the two groups.17

Consistent with these findings, we found that Upward Bound had large impacts on college

attendance for students with lower initial expectations (Table III.8). Upward Bound led to a 6-point

increase for students with lower expectations and a 12-point increase for attending four-year

colleges. Furthermore, Upward Bound had a large impact on earning credits in four-year colleges:

students with lower expectations who were selected for Upward Bound earned almost ten credits and

those in the control group earned about three credits. Also, students selected for Upward Bound, on

average, earned fewer remedial credits attending two-year colleges and they attended more selective

colleges than students in the control group. We found no impact on the college-related outcomes

for students with higher initial expectations.

2. Upward Bound Has Substantial Impacts on Boys

Fewer than one-third of the Upward Bound applicants were boys; however, the findings

presented here show that boys often benefitted more from Upward Bound than did girls (see Table

III.9). Boys in Upward Bound expected to complete almost .8 years more schooling than boys in the

control group; for girls we observed a statistically insignificant impact of about .2 years. Upward

Bound’s impact on high school course taking for boys was almost two credits--about 11 percent--and
63

TABLE III.8

IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND CREDITS EARNED IN 

COLLEGE FOR STUDENTS WITH HIGHER AND LOWER EDUCATIONAL EXPECTATIONS

All Students Participation

Higher 

Expectations

Lower 

Expectations

Higher

Expectations

Lower

Expectations

Control

Mean Impact

Control

Mean Impact Interactionb Impact Impact

School Status

Attend collegeAttend four-year collegeAttend two-year collegeAttend vocational school

 27%

18

 8 1

-3a

-1-3

 1

 20% 8 10 2

6*

10*

-4

 1

 -4 -1 -4 1

 8* 12* -6  1 

Credit Earned

Four-Year College

Total non-remedial creditsTotal remedial credits

4.80.2

-0.1-0.1

2.00.0

10.3*

0.0

-0.1-0.1

13.2*

0.0

Two-Year College

Total non-remedial creditsTotal remedial credits

1.30.3

-0.2-0.2

1.30.1

-.5

-1.5

-0.3-0.3*

-0.8-0.2*

College Selectivity 0.4 -0.1 0.2 0.1* -0.1 0.1*

Impacts on outcomes concerning school status are expressed as percentage points. a

Thistest of no interaction shows whet her we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting b

the hypothesis of no difference in impacts by chance was .10 or less. 
64

TABLE III.9

IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS AND 

HIGH SCHOOL COURSE TAKING FOR BOYS AND GIRLS

All Students Participants

Girls Boys Girls Boys

 Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact

Educational Expectations (years of schooling) Students’ Fathers’ Mothers’

16.317.017.2

0.2

-0.0-0.1

15.416.116.4

0.8*0.40.5*

0.2

-0.0-0.1

0.9*0.5*0.6*

High School Credits Non-remedial English Non-remedial social studies Non-remedial math Non-remedial science Non-remedial foreign language Non-remedial total for 5 major subjects Non-remedial vocational education Non-remedial computer science Non-remedial other Total Non-remedial Total AP/honors, all subjects Total Credits, includes remedial

4.02.82.82.71.6

13.7

1.80.94.1

20.5

2.2

20.8

-0.0

0.00.1*

-0.0-0.0

0.1

-0.0-0.0-0.0

0.0

-0.1

0.1

3.22.32.12.21.0

10.8

1.50.73.7

16.7

1.2

17.4

0.4*0.4*0.5*0.3*0.2*1.8*

-0.2

0.10.4*2.1*0.31.9*

**********

-0.0

0.10.2*

-0.0-0.0

0.1

-0.1-0.0-0.0

0.20.00.1

0.5*0.5*0.6*0.4*0.2*2.2*

-0.3

0.10.5*2.5*0.4*2.3*

Satisfied New Basics Curriculum 34% -1a 18% 5* -1 7* Cumulative GPA 2.45 -0.06 2.06 0.11* -0.07 0.12*
65

TABLE III.9 (continued)

All Students Participants

Girls Boys Girls Boys

Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact

High School Status Still in high school Dropped out Graduated

 25%

7

68

 7*

-2-5

 35% 15 50

 5  -7*  1 

 8*

-2 -6 

6

 -8*

2

Show-Up Rate 83% 84%

UBSF1206, 1405*Significant at the .10 level using a one-tailed test for impacts and a two-tailed test for interaction.

Impact on New Basics Curriculum expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting b

the hypothesis of no difference in impacts by chance was .10 or less. 
66

for girls, the impact was essentially zero. When we examined each of the subject areas and the

chances of completing the New Basics curriculum, we found a similar pattern--large impacts for

boys and at most, small impacts for girls. Upward Bound also reduced the risk of dropping out of

high school for boys, but not girls.

Upward Bound also led to larger impacts on college going for boys than girls (Table III.10).

Upward Bound increased the percent of boys going to a four-year college by 4 points and increased

the number of credits earned in four-year college by almost 4 credits. Furthermore, being selected

for Upward Bound led to boys attending more selective colleges. For girls, we found that Upward

Bound had no impact on ever attending college, on earning non-remedial postsecondary credits, or

attending a more selective college. However, girls selected for Upward Bound took fewer remedial

courses when attending two-year colleges.

3. Hispanic and White Youth Benefit More from Upward Bound than African American

Youth

The three largest racial/ethnic groups participating in Upward Bound were African Americans

(50 percent), Hispanics (22 percent), and whites (21 percent). When we compared the impacts for

the three groups of students, we found some differences in their educational expectations, high

school course taking, and college outcomes (see Table III.11). Upward Bound had a significant

impact on the expectations of all three groups of students: about .4 years for Hispanic and white

students and about .3 years for African Americans. Upward Bound’s impact on total credits earned

in high school and by subject shows that often, white and Hispanic students experienced large

impacts. For total credits earned while in high school, Upward Bound led to an increase of almost

two credits for Hispanic and white students, which corresponds to about a 10 percent increase. With

the important exception of earning credits in Advanced Placement/honors courses, we found no
67

TABLE III.10

IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND CREDITS 

EARNED IN COLLEGE FOR BOYS AND GIRLS

All Students Participation

Girls Boys Girls Boys

Control

Mean Impact

Control

Mean Impact Interactionb Impact Impact

School Status

Attend CollegeAttend four-year collegeAttend two-year collegeAttend vocational school

 29%

1810

1

 -4a -1 -4 1

 16%

10

33

 4 4*

1

-1

 -5 

-1-5

2

 4* 5* 1  -1 

Credits Earned

Four-Year College

Total non-remedialTotal remedial

5.10.2

.9

-0.1*

2.40.1

3.8*0.0

1.1

-0.1

4.6*0.0

Two-Year College

Total non-remedialTotal remedial

1.30.3

-0.3-0.2

1.10.1

-0.1

0.0

-0.4-0.3

-0.2-0.0

College Selectivity 0.4 -0.1 0.2 0.1* 0.1 0.1*

Impacts on outcomes concerning school status expressed as percentage points. a

Thistest of no interaction shows whet her we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting b

the hypothesis of no difference in impacts by chance was .10 or less. 
68

TABLE III.11

IMPACT OF UPWARD BOUND ON EDUCATIONAL EXPECTATIONS AND HIGH SCHOOL 

COURSE TAKING FOR AFRICAN AMERICAN, WHITE, AND HISPANIC STUDENTS

All Students Participants

African American White Hispanic

African

American White Hispanic

 Control

Mean Impact

 Control

Mean Impact

 Control

Mean Impact

Interactionb Impact Impact Impact

Educational Expectations (years of schooling) Students’ Fathers’ Mothers’

16.417.017.2

0.3*0.2*0.3*

15.516.416.7

0.4*

-0.0

0.1

15.916.516.8

0.4*

-0.0-0.4

* 0.4*

0.3*0.3*

0.5*

-0.0

0.1

0.4*

-0.0-0.4

High School Credits Non-remedial English Non-remedial social studies Non-remedial math Non-remedial science Non-remedial foreign language Non-remedial total for 5 major subjects Non-remedial vocational education Non-remedial computer science Non-remedial other Total Non-remedial Total AP/honors, all subjects Total Credits, includes remedial

3.82.72.72.71.5

13.3

1.70.93.9

19.8

2.1

20.2

-0.1

0.00.1

-0.1-0.1-0.1-0.3-0.1

0.1

-0.5

0.3*

-0.5

3.42.52.32.31.2

11.7

1.80.74.1

18.3

1.0

18.7

0.4*0.3*0.3*0.3*0.11.4*

-0.1

0.1*0.3*1.8*0.11.9*

3.82.62.52.21.6

12.7

1.60.94.2

19.3

2.6

19.9

0.20.20.5*0.2*0.11.30.3*0.2*0.01.9*

-1.0

1.8*

***

-0.1

0.00.1

-0.2-0.1-0.2-0.4-0.1

0.1

-0.6

0.4*

-0.6

0.5*0.4*0.4*0.4*0.11.8*

-0.1

0.2*0.4*2.2*0.22.3*

0.20.20.6*0.2*0.21.40.3*0.3*0.12.1*

-1.1

2.0*

Satisfied New Basics Curriculum 39% -8a 22% 7* 14% 13* * -10% 9%* 15%*Cumulative GPA 2.2 -0.1 2.5 0.1 2.4 -0.0 -0.1 0.2 -0.1
69

TABLE III.11 (continued)

All Students Participants

African American White Hispanic

African

American White Hispanic

 Control

Mean Impact

 Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact Impact

High School Status Still in high school Dropped out Graduated

 35%

4

62

 3*

 0

-2

 20%

2258

2

 -9*

7

 25%

1066

21*

-5*

-16 

**

 4*

-1-3

2

-11*

9

 22*

-5

-17

Show-Up Rate 79% 83% 91%

UBSF1206, 1408*Significant at the .10 level using a one-tailed t-test for impacts and an F-test for differences in impacts across subgroups.

Impacts expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the hypothesis of no difference b

in impacts by chance was .10 or less. 
70

positive impacts for African American students. On average, African American students earned 16

percent more credits in Advanced Placement/honors courses because of Upward Bound.

For two of the five core subjects--math and science--we found large differences in impacts

among the three racial/ethnic groups. The impacts for Hispanic youth ranged from a 9 percent

increase in science credits to a 22 percent increase in math credits; for white students there was about

a 13 percent increase in math and science credits. Given these large impacts, it is not surprising to

find that Upward Bound had large impacts on the percent of Hispanic and white youth who

completed the New Basics curriculum. For Hispanics, Upward Bound led to more than a 90 percent

increase in the percentage completing the curriculum and for whites we found more than a 30 percent

increase. Similar findings pertain to students’ high school status; that is, Upward Bound reduced

the chances of white and Hispanic students dropping out of high school and had no impact on

African American students.

Upward Bound had a large impact on non-remedial credits earned in four-year colleges for

Hispanic students, but no impacts for whites or African American students (Table III.12). For white

students, however Upward Bound led to earning fewer credits in remedial courses. Similarly,

African-American youth who were selected for Upward Bound earned fewer credits in remedial

courses while attending two-year colleges. We also found that Upward Bound led white students

to attend more selective colleges, but Upward Bound had no impact on college selectivity for

Hispanics or African Americans.

4. Giving Students the Opportunity to Participate in Upward Bound Had Substantial

Impacts on Low-income/First Generation Students

According to the federal Upward Bound regulations, at least two-thirds of all program

participants must be from low-income families and be potential first generation college students; the
71

TABLE III.12

IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND CREDITS EARNED 

IN COLLEGE FOR African American, WHITE, AND HISPANIC STUDENTS

All Students Participants

African American

White Hispanics

African

American White Hispanic

Control

Mean Impact

Control

Mean Impact

Control

Mean Impact Interactionb Impact Impact Impact

School Status

Attend collegeAttend four-year collegeAttend two-year collegeAttend vocational school

 27%

18 7 1

-8a

-6-4

 2

 18%

11

 6 2

 5*

410

 28%

1513

 1

 5

 9*

-5

 0

*

-11

 -7  -6 

 2 

 6*

510

 6

 9*

-4

 1

Credits Earned

Four-Year College

Total non-remedialTotal remedial

4.70.2

-0.2-0.1

4.70.1

-1.3-0.1*

3.70.1

8.3*0.0

* -0.3

-0.1

-1.6-0.2*

9.1*

-0.0

Two-Year College

Total non-remedialTotal remedial

0.90.2

-0.6-0.2*

1.70.1

0.5

-0.1

1.50.5

-0.4-0.4

-0.8-0.2*

0.6

-0.1

-0.4-0.4

College Selectivity 0.4 -0.1 0.2 0.1* 0.3 0.0 -0.2 0.1* 0.0

Impacts expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the hypothesis of no difference b

in impacts by chance was .10 or less.
72

remaining participants must either be from a low-income family or be a potential first generation

college student. More than 80 percent of the students participating in the study met both criterion,

and the remaining students were mostly from first generation only households. Comparison of the

impacts for the three groups of students (low-income/first generation, first generation only, and lowincome only) showed that most of the impacts were concentrated in the low-income/first generation

and low-income only groups (see Table III.13). 

For students who were from low-income/first generation households, Upward Bound had

substantial impacts on educational expectations and the courses they took in high school. The

program’s impact on students’ expectations was more than one year for students from low-income

only households and nearly one-third of a year for students from low-income/first generation

households. Among students from first generation only households, we found no significant impact

on students’ expectations, but substantial impacts on their parents’ expectations--as reported by

students.

Being selected for Upward Bound led to large gains in high school credits for students from both

low-income/first generation households and low-income only households. The program’s impacts

on total credits earned for the low-income/first generation and low-income only students were 1 and

2 credits, respectively. This means that being selected for Upward Bound increased low-income only

students’ high school course taking by 12 percent; it increased the course taking of students from

low-income/first generation households by 5 percent. Upward Bound also reduced the chances of

dropping out of high school for low-income/first generation students and low-income only students,

and had no impact on first generation only students.

Upward Bound had a large impact on earning credits in Advanced Placement/honors courses

for students from low-income only households. Giving such students the opportunity to participate
73

TABLE III.13

IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT AND RELATED OUTCOMES

FOR Low-income AND POTENTIAL FIRST GENERATION COLLEGE STUDENTS

All Students Participants

Low-income/

First Generation

(LIFG)

First Generation 

Only (FG)

Low-income Only 

(LI) LIFG FG LI

 Control

Mean Impact

Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact Impact

Educational Expectations (years of schooling) Students’ Fathers’ Mothers’

16.116.817.0

0.3*

-0.0-0.1

16.116.616.7

0.20.6*0.6*

16.317.417.5

1.2*

-0.1

0.0

**

0.4*

-0.0-0.1

0.20.8*0.8*

1.3*

-0.1

0.0

High School Credits Non-remedial English Non-remedial social studies Non-remedial math Non-remedial science Non-remedial foreign language Non-remedial total for 5 major subjects Non-remedial vocational education Non-remedial computer science Non-remedial other Total Non-remedial Total AP/honors, all subjects Total Credits, includes remedial

3.72.62.52.51.4

12.7

1.70.84.0

19.3

1.8

19.7

0.10.2*0.3*0.2*

-0.0

0.8*

-0.0

0.00.10.9*

-0.1

0.9*

3.92.92.92.91.7

14.3

1.81.03.9

20.9

2.4

21.3

0.0

-0.3-0.2-0.5-0.0-0.9-0.5-0.2

0.0

-1.5-0.2-1.4

3.42.42.22.31.1

11.3

1.20.53.9

17.0

0.7

17.3

0.00.30.5*0.30.4*1.6*

-0.1

0.3*0.4*2.2*1.5*2.1*

**

0.10.3*0.4*0.2*

-0.0

0.9*

-0.0

0.00.11.1*

-0.1

1.1*

0.0

-0.3-0.2-0.6-0.1-1.2-0.6-0.2

0.1

-1.9-0.3-1.9

0.00.30.6*0.30.5*1.6*

-0.1

0.3*0.5*2.3*1.6*2.2*

Satisfied New Basics Curriculum 27% 5*a 40% -21 23% 8 6* -28 8Cumulative GPA 2.3 0.0 2.5 -0.1 2.1 0.5* * 0.0 -0.1 0.5*
74

TABLE III.13 (continued)

All Students Participants

Low-income/

First Generation

(LIFG)

First Generation 

Only (FG)

Low-income Only 

(LI) LIFG FG LI

 Control

Mean Impact

 Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact Impact

High School Status Still in high school Dropped out Graduated

 29%

1061

5*

-3*-2 

 22%

 6

72

 15*

 0

-15

 34%

1749

-4 

-17* 

21*

***

 6*

 -4*

-3

 19*

-00-19

-4

-18

 22*

Show-Up Rate 84%

76%

 97% 

UBSF1206, 1410.*Significant at the .10 level using a one-tailed t-test for impacts and an F-test for differences in impacts across subgroups.

Impacts expressed as percentage points. a

This test o f no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the hypothesis of no difference b

in impacts by chance was .10 or less. 
Students with high and low academic performance refer to those who were at the 90th and the 18

10th percentile, respectively, of the distribution of credits earned in the five core subjects and overall

high school grade point average. These estimates are based on students’ experiences in 9th grade.

Some students entered Upward Bound as rising 9th graders and the program may have had a small

impact on their scores; however, we believe that while these values may slightly overestimate

students’ course taking and grade point average before entering Upward Bound, they provide a good

indicator of the kinds of students who apply for the program.

75

in Upward Bound led to their earning three times more credits in these courses (.7 to 2.2). The

impact of Upward Bound on the percentage of students completing the New Basics curriculum is

confined to the low-income/first generation group. For this group, being selected for Upward Bound

led to almost a 5 percent increase in the chances of completing the curriculum.

With only one exception, we found that for college attendance, credits earned, and college

selectivity, Upward Bound had no impacts for any of these groups of students (Table III.14). The

exception to this was that, Upward Bound reduced the number of remedial courses low-income/first

generation students took and completed while attending a two-year college.

5. Upward Bound Has Large Impacts on Students Who Have Lower Academic Performance

as High School Freshmen

Although one of the eligibility criteria for selecting students for Upward Bound is that they have

a need for academic support, the program serves students with a wide range of academic skills. For

example, students who applied for Upward Bound and were near the top in academic performance

had earned five credits as a high school freshman in the core subjects and had a B+ grade average;

students at the lower end of the academic skills distribution had earned about two credits in the core

subjects and had D average. Students who most benefit from Upward Bound, however, are often 18

those with greater need for academic support.

To compare impacts for students who had lower/higher academic performance as high school

freshmen, we constructed an index of academic risk based on (1) credits earned in the five core
76

TABLE III.14

IMPACT OF UPWARD BOUND ON COLLEGE ENROLLMENT AND RELATED OUTCOMES 

FOR LOW-INCOME AND POTENTIAL FIRST GENERATION COLLEGE STUDENTS

All Students Participants

Low-income/

First Generation

(LIFG)

First Generation

Only

(FG)

Low-income Only 

(LI) LIFG FG LI

Control

Mean Impact

Control

Mean Impact

Control

Mean Impact

Interactionb Impact Impact Impact

School Status

Attend CollegeAttend four-year collegeAttend two-year collegeAttend vocational school

 26% 16 8 2

 -3a 0 -3 1

 25%

16

81

 -9%

-2-3

0

 22%

14

71

 -7%

4

-2

0

-2% 

 1 

-4 

1 

 -6 -2 -4 0

 2

4

-2 

0

Credits Earned

Four-Year College

Total non-remedialTotal remedial

4.40.2

2.50.0

4.50.1

-1.2

0.0

3.10.3

0.0

-0.1

2.9

-0.1

-1.5

0.0

0.0

-0.1

Two-Year College

Total non-remedialTotal remedial

1.00.2

-0.2-0.2*

2.10.3

-0.7

0.2

1.00.0

-0.3

0.0

-0.2-0.2

-0.8-0.2

-0.4-0.0

College Selectivity 0.3 -0.4 0.4 -0.2 0.3 0.1 0.0 0.0 0.1 

Impacts on school status expressed as percentage points. a

This test of no intera ction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the hypothesis of no difference in b

impacts by chance was .10 or less.
The index was formed on the basis of a factor analysis and the predicted first principal

19

component extracted from the factor analysis. After we computed the principal component score

(index) for each student, we separated students into two groups--students who had a score above the

typical student (median) and those who had a score below the typical student. Students with a higher

score were labeled as less at risk and those with a lower score were labeled as most at risk of school

failure.

77

subjects and (2) students’ grades in high school. Based on the index, we ranked students as being 19

at low risk or high risk for school failure. The average student we considered most at-risk had

earned three credits in the five core subjects and had a C average. Students who were less at risk had

earned almost five credits and had a B- average. 

Although Upward Bound had similar impacts on students’ educational expectations for the

high/low risk groups (about .3 years), the results in Table III.15 show that substantial impacts were

present for students’ course taking. For the most at-risk group, we found that being selected for

Upward Bound led to increases in all the core subjects except science. On average, we found an

impact of about 1.3 credits across the five subjects--more than a 13 percent increase. We also found

that Upward Bound had large impacts on earning credits in Advanced Placement/honors classes and

on completing the New Basics curriculum for students who were not most at-risk; we observed no

impacts for other students. Among the outcomes showing students’ high school completion status,

we found that Upward Bound reduced the chances of dropping out for both groups, but much more

so for those who were most at-risk: Upward Bound participation reduced the dropout rate by 6

points for this group.

Upward Bound appears to have only a few impacts on college outcomes, despite students’ atrisk status (Table III.16). Upward Bound students in the lower half of the academic skills

distribution were more likely to attend four-year colleges and earned about two more non-remedial 
78

TABLE III.15

IMPACT OF UPWARD BOUND ON STUDENTS’ EDUCATIONAL EXPECTATIONS AND 

HIGH SCHOOL COURSE TAKING, BY STUDENTS’ AT-RISK STATUS

All Students Participants

Lower Risk Higher Risk Lower Risk Higher Risk

Control

Mean Impact

 Control

Mean Impact

Interactionb Impact Impact

Educational Expectations (years of schooling) Students’ Fathers’ Mothers’

16.617.217.3

0.3*0.00.0

15.616.416.7

0.3*0.10.1

0.4*0.00.0

0.4*0.20.1

High School Credits Non-remedial English Non-remedial social studies Non-remedial math Non-remedial science Non-remedial foreign language Non-remedial total for 5 major subjects Non-remedial vocational education Non-remedial computer science Non-remedial other Total Non-remedial Total AP/honors, all subjects Total credits, includes remedial

4.53.23.43.21.9

16.1

1.91.04.4

23.3

3.4

23.4

-0.1

0.0

-0.0

0.1

-0.1-0.2-0.3-0.0

0.1

-0.5-0.4-0.5

3.12.11.82.01.09.91.60.73.8

15.9

0.4

16.6

0.3*0.3*0.5*0.10.2*1.3*0.10.10.2*1.6*0.4*1.6*

*******

-0.1

0.0

-0.0

0.1*

-0.1-0.2-0.3-0.0

0.1

-0.5-0.6-0.5

0.4*0.3*0.7*0.10.2*1.6*0.10.10.22.0*0.6*2.0*

Satisfied New Basics Curriculum 49% 0a 11% 2* 0 3*Cumulative GPA 2.7 -0.1 2.0 0.0 -0.1 0.1
79

TABLE III.15 (continued)

All Students Participants

Lower Risk Higher Risk Lower Risk Higher Risk

Control

Mean Impact

 Control

Mean Impact

Interactionb Impact Impact

High School Status Still in high school Dropped out Graduated

 24%

2

74

 13*

-1*

-12 

 31%

1753

0 

-6*

7*

**

 15*

 -1

-14

0 

-8*

 8*

Show-Up Rate 87% 80% 

PC0102, PC0107*Significant at the .10 level using a one-tailed test for impacts and an F-test for differences in impacts across subgroups.

Impact expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the b

hypothesis of no difference in impacts by chance was .10 or less.
80

TABLE III.16

IMPACT OF UPWARD BOUND ON COLLEGE ATTENDANCE AND CREDITS 

EARNED IN COLLEGE BY STUDENTS’ AT-RISK STATUS

All Students Participants

Lower Risk Higher Risk Lower Risk Higher Risk

Control

Mean Impact

Control

Mean Impact Interactionb Impact Impact

School Status

Attend CollegeAttend four-year collegeAttend two-year collegeAttend vocational school

 34%

2410

 1

 -6 

-3 -4 

0

 17%

862

3 

4*

-2 

1 

 -7 

-4-4

 1

 4 5*

-3

1

Credits Earned Four-Year College

Total non-remedialTotal remedial

6.90.2

1.4

-0.1

1.90.1

2.4*0.0

1.5

-0.1

3.0*0.0

 Two-Year College

Total non-remedialTotal remedial

1.70.4

-0.7-0.3*

0.60.1

0.30.0

-0.8-0.4*

0.40.0

College Selectivity 0.5 -0.1 0.2 0.0 -0.1 0.0

PC0113R, PC0116

Impact on school status outcomes expressed as percentage points. a

This test of no interaction shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability b

of rejecting the hypothesis of no difference in impacts by chance was .10 or less. 
We excluded from the analysis all 9th graders who reported that they were in the 8th grade

20

during the 1991/92 school year. When we compared the percentage of high school graduates in the

9th grade and 10th grade cohorts, we found that even with this exclusion restriction, the percentage

of 9th graders who had graduated was lower than for 10th graders--77 percent of the 9th graders had

graduated by 1996 and almost 90 percent of the 10th graders had graduated. This suggests that the

9th grade cohort used in this phase of the analysis was somewhat younger than the 10th grade cohort.

81

college credits than similar students in the control group. For students with stronger academic

performance as a freshman in high school, we found that being selected for Upward Bound led to

them completing fewer non-remedial courses when attending two-year colleges, but the program had

no impact on the other college outcomes.

6. The Impact of Upward Bound Is Similar for 9th- and 10th-Grade Applicants

Students from a wide range of grades may apply for Upward Bound; however, there is little or

no information concerning when it is most beneficial for students to enter the program. To assess

the impact of starting the program at different points during high school, we compared the

experiences of the two largest groups of applicants--9th and 10th graders. We did not include

students who were still in 8th grade or who were in 11th grade when they applied in this phase of

the analysis because there were relatively few of them in the study. The outcomes we used when

computing impacts for these groups were focused on high school course taking and students’

educational expectations. We did not use any of the college related outcomes because the 9th grade

applicants were too young to have attended college when we last collected data.20

The findings concerning Upward Bound’s impact on educational expectations and credits earned

in high school were inconsistent. Among the younger applicants, we found that being selected for

Upward Bound led to substantial impacts on students’ educational expectations and on the

expectations of their mothers and fathers, as reported by students. For the older applicants we found

that Upward Bound had no impact on educational expectations (see Table III.17).
82

TABLE III.17

IMPACT OF UPWARD BOUND ON HIGH SCHOOL OUTCOMES FOR 9TH AND 10TH GRADE APPLICANTS

All Students Participants

9th Grade Applicants

10th Grade

 Applicants

9th Grade

Applicants

10th Grade

Applicants

Control

Mean Impact

Control

Mean Impact Interactionb Impact Impact

Educational Expectations 

Students 16.0 .4* 16.2 0.2 .4* .2Fathers 16.8 .3* 16.9 -0.3 * .3* -.3Mothers 16.9 .3* 17.1 -0.3 * .4* -.4

High School Credits

Non remedial English 3.9 0.3 4.0 0.1 0.3 0.1Non remedial social studies 2.7 0.1 2.8 0.2* 0.1 0.2*Non remedial math 2.4 0.6* 2.8 0.0 * 0.7* 0.0Non remedial science 2.7 -0.1 2.6 0.2* -0.1 0.3Non remedial foreign language 1.3 0.1 1.6 0.1 0.1 0.0Non remedial total for 5 major subjects 12.9 1.0 13.9 0.5 1.1 0.6Non remedial vocational education 1.7 -0.2 1.8 0.3 -0.3 0.3Non remedial computer science 0.8 0.0 0.9 0.0 0.0 0.0Non remedial other 4.1 0.0 4.2 0.0 0.2 0.0Total Non remedial 19.4 0.8 20.8 0.8* 0.9 0.9Total AP/honors, all subjects 2.1 0.2 2.2 0.2 0.2 0.1*Total Credits, includes remedial 20.0 0.8 21.3 0.7* .9 0.9*

Satisfied New Basics Curriculum 34% -5a 39% 9* -0.1 0.1*Cumulative GPA 2.2 -0.1 2.5 0.0 -0.1 -0.1Show-up Rate 86% 83%

UBSF1235, UBSF1411

Impacts expressed as percentage points. a

This test of no inter action shows whether we found a substantial difference in the impacts for the subgroups. A * indicates that the probability of rejecting the hypothesis of no difference b

in impacts by chance was .10 or less. 
83

Comparing the program’s impact on non remedial credits earned, we found that Upward Bound

had some significant impacts for the 10th-grade applicants and almost always statistically

insignificant impacts for 9th- grade applicants. Although the impacts for 9th- grade applicants were

not statistically significant, they were similar to those computed for the older students, suggesting

that Upward Bound may have about the same impact, regardless of whether students enter the

program as 9th graders or 10th graders. For example, the impact on total non-remedial credits

earned for both cohorts was about .8 credits--an increase of about one course more than they would

have completed in the absence of the Upward Bound.

The findings concerning credits earned by the two groups suggests that, at least for this set of

outcomes, little may be gained by serving students before they are in the 10th grade. Before one

concludes, however, that projects should focus their resources on older students, it would be prudent

to wait for results from the next round of data collection that will show the impact of Upward Bound

on the program’s primary outcome--increasing college access.
85

IV. THE IMPACT OF DURATION IN UPWARD BOUND 

AND PROGRAM COMPLETION EDUCATIONAL OUTCOMES

One strength of Upward Bound is that it is a multi-year program. Most applicants enter the

program in the 9th or 10th grade, giving the students an opportunity to be exposed to the program

for up to four years. Although this relatively long exposure to Upward Bound gives the program an

opportunity to make a substantial impact on participants, the typical Upward Bound participant

remains in the program for only about 19 months; the most common Upward Bound experience

includes attending the program for one summer and parts of two academic years. Coupled with these

low levels of participation, we found that, overall, Upward Bound had a modest impact or no impact

on outcomes such as educational expectations, high school course taking, and college going and

credits earned in college.

To assess whether this limited participation in Upward Bound may account for the few impacts

we uncovered in the evaluation when looking at all students, we undertook analyses that compared

the outcomes for participants who were exposed to Upward Bound for different lengths of time and

for those who completed the program and those who did not complete the program. These

comparisons show that longer exposure to the program was often associated with substantial

impacts, including the following:

C Participants who remained in Upward bound for more than two years generally earned

more high school credits, particularly in the five-core subjects, and had higher

educational expectations.

C For older students, greater exposure to Upward Bound was associated with higher

college enrollment rates.
The statistical approach we used required that we assume that all relevant factors that influence 1

both participants length of exposure to Upward Bound and the student outcomes are included in the

statistical model. Since it is likely that some factors were not included in the analysis, the findings

about the relationship between duration and outcomes are not as robust as the findings derived from

the random assignment experiment reported in Chapter III.

86

Furthermore, we found that remaining in Upward Bound until the end of students’ senior year in high

school had no independent impact on their outcomes.

A. ESTIMATING THE EFFECTS OF DURATION ON PARTICIPANTS’ OUTCOMES

To compute the effects of duration on student outcomes, we used an approach that was based

on (1) the statistical matching of Upward Bound participants with similar students in the control

group and (2) the estimation of an analytic model. A simple comparison of outcomes for

1

participants with different levels of exposure to Upward Bound is not sufficient because students

with specific characteristics, such as greater motivation or more parental encouragement to persist

in school, may remain in Upward Bound for different lengths of time and distort the durationstudent outcomes relationship. To compute the effects of duration on the outcomes, we needed to

isolate the effects of duration from these other factors.

To implement our approach, we first sorted participants into three groups based on how long

they remained in Upward Bound. The three groups we used were: (1) participated for 1 to 12

months, (2) participated for 13 to 24 months, and (3) participated for more than 24 months. Next,

we formed a one-to-one match between participants in the three duration groups with similar

students in the control group. The one-to-one match between an Upward Bound participant and a

similar student from the control group provided two outcomes that we compared: (1) an outcome

showing what the Upward Bound participant accomplished and (2) what the participant would have

accomplished without Upward Bound (that is, the outcome for the student selected from the control

group). Comparing the two outcomes within the three duration groups showed the average effect
For students’ high school outcomes we present results for both cohorts; however, for their

2

college outcomes we only present findings for the 10th grade cohort because 9th graders were too

young to have attended college when we last collected data.

We do not look at results for 11th graders because there are so few 11th grade applicants in the 3

study. 

87

of Upward Bound on participants who remained for one year (1-12 months), two years (13-24

months), and more than two years (more than 24 months). The outcomes analyzed in this chapter

were the same we used when comparing impacts for subgroups as discussed in Chapter III.

Since participants who remained in Upward Bound for a longer period had somewhat different

characteristics than those who remained only a few months, such as being older when they applied

for Upward Bound and having parents who were more involved in school related activities (see

Table C.1), we statistically adjusted the effects of duration so that each duration group had similar

background characteristics. This means that the effects we show in this chapter were adjusted so that

differences on selected background characteristics for participants did not distort the relationship

between duration and student outcomes. Some background characteristics we adjusted for were

grade at application, sex, race/ethnicity, educational expectations, parent involvement in school

activities, participation in high school activities, misbehavior in school, and employment. We give

a more detailed description of the matching and estimation process in Appendix C.

Separate results are presented for students who entered Upward Bound in 9th grade and 10th

grade; these are the two most common grades that students are in when they apply for the program. 2 3

We kept these groups distinct because the maximum length of participation in the program varied

with the grade in which students entered Upward Bound. For example, a student who entered in the
FIGURE IV.1

DURATION IN UPWARD BOUND

9th 10th

Grade At Application

0

10

20

30

40

50

60

Percent of Participants

Duration in Months 1-12 13-24 24+

88

fall of grade 9 could have participated in

an Upward Bound program for four years.

A 10th grade applicant, however, could

have participated for a maximum of three

years. The distribution of participants by

length of time in Upward Bound and by

grade at application is shown in Figure

IV.1. 

B. EFFECTS OF DURATION IN UPWARD BOUND ON STUDENT OUTCOMES

1. Participants Who Remained in Upward Bound for Longer Periods Had Higher

Educational Expectations and Earned More Credits in High School

Among 9th and 10th graders, we found that Upward Bound often had substantial impacts on

participants who remained in the program for more than two years (see Table IV.1). And among

those who entered the program as 10th graders, Upward Bound led to substantial impacts for

participants who were in the program for more than a year.

a. Ninth Graders

With only one exception, the findings concerning 9th graders suggest that short-term

participation (less than two years) had no impact on participants’ educational expectations. In other

words, students who remained in Upward Bound for less than two years had similar educational

expectations to students who did not have an opportunity to participate in the program. However,

for those participants who stayed in Upward Bound for more than two years, the program led to an

increase in educational expectations of about 0.6 years. Similarly, Upward Bound increased parents’
89

TABLE IV.1

EFFECTS OF DURATION ON PARTICIPANTS’ HIGH SCHOOL OUTCOMES

Duration

1-12 13-24 24+ 

Months Months Months

9th Grade Cohort 

Educational Expectations (years of schooling)

Students’

Fathers’

Mothers’

-0.3 -0.8 0.6*

0.3 -0.1 0.9*

0.6* -0.1 0.8*

Credits Earned in High School

English -0.2 -0.3 0.7*

Social Studies -0.2 -0.1 0.3

Math 0.5* 0.0 1.2*

Science 0.1 -0.4 0.2

Foreign Language -0.1 0.1 0.2

Total five core subjects 0.1 -0.7 2.6*

Total non-remedial 0.4 -2.0 2.3*

AP/Honors 0.9* 0.6 0.9*

Total non-remedial and remedial 0.5 -1.9 2.4*

Satisfying New Basics 5 15 17

Grade Point Average 0.1 -0.3 -0.1

a

 

10th Grade Cohort 

Educational Expectations (years of schooling)

Students’

Fathers’

Mothers’

0.5 0.6 0.4

0.5 0.2 -0.1

0.5 0.0 -0.3

Credits Earned in High School

English 0.2 0.5* 0.2

Social Studies 0.3 0.4* 0.6*

Math -0.9 0.0 0.6*

Science 0.2 0.0 0.6*

Foreign language -0.1 0.2 0.4*

Total five core subjects 0.6 1.2* 2.3*

Total non-remedial 0.5 1.4* 3.3*

AP/Honors 1.4 1.1* 1.7*

Total non-remedial and remedial 0.8 1.2 3.2*

Satisfying New Basics Curriculum 18* 20* 20*

Grade Point Average -0.2 -0.2 0.1

UBSF0742, 47, 50

*Adjusted effect statistically significant at the .10 level using a one-tailed test.

Effects on Satisfying the New Basics Curriculum are expressed as percentage points. a
90

educational expectations for their children, as reported by their students, by about 0.8 years (about

0.8 years for mothers and almost 0.9 years for fathers).

Upward Bound also led to an increase in the number of credits participants earned in high school.

For the five-core subjects, 9th grade participants with more than two years of exposure to the

program earned almost 2.6 more credits than those who did not participate. This amounts to

participants completing an average of two to three more courses in subjects such as English, math,

science, social studies, or foreign language because of their participation in Upward Bound. Looking

beyond the five-core subjects, we also found that long-term Upward Bound participants earned more

credits in Advanced Placement and honors courses (0.9 credits) and, overall, completed more

remedial and non-remedial high school credits (2.4 credits).

b. Tenth Graders

Among 10th graders who participated for more than two years, we found results concerning

course taking that were often similar to those of the 9th graders. Moreover, for this cohort of

applicants we found that Upward Bound participants with more than one year of exposure to the

program also benefitted; that is, students who participated for 13 to 24 months and the group who

participated for more than 24 months each realized substantial gains. The findings also suggest that

each additional year that students participate may have led to larger effects. For example, 10th

graders who participated for 1 to 12 months earned about 0.6 more credits in the five-core subjects,

those who participated for 13 to 24 months earned about one more credit (1.2), and those who 
The effect estimated for participating in Upward Bound for 13-24 months for total credits

4

earned is not statistically significant at conventional levels of significance; however, the pattern is

consistent with other results and suggests that 10th graders with a moderate amount of exposure to

Upward Bound may also benefit from the program.

91

participated for more than 24 months earned more than two additional credits (2.3) because of their

participation.4

2. Longer Exposure to Upward Bound Led to Higher Postsecondary Attendance Rates,

Attendance at More Selective Schools, and Earning More Postsecondary Credits 

Our results show that 10th-grade applicants who stayed in the program for 13 to 24 months were

more likely to attend college, particularly a four-year college, than if they had not participated in

Upward Bound (see Table IV.2). Participating in Upward Bound for 13 to 24 months led to a 14

percentage point increase in attendance at four-year colleges. Furthermore, these participants

attended more selective colleges than their counterparts and earned five more non-remedial credits

from four-year colleges. Among those who stayed for 1 to 12 months we found that they earned

fewer remedial credits from four-year colleges.

Why did we find that participating for 13-24 months and not more than 24 months resulted in

larger impacts on college outcomes? In part, this finding comes about because for many of the 10th

graders who entered Upward Bound late in their sophomore year, the maximum exposure to the

program was two years. Moreover, many of the 10th graders who were in the program more than

two years were too young to have attended college when we last collected data. Until we collect data

again in 1989/99, it will be unclear whether participants with greater exposure to Upward Bound

have better college outcomes than students with less exposure to the program.
92

TABLE IV.2

EFFECTS OF DURATION ON PARTICIPANTS’ COLLEGE 

OUTCOMES: 10th-GRADE COHORT

Duration

1-12 Months 13-24 Months 24+ Months

School Status:

Attend college -23 15* 1

Attend four-year school -15 14* 7

Attend two-year school -3 3 -1

Attend vocational school -7 -1 -6

 

Credits Earned in College

Four-year college

Non-remedial credits -2.9 5.5* -0.7

Remedial credits -0.8* 0.1 0.0

Two-year college

Non-remedial credits -0.1 1.2 0.0

Remedial credits 0.1 -0.8 0.0

College Selectivity -0.2 0.4* 0.2

*Adjusted effect statistically significant at the .10 level using a one-tailed test.

C. COMPUTING THE EFFECTS OF COMPLETING UPWARD BOUND

To compute the effects of completing the Upward Bound program on students’ outcomes, we

used a procedure similar to that used to estimate the effects of duration. To implement our approach,

we sorted Upward Bound participants into two groups: (1) program completers--students still in

Upward Bound at the end of their senior year of high school and (2) noncompleters. Next, we

formed a one-to-one match between completers and similar students in the control group and
93

noncompleters and similar students in the control group. The one-to-one match between participants

(completers and noncompleters) and similar students in the control group provided two outcomes

that we compared: (1) an outcome showing what the participant accomplished and (2) what the

participant would have accomplished without Upward Bound (that is, the outcome for the student

selected from the control group). Comparing the outcomes within the two groups of participants

show the average effect of Upward Bound on participants who stayed with Upward Bound and

students who participated, but did not stay until the end of their senior year of high school.

D. EFFECT OF PROGRAM COMPLETION ON STUDENT OUTCOMES

Analyses of program completion were restricted to students who were in 10th grade when they

applied for Upward Bound. We focused on this group for two reasons. First, there were too few 9th

graders who had completed Upward Bound when we last collected data. Second, the number of

older students was too small to conduct meaningful analyses.

The results shown in Table IV.3 suggest that except for a few cases, “completing” Upward

Bound has no effect on student outcomes; that is, completers had outcomes that were similar to those

of noncompleters. The most noteworthy exception concerns students’ educational expectations:

Upward Bound completers expect to complete about 0.7 years more schooling that noncompleters.

Are these results inconsistent with the results concerning duration? Probably not. Instead, it shows

that Upward Bound needs time to develop the requisite skills in participants and there does not

appear to be a qualitative impact that results from completing the program. 
94

TABLE IV.3

EFFECTS OF COMPLETING UPWARD BOUND FOR 10TH GRADE APPLICANTS

Outcome Noncompleters on Completers Impact

Effect of Effect of 

Upward Bound on Upward Bound 

Students’ expectations -.26 .43 .69*

Fathers’ expectations -1.10 -.26 .84

Mothers’ expectations -.68 .07 .75

Attend college .27 .04 -.24

Attend two-year college -.04 .01 .05

Attend four-year college .21 .11 -.10

Attend vocational school .09 -.08 -.18

High school credits

English .64 .59 -.05

Social science .32 .45 .12

Math .25 .38 .13

Science .06 .14 .08

Vocational education -.22 .46 .68*

Foreign language .17 .15 -.02

Computer science .01 -.05 -.06

Other .27 1.10 .83*

Five core subjects 1.49 1.70 .27

AP/honors 2.33 .55 -1.78

All 1.57 2.39 .81

New basics .16 .12 -.04

Credits earned in college

Four-year

Remedial .13 .31 .19

Nonremedial 13.09 3.82 -9.17

Two-year

Remedial -.19 -.11 .08

Nonremedial .86 .97 .11

Selectivity .57 .08 -.49

*Indicates that the probability of rejecting the hypothesis of no effect by chance was less than or equal to .10.
95

V. SUMMARY AND CONCLUSIONS

Upward Bound serves each year about 44,000 high school students who are economically

disadvantaged and/or are potential first-generation college students; the intent of the program is to

increase participants’ skills and motivation so that they are more likely to succeed in college. The

first phase of the national evaluation of Upward Bound has drawn on data provided by Upward

Bound project directors and staff, target school liaisons, parents, and students. Analysis of these data

show that Upward Bound projects provide an intensive academic program during the summer and

a more modest program during the school year. Services provided by Upward Bound projects

include formal instruction in high school subjects such as English, math, science, social studies, and

foreign language; tutoring; and counseling. Most applicants are rising 9th graders, high school

freshmen and sophomores; and most of them have high educational expectations. Moore (1997)

describes these findings in detail.

A. SUMMARY OF FINDINGS

Four major findings stand out in this report (see Table V.1). First, while Upward Bound gives

students the opportunity to participate in the program for up to three or four years, nearly 40 percent

of the program participants who were in our study left the program during the first year and we

estimate only about 40 percent of those who began the program will be in Upward Bound at the end

of 12th grade. Because many participants leave the program during the first year and others depart

from Upward Bound as high school juniors and seniors, the typical student gets only about 19

months of exposure to Upward Bound services.
When we refer to Upward Bound students this includes all students who were randomly selected for Upward Bound; this includes both those students who participated in the

a

program and those who did not participate, even when offered the opportunity to attend.

96

TABLE V.1

SUMMARY OF FINDINGS AFTER SECOND FOLLOW-UP OF STUDENTS

Population Population Duration) High School Outcomes College Outcomes

Share Bound

of the (Median

Persistence in 

Upward

All Students 100% 19 months • Upward Bound students earned more credits in • Students in Upward Bound were just as likely to

high school math and social science courses attend a two- or four-year college as students in

than members of the control group.

a

• Upward Bound students had higher educational

expectations than students in the control group.

the control group.

• Upward Bound students earned more nonremedial credits in four-year colleges and fewer

remedial credits in two-year colleges.

• Students in Upward Bound were more actively

engaged in college activities such as talking with

faculty, having informal contacts with an advisor

or faculty, and more often participating in

student study groups than their counterparts in

the control group.

Initial Educational 

Expectations:

Higher (expected to 79% 21 Months • Upward Bound students with higher educational • No impact.

complete a Bachelor’s expectations expect to complete more years of

degree or higher) schooling than students in the control group and

Lower (expected to 21% 15 months • Upward Bound students who had lower initial • Upward Bound led students with lower expectations

complete less than a expectations earned more high school credits than to attend four-year colleges at a higher rate.

Bachelor’s degree) similar students in the control group; they earned

were less likely to drop out of high school.

more credits in the five core subjects, particularly • Students in Upward Bound and who had lower

English and social studies. expectations earned more credits in four-year

• Upward Bound had a large impact on the their counterparts in the control group.

educational expectations of students who started

with lower initial expectations.

colleges and attended more selective schools than
TABLE V.1 (continued)

Population Population Duration) High School Outcomes College Outcomes

Share Bound

of the (Median

Persistence in 

Upward

97

African American 50% 20 months • Upward Bound had an impact on the educational • African American students in Upward Bound earned

White 21% 20 months • Upward Bound led white students to have higher • White students in Upward Bound earned fewer

Hispanic 23% 15 months • Hispanic students in Upward Bound had higher • Hispanic students in Upward Bound were more

expectations of African American students and on fewer remedial credits from two-year colleges than

credits earned in Advanced Placement/honors similar students in the control group.

courses.

educational expectations than similar students in remedial credits from two-year colleges and were

the control group. Furthermore, the program had more likely to attend a postsecondary institution than

substantial impacts on credits earned in the five white students in the control group.

core subjects and total credits earned in high

school, and reduced the chances of dropping out of

high school.

educational expectations, were less likely to drop likely to attend a four-year college and earned more

out of high school, earned more credits in math and credits from four-year colleges than students in the

science courses, and earned more overall credits control group.

than similar students in the control group.

Sex:

Boys 29% 17 months • Upward Bound had an impact on boys’ educational • Boys in Upward Bound were more likely to attend

Girls 71% 20 months • Girls in Upward Bound earned more credits in • Girls in Upward Bound had fewer credits in remedial

expectations, the chances of dropping out of four-year colleges, earned more credits from fourschool, and on credits earned in the five core year colleges, and attended more selective colleges

subjects and overall. than boys in the control group.

• Boys in Upward Bound earned higher grades than

similar students in the control group.

science courses than girls in the control group.. courses when attending four-year colleges than girls

in the control group.
TABLE V.1 (continued)

Population Population Duration) High School Outcomes College Outcomes

Share Bound

of the (Median

Persistence in 

Upward

98

Low-Income/Potential

First Generation College

Status:

Low-Income and First 78% 19 months • Upward Bound students who were from low- • Low-income and first generation students who were

Generation income households and were potential first selected for Upward Bound earned fewer remedial

• No impact.

 First Generation Only 18% 15 months • Among first generation only students, Upward

 Low-Income Only 4% 22 months • Low-income only students in Upward Bound had

generation college students had higher educational college credits, particularly from two-year colleges,

expectations, were less likely to drop out of high than they would have without the program.

school, earned more credits in the five core

subjects, and earned more overall credits than

similar students in the control group.

Bound led to parents having higher educational

expectations.

higher educational expectations, were less likely to

drop out of high school, and earned more high

school credits in the five core subjects and overall

than did low-income only students in the control

group.

• No impact.
TABLE V.1 (continued)

Population Population Duration) High School Outcomes College Outcomes

Share Bound

of the (Median

Persistence in 

Upward

99

At-Risk Status:

Higher Risk (students 50% 20 months • Among higher risk students, Upward Bound had • Upward Bound led to a substantial increase in

who earned fewer substantial impacts on students’ educational attendance at four-year colleges and to students

credits and had a lower expectations, credits earned in the five core earning more postsecondary credits; however, we

grade point average as a subjects, the chances of dropping out of high did not find a statistically significant impact on

high school freshman school, credits earned in Advanced earning credits from four-year colleges.

than the typical student Placement/Honors courses, and total credits earned

who applied for Upward in high school.

Bound)

Lower Risk (students 50% 23 months • Upward Bound had an impact on the educational • Lower risk students in Upward bound earned fewer

who earned more expectations of lower risk students and on their total credits in remedial postsecondary course work

credits and had a higher chances of dropping out of high school. than similar students in the control group.

grade point average as a

high school freshman

than the typical Upward

Bound applicant)

Grade at Application:

9th Grade 46% 23 months • Upward Bound had an impact on the educational • Not computed because applicants who were in the

expectations of students who applied for Upward 9th grade were too young to have attended college.

Bound while in 9th grade. The program had no

impact on credits earned in high school.

10th Grade 31% 19 months • Upward Bound students who applied while in 10th

grade earned more high school credits than their

counterparts in the control group. The program had

no impact on their educational expectations.
TABLE V.1 (continued)

Population Population Duration) High School Outcomes College Outcomes

Share Bound

of the (Median

Persistence in 

Upward

100

Duration in Upward

Bound:

1 to 12 Months 30% NA • No effect. • Participants with 1 to 12 months of exposure to

13 to 24 Months 18% NA • Among 10th grade applicants, Upward Bound led • Participants were more likely to attend college and

More than 24 Months 51% • Participants with more than 24 months of exposure • Upward Bound had no impact on the college

(9th grade Upward Bound had outcomes similar to those they

applicants) would have achieved without the program.

34%

(10th grade

applicants)

(9th grade to an increase in educational expectations and earn credits from four-year colleges than similar

applicants) credits earned in the five core subjects; there was students in the control group.

41% NA

(10th grade

applicants)

(9th grade to Upward Bound earned more high school credits outcomes of students who stayed in Upward Bound

applicants) and had higher educational expectations than their for more than two years.

25%

(10th grade

applicants)

no similar effect for 9th graders.

counterparts.
101

Second, when we look across all students served by Upward Bound, we find that the program

has small impacts on students’ high school course taking and educational expectations. Since many

students we selected for Upward Bound were too young to have attended college when we last

collected data, it is too early to make definitive statements concerning the program’s impacts on

college attendance and related outcomes. Furthermore, our findings concerning college access and

retention reflect the experiences of students who entered Upward Bound later in high school and do

not represent the experiences of those who entered, for example, as 9th graders. So far, our findings

suggest that Upward Bound did not have an impact on the percentage of students who attended a

two- or four-year college, nor on the selectivity of the college attended. It did, however, lead

students to earn more credits from four-year colleges and to become more actively engaged in some

college activities, such as talking with college faculty and working in study groups. 

Third, Upward Bound had substantial impacts for some groups of students, such as boys, white

and Hispanic youth, and students who were most at risk of academic failure--those with lower

educational expectations and those who had earned fewer credits in the five core subjects as high

school freshmen. Upward Bound students completed more credits in the five-core subjects than their

counterparts in the control group, and they were more likely to attend college and earn more credits

from four-year colleges. The findings also show that the impacts on completing high school courses

were similar for the 9th- and 10th-grade applicants; this suggests that at least for this outcome, there

may be no benefit to starting the program in 9th grade instead of 10th grade.

Fourth, Upward Bound participants who remained in the program for more than two years often

benefitted more from participation than those with less exposure to the program. Students who

remained in Upward Bound for shorter periods, particularly 9th-grade applicants, often experienced

smaller impacts or no impacts.
102

B. IMPLICATIONS OF FINDINGS FOR PROGRAM IMPROVEMENT

Overall, Upward Bound’s impacts on students’ high school outcomes were small. Furthermore,

it is too early to tell if the program leads to increased college attendance and completion. We found,

however, that the program can have large impacts for some groups of students and for those who

remained for longer periods. These findings have broad implications for program improvement and

suggest the Upward Bound program and its projects may consider two issues for increasing the

program’s impact on student outcomes.

1. Retaining Upward Bound Participants

The national evaluation showed that longer participation was associated with larger impacts, but

only about one-third of the participants persisted for more than two years in Upward Bound. The

most common reason that participants listed for leaving Upward Bound was to take a job. We

suggest here, as we did in Myers and Schirm (1997), that one way to hold Upward Bound

participants in the program is through the provision of employment opportunities that complement

the design and curriculum of the Upward Bound programs.

Fitting employment opportunities into the Upward Bound schedule, however, may pose many

challenges for project staff since Upward Bound projects generally meet once per week during the

school year and for five to seven weeks each summer in a residential setting. The summer is

particularly challenging because students participate in formal courses or other scheduled activities

for much of the day. As a consequence, there is little time available for students to work without

reducing the time spent on scheduled activities. Incorporating employment opportunities into the

program may be unappealing because it appears to dilute the program’s academic emphasis.

However, if employment opportunities keep students in the program for an additional year or more,
We computed the expected increase in impacts by assuming that Upward Bound’s impact on 1

individual students would remain constant even when there is a shift in the composition of students

participating in Upward Bound. Carrying out such a plan would require working with youth who

are even more at-risk of school failure than current applicants and maintaining the same basic

program of instruction and services already being offered.

103

then this short-term reduction in academic services should be more than made up in subsequent

years.

2. Serving More Students Who are At Greater Risk of Educational Failure

The second issue concerns the concentration of impacts among students who were at greater risk

of academic failure; these students were defined as having lower educational expectations, or as

students who, as high school freshmen, earned fewer credits in the core subjects and had lower grade

point averages than the typical Upward Bound applicant. We found that these students often

benefitted more from Upward Bound than did other students. These findings suggest the program

may achieve more substantial impacts by changing how students are recruited and selected for

Upward Bound. For example, doubling the percentage of students with lower educational

expectations in the program (20 percent to 40 percent) could increase the average impact of Upward

Bound on credits earned in the five-core subjects by almost 50 percent.1

Possible steps for identifying these students include the following:

C Recruiting and selecting more students who have earned few credits in core subjects

such as English, math, science, social studies, and foreign language

C Recruiting and selecting more students with D and C grade point averages

C Working with high school guidance counselors to identify students who have the

potential to complete college, but have lower educational expectations such as expecting

to attend only a two-year college or expecting to attend a four-year college, but not

expecting to complete a bachelor’s degree
104

Project staff already recruit some students with these characteristics; however, they do not recruit

them in large numbers. For example, among the projects who participated in the impact study, fewer

than 10 percent of the projects had 40 percent or more of their applicants report they had lower

educational expectations.

Some project directors have expressed concern about adjusting the mix, claiming that part of

the treatment effect we observed for students with lower expectations was related to the presence of

students in the project with higher educational expectations. Although our statistical analyses have

shown no relationship between project impacts and the percentage of participants in a project with

lower expectations, the effects of changing the mix should be monitored carefully. One approach

for testing this shift in recruitment and selection policy is to set up demonstration projects that enroll

different concentrations of students with lower expectations or poorer academic records, but provide

the same services, and to compare the impacts under the alternative conditions.

C. ISSUES TO BE CONSIDERED IN PHASE II OF THE NATIONAL EVALUATION OF

UPWARD BOUND

Phase I of the national evaluation followed a group of students who entered Upward Bound in

the 1992/93 and 1993/94 school years. Most of these students had completed high school or just

entered college when we last collected data. The U.S. Department of Education (ED) has awarded

a contract to MPR to follow this group of students for five more years--through 2002--to assess the

program’s impact on a series of college related outcomes, such as access, retention, and completion;

selectivity of the colleges attended; college major; and employment related outcomes. Reports

describing program impacts will be available in 1999 and 2001. Besides continuing the evaluation

along the same lines as Phase I, ED has funded two studies linked to the ongoing evaluation. The

first is an assessment of the practices used by nine Upward Bound projects that have potentially
105

large impacts on key outcomes such as educational expectations and credits in the five core high

school subjects. MPR staff and its subcontractors will visit these projects twice: once during the

school year and another time during the summer program in 1998. The final report describing the

findings from the case study visits will be available in early 1999. The second new study is an

evaluation of the effects of Math/Science Centers (MSCs), funded under the Upward Bound

Math/Science Initiative, on students’ outcomes, such as college major. This evaluation will compare

the experiences of a sample of MSC participants with those of similar students who are already part

of the national evaluation (that is, participants in regular Upward Bound and students who were

selected for the control group). We will include the sample of MSC participants in all future data

collection efforts undertaken for the national evaluation. Reports describing the effects of MSC

participation on student outcomes will be available in late 1999 and 2001.
APPENDIX A

SAMPLE DESIGN, WEIGHTING,

AND ERROR ESTIMATION
Some projects funded in the 1989-92 grant cycle were defunded in the 1992-95 grant cycle and 1

therefore eliminated from the universe. Projects newly funded in the 1992-95 and later grant cycles

were also excluded from the universe.

A-3

In this appendix, we describe how the impact study sample was selected, how the baseline and

second follow-up data were weighted, and how standard errors of sample estimates were calculated

to reflect the sample design.

A. SAMPLE DESIGN

For the impact study, we selected a nationally representative sample of eligible Upward Bound

applicants in two stages. First, we selected a nationally representative sample of Upward Bound

projects to serve as “primary sampling units” (PSUs). Second, we selected eligible applicants to those

projects and randomly assigned students to treatment and control groups.

1. First Stage Sampling: Selection of Projects

The “universe” of projects for the impact study--the collection of projects whose students are

targeted for study and eligible to be selected for the study sample--consists of active regular projects

that are (1) located in the 50 states and the District of Columbia; (2) hosted by postsecondary

educational institutions; (3) mature, having operated for at least three years by October 1992; and (4)

not serving only students with physical disabilities. Veterans projects and math/science projects are

not considered regular projects. During the period when the impact study sample of students was

being selected (roughly May 1992 through March 1994), there were 395 Upward Bound projects that

met the definition of the universe.1

From the universe of 395 projects, we selected a sample of 70 projects using stratified simple

random sampling: each project in the universe was assigned to a group of projects (a stratum) and a

sample was drawn from each stratum. Sampling rates varied across strata, so some projects had a
Three of the projects in the sample are “backups,” selected randomly from the same strata as three 2

originally selected projects for which it was determined that random assignment would be

inappropriate. Two of the three originally selected projects were operating under special

administrative provisions, and the third project had, for several years, been unable to fill all available

openings. These three projects that were replaced by backups are included in the universe counts in

Table A.1.

A-4

greater chance of being selected than other projects. Stratification with disproportionate sampling

(unequal sampling rates) was used to ensure that enough projects and, therefore, enough students were

selected to support precise estimates for relatively small, but important analytic subgroups, such as

students in large projects or students in projects hosted by two-year postsecondary institutions.

Table A.1 displays the 46 strata used to select projects in the first-stage sampling for the impact

study. The table also shows, for each stratum, the number of projects in the universe, the number of

projects selected for the sample, and the number of projects in which random assignment of students

was carried out. Within each stratum, projects were selected using simple random sampling without

replacement. Thus, although selection probabilities varied across strata, each project in a given stratum

had the same chance of being selected. That chance equals the number of projects selected divided

by the number of projects in the universe in that stratum.2

Strata are defined, in part, by cross-tabulating three stratifying variables:

1. Location of the host institution

2. Type and control of the host institution

3. Project size

The location variable has two categories: (1) urban and (2) rural. A project is classified as urban if the

host institution is located in a Metropolitan Statistical Area (MSA), as defined by the U.S. Bureau of

the Census. The type and control variable has three categories: (1) public, four-year; (2) private, fouryear; and (3) two-year. Type and control was ascertained from the 1990-91 Integrated Postsecondary
A-5

TABLE A.1

DISTRIBUTION OF UPWARD BOUND PROJECTS: IMPACT STUDY

Number of Projects

Sample

Stratum Universe Selected Respondents1

Urban: four-year, public

 Small:

African American 14 2 2

2

Latino 4 1 1

Other 7 1 1

 Medium:

Asian 5 2 2

Native American 2 1 1

Latino 9 2 2

Other 56 1 1

 Large:

African American 25 3 3

Latino 6 3 3

White 2 1 1

Other 6 1 1

Urban: four-year, private

 Small:

African American 8 1 1

Other 5 1 1

 Medium:

Asian 4 1 1

African American 38 3 3

Latino 3 2 2

Other 5 1 1

 Large:

Asian 2 1 1

African American 22 5 3

Other 3 1 1
TABLE A.1 (continued)

Number of Projects

Sample

Stratum Universe Selected Respondents1

A-6

Urban: two-year

 Small:

Native American 1 1 1

African American 9 3 3

Latino 3 1 1

Other 5 1 1

 Medium:

Asian 2 1 1

African American 10 3 3

Other 4 1 1

 Large 3 1 1

Rural: four-year, public

 Small:

White 6 1 1

Other 6 1 1

 Medium:

Native American 7 3 2

Latino 4 1 1

Other 30 1 1

 Large:

African American 5 1 1

Other 10 2 2

Rural: four-year, private

 Small 7 1 1

 Medium 14 2 2

 Large 4 1 1
TABLE A.1 (continued)

Number of Projects

Sample

Stratum Universe Selected Respondents1

A-7

Rural: two-year

 Small:

African American 4 2 2

White 5 1 1

Other 6 1 1

 Medium:

African American 5 1 1

White 8 2 2

Other 5 1 1

 Large:

White 3 1 1

Other 3 1 1

Total 395 70 67

Respondents are projects in which random assignment was carried out. 1

At least 50 percent of the students served by “African American projects” are classified as African 2

American according to the 1990-91 Upward Bound performance reports. Native American, Latino,

and white projects are similarly defined. (Native American includes Alaskan Native.) For Asian

projects, at least 25 percent of the students served are classified as Asian or Pacific Islander.
A-8

Education Data System (IPEDS) Institutional Characteristics file. The project size variable has three

categories: (1) small (60 or fewer students), (2) medium (61 to 99 students), and (3) large (100 or more

students). Enrollment figures were obtained from the 1990-91 Upward Bound performance reports.

Although some strata are defined entirely in terms of the location, type and control, and project

size variables, many strata are defined by also taking into account projects' racial/ethnic compositions.

At least 25 percent of the students served by “Asian projects” are classified as Asian or Pacific

Islander. For a Native American (including Alaskan Native), African American, Latino, or white

project, at least 50 percent of the students served are classified as members of the specified

racial/ethnic group. Data on race/ethnicity were obtained from Upward Bound performance reports.

When possible, projects were sampled proportionately by racial/ethnic composition within

classifications based on the other three stratifying variables. Thus, differences by racial/ethnic

composition in the overall rates at which projects were sampled are largely due to disproportionate

sampling by, mainly, size and type and control. Small projects, large projects, and projects hosted by

two-year postsecondary institutions were oversampled to provide adequate sample sizes for subgroup

analyses.

2. Second Stage Sampling: Selection of Students

For each project selected in the first stage, we identified its main recruiting period(s)--typically

spring 1993, fall 1993, or both--that fell during the student sample intake period for the impact study

(roughly October 1992 to March 1994). All eligible students applying to Upward Bound during a

project’s main recruiting period(s) were selected with certainty for the baseline impact study sample

and subject to random assignment to treatment (Upward Bound) and control groups. The exceptions

were students designated as “exempt” from random assignment and students who could have been
As we discuss later, students designated as post-initial treatments will not necessarily be members 3

of the treatment group for baseline or follow-up analyses.

Asindicated in Table A.1, random assignment was not carried out in 3 of the 70 projects selected 4

for the sample. The stated policy of one of those three projects was to serve all eligible applicants.

Although not policy, the practice of another project was also to serve all eligible applicants because

there were few eligible students attending the project’s target schools--just enough students to fill

program openings, leaving none to form a control group. The third project had its funding cut and had

no openings for new students. These three projects could not be replaced by backups even though, as

noted earlier, three other projects in which random assignment could not be carried out had been

replaced. Backups could not be selected because random assignment was determined to be infeasible

only after it had been announced that no additional projects would be selected for the impact study.

Failure to carry out random assignment in originally selected projects may introduce bias of unknown

direction and magnitude into sample estimates.

A-9

randomly chosen as “givebacks.” We discuss these exceptions after describing how random

assignment was conducted.

a. Random Assignment

When a project had completed recruiting for a given recruiting period, we selected eligible

applicants at random to fill all available program openings. Eligible applicants not selected for

Upward Bound and assigned to the treatment group were assigned to the control group or, more

accurately, to a waiting list that could be used to fill certain future program openings. The waiting list

and students selected from it, so-called “post-initial treatments (PITs),” are discussed in the next

section.3

At least one round of random assignment was conducted in each of 67 projects. In 17 projects 4

with more than one recruiting period, there were two or more rounds of random assignment. We

conducted a total of 87 rounds of random assignment.

Many Upward Bound project directors were concerned that the element of chance introduced by

random assignment could severely unbalance the student composition of their programs. For example,

it would be possible for an entire cohort of students to be from just one target school or all female. The

former outcome could have seriously damaged relationships with target schools whose students were
Stratification was also needed in some instances to ensure that a project did not violate the federal 5

requirement that two-thirds of the project’s students be both low income and potential first generation

college. For projects with multiple rounds of random assignment, each round had its own set of strata.

A-10

not selected while the latter might have hampered program operations if there were not enough

dormitory rooms available for females during the summer session. Therefore, project directors were

allowed to specify random assignment strata and, subject to there being enough eligible applicants, to

allocate available program openings across the strata to obtain the desired mix of students. In all,

5

there were 339 random assignment strata. Within a given stratum, random assignment was conducted

as described before. The eligible applicants in the stratum were chosen at random to fill the available

openings in the stratum. Students not picked for Upward Bound were assigned to the control

group/waiting list.

b. Exemptions, Givebacks, and PITs

Exemptions. At the request of Upward Bound project directors, a very small number of students

applying to Upward Bound were exempt from random assignment because it was determined that

allowing the assignment of such students to be subject to the vagaries of chance could be unusually

or permanently disruptive to normal program operations. For example, if a project and a local child

protective services agency had a prior agreement that all eligible students referred by the agency would

be accepted into Upward Bound, that agreement was not violated for the evaluation, and students

referred during the sample intake period were exempt from random assignment. Strict policies of

treating applicants from the same family the same were also honored and accounted for a few

exemptions. For instance, one of a pair of twins applying to a project with such a policy was exempt

from random assignment while the other twin was subject to random assignment. The exempt twin

would be allowed to participate in Upward Bound only if the nonexempt twin was randomly assigned
Even students who applied to Upward Bound and were placed on a project’s waiting list prior 6

to the sample intake period for the study were generally subject to random assignment. The only

exceptions were students who had been previously promised admission when openings became

available. Such students were among the small number of exemptions.

A-11

to the treatment group. All exempt students are nonresearch cases, meaning they were excluded from

all future data collection activities and from all analyses.

Givebacks. Some projects recruited many more eligible applicants than were needed to fill

available program openings and form a control group of adequate size. In such instances, we randomly

selected students from the control group and “gave them back” to the projects immediately after

random assignment. These givebacks could be selected by the projects to fill program openings when

the impact study student intake period had ended. Although subject to random assignment, givebacks

are not part of the baseline or follow-up samples. All 97 givebacks (distributed across 11 projects) are

nonresearch cases.

PITs. After being selected for Upward Bound, some students never enter the program. Other

students enter, but leave before completing the program. Therefore, Upward Bound projects typically

maintain waiting lists of students so that program openings can be filled without having to either mount

a full-scale recruiting effort or wait until the next recruiting period.

During the sample intake period for the impact study, projects were not allowed to have their own

waiting lists--all nonexempt applicants were subject to random assignment. To enable projects to

6

maintain full enrollment under such conditions, we assigned students not selected for Upward Bound

to an evaluation waiting list, rather than a strict control group. Students could be randomly selected

from the evaluation waiting list to fill program openings, although such use of the waiting list was

subject to time and size restrictions. Specifically, students could not be selected off the waiting list

after a certain date, typically the start of the next recruiting period. Also, for a given random

assignment stratum, a student could not be selected from the waiting list if the selection of a student
For example, without weighting, a total estimated from a simple one-in-two random sample

7

would, on average, fall short of the true (population) total by 50 percent.

A-12

reduced the number of students remaining on the waiting list to less than about two-thirds the number

of students originally assigned to the treatment group. Students randomly selected from the evaluation

waiting list are designated as PITs. The next section discusses how PITs are used in baseline and

second follow-up analyses.

B. WEIGHTING

Students were assigned sample weights that we have used in analyzing impact study data. A

student's weight indicates how many students in the universe she/he represents. A student with a

weight of four represents herself/himself and three other students that were not selected for the sample

(or did not respond to a survey questionnaire or other data collection instrument).

Weighting has three purposes. First, weighting ensures that the sample “weights up” to the

universe, producing correct totals (subject to sampling variability). Second, for purposes of

7

estimation, weighting “undoes” the effects of disproportionate sampling so that two strata with the

same number of students in the universe are counted equally even if they have different numbers of

students in the sample. Third, weighting adjusts for nonresponse.

In the following sections, we describe how we assigned baseline and second follow-up weights.

We constructed separate second follow-up weights for analyzing data from the survey, high school

transcripts, and postsecondary transcripts. To exclude exemptions and givebacks from all analyses,

we assigned them zero baseline and second follow-up weights. In contrast, all PITs were included in

baseline analyses and received nonzero baseline weights. Whether a PIT received a nonzero second

follow-up weight depended on when that student was selected off the evaluation waiting list, as

discussed below.
w '

1

project selection probability

×

(number of applicants)

s

(number of applicants & number of givebacks)

s

,

p ' project selection probability ×

(number of applicants & number of givebacks)

s

(number of applicants)

s

.

For the baseline sample, students were designated as treatments or controls based on their initial 8

random assignment status. Students initially selected for Upward Bound are treatments, while students

initially placed on the evaluation waiting list, including students who later became PITs, are controls.

In the expressions in the text, “number of applicants” is, more precisely, the number of

9

nonexempt eligible applicants. Also, the number of applicants minus the number of givebacks is the

number of treatments plus the number of controls.

A-13

1. Baseline Weights

We assigned nonzero baseline weights to 3,028 students--all nonexempt students except

givebacks--divided into 1,479 treatments and 1,549 controls. A student’s baseline weight is: 8

where s indexes the student’s random assignment stratum. This baseline weight is the inverse of the

student’s probability of being selected for the baseline sample. That selection probability is:

The first term on the right side of this last expression is the selection probability for the project to

which the student applied (adjusted for nonresponse, that is, failure to carry out random assignment).

The second term is the selection probability for the student conditional on the student’s project having

been selected and random assignment carried out. In other words, the two terms are, respectively, the

first and second stage selection probabilities. Their product gives the student’s overall (unconditional)

selection probability.9

The first and second stage selection probabilities are easy to calculate. The first stage probability,

the “project selection probability” in the last expression, is the number of projects in which random

assignment was carried out--given in the last column of numbers in Table A.1--divided by the number
A-14

of projects in the universe--given in the first column of numbers in Table A.1. This ratio is calculated

within a project’s first stage sampling stratum. The second stage selection probability equals one for

all but 239 students who applied to the 11 projects that received givebacks. Although they applied to

projects that received givebacks, these 239 students are not themselves givebacks and, therefore,

received nonzero baseline weights.

Two simple examples illustrate how we calculated baseline weights. For an applicant to a large,

rural project hosted by a private, four-year university, the project selection probability is 1/4.

(According to Table A.1, random assignment was carried out in 1 of the 4 large, rural projects hosted

by private, four-year universities.) If there were 7 other applicants (8 applicants in all) and no

givebacks in the student’s random assignment stratum, the second stage selection probability equals

1, and the overall selection probability equals 1/4 × 1 = 1/4. Therefore, the student’s baseline weight

is 4 (the inverse of 1/4), implying that the student represents herself/himself and three other students

who applied to projects in which random assignment was not carried out (because those projects were

never selected for the impact study sample). Alternatively, if there were 4 givebacks instead of none,

the second stage selection probability equals 4/8, and the overall selection probability equals 1/4 × 4/8

= 1/8. Then, the student’s baseline weight is 8. We can think of the selected student as representing

herself/himself, one giveback, and six other students (two applicants to each of three projects not

selected for the impact study sample).

In the first example (with no givebacks), each of the 8 students in the baseline sample gets a

weight of 4, and, together, the 8 students weight up to 8 × 4 = 32 students. In the second example

(with 4 givebacks), each of the 4 students in the baseline sample gets a weight of 8, while each of the

4 givebacks gets a weight of 0. Together, the 4 students in the baseline sample weight up to 4 × 8 =

32 students, the correct weighted total.
It is easy to see the consequences of not adjusting for nonresponse. Suppose all 22 large, urban, 10

predominantly African American projects hosted by private, four-year universities had 45 applicants

during the study intake period for 22 × 45 = 990 applicants in all. With random assignment carried

out in 3 of the 22 projects, our baseline sample consists of 3 × 45 = 135 applicants. If we do not adjust

for nonresponse, taking the project selection probability as 5/22 instead of 3/22, each student in the

baseline sample has an overall selection probability of 5/22 × 1 = 5/22 (assuming no givebacks) and

a baseline weight of 22/5. The 135 students in the baseline sample weight up to 135 × 22/5 = 594,

which is short of the correct total (990) by the nonresponse rate (40 percent). If we adjust for

nonresponse, those students weight up to 135 × 22/3 = 990, the correct total.

The nonresponse adjustment allows applicants to the responding projects to stand in for applicants

to the nonresponding projects by distributing weight from the latter to the former. Had random

assignment been carried out in all five projects selected for the sample, there would have been 5 × 45

= 225 students in the baseline sample, and each would have had a weight of 22/5. When random

assignment was not carried out in two of the five projects, 2 × 45 = 90 students did not make it into

the sample. So, we divided the total weight that those 90 students would have received (90 × 22/5 =

396) equally among the 3 × 45 = 135 students who made it into the sample when random assignment

was carried out in the three projects to which they applied. The adjusted weight for each of these 135

students is the unadjusted weight plus the distributive share, or 22/5 + 396/135 = 22/3 (after

simplification).

Item nonresponse, that is, failure to answer individual questions, created little missing data above 11

and beyond that created by unit nonresponse. (Unit nonresponse is the failure to answer any questions.)

A-15

Our simple examples involve applicants to a large, rural project hosted by a private, four-year

university. Although random assignment was carried out in the one such project selected for the

impact study sample, project selection probabilities were adjusted when necessary for nonresponse,

that is, failure to carry out random assignment. For instance, the adjusted selection probability for the

three large, urban, predominately African American projects hosted by four-year, private universities

is 3/22. Using that selection probability, rather than 5/22, in weighting allows applicants to the 3

responding projects to represent applicants to the 2 nonresponding projects (as well as applicants to

the 17 projects that were never selected in the first place).10

2. Second Follow-up Survey Weights

As discussed in the previous section, 3,028 students received nonzero baseline weights. We

wanted second follow-up survey data for all of these students and succeeded in obtaining at least

partially completed survey questionnaires for 2,608 students--an 86 percent response rate. We next 11
Project directors often do not regard a slot as open until there is strong evidence that a previously 12

enrolled student has dropped out. Therefore, rather than delaying student selection until the “last

minute,” some slots that were later confirmed as open were not filled in the initial random assignment.

A-16

discuss how we weighted students for analyses of the second follow-up survey data and how, in

weighting students, we adjusted for unit nonresponse. We begin by describing how we designated

students as treatments, controls, or nonresearch cases.

a. Designating Students as Treatments, Controls, or Nonresearch 

Of the 3,028 students who received nonzero baseline weights, 1,524 were designated as

treatments, 1,320 as controls, and 184 as nonresearch cases for second follow-up analyses.

Designations were made as follows. All 1,479 students assigned to the treatment group at initial

random assignment are treatments for second follow-up analyses. Likewise, all 1,320 students

assigned to the evaluation waiting list at initial random assignment and not randomly selected off it (as

PITs) are controls. Of the 229 PITs, 45 are treatments for second follow-up analyses, and the rest are

nonresearch cases.

A PIT was designated as a treatment if two conditions were satisfied. First, the PIT had the

opportunity to begin participating in Upward Bound at essentially the same time (often the same day)

as the “original” treatments in the PIT’s random assignment stratum. Second, the PIT did not replace

a treatment who dropped out of Upward Bound (or never showed up). PITs satisfying these two

conditions were designated as treatments because it is assumed that they would have been original

treatments had the Upward Bound project director not underestimated the number of open slots that

were available at the initial random assignment. As noted before, all other PITs were designated as 12

nonresearch cases.
Because of nonresponse, 30 of the original 339 random assignment strata “lost” either all of their 13

treatments or all of their controls (but never both as it turned out) and were combined with other strata.

Strata were combined based on propensity scores, whose estimation is described later in the text. We

combined an empty stratum with a nonempty stratum based on the similarity of students’ propensity

scores, as measured by the average squared difference in propensity scores between students in a given

empty stratum and students in a nonempty stratum with which the empty stratum might be combined.

The nonempty stratum with the lowest average squared difference was judged the most similar to the

empty stratum. We did not combine strata across projects or from random assignments that occurred

at widely separated points in time. Propensity scores were used to combine strata because they reflect

a broad range of characteristics related to nonresponse and to the outcomes that are examined in the

impact analysis. There were 42 respondents and 54 nonrespondents or nonresearch cases in the 30

empty strata.

The probability was obtained from a logistic regression model that related, for all 3,028 students, 14

a binary response variable to a large set of predictor variables. This set included 30 variables in all--

stratifying variables, variables measuring student baseline characteristics, and interaction variables

obtained by multiplying pairs of characteristics variables. The characteristics variables and interaction

variables in the “best” logit model, that is, the model used to estimate propensity scores were selected

using a forward selection procedure with a liberal inclusion criterion.

A variable included in the model should have two properties: (1) it should be a good predictor of

the propensity to respond and (2) it should be a good predictor of the outcomes of interest in the impact

analysis. We tried to ensure satisfaction of the first property by using the forward selection procedure.

We tried to ensure satisfaction of the second property by applying our substantive knowledge about

what factors have been shown to influence education and education-related outcomes. This led us to

A-17

b. Preliminary Weights 

Each of the 3,028 students with a nonzero baseline weight, whether designated as treatment,

control, or nonresearch for second follow-up analyses, was assigned her/his baseline weight as a

preliminary second follow-up survey weight.

c. Final Weights 

To develop final weights that both facilitate comparisons of treatments and controls by weighting

them up to the same totals and incorporate an adjustment for nonresponse, we calculated a set of

control totals. Specifically, we summed the preliminary weights of all students (treatment, control, and

nonresearch) in each random assignment stratum to obtain 309 control totals. After deriving the

13

control totals, we multiplied each student’s preliminary weight by the inverse of her/his propensity

score. This score is an estimated probability that the student was a respondent. This product was 14
consider for possible inclusion in the model only some of the many variables measured at baseline.

In fact, only 17 characteristics variables were considered, and only two were included in the model by

the selection procedure used. Counting stratifying and interaction variables as well as characteristics

variables, we considered over 130 variables for inclusion in the model. 

The ratio adjustment procedure works as follows. Suppose that the product of the baseline

15

weight and the inverted propensity score sums to 100 across the treatment group respondents. Then,

if the control total for the random assignment stratum is 120, we multiply, for every treatment group

respondent, the product of the baseline weight and the inverted propensity score by the ratio 120/100

= 1.2 to obtain a final weight. After this adjustment, the final weights sum to 120, the control total for

the stratum.

In addition to the weights for the analyses of second follow-up data, we created weights for the 16

analyses in Chapter II of Upward Bound participation, completion, and persistence. Each of the 1,524

students designated as a treatment received a nonzero weight, while all other students received weights

of zero. The “participation” weights for the 1,524 treatments were constructed so that the weighted

distribution of treatments across random assignment strata (calculated using participation weights) is

the same as the weighted distribution of eligible applicants across random assignment strata at baseline

(calculated using baseline weights). Thus, for treatments, the participation and second follow-up

weights are constructed in the same way, except that the participation weights do not need to be

adjusted for unit nonresponse because there was none. 

A-18

then ratio-adjusted to obtain a final weight, such that the final weights for treatments and controls

summed (separately) to the control totals. Final weights for all nonrespondents and nonresearch

15

cases equal zero. 

The final weights are constructed so that the weighted distribution of treatments across random

assignment strata (calculated using the final weights) is the same as the weighted distribution of

eligible applicants across random assignment strata at baseline (calculated using baseline weights).

Likewise, the weighted distribution of controls is the same as the weighted distribution of eligible

applicants. Therefore, the weighted distributions of treatments and controls are the same.16
Since there was higher nonresponse to the high school transcripts, a total of 34 of the original 17

339 random assignment strata “lost” either all of their treatments or all of their controls. These 34

strata were combined with other strata, as described earlier. There were 53 respondents and 58

nonrespondents or nonresearch cases in the 34 empty strata. The logistic regression model used to

predict the probability of response to the high school transcript differed slightly from the model used

to predict response to the second follow-up survey. 

A-19

3. Second Follow-up High School Transcript Weights

To construct nonresponse-adjusted weights for the second follow-up analyses of high school

transcript data, we followed the same procedures used to construct weights for the second follow-up

survey data. As with the survey, we wanted high school transcript data for all of the 3,028 students

and succeeded in obtaining transcripts for 2,494 students--an 82 percent response rate. To obtain a

high school transcript for a sample member, we needed to know the names of the high schools the

student attended. Therefore, all of the 420 students who are second follow-up survey nonrespondents

are also nonrespondents for the high school transcripts. In addition, we were unable to obtain high

school transcripts for 114 of the second follow-up survey respondents. A student’s designation as a

treatment, control, or nonresearch case is the same for all components (survey, high school transcript,

and postsecondary transcript) of the second follow-up study. 

a. Preliminary Weights 

Preliminary high school transcript weights are the same as the preliminary second follow-up

survey weights. 

b. Final Weights 

Final high school transcript weights were constructed using the same methods described for the

second follow-up survey weights.17
Thus, the final postsecondary transcript weight equals the nonzero second follow-up survey

18

weight for the 1,967 survey respondents who did not report any postsecondary attendance and the zero

second follow-up survey weight for the 420 survey nonrespondents.

There were too few college attendees to control to random assignment stratum totals. So, we 19

controlled to project totals instead.

A-20

4. Second Follow-up Postsecondary Transcript Weights

In the second follow-up survey, 641 students said they had attended some type of postsecondary

institution. Using the name of this institution provided by the survey respondent, we succeeded in

obtaining college transcripts for 551 of the 641 students--an 86 percent response rate. In the

development of the postsecondary transcript weights, the purpose of adjusting for nonresponse is to

distribute the weight of the 90 nonrespondents across the 551 respondents. These respondents stand

in for themselves and for their college-going peers for whom we do not have postsecondary transcripts.

a. Preliminary Weights 

Each of the 3,028 students with nonzero baseline weights was assigned a preliminary

postsecondary transcript weight equal to her/his final second follow-up survey weight.

b. Final Weights 

The final postsecondary transcript weight is equal to the preliminary weight for each of the

2,387 students not reporting postsecondary attendance in the second follow-up survey. In contrast, 18

for the 641 postsecondary attendees, final weights do not equal preliminary weights. The final

weights for the 90 postsecondary transcript nonrespondents were set equal to zero. The final weights

for the 551 respondents were adjusted to compensate for the missing 90, as described next.

To derive control totals for the nonresponse adjustment, we summed (separately for treatments

and controls) the preliminary weights of all postsecondary attendees in each project. Then, we

19
The logistic regression model used to predict the probability of response to the postsecondary 20

transcripts differed significantly from the models used for the second follow-up survey and high school

transcript weights. The postsecondary transcript model was estimated only across the sample of 641

college attendees. Due to the smaller sample size, we eliminated all 105 interaction terms that were

allowed in previous models. Consequently, the “best” model included only 10 predictors, nine of

which were stratifying variables. 

Aminor exception to this property arises because there were five nonrespondents in projects that 21

did not include any college-attending respondents. Since we could not make the nonresponse

adjustment for these students within their respective projects, we distributed their preliminary weights

across all of the other respondents, separately for treatments and controls, so that the sum of the

preliminary weights equals the sum of the final weights across the sample of 641 college attendees.

Despite the fact that the second follow-up survey weights and not the baseline weights were used 22

as preliminary weights, this statement is true because the second follow-up survey weights were

controlled to the baseline distribution of eligible applicants. The issue mentioned in the preceding note

causes small differences from the baseline distribution.

A-21

multiplied each student’s preliminary weight by the inverse of her/his propensity score. The

20

resulting product was ratio-adjusted to obtain a final weight, such that the final weights for

treatments and controls reporting postsecondary attendance summed (separately) to the project-level

control totals.21

The final weights are constructed so that the weighted distribution of treatments across projects

(calculated using the final weights) is the same as the weighted distribution of eligible applicants across

projects at baseline (calculated using baseline weights). Likewise, the weighted distribution of

22

controls is the same as the weighted distribution of eligible applicants. Therefore, the weighted

distributions of treatments and controls are the same.

C. CALCULATING STANDARD ERRORS

Throughout this report, we present many estimates, such as percentages and means. These

estimates are called “point” estimates because they are single values, as opposed to ranges of values.

We can also obtain standard errors for these point estimates. As its name implies, a standard error is

an estimate of the error in a point estimate, that is, an expression of our uncertainty. Typically,
We estimate that there were nearly 22,000 eligible students who applied to Upward Bound

23

during the impact study sample intake period.

For purposes of illustration, we have described here a “two-tailed” test, where any estimate far 24

enough from zero--either above or below--casts doubt on the hypothesis that the true value is zero.

A-22

standard errors are used to construct “interval” estimates or “confidence intervals” that give a range

of possible values. A “95-percent” confidence interval extends from two standard errors below the

point estimate to two standard errors above the point estimate. Thus, when we estimate from baseline

data that 11 percent of eligible Upward Bound applicants do not expect to attend college and the

standard error for this estimate is 2 percent, the 95-percent confidence interval runs from 11 ! 2 × 2

to 11 + 2 × 2, or from 7 to 15 percent. One interpretation of this confidence interval is that if we

repeated our sampling and estimation procedures 100 times (drawing a new random sample each time),

about 95 percent or 95 of the 100 confidence intervals that we construct will contain the true

percentage of eligible Upward Bound applicants who do not expect to attend college. That true

percentage is the percentage that would have been obtained if we had surveyed all applicants in the

universe, rather than a sample of 3,028 applicants. In our example, we are 95-percent “confident” 23

that the true percent of applicants who do not expect to attend college lies between 7 and 15 percent.

Of course, the true percentage either does or does not lie in that range.

Instead of confidence intervals, we can derive “t-statistics,” which are closely related to

confidence intervals. Dividing a point estimate by its standard error gives a t-statistic. If that t-statistic

is less than two in absolute value (that is, between -2 and 2), we conclude that, at the 95-percent

confidence (5-percent significance) level, the point estimate is not “significantly different” from zero;

in other words, the observed difference from zero may be due entirely to the element of chance

introduced by sampling. Determining whether the t-statistic is less than two in absolute value is the

same as determining whether the confidence interval includes zero. When the confidence interval

includes zero, we are not confident that the true value is different from zero.24
Since the aim of most analyses presented in this report is to determine whether Upward Bound has a

beneficial impact and not whether it just has some impact, good or bad, we have usually conducted

“one-tailed” tests. With a one-tailed test, only an impact estimate far enough above zero supports the

hypothesis that Upward Bound has a beneficial impact. Any other impact estimate--including a large,

but negative estimate--casts doubt on that hypothesis.

A-23

To estimate error and express our uncertainty accurately, standard errors must be calculated using

methods that reflect how the sample was drawn. For almost all analyses, we used SUDAAN or

STATA, two computer software packages that use the Taylor series linearization method to calculate

standard errors based on the user’s coded description of the sample design.
APPENDIX B

ESTIMATION OF PROGRAM IMPACTS
B-3

We estimated the impact of Upward Bound on student outcomes using three approaches. To

compute the impact for all students who were selected for Upward Bound, we used subclassification

analysis. Estimation of subgroup impacts for students who were selected for Upward Bound was

based on an analytic model. Finally, to compute the overall impact and subgroup impacts for

participants we used statistical model the specified the relationship between selection into the

treatment and control groups, participation, and student outcomes. We describe each of these

approaches below.

A. SUBCLASSIFICATION ANALYSIS

Since we used random assignment to construct the two groups, we would ideally compute the

program’s impact by comparing the average for an outcome for the treatment group with the average

for the control group. However, small differences may exist between the treatment group and the

control group (see Appendix D). To compute the overall impact of Upward Bound on student

outcomes when taking into account differences in their background characteristics, we selected

subclassification analysis; this approach requires few assumptions concerning the structure of the

data (see, for example, Rosenbaum and Rubin, 1983 and 1984). For the subclassification analysis,

we formed groups of similar students and within these groups we computed the difference between

the average outcome for students who were selected for Upward Bound and the average outcome for

students who were in the control group. To arrive at the overall impact, we averaged the within

group estimates. 

The process for constructing the groups and for computing program impacts included the

following steps:

1. For each student, we computed the probability of being in the treatment group. The

probabilities were predicted using a logit model that included as predictors items
y'$0

%$1T%$2G1%$3(TxG1)%$4P%,

V'j

6

1

p

2

j

(Vj) Vjpj

T G P

$0$1$2$3$4

B-4

such as sex, race/ethnicity, grade at application, students’ educational expectations,

and participation in high school activities.

2. With the predicted probabilities for the treatment group, we constructed six

categories, each with an equal number of students. We sorted students in the

treatment group and the control group into the six groups based on their predicted

probabilities. The idea is that students, who have similar predicted probabilities of

being selected for Upward Bound, will be similar in terms of the characteristics used

to predict the probabilities (Rosenbaum and Rubin 1983).

3. Within each of the six groups, we computed the program’s impact by taking the

difference between the average of an outcome for the treatment group and the control

group. The within group impact shows how much being selected for Upward Bound

effects similar students’ outcomes. Computing the average of the six impact

estimates shows the overall impact of Upward Bound.

4. We computed the variances of the overall impact estimates with the following

expression: where is the within group variance and is the

proportion of students from the treatment group who were in the jth category.

B. IMPACTS FOR SUBGROUPS

We used an analytic model to compute subgroup impacts. The analytic model included the

student outcome as the dependent variable, and as independent variables in the model we included

treatment status (Upward Bound or control group), an indicator for the subgroup, and the estimated

probability of being in the treatment group. The type of analytic model we estimated was based on

the measurement of the outcome variable. For outcomes measured on an interval scale such as, years

of schooling or high school credits, we used a regression model. For outcomes measured on a 0/1

scale, such as college attendance, we used a logit model. The general specification of the analytic

model was:

where , , and correspond to the treatment indicator, an indicator variable that shows which

subgroup a student was in, and the probability of being in the treatment group; , , , , and 
,

$1$3

 This model specification permits an analysis of two subgroups such as boys and girls, or

1

students with lower or higher initial educational expectations. The model can be expanded to

incorporate more subgroups by including additional subgroup indicators and interaction terms.

B-5

are parameters to be estimated, respectively; and shows the effect of unmeasured factors on the

outcome. The two parameters of direct interest are and because they can be used to compute 1

the impact of Upward Bound on the subgroups and test whether a statistically significant difference

exists between the impacts.

This approach was used instead of the subclassification analysis because some groups were

small when we computed impacts for specific groups such as students with lower initial education

expectations. To use the analytic models approach when estimating subgroup impact, we needed

to assume that the relationship between the probability of being selected for the treatment group and

the outcome was linear and did not differ for the subgroups. For the subclassification analysis we

did not need to make this assumption.

C. IMPACTS FOR PROGRAM PARTICIPANTS

Both approaches for computing program impacts described in sections A and B provide

information about program impacts for all students who were selected for Upward Bound. Since

some students we selected for Upward Bound decided not to participate in the program, it is useful

to ask about the impacts for those who showed up for services (about 82 percent of all students

selected for Upward Bound participated). To compute the impact for participants we use a

framework suggested by Angrist, Imbens, and Rubin (1996).

We can form an impact estimate for participants by using the following framework. First, we

express the mean (expected value) for an outcome such as academic credits earned in high school

for students in the treatment group and the control group as:
E(y

T

)'E(y

T

*S'1)Pr(S'1)

T

%E(y

T

*S'0)[1&Pr(S'1)

T

]

E(y

C

)'E(y

C

*S'1)Pr(S'1)

C

%E(y

C

*S'0)[1&Pr(S'1)

C

]

E(y

T

*S'1)'[E(y

T

)&E(y

T

*S'0)[1&Pr(S'1)

T

]]/Pr(S'1)

T

E(y

C

*S'1)'[E(y

C

)&E(y

C

*S'0)[1&Pr(S'1)

C

]/Pr(S'1)

C

E(yT) E(yC) E(yT*S'1)

E(yC*S'1)

E(yT*S'0) E(yC*S'0)

Pr(S'1)

T

Pr(S'1)C

B-6

and

where and are the means for the outcome in the treatment group and the control group, 

and are the means for the treatment group and the control group for students who showed

up for services or who would have shown up for services if they had been given the opportunity to

participate in Upward Bound, and are the means for the outcomes for students

in the treatment group and the control group who did not show up for services and whom we would

expect to not show up if given the opportunity to participate, and and are the

probabilities that students in the treatment group and the control group showed up or would show

up for services if given the opportunity to participate. 

For treatment group students, we are interested in the expected value of the student outcome,

given that students participated in Upward Bound when offered the opportunity to participate. For

students in the control group we are interested in the expected value of the student outcome, given

that the students would have participated if given the opportunity. Using the expressions we have

already defined, we can express the desired quantities as:

and 
I(participants)'E(y

T

*S'1)&E(y

C

*S'1)

I(participants)'[E(yT)&E(yC)]/Pr(S'1)

T

di'"0%"1Ti%,i1

y

i

'$0%$1di%,i2

E(yT*S'0)'E(yC*S'0)

Pr(S'1)

T

'Pr(S'1)C

diTiyi

" $ $1

B-7

Then, the impact for participants is:

If we assume that Upward Bound had no impact on the students who did not show up (that is,

) and the probability of being a no-show in the treatment group and the control

group are the same (that is, , then we can estimate the impact for participants

as:

As Angrist. Imbens, and Rubin (19xx) show, Bloom’s estimator is equivalent to the instrumental

variables estimator when using the following setup:

and

where , , and correspond to an indicator of whether a student participated in Upward Bound

(0=no and 1=yes), a treatment status indicator, and the outcome for the ith student, respectively; the

‘s and ‘s are parameters to be estimated. Here, an estimate of the parameter shows the impact

of participating in Upward Bound on students’ outcomes. To adjust for possible differences in the

characteristics of the students in the treatment group and the control group at baseline, we included

in the two equations as an independent variable the predicted probability of being in the treatment

group at the time of randomization. To compute statistical tests, we used the standard errors
B-8

estimated for the parameters in the model and computed t-tests. We extended this model

specification to include subgroup indicators when we computed subgroup impacts for participants.

Although the specific formulation for estimating impacts for participants is best suited for

outcomes measured on an interval scale, we have applied it to binary outcomes as well (for example,

whether a student went to college). Estimation of logit models or similar models in this context is

not readily carried out. With the approach we applied here, we obtain unbiased estimates; however,

they are less precise than if we had used an approach explicitly designed for analyzing binary

outcomes.
APPENDIX C

ESTIMATION OF THE EFFECTS OF DURATION

ON STUDENT OUTCOMES
 Rosenbaum and Rubin (1983) provide a detailed discussion of this approach. 1

C-3

We used a combination of statistical matching and modeling procedures to compute the effects

of duration in Upward Bound on participants’s outcomes. Below, we describe each of these

procedures.

A. FORMING A ONE-TO-MATCH USING PROPENSITY SCORES

Students who participated in Upward Bound were sorted into one of three duration groups: (1)

participated for 1 to 12 months, (2) participated for 13 to 24 months, and (3) participated for more

than 24 months. After sorting participants into the groups, we matched each participant with a

similar student from the control group using a propensity score. The propensity score corresponds 1

to the estimated probability of a student being in Upward Bound for a specified amount of time (for

example, 1 to 12 months).

To estimate the propensity score, we took all participants from one duration group and pooled

them with the students in the control group. A logit model was then used to predict each student’s

propensity score. We coded the dependent variable for the logit model as 1 for students who

participated in Upward Bound for a specific amount of time and 0 for those in the control group.

The variables used to predict the chances of staying in Upward Bound included sex, race/ethnicity,

educational expectations, years in the United States, misbehavior in school, employment status, and

parental involvement in school related activities. Separate logit models were estimated for students

in each duration group.

After we computed the propensity scores, we then took each Upward Bound participant and

matched them with a student in the control group who had the closest propensity score. This one-toone match formed the basis for computing the effects of duration on participants’ outcomes. The
di

'$0%$1Di1%$2Di2%Xi"%,i

di Di1

Di2

Xi

,i

$ "

$0%X ̄" $0%$1%X ̄"

$0%$2%X ̄" X ̄

C-4

outcome from the Upward Bound participant shows what the student achieved when attending and

the outcome for the matched-control group student shows what the participant would have achieved

if they had not participated in Upward Bound. Comparing the two outcomes shows the impact of

Upward Bound for each student.

B. ADJUSTING FOR DIFFERENCES AMONG DURATION GROUPS AND COMPUTING

EFFECTS

With the three groups of participants--students who remained in Upward Bound for 1 to 12

months, 13 to 24 months, and more than 24 months--we can compute the impact of Upward Bound

for students who remained in the program for each of these periods. Since participants who

remained in Upward Bound for longer periods differed on a variety of observed characteristics which

may have affected their outcomes (see Table C.1), we statistically adjusted for these differences

when we computed the impacts for each duration group. To make these adjustments we used a

regression model. The regression model was specified as:

where is the difference in outcomes for participant i, shows if a participant attended an

Upward Bound program for 13-24 months, indicates if a participant attended for more than 24

months, is a vector of student characteristics, is factor that shows the effects of unobserved

characteristics on the differences in outcomes, and the ‘s and are parameters (or vectors of

parameters) to be estimated. The adjusted impact of attending an Upward Bound program for 1 to

12 months corresponds to , the impact for participating for 12 to 24 months is ,

and the impact of participating for more than 24 months is where is a vector of means
C-5

TABLE C.1

BASELINE CHARACTERISTICS OF TREATMENT GROUP,

BY DURATION IN UPWARD BOUND

Months in Upward Bound

Baseline Variable Months Months or More

1-12 13-24 25 Months

Students’ educational expectations 16.7 16.6 16.6

Fathers’ educational expectations 16.6 16.6 16.7

Mothers’ educational expectations 16.7 16.5 17.1

Grade at application 9.4 9.7 9.0

Number of siblings 2.5 2.7 2.2

Female 0.70 0.69 0.69

Race/Ethnicity:

 Black

 Hispanic

 White

 Asian

 Native American

0.45 0.43 0.49

0.28 0.20 0.26

0.19 0.29 0.19

0.03 0.04 0.04

0.06 0.04 0.02

Talked with parents sometimes/often about:

Courses

School activities

Studies

Grades

Transferring to another school

Taking ACT/SAT exam

College plans

0.62 0.81 0.76

0.72 0.78 0.79

0.57 0.66 0.69

0.79 0.81 0.88

0.20 0.26 0.17

0.28 0.41 0.31

0.77 0.83 0.85

Parent did the following sometimes/often:

Checked on homework

Helped with homework

Gave special privileges

Limited privileges

Required chores

Limited TV/video time

Limited time with friends

0.68 0.70 0.76

0.54 0.55 0.72

0.62 0.61 0.71

0.60 0.50 0.59

0.82 0.87 0.93

0.46 0.47 0.63

0.69 0.76 0.71
TABLE C.1 (continued)

Months in Upward Bound

Baseline Variable Months Months or More

1-12 13-24 25 Months

C-6

Times:

Late for school

Skipped classes

Missed a day of school

In trouble for not following school rules

Put on in-school suspension

Suspended

Transferred for disciplinary reasons

Arrested

Spent time in juvenile home

3.1 2.3 1.9

0.8 0.6 0.6

4.2 4.1 3.1

1.3 0.8 1.1

0.5 0.2 0.4

0.3 0.2 0.2

0.0 0.0 0.0

0.1 0.0 0.0

0.0 0.0 0.1

Parent:

Attended school meeting

Spoke with teachers

Visited classes

Attended school event

0.61 0.67 0.65

0.66 0.77 0.64

0.46 0.45 0.54

0.61 0.57 0.73

Hours spent:

On homework

On school-sponsored activities

Working (school year)

Working (summer)

7.0 7.5 7.0

4.2 4.2 4.5

1.4 1.6 0.6

7.2 6.9 3.0

Number of high school activities 3.4 2.8 3.5
We have used the standard errors computed from a bootstrap procedure. The bootstrap is based 2

on the idea of computing the direct variability in the parameter estimates of the regression equation

when there is repeated sampling from the same population (for these analyses we used 200 bootstrap

samples to estimate the variability in the estimates). The estimates of the standard errors only

capture variability in the parameter estimates of the regression equation after the matches were

formed. That is, we do not incorporate the variability that may have occurred in the matching

process of different bootstrap samples that were used at two stage of the analysis. A consequence

of not capturing variability in the matching process may be an underestimate of the standard errors

and thus concluding that there were statistically significant effects when in fact there were none. 

C-7

for the independent variables in the regression model. To compute tests of statistical significance

we use the standard errors of the estimates.2
APPENDIX D

USE OF SUPPLEMENTAL SERVICES
D-3

TABLE D.1

PERCENT OF STUDENTS USING SUPPLEMENTAL 

SERVICES: BY SUBGROUP

Upward Bound Control Group Students

Any Year Summer Any Year Summer

Academic Academic

Overall 41% 39% 15% 58% 54% 21%

Educational Expectations:

Higher 44 42 16 57 54 24

Lower 37 36 14 64 59 17

Race/Ethnicity:

African American 44 42 17 63 58 28

White 40 39 12 45 43 9

Hispanic 36 34 10 55 53 16

Sex:

Boys 40 37 17 56 54 14

Girls 42 40 14 58 54 24

Low Income/First Generation:

LIFG 41 39 15 57 54 22

LI 67 66 34 68 67 19

FG 34 31 10 57 52 23

Academic Risk:

Lower 43 41 14 62 56 26

Higher 40 38 16 53 51 16