U . S . D E P A R T M E N T O F E D U C A T I O N

Upward Bound Math-Science: 

Program Description and Interim 

Impact Estimates
Upward Bound Math-Science: 

Program Description and Interim Impact Estimates 

Prepared by: 

Robert Olsen 

Neil Seftor 

Tim Silva 

David Myers 

David DesRoches 

Julie Young 

Mathematica Policy Research, Inc. 

For: 

U.S. Department of Education 

Office of Planning, Evaluation and Policy Development 

2007 
This report was produced for the U.S. Department of Education under Contract Number ED-04-CO-0152. The 

project monitor was Marisol Cunnington in the Policy and Program Studies Service. The views expressed herein do 

not necessarily represent the positions or policies of the Department of Education. No official endorsement by the 

U.S. Department of Education of any product, commodity, service or enterprise mentioned in this publication is 

intended or should be inferred. 

U.S. Department of Education 

Margaret Spellings 

Secretary

Office of Planning, Evaluation and Policy Development 

Doug Mesecar

Acting Assistant Secretary

Policy and Program Studies Service 

Alan L. Ginsburg 

Director

Program and Analytic Studies Division 

David Goodwin 

Director 

April 2007

This report is in the public domain. Authorization to produce it in whole or in part is granted. Although permission 

to reprint this publication is not necessary, the citation should be: U.S. Department of Education, Office of 

Planning, Evaluation and Policy Development, Policy and Program Studies Service, Upward Bound Math-Science: 

Program Description and Interim Impact Estimates, Washington, D.C., 2007. 

 

To order copies of this report, write: 

ED Pubs 

Education Publications Center 

U.S. Department of Education 

P.O. Box 1398 

Jessup, MD 20794-1398; 

Or via electronic mail, send your request to: edpubs@inet.ed.gov. 

You may also call toll-free 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet available in your area, 

call 1-800-872-5327 (1-800-USA-LEARN). Those who use a telecommunication device for the deaf (TDD) or a 

teletypewriter (TTY) should call 1-800-437-0833. 

To order online, point your Internet browser to: www.edpubs.org. 

This report is available on the Department’s Web site at: http://www.ed.gov/about/offices/list/opepd/ppss/index.html. 

On request, this publication is available in alternative formats, such as Braille, large print, or computer diskette. For 

more information, please contact the Department’s Alternate Format Center at (202) 260-9895 or (202) 205-8113. 
iii 

CONTENTS 

Chapter Page 

LIST OF EXHIBITS..................................................................................................................v 

ACKNOWLEDGMENTS ...................................................................................................... vii 

EXECUTIVE SUMMARY ..................................................................................................... ix 

I INTRODUCTION ...........................................................................................................1 

 A. UPWARD BOUND MATH-SCIENCE PROGRAM .............................................2 

 

 B. EVALUATION OF THE UPWARD BOUND MATH-SCIENCE 

 PROGRAM..............................................................................................................3 

 

 C. OVERVIEW OF THIS REPORT............................................................................4 

II THE OPERATION OF THE UPWARD BOUND MATH-SCIENCE PROGRAM ......5 

 A. PROJECT HOSTS AND STAFF ............................................................................6 

1. Host Institutions: Two- and Four-Year Colleges and Universities..................6 

2. Summer Program Staff and Project Director....................................................6 

 B. ELIGIBILITY, RECRUITMENT AND ENROLLMENT, AND 

 STUDENT CHARACTERISTICS..........................................................................8 

1. Eligibility ..........................................................................................................8 

2. Recruitment and Enrollment...........................................................................10 

3. Student Characteristics ...................................................................................12 

 C. PROGRAM DESCRIPTION.................................................................................13 

1. Setting.............................................................................................................14 

2. Goals of the Program......................................................................................14 

3. Academic Orientation.....................................................................................14 

4. Summer Offerings ..........................................................................................16 

5. Academic-Year Offerings...............................................................................18 

6. Instructional Approaches ................................................................................19 

 D. INTENSITY AND QUANTITY OF SERVICES .................................................20 
CONTENTS (continued) 

iv 

Chapter Page 

III THE IMPACTS OF THE UPWARD BOUND MATH-SCIENCE PROGRAM .........23 

 A. STUDY DESIGN...................................................................................................23 

1. Selecting the Samples .....................................................................................25 

2. Collecting Baseline Data ................................................................................27 

3. Selecting a Matched Comparison Sample for the Impact Analysis................28 

4. Estimating the Impacts of UBMS Participation .............................................31 

 B. ANALYSIS AND FINDINGS ..............................................................................31 

1. The Effect of UBMS on High School Outcomes ...........................................32 

2. The Effect of UBMS on Postsecondary Attendance, Persistence, and 

Completion .....................................................................................................33 

3. The Effect of UBMS on Postsecondary Field of Study..................................36 

 C. INTERPRETATION OF THE FINDINGS...........................................................37 

REFERENCES ........................................................................................................................41 

APPENDIX A: DATA COLLECTION............................................................................... A-1 

APPENDIX B: PROGRAM IMPACTS...............................................................................B-1 

 

APPENDIX C: SENSITIVITY OF IMPACTS TO AN ALTERNATIVE 

 MEASURE OF POSTSECONDARY ATTENDANCE ............................C-1 

APPENDIX D: SAMPLE SIZES AND STANDARD ERRORS ....................................... D-1 
v 

EXHIBITS 

Exhibit Page

II.1 Types of Institutions that Hosted UBMS Projects, 1995.........................................7 

II.2 Predominant Racial and Ethnic Group for UBMS Staff, Summer 1998 .................9 

II.3 Racial and Ethnic Distribution for UBMS Staff and Project Directors, 

 Summer 1998...........................................................................................................9 

II.4 Targets for Recruiting by UBMS Projects, 1994...................................................11 

 

II.5 Geographic Scope of Recruiting by UBMS Projects, 1994...................................11 

 

II.6 Predominant Racial and Ethnic Group for Participants at UBMS Projects, 

 Summer 1998.........................................................................................................13 

II.7 Most Important Goals of UBMS Projects, Summer 1994 .....................................15 

II.8 Academic Orientation of UBMS Projects..............................................................15 

II.9 Instruction Offered by UBMS Projects, by Subject Area......................................17 

II.10 Noninstructional Services Offered by UBMS Projects .........................................18 

II.11 Academic Year Services Offered by UBMS Projects, 1994-95............................19 

II.12 Instructional Methods Used by UBMS Projects, 1994..........................................21 

II.13 Per-Capita Funding for UBMS Projects, FY 2004 ................................................22 

 

III.1 List of Baseline Variables......................................................................................28 

III.2 Summary Statistics from the Baseline Variables...................................................29 

III.3 Math and Science Courses Taken in High School.................................................33 

III.4 Postsecondary Attendance, Highest Level Attended and College Selectivity.......34 

III.5 Years of College and Degree Completion .............................................................36 

III.6 Field of Study.........................................................................................................38

vii

ACKNOWLEDGMENTS 

This report on Upward Bound Math-Science reflects the contributions of many individuals. 

The authors would like to thank David Goodwin from the Policy and Program Studies Service 

(PPSS) of the U.S. Department of Education for his support and guidance throughout the study. 

The authors would also like to thank Maggie Cahalan, who directed the initial student survey of 

former participants, and Marisol Cunnington from PPSS; Frances Bergeron from the office of 

Federal TRIO Programs; an anonymous peer reviewer; and Mark Dynarski, Paul Decker, and 

John Deke from Mathematica Policy Research, inc., for providing suggestions that helped us to 

improve the report. 

The authors are also grateful to the individuals who contributed to the data collection, the 

data analysis, and the production of the report. Amy Johnson led the effort to obtain rosters of 

participants from Upward Bound Math and Science Centers, and Susan Mitchell directed the 

survey of those centers. Melissa Thomas contributed to the follow-up survey design, oversaw 

student survey operations, and prepared the survey data file. Stephanie Naber managed the dayto-day operations of the follow-up student survey; she also led our efforts to locate sample 

members for the survey, collect student transcripts, and enter the transcript data. Giuseppe 

Scaturro and Mary Grider processed the transcript data. Grider and Heather Berry matched the 

colleges that students reported attending to the integrated Postsecondary Education Data System 

(IPEDS). Miki Satake computed the impact estimates. Carol Soble edited the report and Micki 

Morris provided production support. 

ix

EXECUTIVE SUMMARY 

For many years, policymakers have been concerned by the relatively low levels of academic 

achievement by economically disadvantaged K-12 students in math and science, by the 

underrepresentation of disadvantaged college students in math and science majors, and by the 

underrepresentation of people from disadvantaged groups in math and science careers. While 

racial gaps in math and science test scores narrowed somewhat in the 1970s and 1980s, 

substantial gaps persisted through the 1990s to the present. 

To help address these disparities, the U.S. Department of Education (ED) established a math 

and science initiative in 1990 within Upward Bound, a federal grant program designed to provide 

disadvantaged high school students with skills and experiences that will prepare them for college 

success. The initiative, referred to as Upward Bound Math-Science (UBMS), awards grants to 

institutions—largely colleges and universities—to operate UBMS projects. These projects were 

designed to differ from “regular” Upward Bound projects in several respects. To ensure that 

participants receive an intensive math and science precollege experience, UBMS projects 

provide instruction that includes hands-on experience in laboratories, computer facilities, and at 

field sites. Opportunities are also provided to learn from mathematicians and scientists 

employed at the host institution or engaged in research or applied science in other institutions in 

the community. A six-week summer program providing intensive instruction in laboratory 

science and mathematics through precalculus is also offered. 

Initially, ED funded 30 UBMS projects. by FY 2004, there were 127 UBMS projects 

serving 6,845 students at a total cost of $32.8 million. Therefore, the annual cost per student—

approximately $4,800—is comparable in cost to regular Upward Bound but much more 

expensive than other federally funded precollege programs. More than 80 percent of UBMS 

projects are hosted by four-year colleges and universities; most of the rest are hosted by two-year 

colleges (Curtin and Cahalan 2004). 

Participants in UBMS must meet the same eligibility requirements as regular Upward Bound 

students: students must (1) belong to families classified as low-income (taxable income of no 

greater than 150 percent of the poverty line), or (2) be a potential first-generation college student 

(neither parent has a bachelor’s degree). Some students who participate in UBMS summer 

programs are referred from regular Upward Bound programs and then return to those programs 

during the academic year. However, as would be expected, UBMS projects are more likely to 

consider students’ interests in math and science when reviewing applications than are most 

regular Upward Bound projects (Moore 1997b). While 25 percent of participants are white, most 

program participants are from underrepresented minority groups: about 60 percent of 

participants are African American or Hispanic (Curtin and Cahalan 2004). 

EVALUATION OF UPWARD BOUND MATH-SCIENCE 

Since 1991, Mathematica Policy Research, Inc., (MPR) has been conducting the National 

Evaluation of Upward Bound for ED. The centerpiece of this evaluation has been a random 

assignment evaluation of regular Upward Bound. In 1997, ED added a new component to the 

evaluation that is focused on UBMS. In 1998, MPR selected a random sample of the students 
x

who participated in UBMS between 1993 and 1995 at projects that were still operating at that 

time. This report constitutes the first of two evaluation reports on UBMS, and it is based on 

participant surveys and student transcripts collected for this sample between 1998 and 1999 and 

again between 2001 and 2002. The second report is scheduled for completion in 2006 and will 

be based on data collected between 2003 and 2005. 

The evaluation of UBMS has two components: a descriptive analysis and an impact 

analysis. The descriptive analysis relies primarily on a survey of project directors to describe the 

resources available to UBMS projects; the types of institutions that host them; the credentials and 

demographic characteristics of project staff; recruitment, eligibility, and enrollment of students; 

student characteristics; and program offerings. The impact analysis is designed to measure the 

effects of UBMS on (1) performance in high school, especially in math and science courses; (2) 

postsecondary attendance, persistence and completion; and (3) the likelihood of completing a 

postsecondary degree in mathematics or a scientific field. 

The impact analysis is based on a comparison of UBMS participants with a sample of 

students that (1) applied to enroll in regular Upward Bound programs in the early 1990s, (2) 

never participated in UBMS and (3) have been tracked by MPR as part of the national 

evaluation. This comparison group was selected to ensure that it had similar characteristics to 

the sample of UBMS participants, and we controlled statistically for the small remaining 

differences in these characteristics between UBMS participants and the comparison group. 

If UBMS participants are more interested or skilled in math and science than the students in 

the comparison group, the estimated effects of the program may be subject to “selection bias” 

and may overstate the true effects of participating in UBMS. However, the comparison group 

we selected was probably the best available short of a randomized control group because the 

students in the comparison group exhibited the motivation to pursue Upward Bound services, 

and our analysis shows that the participant and comparison groups are similar in other ways as 

well. In addition, we implemented a data collection and analysis plan designed to minimize 

selection bias (see Chapter III for more details). While a control group from a randomized 

experiment would have prevented selection bias, the comparison group we selected greatly 

reduced the cost of the evaluation because we were already collecting data for this group as part 

of the national evaluation. 

Note that the descriptive findings and impact estimates presented in this report describe the 

operations and effects of the Upward Bound Math-Science Program as it operated in the mid1990s. At that time, it was a relatively new program, and some changes have occurred in how 

UBMS projects operate. In Chapter II, we mention some of these changes as they are reflected 

in information provided to us by UBMS project directors in a survey of grantees. It is certainly 

possible that some of the changes in the program since the mid-1990s have influenced the 

effectiveness of UBMS projects, and the evaluation does not attempt to measure any changes in 

effectiveness since that time. In this report, we measure the effects of the program on people 

who participated between 1993 and 1995 and describe the operations of the program at that time. 
xi

REPORT FINDINGS 

From our descriptive analysis, we found that UBMS projects: 

• Provide a large quantity of academic instruction. in the summer, the average 

UBMS project provided a total of 240 hours of academic instruction, and 

participation in the program is roughly full-time for a six-week period. 

• Are most active during the summers. UBMS projects typically provide services, 

such as tutoring or study sessions, during the school year, but they provide most of 

their services during summer residential programs at the colleges or universities 

hosting the program. 

• Provide academic enrichment in math and science subjects. Many UBMS projects 

offer courses in algebra II, geometry, precalculus, biology, chemistry, physics and 

computer software; in contrast, few offer courses in Social Studies (though many 

offer English courses in addition to their math and science offerings). At most 

projects, the course work is designed to provide academic enrichment instead of 

academic remediation. 

• Provide instruction through a combination of single-subject courses and 

interdisciplinary instruction. While other instructional techniques were used, three 

out of four projects provided instruction primarily through single-subject academic 

courses or the combination of these courses with interdisciplinary instruction. 

Given the academic services provided by UBMS, it is natural to ask whether participating in 

UBMS affects the educational outcomes of the students that participate. From our impact 

analysis, we found that UBMS: 

• Improved high school grades in math and science and overall. UBMS had a 

positive effect on high school grades, increasing the average GPA in math courses 

from 2.7 to 2.8, the average GPA in science courses from 2.7 to 2.9 and the average 

GPA overall. 

• Increased the likelihood of taking chemistry and physics in high school. UBMS 

increased the likelihood that participants took upper-level science courses in high 

school, raising the percentage of students taking chemistry from 78 percent to 88 

percent and raising the percentage of students taking physics from 43 percent to 58 

percent. in contrast, UBMS did not affect coursetaking in advanced math subjects 

(see Chapter III, Exhibit III.3). 

• Increased the likelihood of enrolling in more selective four-year institutions. 

UBMS increased the percentage of students that attended four-year colleges and 

universities from 71 percent to 82 percent. The increase in four-year attendance is 

particularly pronounced for more selective schools (those rated as “most selective”, 

“highly selective” or “very selective” by the Barron’s Guide): UBMS increased the 

percentage of students that attended more selective four-year colleges from 23 

percent to 33 percent (see Chapter III, Exhibit III.4). 
xii

• Increased the likelihood of majoring in math and science. UBMS affected 

students’ choice of major, increasing the percentage majoring (or planning to major) 

in math or science from 23 percent to 33 percent and decreasing the percentage 

majoring in a field outside of math or science and the social sciences from 51 to 42 

percent. UBMS also seems to increase the percentage of participants majoring in the 

social sciences (see Chapter III, Exhibit III.6). 

• Increased the likelihood of completing a four-year degree in math and science. 

UBMS increased the percentage of students that earned a bachelor’s degree in a math 

and science field from 6 percent to 12 percent and decreased the percentage that 

earned a bachelor’s degree outside of math, science, and the social sciences from 20 

to 14 percent (see Chapter III, Exhibit III.6). Because 47 percent of participants in 

our sample were still in college when we interviewed them in 2002, findings related 

to degree completion should be treated as preliminary, and a final assessment will be 

presented in a subsequent report. 

In addition, we computed separate impact estimates for subgroups defined by sex, race and 

ethnicity, and prior participation in regular Upward Bound. For some outcomes, we found 

differences in subgroup impacts that were statistically significant. For example, the effect of 

UBMS on four-year college attendance was larger for women than for men. However, the 

number of significant differences between subgroups was relatively small, and there was no 

obvious pattern to the findings suggesting that particular groups benefited more from UBMS 

than other groups. Therefore, it is not clear whether the significant subgroup differences are due 

to chance or to systematic differences in the effects of UBMS on different groups of participants. 

To summarize the report’s findings, UBMS provides intensive academic instruction in math 

and science, and our impact estimates suggest that it improves several student outcomes in high 

school and college. In addition, and consistent with the objectives of the program, preliminary 

estimates suggest that UBMS participation increases the odds of majoring in math or science. In 

the next report, we will reexamine the effects on college completion, examine the effects on 

labor market outcomes, such as employment in the sciences, and weigh the benefits of the 

program against the costs. 

It is tempting to compare the estimated impacts of UBMS to the estimated impacts of 

regular Upward Bound presented in earlier reports. However, it is important to recognize that 

the two studies used different methods: while the evaluation of regular Upward Bound is based 

on an experimental design, the “gold standard” in evaluation research, the evaluation of UBMS 

is based on nonexperimental methods that may suffer from selection bias, as described earlier. If 

the estimated effects of UBMS are inflated due to selection bias, then the impression based on 

our findings that UBMS is more effective than regular Upward Bound might be attributable to 

differences in the methods used to estimate the impacts instead of differences in the effectiveness 

of the two programs. 
1

I. INTRODUCTION 

For many years, policymakers have been concerned by the relatively low levels of academic 

achievement by economically disadvantaged K-12 students in math and science, by the 

underrepresentation of disadvantaged college students in math and science majors, and by the 

underrepresentation of people from disadvantaged groups in math and science careers. National 

statistics show that while the gaps between minorities’1

 and whites’ math and science test scores 

narrowed somewhat in the 1970s and 1980s, gaps in test scores and other educational outcomes 

persisted through the 1990s to the present. 

• Disadvantaged students take fewer math and science courses in high school. in 

the 1991–92 school year, 57 percent of seniors in the lowest socioeconomic status 

(SES) quartile took a math course, compared with 75 percent of seniors from the 

highest SES quartile; 37 percent of seniors from the lowest SES quartile took a 

science course, compared with 61 percent of seniors from the highest SES quartile 

(U.S. Department of Education 1996b). in 1994, only 58 percent of black high 

school graduates had completed geometry while in high school, compared with 73 

percent of white high school graduates. in the same year, only 13 percent of black 

and Hispanic graduates had completed the common triad of science courses—

biology, chemistry, and physics—compared with 23 percent of white graduates (U.S. 

Department of Education 1996a). 

• Minority college students are less likely to take math and science courses or earn a 

degree in math or science. Ten percent of black college students and 14 percent of 

Hispanics received credit for calculus or advanced math courses in the late 1980s, 

compared with 22 percent of whites. Sixteen percent of blacks and 21 percent of 

Hispanic college students earned course credits in chemistry, compared with 27 

percent of whites, and 8 percent of blacks and 11 percent of Hispanics earned college 

credit for physics, compared with 18 percent of white students (U.S. Department of 

Education 1994). Because minorities earned fewer college credits in math and 

science than whites, it is not surprising that they were less likely to earn degrees in 

those subjects. Black students earned 7 percent of all bachelor’s degrees in 1995-96, 

but just 7 percent of all bachelor’s degrees in math and science fields. in the same 

year, Hispanic students earned 5 percent of all bachelor’s degrees, but just 4 percent 

of all bachelor’s degrees in math and science (U.S. Department of Education 1999).2

 

 1 Ideally, socioeconomic measures such as income would be used to define groups, rather than race or 

ethnicity. For most education outcomes of interest, however, data are not presented on different income groups. 

Because racial and ethnic minorities are disproportionately lower-income (U.S. Census Bureau 2001:40), data based 

on race and ethnicity offer a reasonable, albeit imperfect, estimate of economically disadvantaged students’ 

educational experiences. 

2

 The following subjects were classified as math or science: biological sciences and life sciences, computer 

and information sciences, engineering, engineering-related technologies, mathematics, and physical sciences and 

science technologies. 
2

• Minorities are less likely than whites to enter careers in math and science. Among 

people who were working in a scientific field in 1995 and had obtained their college 

degree in the previous five years, only 6 percent were black (National Science 

Foundation 1995). However, in 1990, around the time those individuals would have 

been in college, blacks accounted for 14 percent of the U.S. population aged 18-24 

years old (Census Bureau 1990a, Census Bureau 1990b). 

A. UPWARD BOUND MATH-SCIENCE PROGRAM 

To help address these disparities, the U.S. Department of Education (ED) in 1990 

established the Upward Bound Math-Science Program (UBMS) within Upward Bound, a federal 

grant program designed to provide disadvantaged high school students with skills and 

experiences that will prepare them for college success. UBMS was designed to differ from 

“regular” Upward Bound in a few key respects. To ensure that participants receive an intensive 

math and science precollege experience, ED requires UBMS projects to provide instruction that 

includes hands-on experience in laboratories, computer facilities, and at field sites. Also 

provided are the following: opportunities to learn from mathematicians and scientists employed 

at the host institution or engaged in research or applied science in other institutions in the 

community;3

 involvement with tutors and counselors who are graduate and undergraduate math 

and science majors; and a six-week summer program consisting of daily course work and 

activities, instruction in laboratory science and mathematics through precalculus (in addition to 

foreign language, composition and literature, which are also required offerings at regular Upward 

Bound projects). 

Initially, ED funded 30 UBMS projects. By FY 2004, there were 127 UBMS projects 

serving 6,845 students at a total cost of $32.8 million. Therefore, the annual cost per student—

approximately $4,800—is comparable in cost to regular Upward Bound but much more 

expensive than other federally funded precollege programs. More than 80 percent of UBMS 

projects are hosted by four-year colleges and universities; most of the rest are hosted by two-year 

colleges (Curtin and Cahalan 2004). 

UBMS participants must meet the same eligibility requirements as regular Upward Bound 

participants: they must (1) come from families that are classified as low-income (taxable income 

not over 150 percent of the poverty line), or (2) be a potential first-generation college student 

(neither parent has a bachelor’s degree). Some students who participate in UBMS are referred 

from regular Upward Bound programs and then return to those programs during the academic 

year. However, as would be expected, UBMS projects are more likely to consider students’ 

interests in math and science when reviewing applications than are most regular Upward Bound 

projects (Moore 1997b). While 25 percent of participants are white, most program participants 

are from underrepresented minority groups: about 60 percent of participants are African 

American or Hispanic (Curtin and Cahalan 2004). 

 3

 This requirement may have stemmed from concern that too much math and science instruction in high school 

is provided by teachers teaching out of their own field. 
3

Despite coming from low-income families, the evidence suggests that on average, UBMS 

serves students who do well in high school and attend college at higher rates than the average 

low-income student. Data reported by Upward Bound projects suggest that prior to participating 

in Upward Bound, UBMS participants earned higher grades on average than regular Upward 

Bound participants (Curtin and Cahalan 2004). In addition, the national evaluation has shown 

that regular Upward Bound participants would have attended college at much higher rates than 

the average low-income student even if they had not participated in Upward Bound (Myers et al. 

2004).4 Therefore, the evidence strongly suggests that UBMS serves high school students who 

are much more likely to attend college than the average low-income student. 

B. EVALUATION OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 

The legislation establishing Upward Bound authorizes ED to sponsor studies of it, including 

examinations of program effectiveness. In 1991, ED awarded a contract to Mathematica to 

conduct the National Evaluation of Upward Bound. This evaluation has several components, but 

its signature feature is an experiment to measure the effects of participating in regular Upward 

Bound. We selected a random sample of Upward Bound projects (excluding UBMS projects); 

for each of these projects, we randomly assigned eligible applicants to a treatment group, which 

was offered the chance to participate in the program, or a control group, which was not. The 

evaluation is ongoing, and it was one of the first to use experimental methods to measure the 

effects of a federally funded education program. 

This report presents the results of an evaluation of the Upward Bound Math-Science 

Program. In 1997, Mathematica completed two reports on UBMS. One provided a descriptive 

analysis of the program based primarily on site visits to a representative sample of 14 UBMS 

projects (Moore 1997a). The other provided an assessment of the feasibility of conducting a 

rigorous evaluation of the effects of UBMS on student outcomes (Myers 1997). When ED 

awarded a contract to Mathematica in 1997 to extend its evaluation of the effects of regular 

Upward Bound, it also specified an evaluation of the effects of UBMS. This evaluation consists 

of two components: a descriptive analysis and an impact analysis. The descriptive analysis 

relies primarily on a survey of UBMS project directors conducted in the spring of 1998. The 

analysis is designed to describe the resources available to UBMS projects; the types of 

institutions that host them; the credentials and demographic characteristics of project staff; 

recruitment, eligibility, and enrollment of students; student characteristics; and a description of 

the program, including its goals, academic orientation, instructional methods and the intensity 

and quantity of the services provided. 

The UBMS impact study is designed to measure the effects of participating in UBMS on 

college enrollment, choice of major, and other outcomes for students who participated during the 

 4

 While the last report from the national evaluation shows that about 70 percent of regular Upward Bound 

participants would have attended a postsecondary institution even if they had not participated in Upward Bound 

(Myers et al. 2004), only 53 percent of 1992 high school graduates from the lowest SES quartile attended a 

postsecondary institution by 1994 (U.S. Department of Education 1997). 
4

summer of 1993, 1994, or 1995.5 Conceptually, the study contrasts how participants fared with 

how they would have fared if they had not participated in UBMS. We compared UBMS 

participants with eligible applicants to the regular Upward Bound projects participating in the 

national evaluation. From this pool, we systematically selected a matched comparison group of 

students who were as similar as possible to UBMS participants in terms of characteristics and 

experiences that could potentially predict later outcomes. These characteristics included 

demographics—such as sex, race, and ethnicity—and prior academic achievement such as grade 

point average and math and science courses taken in 9th grade. The key difference was that the 

matched comparison students did not participate in UBMS. 

The selection of matched comparison students also took into account experiences in other 

precollege programs, in particular regular Upward Bound. Because regular Upward Bound is an 

intensive program that can influence high school achievement and postsecondary outcomes 

(Myers and Schirm 1999; Myers et al. 2003), it is important to account for exposure to regular 

Upward Bound when estimating how UBMS participants would have fared if they had not 

participated in UBMS. For UBMS participants who had previously participated in a regular 

Upward Bound program—perhaps during the academic year—we selected comparison students 

who had also participated in the regular Upward Bound.6

 For UBMS participants who had not 

participated in regular Upward Bound, we selected comparison students who did not participate 

in regular Upward Bound. 7

Several data sources play a key role in the impact analysis. Baseline characteristics were 

collected for comparison group members through the baseline survey for the evaluation of 

regular Upward Bound; baseline information on many of the same characteristics was collected 

for UBMS participants through a follow-up survey conducted in 1999.8 This follow-up survey 

was used to collect information about educational outcomes for UBMS participants, and a 

similar survey was used to collect analogous information for comparison students. Finally, 

secondary and postsecondary transcripts were collected for both types of students to assess 

academic achievement. 

C. OVERVIEW OF THIS REPORT 

The remainder of this report is organized as follows. Chapter II describes the operation of 

the UBMS program. Chapter III presents findings from the impact analysis. 

 5 Because the sample was not selected until 1998, we restricted the sample to participants at UBMS projects 

that were still operating that year: obtaining lists of participants from programs that were no longer operating in 

1998 would have been nearly impossible. 

6

 These comparison students were selected from the treatment group for the evaluation of regular Upward 

Bound. For a more thorough discussion of how the treatment group was selected, see Myers et al. (2004), Appendix 

A. 

7

 These comparison students were selected from the control group for the evaluation of regular Upward Bound. 

For a more thorough discussion of how the control group was selected, see Myers et al. (2004), Appendix A. 

8

 While the 1999 survey was conducted four to six years after our sample had participated in the program, most 

of the baseline information collected—including sex, race, and ethnicity—is time-invariant. 
5

II. THE OPERATION OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 

To interpret information on the impacts of UBMS, it is necessary to understand what the 

program entails. This chapter describes key features of the operations of UBMS projects, 

including the characteristics of host institutions and staff, projects’ recruitment practices and 

enrollment levels, participants’ characteristics and projects’ goals and services. For context, this 

chapter presents comparable information on the operations of regular Upward Bound when 

possible. 

The primary data source for this chapter is a survey of UBMS projects conducted in the 

spring of 1998. The survey sample consisted of all 81 projects operating at the time, and 74 of 

the 81 projects responded to the survey.9,10 the survey requested information about program 

operations in two separate years—(1) 1994 (in the middle of the period over which our sample 

was participating in UBMS) and (2) 1998 (the year prior to the survey)—but some questions 

were specific to 1998. When possible, we focus our analysis of program operations on 1994 to 

facilitate comparisons with regular Upward Bound projects operating in 1993, as reported in 

Fasciano and Jacobson (1997) and to describe the programs that served the same cohorts of 

participants for whom we measured the impacts of the program (see Chapter III).11 To augment 

the information provided by the survey of UBMS projects, we also use information from case 

studies and annual performance reports (Moore 1997b). 

The findings in this chapter indicate that UBMS projects provide intensive academic 

enrichment to disadvantaged high school students in math and science using staff with strong 

academic credentials in those subjects. Some of the features that make UBMS projects 

distinctive, even from regular Upward Bound projects, are: (1) high levels of annual funding per 

student and low student-teacher ratios, (2) recruiting strategies that attract students from wide 

geographic areas, (3) service provision that is heavily concentrated in residential programs 

during the summer, (4) course offerings that focus on math and science relative to other subjects, 

(5) academic preparation over nonacademic college preparatory activities, and (6) academic 

enrichment over remediation. The remainder of the chapter provides a description of UBMS and 

an assessment of its distinctive features. 

 9

 We did not adjust (weight) for survey nonresponse, reasoning that the number of nonrespondents was low 

enough to eliminate any serious concerns about data representativeness. Also, rarely did more than three UBMS 

projects fail to respond to any particular item on the questionnaire. 

10 We excluded the one project that reported serving only veterans in 1998. Note that veterans’ projects were 

also excluded from the survey of regular Upward Bound grantees, so the comparisons that are made in this chapter 

between the two types of Upward Bound programs are based on Upward Bound projects that did not exclusively 

serve veterans. 

11 Unless noted otherwise, the results for 1998 were generally similar to those for 1994. 
6

A. PROJECT HOSTS AND STAFF 

The impacts of UBMS projects on student outcomes may depend on the types of institutions 

that host them and the people they hire to serve as instructors and other staff. In this section, we 

describe the types of institutions that host UBMS projects and the staff that provide services to 

program participants. 

1. Host institutions: Two- and Four-Year Colleges and Universities 

The types of institutions that host a UBMS project may influence where students attend 

college. Most Upward Bound programs are hosted by either two- or four-year postsecondary 

institutions. Evidence from the national evaluation of regular Upward Bound suggests that 

participation at projects hosted by four-year colleges raises the probability of attending a fouryear college, and participation at projects hosted by two-year colleges raises the probability of 

attending a two-year college (Myers et al. 2002b). Therefore, the types of institutions that host 

UBMS projects may influence the types of postsecondary institutions that program participants 

subsequently attend. 

Nearly nine out of ten UBMS projects operating in the mid–1990s were hosted by four-year 

colleges, a substantially higher proportion than among regular Upward Bound projects (see 

Exhibit II.1). Four-year colleges may find it easier than other potential host institutions to meet 

some of ED’s guidelines for UBMS, including offering hands-on experience in laboratories and 

computer facilities, opportunities to learn from mathematicians and scientists engaged in 

research or applied science and involvement with tutors and counselors who are graduate and 

undergraduate students in math and science. 

2. Summer Program Staff and Project Director 

UBMS projects are directed by highly educated individuals and staffed by people with 

strong credentials in math and science. These staff have responsibility for a relatively small 

number of students, which may provide opportunities for individual instruction. At the typical 

project, the project director and staff can provide same-race role models for many of the students 

they serve. The sections below provide more detail on our findings concerning staff size and 

composition by job title, staff credentials, and the racial composition of project staff. 

a. Staff Size and Composition by Job Title 

In 1994, UBMS projects had an average of 24 staff members, comprising roughly eight 

instructors, five resident counselors, four mentors, three tutors, two administrators, one academic 

or guidance counselor and one clerical staff member.12 Overall, the average student-staff ratio in 

summer 1998 was 2:1, with a range from about 1:1 to 5:1. 

 12 By 1998 the average MSC had almost 26 staff, including 9 instructors. 
7

Exhibit II.1 

Types of Institutions that Hosted UBMS Projects, 1995 

Type of institution Upward Bound Math-Science Regular Upward Bound 

Four-year college or university 88% 68% 

Two-year college 11 28 

Other institution 2 4 

NOTE: Percentages may not sum to 100 percent due to rounding. 

SOURCE: Moore 1997b, Exhibit II.1, p. 15. 

These findings, combined with findings from Moore (1997b), suggest that student-staff 

ratios are typically lower in UBMS projects than in regular Upward Bound projects. The survey 

of grantees did not collect information on the number of full-time-equivalent (FTE) staff, but the 

information available suggests that UBMS projects typically maintain student-staff ratios that are 

substantially lower than in regular Upward Bound. Moore (1997b) found that 14 randomly 

selected UBMS projects visited in summer 1996 had an average of 2.6 students per FTE staff, 

including administrators, and 8.2 students per FTE instructional staff (Moore 1997b, Exhibit 

II.6). In contrast, tabulations from the data used by Fasciano and Jacobson (1997) indicate that 

in summer 1992, regular Upward Bound projects had more students per staff member—5.1 

students per FTE staff and 13.6 students per FTE instructional staff (Moore 1997b, Exhibit II.6). 

b. Credentials 

At the average UBMS project in 1998, most staff were highly educated and had educational 

backgrounds in math and science. About one-quarter had attended some college but not obtained 

a degree, another quarter had obtained a bachelor’s or associate’s degree (mostly bachelor’s 

degrees) and the rest had done graduate work or obtained a graduate degree. Most staff members 

without undergraduate degrees were undergraduate students who served as UBMS mentors, 

tutors and resident counselors while working toward bachelor’s degrees in math, science, or 

education. Approximately two out of five staff members had their highest degree in science or 

the social sciences (31 percent) or math (10 percent); additionally, most staff members without a 

degree were working toward a bachelor’s degree in math, science, or education. 

Most instructors at the average UBMS project had experience teaching math or science. 

During the school year, most instructors were either high school teachers (41 percent) or 

postsecondary teachers (31 percent); one-fifth were graduate students (14 percent) or 

undergraduates (6 percent). Moreover, at the typical project, two-thirds of the high school 

teachers and three-quarters of the postsecondary teachers taught in a math or science field. 

The professional and educational backgrounds of UBMS project directors provide insight on 

their credentials to direct projects. One-fifth of the directors were faculty members at the host 

institution or another college, roughly the same percentage as in regular Upward Bound. In 

1998, two-thirds of the directors held a master’s degree and one-fifth held a doctorate. About 

half had their highest degree in education, and less than one-fifth had their highest degree in 
8

engineering, mathematics, or physical sciences.13 Although UBMS project directors were less 

likely than program staff as a whole to have a background in math or science, this may not be 

surprising since subject area expertise is probably less important for administrators than for other 

staff. Compared with regular Upward Bound project directors, UBMS project directors were 

more highly educated. They were, for example, twice as likely to have a doctorate. 

c. Race and Ethnicity 

At about 9 out of 10 UBMS projects operating in 1998, one racial or ethnic group accounted 

for a majority of the staff (see Exhibit II.2). In many cases this pattern may have reflected a 

conscious strategy, also used in regular Upward Bound, to provide minority students with samerace role models. For example, at 21 UBMS projects, a majority of the staff members were 

black; at 18 of these, a majority of the students were also black. 

The racial and ethnic profile of UBMS project directors was similar to that of UBMS project 

staff (see Exhibit II.3). Both staff and project directors were nearly evenly split between white 

and nonwhite; project directors were slightly more likely to be black and less likely to be Asian 

than other staff. The race and ethnicity of the UBMS project director often matched that of the 

predominant student racial and ethnic group: more than three-fourths of the minority directors 

headed programs where students from the same group constituted a plurality of participants. 

B. ELIGIBILITY, RECRUITMENT AND ENROLLMENT, AND STUDENT 

CHARACTERISTICS 

To shed light on the types of students that participate in UBMS, we examine the eligibility 

criteria that students must meet and the recruiting strategies that UBMS projects use to attract 

students. We also examine the characteristics of participants as reported by project directors. 

1. Eligibility 

UBMS projects have to meet the same federal rules as regular Upward Bound projects 

concerning the composition of participants. At each project, at least two-thirds of the 

participants must be both low-income and potential first-generation college students; the 

remaining students must meet either of these two criteria. At the average project in summer 

1998, about 77 percent of students met both of these eligibility criteria, about 14 percent were 

first-generation only and about 9 percent were low-income only, very similar to the distribution 

in regular Upward Bound during 1992-93. In addition, UBMS and regular Upward Bound 

projects are only allowed to serve students who have completed 8th grade. 

 13 An additional 12 percent had their highest degree in the social sciences. 
9

Exhibit II.2 

Predominant Racial and Ethnic Group for UBMS Staff, Summer 1998 

No majority racial 

and ethnic group

12%

Majority white

54%

Majority Hispanic 

or Latino

3%

Majority black 

30%

Majority Asian

1%

NOTE: Percentages may not sum to 100 percent due to rounding. The 12 percent of projects with no 

majority racial and ethnic group included 6 percent of projects with a plurality of blacks and 4 

percent with a plurality of whites; the remainder had no plurality. 

SOURCE: 1998 survey of UBMS projects.

Exhibit II.3 

Racial and Ethnic Distribution for UBMS Staff and Project Directors, Summer 1998 

Race and Ethnicity Staff Director 

White 49% 49% 

Black 33 39 

Hispanic or Latino 7 7 

Asian 4 1 

American indian or Alaskan Native 2 4 

NOTE: Percentages may not sum to 100 percent because (1) people may fall into multiple categories, (2) the 

Pacific Islander category was excluded from the exhibit because only nine staff members nationwide fell 

into this category and (3) some staff members may not have been classified by race or ethnicity. 

SOURCE: 1998 survey of UBMS projects.
10

Most UBMS projects also adopt additional student eligibility criteria for enrollment in the 

program—for example, requirements about grade level, school course work, or 

recommendations. In 1994, over three-fourths of projects required students to have finished 9th 

grade; a few projects required 10th grade completion. In addition, nearly all UBMS projects 

required a teacher recommendation and completion of at least one high school course in math or 

science, and applicants enrolled in regular Upward Bound commonly needed a recommendation 

from the director. Finally, about 30 percent of UBMS projects prohibited students from 

returning from the previous summer’s program, and almost half prohibited students from 

returning unless they met certain criteria.14

2. Recruitment and Enrollment 

To find a pool of potentially eligible applicants, UBMS projects focused mainly on other 

precollege programs or secondary schools (see Exhibit II.4). Among UBMS projects operating 

in summer 1994, nearly all recruited from regular Upward Bound projects, while substantial 

majorities also recruited from Talent Search projects and from middle or high schools directly. 

However, it is important to note that while almost all UBMS projects recruited from regular 

Upward Bound projects in 1994, data from the evaluation suggest that fewer than one in five 

UBMS participants had previously participated in regular Upward Bound. 

Historically, UBMS projects have cast a wide net in recruiting students beyond the local 

areas of the host institutions. In 1994, only about one in ten UBMS projects recruited 

exclusively from a specific and typically local set of feeder schools or Upward Bound projects; 

the rest recruited from state-wide or regional lists of schools and programs (see Exhibit II.5). By 

1998, however, the percentage of UBMS projects that recruited exclusively from a specific set of 

feeder schools or Upward Bound projects had tripled. Therefore, it appears that over time, more 

UBMS projects are taking local recruitment strategies like regular Upward Bound projects. 

By design, UBMS projects are smaller than regular Upward Bound projects. Through 

recruitment, UBMS projects received an average of 108 applications for the summer of 1994, 

ranging from a low of 50 to a high of 300, and they enrolled between 40 and 53 students. In 

contrast, regular Upward Bound programs enrolled an average of about 75 students in the mid1990s (Moore 1997b). 

 14 These practices changed dramatically by 1998, when only 7 percent of UBMS projects prohibited all 

students from returning from the prior summer, and 71 percent prohibited students from returning unless they met 

certain criteria. The questionnaire did not address the specific types of criteria that projects imposed. 
11

Exhibit II.4 

Targets for Recruiting by UBMS Projects, 1994

SOURCE: 1998 survey of UBMS projects.

Exhibit II.5 

Geographic Scope of Recruiting by UBMS Projects, 1994

NOTE: Percentages may not sum to 100 percent due to rounding. 

SOURCE: 1998 survey of UBMS projects.

43%

82%

88%

95%

0% 20% 40% 60% 80% 100%

 Other programs

 Middle or high

schools

 Talent Search

projects

 Regular Upward

Bound projects

Percent of Upward Bound Math-Science Projects

Statewide or 

Regional

45%

Local Only

Both 11%

45%
12

3. Student Characteristics 

In our 1998 survey of UBMS projects, project directors provided information on the 

distribution of students participating in their projects by sex, race, grade level, and place of 

residence: 15

• Sex. At the average UBMS project, like the average regular Upward Bound project, 

60 percent of students were female.16 However, this varied considerably across 

UBMS projects from a low of 25 percent to a high of 78 percent. 

• Race. On average, UBMS projects served an ethnically diverse group of students: 

42 percent black, 27 percent white, 15 percent Hispanic, 8 percent Asian, 5 percent 

American indian and 1 percent Native Hawaiian or other Pacific Islander. However, 

most UBMS projects serve participants where one racial and ethnic group 

constituted a majority (see Exhibit II.6).17 Furthermore, some UBMS projects 

exclusively served students from a single racial or ethnic group. For example, six 

UBMS projects reported that all of its participating students were black. 

• Grade level. The eligibility guidelines discussed above, along with other factors, 

can affect the distribution of students across different grade levels. At the average 

project, 29 percent of participants were entering 12th grade, 37 percent were 

entering 11th grade, 27 percent were entering 10th grade and 6 percent were 

entering 9th grade. These exhibits suggest that on average, UBMS projects serve 

students who are slightly closer to graduation than is the case at regular Upward 

Bound projects.18 However, there was substantial variation in the grade level 

distribution of participants across projects in 1998. for example, one UBMS project 

reported that all of its participants were entering 12th grade, while four reported that 

none were rising seniors.19 

• Place of residence. Given that most projects recruited across the state or region, it 

is not surprising that many UBMS participants came from outside the grantee’s local 

city or town. At the average project, only about 25 percent of the students were 

locals. As we would expect, projects that recruited only from a set of local schools 

or regular Upward Bound projects served considerably higher percentages of 

students from the local area than other UBMS projects. 

 15 Most of these questions were focused on 1998 participants, so the analysis in this section is focused on 1998 

instead of 1994. 

16 Data based on all UBMS projects operating in 1998 and all regular Upward Bound projects operating in 

1992. 

17 In comparison, at 87 percent of regular Upward Bound projects operating in 1992–93, one racial and ethnic 

group accounted for a majority of participants. 

18 In regular Upward Bound during the summer of 1992, 20 percent of participants were entering 12th grade, 

32 percent were entering 11th grade, 31 percent were entering 10th grade, and 16 percent were entering 9th grade. 

19 In addition, the proportion of rising juniors ranged from 0 to 72 percent, and the proportion of rising 

sophomores ranged from 0 to 60 percent. 
13

Exhibit II.6 

Predominant Racial and Ethnic Group for Participants at UBMS Projects, 

Summer 1998 

Majority white

15%

Majority

Hispanic or Latino

4%

Majority black

42%

Majority

American Indian or 

Alaskan Native

1%

No majority

racial and ethnic 

group

37%

NOTE: Percentages may not sum to 100 percent due to rounding. The 37 percent of UBMS projects 

with no majority racial and ethnic group of students included 17 percent with a plurality of white 

students, 11 percent with a plurality of Hispanics, 6 percent with a plurality of blacks, 3 percent 

with a plurality of American indians and 1 percent with a plurality of Asians. 

SOURCE: 1998 survey of UBMS projects. 

Moore (1997b) described UBMS participants as a more select group than regular Upward 

Bound participants based on having earned somewhat higher grades and having greater interest 

in math and science (pp. 23, 26) prior to participating in Upward Bound. Discussions with 

UBMS and regular Upward Bound staff revealed that UBMS participants were typically 

considered “more serious about school” than regular Upward Bound participants (Moore 1997b, 

p. 26). 

C. PROGRAM DESCRIPTION 

In a college-like setting, UBMS projects offer academic enrichment in math and science to 

improve student achievement in those subjects and expose students to math and science careers. 

In this section, we describe the following features of UBMS projects in more detail: the setting 

in which these projects provide services; the goals, academic orientation, academic offerings, 

and instructional approaches of these projects; and the intensity and quantity of services the 

UBMS projects provide. 
14

1. Setting 

As described earlier, UBMS projects are typically hosted by two- and four-year 

postsecondary institutions (see Section A.1). Most UBMS projects are hosted by four-year 

colleges and universities, and most of these institutions have dormitories to house their students. 

These dormitories are often available in the summers to house participants of summer programs 

hosted by these institutions. 

UBMS projects typically exposed participants to a college setting during the summer 

program by housing them in the college dormitories. Virtually all the UBMS projects we 

surveyed (100 percent in 1994, 97 percent in 1998) offered a residential component to their 

summer programs, compared with 87 percent of regular Upward Bound programs in 1992 

(Moore 1997a). At almost all UBMS projects, students lived in the dormitories for the entire 

summer program, which lasted about six weeks on average.20 Therefore, for six weeks, 

participants lived on campus like many college students do during the academic year. 

2. Goals of the Program 

As mentioned in Chapter I, the general objective of the Upward Bound Math-Science 

program is to prepare participating students for postsecondary programs leading to careers in 

math and science. Seven out of ten UBMS projects operating in 1994 rated “academic 

performance in math and science” as their most or second most important goal (see Exhibit II.7). 

The focus on academic improvement was similar to the focus of regular Upward Bound projects 

operating around the same time.21 

However, two goals that regular Upward Bound projects considered moderately important 

were not considered important by UBMS projects. First, only 13 percent of UBMS projects 

reported that one of their top two goals was fostering students’ personal skills (e.g., goal 

orientation, ability to adapt to new settings), compared with 31 percent of regular Upward Bound 

programs. Second, none of the UBMS projects cited improving students’ access to financial aid 

as one of their top two goals, compared with 35 percent of regular programs. 

3. Academic Orientation 

UBMS projects try to provide academic enrichment beyond what students are exposed to in 

school (see Exhibit II.8). Very few projects emphasized remedial instruction in 1994.22 While 

 20 Only 4 percent of UBMS projects in 1994 had a residential component shorter than the summer program, but 

by 1998, the rate had increased to 11 percent. 

21 Eighty-seven percent of regular Upward Bound projects rated “academic improvement” as their most or 

second most important goal in 1993. If the regular Upward Bound grantee survey had also listed “academic 

improvement in math and science,” it is possible that some respondents would have cited that as one of their top two 

goals: Fasciano and Jacobson (1997) characterized 37 percent of regular Upward Bound projects as having a strong 

emphasis on math and science. 

22 By 1998, the relative focus on enrichment was even greater, with 83 percent of UBMS projects citing 

enrichment as a major emphasis and only 1 percent citing remediation. 
15

Exhibit II.7 

Most Important Goals of UBMS Projects, Summer 1994

0% 20% 40% 60% 80% 100%

Exposure to college

General academic

improvement

Perseverance in math

and science

Exposure to careers in

math and science

Academic improvement

in math and science

Percent of UBMS Projects

Most important Second most important

SOURCE: 1998 survey of UBMS projects. 

Exhibit II.8 

Academic Orientation of UBMS Projectsa

Major Emphasis 

UBMS 

Summer 1994 

Regular Upward Bound 

Summer 1992b

Support—instruction that parallels what students are 

taught in regular school courses 33% 55% 

Remediation—instruction that concentrates on 

fundamental concepts and skills that were taught in 

earlier grades 6 23 

Enrichment—instruction in concepts and material 

beyond what students are exposed to in regular school 

classes 73 69 

a Percentages do not sum to 100 because grantees were allowed to rate more than one approach as a major emphasis. 

b Excludes summer bridge programs for Upward Bound participants who have just graduated from high school. 

SOURCES: 1998 survey of UBMS projects, Fasciano and Jacobson 1997. 
16

about one in four regular Upward Bound programs emphasized the provision of remedial 

instruction, fewer than one in ten UBMS projects reported doing the same. 

The focus of UBMS on academic enrichment over remediation is consistent with the types 

of students served by the program. As described earlier, findings in Moore (1997b) indicate that, 

on average, UBMS participants probably had less need for remedial support than regular Upward 

Bound participants. 

4. Summer offerings 

In accordance with program guidelines, UBMS projects offered instruction in a diverse array 

of academic subject areas (see Exhibit II.9). Seventy-five percent or more of these projects in 

1994 offered instruction in the following subjects: writing and composition, algebra II, 

geometry, precalculus, computer applications and software use, biology, chemistry and physics. 

The average total number of offerings in 1994 was about 14, with a range of 2 to 22. The 

average number of offerings in math and science was about 7, with a range of 1 to 11.23 Thus, 

on average, math and science courses accounted for roughly half of UBMS projects’ total 

offerings. 

UBMS projects clearly differed from regular Upward Bound projects in their relative 

emphasis on certain subjects. First, as expected, they concentrated their offerings more on math 

and science. Although UBMS projects were no more likely than regular Upward Bound projects 

to offer certain math or science courses (for example, algebra II, geometry precalculus, calculus, 

biology and chemistry), UBMS projects were much less likely to offer instruction in areas 

outside of math and science, such as social science or history courses or electives or nonacademic courses. Second, consistent with their greater emphasis on enrichment than on 

remediation, UBMS projects were less likely than regular Upward Bound programs to offer lowend courses such as reading comprehension and vocabulary, pre-algebra, and earth science (see 

Exhibit II.9). 

To help prepare students for a postsecondary education and post-collegiate careers in math 

or science, UBMS projects also offered a range of support services and activities (see Exhibit 

II.10). Among the most common activities were field trips (e.g., to math or science facilities) 

and assistance with college and financial aid applications. The average number of these noninstructional offerings in 1994 was about 10, with a range of 3 to 15. 

UBMS projects were substantially less likely than regular Upward Bound projects to offer 

services focused on preparing for college. Many regular Upward Bound projects would have 

provided these services during the academic year: both regular Upward Bound and Math 

Science programs focus on academics during the summer. Because many UBMS participants 

participated in other precollege programs during the academic year, UBMS project staff could 

 23 It is not clear why some UBMS projects apparently had such a small number of instructional offerings. One 

possibility is that their instructional approach centered on interdisciplinary courses or large projects that covered 

multiple subject areas. However, only 3 of the 12 projects that reported five or fewer math and science offerings in 

1994 also reported that their primary instructional method was interdisciplinary courses or large projects and 

experiments. A second possibility is underreporting of instructional offerings by project directors. 
17

Exhibit II.9 

Instruction Offered by UBMS Projects, by Subject Areaa

 UBMS 

Summer 1994 Regular Upward Bound 

1992b

English/Language Arts 

Writing/Composition 93% 100% 

Literature 60 83 

Reading Comprehension and Vocabulary 65 98 

English as a Second Language 13 11 

foreign Language 54 35 

Other 9 13 

Mathematics 

Pre-Algebra 36 82 

Algebra I 69 96 

Algebra II 81 95 

Geometry 80 95 

Precalculus 80 80 

Calculus 52 58 

Statisticsc

 17 c

Trigonometryc

 7 c

Other 9 24 

Computers 

Programming 43 47 

Applications/Software Use 85 79 

internet/Web Page Designc

 7 c

Other 7 6 

Science 

Physics 76 63 

Biology 87 89 

Chemistry 81 81 

Earth Science 48 66 

Other 15 19 

Social Science/History 

History 11 47 

Geography 9 24 

Sociology 4 17 

Psychology 8 15 

Government/Civics 9 40 

Other 8 13 

Electives/Non-Academic Courses 

Performing Arts 31 53 

Art 26 53 

Journalism 28 52 

Speech/Public Speaking 48 59 

Physical Fitness 56 69 

Other 6 26 

a

 UBMS projects offer instruction in many areas besides math and science, either to meet regulatory requirements or 

simply to ensure that their program will interest and benefit students in many ways. 

b 1992 non-bridge summer programs or 1992-93 academic year. 

c

 Neither survey listed statistics, trigonometry, or internet or Web page design, but enough project directors specified 

them under “other courses” that we present data on these courses separately. 

SOURCES: 1998 survey of UBMS projects, Fasciano and Jacobson (1997:39).
18

Exhibit II.10 

Noninstructional Services Offered by UBMS Projects

UBMS 

Summer 1994 

Regular Upward Bound 

1992a

College preparation/skills 

Campus visits 74% 98% 

Adjusting to college living 98 92 

ACT/SAT preparation 65 97 

PSAT/PLAN or PACT preparation 24 73 

Help with financial aid or scholarshipsb

 80 100 

Assistance with college applications 78 99 

Assistance with financial aid applications 72 100 

Career/employment assistance 

Site visits to employersc

 65 59 

On-campus (employers or career representatives) 63 78 

Project-related work experience 49d

JTPA job 0 d

Work-study job 4 d

Math or science internships 24 d

Job through other partnerships 7 d

Field trips to 

Academic science or math facilities 98 e 

Non-academic science or math facilities 94 e

Conduct math- or science-related field work 85 e

Other 93 e

a

1992 nonbridge summer programs or 1992-93 academic year. 

b

 In the regular Upward Bound grantee survey, this item was phrased, “Identify sources of financial aid.” 

c In the regular Upward Bound grantee survey, this item was phrased, “Site visit to employers or job shadowing.” 

d

Although the regular Upward Bound grantee survey had three separate items about JTPA, work-study and other 

partnerships, the results were reported only in the aggregate, and it did not ask about math or science internships 

(Fasciano and Jacobson 1997:54). 

e

the regular Upward Bound grantee survey asked about field trips of varying lengths, not the destinations. 

SOURCES: 1998 survey of UBMS projects, Fasciano and Jacobson (1997:54).

have reasonably expected that those other programs were assisting students in preparing for 

college. 

5. Academic-Year offerings 

 While UBMS projects also provided services to students during the academic year in the 

mid-1990s, these services were minimal compared to UBMS summer services and also typically 

far less numerous and less intense than academic year services provided in regular Upward 

Bound. During the 1994-95 academic year, about one-third of UBMS projects provided tutoring 

or study sessions, and just over half provided assistance with college applications (see Exhibit 
19

Exhibit II.11 

Academic Year Services Offered by UBMS Projects, 1994-95 

SOURCE: 1998 survey of UBMS projects. 

II.11). The average UBMS project provided about three types of these services during 1994-

95.24 Not surprisingly, geographic proximity to their participating students influenced whether 

UBMS projects provided certain services during the academic year. UBMS projects were 

substantially more likely to provide tutoring and workshops during the academic year if a 

relatively large percentage of their participants lived in the same city or town as the program 

host.25

6. Instructional Approaches 

In 1994, the most common instructional approach taken by UBMS projects was the 

provision of instruction through courses in separate subjects. Four out of five UBMS projects 

offered courses in separate subjects (see Exhibit II.12, Panel A). In three out of four UBMS 

projects, the primary method of instruction was either the provision of these courses (37 percent) 

 24 By 1998-99, the percentage of UBMS projects regularly providing these services had increased substantially. 

For example, the percentage providing tutoring or study sessions rose from 81 percent to 93 percent, and the 

percentage providing assistance with college applications rose from 52 percent to 78 percent. This probably reflects 

the establishment of more locally oriented UBMS projects. 

25 For example, more than three-quarters of UBMS projects with a relatively large percentage of participants 

from the local area (above the median) offered tutoring during the academic year, versus less than half of UBMS 

projects with a relatively small percentage (below the median).

15%

17%

35%

39%

46%

52%

0% 20% 40% 60% 80% 100%

Other

Employment or

internship assistance

Tutoring or study

sessions

Individualized projects

Workshops

College application

help before 12th grade

Percent of UBMS Projects
20

or the combination of these courses with interdisciplinary courses (also 37 percent, see Exhibit 

II.12, Panel B). 

However, UBMS projects frequently employed other instructional methods. The majority of 

projects (63 percent) offered interdisciplinary courses, and in a large minority (35 percent), at 

least some students worked on a large project or experiment that spanned multiple academic 

subject areas (see Exhibit II.12, Panel A). 

UBMS projects vary considerably in how they sort students into classes or groups. in the 

summer of 1994, about half of projects placed their students in instructional groups based on 

proficiency level (37 percent) or grade level (16 percent). About one-fourth placed students with 

diverse proficiency levels in the same group to facilitate learning (presumably the learning of 

less proficient students), and the remaining projects grouped students by their interests or in 

some other way. 

UBMS participants do not spend most of their time in traditional lecture-style classes. At 

the average project during the summer of 1994, only one-fourth of the time was spent in lecturestyle classes like those offered in most schools. The remaining time was spent in small group, 

teacher-led instruction (32 percent), laboratories (29 percent), computer-based instruction (12 

percent), and other settings (4 percent). 

D. INTENSITY AND QUANTITY OF SERVICES 

The services that UBMS projects offer and the length of their summer residential summer 

programs suggest that these projects offer intensive programs that provide students with a “large 

dose” of services, at least for one summer. Furthermore, larger doses of effective services may 

yield larger impacts than smaller doses, as we found for regular Upward Bound (Myers et al. 

2004). Summary measures of program intensity presented in this section indicate that UBMS 

projects offer an intensive program that might be expected to improve the math and science 

preparation of program participants. 

UBMS is a resource-intensive program. Program grants to UBMS projects provided an 

average of approximately $4,800 per student in FY 2004 (see Exhibit II.13). This is comparable 

to funding for regular Upward Bound—approximately $4,500 per student—and much more 

expensive than most precollege programs. UBMS funding supports an extensive package of 

instruction and services, as described earlier in the chapter. 

Participants devote a substantial amount of time to the program, and most of this time is 

spent on academics. At the average UBMS project in the summer of 1994, students spent about 

29 hours per week receiving instruction and almost 11 hours per week on tutoring and 

homework. Thus, participating in UBMS over the summer is somewhat like having a full-time 

job requiring 40 hours per week. Because the vast majority of UBMS summer programs last six 

weeks,26 participants at the average project spend 240 total hours on academics during the 

summer. 

 26 Only five UBMS projects reported a program of a different length—three had a five-week session, one had a 

seven-week session, and one had an eight-week session. 
21 

Exhibit II.12 

Instructional Methods Used by UBMS Projects, 1994

A. Use of Different Instructional Methods

4%

35%

63%

81%

0% 20% 40% 60% 80% 100%

Other

Large project

Interdisciplinary courses

Separate subjects

Percent of UBMS Projects

B. Primary Method Used

Large project Interdisciplinary 11%

courses

9%

Other

6%

Separate subjects

37%

Separate subjects 

and 

interdisciplinary 

courses

37%

NOTE: Percentages may not sum to 100 percent due to rounding. 

SOURCE: 1998 survey of UBMS projects. 
22

Exhibit II.13 

Per-Capita Funding for UBMS Projects, FY 2004

SOURCE: U.S. Department of Education Web page (www.ed.gov/programs/triomathsci/funding.html and 

www.ed.gov/programs/trioupbound/funding.html accessed June 2006).

The amount of time devoted to core program activities varied across UBMS projects. For 

example, four projects reported that students spent over 40 hours per week in instruction alone, 

while a handful of projects reported estimates of less than 10 hours per week. Estimates of time 

spent on tutoring or homework ranged from 0 to 20 hours per week. And the estimates of 

average total time spent on academics during the summer varied considerably, from under 100 

hours at some projects to over 340 hours at some others. 

Most UBMS participants seem to stay for the entire summer. At the average project in the 

summer of 1994, 94 percent of participants completed all the requirements of the program. A 

few projects reported completion rates of 50 percent or less, but this does not necessarily indicate 

a high dropout rate. Students might have attended the summer program for its full length but be 

counted as failing to “complete all the requirements of the program”—the wording in the 

questionnaire—because they did not, for example, turn in all their assignments. 

Some students participate for multiple summers or receive services during the academic 

year. As indicated earlier, about 70 percent of UBMS projects operating in summer 1994 

allowed students to return from the previous summer. Among those projects, an average of 35 

percent of 1994 participants had also participated during 1993.27 Furthermore, some UBMS 

projects extended students’ exposure to the program beyond the summer by providing services 

during the academic year (see Section C.4 for more details). 

 27 In summer 1998, not only did more UBMS projects (over 90 percent) allow students to return, but at those 

projects an average of 52 percent of students had also participated in summer 1997. 

$4,794

$4,514

$0 $2,000 $4,000 $6,000

UBMS

Regular Upward

Bound
23

III. THE IMPACTS OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 

UBMS was established to increase economically disadvantaged students’ achievement in 

high school math and science courses, to increase the likelihood that they would major in math 

and science in college, and ultimately to increase their representation in math and science 

careers. Until now, no rigorous studies have measured the extent to which the program achieves 

its goals. In this chapter, we assess the effects of UBMS on important outcomes for program 

participants four to eight years after graduating from high school. 

This chapter presents our estimates of the effects of UBMS on (1) high school preparation 

for college and for majoring in math and science; (2) college enrollment, highest level of college 

attended, college selectivity, years of college completed and highest degree earned; and (3) field 

of study in college. These estimates are based on information collected in 2002 or earlier, and 

another round of data collection is currently underway. Given many students in our sample may 

have, for example, selected a major or completed a degree between 2002 and 2004, the findings 

in this report should be treated as an initial assessment of the effects of the program. Our final 

assessment will be based on the data collected in 2004. 

The evidence suggests that in addition to improving students’ grades in high school, UBMS 

increases the likelihood of: 

• Taking chemistry and physics in high school 

• Enrolling in more selective four-year colleges and universities 

• Completing a postsecondary degree in a math and science field 

The effects of UBMS are nontrivial and in the expected direction. For example, UBMS 

increased the likelihood of earning a bachelor’s degree in a math or science field four to eight 

years after high school graduation from 12 percent to 20 percent. in the next report on UBMS, 

we will weigh the size of the program’s benefits against its costs. 

Because UBMS participants in our sample were not randomly selected to participate in the 

program, the impact estimates presented in this chapter may suffer from selection bias. 

However, in designing the study, we identified several likely sources of bias and have addressed 

them through a combination of data collection and statistical methods described later in the 

chapter. Therefore, the research design is stronger than in the typical nonexperimental study, 

and as a result, the impact estimates should be more credible. 

A. STUDY DESIGN 

UBMS provides intensive academic enrichment in math and science. Like regular Upward 

Bound projects, most UBMS projects also offer some assistance in preparing for college, such as 

assistance with college applications (see Chapter II for more details). The combination of 
24

intensive academic enrichment in math and science with college preparation assistance suggests 

that UBMS might have positive effects on the following outcomes for program participants: (1) 

performance in high school, especially in math and science courses; (2) postsecondary 

attendance, persistence, and completion; and (3) the likelihood of completing a postsecondary 

degree in a math and science field. Therefore, we designed the analysis to answer the following 

three research questions: 

1. What are the effects of UBMS participation on student performance in high school 

overall and in math and science courses in particular? 

2. What are the effects of UBMS participation on college attendance, attendance at 

different types of colleges and universities, years of college, and college completion? 

3. What are the effects of UBMS participation on the likelihood of completing a degree 

in math or science? 

We have designed the analysis to measure the impacts of UBMS on two important 

subgroups: students who had previously participated in regular Upward Bound and students who 

had not participated. UBMS participants who previously participated in regular Upward Bound 

may have received a large dose of precollege services and academic preparation before 

participating in UBMS. However, most of the other UBMS participants entered UBMS without 

having received such intensive services. It is reasonable to expect UBMS to have larger effects 

on the students who had not previously received intensive services: students who have already 

received them may have already received the boost they needed to succeed. On the other hand, 

UBMS participants may be better prepared to benefit from their participation if they have 

previously participated in intensive services. in our analysis, we compute the effects separately 

for students who had previously participated in regular Upward Bound and students who had not 

participated to assess whether the effects of UBMS depend on the amount of precollege services 

students have received to that point. Regular Upward Bound is just one of the other programs in 

which UBMS participants could have participated. However, in our analysis, regular Upward 

Bound deserves special consideration because many UBMS participants participate in regular 

Upward Bound, and because few other programs are as intensive as UBMS.28

It is important to clarify we have not attempted to measure the effects of UBMS versus no 

precollege services. The analysis was designed to measure the effects of participating in UBMS 

relative to what students would have participated in otherwise—which might include regular 

Upward Bound—not the effects of participating in UBMS relative to no program participation. 

Most UBMS participants in our sample did not participate in regular Upward Bound, and only a 

few other precollege programs are as intensive as Upward Bound. Therefore, most UBMS 

participants would have participated in less intensive precollege services if they had not 

participated in UBMS. 

 28 Moore (1997), Appendix A includes a list of 28 alternative math and science precollege programs. 

However, some other precollege programs without a specific precollege focus, such as regular Upward Bound, may 

share some common features with UBMS. Annual per student costs were available for 17 of the 28 programs; of 

these 17, only two were more expensive per student-year than UBMS. 
25

 In this section, we describe the design of the analysis used to measure the impacts of 

UBMS. The impact analysis is based on a matched comparison group that attempts to reduce 

two types of bias common to many nonexperimental studies: selection bias and bias attributable 

to different data collection protocols for the participant and comparison groups. The strength of 

the impact analysis rests on three features of its design: 

1. How we selected our initial samples, particularly the comparison sample 

2. How we collected “baseline” (preprogram) information on the two samples 

3. How we used that information to select a matched comparison group 

In the remainder of this section, we describe these three features of the study and our 

approach to estimating the effects of UBMS on student participants. 

1. Selecting the Samples 

For the impact analysis, we obtained our sample of program participants from the projects 

themselves. In 1998, we contacted the 62 Upward Bound Math and Science Centers (MSCs) 

that were operating at that time and that had been operating between 1993 and 1995.29 From 

these MSCs, we requested lists of the students who had participated in their program in the 

summer of 1993, the summer of 1994 or the summer of 1995.30 To reduce the costs of collecting 

the necessary data, we selected one out of every four of the students from these lists for our 

analysis sample. 

A primary feature of any nonexperimental evaluation is the choice of a comparison group. 

Experiments yield the best comparison groups because differences in outcomes between treated 

and untreated cases cannot be attributed to selection bias. In the absence of an experiment, the 

strength of an evaluation depends on the comparability between the participant and comparison 

groups. 

The most convenient comparison group for the impact analysis is also a compelling one—

and the one we used. For the comparison group, we selected students from the evaluation of 

regular Upward Bound who reported that they had not participated in an MSC.31 In doing, so, 

we selected a comparison group with three desirable attributes: 

 29 While we would have been interested in obtaining lists of students from projects that were no longer 

operating in 1998, we believed that it would be very difficult to obtain such lists. If MSCs that closed before 1998 

operated less effective programs than those that remained open, then the results presented in this chapter may 

overstate the effectiveness of MSCs operating between 1993 and 1995. 

30 We received participant lists from all but one of these MSCs. 

31 If UBMS projects admitted eligible applicants on a fairly random basis, rejected applicants would probably 

constitute the best comparison group. However, MSCs are not required to select randomly from eligible applicants, 

and the evidence suggests that they do not. Case studies of 14 MSCs in the mid-1990s suggest that many MSCs 
26

1. Like UBMS participants, comparison students applied to participate in Upward 

Bound. Therefore, both UBMS participants and comparison students revealed a 

high level of motivation to pursue precollege services. This provides some 

protection against a common source of selection bias in nonexperimental studies—

bias from comparing more motivated participants to less motivated nonparticipants. 

2. Like UBMS participants, comparison students met the federal eligibility 

requirements to participate in some type of Upward Bound program—regular or 

math and science. The federal eligibility requirements are the same for both regular 

Upward Bound and UBMS. To be included in the sample for the evaluation of 

regular Upward Bound—the sample from which we selected our comparison 

group—a student must have applied to a regular Upward Bound project and been 

determined eligible to participate. Therefore, both UBMS participants and 

comparison students in our sample must either have come from “low-income” 

families (income below 150 percent of the poverty line) or potential “firstgeneration” families (neither parent had earned a bachelor’s degree). 

3. Like UBMS participants, many comparison students would have met projectspecific eligibility requirements imposed by some MSCs. Moore (1997) indicates 

that MSCs often apply additional admissions criteria in selecting applicants—criteria 

that include a minimum grade point average (GPA) in math and science. in this 

chapter, we show that many comparison students met the same criteria: many were 

successfully “matched” to UBMS participants who took similar courses and earned 

similar grades in math and science in ninth grade. 

While the regular Upward Bound sample is a useful comparison group for measuring the 

effects of UBMS, it is not a perfect one. Data from the 1990s suggest that MSCs typically had 

more stringent minimum GPA requirements than regular Upward Bound projects; and MSC 

staff—who typically had recent experience with regular Upward Bound participants—reported 

that UBMS participants tended to be “more serious about school” than regular Upward Bound 

participants (Moore 1997, p. 26). Therefore, UBMS participants might fare better than regular 

Upward Bound participants even without any assistance from UBMS, and simple differences in 

mean outcomes between the UBMS participant and comparison groups may overstate the effects 

of UBMS due to selection bias. 

To reduce selection bias, we selected a matched comparison sample from the regular 

Upward Bound sample. More specifically, we matched each UBMS participant to one or more 

 

(continued) 

impose additional eligibility requirements beyond the federal requirements, including an interest in math and science 

(Moore 1997). Therefore, it is unlikely that rejected applicants would have the virtues of a randomly selected 

control group for the purposes of this evaluation. Furthermore, the difficulty in obtaining lists of program 

participants from MSCs several years after they participated in UBMS suggests that many MSCs would not have 

been able to provide information on rejected eligible applicants. 
27

regular Upward Bound sample members with similar characteristics based on data collected from 

student surveys and transcripts.32 See Section 3 for more details. 

2. Collecting Baseline Data 

The baseline variables constructed for the impact analysis characterize members of the two 

samples early in high school before UBMS participants in our sample entered UBMS. These 

variables are critical because they allow us to account—through a combination of matching and 

regression adjustments—for many preexisting differences between the two groups that might 

otherwise bias our impact estimates. To collect the information necessary to create baseline 

variables, we conducted student surveys and collected high school transcripts (see Appendix A). 

An important strength of the study’s design is that the data collection strategy was similar 

for both samples. Heckman et. al. (1997) argue that different survey questionnaires for the 

participant and comparison samples can be an important source of bias in nonexperimental 

studies. In this evaluation, we developed the initial survey questionnaire for UBMS participants 

from the baseline survey questionnaire for the regular Upward Bound sample. Therefore, the 

survey questions from which we constructed baseline variables were often identical and always 

similar for the two samples.33 Furthermore, the approach to collecting and coding high school 

transcripts was the same for the two samples. Therefore, it is unlikely that differences in the data 

between the two samples have biased the impact estimates presented in this chapter. 

The baseline variables for the impact analysis fall into the following three categories: (1) 

demographic and family characteristics, (2) participation in other precollege programs and (3) 

ninth-grade academic achievement in math and sciences and more generally (see Exhibit III.1). 

We believe that the measures of students’ ninth-grade academic achievement are critical to the 

strength of the study. Given the findings in earlier reports, it seems entirely possible that even 

among students with similar demographic and family characteristics, students who participated 

in UBMS might have a higher academic aptitude and interest in math and science than students 

who participated in regular Upward Bound. Therefore, we use information on ninth-grade 

courses taken and ninth-grade grade point average—overall and specifically math and science—

to control for differences between the two samples in early high school achievement. 

 32 Very few of students in the regular Upward Bound sample participated in UBMS, and those who reported 

participating in UBMS were excluded from the comparison group. 

33 Two differences in the two surveys were addressed in constructing baseline variables for the MSC impact 

analysis: (1) the two survey questionnaires were different, and (2) the surveys were conducted at different times—

after high school for the participant group and early in high school for the comparison group. To address the 

differences in the survey questionnaires, we used data from survey questions that are either identical or almost 

identical. To address the difference in the timing of the surveys, we used data from survey questions only when the 

timing of the survey was unlikely to affect the answer to the question. 
28

Exhibit III.1 

Baseline Variables 

Category Variables Source 

Demographic and family Sex 

Race and ethnicity 

Census region 

Native English speaker 

Mother’s time in the U.S. 

Sibling participated in Upward Bound 

High school cohort 

Initial surveys 

Prior program participation Sample member participated in regular Upward Bound Initial survey and project records 

Ninth-grade achievement GPA in math and science 

GPA in other subjects 

Math course taken 

Science course taken 

High school transcripts 

NOTE: The initial surveys were conducted in 1999 for UBMS participants and in 1992-94 for comparison 

students in the regular Upward Bound sample. To identify UBMS participants who had previously 

participated in regular Upward Bound, we used responses to the 1999 initial survey; to identify 

comparison students who participated in regular Upward Bound, we used participation information 

provided by projects. We collected high school transcripts in 2000 and 2003 for UBMS participants and 

in 2000 and earlier years for comparison students. 

3. Selecting a Matched Comparison Sample for the Impact Analysis 

The data collected on UBMS participants and regular Upward Bound sample members are 

useful in identifying many similarities between the two groups—and some differences as well 

(see Exhibit III.2). In both groups, 37 percent of the students were African American and 11 

percent had a sibling who had participated in Upward Bound. The two groups were similar with 

respect to many other characteristics as well, including the percentage taking algebra in ninth 

grade. However, the two groups exhibit differences on several dimensions. For example, 

UBMS participants were more likely to be male and tended to have higher grades than regular 

Upward Bound sample members. 

To address possible selection bias, we selected a matched comparison group from the 

regular Upward Bound sample using propensity score matching methods.34 The goal in 

 34 Many studies adjust for baseline differences of these types using standard covariance adjustments—that is, 

by controlling for these variables in a regression analysis. However, the differences between the UBMS participant 

sample and the regular Upward Bound sample are too large to expect covariance adjustment to be reliable. 

Regression adjustments are likely to be unreliable if the means of the propensity scores are more than half a standard 

deviation apart (Rubin 2002). For the participant sample, the mean and standard deviation of the propensity score 

are 0.58 and 0.22, respectively; for the comparison sample, the mean and standard deviation of the propensity score 

are 0.25 and 0.23, respectively. Therefore, the difference in mean propensity scores is more than one standard 

deviation, and regression adjustments alone are likely to be unreliable. 
29

Exhibit III.2 

Summary Statistics from the Baseline Variables 

(Percentage unless otherwise noted) 

Characteristic 

UBMS 

Participants 

Regular Upward 

Bound Sample 

Matched Comparison 

Group 

Participated in Regular Upward Bound 18 55*** 18 

Female 59 72*** 59 

Race and Ethnicity 

 African American 37 37 37 

 White 25 34*** 30 

 Hispanic 18 20 16 

 Other race 20 9*** 17 

Region 

 Northeast 11 9 12 

 Midwest 23 19** 28 

 South 40 45** 35 

 West 25 26 24 

Entry to High School 

 1991-92 32 25*** 28 

 1992-93 39 49*** 37 

 1993-94 29 26 35 

Other Characteristics 

 Native English speaker 80 87*** 86** 

 Mother in U.S. most of her life 79 87*** 83* 

 Siblings in Upward Bound 11 11 12 

Ninth-Grade Math Course 

 Lower than algebra 16 32*** 14 

 Algebra 55 53 59 

 More than algebra 29 14*** 27 

Ninth-Grade Science Course 

 Biology, chemistry, or physics 37 26*** 37 

Ninth-Grade GPA (mean) 

 Math and science 2.69 1.65*** 2.71 

 Other subjects 3.24 2.64*** 3.25 

*/**/*** Significantly different from UBMS participants at the 0.10 / 0.05 / 0.01 level.
30

matching was to select a matched comparison sample from the regular Upward Bound sample 

such that the distributions of the baseline variables for the UBMS participant sample and the 

matched comparison sample were similar.35 Matching was conducted separately for sample 

members who had previously participated in regular Upward Bound and for those that had not: 

• UBMS participants who had previously participated in regular Upward Bound were 

matched to members of the treatment group in the evaluation of regular Upward 

Bound. 

• UBMS participants who had not previously participated in regular Upward Bound 

were matched to regular Upward Bound control group members who did not 

participate in regular Upward Bound. 

Regular Upward Bound sample members that were matched to at least one UBMS participant 

were included in the matched comparison sample. 36, 37

Using matching procedures, we were able to select a matched comparison group that is 

highly similar to the sample of UBMS participants on many dimensions (see the last column of 

Exhibit III.2). Only two of the differences between the groups are statistically significant—the 

difference in the percentage of sample members who are native English speakers and the 

percentage whose mother has lived in the United States for all or almost all of her life. 

Furthermore, given the number of baseline variables, two is a small number of significant 

differences: we would expect about two significant differences even if the differences between 

the two groups were purely random. While there is no guarantee that matching removed all 

unmeasured differences between the two samples, matching removed differences on a broad 

range of baseline variables—differences that might otherwise bias the impact estimates. 

 35 More specifically, the goal is to ensure that the distributions of the baseline variables for the UBMS 

participant sample and the matched comparison sample are similar enough such that covariance adjustments will 

produce reliable impact estimates 

36 The propensity score matching and the impact analysis are restricted to sample members who entered high 

school between 1991 and 1993. While some UBMS participant sample members entered high school before 1991 

and after 1993, relatively few comparison sample members did so. Furthermore, high school cohort is related to the 

likelihood of participating in the program and to the outcomes of interest because earlier cohorts had more time to 

enter college and select a field of study by the time they were interviewed for the evaluation. Therefore, to protect 

the internal validity of the study, we focused the analysis on students who entered high school between 1991 and 

1993. 

37 UBMS participants could be matched to more than one regular Upward Bound sample member, and regular 

Upward Bound sample members could be matched to more than one UBMS participant. To be matched, a pair of 

students must satisfy the following condition: the difference between matched students in the log odds of the 

propensity scores was less than 0.20 times the standard deviation of the log odds. Smith and Todd (2003) refer to 

this type of matching as “radius matching.” 
31

4. Estimating the Impacts of UBMS Participation 

To measure the effects of UBMS participation on participating students, we used a 

regression-based approach that allows us to (1) adjust for the small remaining differences 

between the UBMS participant sample and the matched comparison group and (2) increase the 

precision of our impact estimates. The regression models yield estimates of the effect of UBMS 

on students who participated in the program.38 

We estimated the effects of UBMS for the entire sample and separately for selected 

subgroups of students. The effects of UBMS may depend on the amount of other precollege 

services received. Therefore, we present separate impact estimates for those who participated in 

regular Upward Bound and those who did not participate. Furthermore, we present separate 

estimates by sex and race since achievement in math and science often varies along these 

dimensions, and the effects of UBMS may vary along the same dimensions.39,40 

B. ANALYSIS AND FINDINGS 

In this section, we present the estimated effects of UBMS on students’ outcomes. 

Consistent with program objectives, participation in UBMS seems to promote postsecondary 

study in math and science. Specifically, our estimates suggest that UBMS participation (1) raises 

grades in high school and course-taking in chemistry and physics, (2) increases enrollment in 

more selective four-year institutions and the number of years of college completed, and (3) raises 

the likelihood of both pursuing postsecondary studies and completing a four-year degree in math 

or science. In contrast, UBMS does not affect course-taking in advanced math in high school. 

 38 We regressed each outcome on a set of control variables and an indicator of whether the student participated 

in UBMS. The control variables included the variables used in selecting the matched comparison group: prior 

participation in regular Upward Bound, siblings in Upward Bound, sex, race, ethnicity, mother’s native language 

and immigrant status, high school cohort, region of the country, and several variables describing academic 

achievement in ninth grade, including GPA—separately for math and science courses and for other courses—math 

course taken, and science course taken. For continuous variables, such as number of college credits, we estimated 

linear regression models; for categorical outcomes, such as whether the sample member pursued postsecondary 

studies in math or science, we estimated logistic regression models or “logit” models. In estimating standard errors, 

we accounted for clustering by project and used the Taylor series linearization methods employed by the SUDAAN 

statistical analysis software. 

39 Other reports from the National Evaluation of Upward Bound have shown that the effects of regular Upward 

Bound vary with the educational expectations of student applicants (see Myers et al. 2004 and Myers and Schirm 

1999). In particular, regular Upward Bound greatly increased the likelihood of attending a four-year college or 

university for students who did not expect to earn a bachelor’s degree when they applied to participate in regular 

Upward Bound. Because we first interviewed UBMS participants after they finished participating in UBMS, we 

lack information on their educational expectations when they applied to participate in UBMS. Therefore, we cannot 

assess whether the effects of UBMS vary with students’ educational expectations before participating. 

40 In the national evaluation, we have estimated the effects of regular Upward Bound separately by eligibility 

category (low-income, first-generation, or both). However, student data was provided by project directors when 

they submitted students to Mathematica for random assignment. Because we did not conduct random assignment 

for the evaluation of UBMS, we lack data on eligibility for UBMS participants. While we did obtain rosters of prior 

participants from UBMS projects in 1998, we did not request data on eligibility because we did not believe that 

projects could easily provide them for students who participated three or more years ago. 
32

1. The Effect of UBMS on High School Outcomes 

The UBMS program is designed to strengthen participants’ math and science skills. MPR 

conducted follow-up student surveys and collected high school transcripts to determine whether 

UBMS participation increased the amount of course work in math and science as well as total 

high school credits, high school graduation and GPA in math and science and overall.41 

UBMS raised the likelihood of taking upper-level high school courses in science but not in 

math (see Exhibit III.3). Our estimates indicate that UBMS increased the likelihood of taking 

upper-level science courses in high school, raising the percentage of students taking chemistry 

from 78 to 88 percent and raising the percentage of students taking physics from 43 to 58 

percent.42 The estimated effects of UBMS on the likelihood of taking algebra II, trigonometry, 

precalculus and calculus are statistically insignificant. 

It is not clear why UBMS affected course-taking in the sciences but not in math. One 

possibility involves differences between the type of instruction that MSCs offer in math and the 

type of instruction they offer in sciences. Almost two-thirds of MSCs reported offering 

interdisciplinary classes, and about a third of them offered instruction that revolved around a 

large project or experiment (see Chapter II). However, findings from Moore (1997) suggest that 

in MSCs, laboratories and hands-on instruction were much more common in science instruction 

than in math instruction, and that MSC instructors used math to help student prepare for careers 

in science and technology (see Moore 1997, pp. 41-42). It is possible that the combination of 

hands-on instruction offered only in science with the science orientation of MSC instructors is 

responsible for UBMS’s effects on science course-taking. 

Additional estimates suggest UBMS raises overall achievement and educational attainment 

in high school (see Appendix B). UBMS had a positive effect on high school grades, increasing 

the average GPA in math courses from 2.7 to 2.8, the average GPA in science courses from 2.7 

to 2.9 and the average GPA overall. It also raised the graduation rate from 96 to 99 percent. 

UBMS appears to have had larger effects on grades and science course work for Hispanics 

than for African Americans (see Appendix B). For example, UBMS raised the average GPA in 

math courses from 2.4 to 2.5 for African Americans and from 2.4 to 2.7 for Hispanics. In 

addition, UBMS raised the likelihood of taking chemistry and physics by 7 percentage points 

each for African Americans and by 17 and 27 percentage points, respectively, for Hispanics.43 

Comparisons of males to females and students who had participated in regular Upward Bound to 

other students revealed few significant differences. 

 41 With the exception of the outcome variables describing high school completion, which were constructed 

from student survey responses, the high school outcomes were constructed using data from high school transcripts. 

42 These increases in part reflect increases in the total number of courses taken in high school: UBMS raised 

the average number of high school credits earned from 25 to 26. 

43 For African Americans, the estimated effect on the likelihood of taking physics was not statistically 

significant. 
33

Exhibit III.3 

Math and Science Courses Taken in High School 

(Percentage of students) 

 UBMS Participants Matched Comparison Group Impact 

 

Math Courses 

 Algebra 97 97 0 

 Geometry 86 86 0 

 Algebra II 73 74 -1 

 Trigonometry 35 30 5 

 Analysis or Precalculus 36 31 5 

 Calculus 26 24 2 

 

Science Courses 

 Biology 97 98 -1 

 Chemistry 88 78 10*** 

 Physics 58 43 15*** 

*/**/*** Statistically significant at the 0.1 / 0.05 / 0.01 level. 

NOTES: Impact may not exactly equal the difference between UBMS participants and the matched comparison 

group due to rounding. See Table B.1 in Appendix B for more details and additional high school 

outcomes and Tables B.2 through B.4 for estimates for subgroups of UBMS participants. 

2. The Effect of UBMS on Postsecondary Attendance, Persistence, and Completion 

While UBMS focuses on preparing students to major in math and science and to complete a 

degree in a math and science field, a person must enroll in college before choosing a major and 

must complete college to earn a degree in a math and science field. Even if UBMS had little 

effect on students’ choice of major, UBMS might be a cost-effective strategy to increase college 

enrollment and completion for disadvantaged students. Therefore, we assess whether UBMS 

promotes postsecondary attendance, persistence, and completion before examining its effects on 

college major.

The evidence suggests that UBMS participation increases the likelihood of attending a 

postsecondary institution. Estimates for the matched comparison group suggest that 90 percent 

of UBMS participants would have attended a postsecondary institution if they had not 

participated in UBMS (see Exhibit III.4). This indicates that UBMS participants are much more 

likely to attend college than the average low-income high school student, and a 90-percent 

attendance rate leaves little room for improvement. Despite this, the evidence suggests that 

UBMS participation increases the likelihood of attending a postsecondary institution from 90 to 

95 percent, although as shown in Appendix C, this result is sensitive to how postsecondary 
34 

Exhibit III.4 

Postsecondary Attendance, Highest Level Attended and College Selectivity 

(Percentage of students) 

 UBMS Participants Matched Comparison Group Impact 

 

Postsecondary Attendance 95 90 5*** 

 

Highest Level Attended 

 Four-year college or university 82 71 11*** 

 Two-year college 11 16 -5*** 

 Vocational institution 2 4 -2 

 

Most Selective Four-Year institution 

 More selective 33 23 11 

 Less selective 48 48 0 

*/**/*** Statistically significant at the 0.1 / 0.05 / 0.01 level. 

NOTES: Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison 

Group due to rounding. “Most Selective College or University” refers to four-year colleges and 

universities only, and some four-year institutions were not classified in Barron’s Profiles of American 

Colleges (2003). See Table B.5 in Appendix B for more details and additional postsecondary outcomes 

and Tables B.6 through B.8 for estimates for subgroups of UBMS participants. 

attendance is measured and, specifically, to whether college transcripts are used to verify 

attendance.44 

The evidence also suggests that UBMS participation increases the likelihood of attending a 

four-year college or university from 71 percent to 82 percent. The increase in four-year 

attendance can be attributed to increased attendance at more selective schools as UBMS raises 

the likelihood of attending such a school from 23 to 33 percent.45 It appears that some of this 

 44 This 95-percent figure suggests that almost all UBMS participants attend a postsecondary institution in the 

first few years after high school. In contrast, findings from a recent analysis of performance data submitted by 

UBMS projects seem to suggest a much lower rate of postsecondary attendance. Analyzing the performance data, 

Curtin and Cahalan (2004) find that only 68 percent of UBMS participants who graduated from high school in 1999-

2000 were reported as attending a postsecondary institution in 2000-01. The difference between this figure and the 

enrollment rate presented in this report reflects substantial differences in how the two estimates were derived. 

According to information provided to us by ED, the estimate in Curtin and Cahalan (2004) indicates the percentage 

of former program participants for whom UBMS project directors could confirm postsecondary attendance. The 

estimate was not intended to measure the postsecondary enrollment rate per se, and it understates the true rate for 

UBMS participants for two related reasons. First, the estimate includes in the denominator but not the numerator 

UBMS participants from projects that were no longer funded and, therefore, did not submit performance data. 

Second, it excludes from the numerator about one-fifth of the participants who are included in the denominator 

because UBMS project staff were unable to confirm college attendance for these former participants. For our 

estimate, in contrast, we exclude from both the numerator and denominator the UBMS participants in our sample for 

whom we are missing data due to survey nonresponse and, then, use standard weighting techniques to adjust for the 

missing observations. 

45 If a school was rated as “most competitive,” “highly competitive,” or “very competitive,” then we classified 

the school as “more selective.” If a school was rated as “competitive,” “less competitive,” “noncompetitive,” or 
35 

increase in four-year college enrollment can be attributed to students who would otherwise have 

attended only two-year colleges: UBMS reduced the likelihood of enrolling in a two-year 

college but not in a four-year college or university from 16 to 11 percent. 

UBMS increased the number of years that students spend in four-year colleges and 

universities without increasing the completion rate at these schools. Our estimates suggest 

UBMS increased the number of years enrolled in a four-year institution from 2.4 to 2.9 (see 

Exhibit III.5). While UBMS increased the amount of time participants spent in college, it had no 

effect on the likelihood of their completing a bachelor’s degree as of the time we last interviewed 

them in 2002. Given that almost half of UBMS participants in our sample (47 percent) were 

attending four-year colleges and universities when we last interviewed them, it is too early to 

reach firm conclusions about the effects of UBMS on the number of years of college attended 

and on college completion. Our final report on the Upward Bound Math-Science Program will 

be based on data collected in 2004. 

The average effects of UBMS mask some interesting differences between men and women. 

for both men and women, UBMS increased the percentage attending a selective four-year 

institution—from 23 to 34 percent for men and from 21 to 32 percent for women. However, 

UBMS only affected overall four-year enrollment for women. For women, UBMS increased the 

likelihood of attending a four-year institution from 68 to 82 percent and reduced the likelihood of 

attending a two-year institution (without also attending a four-year institution) from 18 percent to 

10 percent; for men, both of these effects were statistically insignificant. 

The effects on college completion also differ between men and women. For women, the 

positive effect of UBMS on enrollment in four-year institutions—along with positive effects on 

credits earned in these institutions—translates into higher graduation rates from four-year 

institutions. UBMS raised the percentage of women earning a bachelor’s degree from 32 to 40 

percent. For men, the effect of UBMS on the percentage completing a bachelor’s degree is 

statistically insignificant; however, UBMS increased the percentage completing an associate’s 

degree from 4 to 8 percent. 

In the last report from the national evaluation (Myers et al. 2004), we presented two sets of 

estimates for postsecondary attendance and highest level attended—the first based entirely on the 

information reported by the student, and the second that required verification of enrollment from 

the institution. In computing the second set of estimates, we made the strong assumption that a 

student did not attend the school they reported to us unless we received verification from the 

school in the form of a college transcript (or a reason for not providing one that clearly indicated 

that the student had attended the school). However, additional analysis of the data suggests that 

in most cases for which we did not receive a transcript, there is no reason to doubt that the 

students actually attended the schools they reported. Schools provided a variety of reasons for 

 

(continued) 

“special,” or the school was either excluded from Barron’s or was included but not rated, then we classified the 

school as “less selective.” Students were then classified as “more selective” school if they attended one or more of 

the “more selective” institutions, and “less selective” if they attended one or more of the “less selective” institutions 

and did not attend a “more selective” institution. 
36 

Exhibit III.5 

Years of College and Degree Completion 

UBMS Participants Matched Comparison Group Impact 

 

Years of College 

 Four-year college or university 2.9 2.4 0.4** 

 Two-year college 0.4 0.4 0.0 

 

Degree Completion (percent) 

 Bachelor’s degree 35 33 2 

 Associate’s degree 7 7 -1 

*/**/*** Statistically significant at the 0.1 / 0.05 / 0.01 level. 

NOTES: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section 

A.4). Impact may not exactly equal the difference between UBMS Participants and the Matched 

Comparison Group due to rounding. Years of Postsecondary Education were computed by dividing the 

number of credits earned by 30. See Table B.5 in Appendix B for more details and additional 

postsecondary outcomes and Tables B.6 through B.8 for estimates for different subgroups of UBMS 

participants. 

not providing transcripts, including confidentiality considerations and money that the student 

owed to the school. Therefore, for all outcome variables except years of college, which are 

based on information from college transcripts, the estimates presented in this chapter are based 

on the information reported by the students themselves. However, we conducted a sensitivity 

analysis to see whether requiring verification of enrollment affected our impact estimates, and in 

most cases, it did not. See Appendix C for more details about the sensitivity analysis. 

3. The Effect of UBMS on Postsecondary Field of Study 

The primary objective of UBMS is to prepare students for postsecondary studies in math and 

science. Using information reported by students in 2002, we conducted an analysis to examine 

whether UBMS participation increases the likelihood of pursuing postsecondary studies in math 

and science or the likelihood of earning a postsecondary degree in a math or science fields. To 

determine whether a sample member had pursued postsecondary studies in math and science, we 

asked sample members in the 2002 survey for their “most recent or intended field of study,” and 

we classified their responses according to the same classification system used in the National 

Science Foundation’s Scientists and Engineers Statistical Data System (SESTAT).46 

Furthermore, for the analysis, we separated the social sciences from other math and science 

fields (which we refer to as “math or science”) because the objectives of the program are more 

closely tied to the latter than the former. 

 46 The fields classified as science and engineering were biological sciences, computer science, engineering, 

mathematics, physical sciences, and technical fields. The fields classified as non-science and engineering were 

agriculture, arts, business, education, clerical or legal assistance, communications, health-related fields, humanities, 

trade and industry, protective services and consumer or personal services. A small number of fields reported by 

sample members could not be classified as either science and engineering or nonscience and engineering. (For more 

details about NSF’s classification of fields of study, see www.nsf.gov/sbe/srs/nsf99337/pdf/appa.pdf.) 
37 

The evidence suggests that UBMS participation encourages students to pursue 

postsecondary studies in math or science and also in the social sciences. UBMS increased the 

likelihood of majoring or intending to major in math or science from 23 to 33 percent overall and 

from 18 to 28 percent if we focus on majors at four-year colleges and universities (see Exhibit 

III.6).47 The effects are also positive but smaller in magnitude for the social sciences: UBMS 

increased the likelihood of pursuing postsecondary studies in the social sciences from 7 to 11 

percent overall and from 7 to 10 percent if we focus on majors at four-year institutions. 

The evidence also suggests that UBMS participation may encourage students to complete 

postsecondary degrees in math or science. However, with almost half of participants still in 

college, it is too early to reach final conclusions about UBMS’s effects on college completion 

overall or in specific fields. As of the 2002 survey, the effect of UBMS on the likelihood of 

earning a degree or certificate in math and science was statistically insignificant. However, if we 

focus on four-year institutions, UBMS altered the types of fields in which students earned 

degrees. More specifically, it raised the likelihood of earning a bachelor’s degree in math or 

science from 6 to 12 percent and reduced the likelihood of earning a bachelor’s degree outside of 

math or science or the social sciences from 20 to 14 percent. 

The effect of UBMS on the likelihood of majoring in math and science was larger for men 

than for women. While UBMS increased the likelihood of pursuing postsecondary studies in 

math and science from 38 to 58 percent for men, the estimated effect for women is much smaller 

and statistically insignificant. In contrast, there is no evidence that the effect of UBMS on the 

likelihood of completing a math or science degree differs between the two groups. 

C. INTERPRETATION OF THE FINDINGS 

The estimated effects of UBMS paint a fairly consistent picture of UBMS helping its 

participants onto an educational path that could lead to careers in math and science. Our findings 

suggest that UBMS participation improves student outcomes in high school and college, and—

consistent with the objectives of the program—increases the odds of completing a college degree 

in math and science. 

While we took several steps to reduce selection bias, it is certainly possible that the true 

effects of participating in UBMS are smaller than our estimates suggest. As reported earlier in 

the chapter, UBMS participants may be more serious about school than regular Upward Bound 

participants on average. This difference is reflected in our data: UBMS participants in our 

sample had higher GPAs and took more advanced math and science courses in ninth grade than 

members of the regular Upward Bound sample. We accounted for these differences by selecting 

a matched comparison group that resembled the UBMS participant sample in both grades earned 

and course-taking in ninth grade. However, it is possible that despite earning similar grades and 

taking similar courses early in high school, the UBMS participant sample is somewhat more 

serious about school, more serious about math and science, or is different from the matched 

comparison group in some other way that would lead our analysis to overstate the effects of 

UBMS. 

 47 For convenience, we use the terms “major” and “field of study” interchangeably. 
38 

Exhibit III.6 

Field of Study 

(Percentage of students) 

UBMS Participants Matched Comparison Group Impact 

All Postsecondary institutions 

Postsecondary Studies 

 Math or science 33 23 10*** 

 Social science 11 7 4* 

 Other 42 51 -9*** 

 

Postsecondary Studies Completed 

 Math or science 15 12 3 

 Social science 8 4 3* 

 Other 21 28 -7** 

Four-Year Colleges and Universities 

Postsecondary Studies 

 Math or science 28 18 10*** 

 Social science 10 7 4* 

 Other 36 39 -3 

 

Postsecondary Studies Completed 

 Math or science 12 6 6*** 

 Social science 6 4 3 

 Other 14 20 -6** 

*/**/*** Statistically significant at the 0.1 / 0.05 / 0.01 level. 

NOTES: Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison 

Group due to rounding. See Table B.9 in Appendix B for more details and additional outcomes and Tables 

B.10 through B.12 for estimates for subgroups of UBMS participants. 

While we cannot measure the extent of selection bias, some informed speculation is helpful 

in interpreting the impact estimates. As we indicated earlier, students in both groups—the 

UBMS participant sample and the matched comparison group—exhibited some motivation to 

improve academically by applying to participate in Upward Bound. Therefore, motivational 

differences between the two groups are likely to be small and unlikely to bias the impact 

estimates. However, the impact estimates would overstate the true impacts if: 

• Members of UBMS participant sample were higher achievers than members of 

matched comparison group. by the end of high school and after students in the 

UBMS sample had participated in UBMS, the average GPA was slightly but 

significantly higher for UBMS participants (3.14) than for matched comparison 

students (3.06). Even if the entire difference were attributable to selection bias 

instead of the effects of the program, the average student in each group was a B 

student. Therefore, if the UBMS participant sample contains higher achieving 

students than the matched comparison group, the difference seems to be small. 
39 

• Members of the UBMS participant sample were better at math and science than 

members of the matched comparison group. If students in the UBMS participant 

sample were better at math and science than students in the matched comparison 

group, it is not reflected in their math and science course-taking or grades during 

ninth grade: the two groups had similar grades in math and science, were equally 

likely to take algebra or geometry and were equally likely to take biology, chemistry 

or physics. While we did not explicitly examine whether UBMS participants were 

more likely to enroll in the advanced sections of courses before entering UBMS, 

other findings suggest that UBMS participants were no more likely to have taken 

Advanced Placement classes in math or science by the time they finished high 

school. Finally, focus groups conducted in 1996 suggest that UBMS participants do 

not view themselves as particular strong in math and science (Moore 1997, p. 28). 

Therefore, the information we have collected provides no reason to believe that when 

they entered UBMS, the participant sample was better at math and science than the 

matched comparison group. 

• Members of the UBMS participant group were more interested in math and 

science than members of the matched comparison group. None of the information 

that we extracted from student transcripts would suggest that the UBMS participant 

sample had greater interest in math and science. It is possible UBMS participants 

had greater interest in careers in math and science that simply was not reflected in 

their high school course-taking or grades early in high school. in 1996 focus groups, 

many UBMS participants expressed interest in pursuing careers in scientific fields, 

such as engineering, medicine, and nursing (Moore 1997, p. 28). However, many of 

these students indicated that their career interests had developed just that summer, 

and the expression of those interests could have been influenced by the fact that the 

focus groups were conducted on site at projects that emphasized math and science. 

We suspect that the impacts most vulnerable to selection bias are those that are most closely 

related to a person’s interest in pursuing careers in a math or science field. To gauge students’ 

interest in math and science early in high school would probably require conducting assessments 

or survey interviews at that time. However, we first interviewed members of the UBMS 

participant sample after they had completed high school. The possibility that UBMS participants 

might have had greater interest in pursuing a career in science than matched comparison students 

raises the question of whether our estimates overstate the effects of UBMS on the outcomes that 

are most closely related to one’s career interests, such as majoring in math or science in college. 

Therefore, while the findings in this report are promising, a note of caution is appropriate. 

We speculate that the selection bias is likely to be largest for outcome variables most closely tied 

one’s interest in pursuing math and science careers, but it is not possible to measure the selection 

bias. While we took several steps to reduce selection bias, the estimated effects of UBMS may 

overstate the true effects of the program. 

 

41

REFERENCES 

Cahalan, Maggie, Tim Silva, Justin Humphrey, Melissa Thomas and Kusuma Cunningham. 

Implementation of the Talent Search Program, Past and Present. Final Report from Phase I 

of the National Evaluation. Report prepared for the U.S. Department of Education, Planning 

and Evaluation Service. Washington, D.C.: Mathematica Policy Research, Inc., October 

2002. 

Curtin, Thomas R., and Margaret W. Cahalan. A Profile of the Upward Bound Math-Science 

Program: 2000-2001. Washington, D.C.: U.S. Department of Education, Office of 

Postsecondary Education, December 2004. 

Fasciano, Nancy J., and Jon E. Jacobson. “Grantee Survey Report” in A 1990s View of Upward 

Bound: Programs Offered, Students Served, and Operational Issues, a monograph prepared 

by Mathematica Policy Research, Inc., for the U.S. Department of Education, Office of the 

Under Secretary, as part of the National Evaluation of Upward Bound. Washington, D.C.: 

May 1997. 

Moore, Mary T. A 1990s View of Upward Bound: Programs Offered, Students Served, and 

Operational Issues. In the monograph of the same name, prepared by Mathematica Policy 

Research, Inc., for the U.S. Department of Education, Office of the Under Secretary, as part 

of the National Evaluation of Upward Bound. Washington, D.C., May 1997. 

Moore, Mary T. Developing Math and Science Skills Among Disadvantaged Youth: A Review of 

the Upward Bound Precollege Math/Science Centers. Report prepared for the U.S. 

Department of Education, Planning and Evaluation Service. Washington, D.C.: 

Mathematica Policy Research, Inc., September 1997. 

Myers, David. Measuring the Impact of the Upward Bound Math/Science Initiative: Feasibility 

of an Evaluation. Report for the Department of Education. Washington, D.C.: 

Mathematica Policy Research, Inc., May 7, 1997. 

Myers, David, Rob Olsen, and Neil Seftor. The Impacts of Upward Bound: High School 

Experiences and College Access. Presentation at Annual Conference, Council for 

Opportunity in Education, Washington, D.C., September 2002. 

Myers, David, Robert Olsen, Neil Seftor, Julie Young, and Christina Tuttle. The Impacts of 

Regular Upward Bound: Results from the Third Follow-up Data Collection. Report 

prepared for the U.S. Department of Education, Policy and Program Studies Service. 

Washington, D.C.: Mathematica Policy Research, Inc., April 2004. 

Myers, David, and Allen Schirm. The Impacts of Upward Bound: Final Report for Phase I of 

the National Evaluation. Prepared for the U.S. Department of Education, Planning and 

Evaluation Service. Washington, D.C.: Mathematica Policy Research, Inc., April 1999. 
42

National Science Foundation. “Table C-11. Employed U.S. Scientists and Engineers, by Level 

and Broad Field of Highest Degree Attained, Race/Ethnicity, and Years Since Degree: 

1995.” Available at http://srsstats.sbe.nsf.gov/preformatted-tables/1995/tables/tbC11.pdf, 

accessed June 2006. 

Rubin, Donald B. “Using Propensity Scores to Help Design Observational Studies: Application 

to the Tobacco Litigation.” Health Services and Outcomes Research Methodology, 2, pp. 

169-188, 2002. 

U.S. Census Bureau. Statistical Abstract of the United States: 2001. Washington, D.C., 

November 2001. 

U.S. Census Bureau. “DP-1. General Population and Housing Characteristics: 1990.” Online 

data table from 1990 Census. (http://factfinder.census.gov/servlet, accessed June 2006). 

U.S. Census Bureau. “QT-P1D. Age and Sex of the Black Population: 1990.” Online data 

table from 1990 Census (http://factfinder.census.gov/servlet, accessed June 2006). 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. NAEP 1999 Trends in Academic Progress: Three Decades 

of Student Performance. NCES 2000-469, by J.R. Campbell, C.M. Hombo, and J. Mazzeo. 

Washington, D.C., 2000. 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. Digest of Education Statistics 1998. NCES 1999-036. 

Washington, D.C., 1999. 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. Confronting the Odds: Students At Risk and the Pipeline to 

Higher Education. NCES 98-094. Washington, D.C., 1997. 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. The Condition of Education 1996. NCES 96-304. 

Washington, D.C., 1996a. 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. The Condition of Education 1994. NCES 94-149. 

Washington, D.C., 1994. 

U.S. Department of Education, Office of Educational Research and Improvement, National 

Center for Education Statistics. National Education Longitudinal Study of 1988. High 

School Seniors’ instructional Experiences in Science and Mathematics. NCES 95-278. 

Washington, D.C., 1996b. 
 

APPENDIX A 

DATA COLLECTION

A-3 

This appendix describes and assesses the procedures for collecting the data that we used to 

construct student outcome measures for the impact analysis presented in Chapter III. These data 

come from two different sources: 

1. The fourth follow-up survey of students 

2. Secondary and postsecondary transcripts 

This appendix focuses on procedures for obtaining completed interviews in the fourth 

follow-up survey and for collecting academic transcripts. 

A. Fourth Follow-Up Survey of Students 

The fourth follow-up survey was conducted between April 2001 and December 2002. It was 

designed to collect information on secondary and postsecondary educational outcomes 

approximately five to seven years after scheduled completion of high school. 

1. Data Collection Modes 

One week before we began interviewing, we sent a letter to all study participants. The letter 

indicated that we would call them to complete an interview for an important study, and it 

encouraged them to participate. In addition, the letter indicated that we would pay them $10 for 

completing the interview. 

Most interviews were administered using computer-assisted telephone interviewing (CATI). 

CATI interviews took about 30 minutes to complete. When a CATI interview was not possible, 

we attempted to obtain a completed questionnaire through the mail. Study participants were also 

offered the option of completing the survey on the Web. In June 2001, questionnaires were 

mailed to study participants that could not be reached by telephone. Three additional follow up 

mailings were conducted after the first mailing, with the last set of questionnaires being sent out 

in January 2002. 

2. Locating 

Throughout the data collection period, locating staff used services such as LexisNexis and 

Internet databases to obtain updated addresses and phone numbers for study participants that 

were difficult to reach. 

3. Incentives 

Financial incentives for survey completion were used to obtain a high response rate. Study 

participants were offered a $10 incentive for participating in the survey. Incentive checks were 

mailed after the sample member completed the interview. 
A-4 

4. Response Rates 

The eligible sample consisted of 1,759 UBMS participants and 2,830 sample members from 

the evaluation of regular Upward Bound. (See Chapter III for more details on the samples.) We 

obtained completed interviews for 1,425 UBMS participants and 2,146 regular Upward Bound 

sample members for response rates of 81 percent and 76 percent, respectively (see Table A.1). 

B. Transcript Data Collection 

Secondary and postsecondary transcripts were collected between July 2002 and March 2003. 

Academic transcripts provided the primary source of information on postsecondary achievement. 

Transcript requests were made from institutions that were reported by sample members in the 

fourth follow-up survey of students and in earlier surveys. 

1. Preparation for Requesting Transcripts 

Information about students’ secondary and postsecondary enrollment was primarily obtained 

from follow-up interviews. Students reported the secondary and postsecondary institutions that 

they had attended. Secondary transcripts were only requested from UBMS sample members 

selected for the impact analysis; postsecondary transcripts were requested from all sample 

members—both UBMS and regular Upward Bound—who reported or confirmed having 

attended a particular postsecondary institution.48 

To obtain mailing addresses for the schools that were attended by sample members, we 

matched schools that were reported by survey respondents to directories of secondary and 

postsecondary schools maintained by the U.S. Department of Education. Secondary schools 

were matched to the Common Core of Data (CCD); postsecondary schools were matched to the 

integrated Postsecondary Education Data System (IPEDS).49

 48 We did not collect secondary transcripts for regular Upward Bound sample members because we had already 

collected these transcripts for a large percentage of the sample in previous waves of data collection. 

49 Students were asked to provide the name and state of each secondary and postsecondary school they 

attended, but sometimes misspellings or incomplete information resulted in some invalid requests for student 

transcripts as schools were matched with an incorrect address and transcripts were requested from the wrong school. 

When a school indicated that they could not fill a request because they had no record of the student whose transcript 

we requested, it was sometimes due to such mismatches. In these cases, we attempted to learn the correct name and 

address of the school where the student was enrolled and make a new transcript request. 
A-5 

2. Procedures for Requesting Transcripts 

Each school was sent a transcript request packet that included: 

• A letter, printed on Department of Education letterhead, which explained the 

purpose of the study and the reason we were requesting transcripts. 

• A statement of Authorization and Confidentiality, which cited the Family 

Educational Rights and Privacy Act and included questions and answers regarding 

consent and confidentiality. 

• A transcript checklist of all the materials that we requested from the school, 

including student transcripts, a course catalog, grade descriptions and a transcript 

reimbursement form, which would indicate the reimbursement that the school 

required for providing the requested transcripts. 

• A postage-paid business reply envelope for sending the transcripts. 

• A disclosure notice to be placed in each student’s file, indicating that a copy of his 

or her transcript was released to Mathematica Policy Research as an agent to the 

U.S. Department of Education. 

3. Follow-Up Procedures 

For schools that did not respond to our initial request for transcripts, we mailed another 

request for student transcripts. These mailings were done periodically as we tracked the schools 

that had not yet sent the requested transcripts and corrected requests that contained errors. 

As the targeted end date for collecting transcripts approached, interviewers started calling 

schools directly to inquire about the status of our requests. Many schools responded to these 

calls by faxing us the requested transcripts. When the school indicated that they could not 

provide one or more of the requested transcripts, the interviewer completed a problem sheet 

indicating the reason. The reason generally fell into one of the following categories: 

Completed Interview 1,425 2,146 3,571

Eligible Nonrespondent 334 684 1,018

Ineligible - Deceased 7 14 21

Total 1,766 2,844 4,610

Table A.1

Fourth Follow-up Survey of Students

UBMS Sample Regular UB Sample Full Sample
A-6 

• The student was never enrolled at the school according to the school’s records. 

When this occurred, our first response was to call the school and provide more 

information on the student (e.g., provide or verify date of birth and dates of 

attendance) to see if a transcript could be located with additional information. In 

many cases, the school was able to locate and provide transcripts once additional 

information was provided. In other cases, the school provided some information 

that helped us determine where we might obtain the needed transcripts.50 If the 

school had no record of the student having ever attended and we were unable to 

obtain additional information, we marked the case as an invalid request. 

• Transcripts were held by the school district. Some schools only held the 

transcripts of currently enrolled students and all other transcripts were sent to the 

school district. in this situation, the school would sometimes forward the request 

packet to the district. Other times, the school returned the materials to us, and we 

sent them to the school district. 

• The student transferred to another school. When the student had transferred to 

another school, a transcript was requested from the school to which the student had 

transferred. In some cases, the registrar or school secretary forwarded the request 

materials to the transfer school. in other cases, the request materials were sent back 

to us and we sent a new request to the transfer school. 

• The school would not release any transcript without student’s written consent. A 

few schools returned the transcript request materials with no transcripts, indicating 

that they required written consent from each student whose transcript we were 

requesting. A problem sheet was completed for these cases, and they were 

forwarded to the survey manager for follow-up. As a first step, the survey manager 

called the school to explain that, as an agent of the Department of Education, 

Mathematica Policy Research was authorized to collect student transcripts for the 

purposes of this study and that, according to the laws of FERPA, schools are 

permitted to release student transcripts to the Department of Education without the 

written consent of students participating in the study. It was also explained that 

students had given verbal consent over the telephone or written consent when they 

completed the mail survey, and that we did not request transcripts for any students 

who refused consent. Some schools agreed to send the requested transcripts upon 

hearing this explanation. Others reiterated that signed consent was required by 

school policy. in this case, we sent written consent forms to the students for them 

to sign and return to Mathematica so that we could obtain their student transcript 

for the impact study. A postage-paid return envelope was included with the 

consent form. A small number of students did sign and return the consent form, 

but most of the letters came back unopened because we no longer had a valid 

address for the student. 

 50 For example, some school principals and registrars indicated that their school was often confused with 

another school having the same or a similar name and suggested that we direct our request to the other school. In 

this case, we would call the alternate school to find out if the student was ever enrolled there. If so, we made a 

correction to the database and sent a request to the newly identified school. 
A-7 

• The school would not release transcripts without advance payment. In these 

cases, we sent a check to cover the cost of each transcript, along with a list of the 

students whose transcripts we were requesting. 

• The school would not release a transcript until the student paid an outstanding 

debt. In some cases we were eventually able to obtain these transcripts as students 

paid whatever bills they owed the school. When the debt remained unpaid, 

however, there was no way we could get the transcript. These cases were marked 

as unfilled requests. 

4. Response Rates 

From the samples used in the impact analysis described in Chapter III, 1,365 students 

reported having attended at least one postsecondary institution that could be matched to IPEDS. 

for each of these 1,365 students, we requested a transcript from each of the postsecondary 

institutions that he or she reported attending. In total, we requested 2,029 transcripts. We 

received 1,821 of the 2,029 transcripts requested (90 percent), and we obtained a complete 

transcript record—that is, transcripts for all postsecondary institutions attended—for 1,109 

students (see Table A.2). 

 

Postsecondary Students UBMS Sample Regular UB Sample Full Sample

Complete Transcript Record 472 637 1,109

Incomplete Transcript Record 101 155 256

Total Postsecondary Students 573 792 1,365

Table A.2

Postsecondary Transcript Data Collection for the Upward Bound Math-Science Impact Analysis

APPENDIX B 

PROGRAM IMPACTS 

B-3 

 

Overall

Total Credits 25.7 24.5 1.2 ***

Grade Point Average (GPA) 3.1 3.1 0.1 *** 

Math Courses Taken 

Algebra (%) 97 97 0 !

Geometry (%) 86 86 0 

Algebra II (%) 73 74 -1 

Trigonometry (%) 35 30 5 

Analysis / Precalculus (%) 36 31 5 

Calculus (%) 26 24 2 

Advanced Placement Calculus (%) 6 10 -3 *

GPA in Algebra or above 2.8 2.7 0.1 *** 

Science Courses Taken 

Biology (%) 97 98 -1 

Chemistry (%) 88 78 10 ***

Physics (%) 58 43 15 ***

Advanced Placement Biology, Chemistry, or 

Physics (%) 4 9 -5 !

GPA in Biology, Chemistry, and Physics 2.9 2.7 0.2 *** 

High School Status (%) 

Graduated 99 96 2 ***

Dropped out 1 2 -2 !

General Educational Development (GED) 0 1 -1 !

Source: MPR analysis file "UBMSImpact6.sas."

*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.

Note 1: All estimates are weighted to account for missing data. 

Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

! Statistical significance could not be assessed due to complete or semi-complete separation.

Table B.1

Impact of Upward Bound Math Science on High School Outcomes

UBMS Participants Comparison Group Impact
B-4 

Overall

Total Credits 24.6 24.5 0.1 26.0 24.5 

Grade Point Average (GPA) 3.1 3.0 0.0 3.2 3.1 

Math Courses Taken

Algebra (%) 97 98 -1 ! 97 97 Geometry (%) 88 88 0 86 85 Algebra II (%) 66 78 -12 ** 74 73 Trigonometry (%) 38 27 11 ** 34 30 Analysis / Precalculus (%) 34 29 5 37 32 Calculus (%) 22 21 2 27 25 Advanced Placement Calculus (%) 6 7 -2 7 10 GPA in Algebra or above 2.7 2.6 0.1 2.8 2.7 

Science Courses Taken

Biology (%) 98 97 1 ! 97 98 Chemistry (%) 91 78 13 *** 88 79 Physics (%) 53 41 12 ** 59 43 Advanced Placement Biology, Chemistry, or Physics (%) 2 5 -4 ! 5 10 GPA in Biology, Chemistry, and Physics 2.8 2.7 0.1 ** 2.9 2.7 

High School Status (%)

Graduated 100 99 1 ! 99 96 Dropped out 0 1 -1 ! 1 3 General Educational Development (GED) 0 0 0 ! 1 1 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on students who participated in regular Upward Bound at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

UBMS 

Participants

Participated in Regular Upward Bound

UBMS 

Participants

Comparison 

Group Impact

Comparison 

Group

Table B.2

Impact of Upward Bound Math Science on High School Outcomes by Prior Participation in Regular Upward Bound

Did Not Participate in Regular Up
B-5 

Overall

Total Credits 25.3 24.6 0.7 26.1 24.5 

Grade Point Average (GPA) 3.1 3.0 0.1 * 3.2 3.1 

Math Courses Taken

Algebra (%) 97 98 0 ! 97 97 Geometry (%) 84 87 -3 88 84 Algebra II (%) 77 73 4 70 74 Trigonometry (%) 33 34 -1 36 28 Analysis / Precalculus (%) 42 35 7 32 28 Calculus (%) 25 22 3 27 25 Advanced Placement Calculus (%) 7 9 -2 6 10 GPA in Algebra or above 2.7 2.6 0.1 2.8 2.7 

Science Courses Taken

Biology (%) 98 99 -1 ! 97 98 Chemistry (%) 87 77 11 *** 89 80 Physics (%) 60 48 12 ** 56 39 Advanced Placement Biology, Chemistry, or Physics (%) 5 7 -2 ! 4 10 GPA in Biology, Chemistry, and Physics 2.9 2.6 0.3 *** 2.9 2.8 

High School Status (%)

Graduated 98 98 0 ! 99 96 Dropped out 1 1 0 ! 0 2 General Educational Development (GED) 1 1 0 ! 0 2 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on males at the 0.10 level.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).! Statistical significance could not be assessed due to complete or semi-complete separation.

UBMS 

Participants

Male

UBMS 

Participants

Comparison 

Group Impact

Comparison 

Group

Table B.3

Impact of Upward Bound Math Science on High School Outcomes by Sex

Female
B-6 

Overall

Total Credits 25.5 24.1 1.4 ** 26.0 25.2 0.8 26.4 2

Grade Point Average (GPA) 2.9 2.9 0.1 * 3.3 3.3 0.0 3.2 

Math Courses Taken

Algebra (%) 96 97 -1 ! 100 99 1 ! 96 Geometry (%) 88 89 -1 91 87 5 ** 81 Algebra II (%) 75 75 0 79 81 -2 ! 67 Trigonometry (%) 35 32 2 38 31 7 35 Analysis / Precalculus (%) 33 27 5 30 22 8 42 Calculus (%) 17 16 1 27 27 1 29 Advanced Placement Calculus (%) 4 4 0 ! 4 12 -8 ! 9 GPA in Algebra or above 2.5 2.4 0.1 3.2 3.1 0.1 2.7 

Science Courses Taken

Biology (%) 97 99 -2 98 98 0 ! 98 Chemistry (%) 90 83 7 ** 87 77 10 ! 87 Physics (%) 53 46 7 59 36 22 *** # 59 Advanced Placement Biology, Chemistry, or Physics (%) 4 7 -3 ! 2 6 -4 ! 6 GPA in Biology, Chemistry, and Physics 2.7 2.5 0.2 *** 3.3 3.0 0.3 *** 2.9 

High School Status (%)

Graduated 100 97 3 ! 99 97 2 ! 98 Dropped out 0 2 -2 ! 0 2 -2 ! 2 General Educational Development (GED) 0 0 0 ! 1 1 0 ! 0 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on African Americans at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Table B.4

Impact of Upward Bound Math Science on High School Outcomes by Race and Ethnicity

African American

UBMS 

Participants

Comparison 

Group Impact

Comparison 

Group

UBMS 

Participants

Comp

Gr

White

Impact

H

UBMS 

Participants
B-7 

Postsecondary Enrollment (%)

Any postsecondary institution 95 90 5 ***

Highest level of schooling attended 

 Four-year college or university 82 71 11 ***

 Two-year college 11 16 -5 **

 Vocational institution 2 4 -2 

College Selectivity (%) 

Most selective four-year college or university 

 More selective 33 23 11 ***

 Less selective 48 48 0 

Postsecondary Credits Earned (mean) 

Two- and four-year colleges and universities 98.1 85.0 13.1 ***

Four-year colleges and universities 86.6 73.1 13.5 **

Two-year colleges and universities 11.5 11.9 -0.5 

Postsecondary Completion (%) 

Any degree, certificate, or license 47 46 1 

Highest degree, certificate, or license earned 

 Bachelor's degree or higher 35 33 2 

 Associate's degree 7 7 -1 

 Certificate or license 5 5 -1 

Source: MPR analysis file "UBMSImpact6.sas."

*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.

Note 1: All estimates are weighted to account for missing data. 

Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

UBMS Participants Comparison Group Impact

! Statistical significance could not be assessed due to complete or semi-complete separation.

Table B.5

Impact of Upward Bound Math Science on Postsecondary Outcomes
B-8 

Postsecondary Enrollment (%)

Any postsecondary institution 99 91 8 *** 95 90 Highest level of schooling attended

 Four-year college or university 81 71 10 * 82 70  Two-year college 18 16 1 10 16  Vocational institution 0 4 -4 ! 3 4 

College Selectivity (%)

Most selective four-year college or university

 More selective 35 24 11 ** 33 22  Less selective 46 47 -1 48 48 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 103.4 84.2 19.2 ** 96.9 85.2 Four-year colleges and universities 90.1 69.2 20.9 ** 85.9 74.3 Two-year colleges and universities 13.3 15.0 -1.7 11.1 10.9 

Postsecondary Completion (%)

Any degree, certificate, or license 42 43 -2 48 46 Highest degree, certificate, or license earned

 Bachelor's degree or higher 31 29 2 36 34  Associate's degree 7 5 2 7 8  Certificate or license 4 8 -4 5 5 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on students who participated in regular Upward Bound at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Did Not Participate in R

Table B.6

UBMS 

Participants

Participated in Regular Upward Bound

UBMS 

Participants

Comparison 

Group Impact

Compariso

Group

Impact of Upward Bound Math Science on Postsecondary Outcomes by Prior Participation in Regular Upward Bou
B-9 

Postsecondary Enrollment (%)

Any postsecondary institution 96 89 7 *** 95 90 

Highest level of schooling attended

 Four-year college or university 82 76 6 82 68  Two-year college 12 12 0 10 18  Vocational institution 2 2 0 2 4 

College Selectivity (%)

Most selective four-year college or university

 More selective 34 23 11 ** 32 21  Less selective 48 52 -5 48 46 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 96.6 85.7 10.9 99.1 85.2 Four-year colleges and universities 83.2 73.5 9.7 88.9 72.7 Two-year colleges and universities 13.4 12.2 1.2 10.1 12.5 

Postsecondary Completion (%)

Any degree, certificate, or license 40 43 -3 51 46 

Highest degree, certificate, or license earned

 Bachelor's degree or higher 27 33 -6 40 32  Associate's degree 8 4 4 * 6 10  Certificate or license 5 5 0 5 5 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on males at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Comparison 

Group

Table B.7

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants

Female

Impact of Upward Bound Math Science on Postsecondary Outcomes by Sex

Male
B-10 

Postsecondary Enrollment (%)

Any postsecondary institution 97 93 5 ! 95 87 8 ! 95 

Highest level of schooling attended

 Four-year college or university 87 74 13 *** 79 67 12 ! 80  Two-year college 9 15 -6 12 19 -7 ! 14  Vocational institution 1 3 -2 ! 5 1 4 ! 0 

College Selectivity (%)

Most selective four-year college or university

 More selective 29 19 10 ** 24 11 13 *** 38 

 Less selective 57 54 3 54 55 0 42 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 105.9 87.0 18.9 ** 92.9 85.8 7.2 97.6 Four-year colleges and universities 99.4 79.2 20.2 ** 81.6 73.3 8.4 81.2 Two-year colleges and universities 6.5 7.8 -1.3 11.3 12.5 -1.2 16.4 

Postsecondary Completion (%)

Any degree, certificate, or license 49 46 3 46 35 10 ** 49 

Highest degree, certificate, or license earned

 Bachelor's degree or higher 38 36 1 36 19 17 *** # 33  Associate's degree 5 4 2 ! 8 10 -3 8  Certificate or license 6 6 -1 ! 2 6 -4 ! 8 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on African Americans at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Impact of Upward Bound Math Science on Postsecondary Outcomes by Race and Ethnicity

Table B.8

White

UBMS 

Participants

African American

Comparison 

Group Impact

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants
B-11 

Field of StudyField of Study at Most Recent PS Institution (%) All postsecondary institutions

 Math and science fields 33 23 

Social science fields 11 7 Other fields 42 51 

Four-year colleges and universities Math and science fields 28 18 

Social science fields 10 7 Other fields 36 39 

Earned Degree or Certificate in Field (%) All postsecondary institutions

 Math and science fields 15 12 

Social science fields 7 4 Other fields 21 28 

Four-year colleges and universities Math and science fields 12 6 

Social science fields 6 4 Other fields 14 20 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Table B.9

Impact of Upward Bound Math Science on Postsecondary Field of Study

! Statistical significance could not be assessed due to complete or semi-complete separation. 

UBMS Participants Comparison Group Imp
B-12 

Field of StudyField of Study at Most Recent PS Institution (%) All postsecondary institutions

 Math and science fields 35 26 10 * 32 22 

Social science fields 13 7 6 * 11 8 Other fields 38 47 -9 43 52 

Four-year colleges and universities Math and science fields 29 23 6 28 17 

Social science fields 12 6 6 10 7 Other fields 31 35 -3 37 40 

Earned Degree or Certificate in Field (%) All postsecondary institutions 

 Math and science fields 13 12 1 15 11 

Social science fields 6 3 3 7 4 Other fields 20 24 -4 22 29 

Four-year colleges and universities Math and science fields 8 10 -2 ! 13 5 

Social science fields 4 3 1 7 4 Other fields 15 13 2 14 22 

Source: UBMSImpact6.sas*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on students who participated in regular Upward Bound at the 0.10 level.

! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Did Not Participate in R

Comparison 

Group 

Table B.10

Impact of Upward Bound Math Science on Postsecondary Field of Study by Prior Participation in Regular Upward Bound

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants

Participated in Regular Upward Bound
B-13 

Field of StudyField of Study at Most Recent PS Institution (%) All postsecondary institutions

 Math and science fields 44 28 16 *** 24 19 

Social science fields 9 5 4 12 9 Other fields 31 46 -15 *** 50 53 

Four-year colleges and universities Math and science fields 38 25 13 ** 22 13 

Social science fields 8 4 5 * 12 9 Other fields 27 38 -12 ** 42 39 

Earned Degree or Certificate in Field (%) All postsecondary institutions 

 Math and science fields 16 10 6 * 14 11 

Social science fields 5 3 2 ! 8 5 Other fields 15 25 -10 ** 26 28 

Four-year colleges and universities Math and science fields 11 6 5 * 13 5 

Social science fields 4 3 1 ! 8 4 Other fields 9 19 -11 *** 18 20 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on males at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation. Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Comparison 

Group 

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants

Table B.11

Impact of Upward Bound Math Science on Postsecondary Field of Study by Sex

Male Female
B-14 

Field of StudyField of Study at Most Recent PS Institution (%) All postsecondary institutions

 Math and science fields 32 25 7 34 26 8 

Social science fields 13 6 6 ** 9 4 5 ! Other fields 42 59 -17 *** 42 50 -8 

Four-year colleges and universities Math and science fields 30 20 10 * 27 20 7 

Social science fields 13 6 7 ** 8 4 4 ! Other fields 38 44 -7 35 38 -4 

Earned Degree or Certificate in Field (%) All postsecondary institutions 

 Math and science fields 14 10 4 ! 16 9 6 

Social science fields 8 4 4 * 5 2 4 ! Other fields 21 32 -11 ** 21 20 1 # 

Four-year colleges and universities Math and science fields 13 6 7 ! 13 5 8 ! 

Social science fields 7 3 4 * 5 2 4 ! Other fields 15 23 -8 * 13 9 4 # 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on African Americans at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation. Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

U

Impact Par

UBMS 

Participants

Comparison 

Group Impact

Table B.12

Impact of Upward Bound Math Science on Postsecondary Field of Study by Race and Ethnicity

African American

White 

UBMS 

Participants

Comparison 

Group
APPENDIX C 

SENSITIVITY OF IMPACTS TO AN ALTERNATIVE MEASURE OF 

POSTSECONDARY ATTENDANCE 

C-3

A. Verification of Students’ Postsecondary Attendance 

In Chapter III, we presented estimates of the effects of UBMS on postsecondary attendance 

based on self-reported attendance—specifically, the schools that sample members reported 

attending after high school. However, in some instances, sample members may not have actually 

attended the schools they reported. For example, some may have reported schools that they 

planned to attend but never attended, and others may have reported schools at which they 

participated in a noncredit program but were not enrolled as students. In this appendix, we 

conduct an analysis to determine if the impact estimates presented in Chapter III are sensitive to 

an alternative measure of postsecondary attendance, one that requires verification by the 

postsecondary institution that students reported attending. 

Myers et al. (2004) present the results of this analysis in the text of the report along with the 

analysis based on self-reported data, which effectively provided two impact estimates for each 

outcome. When one of the two impact estimates was statistically insignificant, the results were 

characterized as inconclusive because at that time, we were unable to persuade ourselves that one 

method was more reliable than the other. However, we have since conducted an investigation of 

the verification process, and the results of this analysis indicate that on average, the self-reported 

measures of postsecondary attendance are more accurate than the measures requiring verification 

by the institution. While this may seem counterintuitive, in constructing postsecondary 

attendance measures that required verification, we assumed that students did not attend the 

schools they reported unless attendance was verified by the school, typically through the 

provision of a transcript. However, the transcript collection process was not originally designed 

to verify attendance.51 And in most instances when a school was unable or unwilling to provide 

a transcript, a careful examination of the data suggests that the student probably did attend the 

school—or at the very least, the school gave us no reason to doubt the student had attended when 

explaining why they would not provide a transcript. 

The verification process worked as follows. In the third and fourth follow-up surveys, 

students reported all postsecondary institutions that they had attended since high school. We 

then attempted to match all reported schools to the 1997-98 Integrated Postsecondary Education 

Data System (IPEDS) maintained by the National Center for Education Statistics (NCES) to 

determine whether they met NCES’s definition of a postsecondary institution and to obtain 

school contact information. Transcript requests were then sent to all schools a student reported 

attending that we were able to match to IPEDS and schools that did not initially respond were 

followed up. If a school provided a transcript that we requested for a student in the sample, then 

the student clearly attended that school. Furthermore, some of the reasons given by school staff 

for not providing transcripts can be treated as verification of attendance, for example, a college 

indicating that it could not provide a transcript for a student because he or she owed money to the 

school. In many instances, however, the reason given for not providing transcripts does not 

clearly indicate whether the student attended the school. For example, some schools required 

written consent from the students themselves even though the law does not require it, and we 

typically obtained only oral consent (see Appendix B); in these cases, lack of verification casts 

no doubt on the student’s self-reported attendance. 

 51 For the final report of this evaluation, which will use postsecondary transcript data collected in 2004-05, the 

transcript data collection process has been modified to specifically verify attendance if possible. 
C-4

The information obtained while collecting transcripts is therefore useful in verifying 

attendance in some but not all cases. In the vast majority of cases considered (approximately 80 

percent) we were able to verify the student’s attendance. However, for approximately 220 

students, the fact that we were unable to verify attendance for at least one of their reported 

postsecondary institutions changed at least one outcome of interest.52 In investigating these 

cases, we found little evidence that contradicted a student’s report of their postsecondary schools 

attended. One concern was that some Upward Bound participants would report the school at 

which they attended an Upward Bound summer bridge program but did not actually enroll; we 

found no evidence that this happened.53 Similarly, there was little evidence that many students 

reported attending institutions that are not classified by IPEDS as postsecondary institutions, and, 

in fact, non-IPEDS institutions such as Job Corps programs were excluded through the IPEDS 

match. The lack of verification more often appeared to be due to schools that would not release 

transcripts without written consent, ambiguities in the exact campus attended by the student 

within a large state system, or insufficient information about the school reported by the student to 

know which school to contact for transcript information. More often than not, lack of 

verification seemed to mainly reflect limitations of the verification process rather than 

inaccuracies in the information provided by students. 

B. Differences in Impacts 

This section presents estimates of the impacts of Upward Bound that use information only 

on schools that we were able to verify the student attended. These estimates make the strong 

assumption that lack of verification of attendance implies that the student did not attend the 

school in question or any school like it. 

Because most attendance reported by students was verified through transcript receipt, most 

of the impacts presented here are very similar to those estimated using information on all schools 

that students reported attending. There are a few differences between the estimates presented 

here and the estimates presented in Chapter III, but there is probably only one worth noting: the 

finding that UBMS has a positive effect on the likelihood of attending some type of 

postsecondary institution is sensitive to the verification process. If we require verification of 

postsecondary attendance, the estimated effect becomes statistically insignificant (compare Table 

C.1 to Table B.5). For the full set of estimates from the sensitivity analysis, see Tables C.1 - C.4. 

 52 There may be cases in which we were not able to verify a student’s attendance at all schools but for which 

that did not affect the outcome variables of interest. For example, if we verified a student’s attendance at a four-year 

college or university but could not verify his or her attendance at a vocational school they reported attending, he or 

she will still be classified as having their highest level of schooling as a four-year college or university. 

53 Students participate in Upward Bound summer bridge programs the summer before enrolling in college. 

Because the survey question simply asks students to list all postsecondary institutions attended since high school 

graduation, there was concern that some students may list the school at which they participated in a summer bridge 

program. 
C-5

Postsecondary Enrollment (%)

Any postsecondary institution 89 87 2 

Highest level of schooling attended 

 Four-year college or university 77 68 9 ***

 Two-year college 11 16 -5 **

 Vocational institution 1 3 -2 * 

College Selectivity (%) 

Most selective four-year college or university 

 More selective 30 21 9 ***

 Less selective 46 47 -1 

Postsecondary Credits Earned (mean) 

Two- and four-year colleges and universities 84.8 75.2 9.6 **

Four-year colleges and universities 75.3 64.8 10.5 **

Two-year colleges and universities 9.6 10.4 -0.8 

Postsecondary Completion (%) 

Any degree, certificate, or license 43 43 0 

Highest degree, certificate, or license earned 

 Bachelor's degree or higher 33 32 1 

 Associate's degree 6 7 -1 

 Certificate or license 4 4 1 

Source: MPR analysis file "UBMSImpact6.sas."

*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.

Note 1: All estimates are weighted to account for missing data. 

Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Table C.1

Impact of Upward Bound Math Science on Postsecondary Outcomes,

Excludes Unverified Enrollment and Completion

UBMS Participants Comparison Group Impact

! Statistical significance could not be assessed due to complete or semi-complete separation.
C-6 

Postsecondary Enrollment (%)

Any postsecondary institution 91 86 4 89 87 Highest level of schooling attended

 Four-year college or university 75 67 8 77 68  Two-year college 16 17 0 10 16  Vocational institution 0 2 -2 ! 1 3 

College Selectivity (%)

Most selective four-year college or university

 More selective 31 23 8 30 20  Less selective 44 43 0 46 47 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 83.5 65.7 17.8 *** 85.1 77.1 Four-year colleges and universities 72.8 53.9 18.9 ** 75.8 67.0 Two-year colleges and universities 10.7 11.7 -1.1 9.3 10.1 

Postsecondary Completion (%)

Any degree, certificate, or license 36 41 -6 45 43 Highest degree, certificate, or license earned

 Bachelor's degree or higher 25 28 -2 35 33  Associate's degree 6 5 1 6 7  Certificate or license 4 7 -2 4 3 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on students who participated in regular Upward Bound at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation. Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Impact of Upward Bound Math Science on Postsecondary Outcomes by Prior Participation in Regular Upward Bound,

Did Not Participate in Regular U

Table C.2

UBMS 

Participants

Comparison 

Group

Participated in Regular Upward Bound

UBMS 

Participants

Comparison 

Group Impact

Excludes Unverified Enrollment and Completion
C-7 

Postsecondary Enrollment (%)

Any postsecondary institution 88 88 1 89 85 Highest level of schooling attended

 Four-year college or university 74 74 0 78 64  Two-year college 13 13 0 10 18  Vocational institution 1 1 0 ! 1 4 

College Selectivity (%)

Most selective four-year college or university

 More selective 30 21 9 ** 30 19  Less selective 43 53 -9 * 47 44 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 80.2 76.3 3.9 88.1 73.9 Four-year colleges and universities 69.7 67.6 2.0 79.2 62.0 Two-year colleges and universities 10.6 8.7 1.9 8.9 11.9 

Postsecondary Completion (%)

Any degree, certificate, or license 37 42 -5 48 43 Highest degree, certificate, or license earned

 Bachelor's degree or higher 26 33 -7 38 31  Associate's degree 7 3 4 * 6 10  Certificate or license 4 4 0 4 3 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on males at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

Impact of Upward Bound Math Science on Postsecondary Outcomes by Sex,

Male

Comparison 

Group

Table C.3

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants

Excludes Unverified Enrollment and Completion

Female
C-8 

Postsecondary Enrollment (%)

Any postsecondary institution 91 89 1 ! 89 86 4 ! 

Highest level of schooling attended

 Four-year college or university 82 72 11 ** 74 67 7 !  Two-year college 8 16 -8 ** 14 19 -5 !  Vocational institution 1 3 -2 ! 2 1 1 ! 

College Selectivity (%)

Most selective four-year college or university

 More selective 26 17 9 ** 23 11 12 ***  Less selective 55 54 1 50 54 -5 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 88.5 73.8 14.8 ** 86.2 77.2 9.0 82

Four-year colleges and universities 83.3 66.7 16.7 ** 75.9 65.7 10.2 67Two-year colleges and universities 5.2 7.1 -1.9 10.3 11.4 -1.2 14

Postsecondary Completion (%)

Any degree, certificate, or license 44 41 3 43 35 9 * 4Highest degree, certificate, or license earned

 Bachelor's degree or higher 35 36 -1 35 19 16 *** #  Associate's degree 4 4 1 ! 8 10 -2  Certificate or license 5 2 3 ! 1 6 -5 ! 

Source: MPR analysis file "UBMSImpact6.sas."*/**/*** Impact estimate is statistically significant at the 0.10 / 0.05 / 0.01 level.# Impact estimate is significantly different from the impact on African Americans at the 0.10 level.! Statistical significance could not be assessed due to complete or semi-complete separation.Note 1: All estimates are weighted to account for missing data. Note 2: The comparison group estimates and impact estimates are regression-adjusted (see Chapter III, Section A.4).

UBM

Particip

African American

Comparison 

Group Impact

UBMS 

Participants

Comparison 

Group Impact

UBMS 

Participants

Impact of Upward Bound Math Science on Postsecondary Outcomes by Race and Ethnicity,

Excludes Unverified Enrollment and Completion

Table C.4

White
APPENDIX D 

SAMPLE SIZES AND STANDARD ERRORS 

D-3 

Overall

Total Credits 1,677 0.40

Grade Point Average (GPA) 1,677 0.02

Math Courses Taken 

Algebra (%) 1,677 0.70

Geometry (%) 1,677 2.35

Algebra II (%) 1,677 3.80

Trigonometry (%) 1,677 4.16

Analysis / Precalculus (%) 1,677 4.53

Calculus (%) 1,677 2.77

Advanced Placement Calculus (%) 1,677 1.43

GPA in Algebra or above 1,677 0.05

Science Courses Taken 

Biology (%) 1,677 0.91

Chemistry (%) 1,677 1.61

Physics (%) 1,677 3.07

Advanced Placement Biology, Chemistry, or 

Physics (%) 1,677 1.38

GPA in Biology, Chemistry, and Physics 1,677 0.04

High School Status (%) 

Graduated 1,677 0.51

Dropped out 1,677 0.37

General Educational Development (GED) 1,677 0.38

Source: MPR analysis file "UBMSImpact6.sas."

Note: Standard errors account for project clustering and were estimated using Taylor series 

linearization methods.

Table D.1

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.1

Sample Size Standard Error
D-4 

Overall

Total Credits 640 0.52 1,037 0.43

Grade Point Average (GPA) 640 0.03 1,037 0.03

Math Courses Taken

Algebra (%) 640 1.10 1,037 0.63Geometry (%) 640 2.84 1,037 2.57Algebra II (%) 640 5.85 1,037 3.66Trigonometry (%) 640 6.08 1,037 4.59Analysis / Precalculus (%) 640 5.55 1,037 4.91Calculus (%) 640 4.21 1,037 3.17Advanced Placement Calculus (%) 640 2.55 1,037 1.69GPA in Algebra or above 640 0.08 1,037 0.05

Science Courses Taken

Biology (%) 640 1.05 1,037 1.05Chemistry (%) 640 2.44 1,037 1.88Physics (%) 640 5.27 1,037 3.43Advanced Placement Biology, Chemistry, or Physics (%) 640 1.44 1,037 1.66GPA in Biology, Chemistry, and Physics 640 0.06 1,037 0.05

High School Status (%)

Graduated 640 0.00 1,037 0.64Dropped out 640 0.00 1,037 0.47General Educational Development (GED) 640 0.00 1,037 0.49

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.2

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.2

Did Not Participate in Regular 

Sample Size Standard Error

Participated in Regular Upward 

Sample Size Standard Error
D-5 

Overall

Total Credits 578 0.47 1,098 0.47

Grade Point Average (GPA) 578 0.04 1,098 0.03

Math Courses Taken

Algebra (%) 578 1.05 1,098 0.77Geometry (%) 578 2.91 1,098 2.49Algebra II (%) 578 4.40 1,098 4.43Trigonometry (%) 578 5.84 1,098 4.64Analysis / Precalculus (%) 578 4.87 1,098 5.10Calculus (%) 578 4.47 1,098 3.29Advanced Placement Calculus (%) 578 2.62 1,098 1.90GPA in Algebra or above 578 0.08 1,098 0.05

Science Courses Taken

Biology (%) 578 1.04 1,098 1.14Chemistry (%) 578 2.62 1,098 1.88Physics (%) 578 5.19 1,098 4.10Advanced Placement Biology, Chemistry, or Physics (%) 578 2.23 1,098 1.68GPA in Biology, Chemistry, and Physics 578 0.07 1,098 0.05

High School Status (%)

Graduated 578 0.84 1,098 0.66Dropped out 578 0.88 1,098 0.32General Educational Development (GED) 578 0.09 1,098 0.59

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.3

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.3

Female

Sample Size Standard Erro

Male

Sample Size Standard Error
D-6 

Overall

Total Credits 633 0.55 522 0.82 

Grade Point Average (GPA) 633 0.04 522 0.03 

Math Courses Taken

Algebra (%) 633 1.19 522 0.00 Geometry (%) 633 2.82 522 2.07 Algebra II (%) 633 5.78 522 5.67 Trigonometry (%) 633 5.48 522 7.27 Analysis / Precalculus (%) 633 6.47 522 6.71 Calculus (%) 633 3.87 522 4.23 Advanced Placement Calculus (%) 633 2.46 522 1.78 GPA in Algebra or above 633 0.08 522 0.06 

Science Courses Taken

Biology (%) 633 1.69 522 1.33 Chemistry (%) 633 2.67 522 3.26 Physics (%) 633 5.79 522 6.16 Advanced Placement Biology, Chemistry, or Physics (%) 633 2.43 522 2.22 GPA in Biology, Chemistry, and Physics 633 0.06 522 0.06 

High School Status (%)

Graduated 633 0.00 522 1.09 Dropped out 633 0.00 522 0.00 General Educational Development (GED) 633 0.00 522 0.84 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.4

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.4

African American

Sample Size Standard Error Standard Error Samp

White 

Sample Size
D-7 

Postsecondary Enrollment (%)

Any postsecondary institution 1,438 0.99

Highest level of schooling attended 

 Four-year college or university 1,438 2.28

 Two-year college 1,438 1.99

 Vocational institution 1,438 0.96

College Selectivity (%) 

Most selective four-year college or university 

 More selective 1,438 3.44

 Less selective 1,438 4.05

Postsecondary Credits Earned (mean) 

Two- and four-year colleges and universities 1,158 5.0

Four-year colleges and universities 1,158 5.7

Two-year colleges and universities 1,158 2.3

Postsecondary Completion (%) 

Any degree, certificate, or license 1,438 3.54

Highest degree, certificate, or license earned 

 Bachelor's degree or higher 1,438 3.70

 Associate's degree 1,438 1.70

 Certificate or license 1,438 1.52

Source: MPR analysis file "UBMSImpact6.sas."

Note: Standard errors account for project clustering and were estimated using Taylor series linearization 

methods.

Table D.5

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.5

Sample Size Standard Error
D-8 

Postsecondary Enrollment (%)

Any postsecondary institution 552 1.15 886 

Highest level of schooling attended

 Four-year college or university 552 4.54 886  Two-year college 552 4.89 886  Vocational institution 552 0.00 886 

College Selectivity (%)

Most selective four-year college or university

 More selective 552 5.56 886  Less selective 552 5.88 886 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 432 7.40 726 Four-year colleges and universities 432 8.73 726 Two-year colleges and universities 432 4.71 726 

Postsecondary Completion (%)

Any degree, certificate, or license 552 5.88 886 Highest degree, certificate, or license earned

 Bachelor's degree or higher 552 5.19 886  Associate's degree 552 2.77 886  Certificate or license 552 2.08 886 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.6

Sample Size 

Participated in Regular Upward 

Sample Size Standard Error

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.6

Did Not Particip
D-9 

Postsecondary Enrollment (%)

Any postsecondary institution 480 1.62 957 Highest level of schooling attended

 Four-year college or university 480 3.66 957  Two-year college 480 3.80 957  Vocational institution 480 1.11 957 

College Selectivity (%)

Most selective four-year college or university

 More selective 480 5.25 957 

 Less selective 480 5.76 957 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 377 7.80 780 

Four-year colleges and universities 377 8.97 780 Two-year colleges and universities 377 4.73 780 

Postsecondary Completion (%)

Any degree, certificate, or license 480 5.17 957 Highest degree, certificate, or license earned

 Bachelor's degree or higher 480 4.70 957  Associate's degree 480 3.35 957  Certificate or license 480 2.43 957 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.7

Sample Size Standard Error Sample Size

Fema

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.7

Male
D-10 

Postsecondary Enrollment (%)

Any postsecondary institution 523 1.46 468 1.99 

Highest level of schooling attended

 Four-year college or university 523 3.28 468 4.52  Two-year college 523 3.37 468 3.98  Vocational institution 523 1.00 468 2.58 

College Selectivity (%)

Most selective four-year college or university

 More selective 523 5.41 468 5.92  Less selective 523 6.33 468 6.07 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 405 7.24 394 8.10 Four-year colleges and universities 405 8.08 394 9.81 Two-year colleges and universities 405 2.63 394 4.33 

Postsecondary Completion (%)

Any degree, certificate, or license 523 6.30 468 5.01 Highest degree, certificate, or license earned

 Bachelor's degree or higher 523 6.09 468 4.96  Associate's degree 523 3.29 468 3.46  Certificate or license 523 2.50 468 1.23 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.8

White

African American

Sample Size Standard Error Sample Size Sam

Standard Error

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.8
D-11 

Field of Study at Most Recent PS Institution

 All PS institutions

 Math and science fields 1,438 3.73

Social science fields 1,438 2.37

Other fields 1,438 3.33 

Four-year colleges and universities 

 Math and science fields 1,438 3.53

Social science fields 1,438 2.37

Other fields 1,438 3.23 

Earned Degree or Certificate in Field 

 All PS institutions

 Math and science fields 1,438 2.84

Social science fields 1,438 2.13

Other fields 1,438 2.69 

Four-year colleges and universities 

 Math and science fields 1,438 2.78

Social science fields 1,438 2.17

Other fields 1,438 2.56

Source: MPR analysis file "UBMSImpact6.sas."

Note: Standard errors account for project clustering and were estimated using Taylor series 

linearization methods.

Table D.9

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.9

Sample Size Standard Error
D-12 

Field of Study at Most Recent PS Institution All PS institutions

 Math and science fields 552 5.98 886 

Social science fields 552 4.70 886 Other fields 552 6.10 886 

Four-year colleges and universities Math and science fields 552 5.76 886 

Social science fields 552 4.55 886 Other fields 552 5.72 886 

Earned Degree or Certificate in Field All PS institutions 

 Math and science fields 552 3.99 886 

Social science fields 552 2.90 886 Other fields 552 4.59 886 

Four-year colleges and universities Math and science fields 552 2.89 886 

Social science fields 552 2.94 886 Other fields 552 3.75 886 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.10

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.10

Sample Size 

Participated in Regular Upward Did Not Parti

Sample Size Standard Error
D-13 

Field of Study at Most Recent PS Institution All PS institutions

 Math and science fields 480 6.16 957 

Social science fields 480 4.21 957 Other fields 480 4.81 957 

Four-year colleges and universities Math and science fields 480 5.94 957 

Social science fields 480 3.72 957 Other fields 480 4.55 957 

Earned Degree or Certificate in Field All PS institutions 

 Math and science fields 480 3.87 957 

Social science fields 480 2.42 957 Other fields 480 3.69 957 

Four-year colleges and universities Math and science fields 480 3.66 957 

Social science fields 480 2.18 957 Other fields 480 2.84 957 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.11

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.11

Sample Size 

Male Fe

Sample Size Standard Error
D-14 

Field of Study at Most Recent PS Institution All PS institutions

 Math and science fields 523 5.84 468 6.00 

Social science fields 523 3.42 468 3.23 Other fields 523 5.26 468 5.09 

Four-year colleges and universities Math and science fields 523 6.25 468 4.94 

Social science fields 523 3.63 468 3.18 Other fields 523 4.68 468 4.48 

Earned Degree or Certificate in Field All PS institutions 

 Math and science fields 523 4.64 468 5.17 

Social science fields 523 3.11 468 3.08 Other fields 523 4.70 468 3.67 

Four-year colleges and universities Math and science fields 523 5.85 468 4.82 

Social science fields 523 3.06 468 3.08 Other fields 523 4.10 468 3.66 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.12

Sample Sizes and Standard Errors for Reported Impact Estimates: Table B.12

African American 

Sam

White

Sample Size Standard Error Sample Size Standard Error
D-15 

Postsecondary Enrollment (%)

Any postsecondary institution 1,438 1.75

Highest level of schooling attended 

 Four-year college or university 1,438 2.54

 Two-year college 1,438 1.97

 Vocational institution 1,438 0.70

College Selectivity (%) 

Most selective four-year college or university 

 More selective 1,438 3.34

 Less selective 1,438 3.78

Postsecondary Credits Earned (mean) 

Two- and four-year colleges and universities 1,438 4.53

Four-year colleges and universities 1,438 4.86

Two-year colleges and universities 1,438 1.78

Postsecondary Completion (%) 

Any degree, certificate, or license 1,438 3.50

Highest degree, certificate, or license earned 

 Bachelor's degree or higher 1,438 3.71

 Associate's degree 1,438 1.61

 Certificate or license 1,438 1.53

Source: MPR analysis file "UBMSImpact6.sas."

Note: Standard errors account for project clustering and were estimated using Taylor series 

linearization methods.

Table D.13

Sample Sizes and Standard Errors for Reported Impact Estimates: Table C.1

Sample Size Standard Error
D-16 

Postsecondary Enrollment (%)

Any postsecondary institution 552 3.46 886 

Highest level of schooling attended

 Four-year college or university 552 5.18 886  Two-year college 552 4.89 886  Vocational institution 552 0.00 886 

College Selectivity (%)

Most selective four-year college or university

 More selective 552 5.18 886  Less selective 552 5.87 886 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 552 6.67 886 Four-year colleges and universities 552 7.39 886 Two-year colleges and universities 552 3.36 886 

Postsecondary Completion (%)

Any degree, certificate, or license 552 5.98 886 Highest degree, certificate, or license earned

 Bachelor's degree or higher 552 4.97 886  Associate's degree 552 2.77 886  Certificate or license 552 2.02 886 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.14

Sample Size S

Participated in Regular Upward 

Sample Size Standard Error

Sample Sizes and Standard Errors for Reported Impact Estimates: Table C.2

Did Not Participa
D-17 

Postsecondary Enrollment (%)

Any postsecondary institution 480 3.07 957 

Highest level of schooling attended

 Four-year college or university 480 4.09 957  Two-year college 480 3.90 957  Vocational institution 480 0.81 957 

College Selectivity (%)

Most selective four-year college or university

 More selective 480 5.11 957  Less selective 480 5.59 957 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 480 6.71 957 Four-year colleges and universities 480 7.10 957 Two-year colleges and universities 480 2.95 957 

Postsecondary Completion (%)

Any degree, certificate, or license 480 4.92 957 Highest degree, certificate, or license earned

 Bachelor's degree or higher 480 4.57 957  Associate's degree 480 2.87 957  Certificate or license 480 2.21 957 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Male

Stand

Table D.15

Sample Size Standard Error Sample Size

Female

Sample Sizes and Standard Errors for Reported Impact Estimates: Table C.3
D-18 

Postsecondary Enrollment (%)

Any postsecondary institution 523 2.64 468 3.43 Highest level of schooling attended

 Four-year college or university 523 3.79 468 4.96  Two-year college 523 2.98 468 4.74  Vocational institution 523 0.96 468 1.52 

College Selectivity (%)

Most selective four-year college or university

 More selective 523 4.93 468 5.85 

 Less selective 523 5.90 468 6.13 

Postsecondary Credits Earned (mean)

Two- and four-year colleges and universities 523 7.08 468 7.12 Four-year colleges and universities 523 7.25 468 8.83 Two-year colleges and universities 523 2.10 468 4.00 

Postsecondary Completion (%)

Any degree, certificate, or license 523 6.33 468 4.84 Highest degree, certificate, or license earned

 Bachelor's degree or higher 523 6.12 468 4.94  Associate's degree 523 3.18 468 3.47  Certificate or license 523 2.70 468 0.99 

Source: MPR analysis file "UBMSImpact6.sas."Note: Standard errors account for project clustering and were estimated using Taylor series linearization methods.

Table D.16

White

African American

Sample Size Standard Error Sample Size Sam

Standard Error

Sample Sizes and Standard Errors for Reported Impact Estimates: Table C.4
The Department of Education’s mission is to provide equal access to education 

and promote educational excellence throughout the nation.

Our mission is to provide equal access to education 

and promote educational excellence throughout the nation.