### Upward Bound Grantee-level Performance Results: 2004-05

**Why is the U.S. Department of Education calculating grantee-level
performance measures?**

The Department is committed to continually improving its management of
programs and improving the educational outcomes of students.
Improvements are guided by monitoring and evaluating performance,
improving the data used for evaluation, collaborating with stakeholders,
implementing recommendations, and re-evaluating performance. The Office
of Management and Budget's [Program Assessment Rating
Tool](http://www.whitehouse.gov/omb/expectmore/part.html), or PART, is
one tool for assessing the performance of every Federal program to make
sure it is working well for the American people. Providing data to the
public is a key element in promoting improvement and collaborating with
stakeholders.

**What are the performance measures for Upward Bound projects?**

For regular Upward Bound (UB) and Upward Bound Math-Science (UBMS)
projects, the performance measures are the postsecondary school
enrollment rate and the gap between the cost per high school persister
and college enrollee, on the one hand, and the cost per program
participant, on the other. The attached table shows for each UB and UBMS
project the number of participants who were expected (see explanation
below) to graduate from high school in 2003--04 and the number and
percentage of those who subsequently enrolled in a postsecondary
education program by fall of 2005. That semester was used rather than
fall 2004 because projects do not necessarily become aware of prior
participants' postsecondary enrollments until a year or more after the
students' high school graduation. Enrollment data submitted by projects
can be supplemented or cross checked by postsecondary financial aid
data, which indicates whether a student is enrolled if the student
received financial aid. Unfortunately, postsecondary financial aid data
are not available for analysis until approximately two years after high
school graduation. Projects\' awareness of postsecondary enrollment
rates thus tends to increase over several years.

Table 1 shows all regular UB and UBMS projects that were funded in both
the 1999--03 and the 2003--07 cycles alphabetically by grantee name.
Projects funded in the first but not the second funding cycle were
excluded because the projects no longer exist and therefore no longer
provide performance report data on their participants. Projects that
were first funded in 2003--04 (i.e., those that were starting a new UB
or UBMS project and not continuing a previously established one) were
excluded because they had very few participants expected to graduate
from high school in 2003--04 and because the participants who were
expected to graduate received only one year of Upward Bound services.
Projects do not typically recruit high school seniors. Consequently, 674
UB and 110 UBMS grantees were included in the table; 53 UB (6 percent of
UB projects) and 11 UBMS (8 percent of UBMS projects) projects funded
only in the first funding cycle and 90 UB (11 percent of UB projects)
and 17 UBMS (12 percent of UB projects) projects funded only in the
second funding cycle were excluded.

## How did the U.S. Department of Education calculate the Upward Bound project-level performance measure for 2004--05?

The project's postsecondary enrollment rate is calculated by dividing
the number of participants who were expected to graduate from high
school during 2003--04 who have evidence of postsecondary enrollment by
the total number of participants who were expected to graduate from high
school in 2003--04 and multiplying by 100.

These analyses examine participants who were expected to graduate from
high school during the 2003--04 academic year. Participants were
included in this group if their expected high school graduation date
reported by projects on the APR was during 2003--04. For cases missing
this information, expected high school graduation year was derived using
project entry date and grade level at the time of project entry.

Data from Upward Bound Annual Performance Reports (APRs), which are
submitted by UB grantees, supplemented by data from the federal
financial aid database, are used to calculate postsecondary school
enrollment rates. The following APR fields were used to determine a
participant's postsecondary enrollment status: postsecondary grade level
(field #97, values 1 through 11 and 77), enrollment status (field #96,
values 1 through 3), financial aid (field #95, values 1 through 11), or
institution code (fields #93 and #94, values greater than 000000 except
888888 and 999999). Additionally, the participant was classified as
having enrolled in postsecondary education if financial aid was received
(total disbursement greater than \$0) according to federal financial aid
records. If there is any evidence of postsecondary enrollment in any of
these fields, a participant was considered to have enrolled in a
postsecondary program. (The APR may be found at
<http://www.ed.gov/programs/trioupbound/report.html>.)

**What do the data show and what might account for the variation in
postsecondary enrollment rates among grantees?**

First, it is important to note that the postsecondary enrollment rate is
an outcome measure of project performance. The limitations of the data
set used for this analysis (the annual performance reports) do not
permit us to determine project impacts, such as the extent to which the
postsecondary enrollment rate can be attributed to participation in UB.

Postsecondary enrollment rates range from 14 percent to 100 percent for
the projects (Figures 1a and 1b). Nearly two-thirds of all regular UB
projects had postsecondary enrollment rates of 70 percent or higher;
approximately 70 percent of UBMS projects had postsecondary enrollment
rates of 80 percent or higher. An individual project director may wish
to consider his or her participants\' enrollment rates as shown in the
table in the context of the frequency distributions shown in figures 1a
and 1b. The Department's program goal for postsecondary enrollment is 65
percent.

While a number of factors are doubtless involved, at this time we want
to draw attention to the role of **students\' duration in the program**.
Particularly in regular UB, if participants stay in the program for a
long period of time, they tend to have higher postsecondary enrollment
rates, as is clear from figure 2. To show how this relationship is
manifested at the level of the individual grantee, table 1 includes the
percentage of participants in each project who were enrolled in the
program for 36 months or more and the postsecondary enrollment rate for
those participants. The role of persistence in the program is
observable, but less pronounced, in UBMS. The difference is presumably
due to inherent differences in the programs. Some UBMS projects operate
as regional centers that typically serve students for only one or two
summer sessions; also, the percentage of UBMS students entering the
program as freshmen is smaller than that of regular UB (12.8 percent and
17.7 percent, respectively). Thus, for many UBMS projects, program
participation for 36 months or more may not be a useful comparison.

Because we do not fully understand the role of the various factors that
may affect postsecondary enrollment rates in individual projects, **the
data should be interpreted with caution; comparing rates among specific
projects could lead to unwarranted conclusions.** For example, a project
may have a lower than average postsecondary enrollment rate because the
project may be serving more students with a high risk of academic
failure, who have low educational aspirations, and/or who are in low
performing high schools.

Four important data issues that also could significantly affect outcomes
are these:

• For some projects, only a small number of students were expected to
graduate in 2003--04. Where only a small number of graduates exist,
small changes in numbers can cause significant changes in percentages.
For example, a grantee that expects five students to graduate in 2004
will have an enrollment rate of 100 percent if all enroll in
postsecondary education, but a rate of only 80 percent if just one
student does not matriculate.

-   A couple of projects did not submit complete data on all students
    served by the project, resulting in no students included in the
    number of expected high school graduates for 2003-04.

• Some projects experienced difficulty in following up with prior
participants and thus may have been unaware of enrollments that did, in
fact, occur.

• Additional years of data could significantly change the picture.