**8** | MCNAIR POSTBACCALAUREATE ACHIEVEMENT PROGRAM PERFORMANCE RESULTS: 2019–20

U.S. Department of EducationOffice of Postsecondary EducationHigher Education Programs Student ServiceFederal TRIO ProgramsRonald E. McNair Postbaccalaureate Achievement ProgramProject and Program Measures for Government Performance and Results Act (GPRA)Reporting Period: 2020-21

Table of ContentsIntroduction	1Program Performance	1Program Targets	1Outcome Summary	2Performance Measure Definitions	2Research Doctorate Completion Rate Definition	2Grantee-level Data	3Findings	4Data Limitations	5Appendix: Calculation Rules	7Graduate School Enrollment Rate	7Graduate School Persistence Rate	7Research Doctorate Completion Rate	8

## Introduction

This document presents the program performance for the Ronald E. McNair Postbaccalaureate Achievement Program (McNair) in the 2020–21 reporting year for *Government Performance and Results Act* (GPRA) reporting. McNair program performance is measured by two student achievement outcomes: graduate school enrollment within three years of bachelor’s degree completion and one-year graduate school persistence. These rates are calculated at the grantee- and program-levels.

This document provides a description of the methodology and data used to calculate and analyze the three-year graduate school enrollment rates, the one-year graduate school persistence rates, and the research doctorate completion rates of McNair projects; and a summary of the preliminary findings. The accompanying Excel workbook contains grantee- and program-levels data describing graduate school enrollment (Table 1); graduate school persistence (Table 2); and program-level data describing research doctorate completion (Table 3). The analyses were based on the data provided by the grantees in their *Annual Performance Reports* (APRs) for 2014–15 through 2020–21. Data should be interpreted with caution given considerations described in the section below on data constraints.

## Program Performance

### Program Targets

The U.S. Department of Education is committed to continually improving its management of programs and advancing the educational outcomes of participating students. Improvements are guided by monitoring and assessing performance; improving the data used for these assessments; collaborating with stakeholders; implementing recommendations; and reassessing performance. Providing data to the public is a key element in promoting improvement and collaborating with stakeholders.

The 2020–21 performance measures for the McNair program are measurable educational outcomes for the projects funded by the program. McNair has two specific targets for this reporting year:

1. Three-year graduate school enrollment rate: 71.0 percent of participants who received their bachelor’s degree in 2017–18 enroll in graduate degree programs within three years; and

2. One-year graduate school persistence rate: 85.0 percent of participants reported for the first time as graduate students in 2019–20 persist in their graduate studies into the 2020–21 academic year.

In addition to these two performance targets, program-level, research doctorate completion rates were calculated for the 2002–03 through 2010–11 bachelor’s degree cohorts.[footnoteRef:2] McNair did not specify a target for the research doctorate completion rate. [2:  Research doctorate degrees include Doctor of Philosophy, Doctor of Education, and other research-intensive doctorates (e.g., Doctor of Engineering, Doctor of Science, Doctor of Nursing Science, and Doctor of Public Health).] 

### Outcome Summary

The cumulative three-year graduate school enrollment rate was 64.1 percent, 6.9 percentage points lower than the McNair program’s target goal of 71.0 percent (Table 1). The overall graduate school persistence rate for participants who were first-year graduate students in 2019–20 (i.e., the percentage of this cohort who persisted in graduate school into 2020–21) was 85.7 percent, 0.7 percentage points higher than the McNair program’s 2020–21 target goal of 85.0 percent (Table 2). The overall 2020–21 research doctorate completion rate for the McNair participants who earned bachelor’s degrees in 2010–11 was 14.5 percent (Table 3).

#### Performance Measure Definitions

Because the McNair program prepares students who are traditionally underrepresented among doctoral degree recipients, a primary performance measure is the graduate school enrollment rate of program participants. The three-year graduate school enrollment rate is the percentage of McNair bachelor's degree recipients who enroll in graduate school anytime during the three academic years following attainment of their bachelor’s degree. Although this three-year span will not capture all program successes (as some participants enroll many years following college graduation), the three-year time frame has been established for measuring and comparing program outcomes among currently funded McNair projects. Table 1 presents the three-year graduate school enrollment rate of the 2017–18 cohort of McNair participants, based on bachelor’s degree attainment.

An additional performance measure for the McNair program is the one-year graduate school persistence rate, which is defined as the percentage of participants who were reported for the first time as enrolled in graduate school and who were still enrolled in the following academic year. In combination with graduate school enrollment, graduate school persistence may serve as evidence of successful undergraduate preparation for graduate education, as poorly prepared students are less likely to persist. For all currently funded McNair grantees, the graduate school persistence rate is the rate at which participants who were first reported as graduate students during academic year 2019–20 and were still enrolled in graduate school in the 2020–21 academic year (Table 2).

#### Research Doctorate Completion Rate Definition

The purpose of the McNair program is to prepare participants for research doctorate degrees, including Doctor of Philosophy, Doctor of Education, and other research-intensive doctorate degrees (e.g., Doctor of Engineering, Doctor of Science, Doctor of Nursing Science, Doctor of Public Health). Grantees track participants for 10 years following bachelor’s degree completion in order to report information on their highest degree earned, including completion of research doctorate degrees. However, this measure underreports true program successes, as it often requires more than 10 years after bachelor’s degree attainment to earn a doctorate degree. The research doctorate completion rate**is defined as the percentage of participants who had earned research doctorate degrees within 10 years of earning bachelor’s degrees. Table 3 displays the McNair program-level, research doctorate completion rates for the 2002–03 through 2010–11 bachelor’s degree cohorts.

## Grantee-level Data 

The accompanying Excel workbook contains grantee-level data describing enrollment and persistence for all grantees included in these results and summarizes the rate of doctoral-degree completion.

The data in Table 1 (enrollment) and Table 2 (persistence) in the Excel workbook are organized alphabetically by state and grantee name for all McNair grantees funded in 2020–21 (total 187 grantees). Table 1 contains a note column to identify those grantees that were first funded in 2017–18 (and therefore did not have any participants in the 2017–18 bachelor’s degree cohort); the number of bachelor’s degree recipients in 2017–18 was fewer than four; and grantees that had relatively low (less than 35 percent) graduate school enrollment rates (see the summary in Exhibit 1 below). 

**Exhibit 1. *Number of Grantees by the Content of Note in the Enrollment Table***

Table 2 in the Excel workbook also contains a note column to identify those grantees that were first funded in 2017–18, those that did not have first-time reported graduate students in 2019–20, and those that had relatively small cohorts of first-time reported graduate students in 2019–20 (see the summary in Exhibit 2 below). 

****

**Exhibit 2. *Number of Grantees by the Content of Note in the Persistence Table***

Table 3 (completion) describes the total number of research doctorates earned for nine cohorts of participants. Due to the risks of data disclosure, many of the values in the tables are either suppressed or recoded if the denominator or numerator has three or fewer participants and/or the difference between them is three or less. However, the patterns of findings are summarized in the section below.

## Findings

The overall three-year graduate school enrollment rate for McNair participants who received their bachelor’s degree in 2017–18 was 64.1 percent, a decrease of 0.6 percentage points from the 2016–17 cohort’s three-year rate of 64.7 percent.[footnoteRef:3] This enrollment rate was 6.9 percentage points lower than the McNair program’s target goal of 71.0 percent. For grantees with at least one participant in the 2017–18 bachelor’s degree cohort, three-year graduate school enrollment rates ranged from 0.0 to 100.0 percent. Fourteen grantees representing 88 participants attained a three-year graduate enrollment rate of 100.0 percent after bachelor’s degree completion in 2017–18. Sixty-five grantees (out of the 155 grantees with at least one participant in the 2017–18 bachelor’s degree cohort) met or exceeded the enrollment target of 71.0 percent. Table 1 shows three-year graduate school enrollment rates for McNair participants who received their bachelor’s degree in 2017–18. [3:  Beginning with the 2019–20 GPRA reporting year, an adjustment was made to the methodology for assigning participants to bachelor’s degree cohorts for the purposes of calculating the graduate school enrollment rate (see also the Appendix to this report). In this findings section, the graduate school enrollment rates for both the 2016–17 and the 2017–18 cohorts were calculated using the new methodology.] 

Overall, the one-year graduate school persistence rate for McNair participants who were first reported as graduate students in 2019–20 was 85.7 percent. This rate represents a decrease of 1.1 percentage points from the 86.8 percent one-year persistence rate of participants who enrolled in 2018–19; it is 0.7 percentage points above the McNair program’s 2020–21 target goal of 85.0 percent for participants who were first reported as graduate students in 2019–20. For 182 grantees with at least one participant reported as a first-time graduate student in 2019–20, one-year graduate school persistence rates ranged from 0.0 to 100.0 percent, with 63 grantees attaining a one-year graduate school persistence rate of 100.0 percent. These 63 grantees reported between one and 17 participants entering graduate school in 2019–20. One hundred and six grantees met or exceeded the persistence target of 85.0 percent. Table 2 shows the one-year graduate school persistence rate for McNair participants who were first reported as graduate students in 2019–20 and continued to be enrolled in graduate school in 2020–21.

The overall 2020–21 research doctorate completion rate for the McNair participants who earned bachelor’s degrees in 2010–11 was 14.5 percent. This rate was 0.2 percentage point higher than the 2019–20 research doctorate completion rate for participants who earned bachelor’s degrees in 2009–10 (14.3 percent). Table 3 displays the McNair program-level, research doctorate completion rate for participants who earned bachelor’s degrees in 2002–03 through 2010–11. 

## Data Limitations 

These results were based on the data provided by the grantees in the APRs for 2014–15 through 2020–21. A couple of factors may have affected the calculations of the enrollment rate prior to the 2019–20 reporting year when the bachelor’s degree cohort was determined by multiple APR fields. [footnoteRef:4] An examination of the data quality indicated various reporting issues such as inconsistent responses between data fields and/or across reporting years. For instance, the data shows a participant got a bachelor’s degree for the first time in 2015–16 but had the degree date beyond the possible range. The extent to which these factors impacted calculations of the number of students in each cohort and the number enrolling in graduate school is unknown.  [4:  In previous GPRA reporting years, participants were assigned to bachelor’s degree cohorts based on multiple APR fields (i.e., BADegree, BADegreeDate, EnrollCD, EnterGradeLV, and YRGradStudy). For example, using the old methodology for bachelor’s degree cohort assignment, participants would have been assigned to the 2016–17 bachelor’s degree cohort using information from the 2016–17 APR as follows: (1) determine if a student earned a bachelor’s degree prior to 2016–17 and, if so, the student would not be included in the cohort; (2) for students not already assigned to a cohort, examine the bachelor’s degree status field (BADegree) to see if a degree was earned; if a degree was earned, the date the bachelor’s degree was earned (BADegreeDate) field would be checked to see if the date is between September 1, 2016, and August 31, 2017 (if so, participant would be included in the 2016–17 cohort); and (3) if the date that the bachelor’s degree was earned was out-of-range or unknown, then the responses to the fields recording current year of graduate study at the end of the spring/summer term (YRGradStudy), college grade level at entry into the project (EnterGradeLV), and enrollment status for academic year being reported (EnrollCD) would be examined to determine whether or not a student should be included in the cohort.] 

The calculations for the McNair performance measures require participant-level longitudinal data covering a window of time up to 10 years after bachelor’s degree completion, depending on the measure. Program year 2020–21 was the fourth year in the 2017–22 grant cycle; therefore, some 2020–21 performance measures are not calculated for projects that were first funded in the 2017–22 grant competition.[footnoteRef:5] For grantees who continue to be funded in subsequent cycles, tracking prior participants for up to 10 years can be challenging and, as found in an internal analysis of McNair research doctorate recipients, 10 years is frequently insufficient time to capture the completion of a research doctorate.[footnoteRef:6] [5:  Grantees that were first funded in the 2017–22 grant competition would not have served any members of the 2010–11 bachelor’s degree cohort (the basis for the research doctoral completion measure); therefore, participants served by such grantees are necessarily excluded from the calculations for these performance measures. However, small numbers of 2019–20 first-time graduate students were reported by grantees who were first funded in the 2017–22 grant competition, and these participants are included in the calculations for the graduate school persistence measure (see also Table 2).]  [6:  In this analysis, McNair APR data were matched, at the individual participant-level, with data from the National Science Foundation’s Survey of Earned Doctorates (SED). The SED provided additional information about McNair scholars who completed a research doctorate. The study found that for 30 percent of the McNair participants who completed a bachelor’s degree in 1998–99 and completed a research doctorate by 2014, more than 10 years had elapsed between earning a bachelor’s degree and obtaining a research doctorate.] 

## Appendix: Calculation Rules

The data sources used for calculating the graduate school enrollment and persistence rates and the doctorate degree completion rate were the 2014–15 through 2020–21 McNair APRs submitted by grantees. Calculations are described below.

### Graduate School Enrollment Rate

The *graduate school enrollment rate* describes the percentage of project participants who enroll in graduate school within three years of completing their baccalaureate degree.

**To calculate the three-year cumulative enrollment rates of the 2017–18 cohort,** the number of students enrolling in graduate school in 2018–19, 2019–20, and/or 2020–21 (i.e., within three years after receiving a bachelor’s degree) is divided by the number of students receiving a bachelor’s degree in 2017–18 and multiplied by 100. This calculation provides a cumulative rate that includes in the numerator the students who first enrolled in graduate school within one, two, and/or three years after receiving a bachelor’s degree (i.e., enrolled in graduate school in 2018–19, 2019–20, and/or 2020–21). 

· **Beginning in the 2019–20 GPRA reporting year, the number of students receiving bachelor’s degrees** (i.e., bachelor’s degree cohort) is based on information reported in a single APR field: BADegCohortYR (APR field #29). More specifically, for the purposes of calculating the GPRA graduate school enrollment rate, participants are assigned to the 2017–18 bachelor’s degree cohort if:  

they were associated with a grantee that was funded in 2020–21; **and**

they were reported as a 2017–18 bachelor’s degree recipient per the BADegCohortYR field in the 2017–18 APR; **and**

they were not assigned to an earlier bachelor’s degree cohort per the BADegCohortYR field from an earlier year’s APR.

· **The number of students enrolling in graduate school in****2018–19, 2019–20,****and/or 2020–21**is calculated using the fields recording current year of graduate study during the 2018–19, 2019–20, and 2020–21 academic years, respectively. If the participant was recorded as a first-, second-, third-, or beyond third-year graduate student, the participant is included in the numerator.

### Graduate School Persistence Rate

The *graduate school persistence rate*is defined as the percentage of participants who enrolled in graduate school in one academic year and persisted into the next academic year. 

**To calculate the one-year graduate school persistence rate of the 2019–20 cohort,** the number of participants who were reported for the first time as graduate students in 2019–20 and continued to be enrolled in graduate school in the 2020–21 academic year is divided by the number of students who were reported for the first time as graduate students in 2019–20 and multiplied by 100.

· **The number of students who were reported for the first time as graduate students in 2019–20** is calculated from various fields on the 2019–20 APR using the following criteria.

The first criterion in determining whether to assign a participant to the 2019–20 first-time graduate student cohort is to determine whether the student had been identified as part of a prior year’s cohort of first-time-reported graduate students. If a student had been identified as part of a prior year’s cohort of first-time-reported graduate students, the student is not included in the current cohort of first-time-reported graduate students.

The second criterion is to examine the responses to the field recording year of graduate study. The participant is included in the 2019–20 cohort of first-time graduate students if in the 2019–20 APR the year of graduate study is recorded as “first-year graduate student.”

Among students who had not been identified as part of a prior year’s cohort of first-time-reported graduate students and who were not identified as first-year graduate students using the first two criteria, students were included in the cohort of first-time-reported graduate students if reported as enrolled in graduate school and as a second- or third-year graduate student.

· **For the number of first-time-reported graduate students in 2019–20 who continued to be enrolled in the 2020–21 academic year,** data are derived from two fields on the 2020–21 APR. Among first-time-reported graduate students in 2019–20, a student is included among those still enrolled in the 2020–21 academic year if the enrollment status (EnrollCD, APR field #23) is recorded as “enrolled” and the current year of graduate study (YRGradStudy, APR field #40) is recorded as a second-, third-, or beyond-third-year graduate student. 

### Research Doctorate Completion Rate

The *research doctorate completion rate*is defined as the percentage of participants in each bachelor’s degree cohort who earned research doctorates within 10 years of earning bachelor’s degrees.

**To calculate the research doctorate completion rate of the 2010–11 bachelor’s degree cohorts,** the number of participants in the cohort who were reported as having earned a research doctorate by the 2020–21 academic year is divided by the number of participants who were reported as having earned bachelor’s degrees in 2010–11 and multiplied by 100.

· **The number of students who earned bachelor’s degrees in 2010–11** is established from the 2020–21 APR field recording bachelor’s degree cohort. Participants recorded as “prior-year participant, deceased” are excluded from the research doctorate completion rate if they did not attain a doctorate.

· **The number of students who earned research doctorates by 2020–21**is established from the 2020–21 APR field recording highest graduate degree earned. Research doctorate degrees include Doctor of Philosophy, Doctor of Education, and other research-intensive doctorate degrees (e.g., Doctor of Engineering, Doctor of Science, Doctor of Nursing Science, and Doctor of Public Health).

2

2

# image1.png