Passing The College Stress Test
Tags: Economics, Education, Data Visualization, Higher Education

Education 231: Higher Education Economics

The authors divide institutions into one of five classes based on graduation rates (here I quote from p. 24):

Medallion: the segment comprising the nation’s most competitive institutions and students; a segment for which prestige-based ranking plays a substantial role in defining institutional ambitions and quality. Name Brand: a segment largely populated by well-known institutions. Most practice selective admissions, though their appeal is more likely to be regional than national. Many but not all of these institutions would like to be Medallions. Good Buy: a segment comprising a variety of institutions, for the most part offering full-scale undergraduate programs at prices substantially less than those of higher-ranked institutions. Good Opportunity: a segment comprising institutions and students who see higher education as a special opportunity. Many students who shop in this segment are the first in their families to attend college. Convenience: the one segment in which part-time and intermittent learners dominate. Students in this segment often shop for a friendly environment at an institution that understands special needs, including the need to take courses at convenient times.

We can get get a visual of the distribution of these market segments by looking at a k-means plot based on IPEDS data. I’m using the six-year graduation rate and net average price as my variables.

K-means clustering four-year schools on graduation rate and average net price

We can se pretty clearly that there are divisions at the top of the market with some very expensive schools that graduate students at high rates, but lots of clustering near the means on both axes. In fact, the dense cluster near the center of the axes might make the clustering a bit suspect in this case. Certainly the medallions (yellow, in the upper right) make a distinct cluster, but when we come in near the sample mean there’s relatively little to differentiate some of the Name Brands (blue) from Convenience (pink) or Good Buys (light blue). These distinctions might be more apparent (and might especially differentiate the Convenience segment) if we included two-year colleges in the analysis.

I generated a scree plot to confirm this intuition. Looks like we could actually get away with as few as three or four clusters, but five doesn’t look unreasonable. It’s worth noting here that about 24% of the schools analyzed for clustering were placed in the first “Convenience” cluster, suggesting that our findings, using slightly updated data, are roughly in line with TCST.

How many school categories are there?

Zemsky et al. identify schools in the lower left as those most likely to suffer financial distress in the coming years, while they expect “medallions” to thrive. “The winners [in their analysis] were the big, top-ranked institutions. Conversely, being small, being located in the Mid-Atlantic, the Plains, or the Great Lakes, proved to be a disadvantage, as was being at or near the bottom of the market in terms of competitiveness.” (27)

The parameters of the titular “stress test” vary by class, but broadly cover two areas: enrollment and finances. For example, here are the metrics the authors use to evaluate the current and future health of a public four-year college or university:

Percent change in freshman enrollment; First-year retention rate; Market price (i.e. tuition less institution-granted financial aid); Percent change in ratio of endowment to expenses.

The metrics are similar for four-year private, for-profit and two-year institutions.

Scores, which range from 0–12 for four-year institutions, are assigned as follows:

Each college and university can receive a risk score for a given factor ranging from 0 to 3. An institution receives a 0 for a particular component if (1) the change over time (or for retention, the rate) does not reach the alert line and (2) the trend line of the data does not have a negative slope that is projected to reach the alert line…. By contrast, an institution receives a 3 if (1) the negative trend reaches the alert line, (2) the “change over time” both falls below the alert line and (3) reaches or falls below the warning line…. Risk scores of 1 or 2 are achieved if some but not all of the criteria are met. (49)

In other words, we’ll divide our data into classes and look at each distribution along the variables outlined above. We assign critical values (“alert” and “warning” lines) based on those distributions. For example, we can take a quick look at changes in first-year enrollment among four-year colleges and universities and establish our critical values (“warning” in orange, “alert” in red):

If a data point falls below the “alert” line, that’s one point; if the trend line for a component crosses the “alert” line, that’s a point; if the trend line crosses the “warning” line, that’s two points. The sum of those points (which ranges from 0–12 across four metrics) is our stress test score.

Architecture 112: Design Thinking

We can see whether it works and compare our results with what’s in TCST by way of confirmation.

I’m using Jason Bryer’s ‘ipeds’ package for R to get the data. It’s worth mentioning now that I could not get this to work on R version 4.0.0. I was successful after upgrading to R 4.0.3.

First, I download and explore the data to identify the points we need, then I store the necessary tables in a data folder that the app will have access to (we might be able to script the download on the server side of the Shiny app, but this will be faster and cleaner).

These are the available IPEDS data sets by year (note that I hadn’t downloaded all these at the time I wrote this piece of code):

require(ipeds)

require(tidyverse) available_ipeds() %>% tail(15) %>% kable() # for formatting

Available IPEDS data sets 2007–2020

We can see that some are final and some are provisional . The difference is that final data are fully revised and are released about one calendar year later than the provisional data, which are released about a year after data is collected. Thus, for the finalized 2017–2018 IPEDS data set, we’re looking at data that were collected in 2015–2016; for the provisional 2018–2019 data set, we’re looking at data collected in 2017–2018.

Each of the IPEDS years listed here represents a series of tables. We need to identify the tables (and years) we need. Once we’ve done that, we can discard some of the additional (unused) data, and create a directory for our app.

TCST lays out in Appendix A how to build your own stress test. The workflow used there is the same one we’ll adapt to the app. The steps are:

Get the data and chart it;

Tabulate a stress index;

Determine “alert” and “warning” values (these are based on the distribution of data points along each of the metrics).

TCST looks at long-term changes in a college’s fortunes. The authors chart their metrics over an eight-year period, between 2007–2008 and 2015–2016. Because we’re going to be fitting a regression line to our data, we’re going to need to extract data from several years of surveys.

To recap, we have four primary metrics that we’re looking at: enrollment numbers, first-year retention rate, market price, and endowment / expense ratio. The IPEDS tables that correspond to these metrics are:

Institutional Characteristics, where we find institutional information like public vs. private, two-year vs four-year, and primary keys for each school; Fall Enrollment, where we get enrollment data; Finance, where we get information on endowments and expenses.

I’ve posted an R notebook detailing the data extraction process in the GitHub repo, here.

I’ll walk through the process briefly here.