K-Means Clustering of University Data | by Chris Woodard | Towards Data Science

Hello everybody! For this project, I want to introduce K-means clustering, one of the most common clustering algorithms in machine learning. Along the way, we’ll do some exploratory data analysis to understand more about a dataset about universities.
For those of you who may be new to machine learning, clustering is an example of an unsupervised learning model, when we feed uncategorized training data into a model and have our model attempt to categorize that data in some way according to its features.
The alternative is a supervised learning model, where we assign labels to our training data in advance indicating which category each data point belongs to, and the model attempts to predict which category a new data point belongs to.
I worked in education in China for almost 4 years, and my work involved using data analytics to estimate students’ likelihood of admission to universities on their school list given their profile. Just for fun, I’ll create a model in Python using K-means clustering to cluster universities into two groups: private and public. In reality, a student who does at least cursory research on a university website can find out in under a minute whether a school is private or public, but we’ll see if a model can correctly categorize schools based on features that may seem very relevant (out-of-state tuition, graduation rate) and others that might not seem relevant at all (percentage of alumni who donate).
NOTE: we actually have the labels for this data set, but we will NOT use them for the K-means clustering algorithm, since that is an unsupervised learning algorithm. At the end, I’ll compare K-means to a supervised learning algorithm to compare the performance of the two models.
Here are the features in this dataset — feel free to glance at them and think about how well or poorly they might predict a school being public or private:
Apps: Number of applications received
Accept: Number of applications accepted
Enroll: Number of new students enrolled
Top10perc: % of new students from top 10% of their high school class
Top25perc: % of new students from top 25% of their high school class
F.Undergrad: Number of full-time undergraduates
P.Undergrad: Number of part-time undergraduates
Outstate: Out-of-state tuition
Room.Board: Room and board costs
Books: Estimated book costs
Personal: Estimated personal spending
PhD: % of faculty with PhDs
Terminal: % of faculty with a terminal degree (PhD/JD/MD/MBA/etc)
S.F.Ratio: Student/faculty ratio
perc.alumni: % alumni who donate
Expend: Instructional expenditure per student
Grad.Rate: Graduation rate
A good way to start any analysis is to use Pandas.DataFrame.describe() to get some basic statistics about your data.
import numpy as npimport pandas as pdimport matplotlib.pyplot as pltimport seaborn as snscolleges = pd.read_csv(‘College_Data’,index_col=0)colleges.describe()
Purely by looking at the max value of some of these columns, a few things stand out:
It’s common knowledge that top private schools can charge up to $60k/year, but the max out-of-state tuition here is only $21,700, so we can assume this tuition is listed on a per semester basis rather than per year.There’s a school that has above 100% of faculty with PhDs! This seems fishy, but it can probably be explained by a handful of over-achieving faculty who may have a dual PhD that has been double counted.In the same vein, we’ve also got a school with a graduation rate of 118%! We’ll explore this anomaly later...
It’s common knowledge that top private schools can charge up to $60k/year, but the max out-of-state tuition here is only $21,700, so we can assume this tuition is listed on a per semester basis rather than per year.
There’s a school that has above 100% of faculty with PhDs! This seems fishy, but it can probably be explained by a handful of over-achieving faculty who may have a dual PhD that has been double counted.
In the same vein, we’ve also got a school with a graduation rate of 118%! We’ll explore this anomaly later...
Feel free to poke around the mean, median, and other statistics to see what sticks out to you! That’s what makes it exploratory data analysis.
Let’s figure out which college had the highest number of applicants.
colleges.loc[colleges[‘Apps’]==np.max(colleges[‘Apps’])]
Turns out Rutgers has the highest number of applicants and acceptances, but not the highest number of enrollments. Another major bragging point for high schools and universities alike is their student-faculty ratio: the lower the better.
colleges.loc[colleges[‘S.F.Ratio’]==np.min(colleges[‘S.F.Ratio’])]
The winner here is University of Charleston, with a very low ratio of 2.5, which is practically as intimate a teaching environment as homeschooling or private tutoring. Considering that the 25th percentile of the student-faculty ratio is 11.5, U of C is a major outlier.
Now let’s talk money: which university has the highest alumni donation rate?
colleges.loc[colleges[‘perc.alumni’]==np.max(colleges[‘perc.alumni’])]
Hardly surprising that Williams takes the prize as it’s a small liberal arts college (~2000 students total) that caters to largely wealthy families with a strong network. Calling .describe() earlier told us that the median donation percentage was a measly 21%, hence the abundance of on-campus job opportunities as student callers.
Now here’s the juicy stuff: let’s see who’s really getting their dollar’s worth from their university education by looking at the ratio of a school’s expenditure on each student to out-of-state tuition.
colleges[‘expense ratio’] = colleges[‘Expend’]/(colleges[‘Outstate’])colleges.describe()[‘expense ratio’]
WOAH! The fact that the mean is less than 1 tells me that the average school is losing money on its students when considering tuition as the only source of income. In fact, ~69% of universities are investing more money in their students than they’re charging, which does not bode well for the future of most universities. The school with the highest ratio is probably some Ivy League school, right?
colleges.loc[colleges[‘expense ratio’]==np.max(colleges[‘expense ratio’])]
WRONG! The best “Bang for Your Buck” award goes to...*drum roll*...University of Alabama at Birmingham with an expense ratio of 3.68! However, a more cynical way of looking at it is if UAB’s spending nearly 4x the out-of-state tuition on every student, then it might not be in business much longer...so if you’re in the HS class of 2021, get in while they’re still afloat, but hold tight to your wallet because those student callers will be working overtime to snag your donation dollars!
For anyone who was wondering, the biggest thief (the college with the lowest expense ratio) is Lindenwood College with a ratio of 0.378. Yikes!
It’s time to create some data visualizations!
Let’s investigate how tuition varies with the number of students and by whether a school is public or private. I suspect that tuition is higher for private schools because they don’t receive government funding like public schools. The number of full-time students could push the needle either way, as more students means more staff and more buildings, but it’s possible that a university could benefit from economies of scale for construction projects for such a large number of students.
plt.figure(figsize=(8,6))sns.scatterplot(data=colleges,x='Outstate',y='F.Undergrad',hue='Private')
Hmmm...very interesting how the school’s undergrad population is almost completely uncorrelated with tuition, which begs the question: where is that money going? Let’s zoom in on out-of-state tuition and take a look at graduation rate as well.
sns.set_style('darkgrid')g = sns.FacetGrid(colleges,hue='Private',palette='coolwarm',height=6,aspect=1.5)g = g.map(plt.hist,'Outstate',bins=20,alpha=0.7)plt.legend(labels=['Private','Public'])
grid = sns.FacetGrid(colleges,hue='Private',palette='coolwarm',height=6,aspect=1.5)grid = grid.map(plt.hist,'Grad.Rate',bins=20,alpha=0.7)plt.legend(labels=['Private','Public'])
As expected, out-of-state tuition is higher for private schools, and so is graduation rate — perhaps because if a student is investing that much money into their education, their parents are thinking, “You better finish...or else!”
Now back to that school with a graduation rate of higher than 100%...turns out the culprit is Cazenovia College. Maybe this school pulled a fast one and counted double majors as having graduated twice? In reality, this makes no sense, so I’m going to change this graduation rate to 100%.
Now on to the moment you’ve all been waiting for: K-means clustering!
First we’ll import KMeans from Scikit-learn, the most popular package for machine learning in Python. The way K-means works is the following:
Pick the number of clusters. Whether you choose 2 or 4 or 10 is dictated by the domain of the specific problem. In this case, we’re sorting into public and private schools, so we will choose the number of clusters to be 2. If you were analyzing genetic variations in a population and knew beforehand there were 7 known variants, then you would choose 7.Randomly assign each data point to a category.Take the centroid of the data points in each category. For those of you who think a centroid is some cool-sounding type of asteroid, it’s multivariable calculus-speak for “the average of all the data in each cluster.” You already understand this intuitively for 1-D data: the average price of grocery items gets you a single number like $52. If you calculate the average price AND average quantity of grocery items (2 dimensions), you’ll get two numbers like $52 and 2 items. In this dataset, we have 18 features, so each centroid corresponds to a 18-D set of coordinates. And no, we’re not going to try and visualize this.Re-assign each data point to the category corresponding to the nearest centroid.Repeat Steps 3 and 4 until there are no more changes in category.
Pick the number of clusters. Whether you choose 2 or 4 or 10 is dictated by the domain of the specific problem. In this case, we’re sorting into public and private schools, so we will choose the number of clusters to be 2. If you were analyzing genetic variations in a population and knew beforehand there were 7 known variants, then you would choose 7.
Randomly assign each data point to a category.
Take the centroid of the data points in each category. For those of you who think a centroid is some cool-sounding type of asteroid, it’s multivariable calculus-speak for “the average of all the data in each cluster.” You already understand this intuitively for 1-D data: the average price of grocery items gets you a single number like $52. If you calculate the average price AND average quantity of grocery items (2 dimensions), you’ll get two numbers like $52 and 2 items. In this dataset, we have 18 features, so each centroid corresponds to a 18-D set of coordinates. And no, we’re not going to try and visualize this.
Re-assign each data point to the category corresponding to the nearest centroid.
Repeat Steps 3 and 4 until there are no more changes in category.
Here’s a great image to help illustrate:
Coding a K-means model in Python is very simple, as shown below, and the clusters that are generated by the model can be referenced with km.labels_ (don’t forget the underscore at the end!).
from sklearn.cluster import KMeanskm = KMeans(n_clusters=2)km.fit(colleges.drop(‘Private’,axis=1))
There is no perfect way to evaluate clustering if you don’t have the labels. However, in this case, the colleges data set told us whether each school was public or private, so we can cross-validate our K-means model with these labels to compare the performance of supervised and unsupervised models in general.
Two quick ways to evaluate the performance of a machine learning model is to look at a confusion matrix and a classification report.
def convertToCluster(cluster):    if cluster=='Yes':        return 1    else:        return 0colleges['Cluster'] = colleges['Private'].apply(convertToCluster)from sklearn.metrics import classification_report,confusion_matrixprint(confusion_matrix(colleges['Cluster'],km.labels_))print(classification_report(colleges['Cluster'],km.labels_))
If you’re used to supervised learning algorithms, you may look at these results and think, “22% accuracy?! This model is garbage! What are unsupervised models good for anyway?” But remember the definition of unsupervised: the model is aiming to make sense out of the mess of 18 features that we’ve given it, with no labels. This is not an easy task, so let’s give credit to K-means where credit is due.
Let’s actually compare the results we just got to that of a supervised learning model to evaluate their performance (or at the very least, set our expectations to the proper low level for unsupervised learning 😉). Let’s just go for a classic logistic regression.
from sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LogisticRegressionunis = collegesX = unis.drop('Cluster',axis=1)y = unis['Cluster']X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)logmodel = LogisticRegression()logmodel.fit(X_train,y_train)preds = logmodel.predict(X_test)print("Using K means clustering (unsupervised):\n")print(confusion_matrix(colleges['Cluster'],km.labels_))print(classification_report(colleges['Cluster'],km.labels_))print("Using logistic regression (supervised):\n")print(confusion_matrix(y_test,preds))print(classification_report(y_test,preds))
Behold the power of labeling, folks. Using a simple logistic regression, a supervised learning model, the accuracy shot up from 22% to 91%, which probably could be improved upon with another model choice.
Hopefully you can begin to see how K Means is useful for making sense out of messy data! Feel free to explore other areas using Exploratory Data Analysis and see what you find!