Predicting MS Admission. It’s almost admission season and I’ve... | by Naman Doshi | Towards Data Science

It’s almost admission season and I’ve couple of friends who are in panic mode waiting for a call from the universities they’ve applied at.This made me think — How can we predict whether a student will get an admit or not? What are the parameters for selection? Can it be mathematically expressed? All of these questions started popping up, so I decided to work out this problem and help my friend calm his nerves. (This is the first technical tutorial I’m ever writing so please go easy on me).
In order to solve this problem, I’m going to take you through what I call a “Data Science Pipeline”. Given any problem statement, follow this pipeline so that your approach is structured.
Define the problem -
Define the problem -
Write down the problem statement and understand what you’re trying to solve. In this case, our objective is to predict whether a student will get an admit or not. So it means, that this is a binary classification problem.
2. Generate your own hypothesis -
Next, list down all the things that you think can affect our objective viz. List down all the possible features with respect to our target feature.In our case, we got ask this question — What are the factors that can affect a student’s admission?Take a piece of paper and write down your own hypothesis. I spent some time and came up with the following features- GRE Score- TOEFL Score- Statement of Purpose (SOP)- Letter of Recommendation (LOR)- Academic Performance (GPA)- Extra Curricular Activities (Sports, Math Olympiad etc..)- Outstanding Achievements- Projects and Research Make sure that you write down at least 10–20 for any problem statement. This helps us get a deeper understand of what we are trying to solve and prompts us to think beyond the available dataset.
3. Get the Dataset -
Next step is to get the data. We’re using UCLA hypothetical data for graduate admissions. You can download the dataset here.You can now map your hypothesis with the given dataset. See how many features you can add or modify in order to improve the quality of the dataset. Identify each feature as either (a) Continous Variable or (b) Categorical Variable
4. Data Cleaning -
Most of the time, the dataset will have lot of anomalies like missing values, outliers and so on. Always preprocess the data before moving on to the next step.- Missing Value Treatment: You can use Mean-imputation (for continous variables) or Mode-imputation (for categorical variables)
- Outlier treatment: An outlier is an observation that lies an abnormal distance from other values in a random sample.
There are four ways to treat it — Deleting observations, Imputing, Creating Bins and Treat Separately.- Feature Engineering: Combining, adding, deleting, splitting, scaling features in order to increase the accuracy of the model
5. Exploratory Data Analysis (EDA) -
Now its time to get our hands dirty. Let us explore the data and understand the given features.This dataset has a binary response (outcome, dependent) variable called admit. There are three predictor variables: gre, gpa and rank. We will treat the variables gre and gpa as continuous. The variable ranktakes on the values 1 through 4. Institutions with a rank of 1 have the highest prestige, while those with a rank of 4 have the lowest.
EDA helps us get a clear idea about our features. It also helps us capture any trend or seasonality of data points.In our case, we can see that Higher the GRE score and GPA, more the chances of getting admit.
6. Predictive Modeling -This is a binary classification problem. The output has only two possibilities either Yes (1) or No (0). There are lot of classification algorithms out there so how do we know which one is the best?We don’t. This is something that comes with experience and expertise. But not to worry, we do have a work around. We can fit multiple algorithms onto our data points and then evaluate the model. This way we can choose the best model which has the least error.First, let us import all the necessary libraries
from matplotlib import pyplotfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import classification_reportfrom sklearn.metrics import confusion_matrixfrom sklearn.metrics import accuracy_scorefrom sklearn.model_selection import KFoldfrom sklearn.model_selection import cross_val_scorefrom sklearn.linear_model import LogisticRegressionfrom sklearn.tree import DecisionTreeClassifierfrom sklearn.neighbors import KNeighborsClassifierfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysisfrom sklearn.naive_bayes import GaussianNBfrom sklearn.svm import SVC
We’ll be using Cross Validation as an evaluation metric where in we will split our dataset into Train and Test.
#Step 1: Convert Dataframe into matrixdataArray = dataset.values#Step 2: Splitting Input features & Output VariablesX = dataArray[:,1:4]y = dataArray[:,0:1]#Step 3: Splitting training & testingvalidation_size = 0.10seed = 9X_train, X_test, Y_train, Y_test = train_test_split(X,y,test_size=validation_size, random_state = seed)print(X_train.shape)print(X_test.shape)print(Y_train.shape)print(Y_test.shape)
Next we are going to apply the following Machine Learning algorithms on our Training data- Logistic Regression (LR)- Linear Discriminant Analysis (LDA)- K Nearest Neighbours (KNN)- Decision Tree (CART)- Random Forest (RF)- Gaussian Naive Bayes (NB)- Support Vector Machines (SVM)Initialize max number of features and number of trees.
num_trees = 200max_features = 3models = []models.append(('LR', LogisticRegression()))models.append(('LDA', LinearDiscriminantAnalysis()))models.append(('KNN', KNeighborsClassifier()))models.append(('CART', DecisionTreeClassifier()))models.append(('RF', RandomForestClassifier(n_estimators=num_trees, max_features=max_features)))models.append(('NB', GaussianNB()))models.append(('SVM', SVC()))#Fit Models and Evaulateresults = []names = []scoring = 'accuracy'#Cross Validationfor name, model in models: kfold = KFold(n_splits = 10, random_state=7) cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring = scoring) results.append(cv_results) names.append(name) msg = "%s: %f (%f)" % (name,cv_results.mean(), cv_results.std()) print(msg)
What we have done here is run all 7 algorithms on our data point and tried to evaluate it. As you can see LR and LDA perform the best.We can also implement a comparison plot
So now we have fit the ML algorithm onto our dataset. Next, we have to create a predictive model using the best algorithm.For the sake of simplicity, I’m going to pick LR.
#Step 1 - Create prediction modelmodel = LogisticRegression()#Step 2 - Fit modelmodel.fit(X_train, Y_train)#Step 3 - Predictions predictions = model.predict(X_test)#Step 4 - Check Accuracyprint("Model --- LogisticRegression")print("Accuracy: {} ".format(accuracy_score(Y_test,predictions) * 100))print(classification_report(Y_test, predictions))
I love to make plots. So lets make an another one ;)
#plotting confusion matrix on heatmapcm = confusion_matrix(Y_test, predictions)sns.heatmap(cm, annot=True, xticklabels=['reject','admit'], yticklabels=['reject','admit'])plt.figure(figsize=(3,3))plt.show()
Congratulations! You made our very own predictive model with an accuracy of 77.5(Not bad at all for beginners)You can try to increase this score by fitting different classification algorithms or by doing a little bit of feature engineering. Best way is to use Ensembling methods, but that is for some other tutorial.
But wait...How do we know that this works? Yes, exactly right? We have come this far, how can we not let our model make some predictions!
#Making predictions on some new data#Like a bossnew_data = [(720,4,1), (300,2,3) , (400,3,4) ]#Convert to numpy arraynew_array = np.asarray(new_data)#Output Labelslabels=["reject","admit"]#Let's make some kickass predictionsprediction=model.predict(new_array)#Get number of test cases usedno_of_test_cases, cols = new_array.shapefor i in range(no_of_test_cases): print("Status of Student with GRE scores = {}, GPA grade = {}, Rank = {} will be ----- {}".format(new_data[i][0],new_data[i][1],new_data[i][2], labels[int(prediction[i])]))
AND THAT’S IT. Our very own predictive model that can decide your friend’s future (actually no, it cant)But our model can definitely predict whether he/she will get an admit or not!
Motives behind DSW —
I’ve been learning and doing data science from about a year now. During this time I have worked on lot of problem statements. But the thing is sometimes the dataset is pretty interesting but other times its not. You have to care about the problem you’re trying to solve, only then you’ll be able to do a good job.
So find a problem that you wanna solve and use data to solve it.
You can find the code for this tutorial here on Github —
I’ll be writing down some kickass tutorials every week so if you’re someone who can learn only by doing, follow my publication here.
You can also connect with on LinkedIn.
I hope you have a productive week ahead.
Until then, keep hustling!