A Beginner’s Guide to Machine Learning in Python | by Natassha Selvaraj | Towards Data Science

There are two approaches you can take to study any topic — top-down, and bottom-up.
When you learnt subjects like math and science in school, you would’ve been taught using the bottom-up approach.
First, you were taught the foundations of the topic, and then gradually progressed through the material.
However, when learning subjects like data science and machine learning, the top-down approach is often easier to grasp — especially if you don’t have a strong mathematical background.
This is easy to do due to the number of packages available in languages like Python and R.
The democratization of machine learning allows you to create end-to-end machine learning models with little understanding of how they work and the math behind the algorithm.
Of course, you still need to gain understand how these algorithms work, but this can come later (once you’ve learnt how to build these models in practice).
In this article, I am going to walk you through an end-to-end machine learning project with the Pima Indian Diabetes dataset on Kaggle.
I will provide codes for data pre-processing, analysis, model training, and evaluation. You can use this as starter code for any machine learning project you work on in the future.
I will also provide additional learning resources for each technique and algorithm I use, so you can read up on them and gain a better understanding of they work.
Note:
I have already done a data analysis tutorial on the same dataset, which you can check out here. For that reason, I will not be going into too much detail on data analysis and visualization in this article. Instead, I will put more focus into building the machine learning algorithms.
To follow along to this tutorial, you need to have a Python IDE installed on your computer.
I suggest using a Jupyter Notebook because that’s what I used to create this tutorial, and Jupyter allows you to display visualizations in the same document that your code is in.
You also need to have the following packages pre-installed — Pandas, Numpy, Matplotlib, Seaborn, and Scikit-Learn. Follow the instructions in their documentation to install these libraries.
This is the dataset we will be using throughout the tutorial. Once you have downloaded it and have your environment ready, run the following lines of code:
# importsimport numpy as npimport pandas as pdimport matplotlib.pyplot as pltimport seaborn as sns# Reading the data-framedf = pd.read_csv('pima.csv')
Once you have read the data-frame, run the following lines of code to take a look at the different variables:
df.head()
You will see the following output:
The different variables in the data-frame include:
Pregnancies — Number of times pregnant
Glucose — Plasma glucose concentration a 2 hours in an oral glucose tolerance test
Blood Pressure — Diastolic blood pressure
SkinThickness — Triceps skin fold thickness
Insulin — 2-Hour serum insulin
BMI — Body Mass Index
DiabetesPedigreeFunction — Diabetes pedigree function
Age — Age
Outcome — Whether or not the person is diabetic
The outcome variable is our target, and all other variables are the predictors. We need to use the remaining variables to predict the outcome with a machine learning model.
To take a look at some descriptive statistics, run the following lines of code:
df.describe()
We can see some simple dataset statistics:
Now, we can run a few lines of code to view the relationship between all the variables in the dataset:
# pairplot:sns.pairplot(df,hue='Outcome')
The following plot is rendered:*I didn’t include the entire image because it was too large
A pair plot is a very useful visualization. It allows you to take a look at the relationship between all the variables in the dataset at once.
At a first glance, you will be able to notice strong relationships between variables that stand out. You can then go on to examine these relationships, and remove highly correlated attributes that might cause any multicollinearity in your model.
Now, we can check for missing values in the data-frame:
print(df.isnull().values.any())
The lines above should return False. There are no missing values in the data frame.
Now, we are going to standardize our variables to get all of them around the same scale. You can read more about why standardization is important here. If you prefer watching over reading, I suggest taking a look at Krish Naik’s explanation on the importance of feature scaling.
We will be doing something called the Z-Score standardization. This will take all our variables and transform them to follow a normal distribution having a mean of 0 and a standard deviation of 1.
Run the following lines of code:
X = df.drop('Outcome',axis=1)y = df['Outcome']# standardizationX = (X-X.mean())/X.std()
Once this is done, inspect the head of the data frame again:
X.head()
The transformed data frame will look like this:
Now that we are done with data pre-processing, we can start building the machine learning model.
First, we need to split the data frame into a train and test set. We will be training the model on one set of data, and then evaluating its performance on data that it has never seen before.
There are many ways to perform model validation, one of the most popular being K-fold cross validation. You can read about the different cross-validation methods here.
However, I will cover the simplest validation approach in this article — dividing the dataset into two, and holding out one set of data for testing.
Here is some code to do this:
from sklearn.model_selection import train_test_splitX_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.25)
After doing this, we will get separate datasets to train and test our model. We can now start with model creation.
The first model we will build is a logistic regression classifier. This model fits a logistic function onto data to identify the class that each data point belongs to.
If you have learnt about linear regression, you should know that logistic regression uses the same linear function (y=a+bX) to separate values of each class. However, it modifies the linear function a little such that the values will not go below 0 or above 1.
As a result, a logistic regression classifier outputs probabilities. By default, the model prediction is the class with the highest probability.
You can change this by tailoring the threshold value of the logistic regression model to your needs. By default, it is 0.5. We will be using this default threshold value of 0.5 in this tutorial.
If you want to learn more about logistic regression, I suggest watching this video.
You can run the following lines of code to fit a logistic regression model onto the training data:
from sklearn.linear_model import LogisticRegressionlr = LogisticRegression()lr.fit(X_train,y_train)
Now, we can make predictions on our test data:
lr_preds = lr.predict(X_test)
And we’re done! Now, we can fit the data on the next model.
A decision tree classifier is a tree-based model. It will take all the variables in the dataset and split on them.
At each split, it either assigns the data point to a class or splits on a different variable. This process continues until there are no more features to split on, or a stopping criteria is reached.
The decision tree classifier decides what to split on by selecting the feature that minimizes the loss at each split. The Gini index and entropy are two common loss functions used in decision tree classifiers.
You can watch this video to gain a better understanding of how decision trees work.
To fit a decision tree classifier on the data, run the following lines of code:
from sklearn.tree import DecisionTreeClassifierdt = DecisionTreeClassifier()dt.fit(X_train,y_train)
Lets use this model to make predictions on the test data:
dt_preds = dt.predict(X_test)
Done! Lets move on to the next model.
A random forest model uses multiple decision trees to come up with a prediction. In practice, random forests often outperform decision trees and linear classifiers.
It uses a technique called bagging, which stands for bootstrap aggregation. The training dataset is randomly sampled many times, and a decision tree is fit onto each data sample.
Only a subset of features are considered for division at each node, ensuring a fair representation of each variable in the model.
In classification problems like this one, the output is the majority class prediction of all the decision trees.
To learn more about how a random forest algorithm works, you can read this article. If you learn better visually, you can watch this video instead.
Run the following lines of code to fit the random forest classifier on the training set:
from sklearn.ensemble import RandomForestClassifierrf = RandomForestClassifier()rf.fit(X_train,y_train)
Now, run the line below to make predictions on the test set:
rf_preds = rf.predict(X_test)
Done! Lets move on to the final model we will be building for this tutorial.
Another extension of tree based models, an XGBoost classifier uses a technique called boosting to improve the performance of decision trees.
The way gradient boosting algorithms work is pretty intuitive. A base learner is first implemented that makes an initial prediction. The residuals of the initial model is calculated, and a second decision tree is added, that predicts the residuals of the initial model.
Just like this, a chain of decision trees are added to form a sequential model that minimizes the overall residual.
To learn more about gradient boosting, you can watch this video.
Gradient boosting is one of the most popular supervised learning technique, and it used often by data scientists in Kaggle competitions. In practice, they tend to perform a lot better than decision trees.
from xgboost import XGBClassifierxgb = XGBClassifier()xgb.fit(X_train,y_train)
Now, run these lines of code to make predictions on the test set:
xgb_preds = xgb.predict(X_test)
And that’s it! We are done training the model and saving their predictions. Lets take a look at model performance now.
We will take a look at the simplest metric used to evaluate model performance — accuracy.
The following lines of code will create a bar chart that displays the accuracy score of all four models:
model = np.array(['Logistic Regression','Decision Tree','Random Forest','Gradient Boosting'])from sklearn.metrics import accuracy_scorescores = np.array([accuracy_score(lr_preds,y_test),accuracy_score(dt_preds,y_test),accuracy_score(rf_preds,y_test),accuracy_score(xgb_preds,y_test)])df = {'model': model, 'scores': scores}sns.barplot(x='model',y='scores',data=df)
These lines of code will render a chart that looks like this:
The accuracy of each model are as follows:
Logistic Regression— 0.73
Decision Tree Classifier — 0.77
Random Forest Classifier — 0.81
XGBoost Classifier — 0.79
It looks like the random forest classifier has outperformed the other models, and XGBoost is a close second.
Keep in mind that your results may differ slightly from mine when you run the same lines of code, because the data will be partitioned differently every time you do a train-test split.
Also, there are other important classification metrics that you need to keep in mind to ensure that your model is performing well — such as precision and recall.
If you are working on problems like disease classification where one class of data is under-represented in your dataset, you might run into an issue where your model only predicts the majority class.
Having high validation accuracy isn’t the only indicator of a well performing model.
If you want to learn more about other classification metrics, you can watch this video. To learn more about solving the imbalanced dataset issue in classification problems, you can follow along to this tutorial.
That’s all for this article! I hope you learnt something from this tutorial.
Please do take a look at the additional resources I linked throughout the article.
Now that you have a high level understanding of how machine learning modelling works, these resources will help you build a strong understanding of everything you implemented.
They will help cement your understanding of the different data pre-processing techniques, how different algorithms work, and the different metrics you can use to assess model performance.