A Guide for Your Very First Machine Learning Project
Tags: Logistic Regression, Supervised Learning, AI, Knn, Machine Learning

A Guide for Your Very First Machine Learning Project

If you have read my previous article on how to get started in Machine Learning here, this is the perfect next article because in this article, we will dive deeper in the practical way of all the things that was mentioned in that article, for instance, drawing various graphs for data visualisation, using libraries (if you still remember, these are all the functions in Machine Learning that you can use without actually code from scratch, such as for importing data, drawing graph, even applying basic ML algorithms directly), and the steps to successfully executing ML algorithm from the given dataset step by step.

Your Very First Machine Learning Project

If you do not know which project you should pick first in Machine Learning, it is very common among Machine Learning Engineers to recommend the Iris dataset as your very first project in Machine Learning, and in fact, it is dubbed as the ‘Hello World’ project of Machine Learning. Before we discuss further about what is the Iris dataset, first we have to understand two types of Machine Learning, which are supervised and unsupervised learning.

Supervised Learning

As we understood, Machine Learning is an algorithm which relies on data to mimic human intelligent to give a highly accurate decision. The term supervised learning means the algorithm make a decision based on data that has been labelled accordingly. This way, we will be able to assess on how accurate the algorithm in performing its task.

2. Unsupervised Learning

However, not every given data is labelled and it is even considered expensive to label every data. The tasks can be grouping the picture of birds compared to the picture of airplanes based on their features or understanding customer purchasing behaviour. Hence, a Machine learning algorithm that performs its task without any label on its data is called unsupervised learning.

Supervised learning is tend to be preferred as the first Machine Learning project compared to unsupervised learning as it is more manageable and we can assess our performance based on the evaluation metrics. To dive deeper into supervised learning, it can be differentiate into two types, which are classification and regression. Regression is a supervised machine learning to predict the future data based on its historical data. For instance, this can be predicting the stock market price or forecasting future sales in a business.

On the other hand, classification, as it is called, is to classified a given data into two or more categories, for instance, it can be to classify whether an email is spam or or not spam / ham, breast cancer detection, and of course the classification of iris flowers that we will be doing in this project.

Iris Flower Dataset

Iris dataset can be traced back in 1936 when Ronald Fisher, a British statistician, eugenicist, and biologist introduce it in his paper of “The use of multiple measurements in taxonomic problems” as an example of linear discriminant analysis. This dataset is trying to classify 3 types of Iris flowers, which are Iris Sentosa, Iris Virginica, and Iris Versicolor where it contains 50 samples for each type of them. Furthermore, this data contains 4 features (usually represented as number of columns in the dataset) of the flowers, which are the length and width of the sepals and the petals, plus label of each data. This data can be downloaded in the Kaggle website here.

Getting Started

As mentioned in the previous article, almost every Machine Learning project runs on Python, which can be done on either Jupyter Notebook or Google Colab, which has been discussed on the installation as well there. After Jupyter Notebook or Google Colab has been installed and opened, create your first notebook by clicking new and Python 3 for Jupyter Notebook as shown in the picture below or click file and then new notebook for Google Colab.

which will show your notebook here as shown in the picture below:

In the first cell, you need to import the following libraries:

import numpy as np

import pandas as pd

import seaborn as sns

import matplotlib.pyplot as plt

This means the next time you want to use their libraries, you can simply use their acronym. For instance, to use pandas, you can simply type pd and those acronyms can be set as whatever you want. Numpy is useful to construct matrices, while pandas is for data cleansing or exploration, seaborn and matplotlib.pyplot have similarfunction which is to plot graphs, however, seaborn allows us to create more complex graphs, which will be shown later.

After importing those libraries, you will need to import the Iris dataset to your notebook by using following pandas function:

data = pd.read_csv('../First Project/Iris.csv')

As we can see above, we are creating a new variable called data and we are also using pandas library in the form of pd to read the iris dataset, which is in csv form. Make sure the “First Project” is changed into the name of the folder in which your Iris data is being stored. So, next time we want to interact with our Iris dataset, we can always use the data variable.

Next, if we want to make sure that our data has been loaded into our notebook, we can check it by using another pandas function, which is:

data.head()

This will show us the first 5 data in our Iris dataset and if you would like to see more data, simply specify the number inside the round brackets as shown below:

There are also many other data exploratory methods in pandas, such as:

data.info()

To show the types of data, number of filled cells, number of columns, and memory used as shown below:

To calculate all of the descriptive statistics such as count, max, min, mean, median, and standard deviation, can be done instantly by following command:

data.describe()

Which will show an output of:

To count the number of duplicated data of an attribute, we can specify it using square bracket as follows:

data['Species'].value_counts()

Which will show an output of:

And finally, we can delete the entire column by using the following drop function below:

data = data.drop('Id', axis=1)

In here, we are dropping Id column, which will shows the new dataset as:

Data Visualisation

Next, we will enter the interesting part of data exploration, which is data visualisation. First, we need to create a new variable for our graph which is shown below:

plotting = sns.pairplot(data, hue='Species')

In this command, again, we are only using sns acronym from seaborn and from that, we are assigning a new variable called plotting to create a pairplot, which is a plot to show the relationship between each attribute based on Species attribute as mentioned in hue. Which will give us the following plot:

From these graphs, we can see the difference in the length and width of the petal and sepal of the three types of Iris flowers easily.

Another seaborn plot feature is Violin plot, which is similar to box plot but with extra density visualisation where the thicker part shows the more frequent than the thinner part. We can plot each attribute individually for Violin plot. Further note, in order to plot several graphs in a single cell, it is necessary to insert matplotlib command, plt.show() at the end of each graph variable, otherwise, the cell will only plot the last graph assigned from it.

g = sns.violinplot(y='Species', x='SepalLengthCm', data=data)

plt.show()

g = sns.violinplot(y='Species', x='SepalWidthCm', data=data)

plt.show()

g = sns.violinplot(y='Species', x='PetalLengthCm', data=data)

plt.show()

g = sns.violinplot(y='Species', x='PetalWidthCm', data=data)

Which will plot:

Label and Feature

Now, we need to create our label and feature data for the Machine Learning process. As we understood before, the features of the Iris dataset are the length and width of the petal and sepal, while the labels are the name of the flower’s species. This way we need to create two new variables for them, lets say X as the feature, which only contains the feature data and Y as the label, which only contains the label data.

As we learned earlier to delete the entire column, we can use drop function, hence, we can create our feature data X by dropping the species column as shown below:

X = data.drop(['Species'], axis=1)

On the other hand, because label only consists of species column, it is more efficient to assign Y with only species column instead of dropping all the feature columns as shown below:

y = data['Species']

Data Splitting

In this stage, we are learning about data splitting. Data splitting is a very common method that need to be done in supervised learning project as it will allow us to train and test our Machine Learning algorithm with two different datasets and evaluate better than running train and test in both dataset. For example, if we want to evaluate a student, we have to train him with a set of questions and then give him a test with a different kind of questions instead of giving him the training and test questions exactly the same.

Therefore, we need to use a library called Scikit-learn to split the data:

from sklearn.model_selection import train_test_split

and make sure you have installed Scikit-learn by using:

pip install scikit-learn

And then, split both label and feature data into train and test data by using the following function:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=5)

test size indicates the percentage of the whole data that we will use as the test data. For instance, since we chose 0.4 in this project, therefore, from 150 data, 40% of them or 60 of them will be assigned as label test data and feature test data. While the rest of the 90 data are treated as train data.

For extra note, train and test split function from sklearn always assign random data as test and train, which means each time you execute the train and test split function, the data that will be assigned to the train label, train feature, test label, and test feature are always different. In order to make sure the same data is always treated to the same train or test, label or feature data, we need to assign random state. This random state make sure each execution will give the same output as train and test data.

Classifying by Using Machine Learning Algorithm

Now we have reached the climax of this project, which is assigning the data into one of many Machine Learning algorithms available. Specifically in classification algorithm, there are many options, however, these are the most well-known so far:

K-Nearest Neighbour (KNN) Naive Bayes classifier Logistic Regression Random Forest Support Vector Machine (SVM)

Most of Machine Learning Engineers do not know which algorithm best suit for the given dataset as different dataset will perform better in different algorithm, therefore, we need to test at least two algorithms to make sure we have chosen the better performing algorithm for the given dataset.

Let’s say in this project we are going to use two algorithms, KNN and logistics regression to classify the Iris dataset. First question, what is KNN? Well all I can say is its name tells itself. It is a classification algorithm by using the number of nearest neighbour (value of k)to choose which classification it belongs to. Let’s observe the picture below:

We are choosing whether the circled wine is a red wine or a white wine based on the chloride level and sulphur dioxide level. By assigning the value of k as 5, we are assessing 5 nearest wines to it that take the highest number from those wine and classify it as itself. For this case, we can see that there are more red wine in the circle than white wine, therefore, we are assuming that it is a red wine.

However, the challenge in KNN is to choose the value of K that is suitable for our dataset as we do not want overfitting and underfitting happens in our algorithm and we want it to be balanced as shown in the picture below:

We can try each k value by using for function from 1 to 26 and plot each accuracy by using simple matplotlib line graph. Furthermore, to use KNN algorithm, we do not need to code the formula from scratch as we can import KNN algorithm and accuracy calculation from sklearn library here:

from sklearn import metrics

from sklearn.neighbors import KNeighborsClassifier

Therefore, we can execute KNN algorithm and calculate the accuracy from metrics function as follows:

k_range = list(range(1,26))

scores = []

for k in k_range:

knn = KNeighborsClassifier(n_neighbors=k)

knn.fit(X_train, y_train)

y_pred = knn.predict(X_test)

scores.append(metrics.accuracy_score(y_test, y_pred))

As we can see above, to command the KNN algorithm to train the data, we execute the fit function, while to test the data, we use predict function. Subsequently, from the for function output, we can evaluate which k value is the best by plotting each result by using matplotlib as shown below:

plt.plot(k_range, scores)

plt.xlabel('Value of k for KNN')

plt.ylabel('Accuracy Score')

plt.title('Accuracy Scores for Values of k of k-Nearest-Neighbors')

plt.show()

This will give an accuracy output of:

Which is around 0.983 or 98.3% correct accuracy for k = 2, 3, and 5 up to 26. To avoid overfitting, let’s choose not too small k value and take 12 instead.

Let’s try out logistics regression algorithm by using the same fit and predict function for train and test as shown in the code below:

logreg = LogisticRegression()

logreg.fit(X_train, y_train)

y_pred = logreg.predict(X_test)

print(metrics.accuracy_score(y_test, y_pred))

That gives an output of 0.966 or 96.6%, which does not show much difference compared to KNN, however, to make sure we are using the better algorithm, it is wiser to pick KNN as our algorithm to classify the dataset.

Since we are pretty confident with our 98.3% accuracy from KNN, we should try out our algorithm by assigning random length and width of the petal and sepal to see which type of flower will come up by using following code:

knn = KNeighborsClassifier(n_neighbors=12)

knn.fit(X, y)

knn.predict([[6, 3, 4, 2]])

In this code, we are experimenting with assigning 6cm sepal length, 3cm sepal width, 4cm petal length, and 2cm petal width, which our algorithm tells us the flower might be:

Iris Versicolor

However, if we are assigning 10cm sepal length, 9cm sepal width, 5cm petal length, and 4cm petal width, the algorithm will predict that it might be:

Iris Virginica

Wrap It Up

If you have followed my guide up to this point, I am quite impressed and it shows that you have the passion for Machine Learning and have the capacity to learn deeper than this point. My suggestion next for you is to try out on larger dataset which might challenge your data pre-processing technique, which I might write on about in the future. If you are looking for a next project in regression, I have written an in depth guide for the follow up project, which you can find here.

Let me know in the comment about what you think about think project. Overall, thank you so much so stick with me in this tutorial until the very end and good luck with your next projects.