How to Create Your First Machine Learning Model | by Rebecca Vickery | Towards Data Science

Many machine learning tutorials focus on specific elements of the machine learning workflow such as data cleaning, model training or algorithm optimisation. However, if you are very new to machine learning it can be difficult to fully grasp the basic end to end workflow without a complete simple explanation or walkthrough.
In the following post, I am going to provide a very simple tutorial for developing a supervised machine learning model in python. In this post I am going to assume only a very basic knowledge of python programming and that you have a few of the common data science libraries already installed. If you need an introduction to python for data science first, Codeacademy is a good place to start. I will also provide links to the documentation for all libraries that I include in the tutorial.
Machine learning is the ability of a computer to be able to learn the mapping between some inputs (data features) and some known outputs (data labels) without being explicitly programmed. The goal is that, given new inputs with unknown outputs, the machine can correctly predict the labels.
“Machine learning is a thing-labeler, essentially.”, Cassie Kozyrkov
This mapping of inputs to outputs is performed by mathematical functions, primarily from the areas of linear algebra and calculus, that are performed at a speed and scale that could not be achieved without large amounts of computing power. There are many different algorithms that can accomplish this task from simple regression-based methods to more complex deep learning techniques.
Fortunately, the python programming language has a very active community of open source developers who have built a number of libraries that abstract away the need to directly code these algorithms. One of the core machine learning libraries is scikit-learn which is highly accessible for the beginner. In this tutorial, I am going to be focussing on this library.
For simplicity, I will be using machine learning to solve a classification problem. I will use a toy data set from the scikit-learn API which consists of a number of attributes of wine (the features) such as the alcohol volume and colour, for three types of wine (the labels or target).
To load the data and convert it to a pandas data frame use the code shown below.
from sklearn.datasets import load_wineimport pandas as pdwine = load_wine()data = pd.DataFrame(data=wine['data'], columns = wine['feature_names'])data['target'] = wine['target']data.head()
As machine learning algorithms are based on mathematics the data for machine learning must be numeric in nature. This data set is entirely numeric, however, if we had categorical features we would need to perform some preprocessing to convert them to numeric first. I have chosen this data set so that I can show the simplest workflow to develop a model without introducing more advanced concepts around data preprocessing.
When developing a machine learning model it is important to be able to evaluate how well it is able to map inputs to outputs and make accurate predictions. However, if you use data that the model has already seen (during training for example) to evaluate the performance then you will not be able to detect such problems as overfitting.
Overfitting is when a model has learned either too much detail or noise in the training data which won’t necessarily exist in unseen data. In this case, the model will appear to perform well on the training data but will perform poorly on unseen data. This is often referred to as the model not generalising well.
It is a standard in machine learning to first split your training data into a set for training and a set for testing. There is no rule as to the exact size split to make but it is sensible to reserve a larger sample for training — a typical split is 80% training and 20% testing data.
It is important that the data is also randomly split so that you are getting a good representation of the patterns that exist in the data in both sets. Scikit-learn has a tool that performs this process in a single line of code, known as test_train_split. The code below passes the features and the target data to the function and specifies a test size of 20%.
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(data.drop('target', axis=1), data['target'], test_size=0.20, random_state=0)
It is a good idea to next train a dummy classifier to get a baseline score to benchmark further iterations of model development. Scikit-learn has a function that allows you to train a model and make predictions based on simple rules, such as predicting at random. This is useful to evaluate that your model development is improving as you iterate through the next steps.
In the code below I have trained a dummy classifier that always predicts the most frequently occurring class. The dummy classifier, however, has a number of different methods for this. The accuracy score is 0.44. A perfect accuracy score would be 1.0. Now moving through the next stages we will know if we are making an improvement on this baseline.
from sklearn.dummy import DummyClassifierfrom sklearn.metrics import accuracy_scorebaseline = DummyClassifier(strategy='most_frequent', random_state=0).fit(X_train, y_train)y_pred = baseline.predict(X_test)print(round(accuracy_score(y_test, y_pred),4))
Now that we have a baseline model trained we need to evaluate if there is another algorithm that may perform better on our data. Scikit-learn has this useful cheat sheet that will give you an idea of the different algorithms available to solve a classification problem.
The below code loops through a selection of classification algorithms and prints the resulting score. The output is shown below the code.
from sklearn.metrics import accuracy_score, log_lossfrom sklearn.neighbors import KNeighborsClassifierfrom sklearn.svm import SVC, LinearSVC, NuSVCfrom sklearn.tree import DecisionTreeClassifierfrom sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier, GradientBoostingClassifierfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysisfrom sklearn.discriminant_analysis import QuadraticDiscriminantAnalysisclassifiers = [    KNeighborsClassifier(3),    SVC(kernel="rbf", C=0.025, probability=True),    NuSVC(probability=True),    DecisionTreeClassifier(),    RandomForestClassifier(),    AdaBoostClassifier(),    GradientBoostingClassifier()    ]for classifier in classifiers:    model = classifier.fit(X_train, y_train)    print(classifier)    print("model score: %.3f" % model.score(X_test, y_test))
We can see that the RandomForestClassifier performs best for this model so we will select this algorithm to train our final model.
Each machine learning algorithm has a wide number of parameters that are used to control the learning process. These parameters can be changed and depending on the data set can result in an increase in performance for the model. The process of finding the best set of parameters for an algorithm and data set is known as hyperparameter optimisation.
We know from the code we ran in the last section that the random forest classifier performed best on the data set we are using. If you look at the documentation for this model you will see that there are many parameters that can be tuned for this algorithm.
A common method to use for hyperparameter optimisation is known as grid search. Scikit-learn provides a function to perform this called GridSearchCV. We need to pass this function a grid in the form of a python dictionary containing the parameter names and the corresponding list of parameters. This then becomes the parameter space in which the function will search.
The function will then build a model for every combination of parameters for the given classifier. Once this has been performed you will be able to access the results in the form of the best model and the best parameter combination.
For simplicity, in the example below I have chosen to tune only four parameters. They are as follows:
n_estimators: the number of trees in the model.
max_depth: the maximum depth of the tree.
min_samples_split: the minimum number of data points in a node before the node is split.
min_samples_leaf: the minimum amount of data points in each leaf.
The code below performs hyperparameter tuning and prints both the best model score and parameter combination.
from sklearn.model_selection import GridSearchCVn_estimators = [100, 300, 500, 800, 1200]max_depth = [5, 8, 15, 25, 30]min_samples_split = [2, 5, 10, 15, 100]min_samples_leaf = [1, 2, 5, 10]param_grid = dict(n_estimators = n_estimators, max_depth = max_depth,                min_samples_split = min_samples_split,              min_samples_leaf = min_samples_leaf)rf = RandomForestClassifier()grid_search = GridSearchCV(estimator=rf, param_grid=param_grid)best_model = grid_search.fit(X_train, y_train)print(round(best_model.score(X_test, y_test),2))print(best_model.best_params_)
Finally, we use the best model to predict labels on the test set and print a classification report to evaluate its performance in detail. We can see that overall the performance has vastly improved from the baseline model.
from sklearn.metrics import classification_reporty_pred_best = best_model.predict(X_test)print(classification_report(y_test, y_pred_best))
In this article, I have demonstrated the simplest workflow required to develop a machine learning model. There are many more steps typically involved in developing a model, in particular when you are using a real-world data set. These include data cleaning, feature engineering and cross-validation amongst many other possible steps. Once you have grasped the basic steps in this post you can move onto learning about the other elements involved in machine learning.
In this article I only covered classification but for a brief overview of other types of machine learning see — Beginners Guide to the Three Types of Machine Learning.
If you are interested in learning data science see my list of completely free resources for learning data science here— How to Learn Data Science for Free.
Thanks for reading!