Building your first machine learning model
By: Tommy Dangerous
Tags: Development, AI, Developer Tools, Artificial Intelligence, Machine Learning

We’ll show you how to build a basic machine learning model. For a quick background on AI/ML, please check out this blog post for an overview.

Roadmap to develop AI/ML

In this tutorial, we’ll use the Titanic dataset to predict which passengers survived the crash. The dataset includes information on each passenger, the cabin they stayed in, their gender, and more.

High level steps for building an AI/ML model

Data preparation Choose algorithm Hyperparameter tuning Train model Evaluate performance Deploy/Integrate model

Setup

Go to Google Colaboratory, click the top left “File” and click “New notebook”.

Download this file called titanic_survival.csv.

In your “New notebook” on Google Colaboratory, click the folder icon in the top left and then drag the file you just downloaded called “titanic_survival.csv” and drop it into that area.

Data preparation

Here are the steps when preparing data:

Download and split data Add columns Remove columns Impute values Scale values Encode values Select features

1. Download and split data

We need to load the data into memory by downloading it from a website, a database, data warehouse, SaaS tool, etc. Once we download it, we can load it in memory to operate on quickly. Before we split the data, we’ll need to determine which column we want to predict. In this tutorial, we’ll predict which passengers survived.

After that, we need to split the data into 2 parts: 1 for training the AI/ML model (aka train set) and 1 for evaluating the performance of the model (aka test set). The train set will have 80% of the rows from the original data. There are different strategies for splitting the data; however, a common method is to stratify the data so that there is a representative number of rows in both the train set and test set.

2. Add columns

The data you downloaded may not have all the columns you need. You may want to add a few more columns by combining existing columns or performing some sort of calculation. For example, you may want to create a column called “year” which extracts the year of a date value from the birthday column.

3. Remove columns

There may be columns that you don’t think the model should learn from. For example, the model may not care about specific user IDs or email addresses (the email domain might matter). In these cases, we want to remove these columns from the data. By removing these columns, we help the model focus on what matters instead of trying to make sense of data that has no impact on the prediction. For example, a passenger’s ID probably has very little impact on whether they survived the sinking of the Titanic.

4. Impute values

Your data may have missing values in a particular column. The AI/ML model has a hard time knowing what to do with missing values. We can help it by filling in those missing values using some heuristic. For example, there are a lot of missing values in the “Cabin” column. For those with no known cabin, we’ll fill in the value “somewhere out of sight”. For those with missing age, we’ll use the median age to fill in those missing values.

5. Scale values

Adjust the values of number columns to fall within similar ranges so that large numbers (such as seconds since epoch) don’t affect the prediction disproportionately as much as smaller values (such as age).

For example, if you have a column that is in seconds and a column that is in days, the difference in seconds between today and last week is 604,800 seconds. The difference in days between today and last week is 7. If we don’t scale these values, then the model will think the column with seconds has a greater distance between 2 numbers than the column with days.

There are multiple scaling strategies such as standard scaler and normalizer. For more information, check out this thread.

6. Encode values

AI/ML algorithms perform mathematical operations using numbers. We must convert columns that contain strings into a number representation. A common technique is to encode categorical values. For example, we can convert the value “male” to 0 and “female” to 1. Note: we’re going to use one-hot encoding to convert these strings into numbers. For further explanation why, check out this thread.

7. Select features

Now that we’ve prepared our data, we need to select the features we want our model to learn from. There are many techniques for doing this (Mage’s tool handles this automatically for you). For this tutorial, we’ll simply select the features we manually added, scaled, or encoded.

Choose algorithm

Once our data is in a state that is ready to be trained on, we must choose an algorithm to use. Different algorithms are best suited for different types of problems and different types of data. For this tutorial, we’ll use a basic algorithm called logistic regression that’ll help us classify which passengers survived the Titanic crash.

Hyperparameter tuning

An AI/ML model has parameters that aren’t related to the features (aka columns in the data). These “hyper” parameters control how the model behaves throughout its training. When improving AI/ML models, it’s common to try a bunch of different combinations of hyperparameters that’ll yield the best results. We’ll skip this optimization for this tutorial (keep an eye out for a future article on this topic).

Train model

We take the data that was prepared (X_train) and the actual results (y_train) for each row (e.g. whether the passenger survived the Titanic) and feed it into the model. The model will learn from looking at the values in each column and seeing what result it produces (1 for survived, 0 for not survived). Once the model learns from all the data, it will finish training and can be used to make predictions on unseen data.

Evaluate performance

Prepare test data Use model to predict on test data Calculate model accuracy Determine baseline performance and compare

1. Prepare test data

First, we’ll prepare our test data (e.g. add columns, remove columns, impute values, scale values, encode values, and select features) in the same way we did for our train set. One caveat is that we won’t “fit” our standard scaler or our encoders because we only want to “fit” those on the train set.

Note: the code below is an exact copy of the code written above during data preparation for the train set, except we are calling functions on the variable containing the test data. A better engineering practice would be to refactor the code by creating a reusable function that accepts a Pandas dataframe as an argument, calls all the data preparation steps on that dataframe, and returns it.

Here is the code written and not refactored for clarity sake:

2. Use model to predict on test data

Next, we use the model to predict who survives from the test data (remember we split the data earlier during data preparation).

3. Calculate model accuracy

Regression and classification models have different metrics that are used to evaluate the performance of the model. Since we’re using a classification model (even though it’s called logistic regression, it can be used for classifying), we’ll use accuracy as our metric. If there were multiple categories we’re predicting, we’ll also want to use precision and recall as a metric.

4. Determine baseline performance and compare

In order for us to understand how good this accuracy is, we need to establish a baseline. In this specific example, the baseline accuracy will be the number of people who didn’t survive within the test set divided by the number of rows in the test set.

Deploy/Integrate model

Once you trained the model and fine-tuned it to your business needs, it’s time to integrate it into your product or business operations. There are several ways of doing this: you can deploy the model to an online server where the model’s prediction can be accessed via an API request or you can set up your model to perform batch predictions and export those predictions to your data warehouse, data lake, etc.

Deploying your model, maintaining the model, keeping it up-to-date so that it makes relevant predictions, and making sure your feature data is fresh and readily available to retrieve for online predictions is time-consuming, costly, extremely complex, and a non-differentiating skillset. Instead of focusing your energy on this particular aspect, it’s common to rely on other tools for this service. A tool like Mage not only helps you prepare your data and train your model, it also helps you access your model from an API endpoint and keeps the model relevant by retraining it regularly.

Conclusion

Easy AI/ML development tutorial

Here is the link to the entire code.

Additional resources