Predicting House Prices with Machine Learning
By: John Ade-Ojo
Tags: Data Science, Artificial Intelligence, Kaggle, Machine Learning, Python

Intro

I thought I’d try my hand at Kaggle’s advanced regression techniques competition taking you along on the journey with me. If you’re getting into machine learning and want to see a project end to end, please stick around. I will walk you through the steps I’ve taken and attempt to deliver a crash course in machine learning at the same time.

Objective & Data

The competition goal is to predict sale prices for homes in Ames, Iowa. You’re given a training and testing data set in csv format as well as a data dictionary.

Training: Our training data consists of 1,460 examples of houses with 79 features describing every aspect of the house. We are given sale prices (labels) for each house. The training data is what we will use to “teach” our models.

Testing: The test data set consists of 1,459 examples with the same number of features as the training data. Our test data set excludes the sale price because this is what we are trying to predict. Once our models have been built we will run the best one the test data and submit it to the Kaggle leaderboard.

You can familiarize yourself with the data on the competition page.

Task: Machine learning tasks are usually split into three categories; supervised, unsupervised and reinforcement. For this competition, our task is supervised learning.

Supervised learning uses examples and labels to find patterns in data

It’s easy to recognise the type of machine learning task in front of you from the data you have and your objective. We’ve been given housing data consisting of features and labels, and we’re tasked with predicting the labels for houses outside of our training data.

Tools

I used Python and Jupyter notebooks for the competition. Jupyter notebooks are popular among data scientist because they are easy to follow and show your working steps.

Please be aware this code is not for production purposes, it doesn’t follow software engineering best practices. I’ve sacrificed that somewhat for explainability.

Libraries: These are frameworks in python to handle commonly required tasks. I Implore any budding data scientists to familiarise themselves with these libraries:

Pandas — For handling structured data

Scikit Learn — For machine learning

NumPy — For linear algebra and mathematics

Seaborn — For data visualization

Project Pipeline

Generally speaking, machine learning projects follow the same process. Data ingestion, data cleaning, exploratory data analysis, feature engineering and finally machine learning.

The pipeline is not linear and you might find you have to jump back and forth between different stages. It’s important I mention this because tutorials often make you believe the process is much cleaner than in reality. So please keep this in mind, your first machine learning project might be a mess.

Before we begin: I’ll reiterate, machine learning is an iterative process and It’s rarely straightforward! Please do not be discouraged if you find yourself lost in an ML project. Keep reading, keep experimenting, keep asking questions and one day it will click.

The rest of this article will talk through the stages of the project pipeline. Where useful, I will drop in code examples from python. The full end to end project is available to use and play with here. I’ll share a link for this at the end of the article.