End-to-end Data Science Project: Predicting used car prices using Regression
By: José Manuel García Portillo
Tags: Regression, Random Forest, Vehicles, Python, Machine Learning

Image by JOERG-DESIGN from Pixabay

Introduction

Approximately 40 million used vehicles are sold each year. Effective pricing strategies can help any company to efficiently sell its products in a competitive market and making profit.

In the automotive sector, pricing analytics play an essential role for both companies and individuals to assess the market price of a vehicle before putting it on sale or buying it.

There are two main goals I want to achieve with this Data Science Project. First, to estimate the price of used cars by taking into account a set of features, based on historical data. Second, to get a better understanding on the most relevant features that help determine the price of a used vehicle.

Data

The data that will be used for this project is accessible at Kaggle and has been scraped from Craigslist, the world’s largest collection of used vehicles for sale.

The Database consists of 423,857 rows and 25 features, one of which will be the continuous dependent variable (“price”) that we want to predict.

Overview of the Database

Methodology

EDA

The numerical features play a big role in this Regression model, so it is important to understand well how are they distributed in the Database.

Statistical data of the numerical features

Our focus will be “price”, “year” and “odometer”. As shown in the picture above, there is a big difference between both the maximum value/minimum value and the percentiles for each of these three features. This is an indicator of the presence of outliers, which can greatly hinder the performance of our model. They will be handled later.

Missing values & Duplicates

After understanding the database that we will be working with, it is important to always check if there are any missing values (NaN).

Missing values of some features in the Database

Most features have a high amount of missing values, but it is not practical to just delete all the rows with NaN values since that would thin out the database too much.

For now, I will leave it at that and just drop some of the features that don’t bring any added value. Keeping them will just make it harder for the model to interpret the database.

Set of features dropped from the Database

Lastly, it is also necessary to check if there is any case of duplicate values in this database.

In this case, there are several duplicates in the Database. They will be dealt with by dropping the existing duplicate rows except for the first one in each case, which will be kept.

Handling duplicates in the Database

Feature engineering

First, I will be using Regular Expressions on the column “description”. This feature contains the text with the advertisement of each used car.

The goal is to get key information from there in order to fill the NaN values of other columns in the database. The columns we want to fill are those with categorical values that don’t have too many uniques values:

Pattern Matching with Regular Expressions

After doing this, there are values that have been added inconsistently. For example, we have these kinds of values in the same column: ‘awd’ and ‘ awd ’. To unify the strings, the strip() method will be used.

Handling Inconsistent Data Entry

With this Regex step, the database has gotten ten of thousands of new entries, filling numerous missing values with consistent data. Since we have no need for the “description” feature anymore, I will delete it.

Onto the next step, it is time to handle the outliers in the database. The three columns that will be taken care of are: “odometer”, “price” and “year”.

Let’s start by visualising the two first ones:

Distribution of values of Price and Odometer

Since “price” had such a big difference between the minimum value and the 25% percentile as well as between the maximum value and the 75% percentile, I will be leaving out 10% of the values on each end.

Dropping 10% of the values on each end of Price

Next, it is time to focus on “odometer”. This feature has NaN values so it is trickier to deal with. Let’s use scatterplot to visualise the outliers.

Visualising the outliers on Odometer

Since the outliers start at approximately 3,000,000; I will be dropping the values that exceeds that. Also, it is a good idea to drop the minimum value (0) since it greatly differs from the 25% percentile. With this, the feature “odometer” has also been successfully adjusted.

Odometer feature after being adjusted

To end this step, the outliers of the feature “year” will be handled the same way it was done in “odometer”.

Moving forward, there are another two relevant features I want to deal with by using other existing columns in the database. These are: “condition” and “title_status”. They are pretty similar and basically inform us about the state of used cars (clean, dirty, new, excellent, bad…).

To that end, I will be utilising the “odometer” values. The “odometer” feature is a continuous feature so it is better to group its values into several bins to serve as a reference.

Title_status VS Odometer

Most of the values of “title_status” are clean, so I will be adding “clean” to all the NaN values of this column. Next, it is time to check “condition”:

Condition VS Odometer

In this case, there is a wider variance on values. Using the chart above as a guide, I will fill the NaN values for “condition”. Finally, since both “condition” and “title_status” are interconnected, it is a good idea to join them into one feature.

Filling NaN values and joining features

Finally, it is now time to have a look into the missing values chart again. I will apply other methods to deal with them.

Missing values in the Database

Taking into account the number of missing values, I will be dividing the features into three different groups and dropping “size” which has the most NaN values.

First, the ones with less than 5% NaN values will have the rows with missing values dropped.

Dropping the rows with NaN values

Second, the ones with less than 20% NaN values will be filled by using the fillna() function.

Using fillna() in features with missing values

Finally, the last feature “cylinders” will be handled by using another column (“drive”) to infer its values since I feel it is a relevant feature for the model.

Utilising another column to fill NaN values

To end this Feature Engineering section, it is a good idea to join “region” and “state” into one feature since they are deeply related.

Joining Region and State into one feature

Categorical Variable Encoding

In the database there are both numerical and categorical variable. But, in order to carry on with the following steps, it is required to encode those categorical values into numerical data. This will be done with the help of Label Encoding.

Categorical Variable Encoding

Correlations

When two variables are highly correlated, they have the same effect on the dependent variable that we want to predict. In that case, it is recommendable dropping one of them.

Correlation Matrix

In this case, none of the features were highly correlated so we leave them as they are.

Modelling

It is time to start the modelling for this project. This time, I will be only using the Random Forest Regression model to get the “price” predictions.

The Random Forest algorithm is an ensemble of decision trees. It will be surely come in handy for our project since it has the power to handle a large data set with higher dimensionality and it won’t allow over-fitting trees in the model.

To begin the modelling, we want to divide the database into Train and Test Set. It will be done on a distribution of 80:20. As usual, we will first re-index the database and put the dependent variable “price” as the last column for a simpler splitting.

Re-indexing and splitting into Train/Test Set

After the split, it is necessary to implement Feature Scaling since there is a big variance in the values of the independent features. It is really important to always do it AFTER splitting the database to avoid data leakage into the Machine Learning model.

Applying Feature Scaling to X_train and X_test

We then fit the model in the Training Set using the Cross-Validation method in order to estimate the accuracy of the model more precisely. I will be utilising the R²(R-squared) metric to evaluate the performance of the model.

R ² is a scale-free score that implies it doesn’t matter whether the values are too large or too small, the R² will always be less than or equal to 1. The closer to 1, the better.

As a side note, since percentages are more insightful when evaluating the result, I will be transforming the values we get from R² into percentages.

Evaluating the model based on the R² metric

We got a score of more than 85.48% right off the start. Not bad at all. But we can improve it further by tuning the Hyper-parameter.

Once the hyper-parameters of the model are optimised, it is time to train it again (using Cross-Validation) and check its performance on the Training Set one more time.

Hyper-parameter Tuning with the Random Forest Regression model

As we can see from the picture above, the score got even better after doing the optimisation of the Hyper-parameters.

Finally, once the Training is finished and the model has learned from the training data, all that is left is to check how well does it perform in the Test Set. This will be the moment of truth for our model.

Prediction on the Test Set

The results seem to be pretty solid, which is shown by the good performance of our model in both Train and Test Set. The fact that it got even more accurate in its predictions in the Test Set (87.03%) is specially remarkable.

Results

For this project, I have only used a single model in order to predict the price of used cars: the Random Forest Regressor.

It has shown an excellent performance in such a big dataset and it has performed consistently throughout the Training and Testing process. Even more, the results of the Test Set are better than in the Training Set, with a 87.03% of accuracy in its predictions.

One of the goals of the project was to create a model that was able to estimate the price of used cars and we already achieved it.

The second goal is to find which features are the most relevant ones when estimating the dependent variable “price”:

Importance of each feature for the Random Forest model

Conclusion

With this project, we have built a model that can predict with a 87,03% of accuracy the price of used cars, given a set of features. This information can have an enormous value for both companies and individuals when trying to understand how to estimate the value of a vehicle and, more importantly, the key factors that determine its pricing.

As expected, the year of the vehicle is by far the main factor when calculating the price with almost a 43%, followed by odometer. Interestingly enough, I expected the state of the car and the odometer to be deeply related but there is a big gap in the difference of relevance between both measures.

That said, it seems the region also plays a part, which totally makes sense. There might be more general vehicles that are liked everywhere but specialised cars like sport or convertibles would be a better fit in warmer areas whilst bigger trucks and SUVs would play a better role in colder places.

Mastering the art of pricing is not an easy task, but with the study of historical data it is possible to find patterns that lead to accurate results. Acquiring this knowledge can provide you with a comparative advantage before putting a vehicle on sale or buying it on the market.