Car Price Prediction in Python
Tags: python, datascience, beginners, machinelearning
Published: 2021-07-13 19:36:54

Last week, we did some  [Exploratory Data Analysis to a car dataset](https://dev.to/anderrv/exploratory-data-analysis-in-python-47a8) . After working with the dataset and gathering many insights, we'll focus on price prediction today.

 [The dataset comprises cars for sale in Germany](https://www.zenrows.com/datasets/germany-cars?utm_source=devto&utm_medium=blog&utm_campaign=car_price_prediction) , the registration year being between 2011 and 2021. So we can assume that it is an accurate representation of market price nowadays.

### Prerequisites

For the code to work, you will need  [python3 installed](https://www.python.org/downloads/) . Some systems have it pre-installed. After that, install all the necessary libraries by running pip install.

```
# last week
pip install pandas matplotlib squarify seaborn
# new libs
pip install scipy sklearn catboost statsmodels
``` 

## Data Cleaning

Let's say we want to predict car prices based on attributes. For that, we will train a model. Let's start by replacing registration year with age and removing make and model - we won't be using them for the predictions.

```python
cars['age'] = datetime.now().year - cars['year']
cars = cars.drop('year', 1)
cars = cars.drop('make', 1)
cars = cars.drop('model', 1)
```

We talked about outliers last week, and now it's time to remove them. We'll do that by removing the items in which the  [z score](https://en.wikipedia.org/wiki/Standard_score)  for the price, horsepower, or mileage is higher than 3. In short, this means taking out the values that deviate more than three standard deviations from the mean.

Before that, `dropna` will remove all lines with empty or null values.

```python
from scipy import stats 
 
cars = cars.dropna() 
cars = cars[stats.zscore(cars.price) < 3] 
cars = cars[stats.zscore(cars.hp) < 3] 
cars = cars[stats.zscore(cars.mileage) < 3] 
```

Next, we will replace the category values (offer type and gear) with boolean markers. In practice, this means creating new columns for each category type (i.e., for gear, it will be Automatic, Manual, and Semi-automatic).

There is a function in pandas to do that: `get_dummies`.

```python
offerTypeDummies = pd.get_dummies(cars.offerType)
cars = cars.join(offerTypeDummies)
cars = cars.drop('offerType', 1)

gearDummies = pd.get_dummies(cars.gear)
cars = cars.join(gearDummies)
cars = cars.drop('gear', 1)

# Replacing to avoid late collisions with Makes
cars['fuel'] = cars['fuel'].replace('Others', 'OthersFuel')
fuelDummies = pd.get_dummies(cars.fuel)
cars = cars.join(fuelDummies)
cars = cars.drop('fuel', 1)
```

![Car Dummies](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g49favsv7nmpm7gb59i3.jpg)

## Checking Correlation Visually

We are going to plot variable correlation using `seaborn heatmap`. It will show us graphically which variables are positively or negatively correlated.

The highest correlation shows for age with mileage - sounds fine - and price with horsepower - no big news either. And looking at negative correlation, price with age - which also seems natural.

We can ignore the relation between Manual and Automatic since it's evident that you will only have one or the other - there are almost no Semi-automatics.

```python
import seaborn as sns

sns.heatmap(cars.corr(), annot=True, cmap='coolwarm')
plt.show()
```

![Car Correlation Heatmap](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ru9iwi3r9daq6agecbdr.png)

For a double check, we are going to plot horsepower and mileage variables with the price. We'll do it with `seaborn jointplot`.

It will plot all the entries and a line for the regression. For brevity, there is only one code snippet. The second one would be the same, replacing *hp* with *mileage*.

```python
sns.set_theme(style="darkgrid")
sns.jointplot(x="hp", y="price", data=cars,
	kind="reg", color="m", line_kws={'color': 'red'})
plt.show()
```
![Price and Horsepower Correlation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uutqlsugshz0g9izzccz.png)
![Price and Mileage Correlation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eafmnxug5dzl2tk485xo.png)

## Price Prediction

We are reaching the critical part. We will try **three different prediction models** and see which one performs better.

We need two variables, Y and X, containing price and all the remaining columns. We will use these new variables for the other models too. Then, we split the data for training and testing in a 70%-30% distribution.

**Disclaimer**: we did several tests with all the models and chose the best results for each model. Not the same vars apply, and some "magic" numbers will appear. We adjusted those mainly through trial and error.

```python
from sklearn.model_selection import train_test_split

X = cars.drop('price', 1)
Y = cars.price
X_train, X_test, y_train, y_test = train_test_split(
	X, Y, train_size=0.7, test_size=0.3, random_state=100)
```

### linear_model from sklearn

To train the first `LinearRegression` model, we will pass the train data to the *fit* method and then the test data to *predict*.

To check the results, we'll be using *R-squared* for all of them. In this case, the result is **0.81237**.

```python
from sklearn import linear_model
from sklearn.metrics import r2_score

lm = linear_model.LinearRegression()
lm.fit(X_train, y_train)
y_pred = lm.predict(X_test)
print(r2_score(y_true=y_test, y_pred=y_pred)) # 0.81237
```

### Regressor from CatBoost

Next, we'll use `Regressor` from `CatBoost`. The model's created with some numbers that we adjusted by testing. Similar to the previous one, fit the model with the train data and check the score, resulting in **0.92416**.

There is a big difference since this method is much slower, more than 20 seconds. It might be a closer match, but not a good option if it must run instantly.

```python
from catboost import CatBoostRegressor

model = CatBoostRegressor(iterations=6542, learning_rate=0.03)
model.fit(
	X_train, y_train,
	eval_set=(X_test, y_test),
)
print(model.score(X, Y)) # 0.92416
```

### OLS from statsmodels

For `statsmodels`, we will change X's value and take only mileage, hp, and age. The difference is almost 10% better than with the previous values.

R-squared is **0.91823**, and it runs in under two seconds - counting the data load.

```python
import statsmodels.api as sm

X = cars[['mileage', 'hp', 'age']]

model = sm.OLS(Y, X).fit()
predictions = model.predict(X)
print(model.rsquared) # 0.91823
```

## Extra Ball: Best Prediction

What happens if we do not drop make and model? Two of the models would perform worse, but not `CatBoost`. It will take much longer and use more space. We would have more than 700 feature columns. But it is worth it if you are after accuracy.

For brevity, we will not reproduce all the manipulations we did previously. Instead of dropping make and model, create dummies for them and then continue as before.

```python
makeDummies = pd.get_dummies(cars.make)
cars = cars.join(makeDummies)
cars = cars.drop('make', 1)

modelDummies = pd.get_dummies(cars.model)
cars = cars.join(modelDummies)
cars = cars.drop('model', 1)

# the rest of the features, just as before
# split train and test data

model = CatBoostRegressor(iterations=6542, learning_rate=0.03)
model.fit(
	X_train, y_train,
	eval_set=(X_test, y_test),
)
print(model.score(X, Y)) # 0.9664
```

This model exposes a method to obtain feature importance. We can use that data with a bar chart to check which features affect the prediction the most. We will limit them to 20, but as you'll see, two of them - excluding price itself - carry all the weight.

Age not being significant might look suspicious at first glance. But it makes sense. As we saw in the correlation graph, age and mileage go hand in hand. So there is no need for the two of them to carry all the weight.

```python
sorted_feature_importance = model.get_feature_importance().argsort(
	)[-20:]
plt.barh(
	cars.columns[sorted_feature_importance],
	model.feature_importances_[sorted_feature_importance]
)
plt.xlabel("Feature Importance")
plt.show()
```

![Price Prediction Feature Importance](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gbtdwq2zoubi3hf4kdr0.png)

## Estimate Car Price

Let's say that you want to buy or sell your car. You collect the features we provide (mileage, year, etcetera). How to use the prediction model for that?

We'll choose `CatBoost` again and use their `predict` method for that. We would need to go all the way again by transforming all the data with dummies, so we'll summarize. This process would be extracted and performed equally for training, test, or actual data in a real-world app.

We will also need to add all the empty features (i.e., all the other makes) that the model supports.

Here we present an example with three cars for sale. We manually entered all the initial features (price included), so we can compare the output. As you'll see, the predictions are not far from the actual price.

```python
realData = pd.DataFrame.from_records([
	{'mileage': 87000, 'make': 'Volkswagen', 'model': 'Gold',
		'fuel': 'Gasoline', 'gear': 'Manual', 'offerType': 'Used',
		'price': 12990, 'hp': 125, 'year': 2015},
	{'mileage': 230000, 'make': 'Opel', 'model': 'Zafira Tourer',
		'fuel': 'CNG', 'gear': 'Manual', 'offerType': 'Used',
		'price': 5200, 'hp': 150, 'year': 2012},
	{'mileage': 5, 'make': 'Mazda', 'model': '3', 'hp': 122,
		'gear': 'Manual', 'offerType': 'Employee\'s car',
		'fuel': 'Gasoline', 'price': 20900, 'year': 2020}
])

realData = realData.drop('price', 1)
realData['age'] = datetime.now().year - realData['year']
realData = realData.drop('year', 1)

# all the other transformations and dummies go here

fitModel = pd.DataFrame(columns=cars.columns)
fitModel = fitModel.append(realData, ignore_index=True)
fitModel = fitModel.fillna(0)

preds = model.predict(fitModel)
print(preds) # [12213.35324984 5213.058479 20674.08838559]
```

## Conclusion

As a quick note on the three models, `sklearn` performs a bit worse. And the other two are pretty similar in results - if excluding makes and models - but not on time spent training. So it might be a crucial aspect to consider when choosing between them.

If you are after high accuracy, train the `CatBoost` model with all the available data. It might take up to a minute, but it can be stored in a file and instantly loaded when needed.

As you've seen, loading a  [ZenRows generated dataset](https://www.zenrows.com/datasets?utm_source=devto&utm_medium=blog&utm_campaign=car_price_prediction)  into `pandas` is quite simple. Then, there are some steps to perform: describe, explore manually, look at the values, check for empty or nulls.

These are everyday tasks when first testing a dataset. From there, standard practices such as generating dummies or removing outliers.
And then the juicy part. In this case, price prediction using linear regression, but it could be anything.

---

Originally published at [https://www.zenrows.com](https://www.zenrows.com/blog/car-price-prediction-in-python?utm_source=devto&utm_medium=blog&utm_campaign=car_price_prediction)