House Prices Prediction Using Deep Learning | by Mahsa Mir | Towards Data Science

In this tutorial, we’re going to create a model to predict House prices🏡 based on various factors across different markets.
The goal of this statistical analysis is to help us understand the relationship between house features and how these variables are used to predict house price.
Predict the house price
Using two different models in terms of minimizing the difference between predicted and actual rating
Data used: Kaggle-kc_house DatasetGitHub: you can find my source code here
First, Let’s import the data and have a look to see what kind of data we are dealing with:
#import required librariesimport pandas as pdimport numpy as npimport seaborn as snsimport matplotlib.pyplot as plt#import DataData = pd.read_csv('kc_house_data.csv')Data.head(5).T#get some information about our Data-SetData.info()Data.describe().transpose()
The following features have been provided:✔️Date: Date house was sold✔️Price: Price is prediction target✔️Bedrooms: Number of Bedrooms/House✔️Bathrooms: Number of bathrooms/House✔️Sqft_Living: square footage of the home✔️Sqft_Lot: square footage of the lot✔️Floors: Total floors (levels) in house✔️Waterfront: House which has a view to a waterfront✔️View: Has been viewed✔️Condition: How good the condition is ( Overall )✔️Grade: grade given to the housing unit, based on King County grading system✔️Sqft_Above: square footage of house apart from basement✔️Sqft_Basement: square footage of the basement✔️Yr_Built: Built Year✔️Yr_Renovated: Year when house was renovated✔️Zipcode: Zip✔️Lat: Latitude coordinate✔️Long: Longitude coordinate✔️Sqft_Living15: Living room area in 2015(implies — some renovations) ✔️Sqft_Lot15: lotSize area in 2015(implies — some renovations)
Let’s plot couple of features to get a better feel of the data
#visualizing house pricesfig = plt.figure(figsize=(10,7))fig.add_subplot(2,1,1)sns.distplot(Data['price'])fig.add_subplot(2,1,2)sns.boxplot(Data['price'])plt.tight_layout()#visualizing square footage of (home,lot,above and basement)fig = plt.figure(figsize=(16,5))fig.add_subplot(2,2,1)sns.scatterplot(Data['sqft_above'], Data['price'])fig.add_subplot(2,2,2)sns.scatterplot(Data['sqft_lot'],Data['price'])fig.add_subplot(2,2,3)sns.scatterplot(Data['sqft_living'],Data['price'])fig.add_subplot(2,2,4)sns.scatterplot(Data['sqft_basement'],Data['price'])#visualizing bedrooms,bathrooms,floors,gradefig = plt.figure(figsize=(15,7))fig.add_subplot(2,2,1)sns.countplot(Data['bedrooms'])fig.add_subplot(2,2,2)sns.countplot(Data['floors'])fig.add_subplot(2,2,3)sns.countplot(Data['bathrooms'])fig.add_subplot(2,2,4)sns.countplot(Data['grade'])plt.tight_layout()
With distribution plot of price, we can visualize that most of the prices are between 0 and around 1M with few outliers close to 8 million (fancy houses😉). It would make sense to drop those outliers in our analysis.
It is quit useful to have a quick overview of different features distribution vs house price.
Here, I’m breaking the date columns down to years and months to see how is the house price is changing.
#let's break date to years, monthsData['date'] = pd.to_datetime(Data['date'])Data['month'] = Data['date'].apply(lambda date:date.month)Data['year'] = Data['date'].apply(lambda date:date.year)#data visualization house price vs months and yearsfig = plt.figure(figsize=(16,5))fig.add_subplot(1,2,1)Data.groupby('month').mean()['price'].plot()fig.add_subplot(1,2,2)Data.groupby('year').mean()['price'].plot()
Let’s check if we have a Null Data and also drop some columns that we do not need (this data set does not have some missing values)
# check if there are any Null valuesData.isnull().sum()# drop some unnecessary columnsData = Data.drop('date',axis=1)Data = Data.drop('id',axis=1)Data = Data.drop('zipcode',axis=1)
Data is divided into the Train set and Test set. We use the Train set to make the algorithm learn the data’s behavior and then check the accuracy of our model on the Test set.
Features (X): The columns that are inserted into our model will be used to make predictions.
Prediction (y): Target variable that will be predicted by the features
X = Data.drop('price',axis =1).valuesy = Data['price'].values#splitting Train and Test from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=101)
Feature scaling will help us see all the variables from the same lens (same scale), it will also help our models learn faster.
#standardization scaler - fit&transform on train, fit only on testfrom sklearn.preprocessing import StandardScalers_scaler = StandardScaler()X_train = s_scaler.fit_transform(X_train.astype(np.float))X_test = s_scaler.transform(X_test.astype(np.float))
Multiple Linear Regression is an extension of Simple Linear Regression (read more here) and assume that there is a linear relationship between a dependent variable Y and independent variables X
Let’s wrap the training process in our Regression model:
# Multiple Liner Regressionfrom sklearn.linear_model import LinearRegressionregressor = LinearRegression()  regressor.fit(X_train, y_train)#evaluate the model (intercept and slope)print(regressor.intercept_)print(regressor.coef_)#predicting the test set resulty_pred = regressor.predict(X_test)#put results as a DataFramecoeff_df = pd.DataFrame(regressor.coef_, Data.drop('price',axis =1).columns, columns=['Coefficient']) coeff_df
by visualizing the residual we can see that is normally distributed (proof of having linear relationship with the dependent variable)
# visualizing residualsfig = plt.figure(figsize=(10,5))residuals = (y_test- y_pred)sns.distplot(residuals)
Let’s compare actual output and predicted value to measure how far our predictions are from the real house prices.
#compare actual output values with predicted valuesy_pred = regressor.predict(X_test)df = pd.DataFrame({'Actual': y_test, 'Predicted': y_pred})df1 = df.head(10)df1# evaluate the performance of the algorithm (MAE - MSE - RMSE)from sklearn import metricsprint('MAE:', metrics.mean_absolute_error(y_test, y_pred))  print('MSE:', metrics.mean_squared_error(y_test, y_pred))  print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))print('VarScore:',metrics.explained_variance_score(y_test,y_pred))
Let’s create a baseline neural network model for the regression problem. Starting with all of the needed functions and objects.
# Creating a Neural Network Modelfrom tensorflow.keras.models import Sequentialfrom tensorflow.keras.layers import Dense, Activationfrom tensorflow.keras.optimizers import Adam
Since we have 19 features, let’s insert 19 neurons as a start, 4 hidden layers and 1 output layer due to predict house Price.
Also, ADAM optimization algorithm is used for optimizing loss function (Mean squared error)
# having 19 neuron is based on the number of available featuresmodel = Sequential()model.add(Dense(19,activation='relu'))model.add(Dense(19,activation='relu'))model.add(Dense(19,activation='relu'))model.add(Dense(19,activation='relu'))model.add(Dense(1))model.compile(optimizer='Adam',loss='mes')
Then, we train the model for 400 epochs, and each time record the training and validation accuracy in the history object. To keep track of how well the model is performing for each epoch, the model will run in both train and test data along with calculating the loss function.
model.fit(x=X_train,y=y_train,          validation_data=(X_test,y_test),          batch_size=128,epochs=400)model.summary()
loss_df = pd.DataFrame(model.history.history)loss_df.plot(figsize=(12,8))
y_pred = model.predict(X_test)from sklearn import metricsprint('MAE:', metrics.mean_absolute_error(y_test, y_pred))  print('MSE:', metrics.mean_squared_error(y_test, y_pred))  print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))print('VarScore:',metrics.explained_variance_score(y_test,y_pred))# Visualizing Our predictionsfig = plt.figure(figsize=(10,5))plt.scatter(y_test,y_pred)# Perfect predictionsplt.plot(y_test,y_test,'r')
# visualizing residualsfig = plt.figure(figsize=(10,5))residuals = (y_test- y_pred)sns.distplot(residuals)
We made it!💪 we have predicted the house price using two different ML model algorithms.
The score of our Multiple Linear Regression is around 69%, so this model had room for improvement. Then we got an accuracy of ~81% with Keras Regression model. Also, notice that RMSE (loss function) is lower for Keras Regression model which shows that our prediction is closer to actual rating price.
Without surprise, this score can be improved through feature selection or using other regression models.
Thank you for reading🤓. Again feedback is always welcome!