Regression using sklearn on KC Housing Dataset | by Nikhil Kumar Mutyala | Towards Data Science

In order to predict The King County’s home prices, I chose the housing price dataset that was sourced from Kaggle. This dataset contains house sale prices for King County, which includes Seattle. It includes homes sold between May 2014 and May 2015. It has many characteristics of learning, and the dataset can be downloaded from here.
The overall idea of regression is to examine two things: (1) does a set of predictor variables do a good job in predicting an outcome (dependent) variable? (2) Which variables in particular are significant predictors of the outcome variable, and in what way do they–indicated by the magnitude and sign of the beta estimates–impact the outcome variable? These regression estimates are used to explain the relationship between one dependent variable and one or more independent variables.
Simple Linear Regression
‘bedrooms’ vs ‘price’‘grade’ vs ‘price’
‘bedrooms’ vs ‘price’
‘grade’ vs ‘price’
Multiple Regression
‘bedrooms’,’grade’, ‘sqft_living’, ‘sqft_above’‘bedrooms’,’bathrooms’, ‘sqft_living’, ‘sqft_lot’, ‘floors’, ‘waterfront’, ‘view’, ‘grade’, ‘sqft_above’, ‘sqft_basement’, ‘lat’, ‘sqft_living15’
‘bedrooms’,’grade’, ‘sqft_living’, ‘sqft_above’
‘bedrooms’,’bathrooms’, ‘sqft_living’, ‘sqft_lot’, ‘floors’, ‘waterfront’, ‘view’, ‘grade’, ‘sqft_above’, ‘sqft_basement’, ‘lat’, ‘sqft_living15’
Polynomial Regression
degree=2degree=3
degree=2
degree=3
In this dataset the sales price of houses in King County, Seattle are present. It includes homes sold between May 2014 and May 2015. Before doing anything we should first know about the dataset what it contains what are its features and what is the structure of data.
By observing the data, we can know that the price is dependent on various features like bedrooms(which is most dependent feature), bathrooms, sqft_living(second most important feature), sqft_lot, floors etc. The price is also dependent on the location of the house where it is present. The other features like waterfront, view are less dependent on the price. Of all the records, there are no missing values, which helps us creating better model.
import required libraries.
#importing numpy and pandas, seabornimport numpy as np #linear algebraimport pandas as pd #datapreprocessing, CSV file I/Oimport seaborn as sns #for plotting graphsimport matplotlib.pyplot as plt
Read the CSV file.
df = pd.read_csv("../input/housesalesprediction/kc_house_data.csv")
Pandas dataframe.info() function is used to get a concise summary of the dataframe. It comes really handy when doing exploratory analysis of the data. To get a quick overview of the dataset we use the dataframe.info() function.
df.info()
Pandas head() method is used to return top n (5 by default) rows of a data frame or series.
df.head()
Pandas shape fucntion isused to return size, shape and dimensions of data frames and series.
#finding no of rows and columnsdf.shape
Calling sum() of the DataFrame returned by isnull() will give a series containing data about count of NaN in each column.
df.isnull().sum()
Finding the count of no of bedrooms.
df['bedrooms'].value_counts()
Finding the count of waterfront.
Finding the count of grade.
df['grade'].value_counts()
Finding the count of condition.
df['condition'].value_counts()
A countplot is plotted for bedrooms.
sns.countplot(df.bedrooms,order=df['bedrooms'].value_counts().index)
A barplot is plotted between sqft living and prices to get an overview of how the price changes with sqft.
A barplot is plotted between the sqft above and prices to see how the price changes with the sqft above.
fig,axes=plt.subplots(nrows=1,ncols=1,figsize=(15,10))plt.title("house prices by sqft_above")plt.xlabel('sqft_above')plt.ylabel('house prices')plt.legend()sns.barplot(x='sqft_above',y='price',data=df)
A histogram is plotted for sqft living.
plt.hist('sqft_living',data=df,bins=5)
A distplot is plotted for sqft living to see if the data is skewed or not
fig,axes=plt.subplots(nrows=1,ncols=1,figsize=(15,10))sns.distplot(df['sqft_living'],hist=True,kde=True,rug=False,label='sqft_living',norm_hist=True)
A distplot is plotted for sqft above to see if the data is skewed or not
fig,axes=plt.subplots(nrows=1,ncols=1,figsize=(15,10))sns.distplot(df['sqft_above'],hist=True,kde=True,rug=False,label='sqft_above',norm_hist=True)
Finding the mean, mode and median of sqft living.
print('Mean',round(df['sqft_living'].mean(),2))print('Median',df['sqft_living'].median())print('Mode',df['sqft_living'].mode()[0])
Through graphs we observe that the sqft living=1300 has more values.
len(df[df['sqft_living']==1300])
Making sure we covered all the relations, we plot a correlation between all the features using a heatmap.
A heatmap is a two-dimensional graphical representation of data where the individual values that are contained in a matrix are represented as colors.
def correlation_heatmap(df1):    _,ax=plt.subplots(figsize=(15,10))    colormap=sns.diverging_palette(220,10,as_cmap=True)    sns.heatmap(df.corr(),annot=True,cmap=colormap)    correlation_heatmap(df)
Now that we have got enough inofrmation about the data, we start linear regression.
For linear Regression, we are using linear_model from sklearn function.
Scikit-learn provides a range of supervised and unsupervised learning algorithms via a consistent interface in Python. The library is built upon the SciPy (Scientific Python) that must be installed before you can use scikit-learn. Extensions or modules for SciPy care conventionally named SciKits. As such, the module provides learning algorithms and is named scikit-learn.
We import train_test_split. This splits the data into required ratio (ex: 80–20), of which a certain ratio is considered for training the data, and the rest for testing the data. The data is trained to predict a line and then the test data is used to see if the line fits perfectly or not.
PolynomialFeatures generates a new feature matrix consisting of all polynomial combinations of the features with degree less than or equal to the specified degree.
The metrics is imported as the metric module implements functions assessing prediction error for specific purposes.
In KNeighborsRegressor the target is predicted by local interpolation of the targets associated of the nearest neighbors in the training set.
from sklearn.model_selection import train_test_splitfrom sklearn import linear_modelfrom sklearn.neighbors import KNeighborsRegressorfrom sklearn.preprocessing import PolynomialFeaturesfrom sklearn import metricsfrom mpl_toolkits.mplot3d import Axes3D%matplotlib inline
Here we splitting the data into 80:20 ratio of which train_size is 80%, test_size is 20%. train_test_split splits arrays or matrices into random train and test subsets. That means that everytime you run it without specifying random_state , you will get a different result, this is expected behavior. In order to get the same train ans test subsets we declare a random state. Here the x is ‘sqft_living’ and y is ‘price’. We are reshaping the x_train and y_train and the data is fitted. The x_test and y_test are used to predict the accuracy of the model. Here first we are calculating the squared mean error of y_test. The squared mean error for training and test are found. The intercept and coefficient of the line are found.
train_data,test_data=train_test_split(df,train_size=0.8,random_state=3)reg=linear_model.LinearRegression()x_train=np.array(train_data['sqft_living']).reshape(-1,1)y_train=np.array(train_data['price']).reshape(-1,1)reg.fit(x_train,y_train)x_test=np.array(test_data['sqft_living']).reshape(-1,1)y_test=np.array(test_data['price']).reshape(-1,1)pred=reg.predict(x_test)print('linear model')mean_squared_error=metrics.mean_squared_error(y_test,pred)print('Sqaured mean error', round(np.sqrt(mean_squared_error),2))print('R squared training',round(reg.score(x_train,y_train),3))print('R sqaured testing',round(reg.score(x_test,y_test),3) )print('intercept',reg.intercept_)print('coefficient',reg.coef_)
R-squared testing: 0.496
A scatterplot graph is plotted for x_test, y_test. The data is spread over the graph. Now the line obtained from above is plotted to see how it fits for the data.
_, ax = plt.subplots(figsize= (12, 10))plt.scatter(x_test, y_test, color= 'darkgreen', label = 'data')plt.plot(x_test, reg.predict(x_test), color='red', label= ' Predicted Regression line')plt.xlabel('Living Space (sqft)')plt.ylabel('price')plt.legend()plt.gca().spines['right'].set_visible(False)plt.gca().spines['right'].set_visible(False)
Here we are splitting the data in 80:20 ratio, of which train_size is 80% and test_size is 20%. Here x is ‘grade’ and y is ‘price’. We are reshaping the x_train and y_train and the data is fitted. The x_test and y_test are used to predict the accuracy of the model. Here first we are calculating the squared mean error of y_test. The squared mean error for training and test are found. The intercept and coefficient of the line are found.
train_data,test_data=train_test_split(df,train_size=0.8,random_state=3)reg=linear_model.LinearRegression()x_train=np.array(train_data['grade']).reshape(-1,1)y_train=np.array(train_data['price']).reshape(-1,1)reg.fit(x_train,y_train)x_test=np.array(test_data['grade']).reshape(-1,1)y_test=np.array(test_data['price']).reshape(-1,1)pred=reg.predict(x_test)print('linear model')mean_squared_error=metrics.mean_squared_error(y_test,pred)print('squared mean error',round(np.sqrt(mean_squared_error),2))print('R squared training',round(reg.score(x_train,y_train),3))print('R squared testing',round(reg.score(x_test,y_test),3))print('intercept',reg.intercept_)print('coeeficient',reg.coef_)
R-squared testing: 0.46
The boxplot is plotted for ‘grade’, ‘bedrooms’ and ‘bathrooms’ with respective to ‘price’.
fig,ax=plt.subplots(2,1,figsize=(15,10))sns.boxplot(x=train_data['grade'],y=train_data['price'],ax=ax[0])sns.boxplot(x=train_data['bedrooms'],y=train_data['price'],ax=ax[1])_ , axes = plt.subplots(1, 1, figsize=(15,10))sns.boxplot(x=train_data['bathrooms'],y=train_data['price'])
The features we are considering are ‘bedrooms’, ‘grade’, ‘sqft_living’ and ‘sqft_above’. These are considered are one feature namely features1. Now the data is fitted into the model and test_data of features1 are used for prediction. Mean squared error is calculated for y_test. The mean squared error is rounded of upto 2 decimals. R squared error for both training and test is calculated. The intercept of the line is calculated along with coefficient of individual feature.
features1=['bedrooms','grade','sqft_living','sqft_above']reg=linear_model.LinearRegression()reg.fit(train_data[features1],train_data['price'])pred=reg.predict(test_data[features1])print('complex_model 1')mean_squared_error=metrics.mean_squared_error(y_test,pred)print('mean squared error(MSE)', round(np.sqrt(mean_squared_error),2))print('R squared training',round(reg.score(train_data[features1],train_data['price']),3))print('R squared training', round(reg.score(test_data[features1],test_data['price']),3))print('Intercept: ', reg.intercept_)print('Coefficient:', reg.coef_)
R-squared testing: 0.555
The features we are considering are ‘bedrooms’, ’bathrooms’, ’sqft_living’, ’sqft_lot’, ’floors’, ’waterfront’, ’view’, ’grade’, ’sqft_above’, ’sqft_basement’, ’lat’,’sqft_living15'. These are considered are one feature namely features2. Now the data is fitted into the model and test_data of features2 are used for prediction. Mean squared error is calculated for y_test. The mean squared error is rounded of upto 2 decimals. R squared error for both training and test is calculated. The intercept of the line is calculated along with coefficient of individual feature.
features2 = ['bedrooms','bathrooms','sqft_living','sqft_lot','floors','waterfront','view','grade','sqft_above','sqft_basement','lat','sqft_living15']reg= linear_model.LinearRegression()reg.fit(train_data[features1],train_data['price'])pred = reg.predict(test_data[features1])print('Complex Model_2')mean_squared_error = metrics.mean_squared_error(y_test, pred)print('Mean Squared Error (MSE) ', round(np.sqrt(mean_squared_error), 2))print('R-squared (training) ', round(reg.score(train_data[features1], train_data['price']), 3))print('R-squared (testing) ', round(reg.score(test_data[features1], test_data['price']), 3))print('Intercept: ', reg.intercept_)print('Coefficient:', reg.coef_)
R-squared testing: 0.672
Polynomial Regression is a form of linear regression in which the relationship between the independent variable x and dependent variable y is modeled as an nth degree polynomial. Polynomial regression fits a nonlinear relationship between the value of x and the corresponding conditional mean of y, denoted E(y |x).
For degree=2, the linear modelis built. The mean squared error is calculated and r squared is found for training and testing.
polyfeat=PolynomialFeatures(degree=2)xtrain_poly=polyfeat.fit_transform(train_data[features1])xtest_poly=polyfeat.fit_transform(test_data[features1])poly=linear_model.LinearRegression()poly.fit(xtrain_poly,train_data['price'])polypred=poly.predict(xtest_poly)print('Complex Model_3')mean_squared_error = metrics.mean_squared_error(test_data['price'], polypred)print('Mean Squared Error (MSE) ', round(np.sqrt(mean_squared_error), 2))print('R-squared (training) ', round(poly.score(xtrain_poly, train_data['price']), 3))print('R-squared (testing) ', round(poly.score(xtest_poly, test_data['price']), 3))
R-squared testing: 0.759
For degree=3, the linear model is built. The mean squared error is calculated and r squared is found for training and testing.
polyfeat=PolynomialFeatures(degree=3)xtrain_poly=polyfeat.fit_transform(train_data[features1])xtest_poly=polyfeat.fit_transform(test_data[features1])poly=linear_model.LinearRegression()poly.fit(xtrain_poly,train_data['price'])polypred=poly.predict(xtest_poly)print('complex model_4')mean_squared_error=metrics.mean_squared_error(test_data['price'],polypred)print('Mean Squared Error (MSE) ', round(np.sqrt(mean_squared_error), 2))print('R-squared (training) ', round(poly.score(xtrain_poly, train_data['price']), 3))print('R-squared (testing) ', round(poly.score(xtest_poly, test_data['price']), 3))
R-squared testing: 0.664
Complex Model_3 gives us R-squared (testing) score of 0.759. From above reports, we can conclude that Polynomial regression for degree=2, is best solution.
For notebook, refer here. I would be pleased to receive feedback or questions on any of the above.