Understanding Regression Models
By: Ipsita Shee
Tags: Data Science, Regression, Data Visualization, Python, Machine Learning

Using SKlearn for creating linear, ridge, lasso regression models.

Photo by Markus Winkler on Unsplash

What is a linear regression?

Linear regression comes from the equation for a line, y=mx+c, where y is the dependent variable which changes according to x, which is the independent variable. C is the y intercept and m is the slope. In ML terms coefficient of x is the weight and y intercept or c is the bias. “m” and “c” are the variables that our machine will try to learn.

In this article, we will see how we can use scikit-learn to perform different types of regression like linear, ridge, lasso, etc. In order to start let us first import the required libraries.

Importing required libraries

We will start by importing pandas to load the dataset that we require, we will use matplotlib and seaborn for visualization, and similarly, we will import regression models from scikit-learn as and when required.

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

Loading Required Dataset

The dataset we are going to use here is the Boston dataset which can be easily downloaded from Kaggle. This dataset contains the median value of house prices along with all the features on which the price depends. Let us load the dataset and see what are the different columns it has.

df = pd.read_csv("C:/Users/DELL/boston.csv")

df.head()

Boston Dataset(Source: By Author)

Performing Data Analysis

In order to prepare the dataset for machine learning, we need to perform some basic data analysis on our dataset. We will start by dropping any unnecessary column like “Unnamed:0” as we can see in the image above.

df.drop('Unnamed: 0', axis=1,inplace=True)

Now let us see some statistical properties of our dataset using the describe function of data frames.

df.describe()

Statistical Properties(Source: By Author)

Now let us visualize the relationship of our target variable that is “MEDV” with all other feature variables. For this, we will use matplotlib and seaborn.

plt.figure(figsize=(15,30))

cnt=1

for i in df.columns[:-1]:

plt.subplot(10,3,cnt)

sns.regplot(x=df[i],y=df['medv'], color = 'green')

cnt=cnt+1

plt.tight_layout()

plt.show()

Regression Plot Visualization(Source: By Author)

After analyzing all this we can start with the modeling process. For this we need to divide our dataset into testing and training datasets also we need to import the models from sklearn.

from sklearn.model_selection import train_test_split

#Splitting the dataset

X_train, X_test, y_train, y_test = train_test_split(df[['crim','zn','indus','chas', 'nox','rm','age','dis','rad','tax','ptratio','black','lstat']], df['medv'], test_size=0.3, random_state=109) #Importing Regression Models

from sklearn.linear_model import LinearRegression, Ridge, Lasso

Linear Regression Model

We will start by creating a linear regression model and fit the data into it. To know how accurate our model is we will find the model score for testing data.

linreg = LinearRegression()

linreg.fit(X_train, y_train)

print("Coefficients: ", linreg.coef_)

print("Intercept: ", linreg.intercept_) #y intercept

Model Coefficients(Source: By Author)

Now let us check the accuracy of the model using the model score.

print("Accuracy score for training Data", linreg.score(X_train, y_train))

print("Accuracy score for testing Data", linreg.score(X_test, y_test))

Accuracy( Source: By Author)

Similarly, we can use the Ridge and Lasso models also for regression.

Lasso and Ridge Regression

In linear regression, if we have a large number of predictor variables and these predictor variables have a high correlation with each other then we can say that our model encounters a large variance and makes the model unreliable.

In order to overcome this problem, we use regularization techniques. Ridge regression is a technique which aims at decreasing the size of the coefficient using l2 penalty(sum of squared coefficients) while Lasso regression uses l1 penalty(absolute value of coefficients).

Creating Lasso Regression Model

lassoreg = Lasso()

lassoreg.fit(X_train, y_train)

print("Accuracy score for training data", lassoreg.score(X_train,y_train))

print("Accuracy score for test Data", lassoreg.score(X_test,y_test))

Accuracy(Source: By Author)

Creating Ridge Regression Model

ridgereg = Ridge()

ridgereg.fit(X_train, y_train)

print("Accuracy score for training data", ridgereg.score(X_train,y_train))

print("Accuracy score for test Data", ridgereg.score(X_test,y_test))

Accuracy(Source: By Author)

In this article, we saw how we can create different regression models and check their accuracy using a model score. Go ahead try this and in case you find any difficulty please let me know in the response section.

Before You Go

Thanks for reading! If you want to get in touch with me, feel free to reach me on ipsitashee4@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials.