ML | Implementing L1 and L2 regularization using Sklearn

22 Nov, 2021
Prerequisites: L2 and L1 regularizationThis article aims to implement the L2 and L1 regularization for Linear regression using the Ridge and Lasso modules of the Sklearn library of Python. Dataset – House prices dataset.Step 1: Importing the required libraries 
Python3
import pandas as pdimport numpy as npimport matplotlib.pyplot as pltfrom sklearn.linear_model import LinearRegression, Ridge, Lassofrom sklearn.model_selection import train_test_split, cross_val_scorefrom statistics import mean
Step 2: Loading and cleaning the Data 
Python3
# Changing the working location to the location of the datacd C:\Users\Dev\Desktop\Kaggle\House Prices # Loading the data into a Pandas DataFramedata = pd.read_csv('kc_house_data.csv') # Dropping the numerically non-sensical variablesdropColumns = ['id', 'date', 'zipcode']data = data.drop(dropColumns, axis = 1) # Separating the dependent and independent variablesy = data['price']X = data.drop('price', axis = 1) # Dividing the data into training and testing setX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25)
Step 3: Building and evaluating the different modelsa) Linear Regression: 
Python3
# Building and fitting the Linear Regression modellinearModel = LinearRegression()linearModel.fit(X_train, y_train) # Evaluating the Linear Regression modelprint(linearModel.score(X_test, y_test))
b) Ridge(L2) Regression: 
Python3
# List to maintain the different cross-validation scorescross_val_scores_ridge = [] # List to maintain the different values of alphaalpha = [] # Loop to compute the different values of cross-validation scoresfor i in range(1, 9):    ridgeModel = Ridge(alpha = i * 0.25)    ridgeModel.fit(X_train, y_train)    scores = cross_val_score(ridgeModel, X, y, cv = 10)    avg_cross_val_score = mean(scores)*100    cross_val_scores_ridge.append(avg_cross_val_score)    alpha.append(i * 0.25) # Loop to print the different values of cross-validation scoresfor i in range(0, len(alpha)):    print(str(alpha[i])+' : '+str(cross_val_scores_ridge[i]))
From the above output, we can conclude that the best value of alpha for the data is 2. 
Python3
# Building and fitting the Ridge Regression modelridgeModelChosen = Ridge(alpha = 2)ridgeModelChosen.fit(X_train, y_train) # Evaluating the Ridge Regression modelprint(ridgeModelChosen.score(X_test, y_test))
c) Lasso(L1) Regression: 
Python3
# List to maintain the cross-validation scorescross_val_scores_lasso = [] # List to maintain the different values of LambdaLambda = [] # Loop to compute the cross-validation scoresfor i in range(1, 9):    lassoModel = Lasso(alpha = i * 0.25, tol = 0.0925)    lassoModel.fit(X_train, y_train)    scores = cross_val_score(lassoModel, X, y, cv = 10)    avg_cross_val_score = mean(scores)*100    cross_val_scores_lasso.append(avg_cross_val_score)    Lambda.append(i * 0.25) # Loop to print the different values of cross-validation scoresfor i in range(0, len(alpha)):    print(str(alpha[i])+' : '+str(cross_val_scores_lasso[i]))
From the above output, we can conclude that the best value of lambda is 2. 
Python3
# Building and fitting the Lasso Regression ModellassoModelChosen = Lasso(alpha = 2, tol = 0.0925)lassoModelChosen.fit(X_train, y_train) # Evaluating the Lasso Regression modelprint(lassoModelChosen.score(X_test, y_test))
Step 4: Comparing and Visualizing the results 
Python3
# Building the two lists for visualizationmodels = ['Linear Regression', 'Ridge Regression', 'Lasso Regression']scores = [linearModel.score(X_test, y_test),         ridgeModelChosen.score(X_test, y_test),         lassoModelChosen.score(X_test, y_test)] # Building the dictionary to compare the scoresmapping = {}mapping['Linear Regression'] = linearModel.score(X_test, y_test)mapping['Ridge Regression'] = ridgeModelChosen.score(X_test, y_test)mapping['Lasso Regression'] = lassoModelChosen.score(X_test, y_test) # Printing the scores for different modelsfor key, val in mapping.items():    print(str(key)+' : '+str(val))
Python3
# Plotting the scoresplt.bar(models, scores)plt.xlabel('Regression Models')plt.ylabel('Score')plt.show()
shubham_singh
clintra
surinderdawra388
Machine Learning
Python
Machine Learning
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
ML | Monte Carlo Tree Search (MCTS)
Introduction to Recurrent Neural Network
Markov Decision Process
Support Vector Machine Algorithm
DBSCAN Clustering in ML | Density based clustering
Read JSON file using Python
Python map() function
Adding new column to existing DataFrame in Pandas
How to get column names in Pandas dataframe