How to Implement Linear Regression in Scikit-learn
By: The Educative Team
Tags: Python, Data Science, Machine Learning, Scikit Learn, Programming

Standard

If your data instead follows standard deviation, you can use the StandardScaler instead. This scaler fits a passed data set to be a standard scale along with the standard deviation:

import sklearn.preprocessing as preprocessing



std = preprocessing.StandardScaler()

# X is a matrix

std.fit(X)

X_std = std.transform(X)

As above, we first create the scaler on line three, fit the current matrix on line five, and finally transform the original matrix on line six.

Let’s see how this scales our same example from above:

import sklearn.preprocessing as preprocessing import numpy as np X = np.random.randint(2, 10, size=(4, 2)) X2 = np.random.randint(100, 10000, size=(4, 2)) X = np.concatenate((X, X2), axis=1) print("The original matrix") print(X)

std = preprocessing.StandardScaler() std.fit(X) X_std = std.transform(X) print("The transform data using Standard scaler") print(X_std)

Scaling method comparison

Scikit-learn Linear Regression: Implement an Algorithm

Now we’ll implement the linear regression machine learning algorithm using the Boston housing price sample data. As with all ML algorithms, we’ll start with importing our dataset and then train our algorithm using historical data.

Linear regression is a predictive model often used by real businesses. Linear regression seeks to predict the relationship between a scalar response and related explanatory variables to output value with realistic meaning like product sales or housing prices. This model is best used when you have a log of previous, consistent data and want to predict what will happen next if the pattern continues.

From a mathematical point of view, linear regression is about fitting data to minimize the sum of residuals between each data point and the predicted value. In other words, we are minimizing the discrepancy between the data and the estimation model.

As shown in the figure below, the red line is the model we solved, the blue point is the original data, and the distance between the point and the red line is the residual. Our goal is to minimize the sum of residuals.

Sum of residuals, data vs prediction

How to implement linear regression

import sklearn.datasets as datasets from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression import sklearn.metrics as metrics house = datasets.load_boston() print("The data shape of house is {}".format(house.data.shape)) print("The number of feature in this data set is {}".format( house.data.shape[1])) train_x, test_x, train_y, test_y = train_test_split(house.data, house.target, test_size=0.2, random_state=42) print("The first five samples {}".format(train_x[:5])) print("The first five targets {}".format(train_y[:5])) print("The number of samples in train set is {}".format(train_x.shape[0])) print("The number of samples in test set is {}".format(test_x.shape[0])) lr = LinearRegression() lr.fit(train_x, train_y)

pred_y = lr.predict(test_x) print("The first five prediction {}".format(pred_y[:5])) print("The real first five labels {}".format(test_y[:5]))

mse = metrics.mean_squared_error(test_y, pred_y) print("Mean Squared Error {}".format(mse))