Simple Linear Regression in Python | by Tarun Gupta | Towards Data Science

Simple Linear Regression is a statistical method to find relationship between two continuous variables. Out of the two variables present, one is independent variable and the other is dependent variable. Statistical relationship is not accurate in determining relationship between two variables. For example, relationship between height and weight.
This relationship is defined by the famous line equation which you would have learned in high school.
y = b0 + b1*x
where,
y is the dependent variable
x is the independent variable
b0 is the base value of the relationship
b1 is the slope of the line explaining the relationship between y & x.
For instance, y refers to how does a person’s salary change with the years of experience that he has. So in that case salary would be the dependent variable and experience will be the independent variable and the base value will the salary of a person who has no experience. In our code example, we are going to work on such a dataset.
In order to determine what is the best fitting line, let’s have a look at the following curve.
Point with red cross are actual or real data points, whereas points with green cross are predicted value of the regression model. The goal here is to make such a regression model that the sum of squared error is minimum.
Importing the dataset and performing Data Preprocessing
Diving into the dataset of our code today. We have a business problem, where a company wants to establish a relationship between salary of its employees and the experience they have. We are going to investigate that correlation using Simple Linear Regression.
Simple Linear Regression will tell us what is the best fitting line for this relationship. Let’s have a look at the dataset. You can download the dataset here.
Our data consists of total 30 objects and two attributes, YearsExperience and Salary. The attribute YearsExperience is the independent attribute and the attribute Salary is the dependent attribute.
I am going to use Spyder for writing this Machine Learning model. Before diving into Regression part of the model, we need to perform Data Preprocessing on our dataset. I am going to write the code for that directly but if you want to learn more about them, visit the following articles.
The conceptual Data Preprocessing article:
towardsdatascience.com
The Programming Data Preprocessing article:
towardsdatascience.com
# Importing the librariesimport numpy as npimport matplotlib.pyplot as pltimport pandas as pd# Importing the datasetdataset = pd.read_csv('SimpleLinearRegression.csv')X = dataset.iloc[:, :-1].valuesy = dataset.iloc[:, -1].values# Splitting the dataset into the Training set and Test setfrom sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 1/3, random_state = 0)
After executing this code, we have our training and testing sets split along with arrays and vectors of independent and dependent attributes respectively. They should look like this:
Fit the dataset into Simple Linear Regression model
In order to fit the data into our regressor model, we need to import a library, call the method for Regression and fit our training data to that regression model. We do this as follows:
# Fitting Simple Linear Regression to the Training setfrom sklearn.linear_model import LinearRegressionregressor = LinearRegression()regressor.fit(X_train, y_train)
Since, our Machine Learning model already knows the correlation of our training set, we will now predict the values of our testing set and then later compare them with the actual values of the test set.
To predict the values of the test set, we need to write the following code:
# Predicting the Test set resultsy_pred = regressor.predict(X_test)
After execution of code until now, we can manually have a look at the predicted test set values and actual test set values and see the how do they differ.
As you can see, the values differ with each other. Some have a very low difference like the 4th entry and some have a slightly higher difference like the 8th entry.
Visualising the correlation in the dataset
In order to judge the actual correlation amongst the data, we are going to plot two curves.
Visualising the Training set results
We do this using the following code:
# Visualising the Training set resultsplt.scatter(X_train, y_train, color = 'red')plt.plot(X_train, regressor.predict(X_train), color = 'blue')plt.title('Salary vs Experience (Training set)')plt.xlabel('Years of Experience')plt.ylabel('Salary')plt.show()
As it can be seen, the line we have here is not that bad, since only a few points lie very far from it, most of the points lie around the line itself.
Visualising the Test set results
We do this using the following code:
# Visualising the Test set resultsplt.scatter(X_test, y_test, color = 'red')plt.plot(X_train, regressor.predict(X_train), color = 'blue')plt.title('Salary vs Experience (Test set)')plt.xlabel('Years of Experience')plt.ylabel('Salary')plt.show()
Here, there can be a confusion on why the parameter of regressor.predict() is not changed in plt.plot(X_train, regressor.predict(X_train), color = 'blue') to X_test. This is so because, if we do that, we will get a new line altogether but we want to compare our test set to the line we got from our training set. The curve will look like this:
Here, it can be seen that 5 out of 10 data points lie on the line itself. 3 out of 10 are very near to the line and only 2 out of 10 points are a little bit farther.
This concludes this post on Simple Linear Regression in Python for Machine Learning Modelling.
I am giving away a free eBook on Consistency. Get your free eBook here.
If you enjoy reading stories like these, then you should get my posts in your inbox and if want to support me as a writer, consider signing up to become a Medium member. It’s $5 a month, giving you unlimited access to stories on Medium. If you sign up using my link, I’ll earn a small commission at no extra cost to you.
tarun-gupta.medium.com
Thanks for reading. If you liked this, have a read at some of my other posts here: