Exploring Machine Learning Algorithms — Simple Linear Regression
By: Eshita Goel
Tags: Machine Learning, Data Science, Linear Regression, Data Visualization

Step by Step Linear Regression

A good way to practice linear regression for the first time is to take a dataset having 2 variables, an independent variable and a dependent variable, and test the algorithm.

I will be taking a dataset consisting of 2 columns — “Hours” and “Scores”. The “Hours” depict the number of hours studied by a student for a test and the “Scores” depicts the percentage scored by the student on that test. We can imagine that the more a student studies, the higher scores he/she is likely to get. Of course, there can be other factors that we should be considering, but let's focus on just these two variables for now.

We start off with importing our libraries. This is very important as it helps us perform the various tasks further on.

The pandas library allows us to work with dataframes

The numpy library allows us to work with arrays

matplotlib.pyplot will help us in plotting graphs to visualize our data.

seaborn is a library made on top of matplotlib that also allows us in data visulaization.

2. Now import your data. The data I worked with can be found on this link It was provided to me when I was working on this mini-project as part of a Data Science internship with The Sparks Foundation. I will use pandas to import my data into a dataframe.

3. Explore your data: Before you actually use linear regression to make predictions, you should see if your data actually has a linear relationship or not. We can use many visualization techniques to check this. My favourite is using seaborn’s regplot that not only plots our data, but also shows a regression line. This makes it easy to see if our data is has a linear relationship or not.

We can see an obvious positive linear relationship between the Hours and the Scores. As the Hours are increasing, so are the scores and the increase is linear.

Such a regplot is good if you have less number of independent variables. But it's quite inconvenient to plot this for all variables if you have many of them. In that case, I recommend a Heat Map. Plotting the correlation between each independent variable with the dependent variable using a heatmap gives us information about the whole data in practically just 1 line of code.

The correlation between Hours and Scores is very high, Linear Regression to make predictions seems like a good idea.

4. Making the Model: Now that we know we can use linear regression for this data, we make our model.

Firstly, we store our values for independent and dependent variables in X and Y, respectively.

Now we make use of the sklearn library and using model_selection we import train_test_split. This will allow us to split our data into 2 parts: a training part and a testing part. This is useful as we can train our data on one part and use the other part to test our accuracy. If we were to use all the data for training, we would have no way of knowing if any subsequent predictions made by us are accurate or not.

Now we make the model. From sklearn.linear_model, we import LinearRegression. We then create ‘lm’ which will be used for Linear regression.

We fit and train our data and then store the predictions for the testing data in yhat. The r2_score is a metric that allows us to check the accuracy of our data. Here we can see that our data has a r2 score of about 0.9454, which is a very good accuracy score keeping the kind of data we have. We will explore what r2 score is in the further paragraphs.

If we recall, Linear Regression was about finding a straight line that best fits our data points. Our model does just that. Using the coef_ and intercept_ commands we can access the coefficient of the X values and the intercept for our line.

Let us plot the line using these values of coefficient and intercept to see if it fits our data or not.

Our model has given us a well-fitted line. This line is what is being used for the prediction of our data. For each value of x that we provide, the model calculates the corresponding value of y on this line and that is our predicted value.

Evaluating our Model

We will be using a metric called the R-Squared Value to check the accuracy of our predictions and how well our model works.

The R-Squared value is a statistical measure of how close our data is to the predicted line of best fit. A value close to 1 tells us that our model works well and a value close to 0 means that our dependent variable is not at all dependent on our other variables.

The R-squared value can also be negative. This happens when our prediction is actually worse than just using the mean value. What this means is — using the regression algorithm, our predictions are more inaccurate than just using the mean value for the independent variable.

To calculate the R-squared value, we use the following mathematical formula:

Here, we are calculating the residual error (SSres) as the sum of the squares of the errors for each prediction and the total sum of the squared errors (SStot) as the sum of the squares of the distance between our data points and the mean.

The advantage of splitting our data into training and testing datasets is that now wee can easily compare our predicted values with the already available values of Scores in our test data. We create a datafrane to compare our values:

The predicted values are close to our actual values, we can visualise this using a bar graph where we plot the actual values and the predicted values simultaneously.

We can see from the bar graph that using Linear Regression, the prediction made by us are very close to the actual values!