Advanced statistical Methods-Linear Regression
Tags: Python, Data Science, Linear Regression, Statistical Analysis, Machine Learning

The easiest regression model is Simple linear Regression model .

simple linear regression

Let’s check out Geometrical Representation of simple linear regression ,

The point that are scattered are called observed value (x)

b 0 is the intercept of the Regression Line with the y axis

b 1 is the slope of the Regression line it shows how much y changes for each unit change of x

The distance between the observed value and regression line is the estimator for the error term (e^i) is called Residual.

Now, if you draw a perpendicular from an observed point to the Regression line and the intercept between that perpendicular and the regression line is a point with a y value equal to y hat as shown in the above pic.

It’s time for coding , first let’s discus the packages we are going to use.

If you are using Anaconda, the easiest way to check which you already have the above packages is the following

open Anaconda prompt write conda list and hit enter, you should get a list of all packages installed in the current environment

If you are missing some packages, then quietly write

(base) C:\Users\conda install (name of the package)

or

(base) C:\Users\pip install (name of the package)

Let’s do a quick overview of why we need the packages,

Numpy: A third-party package allowing us to work with multidimensional arrays. They represent a powerful way to organize and process data. Numpy is a fundamental library for those who attempt to manipulate large chunks of information.

Pandas: It is a package that enhances numpy even further, it allows us to organize data in a tabular form and to attach the descriptive label to the row and column of the table.

Matplotlib: It is a 2D plotting library specially designed for visualization of numpy computation.

Scipy: Numpy, pandas, and matplotlib are part of a larger scipy group of libraries. Scipy is a python ecosystem containing a lot of tools for scientific calculations suitable for the fields of mathematics, machine learning engineering, etc.

Seaborn: It is a Python visualization library based on matplotlib. It provides a high-label interface for drawing attractive statistical graphics.

Statsmodels: It is yet another package built on top of numpy and scipy, which also integrate with pandas. statsmodels is a python module that provides classes and functions for the estimation of many different statistical models.

Sci-kit-learn: It is a simple and efficient tool for predictive data analysis. sci-kit-learn, is designed to interoperate with the Python numerical and scientific libraries Numpy and Scipy.

Let’s do some more code for simple linear Regression ,

+-

we need to import all relevant libraries for simple linear regression. I have all my data in my CVS file called simple linear regression.csv, Let’s loaded in a new variable called data using pandas method as Pd.read-csv(‘file path’). After running this line of code the data from the csv will be loaded in the data variable, as I am using pandas the data variable will be automatically converted into a data frame.

let’s explore the problem, so we have a sample of eighty-four students who have studied in college. we will create a linear regression that predicts the GPA of a student based on their SAT score.

Remember the equation of simple linear regression that we discussed earlier

y^ = b0 + b1x1, here our dependent variable is GPA, so I’ll create a variable called Y, which contains GPA. Similarly, our independent variable was, SAT loaded in a variable x1. It is always useful to plot your data to understand it better and see if there is a relationship to be found. Use some conventional matplotlib code each point on the graph represent a different student. observing all data points we can see that there is a strong relationship between SAT and GPA.

Next, create a regression, let’s declare new variable x and create another variable named output which, contains the output of the ordinary least squares regression ( OLS) as an argument we must add the dependent variable y, and at the end, we need to define a fit() method that will apply specific techniques (OLS in this case) to obtain the fit of the model.

Let’s plot the regression line on the same scatter plot.

That’s the best fitting line or the line which is closest to all observation simultaneously. So that’s how you create a simple linear regression using python.

I hope it was useful, see you next time.