Introduction to Linear Regression With One Variable
Tags: machinelearning, datascience
Published: 2020-02-25 13:34:23

<!-- wp:paragraph -->
<p>In this post, we present Linear Regression analysis using one variable example. The idea is to keep the examples as simple and straightforward as possible, so you can focus on the intuition behind Linear Regression and don't get confused with too much data preparation or other details about tools and data manipulation. There are many algorithms to perform regression analysis, but Linear Regression is the simplest of them and the recommended algorithm to start with.</p>
<!-- /wp:paragraph -->

<!-- wp:more -->
<!--more-->
<!-- /wp:more -->

<!-- wp:paragraph -->
<p>In Linear Regression we predict an output value <strong>y</strong> using a linear model. In other words, from an input variable <strong>x</strong>  we use a simple equation of the form:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code -->
<pre class="wp-block-syntaxhighlighter-code">y = ax + b</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p> to predict the value of <strong>y</strong>. Because we already chose the type of the model (the linear model), the task of the algorithm is to find the parameters <strong>a</strong> and <strong>b</strong> to define the linear equation which best fits our data. In the example below, we use the data from the House Price dataset from Kaggle and Python tools to build 3 Linear Regression models to predict the sale price of a house (output variable). Each model uses a different attribute as an input variable, and we use the scikit-learn library to build our linear models. Finally, we plot our models together with the real data so we can visualize how well the linear model fits the data. </p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Building a Linear Regression Model with Python</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The first step we load the dataset into a Pandas dataframe, select only the continous variables, and then we print information about our dataset:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">url = 'https://raw.githubusercontent.com/rodmsmendes/' +
     'reinforcementlearning4fun/master/' + 
     'data/house_prices.csv'
df = pd.read_csv(url)
df_float = df.select_dtypes(include=['float64']).copy()
df_float.info()</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>So the resulting dataset has 3 continous attributes, and we find that all of them has missing values:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code -->
<pre class="wp-block-syntaxhighlighter-code">&lt;class 'pandas.core.frame.DataFrame'>
RangeIndex: 1460 entries, 0 to 1459
Data columns (total 3 columns):
LotFrontage    1201 non-null float64
MasVnrArea     1452 non-null float64
GarageYrBlt    1379 non-null float64
dtypes: float64(3)
memory usage: 34.3 KB</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>We complete then using the mean value of the respective attributes:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">df_float['LotFrontage'] = df['LotFrontage']
    .fillna(df['LotFrontage'].mean(), inplace=False)

df_float['MasVnrArea'] = df['MasVnrArea']
    .fillna(df['MasVnrArea'].mean(), inplace=False)

df_float['GarageYrBlt'] = df['GarageYrBlt']
    .fillna(df['GarageYrBlt'].mean(), inplace=False)</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>Once the the missing values are filled we can build our models.  For each model, we create an instance of the <em>LinearRegression</em> class from scikit-learn. Then we use one of the selected continous variables as input and the output variable <em>SalePrice</em> to train the model:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">lotFrontage = df_float[['LotFrontage']]
salePrice = df['SalePrice']

lr1 = LinearRegression()
lr1.fit(lotFrontage, salePrice)</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>What we did here was to use the method <em>fit()</em> to train a linear regression model over the input data <em>lotFrontage</em> and given the output <em>salePrice</em>. The state of the model is stored in the <em>LinearRegression</em> model references by the <em>lr1</em> variable. Using this variable, we print the model coefficients as well as the <em>mean squared error</em> score to asssess the quality of the model:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">print(lr1.coef_)
print(lr1.intercept_)
print(mean_squared_error(salePrice,
    lr1.predict(lotFrontage)))</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>Finally, we plot the resulting models (in orange) together with the real data (blue points), so we can compare then.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"wide","id":368,"sizeSlug":"large"} -->
<figure class="wp-block-image alignwide size-large"><img src="https://launchyourintelligentapphome.files.wordpress.com/2020/02/one-variable-linear-regression-plot.png?w=1024" alt="" class="wp-image-368"/><figcaption>Linear models plot together with real data</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2>Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In this example, we used the Linear Regression as a exploratory data analysis tool. Usign the scikit-learn libray we create Linear Regression models to understand if the target attribute&nbsp;<em>SalePrice</em>&nbsp;can be explained in terms of&nbsp;<em>LotFrontage</em>,&nbsp;<em>MasVnrArea</em>&nbsp;or&nbsp;<em>GarageYrBlt</em>&nbsp;individually. Finally, the results of modeling are presented  ploting them together with the data points.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can find this complete example as a Kaggle kernel. In this kernel, you will find all the steps needed to create a Linear Regression model using scikit-learn and other Python tools. I strongly suggest that you open and execute this notebook to see the code in action. Then, fork the kernel to create your copy and try to modify the code, using other variables, another dataset, creating new visualizations, and so on. You can access the kernel by clicking the link below:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><a class="_e75a791d-denali-editor-page-rtfLink" rel="noopener" href="https://www.kaggle.com/rodolfomendes/linear-regression-with-one-variable" target="_blank"><strong>Linear Regression Regression With One Variable</strong></a></p>
<!-- /wp:paragraph -->