How to scale attributes with normalization and standardization
Tags: machinelearning, datascience, python
Published: 2020-03-20 21:06:10

<!-- wp:paragraph -->
<p>Attributes in different scales are common in Machine Learning projects. For example, a medical record dataset can include in the columns weight, height, and blood pressure. These attributes have different units of measure and vary in different intervals, making their comparison difficult.</p>
<!-- /wp:paragraph -->

<!-- wp:more -->
<!--more-->
<!-- /wp:more -->

<!-- wp:paragraph -->
<p>In these cases, we can apply a process called&nbsp;<strong>scaling</strong>&nbsp;to make this comparison easier. In this process, we change the original data but keeping the relative distance between the data points, so that we preserve the attribute distribution.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2><strong>Normalization</strong></h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In normalization, we scale an attribute by making all data points fit in the interval between 0.0 and 1.0. We express the normalization process using the formula:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>{% katex %} X^{norm} = \frac{X - X_{min}}{x_{max} - x_{min}} 
{% endkatex %}</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>where:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul>
<li>{%katex%}X^{norm}{%endkatex%}: is the new scaled attribute</li>
<li>{%katex%}X_{min}{%endkatex%}: is a column vector where all elements are equal to {%katex%}x_{min}{%endkatex%}</li>
<li>{%katex%}x_{min}{%endkatex%}: is the minimum value of the $X$ attribute</li>
<li>{%katex%}x_{max}{%endkatex%}: is the maximum value of the $X$ attribute&nbsp;</li>
</ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>For each attribute value, we subtract its minimum and then divide the result by the difference of its maximum and its minimum. The mean value and standard deviation will be scaled as well, but the transforming will keep the data distribution.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Standardization</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>In standardization, the attribute is transformed to have a mean equals 0 (zero), and the standard deviation equals 1 (one). The following formula is applied:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>{% katex %}X^{std} = \frac{X - \mu}{\sigma}{% endkatex %}</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Where:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>$X^{std}$: is the new scaled attribute</li><li>$X$: is a column vector representing our attribute</li><li>$\mu$: is a column vector where all elements are the mean value of the attribute</li><li>$\sigma$: is the standard deviation of the attribute</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>In standardization, there are no lower and upper limits for the new data values. But all of them are now expressed as unitarian distances from the mean.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Example</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>In the project&nbsp;<a target="_blank" href="https://www.kaggle.com/rodolfomendes/scalling-attributes-with-normalization-and-standar" rel="noreferrer noopener">Scaling attributes with normalization and standardization</a>, we use vectorized operations to apply normalization and standardization to scale the attributes of the House Data Pricing dataset.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>For example, we used the following code to apply normalization to the&nbsp;<em>LotFrontage</em>&nbsp;attribute:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">df_norm['LotFrontage'] = (df_float['LotFrontage'] - df_float['LotFrontage'].min()) / (df_float['LotFrontage'].max() - df_float['LotFrontage'].min())</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>We also applied standardization to the same attributes and saved the result in a separated&nbsp;<em>DataFrame</em>:</p>
<!-- /wp:paragraph -->

<!-- wp:syntaxhighlighter/code {"language":"python"} -->
<pre class="wp-block-syntaxhighlighter-code">df_std['LotFrontage'] = (df_float['LotFrontage'] - df_float['LotFrontage'].mean()) / df_float['LotFrontage'].std()</pre>
<!-- /wp:syntaxhighlighter/code -->

<!-- wp:paragraph -->
<p>After scaling the attributes, we create linear regression models for each dataset and compared the results.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":437,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://launchyourintelligentapphome.files.wordpress.com/2020/03/norm-std-house-price.png?w=1024" alt="" class="wp-image-437"/><figcaption>Real data vs linear regression models</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2>Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>After comparing the score of the models, we concluded that the scaling of attributes by themselves does not improve the linear regression models. Check the complete example in the link below:&nbsp;&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":1} -->
<h1><a target="_blank" href="https://www.kaggle.com/rodolfomendes/scalling-attributes-with-normalization-and-standar" rel="noreferrer noopener">Scaling attributes with normalization and standardization</a></h1>
<!-- /wp:heading -->