Input data normalization

username: Chame_call
cooked: <p>When is it best to use normalization:</p>
<pre><code class="lang-auto"># consist positive numbers
normalized_data = (data / data.max()) * 2 - 1
</code></pre>
<p>instead of standardization:</p>
<pre><code class="lang-auto">nomalized_data = (data - data.mean()) / sqrt(data.var())
</code></pre>

username: Chame_call
cooked: <p>What we see in <a href="https://sebastianraschka.com/Articles/2014_about_feature_scaling.html#about-min-max-scaling" rel="noopener nofollow ugc">here</a>:</p>
<blockquote>
<p>An alternative approach to Z-score normalization (or standardization) is the so-called  Min-Max scaling (often also simply called “normalization” - a common cause for ambiguities).<br>
In this approach, the data is scaled to a fixed range - usually 0 to 1.<br>
The cost of having this bounded range - in contrast to standardization - is that <strong>we will end up with smaller standard deviations, which can suppress the effect of outliers</strong>.</p>
</blockquote>
<p>Can someone explain the selected part?</p>

username: paganpasta
cooked: <p>I think the code you have mentioned doesn’t correspond to min-max.<br>
Z-score is fine. However, <code>min_max_norm = (data - data.min())/(data.max() - data.min())</code></p>
<p>For normalisation, the values are squashed in [0, 1]. If you have an outlier say data.max() the transformed values will be very small for min_max_norm(max in denominator) for the majority of samples. Thereby affecting the statistics of your transformed distribution.</p>

username: anantguptadbl
cooked: <p>This is a classic fight. But you have to look at the pre and post data to get an idea</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/5/958437070919c517030f721d0b189cfe20ee7514.png" alt="image" data-base62-sha1="lkGpyYawfsC7M2d00v6CVjgIghS" width="595" height="364"><br>
The normalization and standardization affects this array in the following manner</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/6/769340efda1612e9c534581b9beafdea0f6fb1cd.png" alt="image" data-base62-sha1="gUXPwskIbmfiPOEfe7WQuvKyDYV" width="590" height="344"></p>
<p>We can see the Standard Array represents the original array better.</p>
<p>Inference</p>
<ul>
<li>
<p>If you have clusters of data points in your dataset, it is better to use Standard Scaling. Normal Scaling will not create a lot of variance between the points to close to each other, as can be seen in the images</p>
</li>
<li>
<p>If you have anomalies in your data, it is better to use Standard Scaling as Normal Scaling will skew the data</p>
</li>
</ul>

username: Chame_call
cooked: <blockquote>
<p>Normal Scaling will not create a lot of variance between the points to close to each other, as can be seen in the images</p>
</blockquote>
<p>Can you explain this moment in more details if that possible pls?</p>

username: Chame_call
cooked: <blockquote>
<p>I think the code you have mentioned doesn’t correspond to min-max.</p>
</blockquote>
<p>If normalization squashes data into [-1; -1] then is my norm code version possible to exist?</p>

username: paganpasta
cooked: <p>The range will be [-1, 1], however, it is not referred to min-max normalisation. To keep the range [-1, 1], typically min-max normalisation is employed.</p>

username: Chame_call
cooked: <p>But which parameter(property) will be differ in such cases: my version of squashing into [-1;1] and min-max normalization into the same range?</p>

username: paganpasta
cooked: <p>You can compare the mean of the scaled values. I am computing the mean as it’s easy to compute.<br>
<code>u_your = (2*u_old - x_max)/(x_max)</code><br>
whereas, <code>u_minmax = (u_old - x_min)/(x_max - x_min)</code><br>
you can see that the new means are going to be different after applying the scaling. Correspondingly you can also compute the variance to check if they differ.</p>

username: Chame_call
cooked: <p>Maybe you can additionally explain the first statement</p>

username: Juliusz_Ziomek
cooked: <p>In standarization, technically, there is no limit as to what value can the extremes of output have. In case of normalization, however, the most extreme values will be equal to 1 and -1.<br>
This will suppress the effect of outliers. Why?<br>
For example lets consider a situation when in dataset feature A has a very large variance and feature B has very small (i.e. we expect to find a value of A far from mean quite often, but finding a value of B far from mean is something very unusual). This is a vital information, as finding an example with a very extreme value of B might suggest that this example is an anomaly/ positive/ negative or whatever you want to detect, and finding an extreme value of A is not a very outstanding thing.<br>
After standarization, the extreme values of B will be much further from the zero than values of A, whereas after normalization, in both cases the most extreme values will be equal to 1 and -1. Therefore, the information that an extreme value of B is much more important than an extreme value of A is lost, because in the end, values of these extremes will be equal.</p>

username: Chame_call
cooked: <p>Thanks.<br>
You’ve given a good example.</p>