Effect of normalization on activation

username: sparshgarg23
cooked: <p>I know that normalization tends to center the data making the learning process more stable.<br>
But what is it’s effect on the intermediate activation/feature maps.<br>
for example below is an image before  normalization along with it’s histogram<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/3/e38940bf15c58f9a0c48d32605e4fa30167d9839.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e38940bf15c58f9a0c48d32605e4fa30167d9839" title="berlin_1a"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e38940bf15c58f9a0c48d32605e4fa30167d9839_2_690x345.jpeg" alt="berlin_1a" data-base62-sha1="wsSrPdr05Nc4MtrHjLHQWYyZu2d" width="690" height="345" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e38940bf15c58f9a0c48d32605e4fa30167d9839_2_690x345.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e38940bf15c58f9a0c48d32605e4fa30167d9839_2_1035x517.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e38940bf15c58f9a0c48d32605e4fa30167d9839_2_1380x690.jpeg 2x" data-dominant-color="3E4D4A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">berlin_1a</span><span class="informations">1920×960 99 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
histogram of image<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/0/f0b60c12056e17617d6526414193643940639164.png" alt="berlin_hist" data-base62-sha1="ylqCSG6Wynfs6TncqjisWLsch8M" width="393" height="248"><br>
Same image after histogram equalization is done along with it’s histogram<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67fbf3c932cf95ca5b946477a9c9484f25e6e946.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/67fbf3c932cf95ca5b946477a9c9484f25e6e946" title="berlin_histogram"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67fbf3c932cf95ca5b946477a9c9484f25e6e946_2_690x345.jpeg" alt="berlin_histogram" data-base62-sha1="ePSZ3mpvulPm0icSQzp1bzMLk6q" width="690" height="345" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67fbf3c932cf95ca5b946477a9c9484f25e6e946_2_690x345.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67fbf3c932cf95ca5b946477a9c9484f25e6e946_2_1035x517.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67fbf3c932cf95ca5b946477a9c9484f25e6e946_2_1380x690.jpeg 2x" data-dominant-color="679289"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">berlin_histogram</span><span class="informations">1920×960 192 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
image histogram after equalization is done<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/3/43b738e30faeddef59b4fba839f6530767a6db67.png" alt="berlin_hist_after" data-base62-sha1="9F2xCCn75LTmrjgdKrVVdkavyB1" width="393" height="248"><br>
clearly we can see that after the equalization is done,the image looks more dynamic,where you can clearly see the buildings.</p>
<p>so i am thinking that if normalization wasn’t done,then different parts in the image would end up firing differently,thereby making it difficult for it to learn features and discriminate between them. am i right in thinking like this?</p>