Understanding Normalization and Getting better results from training

username: _Allen
cooked: <p>I’ve been working on building a facial detection NN through fine-tuning the pretrained Faster RCNN. I have been able to get boxes to populate in the general area of faces, but it is very hit or miss. I thought that a normalization transformation would give me better results but when I did the transformation my images come out like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b240d9905edec32dbc6ea189784bbc87fed35ddb.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b240d9905edec32dbc6ea189784bbc87fed35ddb" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b240d9905edec32dbc6ea189784bbc87fed35ddb_2_690x458.jpeg" alt="image" data-base62-sha1="pqTVECy6SO9zthWiixcElfQjkz9" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b240d9905edec32dbc6ea189784bbc87fed35ddb_2_690x458.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b240d9905edec32dbc6ea189784bbc87fed35ddb_2_1035x687.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/2/b240d9905edec32dbc6ea189784bbc87fed35ddb.jpeg 2x" data-dominant-color="8A8169"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1126×749 741 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here’s what I get when I do not do my own normalization:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f704555b8db5aa9a340bce39171750a1722fc8c.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/3f704555b8db5aa9a340bce39171750a1722fc8c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f704555b8db5aa9a340bce39171750a1722fc8c_2_690x462.jpeg" alt="image" data-base62-sha1="93cCqwbw5u13xpuoGPbEjZXQdBO" width="690" height="462" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3f704555b8db5aa9a340bce39171750a1722fc8c_2_690x462.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f704555b8db5aa9a340bce39171750a1722fc8c.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/f/3f704555b8db5aa9a340bce39171750a1722fc8c.jpeg 2x" data-dominant-color="493E33"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">778×521 111 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My thought is, maybe the normalization transform is being called twice? Once in the Faster RCNN source code and then once by me. The reason I think it is being called twice is because I received a tensor mismatch error, when I tried changing my images to gray scale when I loaded them in. This same exact error occurred both with and without my Normalization transform.</p>
<p>Here’s the error:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c3e580ddcb18313db60480898df5e8a7c17115b.png" data-download-href="https://discuss.pytorch.org/uploads/default/9c3e580ddcb18313db60480898df5e8a7c17115b" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c3e580ddcb18313db60480898df5e8a7c17115b.png" alt="image" data-base62-sha1="mic3vZXnO2B4bH4dLMXWgw2dCVZ" width="690" height="160" data-dominant-color="191919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1368×319 19.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here’s my transforms:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/6/46e0d333a04d506fc8d4e8109ea15bb9b3763556.png" alt="image" data-base62-sha1="a716IfFBd1klfEdtHI1aW0ROGyO" width="497" height="261"></p>

username: mailcorahul
cooked: <p>Firstly regarding the transform, <code>FasterRCNN</code> has a <code>GeneralizedRCNNTransform</code> module in the beginning which normalizes the input image.<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/detection/transform.py#L46" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/vision/blob/master/torchvision/models/detection/transform.py#L46" target="_blank" rel="nofollow noopener">pytorch/vision/blob/master/torchvision/models/detection/transform.py#L46</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="36" style="counter-reset: li-counter 35 ;">
<li>def forward(self, images, targets=None):</li>
<li>    # type: (List[Tensor], Optional[List[Dict[str, Tensor]]])</li>
<li>    images = [img for img in images]</li>
<li>    for i in range(len(images)):</li>
<li>        image = images[i]</li>
<li>        target_index = targets[i] if targets is not None else None</li>
<li>
</li>
<li>        if image.dim() != 3:</li>
<li>            raise ValueError("images is expected to be a list of 3d tensors "</li>
<li>                             "of shape [C, H, W], got {}".format(image.shape))</li>
<li class="selected">        image = self.normalize(image)</li>
<li>        image, target_index = self.resize(image, target_index)</li>
<li>        images[i] = image</li>
<li>        if targets is not None and target_index is not None:</li>
<li>            targets[i] = target_index</li>
<li>
</li>
<li>    image_sizes = [img.shape[-2:] for img in images]</li>
<li>    images = self.batch_images(images)</li>
<li>    image_sizes_list = torch.jit.annotate(List[Tuple[int, int]], [])</li>
<li>    for image_size in image_sizes:</li>
<li>        assert len(image_size) == 2</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>Do you feed 3-channel(RGB) images as input or do you convert them into grayscale? FasterRCNN expects a 3-channel input image.</p>

username: _Allen
cooked: <p>I feed it 3 channel images</p>