Not clear about the sequence of transforms.Normalize and transforms.ToTensor

username: banikr
cooked: <p>Working with RGB image and binary mask as a target, I am confused about transformations.</p>
<p>Is it necessary to rescale the image and target between [0, 1] before feeding to the network? If so, is there any preference between <code>transforms.ToTensor</code> or <code>F.to_tensor</code>?</p>
<p>Is it also necessary to normalize the RGB images? If yes, I have the following working:</p>
<pre><code class="lang-auto">img_transform = transforms.Compose([
        transforms.ToPILImage(),
        transforms.RandomVerticalFlip(),
        transforms.RandomHorizontalFlip(),
        transforms.RandomCrop(size=(patch_size, patch_size), pad_if_needed=True), #these need to be in a reproducible order, first affine transforms and then color
        transforms.RandomResizedCrop(size=patch_size),
        transforms.RandomRotation(180),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406],
                         std=[0.229, 0.224, 0.225]),
    ])
</code></pre>
<p>But after applying the transformation the data from the loader is not between [0, 1] rather some other value like [+2.6, -2.11].<br>
Do we rescale them again between [0, 1]? Then how to do that as <code>transforms.ToTensor</code> accepts only tensors.</p>
<p>So the output from <code>DataLoader</code> with Normalization:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/9/695252dc38985cb35bd43889db747829b8717b36.jpeg" alt="image" data-base62-sha1="f1IvKkuF21v9GCpzYoPcKXyWceO" width="186" height="185"><br>
removing the Normalization:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/e/deca4c4a486422ad78f459b5f6e9e65ea06fdae1.jpeg" alt="image" data-base62-sha1="vMTq2bCL8a5lF2uPaKrKYJoPZzH" width="186" height="185"></p>

username: ptrblck
cooked: <p>You don’t <em>need</em> to normalize the inputs, but it usually help training the model.<br>
<code>torchvision.transforms.ToTensor()</code> and <code>torchvision.transforms.functional.to_tensor()</code> are applying the same transformation via an object or the functional API (<code>ToTensor</code> calls internally into <code>to_tensor</code> as seen <a href="https://github.com/pytorch/vision/blob/25bc21dfcbb390f2f215e8f83aa5e028c77a0f24/torchvision/transforms/transforms.py#L97">here</a>).</p>
<p><code>Normalize</code> subtracts the mean and divides by the stddev to create an output with zero mean and unit variance, so the observed values would be expected.</p>

username: banikr
cooked: <p>Thanks for replying, so if we use <code>Normalize</code> are we working with data range beyond [0, 1]? And that’s completely ok for a network to function properly?</p>

username: ptrblck
cooked: <p>Yes, the range could be outside of <code>[0, 1]</code> and would ideally have a zero mean and unit variance.<br>
Also yes, that’s beneficial for the network training. ML literature calls this normalization often “whitening” and you should be able to find more explanations e.g. in Bishop’s Pattern Recognition and Machine Learning or other references.</p>

username: Roman_Velichkin
cooked: <p>Could you explain, how would this affect on fine-tuning pretrained models that were trained with [-1, 1] and [0, 1] normalization?</p>

username: ptrblck
cooked: <p>I would assume you might need to fine tune for more epochs if you change the data preprocessing pipeline and pass inputs in another range. Note that this is speculative as I didn’t run a lot of experiments to verify the claim.</p>

username: Roman_Velichkin
cooked: <p>Thank you for an answer! Guess I have to run experiments then</p>