How to avoid image normalization by torchvision.transforms.ToTensor

Ali_Aghababaei: <p>Hi everyone,<br>
I have created a custom dataset for some medical images and wanted to use an ImageNet-based pre-trained model like vgg16 for feature extraction. Then, I will design a custom fully-connected network and use Sklearn StratifiedKfold cross-validation to train and test the model.<br>
Here is the code for the dataset:</p>
<pre><code class="lang-auto"> def __init__ (self, transform, kfold_index, img_path = '/content/drive/MyDrive/projection images', 
                img_data = pd.read_csv ('/content/drive/MyDrive/projection images/images.csv'), 
                img_files = os.listdir ('/content/drive/MyDrive/projection images /projection images)):
    super ().__init__ ()
    self.transform = transform
    self.img_path = img_path
    self.img_label = [img_data.iloc [kfold_index [i], 7] for i in range (len (kfold_index))]
    self.img_files = [img_files [kfold_index [i]] for i in range (len (kfold_index))]
    
  def __len__ (self):
    return len (self.img_files)
  def __getitem__(self, index):
    sample_path = os.path.join (self.img_path, self.img_files [index])
    img = Image.open (sample_path)
    img = self.transform (img)
    
    return img, torch.tensor (self.img_label [index])
</code></pre>
<p>Also, the torchvision transformations for train and test datasets are as follows:</p>
<pre><code class="lang-auto">test_transform = transforms.Compose ([transforms.Resize (256),
                                      transforms.CenterCrop (224),
                                      transforms.Grayscale (num_output_channels= 3)])
                                      transforms.ToTensor ()])
train_transform = transforms.Compose ([transforms.RandomResizedCrop (256, scale = (0.8, 1)),
                                      transforms.ColorJitter (),
                                      transforms.CenterCrop (224),
                                      transforms.RandomAffine (15),
                                      transforms.RandomHorizontalFlip (),
                                      transforms.Grayscale (num_output_channels= 3)])
                                      transforms.ToTensor ()])
</code></pre>
<p>However, in previous works of my colleagues on the same dataset, the images were not normalized/standardized in the classic ways, i.e., all the images were just divided by a number (claculated based on the range of image values), not using min-max normalization or not standardizing by making the mean to zero and std to 1. In those studies, the models worked very well, and thus, I want to use the same approach in my work.<br>
The problem is that when using transforms.ToTensor (), Pytorch automatically normalizes the images. My question is that what is the best way to prevent such normalization?<br>
should I change transforms.ToTensor with transforms.PILToTensor ()?<br>
when I do that, this error is raised:</p>
<pre><code class="lang-auto">RuntimeError: Input type (torch.cuda.ByteTensor) and weight type (torch.cuda.FloatTensor) should be the same
</code></pre>
<p>or should I delete transforms.ToTensor from the transforms.Compose and instead, in the “return” part of the custom dataset, I change :</p>
<pre><code class="lang-auto">return img, torch.tensor (self.img_label [index])
</code></pre>
<p>with<br>
1)</p>
<pre><code class="lang-auto">return torch.tensor  (img), torch.tensor (self.img_label [index])```

</code></pre>
<p>or<br>
2)</p>
<pre><code class="lang-auto">return torch.Tensor (img), torch.tensor (self.img_label [index])
</code></pre>
<p>Actually, other errors were again raised when I did these changes:</p>
<ol>
<li><code>Could not infer dtype of Image</code></li>
<li><code>new(): data must be a sequence (got Image)</code></li>
</ol>
<p>I would be grateful if you could tell me what is the best approach in my case.</p>

eqy: <p>The simplest thing to do is probably either write your own <code>ToTensor</code> that calls a different function (see the function that is currently used here: <a href="https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#to_tensor" class="inline-onebox" rel="noopener nofollow ugc">torchvision.transforms.functional — Torchvision main documentation</a>) or to add a transformation after <code>ToTensor</code> that effectively undoes the normalization (e.g., by multiplying by a range and adding the mean back) as you should know the normalization constants that were used.</p>

Ali_Aghababaei: <p>Thanks a lot for your response.<br>
Actually, despite reading the source code link you provided, I cannot understand how to_tensor function changes the range [0, 255] to [0, 1]. Would you please explain it a bit more?<br>
Also, as I do not know how ToTensor normalizes the image, I cannot undo it. For instance, in min-max normalization, I should know the maximum and minimum of every image value to be able to unnormalize it (regarding your second recommendation).</p>

eqy: <p>I believe <code>ToTensor</code> assumes an 8-bit value with a maximum possible of 255 for each channel, so it simply devices by this maximum rather than explicitly computing the maximum in your dataset. You can see this behavior in the <code>.div</code> calls in the linked code.</p>
<p>You can also try converting to tensors directly without normalization if you can remove it from the <code>Compose</code>, which I realize would probably be simpler than modifying those functions.  e.g., with something like</p>
<pre><code class="lang-auto">    # handle PIL Image
    img = torch.as_tensor(np.array(pic, copy=True))
    img = img.view(pic.size[1], pic.size[0], len(pic.getbands()))
    # put it from HWC to CHW format
    img = img.permute((2, 0, 1))
    return img
</code></pre>

Ali_Aghababaei: <p>I really appreciate your explanation.<br>
In your code example, I think something like <code>dtype = torch.float32</code> should be added in the line <code>img = torch.as_tensor(np.array(pic, copy=True))</code>. Is that okay?<br>
I also wonder what the use of<br>
<code>  img = img.view(pic.size[1], pic.size[0], len(pic.getbands()))</code> is, since in my transformations I have used transforms.Grayscale (num_ouput_channels = 3) which will produce images with the shape (224, 224, 3).</p>