Torchvision.transforms on RGB image in customized Dataset

banikr: <p>Following the example <a href="https://github.com/pytorch/examples/blob/91f230a13b95c6259e5cb22b6cef355de998cede/imagenet/main.py#:~:text=normalize%20%3D%20transforms.Normalize,normalize%2C" rel="noopener nofollow ugc">here</a>, trying to implement <code>transforms</code> in customized Dataset.</p>
<pre><code class="lang-auto">from torchvision import transforms as T
normalize = T.Normalize(mean=[0.485, 0.456, 0.406],
                         std=[0.229, 0.224, 0.225]) # from ImageNet

t = T.Compose([
               T.RandomResizedCrop(224),
               T.RandomHorizontalFlip(),
               T.ToTensor(),
               normalize
              ])
</code></pre>
<p>Since my RGB image size is huge I am tiling the images to before applying <code>transforms</code> in <code>Custom_dataset</code>.</p>
<pre><code class="lang-auto">def __getitem__(self, index):                                  
    im = Image.open(self.imList[index]).convert('RGB')         
    mk = Image.open(self.mkList[index]).convert('L')           # binary mask
    im = np.asarray(im)                                        
    mk = np.asarray(mk)                                        
    im, mk = patchify(im, mk, [224, 224], [100, 100]) 
    i_t_s = torch.empty([64, 3, 224, 224])     # size of im                
    m_t_s = torch.empty([64, 224, 224])     # size of mk                      
    for l in range(0, len(im)):  # for 64 small image patches
        PIL_i = Image.fromarray(im[l, ...]).convert('RGB')     
        PIL_m = Image.fromarray(mk[l, ...]).convert('L')       
        i_t = self.tform(PIL_i) # [3, 224, 224]                
        m_t = T.ToTensor()(PIL_m)                              
        i_t_s[l, ...] = i_t                  
        m_t_s[l, ...] = m_t                                    
    m_t_o = torch.ones(2, 64, 224, 224)                                  
    m_t_o[1, ...] = m_t_s == 1.                                
    m_t_o[0, ...] = m_t_o[0, ...] - m_t_o[1, ...]              
    m_t_o = torch.transpose(m_t_o, 1, 0) #, 2, 3)              
    return i_t_s, m_t_o # size --&gt; [64, 3, 224, 224], [64, 2, 224, 224] 
</code></pre>
<p><strong>Quesion</strong>:</p>
<ol>
<li>How do I apply the same transformation in both image and paired mask patch?</li>
<li>
<code>T.Normalize</code> must be on Tensor and after <code>T.ToTensor()</code> it changes image values from [0, 1] to some normalized value beyond 0 and 1. What is the way to normalize and then <code>T.ToTensor()</code> to keep the value within the range [0, 1]</li>
</ol>
<p>N.B. <code>patchify</code> creates paired patches of RGB and binary mask.</p>

eqy: <ol>
<li>
<p>You can always apply the same transformation if you specify the parameter and use the functional versions of transformations: <a href="https://pytorch.org/vision/stable/_modules/torchvision/transforms/functional.html" rel="noopener nofollow ugc">torchvision.transforms.functional — Torchvision 0.10.0 documentation (pytorch.org)</a>.</p>
</li>
<li>
<p>ToTensor should already normalize to [0, 1]. <a href="https://pytorch.org/vision/stable/_modules/torchvision/transforms/transforms.html#ToTensor" rel="noopener nofollow ugc">torchvision.transforms.transforms — Torchvision 0.10.0 documentation (pytorch.org)</a></p>
</li>
</ol>

banikr: <p>Thanks for replying.<br>
The second ques was to clarify the dilemma. <code>Normalize</code> can not be implemented on a non-Tensor but applying it after <code>ToTensor()</code> changes the value of [0, 1] to [2.6 to -2.11] or something like that.<br>
For example, the one used [here](<a href="https://pytorch.org/vision/stable/_modules/torchvision/transforms/transforms.html#:~:text=%3E%3E%3E%20transform%20%3D%20transforms.Compose(%5B%0A%20%20%20%20%20%20%20%20%3E%3E%3E%20%20%20transforms.RandomHorizontalFlip()%2C%0A%20%20%20%20%20%20%20%20%3E%3E%3E%20%20%20transforms.ToTensor()%2C%0A%20%20%20%20%20%20%20%20%3E%3E%3E%20%20%20transforms.Normalize((0.485%2C%200.456%2C%200.406)%2C%20(0.229%2C%200.224%2C%200.225)" class="inline-onebox" rel="noopener nofollow ugc">torchvision.transforms.transforms — Torchvision 0.16 documentation</a>)%2C)</p>
<pre><code class="lang-auto"> transform = transforms.Compose([
         transforms.RandomHorizontalFlip(),
         transforms.ToTensor(),
         transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)))
</code></pre>
<p>If I use like that and visualize the patch fed to the network as input looks like:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/4/e4d59d5aa52e84d3b17e7df9c4b9f6d0e6ef75ee.jpeg" alt="image" data-base62-sha1="wEmwTvHYeUdu1dj36qSxUZg6RL8" width="204" height="199"></p>
<p>Without normalization:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/b/ab84d921229af5c3f80658a27583874125b29614.jpeg" alt="image" data-base62-sha1="otkgWE1732dVZ4ungr5wY7VSgAs" width="207" height="205"></p>
<p>Visualizing the inputs with the following codes:</p>
<pre><code class="lang-auto">    unloader = T.ToPILImage()
    def tensor_to_PIL(tensor):
        image = tensor.cpu().clone()
        image = image.squeeze(0)
        image = unloader(image)
        return image
    #
    # print(x.shape, y.shape, torch.unique(y), torch.max(x), torch.min(x), torch.unique(y[:, 0, ...]))
    Pimage = tensor_to_PIL(x[32, ...]) # x size --&gt; 64, 224, 224, 3 
    plt.imshow(Pimage)
</code></pre>