Torch vision ToPILImage()

Megh_Bhalerao: <p>Hi all,</p>
<p>I am trying to convert a real valued tensor to a PIL image to save on disk. When going through the documentation of <code>to_pil_image</code>, I have a question in the following line in the pytorch/vision repo -</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/vision/blob/87cde716b7f108f3db7b86047596ebfad1b88380/torchvision/transforms/functional.py#L288">
  <header class="source">

      <a href="https://github.com/pytorch/vision/blob/87cde716b7f108f3db7b86047596ebfad1b88380/torchvision/transforms/functional.py#L288" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/vision/blob/87cde716b7f108f3db7b86047596ebfad1b88380/torchvision/transforms/functional.py#L288" target="_blank" rel="noopener nofollow ugc">pytorch/vision/blob/87cde716b7f108f3db7b86047596ebfad1b88380/torchvision/transforms/functional.py#L288</a></h4>



    <pre class="onebox"><code class="lang-py">
      <ol class="start lines" start="278" style="counter-reset: li-counter 277 ;">
          <li>        # if 2D image, add channel dimension (HWC)</li>
          <li>        pic = np.expand_dims(pic, 2)</li>
          <li>
          </li>
<li>    # check number of channels</li>
          <li>    if pic.shape[-1] &gt; 4:</li>
          <li>        raise ValueError(f"pic should not have &gt; 4 channels. Got {pic.shape[-1]} channels.")</li>
          <li>
          </li>
<li>npimg = pic</li>
          <li>if isinstance(pic, torch.Tensor):</li>
          <li>    if pic.is_floating_point() and mode != "F":</li>
          <li class="selected">        pic = pic.mul(255).byte()</li>
          <li>    npimg = np.transpose(pic.cpu().numpy(), (1, 2, 0))</li>
          <li>
          </li>
<li>if not isinstance(npimg, np.ndarray):</li>
          <li>    raise TypeError("Input pic must be a torch.Tensor or NumPy ndarray, not {type(npimg)}")</li>
          <li>
          </li>
<li>if npimg.shape[2] == 1:</li>
          <li>    expected_mode = None</li>
          <li>    npimg = npimg[:, :, 0]</li>
          <li>    if npimg.dtype == np.uint8:</li>
          <li>        expected_mode = "L"</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<ol>
<li>I do not understand why the image is being multiplied by 255, I understand that the 255 represents the standard 8 bit pixel values, but it does not make sense to multiply the image by 255 <strong>unless the input tensor values are between [0,1]</strong>, since there will be overflows above the [0,255] range and in this case I believe PIL wraps the overflowed values like so -  <code>val%255</code>. I observed this issue in practice when I am saving a real valued tensor to PIL image using the <code>ToPILImage()</code> utility. Hence if the wrapping happens then, the visualization of images will not be a correct one.</li>
<li>The documentation of <code>TensorToPIL()</code> states that <strong>Converts a torch.Tensor of shape C x H x W or a numpy ndarray of shape H x W x C to a PIL Image while preserving the value range</strong>. - I am not sure what they mean by value range, since from the GitHub code it seems like the value range is not preserved due to some operations on the pixel values. Or am I missing something in my understanding of what preserving value ranges mean? I think the documentation - <a href="https://pytorch.org/vision/main/generated/torchvision.transforms.ToPILImage.html" class="inline-onebox" rel="noopener nofollow ugc">ToPILImage — Torchvision main documentation</a> - needs to be clearer regarding this utility.</li>
<li>Even the <code>save_image</code> utility in pytorch internally seems to be multiplying the image by 255 here - <a href="https://github.com/pytorch/vision/blob/87cde716b7f108f3db7b86047596ebfad1b88380/torchvision/utils.py#L158" class="inline-onebox" rel="noopener nofollow ugc">vision/utils.py at 87cde716b7f108f3db7b86047596ebfad1b88380 · pytorch/vision · GitHub</a> - again under the assumption that the input pixel values are between [0,1], which need not be true. I could not find this assumption stated anywhere explicitly. Please do let me know if I am missing something.<br>
Thanks for your help and please let me know if I am missing something.<br>
– Megh</li>
</ol>

ptrblck: <p>I think your explanation is correct and the default assumption is an input image in the range <code>[0, 1]</code> if you are passing a floating point tensor (e.g. the output of <code>ToTensor</code> using a <code>uint8</code> image) unless you want to use the <code>F</code> <code>PIL.Image.mode</code> which uses the <code>float32</code> format.</p>

Megh_Bhalerao: <p>Maybe this could be made clearer in the docs? I would be happy to help and open a PR if needed.<br>
Thanks again!</p>

ptrblck: <p>Sure, that sounds great! Feel free to create a GitHub issue first explaining the confusion and the lack of documentation and mention that you would be interested in updating for docs for more clarify as it could be beneficial for others, too! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Megh_Bhalerao: <p>For reference - this issue is taken to the pytorch/vision GitHub  - <a href="https://github.com/pytorch/vision/issues/6255#issuecomment-1180228170" class="inline-onebox" rel="noopener nofollow ugc">Issue in save_image utility in torch vision.utils · Issue #6255 · pytorch/vision · GitHub</a></p>