Floating point images in torchvision

username: fdan
cooked: <p>Hi folks,</p>
<p>TorchVision noob question.</p>
<p>I’m doing some research using PyTorch in animation and visual effects, where the standard image format is floating point exr files (lossless, high dynaimc range).  From some cursory reading, it seems that TorchVision utilises PIL, which I believe is limited to 8bit int images?</p>
<p>I’m using another library for handling of exr files (OpenImageIO), from which I can load a floating point image, perform some operations such as resize etc, dump into a NumPy array, and from there initialise a torch Tensor.</p>
<p>My question is, is this equivalent functionality to TorchVision?  i.e. in my use case can I just load image data into tensors without using TorchVision?</p>
<p>cheers</p>

username: eqy
cooked: <p><code>torchvision</code> really only uses 8-bit RGB as an intermediate format, as images are usually converted to floating point before being passed to a model with something like <a href="https://pytorch.org/vision/stable/_modules/torchvision/transforms/functional.html#to_tensor" rel="noopener nofollow ugc"><code>to_tensor()</code></a>.</p>
<p>You can also see a typical example of how transformations are applied before being passed to a model in the ImageNet example:</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/examples/blob/cbb760d5e50a03df667cdc32a61f75ac28e11cbf/imagenet/main.py#L207" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/examples/blob/cbb760d5e50a03df667cdc32a61f75ac28e11cbf/imagenet/main.py#L207" target="_blank" rel="noopener nofollow ugc">pytorch/examples/blob/cbb760d5e50a03df667cdc32a61f75ac28e11cbf/imagenet/main.py#L207</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="197" style="counter-reset: li-counter 196 ;">
<li>        print("=&gt; no checkpoint found at '{}'".format(args.resume))</li>
<li>
</li>
<li>cudnn.benchmark = True</li>
<li>
</li>
<li># Data loading code</li>
<li>traindir = os.path.join(args.data, 'train')</li>
<li>valdir = os.path.join(args.data, 'val')</li>
<li>normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],</li>
<li>                                 std=[0.229, 0.224, 0.225])</li>
<li>
</li>
<li class="selected">train_dataset = datasets.ImageFolder(</li>
<li>    traindir,</li>
<li>    transforms.Compose([</li>
<li>        transforms.RandomResizedCrop(224),</li>
<li>        transforms.RandomHorizontalFlip(),</li>
<li>        transforms.ToTensor(),</li>
<li>        normalize,</li>
<li>    ]))</li>
<li>
</li>
<li>if args.distributed:</li>
<li>    train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Loading a floating point image should not be an issue. If for some reason you need to use a transform that only works on PIL Images, you can use the <a href="https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.functional.to_pil_image" rel="noopener nofollow ugc">to_pil_image</a> function.</p>