Creating a dataloader for multi-page tiff images

JVedant: <p>Hi, I am trying to create a custom dataset that loads multi-page tiff files (shape=(h, w, d, c)). Here the channel’s value is 4 and so when applying transforms.ToTensor() on the image, it shows an error of</p>
<pre><code class="lang-auto">ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "&lt;ipython-input-2-61874812cd51&gt;", line 31, in __getitem__
    img = self.img_transform(img)
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 97, in __call__
    return F.to_tensor(pic)
  File "/home/deadsec/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 105, in to_tensor
    raise ValueError('pic should be 2/3 dimensional. Got {} dimensions.'.format(pic.ndim))
ValueError: pic should be 2/3 dimensional. Got 4 dimensions.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-6-9b4568424e81&gt; in &lt;module&gt;
----&gt; 1 for each in train_loader:
      2     print(each["image"].shape)
      3     print(each["mask"].shape)
      4     break

~/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/dataloader.py in __next__(self)
    519             if self._sampler_iter is None:
    520                 self._reset()
--&gt; 521             data = self._next_data()
    522             self._num_yielded += 1
    523             if self._dataset_kind == _DatasetKind.Iterable and \

~/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/dataloader.py in _next_data(self)
   1201             else:
   1202                 del self._task_info[idx]
-&gt; 1203                 return self._process_data(data)
   1204 
   1205     def _try_put_index(self):

~/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/utils/data/dataloader.py in _process_data(self, data)
   1227         self._try_put_index()
   1228         if isinstance(data, ExceptionWrapper):
-&gt; 1229             data.reraise()
   1230         return data
   1231 

~/miniconda3/envs/torchsm_86/lib/python3.8/site-packages/torch/_utils.py in reraise(self)
    423             # have message field
    424             raise self.exc_type(message=msg)
--&gt; 425         raise self.exc_type(msg)
    426 
    427 
</code></pre>
<p>What can be done to solve/overcome this ?</p>

ptrblck: <p><code>ToTensor()</code> checks image arrays to have 2 or 3 dimensions to make sure they are either grayscale or color images. Based on your description you are using a volume (with an additional depth dimension) so depending on your use case you could transform the numpy array to a tensor manually via <code>x = torch.from_numpy(img)</code> and normalize it afterwards if needed.</p>