UCF101 Torchvision Dataset-to-DataLoader: Expected object of scalar type Float but got scalar type Short for sequence element 1

masoncusack: <p>Hi, when I try to iterate on a dataloader object constructed from a <a href="https://pytorch.org/docs/stable/torchvision/datasets.html#torchvision.datasets.UCF101" rel="nofollow noopener">UCF101 dataset object</a> (from torchvision) as follows:</p>
<pre><code class="lang-auto">import torch #TODO: only load in needed parts of the torch library for the module
from torchvision.datasets import UCF101 #Use pre-made torchvision dataloaders for UCF101

def ucf101_dataset(root='../data/UCFvideo', annotation_path='../data/UCFlabels', frames_per_clip=16, step_between_clips=8, **kwargs):

    ucf_dataset = UCF101(root, annotation_path, frames_per_clip, step_between_clips, **kwargs)
    return ucf_dataset
    
def ucf101_dataloader(dataset, batch_size=4, shuffle=True, num_workers=1):

    return DataLoader(dataset=dataset, batch_size=batch_size, shuffle=shuffle, num_workers=num_workers)

test_set = ucf101_dataloader(ucf101_dataset(train=False))

test_set = iter(test_set)
test_set.next()

</code></pre>
<p>I get the following error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-14-32a0d4e18597&gt; in &lt;module&gt;
      2 
      3 test_set = iter(test_set)
----&gt; 4 test_set.next()

~/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py in __next__(self)
    817             else:
    818                 del self._task_info[idx]
--&gt; 819                 return self._process_data(data)
    820 
    821     next = __next__  # Python 2 compatibility

~/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _process_data(self, data)
    844         self._try_put_index()
    845         if isinstance(data, ExceptionWrapper):
--&gt; 846             data.reraise()
    847         return data
    848 

~/.local/lib/python3.6/site-packages/torch/_utils.py in reraise(self)
    383             # (https://bugs.python.org/issue2651), so we work around it.
    384             msg = KeyErrorMessage(msg)
--&gt; 385         raise self.exc_type(msg)

RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 79, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 79, in &lt;listcomp&gt;
    return [default_collate(samples) for samples in transposed]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: Expected object of scalar type Float but got scalar type Short for sequence element 1 in sequence argument at position #1 'tensors'
</code></pre>
<p>Been working on this for a while now and really don’t understand what I’ve done wrong, especially as I haven’t done much.</p>
<p>Any help appreciated. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>I don’t have the dataset locally at the moment, but could you try to pass <code>transform=transforms.ToTensor()</code> to the dataset initialization?</p>

masoncusack: <p>Sure can. That gives us:</p>
<pre><code class="lang-auto">TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/datasets/ucf101.py", line 105, in __getitem__
    video = self.transform(video)
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 101, in __call__
    return F.to_tensor(pic)
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 54, in to_tensor
    raise TypeError('pic should be PIL Image or ndarray. Got {}'.format(type(pic)))
TypeError: pic should be PIL Image or ndarray. Got &lt;class 'torch.Tensor'&gt;
</code></pre>
<p>I notice the UCF101 dataset causes this warning to be thrown by <a href="https://github.com/pytorch/vision/blob/master/torchvision/io/video.py" rel="nofollow noopener">torchvision’s video.py</a>:</p>
<pre><code class="lang-auto">/home/user/.local/lib/python3.6/site-packages/torchvision/io/video.py:107: UserWarning: The pts_unit 'pts' gives wrong results and will be removed in a follow-up version. Please use pts_unit 'sec'.
  "follow-up version. Please use pts_unit 'sec'.")
</code></pre>
<p>Maybe this default could be causing an issue?</p>

masoncusack: <p>Found a solution of a kind. Setting shuffle to “False” in the DataLoader initialisation stops this error from occurring.</p>
<p>Will try digging a little deeper. If this stokes any ideas let me know. Think this should work now but interested in why exactly it doesn’t work if shuffle = True.</p>
<p>We now have:</p>
<pre><code class="lang-auto">def ucf101_dataloader(dataset, batch_size=4, shuffle=False, num_workers=1):

    return DataLoader(dataset=dataset, batch_size=batch_size, shuffle=shuffle, num_workers=num_workers)

test_set = ucf101_dataloader(ucf101_dataset(train=False))

test_set = iter(test_set)
print(test_set.next())
</code></pre>
<p>Producing lovely normal tensors:</p>
<pre><code class="lang-auto">[tensor([[[[[64, 65, 88],
            [64, 65, 88],
            [65, 66, 89],
            ...,
</code></pre>
<p>One thing I notice immediately is that elements of these tensors do not appear to be floats.</p>