Next(iter(dataloader)) error with positional arguments

JohnPolo: <p>As a solution to this <a href="https://discuss.pytorch.org/t/transforms-resize-misidentified-as-attribute/124276/4">question I posed</a>, I changed an import statement for transforms from</p>
<p><code>import transforms as T</code></p>
<p>to</p>
<p><code>from torchvision import transforms as T</code></p>
<p>I did this in order to fix this:</p>
<pre><code class="lang-auto">def get_transform(train):
    transforms = []
    # converts the image, a PIL image, into a PyTorch Tensor
    transforms.append(T.Resize((400*5312/2988,400)))
    transforms.append(T.ToTensor())
    if train:
        # during training, randomly flip the training images
        # and ground-truth for data augmentation
        transforms.append(T.RandomHorizontalFlip(0.5))
    return T.Compose(transforms)
</code></pre>
<p>The linked Q and A can supply more detail.</p>
<p>Now, with the torchvision import, I get a new error:</p>
<blockquote>
<p>TypeError: <strong>call</strong>() takes 2 positional arguments but 3 were given</p>
</blockquote>
<p>I dug around and there were <a href="https://discuss.pytorch.org/t/call-takes-2-positional-arguments-but-3-were-given/112782/5">similar questions</a>. This new error has to do with <code>transforms.Compose</code> only able to take 2 positional arguments. A <a href="https://discuss.pytorch.org/t/t-compose-typeerror-call-takes-2-positional-arguments-but-3-were-given/62529/6">suggested solution</a> was to write a custom Compose statement. I added that to my code. Now the transform section has this:</p>
<pre><code class="lang-auto">from torchvision import transforms as T

class MyCompose(object):
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, img, tar):
        for t in self.transforms:
            img, tar = t(img, tar)
        return img, tar

def get_transform(train):
    transforms = []
    # converts the image, a PIL image, into a PyTorch Tensor
    transforms.append(T.Resize((400*5312/2988,400)))
    transforms.append(T.ToTensor())
    if train:
        # during training, randomly flip the training images
        # and ground-truth for data augmentation
        transforms.append(T.RandomHorizontalFlip(0.5))
    return MyCompose(transforms)
</code></pre>
<p>Now, I get this error:</p>
<blockquote>
<hr>
<p>TypeError                                 Traceback (most recent call last)</p>
<p> in ()<br>
5  collate_fn=utils.collate_fn)<br>
6 # For Training<br>
----&gt; 7 images,targets = next(iter(data_loader))<br>
8 images = list(image for image in images)<br>
9 targets = [{k: v for k, v in t.items()} for t in targets]</p>
<p>3 frames</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py in <strong>next</strong>(self)<br>
515             if self._sampler_iter is None:<br>
516                 self._reset()<br>
 → 517             data = self._next_data()<br>
518             self._num_yielded += 1<br>
519             if self._dataset_kind == _DatasetKind.Iterable and \</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py in _next_data(self)<br>
1197             else:<br>
1198                 del self._task_info[idx]<br>
 → 1199                 return self._process_data(data)<br>
1200<br>
1201     def _try_put_index(self):</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py in _process_data(self, data)<br>
1223         self._try_put_index()<br>
1224         if isinstance(data, ExceptionWrapper):<br>
 → 1225             data.reraise()<br>
1226         return data<br>
1227</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/_utils.py in reraise(self)<br>
427             # have message field<br>
428             raise self.exc_type(message=msg)<br>
 → 429         raise self.exc_type(msg)<br>
430<br>
431</p>
<p>TypeError: Caught TypeError in DataLoader worker process 0.<br>
Original Traceback (most recent call last):<br>
File “/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py”, line 202, in _worker_loop<br>
data = fetcher.fetch(index)<br>
File “/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py”, line 44, in fetch<br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py”, line 44, in <br>
data = [self.dataset[idx] for idx in possibly_batched_index]<br>
File “”, line 75, in <strong>getitem</strong><br>
img, target = self.transforms(img, target)<br>
File “”, line 11, in <strong>call</strong><br>
img, tar = t(img, tar)<br>
File “/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
TypeError: forward() takes 2 positional arguments but 3 were given</p>
</blockquote>
<p>I don’t know where <code>forward()</code> is called exactly, but it appears to be part of the <code>next(iter(dataset))</code> execution. More importantly, I don’t know what the positional arguments are. I mean, I expect them to be whatever part of the dataset are outputs in the objects images and targets. I don’t know what in the MyCompose would have changed to add another positional argument. How should I fix this?</p>

ptrblck: <p>The new error is raised by the internally used <code>torchvision.transforms</code>, which only accept a single input.<br>
The linked post shows a custom <code>Compose</code> implementation, which accepts two inputs. However, note that internally a custom <code>transform_ToNumpy</code> transformation was used, which also accepts two inputs.<br>
In your case you could thus apply the <code>torchvision.transforms</code> on both inputs separately.</p>

JohnPolo: <p>Thanks for the reply, ptrblck.</p>
<p>I did try something like</p>
<pre><code class="lang-auto">def segmentation_transform(image, tar, train=False):
    image_cc = TF.center_crop(image, 1600)
    tar_cc = TF.center_crop(tar, 1600)
    image_T = TF.to_tensor(image_cc)
    tar_T = TF.to_tensor(tar_cc)
    if train:
        image_T = TF.hflip(image_T)
        tar_T = TF.hflip(tar_T)
    return image_T,tar_T
</code></pre>
<p>I’m not sure if that is what you meant. I got similar positional argument error with that when I run</p>
<pre><code class="lang-auto">dataset = four_chs(root = '/content/drive/MyDrive/data', transforms = get_transform(train=True)) 
data_loader = torch.utils.data.DataLoader(
 dataset, batch_size=1, shuffle=True, num_workers=2,
 collate_fn=utils.collate_fn)

# For Training
images,targets = next(iter(data_loader))
</code></pre>
<p>Unfortunately, it is still not clear to me how I can modify either the <code>transforms</code> function, so that I can use a <code>T.Resize()</code> with a size argument or a <code>T.CenterCrop()</code> with a size argument inside of the function and avoid getting the positional argument error (or maybe both transforms inside the function), or write a new class of <code>Compose</code> that can execute the <code>transforms</code> with the Dataset I create and read in with the Dataloader.</p>