T.Compose | TypeError: __call__() takes 2 positional arguments but 3 were given

username: _Allen
cooked: <p>I have been getting this odd error saying that I have passes too many arguments into my <strong>call</strong>() method. I am thoroughly perplexed because I am sure I have only passes 2 arguments.</p>
<p>Here is my Compose class:</p>
<pre><code class="lang-auto">class Compose(object):
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, image, target):
        for t in self.transforms:
            image, target = t(image, target)
        return image, target

</code></pre>
<p>Here’s the function that is call the method:</p>
<pre><code class="lang-auto">def get_transform(train):
    transforms = []
    transforms.append(T.ToTensor())
    if train:
        # during training, randomly flip the training images
        # and ground-truth for data augmentation
        transforms.append(T.RandomHorizontalFlip(0.5))
        transforms.append(T.Resize(INPUT_SIZE))
    return T.Compose(transforms)
</code></pre>
<p>Here’s the error msg:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/3/13774f45842478001bec30362798e8b760b94bbb.png" data-download-href="https://discuss.pytorch.org/uploads/default/13774f45842478001bec30362798e8b760b94bbb" title="error"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/3/13774f45842478001bec30362798e8b760b94bbb.png" alt="error" data-base62-sha1="2McGSS8cU4FNQGEfiUwYPQLbUZZ" width="690" height="106" data-dominant-color="1C1C1C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">error</span><span class="informations">752×116 3.93 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Any help would be appreciated! &lt;3</p>

username: ptrblck
cooked: <p>There are a few issues in the example code:</p>
<ul>
<li>Even though you are defining your own <code>Compose</code> class, it seems you are still using the <code>torchvision.transforms.Compose</code> one, so you might want to remove the <code>T</code> namespace.</li>
<li>Your custom <code>Compose</code> object takes two inputs. However, the underlying <code>torchvision.transforms</code> don’t, so you would have to call the transformation on the image and target separately.</li>
<li>Since you are using (one) random transformation, the image and target will not be transformed using the same random number, which might be wrong (e.g. for a segmentation use case). If you want to apply the same random transform on both inputs, have a look at <a href="https://discuss.pytorch.org/t/torchvision-transfors-how-to-perform-identical-transform-on-both-image-and-target/10606/7">this post</a>
</li>
<li>
<code>RandomHorizontalFlip</code> and <code>Resize</code> work on <code>PIL.Images</code>, so <code>ToTensor</code> should be applied last.</li>
</ul>
<p>Here is a small dummy example:</p>
<pre><code class="lang-python">class Compose(object):
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, image, target):
        for t in self.transforms:
            image = t(image)
            target = t(target)
        return image, target

import torchvision.transforms as transforms

transform = []
transform.append(transforms.RandomHorizontalFlip(0.5))
transform.append(transforms.Resize(10))
transform.append(transforms.ToTensor())
transform = Compose(transform)

to_image = transforms.ToPILImage()
x = to_image(torch.randn(3, 24, 24))
y = to_image(torch.randn(3, 24, 24))

transform(x, y)
</code></pre>

username: _Allen
cooked: <p>Thank you! This did get me a little further. However, when I try to pass my <em>target</em> variable to the transform Resize I run into a problem, because my <em>target</em> variable is a dictionary, not a PIL image, with bounding boxes, labels, iscrowd, etc.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/1/a17f5e238365c779518c0caf9d8a43f3206df236.png" data-download-href="https://discuss.pytorch.org/uploads/default/a17f5e238365c779518c0caf9d8a43f3206df236" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/1/a17f5e238365c779518c0caf9d8a43f3206df236.png" alt="image" data-base62-sha1="n2FKVj72ONtq5r731Hx9KH64RmK" width="690" height="193" data-dominant-color="292A29"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">865×242 16.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It looks like I will have to make a custom resize function for my target.<br>
Do you know of a way to resize BBox coordinates so they correspond to the resized image?</p>

username: _Allen
cooked: <p>I just realized that my dataset gives me the boxes as proportions of the image. All I have to do is apply the resize to the image then multiply the height and width of the transformed image by the given proportions of the bboxes.</p>

username: Manuel_Alejandro_Dia
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, I am having a similar problem but this time using the <code>transforms.Compose</code> class.</p>
<p>What I’ve been trying to do is to create a composition of transformations to apply directly on the <code>torchvision.dataset.Cityscapes</code> dataset object (here I’m using <code>target_type='semantic'</code> ). For simplicity’s sake, let’s say I want to transform the image as a Numpy object with a custom transform, my code for said transform goes like:</p>
<pre><code class="lang-auto">class transform_ToNumpy(object):
    # Converts PIL image to Numpy array.
    def __call__(self, image,semseg):
        if not isinstance(image, PIL.Image) or not isinstance(semseg, PIL.Image):
            raise (RuntimeError("segtransform.ToNumpy() only handle PIL.Image"
                                "[eg: data readed by cv2.imread()].\n"))
        else:
          image = np.array(image)
          semseg = np.array(semseg)
            
        return image, semseg
</code></pre>
<p>And I am using my <code>transforms.Compose</code> object in the dataset class as:</p>
<pre><code class="lang-auto">dataset = Cityscapes(dataset_path, split='train', mode='fine', target_type='semantic', transforms=transforms.Compose([transform_ToNumpy()]))
</code></pre>
<p>Notice that I am using the parameter <code>transforms</code> to specify the transform that will be applied to both source and target images.</p>
<p>But I am still getting the error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

&lt;ipython-input-5-feacc9e6f37e&gt; in &lt;module&gt;()
     26         ]))
     27 
---&gt; 28 img, target = dataset[0] 

/usr/local/lib/python3.6/dist-packages/torchvision/datasets/cityscapes.py in __getitem__(self, index)
    182         if self.transforms is not None:
--&gt; 183             image, target = self.transforms(image, target)
    184 
    185         return image, target

TypeError: __call__() takes 2 positional arguments but 3 were given
</code></pre>
<p>Which I don’t understand since my transform’s <code>__call__</code> method expects <strong>two</strong> inputs: <code>image</code> and <code>semseg</code> .</p>
<p>Could you give me some insight on why is it saying that:</p>
<pre><code class="lang-auto">__call__() takes 2 positional arguments but 3 were given
</code></pre>
<p>When I do not specify any other inputs than the source and semantic segmentation images?</p>
<p>Thank you.</p>

username: ptrblck
cooked: <p>I think the error is raised by <code>Compose</code>, which accepts only a single argument as seen <a href="https://github.com/pytorch/vision/blob/5405739efa803d09db6099771f0a2302faeb0cee/torchvision/transforms/transforms.py#L36-L63">here</a>.</p>
<p>Reproduction of your error:</p>
<pre><code class="lang-python">trans = transforms.Compose([transform_ToNumpy()])
a = TF.to_pil_image(torch.randn(3, 24, 24))
b = TF.to_pil_image(torch.randn(3, 24, 24))
x, y = trans(a, b)
</code></pre>
<p>A workaround would be to write a custom <code>Compose</code> class, which accepts and forwards two arguments:</p>
<pre><code class="lang-python">class MyCompose(object):
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, img, tar):
        for t in self.transforms:
            img, tar = t(img, tar)
        return img, tar


trans = MyCompose([transform_ToNumpy()])
a = TF.to_pil_image(torch.randn(3, 24, 24))
b = TF.to_pil_image(torch.randn(3, 24, 24))
x, y = trans(a, b)
</code></pre>

username: Manuel_Alejandro_Dia
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>!</p>
<p>I really don’t know how I was doing it before, but it was causing an error with my own Compose class (which was essentially the same that you posted).</p>
<p>But everything works now, this is the code:</p>
<pre><code class="lang-auto">train_transform = transform_Compose([
        transform_ToNumpy(),
        transform_id2Train(),
        transform_Resize((512,1024)),
        transform_RandomHorizontalFlip(),
        transform_ToTensor(),
        transform_Normalize(mean=mean, std=std),
        transform_increaseDim(),
        ])

dataset = Cityscapes(dataset_path, split='train', mode='fine', target_type='semantic', transforms=train_transform)

img, target = dataset[0] #PIL Images
</code></pre>
<p>Here the <code>train_transform</code> class takes two arguments.</p>

username: lin_pan
cooked: <p>great help~ thx ptrblck~</p>

username: Santhoshnumberone
cooked: <p>Following this tutorial</p>
<ol>
<li><a href="https://towardsdatascience.com/building-your-own-object-detector-pytorch-vs-tensorflow-and-how-to-even-get-started-1d314691d4ae#8863" rel="noopener nofollow ugc">Building your own object detector — PyTorch vs TensorFlow and how to even get started?</a></li>
<li>
<a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="noopener nofollow ugc">TORCHVISION OBJECT DETECTION FINETUNING TUTORIAL</a> which has a colab version <a href="https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/torchvision_finetuning_instance_segmentation.ipynb" rel="noopener nofollow ugc">here</a>
</li>
</ol>
<p>Cloning and copying the files<br>
According to 1st</p>
<pre><code class="lang-auto">%%bash
git clone https://github.com/pytorch/vision.git
cd vision
git checkout v0.3.0
cp references/detection/utils.py ../
cp references/detection/transforms.py ../
cp references/detection/coco_eval.py ../
cp references/detection/engine.py ../
cp references/detection/coco_utils.py ../

import numpy as np
import torch
import torch.utils.data
from PIL import Image
import pandas as pd
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
from engine import train_one_epoch, evaluate
import utils
import transforms as T

def get_transform(train):
   transforms = []
   # converts the image, a PIL image, into a PyTorch Tensor
   transforms.append(T.ToTensor())
   if train:
      # during training, randomly flip the training images
      # and ground-truth for data augmentation
      transforms.append(T.RandomHorizontalFlip(0.5))
   return T.Compose(transforms)
</code></pre>
<p>2nd says</p>
<pre><code class="lang-auto">%%shell

# Download TorchVision repo to use some files from
# references/detection
git clone https://github.com/pytorch/vision.git
cd vision
git checkout v0.8.2

cp references/detection/utils.py ../
cp references/detection/transforms.py ../
cp references/detection/coco_eval.py ../
cp references/detection/engine.py ../
cp references/detection/coco_utils.py ../

from engine import train_one_epoch, evaluate
import utils
import transforms as T


def get_transform(train):
    transforms = []
    # converts the image, a PIL image, into a PyTorch Tensor
    transforms.append(T.ToTensor())
    if train:
        # during training, randomly flip the training images
        # and ground-truth for data augmentation
        transforms.append(T.RandomHorizontalFlip(0.5))
    return T.Compose(transforms)
</code></pre>
<p>I made the changes as you suggested</p>
<pre><code class="lang-auto">import os
os.chdir("/content/drive/MyDrive/PytorchObjectDetector/")
import numpy as np
import torch
import torch.utils.data
from PIL import Image
import pandas as pd

from torchvision.models.detection.faster_rcnn import FastRCNNPredictor

from engine import train_one_epoch, evaluate
import utils
import torchvision.transforms as transforms
import torchvision

def get_transform(train):
   transforms = []
   # converts the image, a PIL image, into a PyTorch Tensor
   transforms.append(transforms.ToTensor())
   if train:
      # during training, randomly flip the training images
      # and ground-truth for data augmentation
      transforms.append(transforms.RandomHorizontalFlip(0.5))
   return transforms.Compose(transforms)
</code></pre>
<p>On running this code</p>
<pre><code class="lang-auto"># use our dataset and defined transformations
dataset = FrDataset(root= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/",
          data_file= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/data/fr_labels.csv",
          transforms = get_transform(train=True))
dataset_test = FrDataset(root= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset",
               data_file= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/data/fr_labels.csv",
               transforms = get_transform(train=False))
</code></pre>
<p>I get his error</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-9-f0fdd0373b7b&gt; in &lt;module&gt;()
      2 dataset = FemaleConnectorDataset(root= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/",
      3           data_file= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/data/fr_labels.csv",
----&gt; 4           transforms = get_transform(train=True))
      5 dataset_test = F
rDataset(root= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset",
      6                data_file= "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/data/fr_labels.csv",

&lt;ipython-input-8-b85ee1833ff6&gt; in get_transform(train)
      2    transforms = []
      3    # converts the image, a PIL image, into a PyTorch Tensor
----&gt; 4    transforms.append(transforms.ToTensor())
      5    if train:
      6       # during training, randomly flip the training images

AttributeError: 'list' object has no attribute 'ToTensor'
</code></pre>
<p>What would you suggest I do? I am confused</p>

username: ptrblck
cooked: <p>You are importing <code>transforms</code> via:</p>
<pre><code class="lang-python">import torchvision.transforms as transforms
</code></pre>
<p>and are then overwriting this modules as a <code>list</code> via:</p>
<pre><code class="lang-python">transforms = []
</code></pre>
<p>which yields the error:</p>
<pre><code class="lang-python">transforms.append(transforms.ToTensor())
&gt; AttributeError: 'list' object has no attribute 'ToTensor'
</code></pre>
<p>Don’t use the same names for variables and for modules you’ve imported.</p>

username: Santhoshnumberone
cooked: <p>I followed your suggestion, changed the <code>transforms</code> to an alternative name</p>
<p>Now I get a different error when I run this block</p>
<pre><code class="lang-auto">num_epochs = 10
for epoch in range(num_epochs):
   # train for one epoch, printing every 10 iterations
   train_one_epoch(model, optimizer, data_loader, device, epoch,print_freq=10)
# update the learning rate
   lr_scheduler.step()
   # evaluate on the test dataset
   evaluate(model, data_loader_test, device=device)

os.mkdir("/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/")
torch.save(model.state_dict(), "/content/drive/MyDrive/PytorchObjectDetector/fr_dataset/model")
</code></pre>
<p>Error I get</p>
<pre><code class="lang-auto">/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:490: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  cpuset_checked))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-16-8e894f6729ee&gt; in &lt;module&gt;()
      2 for epoch in range(num_epochs):
      3    # train for one epoch, printing every 10 iterations
----&gt; 4    train_one_epoch(model, optimizer, data_loader, device, epoch,print_freq=10)
      5 # update the learning rate
      6    lr_scheduler.step()

5 frames
/usr/local/lib/python3.7/dist-packages/torch/_utils.py in reraise(self)
    455             # instantiate since we don't know how to
    456             raise RuntimeError(msg) from None
--&gt; 457         raise exception
    458 
    459 

TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataset.py", line 471, in __getitem__
    return self.dataset[self.indices[idx]]
  File "&lt;ipython-input-6-8432aadbd103&gt;", line 28, in __getitem__
    img, target = self.transforms(img, target)
TypeError: __call__() takes 2 positional arguments but 3 were given
</code></pre>
<p>This the block of code where the error is pointing to</p>
<pre><code class="lang-auto">class FrDataset(torch.utils.data.Dataset):
    def __init__(self, root, data_file, transforms=None):
        self.root = root
        self.transforms = transforms
        self.imgs = sorted(os.listdir(os.path.join(root, "images")))
        self.path_to_data_file = data_file
    def __getitem__(self, idx):
        # load images and bounding boxes
        img_path = os.path.join(self.root, "images", self.imgs[idx])
        img = Image.open(img_path).convert("RGB")
        box_list = parse_one_annot(self.path_to_data_file, 
        self.imgs[idx])
        boxes = torch.as_tensor(box_list, dtype=torch.float32)
        num_objs = len(box_list)
        # there is only one class
        labels = torch.ones((num_objs,), dtype=torch.int64)
        image_id = torch.tensor([idx])
        area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:,0])
        # suppose all instances are not crowd
        iscrowd = torch.zeros((num_objs,), dtype=torch.int64)
        target = {}
        target["boxes"] = boxes
        target["labels"] = labels
        target["image_id"] = image_id
        target["area"] = area
        target["iscrowd"] = iscrowd
        if self.transforms is not None:
            img, target = self.transforms(img, target)
        return img, target
    def __len__(self):
            return len(self.imgs)
</code></pre>

username: ptrblck
cooked: <p><code>self.transforms(img, target)</code> takes one input argument while you are trying to pass two to it.<br>
I’m not familiar with your use case, but maybe you want to use the <code>transforms</code> from <code>torchvision.detection</code> which take two arguments as seen <a href="https://github.com/pytorch/vision/blob/292117405af46e1cdbf2f9ec6eb4752d276bbbb6/references/detection/transforms.py">here</a>.</p>

username: Santhoshnumberone
cooked: <p>Fixed the issue by following <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html#putting-everything-together" rel="noopener nofollow ugc">Putting Everything Together</a> copying the files from the suggested vision version v0.3.0 from the <a href="https://github.com/pytorch/vision/tree/v0.3.0/references/detection" rel="noopener nofollow ugc">github</a> to the working directory</p>