F.interpolate weird behaviour

Pierre_Ouannes: <p>Hi !</p>
<p>So I might be missing something basic but I’m getting a weird behavior with <code>F.interpolate</code> :</p>
<p>I created a 3D-tensor using :</p>
<pre><code class="lang-auto">t  = torch.randn(4,4,4)
</code></pre>
<p>Or a least I thought it was 3D, but <code>F.interpolate</code> doesn’t seem to agree. The following code :</p>
<pre><code class="lang-auto">F.interpolate(t, scale_factor=(1,2,1))
</code></pre>
<p>gives the error</p>
<pre><code class="lang-auto">ValueError: scale_factor shape must match input shape. Input is 1D, scale_factor size is 3
</code></pre>
<p>What am I missing here ?</p>

ptrblck: <p>The error message might be a bit weird, but it refers to an input of shape <code>[batch_size, channels, *addidional_dims]</code>.<br>
Given that you provide a “1D” input with a length of 4.<br>
Here would be an example using an image tensor:</p>
<pre><code class="lang-python">batch_size, c, h, w = 1, 3, 4, 4
x  = torch.randn(batch_size, c, h, w)
x = F.interpolate(x, scale_factor=(2,1))
print(x.shape)
&gt; torch.Size([1, 3, 8, 4])
</code></pre>
<p>As you can see, the batch size as well as the channels won’t be manipulated, just the spatial dimensions.</p>

jinfagang: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Hi, ptr.</p>
<p>what about a tensor shape like c=1:</p>
<p>torch.randn(B, 1, 512, 512)</p>
<p>I got error:</p>
<pre><code class="lang-auto">ValueError: size shape must match input shape. Input is 1D, size is 2

</code></pre>
<p>When try to … interplolate on it</p>

ptrblck: <p>It works for me:</p>
<pre><code class="lang-python">batch_size, c, h, w = 3, 1, 512, 512
x  = torch.randn(batch_size, c, h, w)
x = F.interpolate(x, scale_factor=(2,1))
</code></pre>
<p>Based on your error message I guess you are using a 1D tensor?</p>

Mona_Jalal: <p>I get this error ValueError: size shape must match input shape. Input is 1D, size is 2</p>
<p>for when I apply transform of resize 2D on a 2D feature. However, I am not fully sure why it tells me my feature is 1D? What do you think?</p>
<p>features is: <code>torch.Size([503, 512])</code></p>
<pre><code class="lang-auto">  transformed_features = self.transform(features)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 297, in forward
    return F.resize(img, self.size, self.interpolation, self.max_size, self.antialias)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 403, in resize
    return F_t.resize(img, size=size, interpolation=interpolation.value, max_size=max_size, antialias=antialias)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py", line 552, in resize
    img = interpolate(img, size=[new_h, new_w], mode=interpolation, align_corners=align_corners)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torch/nn/functional.py", line 3630, in interpolate
    raise ValueError(
ValueError: size shape must match input shape. Input is 1D, size is 2
</code></pre>
<p>and transforms are:</p>
<pre><code class="lang-auto">train_transforms = transforms.Compose(
    [
        transforms.Resize((256, 512)),
        transforms.RandomResizedCrop(256),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
    ]
)

val_transforms = transforms.Compose(
    [
        transforms.Resize((256, 512)),
        transforms.CenterCrop(256),
        transforms.ToTensor(),
    ]
)
</code></pre>
<p>I apply the transform like this:<br>
<code>    dataset_train = GraphDataset(os.path.join(data_path, ""), ids_train, transform=train_transforms)</code></p>
<p>and I have:</p>
<pre><code class="lang-auto">class GraphDataset(data.Dataset):
    """input and label image dataset"""

    def __init__(self, root, ids, target_patch_size=-1, transform=None):
    *code*
    def __getitem__(self, index):
    *code*
     transformed_features = self.transform(features)
     #sample['image'] = features
     sample['image'] = transformed_features
     return sample
</code></pre>
<p>I am not precisely sure what I am doing wrong.</p>

ptrblck: <p>Assuming you are passing a tensor to the transformation it should have at least 3 dims as <code>[channels, height, width]</code> and can have additional leading dimensions (e.g. the batch dimension).<br>
If that’s the case, <code>unsqueeze</code> the channel dimension and remove the <code>ToTensor</code> transformation.</p>
<pre><code class="lang-python">transform = transforms.Resize((256, 512))
x = torch.randn(3, 400, 600)
out = transform(x)
print(out.shape)
# &gt; torch.Size([3, 256, 512])

x = torch.randn(503, 512)
out = transform(x)
# &gt; ValueError: size shape must match input shape. Input is 1D, size is 2
</code></pre>

Mona_Jalal: <p>Thanks a lot for your response. I removed the ToTensor ones from the transformations, and get same error:</p>
<pre><code class="lang-auto">
train_transforms = transforms.Compose(
    [
        transforms.Resize((256, 512)),
        transforms.RandomResizedCrop(256),
        transforms.RandomHorizontalFlip(),
        #transforms.ToTensor(),
    ]
)

val_transforms = transforms.Compose(
    [
        transforms.Resize((256, 512)),
        transforms.CenterCrop(256),
        #transforms.ToTensor(),
    ]
)
</code></pre>
<pre><code class="lang-auto"> transformed_features = self.transform(features)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 297, in forward
    return F.resize(img, self.size, self.interpolation, self.max_size, self.antialias)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 403, in resize
    return F_t.resize(img, size=size, interpolation=interpolation.value, max_size=max_size, antialias=antialias)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py", line 552, in resize
    img = interpolate(img, size=[new_h, new_w], mode=interpolation, align_corners=align_corners)
  File "/SeaExp/mona/venv/dpcc/lib/python3.8/site-packages/torch/nn/functional.py", line 3630, in interpolate
    raise ValueError(
ValueError: size shape must match input shape. Input is 1D, size is 2

torch.Size([2957, 512])
</code></pre>
<p>This is despite the fact that features is a 2D input:<br>
<code>features size:  torch.Size([1813, 512])</code><br>
here:</p>
<pre><code class="lang-auto">print("features size: ", features.shape)
transformed_features = self.transform(features)
</code></pre>

ptrblck: <p>As my code snippet shows, a 3D tensor is expected. You can copy-paste my code and check the error and compare it to the working solution.</p>

4ndr3aR: <p>I’m sorry but this function can’t be correct, with the <code>size</code> parameter it doesn’t work as expected in any way or with any size.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; batch_size, c, h, w = 2, 3, 5, 6
&gt;&gt;&gt; x  = torch.randn(batch_size, c, h, w)
&gt;&gt;&gt; x.shape
torch.Size([2, 3, 5, 6])
&gt;&gt;&gt; x = F.interpolate(x, size=(2, 3, 10, 12))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/oak-d/venvs/depthai/lib/python3.8/site-packages/torch/nn/functional.py", line 3652, in interpolate
    raise ValueError(
ValueError: size shape must match input shape. Input is 2D, size is 4
</code></pre>
<pre><code class="lang-auto">&gt;&gt;&gt; batch_size, c, h, w = 1, 3, 5, 6
&gt;&gt;&gt; x  = torch.randn(c, h, w)
&gt;&gt;&gt; x.shape
torch.Size([3, 5, 6])
&gt;&gt;&gt; x
tensor([[[ 0.1398,  2.6164,  1.5493, -3.3864, -0.4263,  0.0538],
         [ 0.6553,  1.3214, -0.2174, -0.4452, -0.7426, -0.3066],
         [-0.8942,  0.7908, -0.7685,  0.9150,  0.7674,  0.3924],
         [ 0.7903,  0.5168, -0.3655, -0.6867, -0.5429, -1.5122],
         [-0.5823,  0.5922, -0.0183, -0.5923,  0.0494, -0.0694]],

        [[-0.6175, -0.0933,  0.6786, -0.3152, -0.8401, -0.4897],
         [-0.7455,  0.4002, -0.2659, -0.5124, -0.1008,  1.5574],
         [-0.6002, -0.3244, -0.5347,  0.7308, -2.1288,  0.6471],
         [ 0.2617, -1.5748, -0.1861, -0.8950,  1.6602,  0.3907],
         [ 1.9359, -1.3339,  0.1310, -1.5444, -0.4120,  1.6819]],

        [[-1.1157,  1.1354,  0.9137, -2.6809, -0.4930, -0.5975],
         [ 1.4078,  1.0812,  0.3403,  0.9431, -2.5339,  0.0154],
         [-1.4800,  0.3183, -0.2253, -0.3619, -0.0167, -2.0883],
         [ 1.6934, -1.6267,  1.0259, -0.7392,  0.0561, -0.7352],
         [ 1.1638,  0.7155, -0.1465,  1.4306,  0.5155,  0.2121]]])
&gt;&gt;&gt; x = F.interpolate(x, size=(3, 10, 12))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/oak-d/venvs/depthai/lib/python3.8/site-packages/torch/nn/functional.py", line 3652, in interpolate
    raise ValueError(
ValueError: size shape must match input shape. Input is 1D, size is 3
</code></pre>
<pre><code class="lang-auto">&gt;&gt;&gt; x  = torch.randn(h, w)
&gt;&gt;&gt; x.shape
torch.Size([5, 6])
&gt;&gt;&gt; x = F.interpolate(x, size=(10, 12))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/oak-d/venvs/depthai/lib/python3.8/site-packages/torch/nn/functional.py", line 3652, in interpolate
    raise ValueError(
ValueError: size shape must match input shape. Input is 0D, size is 2
</code></pre>
<p>I’ve been trying to resize a 4D-tensor for hours, with <code>torch.nn.functional.interpolate</code> or with <code>torchvision.transforms.Resize</code>, without success. What is the “canonical” way to resize a batch with PyTorch? Upgrading to <code>torch==1.12.1</code> and <code>torchvision==0.13.1</code> has no effect.</p>

4ndr3aR: <p>Ohhh, it’s devious! At line 3066, <code>torch/nn/functional.py</code> does:</p>
<pre><code class="lang-auto">    dim = input.dim() - 2  # Number of spatial dimensions.

    # Process size and scale_factor.  Validate that exactly one is set.
    # Validate its length if it is a list, or expand it if it is a scalar.
    # After this block, exactly one of output_size and scale_factors will
    # be non-None, and it will be a list (or tuple).
    if size is not None and scale_factor is not None:
        raise ValueError('only one of size or scale_factor should be defined')
    elif size is not None:
        assert scale_factor is None 
        scale_factors = None 
        if isinstance(size, (list, tuple)):
            if len(size) != dim: 
                raise ValueError('size shape must match input shape. '
                                 'Input is {}D, size is {}'.format(dim, len(size)))
</code></pre>
<p>So, if the tensor has no <code>bs</code> (a 3D-tensor), <code>len(size)</code> must be 1 (square images only?). And if it has a <code>bs</code> (it’s a 4D-tensor) AND <code>len(size) == 2</code>… it finally works! <img src="https://discuss.pytorch.org/images/emoji/apple/tada.png?v=12" title=":tada:" class="emoji" alt=":tada:" loading="lazy" width="20" height="20"></p>
<p>Nice, so this function works properly only if tensors have a <code>bs</code> (4D-tensors) and the user specifies <code>(h,w)</code> (and nothing more) as size parameter, every other combination throws a <code>ValueError</code> without providing any meaningful insight in the error message. At least we have unveiled also this mystery.</p>