RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 8 and 4 in dimension 1, but the sizes of my tensors only differ in dim 0?

username: H_C
cooked: <pre><code class="lang-auto">for feature in features:
    print("feature.shape = {}".format(feature.shape))
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">feature.shape = torch.Size([4, 2048, 2, 4])
feature.shape = torch.Size([2, 2048, 2, 4])
feature.shape = torch.Size([3, 2048, 2, 4])
feature.shape = torch.Size([7, 2048, 2, 4])
feature.shape = torch.Size([15, 2048, 2, 4])
feature.shape = torch.Size([6, 2048, 2, 4])
feature.shape = torch.Size([5, 2048, 2, 4])
feature.shape = torch.Size([3, 2048, 2, 4])
feature.shape = torch.Size([3, 2048, 2, 4])
feature.shape = torch.Size([8, 2048, 2, 4])
</code></pre>
<p>But when I run<br>
<code>result = torch.stack(features)</code>, I get this error:</p>
<p><code>RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 8 and 4 in dimension 1 at C:/w/1/s/tmp_conda_3.6_045031/conda/conda-bld/pytorch_1565412750030/work/aten/src\THC/generic/THCTensorMath.cu:71</code></p>
<p>Anyone knows why?</p>

username: ptrblck
cooked: <p>I’m not sure, what your code is doing, so could you explain a bit, what the dimensions mean?<br>
Are you using <code>nn.DataParallel</code> or DDP?</p>

username: H_C
cooked: <p>Ah, sorry about that.</p>
<pre><code class="lang-auto">transform = transforms.Compose([
            transforms.ToPILImage(),
            transforms.Resize((240,320)),
            transforms.ToTensor(),
            transforms.Normalize(MEAN, STD)
        ])

for video in videos:
   frames = [transform(frame) for frame in video]
   frames = torch.stack(frames)
   frames = frames.cuda()
   feature = feature_extractor(frames)
   features.append(feature)
</code></pre>
<p>I am trying to perform action recognition using CNN, and this the above is part of the Dataset class I am trying to write. Frames are images at a certain point in the video. I am not too sure what the dimensions mean to be honest… But I’m using Resnet-50 as the feature extractor, so that explains the 2048.</p>
<p>Other things I’ve tried:</p>
<ul>
<li>converting the frames to RGB using PIL.Image (shouldn’t be required since the training videos are already in colour)</li>
<li>setting batch_size = 1</li>
</ul>
<p>Both didn’t work…</p>
<p>As for <code>DataParallel</code> and <code>DDP</code>, I’m not too sure what those two are. I just write a <code>Dataset</code> class, get the data using a <code>dataloader</code> and iterate over it.</p>

username: alex.veuthey
cooked: <p>DataParallel and DistributedDataParallel are ways of leveraging multiple GPUs (and multiple machines for Distributed) to train faster. Pretty practical!</p>
<p>As for your error, what is the type of <code>features</code>? I suspect <code>torch.stack</code> will try to use the first dimension of your ensemble of features, which apparently has shape [10, X, 2048, 2, 4], for stacking. Thus your error at dimension 1, because of the varying sizes. If <code>features</code> is not a list, try to convert it into a list, and <code>torch.stack</code> should work by specifying <code>dim=0</code> (default).</p>

username: H_C
cooked: <p>features is a list! <code>[]</code><br>
I’m a little confused when you say my features have the shape <code>[10, X, 2048, 2, 4]</code>, because based on the printed output I wrote in my question, they have the shape <code>[X,2048,2,4]</code> don’t they?</p>

username: alex.veuthey
cooked: <p>A single feature has shape [X, 2048, 2, 4], but the <code>features</code> list has 10 features, therefore axis 0 has 10 elements…<br>
Can you try <code>torch.stack(features, dim=1)</code> maybe?</p>

username: H_C
cooked: <p>Ah I see! Somehow I get the same error except in a different dimension<br>
<code>RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 2 and 4 in dimension 0 at C:/w/1/s/tmp_conda_3.6_045031/conda/conda-bld/pytorch_1565412750030/work/aten/src\THC/generic/THCTensorMath.cu:71</code></p>
<p>but I was looking through a Github repo and they did what would be the equivalent of<br>
<code>features.append(torch.mean(feature,0))</code> in my code. That seems to solve the issue (though I don’t get why), but I get a different error now.</p>