RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 100 and 25 in dimension 1

Omar2017: <p>Hi,<br>
I want to extract whole frames from videos in the dataset. the dataset contains videos with a different number of frames. when I tried to extract frames using the following code:</p>
<pre><code class="lang-auto">import torch
from torch.utils.data import Dataset
from PIL import Image
import numpy as np
import cv2

class makeDataset(Dataset):
    def __init__(self, dataset, labels, spatial_transform, seqLen):
        self.spatial_transform = spatial_transform
        self.images = dataset
        self.labels = labels
        self.seqLen = seqLen

    def __len__(self):
        return len(self.images)

    def __getitem__(self, idx):
        vid_name = self.images[idx]
        label = self.labels[idx]
        inpSeq = []
        vid=cv2.VideoCapture(vid_name)
        index=0
        frame =None
        ret, frame = vid.read()
        while(index&lt;=self.seqLen):
            # Extract images
            ret, frame = vid.read()
            if not ret:
                break
            index+=1
            img=Image.fromarray(frame)
            inpSeq.append(self.spatial_transform(img.convert('RGB')))
        inpSeq = torch.stack(inpSeq, 0)
        return inpSeq, label
</code></pre>
<p>i got the following error</p>
<pre><code class="lang-auto">Traceback (most recent call last):

  File "&lt;ipython-input-11-947fd8c9ce15&gt;", line 1, in &lt;module&gt;
    runfile('C:/Users/Windows10/Downloads/CodeF/main-run-vr.py', wdir='C:/Users/Windows10/Downloads/CodeF')

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/Windows10/Downloads/CodeF/main-run-vr.py", line 403, in &lt;module&gt;
    __main__()

  File "C:/Users/Windows10/Downloads/CodeF/main-run-vr.py", line 401, in __main__
    evalInterval, evalMode, numWorkers, outDir,modelUsed,pretrained,train_test_split,datasetDir,crossValidation,nFolds)

  File "C:/Users/Windows10/Downloads/CodeF/main-run-vr.py", line 360, in main_run
    modelTrain(folds,modelUsed,pretrained,data,label,class_names,data2,label2,class_names2,numEpochs,evalInterval,evalMode,outDir,numWorkers,lr, stepSize, decayRate, trainBatchSize, seqLen, True)

  File "C:/Users/Windows10/Downloads/CodeF/main-run-vr.py", line 187, in modelTrain
    for i, (inputs, targets) in enumerate(trainLoader):

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\torch\utils\data\dataloader.py", line 346, in __next__
    data = self.dataset_fetcher.fetch(index)  # may raise StopIteration

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\torch\utils\data\_utils\fetch.py", line 47, in fetch
    return self.collate_fn(data)

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\torch\utils\data\_utils\collate.py", line 80, in default_collate
    return [default_collate(samples) for samples in transposed]

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\torch\utils\data\_utils\collate.py", line 80, in &lt;listcomp&gt;
    return [default_collate(samples) for samples in transposed]

  File "C:\Users\Windows10\Anaconda3\envs\New\lib\site-packages\torch\utils\data\_utils\collate.py", line 56, in default_collate
    return torch.stack(batch, 0, out=out)

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 100 and 25 in dimension 1 at C:\w\1\s\tmp_conda_3.6_045031\conda\conda-bld\pytorch_1565412750030\work\aten\src\TH/generic/THTensor.cpp:689
</code></pre>
<p>the sequence length dimension must be the same for all videos. How can I extract all frames from videos that contains a different number of frames?<br>
Thanks in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

JuanFMontesinos: <p>You can return a list instead of a tensor if your dimensions are missmatching.<br>
Obviously torch.stack requires both tensors to share dimensionality.<br>
You can pad one of them up to the size of the other but list seems simpler.</p>
<p>Returning a list you will get a a list of list of frames</p>

midhunharikumar: <aside class="quote no-group quote-modified" data-username="Omar2017" data-post="1" data-topic="61035">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/omar2017/48/11102_2.png" class="avatar"> Omar2017:</div>
<blockquote>
<p>import torch from torch.utils.data import Dataset from PIL import Image import numpy as np import cv2 class makeDataset(Dataset): def <strong>init</strong>(self, dataset, labels, spatial_transform, seqLen): self.spatial_transform = spatial_transform self.images = dataset self.labels = labels self.seqLen = seqLen def <strong>len</strong>(self): return len(self.images) def <strong>getitem</strong>(self, idx): vid_name = self.images[idx] label = self.labels[idx] inpSeq = <span class="chcklst-box fa fa-square-o fa-fw"></span> vid=cv2.VideoCapture(vid_name) index=0 frame =None ret, frame = vid.read() while(index&lt;=self.seqLen): # Extract images ret, frame = vid.read() if not ret: break index+=1 img=Image.fromarray(frame) inpSeq.append(self.spatial_transform(img.convert(‘RGB’))) inpSeq = torch.stack(inpSeq, 0) return inpSeq, label</p>
</blockquote>
</aside>
<p>Theres’ something wrong in your spatial transformation function. Are you sure that the <code>spatial_transform</code> generates output in the format <code>(B,C,H,W)</code> ?. Also <code>self.seqLen</code> can be inferred from the video in OpenCV.</p>

Omar2017: <p>Thank you, but how can I turned again to tensor because I need to make some operations on it?</p>

midhunharikumar: <p>I don’t follow. I think you would return make the tensor conversion the way you are doing now. using <code>torch.stack</code> .</p>