Shape '[2, 96, 8, 56, 56]' is invalid for input of size 602112

whovivkrajput: <p>class Model(torch.nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Model, self).<strong>init</strong>()<br>
self.weight = MViT_V2_S_Weights.DEFAULT<br>
self.model = mvit_v2_s(weights=self.weight)<br>
self.dense = torch.nn.Linear(in_features=400, out_features=1)<br>
self.sigmoid = torch.nn.Sigmoid()</p>
<pre><code>def forward(self, x, labels=None):
    x = self.model(x)
    x = self.dense(x)                                                
    x = self.sigmoid(x)
    if labels != None:
        labels = torch.reshape(labels, (x.shape[0], 1))*1.0
        loss = torch.nn.BCEWithLogitsLoss()(x, labels)
        return x, loss
    return x
</code></pre>
<p>After this i run this piece of code to do some random testing and i get this error…!!<br>
model = Model().to(“cpu”)<br>
model.eval()</p>
<h1>
<a name="modeltorchones2332224224todevice-1" class="anchor" href="#modeltorchones2332224224todevice-1"></a>model(torch.ones(2,3,32,224,224).to(DEVICE))</h1>
<h1>
<a name="modeleval-2" class="anchor" href="#modeleval-2"></a>model.eval()</h1>
<p>for i in range(1,100):<br>
try:<br>
y = model(torch.ones(2,3,i,224,224).to(“cpu”))<br>
print(“select”, i)<br>
except Exception as error:<br>
print(error)<br>
pass</p>
<p>ERROR - shape ‘[2, 96, 8, 56, 56]’ is invalid for input of size 602112</p>

ptrblck: <p>It seems the model is expecting a temporal size of <code>16</code>:</p>
<pre><code class="lang-python">model(torch.ones(2, 3, 16, 224, 224))
</code></pre>
<p>and fails with other sizes.<br>
<a class="mention" href="/u/pmeier">@pmeier</a> do you know if this is expected?</p>

pmeier: <p>Hey <a class="mention" href="/u/whovivkrajput">@whovivkrajput</a>. MaxVit is an image model, but you are passing 5D input. This is not supported. From the <a href="https://pytorch.org/vision/stable/models/generated/torchvision.models.maxvit_t.html#torchvision.models.MaxVit_T_Weights" rel="noopener nofollow ugc">documentation</a></p>
<blockquote>
<p>Accepts <code>PIL.Image</code>, batched <code>(B, C, H, W)</code> and single <code>(C, H, W)</code> image <code>torch.Tensor</code> objects.</p>
</blockquote>
<p>So if you remove the <code>i</code> from</p>
<pre data-code-wrap="python"><code class="lang-python">y = model(torch.ones(2,3,224,224).to(“cpu”))
</code></pre>
<p>It should work.</p>

ptrblck: <p><code>mvit_v2_s</code> seems to be a <code>torchvision.models.video</code> model described <a href="https://pytorch.org/vision/main/models/generated/torchvision.models.video.mvit_v2_s.html#torchvision.models.video.mvit_v2_s">here</a> which expects inputs in:</p>
<blockquote>
<p>Accepts batched <code>(B, T, C, H, W)</code> and single <code>(T, C, H, W)</code> video frame <code>torch.Tensor</code> objects. The frames are resized to <code>resize_size=[256]</code> using <code>interpolation=InterpolationMode.BILINEAR</code>, followed by a central crop of <code>crop_size=[224, 224]</code>. Finally the values are first rescaled to <code>[0.0, 1.0]</code> and then normalized using <code>mean=[0.45, 0.45, 0.45]</code> and <code>std=[0.225, 0.225, 0.225]</code>. Finally the output dimensions are permuted to <code>(..., C, T, H, W)</code> tensors.</p>
</blockquote>
<p>The interesting part is:</p>
<blockquote>
<p>min_size height=224, width=224<br>
min_temporal_size 16</p>
</blockquote>
<p>which seems to be hard-coded as the code fails if the temporal size is changed.<br>
Am I looking at the wrong model?</p>

pmeier: <p>No you are right. I skimmed over that since the code is not properly formatted. <a class="mention" href="/u/whovivkrajput">@whovivkrajput</a> could you  please next time wrap your code in triple backticks?</p>
<p>I’m no expert here, but the <code>clip_len=16</code> is set during training. So if you want to use our pretrained weights, you need to match that.</p>

whovivkrajput: <p>Thank you for your help… <a class="mention" href="/u/pmeier">@pmeier</a> <a class="mention" href="/u/ptrblck">@ptrblck</a> …i do have few questions.</p>
<p>This model can be used for video recognition and looking at the input (2 , 3 , i , 224, 224) . i here is the frame length and according to your answer 16 is the clip_len…so can i use another value for clip_len other than 16???</p>

pmeier: <blockquote>
<p>so can i use another value for clip_len other than 16???</p>
</blockquote>
<p>Nope. This is hardcoded <a href="https://github.com/pytorch/vision/blob/56b0497646653bf1f027ce9c2ba668331252f67c/torchvision/models/video/mvit.py#L886" rel="noopener nofollow ugc">here</a>. You can open an issue if you need other clip lengths and thus being able to set this value in the builder of the model. Note however that this will not change the fact that our weights are trained for 16 frames and that won’t change. Meaning, you will have to train yourself.</p>
<p>For completion, here is full example on how to use the model:</p>
<pre data-code-wrap="python"><code class="lang-python">from torchvision import models
import torch

name = "MViT_V2_S"
builder = models.get_model_builder(name)
weights = models.get_model_weights(name).DEFAULT

model = builder(weights=weights)
transform = weights.transforms()

input = torch.ones(2, 16, 3, 224, 224)
result = model(transform(input))
</code></pre>

pmeier: <p><a class="mention" href="/u/whovivkrajput">@whovivkrajput</a> It seems you are not alone here. Someone raised the issue to us: <a href="https://github.com/pytorch/vision/issues/7345#issue-1601115393" class="inline-onebox" rel="noopener nofollow ugc">The temporal size of MVIT_V2_S can not be greater than 16 · Issue #7345 · pytorch/vision · GitHub</a>.</p>