I am seeing a shape error when try to detect objects in video?

username: Teddy_Salas
cooked: <p>here is the code of model:- class tinyvgg12(nn.Module):<br>
def <strong>init</strong>(self,input,hidden,output):<br>
super().<strong>init</strong>()<br>
self.Conv_block1 = nn.Sequential(<br>
nn.Conv2d(<br>
in_channels = input,<br>
out_channels = hidden,<br>
kernel_size=3,<br>
stride=1,<br>
padding=1<br>
),<br>
nn.ReLU(),<br>
nn.Conv2d(<br>
in_channels = hidden,<br>
out_channels = hidden,<br>
kernel_size=3,<br>
stride=1,<br>
padding=1<br>
),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2,stride=2)<br>
)</p>
<pre><code>self.Conv_block2 = nn.Sequential(
    nn.Conv2d(
        in_channels =hidden,
        out_channels = hidden,
        kernel_size=3,
        stride=1,
        padding=1
    ),
    nn.ReLU(),
    nn.Conv2d(
        in_channels=hidden,
        out_channels=hidden,
        kernel_size=3,
        stride=1,
        padding=1
    ),
    nn.ReLU(),
    nn.MaxPool2d(kernel_size=2,stride=2)
)

self.classifier = nn.Sequential(
    nn.Flatten(),
    nn.Linear(
        in_features=hidden*16*16,
        out_features=output
    )
)
</code></pre>
<p>def forward(self,x):<br>
x = self.Conv_block1(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
x = self.Conv_block2(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
x = self.classifier(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
return x<br>
return self.Conv_block2(self.Conv_block1(x))</p>
<p>here is the code for detecting objects:-<br>
import cv2<br>
import torch<br>
import torchvision.transforms as transforms<br>
from torchvision.models import resnet50</p>
<p>model = model_x_5<br>
device  = “cuda” if torch.cuda.is_available() else “cpu”<br>
model.eval()<br>
transform = transforms.Compose([<br>
transforms.ToPILImage(),<br>
transforms.Resize(size=(224,224)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225])<br>
])<br>
video_path = “/content/data/Baby brushes his teeth.mp4”<br>
cap =  cv2.VideoCapture(video_path)</p>
<p>while cap.isOpened():<br>
ret,frame = cap.read()<br>
if not ret:<br>
break<br>
frame_rgb = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)<br>
input_tensor = transform(frame_rgb)<br>
input_tensor = input_tensor.unsqueeze(0)<br>
with torch.no_grad():<br>
output = model(input_tensor.to(device))</p>
<pre><code>cv2.imshow("object detection",frame)
if cv2.waitKey(1) &amp; 0xFF == ord('q'):
  break
</code></pre>
<p>cap.release()<br>
cv2.destroyAllWindows()</p>
<h2>
<a name="error-1" class="anchor" href="#error-1"></a>error:-</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in &lt;cell line: 19&gt;()<br>
25   input_tensor = input_tensor.unsqueeze(0)<br>
26   with torch.no_grad():<br>
—&gt; 27     output = model(input_tensor.to(device))<br>
28<br>
29     cv2.imshow(“object detection”,frame)</p>
<p>5 frames<br>
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input)<br>
112<br>
113     def forward(self, input: Tensor) → Tensor:<br>
 → 114         return F.linear(input, self.weight, self.bias)<br>
115<br>
116     def extra_repr(self) → str:</p>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x200704 and 16384x1)</p>

username: ptrblck
cooked: <p>Based on the error message the classifier is failing as the feature dimension does not match the features of the input activation. Set <code>in_features</code> to <code>200704</code> in the linear layer and it should work.</p>

username: Teddy_Salas
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> model_x_5 = tinyvgg12(input=3,<br>
hidden=200704<br>
output=len(class_list)) like this?</p>

username: ptrblck
cooked: <p>No, this won’t work since you are initializing the layer as:</p>
<pre><code class="lang-python">nn.Linear(
    in_features=hidden*16*16,
    out_features=output
)
</code></pre>
<p>in which case <code>hidden</code> should be <code>200704 / 16 / 16 = 784</code>.</p>

username: Teddy_Salas
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
model_x_5 = tinyvgg12(input=3,<br>
hidden=784<br>
output=len(class_list))<br>
nn.Linear(<br>
in_features=hidden<em>16</em>16<br>
out_features=output<br>
)<br>
like this ?</p>

username: Teddy_Salas
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  i given model_x_5 = tinyvgg12(input=3,<br>
hidden=784<br>
output=len(class_list))<br>
nn.Linear(<br>
in_features=hidden<em>16</em> 16<br>
out_features=output<br>
) like this but still seeing the same error:----------------------------------------------------------------------------<br>
RuntimeError                              Traceback (most recent call last)<br>
 in &lt;cell line: 19&gt;()<br>
25   input_tensor = input_tensor.unsqueeze(0)<br>
26   with torch.no_grad():<br>
—&gt; 27     output = model(input_tensor.to(device))<br>
28<br>
29     cv2.imshow(“object detection”,frame)</p>
<p>5 frames<br>
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input)<br>
112<br>
113     def forward(self, input: Tensor) → Tensor:<br>
 → 114         return F.linear(input, self.weight, self.bias)<br>
115<br>
116     def extra_repr(self) → str:</p>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x2458624 and 200704x1)</p>

username: ptrblck
cooked: <p>Based on the error message the current input changed in shape which causes the same error with another shape.<br>
Make sure your inputs have the same shape or add adaptive pooling layers if you are working with different input shapes.</p>

username: Teddy_Salas
cooked: <p>how to add adaptive pooling layers &amp; how do i check inputs have same shape.</p>

username: ptrblck
cooked: <p>You can check the shape of the inputs by simply printing it via <code>print(input.shape)</code> where <code>input</code> is the input tensor.</p>
<p>You can check e.g. the <a href="https://github.com/pytorch/vision/blob/a5035df501747c8fc2cd7f6c1a41c44ce6934db3/torchvision/models/resnet.py#L278">ResNet implementation</a> to see how adaptive pooling layers are used.</p>

username: Teddy_Salas
cooked: <p>class tinyvgg12(nn.Module):<br>
def <strong>init</strong>(self,input,hidden,output):<br>
super().<strong>init</strong>()<br>
self.Conv_block1 = nn.Sequential(<br>
nn.Conv2d(<br>
in_channels = input,<br>
out_channels = hidden,<br>
kernel_size=3,<br>
stride=1,<br>
padding=1<br>
),<br>
nn.ReLU(),<br>
nn.Conv2d(<br>
in_channels = hidden,<br>
out_channels = hidden,<br>
kernel_size=3,<br>
stride=1,<br>
padding=1<br>
),<br>
nn.ReLU(),<br>
nn.MaxPool2d(kernel_size=2,stride=2)<br>
)</p>
<pre><code>self.Conv_block2 = nn.Sequential(
    nn.Conv2d(
        in_channels =hidden,
        out_channels = hidden,
        kernel_size=3,
        stride=1,
        padding=1
    ),
    nn.ReLU(),
    nn.Conv2d(
        in_channels=hidden,
        out_channels=hidden,
        kernel_size=3,
        stride=1,
        padding=1
    ),
    nn.ReLU(),
    nn.MaxPool2d(kernel_size=2,stride=2)
)

self.classifier = nn.Sequential(
    nn.Flatten(),
    nn.Linear(
        in_features=hidden*16*16,
        out_features=output
    )
)
self.avgpool = nn.AdaptiveAvgPool2d((3,3))
</code></pre>
<p>def forward(self,x):<br>
x = self.Conv_block1(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
x = self.Conv_block2(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
x = self.classifier(x)<br>
<span class="hashtag">#print</span>(x.shape)<br>
x = self.avgpool(x)<br>
return x<br>
return self.Conv_block2(self.Conv_block1(x))    i put adaptive pooling but i still seeing the error :----------------------------------------------------------------------------<br>
ValueError                                Traceback (most recent call last)<br>
 in &lt;cell line: 9&gt;()<br>
8<br>
9 for epoch in tqdm(range(epochs)):<br>
—&gt; 10   train_step(model=model_x_5.to(device),<br>
11              data_loader=train_dataloader,<br>
12              optimizer=optimizer,</p>
<p>6 frames<br>
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/utils.py in _list_with_default(out_size, defaults)<br>
35         return out_size<br>
36     if len(defaults) &lt;= len(out_size):<br>
—&gt; 37         raise ValueError(<br>
38             “Input dimension should be at least {}”.format(len(out_size) + 1)<br>
39         )</p>
<p>ValueError: Input dimension should be at least 3</p>