VGG19 RuntimeError: stride should not be zero

maxrivera: <p>I am using transfer learning to tune a pretrained VGG19 model. I am using MPS on Mac M1. Here is the printed model and error that occurs during forward pass:<br>
MODEL:<br>
VGG(<br>
(features): Sequential(<br>
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(1): ReLU(inplace=True)<br>
(2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(3): ReLU(inplace=True)<br>
(4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(6): ReLU(inplace=True)<br>
(7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(8): ReLU(inplace=True)<br>
(9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(11): ReLU(inplace=True)<br>
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(13): ReLU(inplace=True)<br>
(14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(15): ReLU(inplace=True)<br>
(16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(17): ReLU(inplace=True)<br>
(18): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(19): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(20): ReLU(inplace=True)<br>
(21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(22): ReLU(inplace=True)<br>
(23): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(24): ReLU(inplace=True)<br>
(25): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(26): ReLU(inplace=True)<br>
(27): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(29): ReLU(inplace=True)<br>
(30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(31): ReLU(inplace=True)<br>
(32): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(33): ReLU(inplace=True)<br>
(34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))<br>
(35): ReLU(inplace=True)<br>
(36): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
)<br>
(avgpool): AdaptiveAvgPool2d(output_size=(7, 7))<br>
(classifier): Sequential(<br>
(0): Linear(in_features=25088, out_features=8192, bias=True)<br>
(1): ReLU(inplace=True)<br>
(2): Linear(in_features=8192, out_features=256, bias=True)<br>
(3): ReLU(inplace=True)<br>
(4): Linear(in_features=256, out_features=2, bias=True)<br>
)<br>
)</p>
<p>ERROR:<br>
Traceback (most recent call last):<br>
File “~/Desktop/ObjectSecurity/adversarial-segmentation/catsdogs_torch.py”, line 124, in <br>
pred = model(x)<br>
File “~/miniforge3/envs/myenv2/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1186, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “~/miniforge3/envs/myenv2/lib/python3.9/site-packages/torchvision/models/vgg.py”, line 67, in forward<br>
x = self.avgpool(x)<br>
File “~/miniforge3/envs/myenv2/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1186, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “~/miniforge3/envs/myenv2/lib/python3.9/site-packages/torch/nn/modules/pooling.py”, line 1183, in forward<br>
return F.adaptive_avg_pool2d(input, self.output_size)<br>
File “~/miniforge3/envs/myenv2/lib/python3.9/site-packages/torch/nn/functional.py”, line 1214, in adaptive_avg_pool2d<br>
return torch._C._nn.adaptive_avg_pool2d(input, _output_size)<br>
RuntimeError: stride should not be zero</p>

eqy: <p>Could you check what the input shape to the adaptive avg pool layer is? My first guess is that could be the cause of the inferred 0 stride.</p>

maxrivera: <p>input shape to avgpool: torch.Size([64, 512, 4, 4])</p>

eqy: <p>Interesting, do you see any errors with the following snippet?</p>
<pre><code class="lang-auto">import torch

avgpool = torch.nn.AdaptiveAvgPool2d((7, 7))
a = torch.randn(64, 512, 4, 4, device='mps')
avgpool(a)
</code></pre>

maxrivera: <p>I get the same error. Any idea why? Should the input size be bigger than the output size?</p>

eqy: <p>I believe this is an issue that is specific to the MPS backend as I cannot observe it on <code>cpu</code> or <code>cuda</code>. Will open a bug to track this issue.</p>
<p>EDIT: issue is here <a href="https://github.com/pytorch/pytorch/issues/80732" class="inline-onebox" rel="noopener nofollow ugc">[MPS] AdaptiveAvgPool2D doesn't accept input spatial size smaller than output shape · Issue #80732 · pytorch/pytorch · GitHub</a></p>