Removing padding from features extracted from pre-trained VGG16

Ellie_Bowler: <p>I’m using a pre-trained vgg16 as the backbone for a U-Net segmentation model. The code i have is working fine, but I was wondering if there is a way of removing the padding from the feature layers?</p>
<p>In the source code for the pre-trained vgg16 model it seems that the padding is set to 1 <a href="https://pytorch.org/docs/stable/_modules/torchvision/models/vgg.html#vgg16" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torchvision/models/vgg.html#vgg16</a> . So i would like to know if there is a way of altering this after importing the pretrained model?</p>
<p>For example in the code:</p>
<pre><code class="lang-auto">import torchvision.models as models

vgg16 = models.vgg16()
conv1 = vgg16.features[0]

print(conv1)

Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
</code></pre>
<p>I would like to remove the padding, to reduce the effect of artifacts at the edges.</p>
<p>Thanks in advance for any advice.</p>

ptrblck: <p>You could just set the new padding values directly in the module.<br>
While setting them to e.g. 0 would work from the technical point of view, I’m not sure how the model performance would suffer from it.<br>
Here is a small example:</p>
<pre><code class="lang-python">vgg16 = models.vgg16()
x = torch.randn(2, 3, 224, 224)

for module in vgg16.features.modules():
    if isinstance(module, nn.Conv2d):
        module.register_forward_hook(lambda m, input, output: print(output.shape))

# Forward pass with shape outputs
output = vgg16(x)
&gt; torch.Size([2, 64, 224, 224])
torch.Size([2, 64, 224, 224])
torch.Size([2, 128, 112, 112])
torch.Size([2, 128, 112, 112])
torch.Size([2, 256, 56, 56])
torch.Size([2, 256, 56, 56])
torch.Size([2, 256, 56, 56])
torch.Size([2, 512, 28, 28])
torch.Size([2, 512, 28, 28])
torch.Size([2, 512, 28, 28])
torch.Size([2, 512, 14, 14])
torch.Size([2, 512, 14, 14])
torch.Size([2, 512, 14, 14])


for module in vgg16.features.modules():
    if isinstance(module, nn.Conv2d):
        module.padding = (0, 0)

# Forward pass with smaller shapes        
output = vgg16(x)
&gt; torch.Size([2, 64, 222, 222])
torch.Size([2, 64, 220, 220])
torch.Size([2, 128, 108, 108])
torch.Size([2, 128, 106, 106])
torch.Size([2, 256, 51, 51])
torch.Size([2, 256, 49, 49])
torch.Size([2, 256, 47, 47])
torch.Size([2, 512, 21, 21])
torch.Size([2, 512, 19, 19])
torch.Size([2, 512, 17, 17])
torch.Size([2, 512, 6, 6])
torch.Size([2, 512, 4, 4])
torch.Size([2, 512, 2, 2])
</code></pre>
<p>Since the last output activation before the <code>nn.AdaptiveAvgPool2d</code> layer is smaller than the kernel size (<a href="https://github.com/pytorch/vision/blob/b8e3e96910d13bf485aa49ae23e2c77e25ba25f1/torchvision/models/vgg.py#L44">line of code</a>), the values would be repeated a lot:</p>
<pre><code class="lang-python">
x = torch.randn(1, 1, 2, 2)
avgpool = nn.AdaptiveAvgPool2d((7, 7))
out = avgpool(x)

print(x)
&gt; tensor([[[[-0.7769,  0.0930],
          [ 1.1264, -0.6808]]]])
print(out)
&gt; tensor([[[[-0.7769, -0.7769, -0.7769, -0.3419,  0.0930,  0.0930,  0.0930],
          [-0.7769, -0.7769, -0.7769, -0.3419,  0.0930,  0.0930,  0.0930],
          [-0.7769, -0.7769, -0.7769, -0.3419,  0.0930,  0.0930,  0.0930],
          [ 0.1747,  0.1747,  0.1747, -0.0596, -0.2939, -0.2939, -0.2939],
          [ 1.1264,  1.1264,  1.1264,  0.2228, -0.6808, -0.6808, -0.6808],
          [ 1.1264,  1.1264,  1.1264,  0.2228, -0.6808, -0.6808, -0.6808],
          [ 1.1264,  1.1264,  1.1264,  0.2228, -0.6808, -0.6808, -0.6808]]]])
</code></pre>
<p>However, of course you don’t need to set <em>all</em> padding arguments to zeros and thus might avoid this issue. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Ellie_Bowler: <p>Brilliant - thanks very much!</p>