Padding_mode not working correctly in Conv2d

paarandika: <p>It seems like all three options for padding_mode parameter: <code>'zeros'</code> ,  <code>'reflect'</code> ,  <code>'replicate'</code> output same 0 paddings. Only <code>'circular'</code> outputs the padding its name suggests. I have used the following code to test this.</p>
<pre><code class="lang-auto">import torch.nn as nn
from PIL import Image
import matplotlib.pyplot as plt
import torchvision.utils as utils
import numpy as np

def imshow(images):
    img = utils.make_grid(images)
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()

for images, labels, names in loader_eval:
    conv1 = nn.Conv2d(1, 1, kernel_size=1, stride=1, padding=10, padding_mode='replicate')
    conv1.state_dict()['weight'].copy_(torch.FloatTensor([[[[1.0]]]]))
    conv1.state_dict()['bias'].copy_(torch.FloatTensor([0.0]))
    img = conv1(images)
    imshow(img.detach())
    break
</code></pre>
<p>Am I doing something wrong or is there a bug in the implementation?<br>
Thanks.</p>

Nikronic: <p>Hi,</p>
<p>I ran your code and everything works just fine. Could you please share the exact input and output that produces wrong padding?</p>
<p>I changed your visualization method and this may help to depict the cases better.</p>
<pre><code class="lang-auto">import torch.nn as nn
from PIL import Image
import seaborn as sn
import pandas as pd
import matplotlib.pyplot as plt
import torchvision.utils as utils
import numpy as np

def imshow(img):
    npimg = img.numpy()
    # npimg = np.transpose(npimg, (1, 2, 0))
    df_cm = pd.DataFrame(npimg[0, 0])
    plt.figure(figsize = (10,7))
    sn.heatmap(df_cm, annot=True)

# I have considered that you have a batch of 1 grayscale image.
img = torch.arange(0, 100, dtype=torch.float32).view(1, 1, 10, 10)
conv1 = nn.Conv2d(1, 1, kernel_size=1, stride=1, padding=(2,2), padding_mode='replicate')
conv1.state_dict()['weight'].copy_(torch.FloatTensor([[[[1.0]]]]))
conv1.state_dict()['bias'].copy_(torch.FloatTensor([0.0]))
img2 = conv1(img)
imshow(img2.detach())
</code></pre>
<p>bests</p>

paarandika: <p>Hi,<br>
Thanks for the reply. I have ran your code and here are the ouputs:<br>
<code>reflect</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4.png" data-download-href="https://discuss.pytorch.org/uploads/default/fdb8fca8560958b230bcd05944600a9000f11ee4" title="reflect"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_517x361.png" alt="reflect" data-base62-sha1="Acx8ebt1vMPpNP5fWgNsKu2sSW0" width="517" height="361" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_517x361.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_775x541.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4.png 2x" data-dominant-color="A98A94"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">reflect</span><span class="informations">1000×700 48.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<code>replicate</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4.png" data-download-href="https://discuss.pytorch.org/uploads/default/fdb8fca8560958b230bcd05944600a9000f11ee4" title="reflect"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_517x361.png" alt="reflect" data-base62-sha1="Acx8ebt1vMPpNP5fWgNsKu2sSW0" width="517" height="361" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_517x361.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4_2_775x541.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdb8fca8560958b230bcd05944600a9000f11ee4.png 2x" data-dominant-color="A98A94"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">reflect</span><span class="informations">1000×700 48.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<code>circular</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/2/120576a82ae16ac00ec2e680fba021f1deceafc0.png" data-download-href="https://discuss.pytorch.org/uploads/default/120576a82ae16ac00ec2e680fba021f1deceafc0" title="circular"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/120576a82ae16ac00ec2e680fba021f1deceafc0_2_517x361.png" alt="circular" data-base62-sha1="2zqiB5lEJByPowAC2bGUm0I9Ixq" width="517" height="361" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/120576a82ae16ac00ec2e680fba021f1deceafc0_2_517x361.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/120576a82ae16ac00ec2e680fba021f1deceafc0_2_775x541.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/2/120576a82ae16ac00ec2e680fba021f1deceafc0.png 2x" data-dominant-color="CC9BA3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">circular</span><span class="informations">1000×700 49.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thanks.</p>

paarandika: <p>Hi,<br>
I think I found the issue. My Pytorch version is 1.4.0 and I was refering to 1.5.0 docs.<br>
Thanks again for the help.</p>

Nikronic: <p>Ow, that is possible, I lost the track of versions.<br>
I think I have to mention that libraries like PyTorch are being consistently updated and there are thousands of issues and pull requests on Github. So, I think the best way to keep our codes stable and more reliable (even faster and more optimized) is to update to latest stable version.</p>
<p>Good luck</p>