Is asymmetric padding of style 'same' available in pytorch?

username: deartonym
cooked: <p>Problem as the title:</p>
<p>When the kernel size is even, we may need asymmetric padding. In Tensorflow we have a padding setting called ‘SAME’, which will make a padding with asymmetric padded pixels to different side of tensor to make the output size matches the input size.</p>
<p>Is there any such setting available in PyTorch?</p>
<p>Thanks.</p>

username: santi-pdp
cooked: <p>I think there is no such thing as ‘SAME’ or ‘VALID’ as in TF/Keras when defining your convolution layer, instead you define your own padding with a tuple, as stated in the docs <code>padding (int or tuple, optional) – Zero-padding added to both sides of the input</code> for <code>torch.nn.Conv2d</code>.</p>

username: santi-pdp
cooked: <p>Now that I realize, <code>padding</code> only accepts symmetric padding in width and height (<code>a tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension</code>). Nonetheless you have many padding layers <code>http://pytorch.org/docs/nn.html?highlight=padding#padding-layers</code> for which you can indeed apply different paddings with 4-tuples.</p>

username: deartonym
cooked: <p>thanks for the hint. I will try to use padding layers before making convolutions or transposed convolutions. Hope that would solve my problem.</p>
<p>Thanks again.</p>

username: ThibaultGROUEIX
cooked: <p>I’m also interested in that topic. I think by combining asymmetric padding and conv2D, one can mimic ‘SAME’ in tensorflow for  <code>tflearn.layers.conv.conv_2d</code> and  <code>tflearn.layers.conv.conv_2d_transpose</code> of stride 1. But I couldn’t find a way to translate   <code>tflearn.layers.conv.conv_2d_transpose</code> with asymmetric padding and stride &gt; 1.<br>
After looking at “Padding, strides, transposed (odd)” from <a href="https://github.com/vdumoulin/conv_arithmetic" rel="nofollow noopener">https://github.com/vdumoulin/conv_arithmetic</a>, I’m convinced that you can’t reproduce   <code>tflearn.layers.conv.conv_2d_transpose</code> with asymmetric padding and stride &gt; 1 using a cascade of  asymmetric padding and convolutions in pytorch.<br>
Anyone has a solution for this ?<br>
Toy exemple :<br>
Trying to translate to pytorch : <code>tflearn.layers.conv.conv_2d_transpose(x, nb_filter = 1, filter_size = [5, 5] , output_shape = [6,8], strides = 2, activation = 'linear')</code>  with <code>x.size = (1, 1, 3, 4) = (BATCH, INPUT_FILTERS, HEIGHT, WIDTH).</code></p>

username: Prabhat-IIT
cooked: <p>Refer to <a href="https://stackoverflow.com/questions/50683039/conv2d-transpose-output-shape-using-formula" rel="nofollow noopener">this</a> answer and calculate the padding needed for conv2D transpose of pytorch. It’ll enable you to convert between <code>tflearn.layers.conv.conv_2d_transpose</code> with non-unit stride to pytorch version of conv2D</p>

username: duang
cooked: <p>Refer to <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.ZeroPad2d" rel="nofollow noopener">torch.nn.ZeroPad</a>, the asymmetric padding is now available by passing a tuple to the Pad function.<br>
For example:</p>
<pre><code class="lang-python">&gt;&gt;&gt; input = torch.randn(1, 1, 3, 3)
&gt;&gt;&gt; input
tensor([[[[-1.7800,  0.6112, -0.0166],
          [-2.1496, -0.5789,  0.8997],
          [-0.5621,  0.9050,  0.4039]]]])
&gt;&gt;&gt; m = torch.nn.ZeroPad2d((1, 2, 1, 2))
&gt;&gt;&gt; m
tensor([[[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000, -1.7800,  0.6112, -0.0166,  0.0000,  0.0000],
          [ 0.0000, -2.1496, -0.5789,  0.8997,  0.0000,  0.0000],
          [ 0.0000, -0.5621,  0.9050,  0.4039,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]])
</code></pre>

username: h-shad
cooked: <p>But it’s not without problems. According to <a href="https://github.com/pytorch/pytorch/issues/4939" rel="nofollow noopener">here</a>:<br>
The padding in Conv2d layer is implemented as implicit padding, i.e. the convolution kernel itself is assuming that the given input is padded and doing the computation. There is no extra memory taken by the operation because of the padding value.</p>
<p>The F.pad layer does padding more explicitly, i.e. each input is padded and an output Tensor is returned. This involves extra memory.</p>

username: pkmandke
cooked: <p>AFAIK, it’s best not to use the functional API(F.pad) for adding layers, especially those that mutate the shape of their input. The layers won’t show up in the model description and will create a problem in inferring how the shape of the feature maps evolve as the network progresses.</p>
<p>torch.nn layers make life easier.(Refer <a class="mention" href="/u/duang">@duang</a>’s example <a href="https://discuss.pytorch.org/t/is-asymmetric-padding-of-style-same-available-in-pytorch/4354/7">above</a>).</p>