Question of 2D transpose Convolution

Tomojit: <p>Hi - I was experimenting with ConvTranspose2d operation. I took a [2 x 2] random tensor and applied transpose conv on it with and without padding. Both the kernel size and stride are set to 2. When I checked the size of the tensor after the operation I found that the size of the output tensor without padding is bigger than with padding.</p>
<p>Code snippet with padding:<br>
d=torch.randn(1,1,2,2)<br>
deconv2=nn.ConvTranspose2d(1,1,kernel_size=2,stride=2,padding=1)<br>
d1=deconv2(d)<br>
d1.shape ==&gt;&gt; it returns torch.Size([1, 1, 2, 2])</p>
<p>Code snippet without padding:<br>
deconv2=nn.ConvTranspose2d(1,1,kernel_size=2,stride=2,padding=0)<br>
d1=deconv2(d)<br>
d1.shape ==&gt;&gt; it returns torch.Size([1, 1, 4, 4])</p>
<p>I thought that the output tensor size with padding will be bigger than without padding.But it’s not the   case. Can anyone please explain this?</p>
<p>Regards,<br>
Tomojit</p>

ptrblck: <p>This should be expected.<br>
Form the <a href="https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html#torch.nn.ConvTranspose2d">docs</a>:</p>
<blockquote>
<p>The  <code>padding</code>  argument effectively adds  <code>dilation * (kernel_size - 1) - padding</code>  amount of zero padding to both sizes of the input. This is set so that when a <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d"> <code>Conv2d</code> </a> and a <a href="https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html#torch.nn.ConvTranspose2d"> <code>ConvTranspose2d</code> </a> are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when  <code>stride &gt; 1</code> , <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d"> <code>Conv2d</code> </a> maps multiple input shapes to the same output shape.  <code>output_padding</code>  is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that  <code>output_padding</code>  is only used to find output shape, but does not actually add zero-padding to output.</p>
</blockquote>
<p>Also, have a look at the <a href="http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html#transposed-convolution-arithmetic">conv arithmetic tutorial</a> for more information.</p>

Tomojit: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> - Thanks for sharing the “conv arithmmetic” tutorial. I checked this tutorial before as well. I am still confused on padding. Consider the first case when I used padding=1. Note I didn’t use dilation. So, according to the PyTorch document (“dilation * (kernel_size - 1) - padding”) which equals to (0*(2-1)-1) = -1 amount of padding will be added. This does not make any sense to me. Am I doing any mistake?</p>

Tomojit: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> - I also observe that when the stride is &gt; 1 (say 2) the transpose Conv can’t reconstruct the original image size. But if I use unit stride then transpose Conv reconstructs the exact image size. See below:<br>
Code snippet for perfect reconstruction:<br>
In [1]: import torch<br>
In [2]: D=torch.randn(1,1,28,28)<br>
In [3]: import torch.nn as nn<br>
In [4]: s,k,p=1,5,0<br>
In [5]: conv1 = nn.Conv2d(1,1,kernel_size=k,stride=s,padding=p)<br>
In [6]: deconv1 = nn.ConvTranspose2d(1,1,kernel_size=k,stride=s,padding=p)<br>
In [7]: D1=conv1(D)<br>
In [8]: D1_t = deconv1(D1)<br>
In [10]: D1_t.shape<br>
Out[10]: torch.Size([1, 1, 28, 28])==&gt; size of the reconstructed image</p>
<p>Code snippet when reconstruction is not perfect when I use stride=2.<br>
In [11]: s,k,p=2,5,0<br>
In [12]: conv1 = nn.Conv2d(1,1,kernel_size=k,stride=s,padding=p)<br>
In [13]: deconv1 = nn.ConvTranspose2d(1,1,kernel_size=k,stride=s,padding=p)<br>
In [14]: D1 = conv1(D)<br>
In [15]: D1_t = deconv1(D1)<br>
In [16]: D.shape<br>
Out[16]: torch.Size([1, 1, 28, 28])<br>
In [17]: D1_t.shape<br>
Out[17]: torch.Size([1, 1, 27, 27]) ==&gt; size of the reconstructed image</p>

vijay_srinivas_tida: <p>Hi<br>
I want to use how the transpose convolution implemented in general for Generative Adversarial Networks using PyTorch framework. For example <a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">DCGAN Tutorial — PyTorch Tutorials 1.11.0+cu102 documentation</a> the code taken from here. Is transpose convolution a combination of upsampling layer and convolution layer used or any other approach I really appreciate your help.</p>
<p>Thanks,<br>
Vijay</p>

ptrblck: <p>The <code>Generator</code> uses <code>nn.ConvTranspose2d</code> layers so directly transposed convolutions (you can think about them as the “reversed” conv layers i.e. the forward pass of a transposed conv equals the backward of a vanialla conv layer and vice versa). For more information about these layers, check <a href="https://github.com/vdumoulin/conv_arithmetic">this repo</a>.</p>

vijay_srinivas_tida: <p>Hi,</p>
<p>Thank you very much for your quick response. So there won’t be any upsampling layer before applying convolution for this implementation? I thought transpose convolution = upsample layer+convolution layer from the paper <a href="https://arxiv.org/pdf/1806.01107.pdf" rel="noopener nofollow ugc">https://arxiv.org/pdf/1806.01107.pdf</a> fig 4.</p>

ptrblck: <p>I’m not familiar with this paper and don’t know if the authors call an upsampling + conv block a “transposed convolution”. You can see in the model architecture of the <a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html#generator">tutorial</a> that <code>nn.ConvTranspose2d</code> is directly used and please check the posed link to see how this layer is working and increasing the spatial size of the input activation.</p>

vijay_srinivas_tida: <p>Hi,<br>
Thanks.I’m working on optimization of transpose convolution layer by avoiding the increasing input spatial size  before applying convolution and to produce the same output. May I please know how the backend code works?</p>