torch.nn.ConvTranspose2d vs torch.nn.Upsample

Neda: <p>What is the difference between ConvTranspose2d  and Upsample in Pytorch?<br>
To implement UNet in Pytorch based on the model <a href="https://arxiv.org/pdf/1505.04597.pdf" rel="nofollow noopener">in this paper</a> for the first upsampling layer some people used</p>
<pre><code class="lang-auto">self.upSample1 = nn.Upsample(size=(1024, 1024), scale_factor=(2, 2), mode="bilinear")

        self.up1 = nn.Sequential(
            ConvRelu2d(1024, 512, kernel_size=(3, 3), stride=1, padding=0),
            ConvRelu2d(512, 512, kernel_size=(3, 3), stride=1, padding=0)
        )
</code></pre>
<p>while some people used</p>
<pre><code class="lang-auto">        self.up = nn.ConvTranspose2d(in_size, out_size, 2, stride=2)
        self.conv = nn.Conv2d(in_size, out_size, kernel_size)
        self.conv2 = nn.Conv2d(out_size, out_size, kernel_size)
        self.activation = F.relu
</code></pre>
<p>I am confused is both Upsample and ConvTranspose2d  do the same things?</p>

justusschock: <p>No they don’t.<br>
<code>ConvTranspose</code> is a convolution and has trainable kernels while <code>Upsample</code> is a simple interpolation (bilinear, nearest etc.)</p>

Neda: <p><a class="mention" href="/u/justusschock">@justusschock</a> you mean ConvTranspose2d is deconvolution?</p>

justusschock: <p>Yes if you want to call it like that.<br>
It is not really an inverse operation though, but the same operation with another kernel.</p>

John1231983: <aside class="quote no-group" data-username="justusschock" data-post="2" data-topic="30574">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/justusschock/48/4637_2.png" class="avatar"> justusschock:</div>
<blockquote>
<p><code>Transpose</code> is a convolution and has trainable kernels while <code>Upsample</code> is a simple interpolation (bilinear, nearest etc.)</p>
</blockquote>
</aside>
<p>Transpose is learning parameter while Up-sampling is no-learning parameters. Using Up-samling for faster inference or training because it does not require to update weight or compute gradient</p>

singhvishal0209: <p>So according to you in the Unet paper what did the author used for Upsampling, nn.Upsample or ConvTranspose2d?<br>
and can you also explain which one to use among above 2 in what condition</p>

vdasu: <p>In the Unet paper, they’ve stated that “up-convolutions” were used. I would assume that this means they would have implemented/used <code>ConvTranspose2d</code>.</p>
<p>As for which one to use, it really depends on the network you’re designing. If you are sure of the kind of upsampling that needs to be done (bilinear, etc.) then you can use <code>nn.Upsample</code>. However, if you think it would be important to “learn” how to upsample instead of just using a hardcoded method then the trainable parametes in <code>ConvTranspose2d</code> would be useful.</p>

zaccharieramzi: <p>Actually, in the U-net paper, they specify what they mean by up-convolutions:</p>
<blockquote>
<p>Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 convolution (“up-convolution”)</p>
</blockquote>
<p>Some of the recent architectures, like <a href="https://github.com/ajbrock/BigGAN-PyTorch/blob/master/BigGAN.py#L165" rel="noopener nofollow ugc">BigGAN</a>, use this scheme to avoid the <a href="https://distill.pub/2016/deconv-checkerboard/" rel="noopener nofollow ugc">cherckerboard artifacts</a>.</p>