Convolutional autoencoder, how to precisely decode (ConvTranspose2d)

witl: <p>I’m trying to code a simple convolution autoencoder for the digit MNIST dataset. My plan is to use it as a denoising autoencoder.</p>
<p>I’m trying to replicate an architecture proposed in a paper. The network architecture looks like this:</p>
<div class="md-table">
<table>
<thead>
<tr>
<th>Network</th>
<th>Layer</th>
<th>Activation</th>
</tr>
</thead>
<tbody>
<tr>
<td>Encoder</td>
<td>Convolution</td>
<td>Relu</td>
</tr>
<tr>
<td>Encoder</td>
<td>Max Pooling</td>
<td>-</td>
</tr>
<tr>
<td>Encoder</td>
<td>Convolution</td>
<td>Relu</td>
</tr>
<tr>
<td>Encoder</td>
<td>Max Pooling</td>
<td>-</td>
</tr>
<tr>
<td>----</td>
<td>----</td>
<td>----</td>
</tr>
<tr>
<td>Decoder</td>
<td>Convolution</td>
<td>Relu</td>
</tr>
<tr>
<td>Decoder</td>
<td>Upsampling</td>
<td>-</td>
</tr>
<tr>
<td>Decoder</td>
<td>Convolution</td>
<td>Relu</td>
</tr>
<tr>
<td>Decoder</td>
<td>Upsampling</td>
<td>-</td>
</tr>
<tr>
<td>Decoder</td>
<td>Convolution</td>
<td>Sigmoid</td>
</tr>
</tbody>
</table>
</div><p>Here is the code I have so far, I never performed “deconvolution” so I’m a bit lost.</p>
<pre><code class="lang-auto"># Conv network
self.convEncoder = nn.Sequential(
	# Output size of each convolutional layer = [(in_channel + 2 * padding - kernel_size) / stride] + 1
	# In this case output = [(28 + 2 * 1 - 5) / 1] + 1 = 26
	nn.Conv2d(in_channels=1, out_channels=10, kernel_size=5, padding=1, stride=1),
	nn.ReLU(),
	nn.MaxPool2d(kernel_size=2),    # End up with 12 channels of size 13 x 13

	# In this case output = [(13 + 2 * 1 - 5) / 1] + 1 = 11
	nn.Conv2d(in_channels=10, out_channels=24, kernel_size=5, padding=1, stride=1),
	nn.ReLU(),
	nn.MaxPool2d(kernel_size=2),  # End up with 24 channels of size 5 x 5
)

# Not sure what to do here
self.convDecoder = nn.Sequential(
	# ???? 
	nn.ConvTranspose2d(??, ??, ??),	# 24, 10, 5 ???
	nn.ReLU(),
	nn.Upsample(scale_factor=??)	# ???

	nn.ConvTranspose2d(??, ??, ??),	# 10, 1, 5 ???
	nn.ReLU(),
	nn.Upsample(scale_factor=??)	# ???

	nn.ConvTranspose2d(??, ??, ??),	# ???
	nn.Sigmoid()
)
</code></pre>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> did you ever face this situation?</p>

ptrblck: <p>I haven’t written an autoencoder using your structure and assume you are wondering which setup to use in the transposed convolutions?<br>
If so, you could start by “inverting” the encoder path and use the inverse channel dimensions. The kernel size, stride etc. should most likely be set in a way to reproduce the input spatial size.<br>
If you don’t want to calculate it manually, add <code>Print</code> layers to the model to check the output activation shape and adapt the setup:</p>
<pre><code class="lang-python">class Print(nn.Module):
    def __init__(self):
        super(Print, self).__init__()
        
    def forward(self, x):
        print(x.shape)
        return x
</code></pre>

witl: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> ty for your reply.</p>
<p>In the paper I’m reading, they show the following architecture<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68661c38ef9ae454cd051d64429662aa18e4c620.png" alt="AcroRd32_joTWxb6NBl" data-base62-sha1="eTyqtPtbz4APViqNkJk6UFgI0rm" width="539" height="378"></p>
<p>Do you think by upsampling they are saying that they’re actually using an upsampling function such as <code>nn.Upsample</code> or they are just using ConvTranspose2d and playing with the stride?</p>
<p>I mean, we can achieve the same downsampling without Max Pooling by playing with stride too right? But if they mention it on the architecture, it means they’re applying it right? What would you say given your experience?</p>
<p>I guess the best would be to contact the author to get more informations.</p>

ptrblck: <p>I would guess that <code>UpSampling</code> would refer to an <code>nn.Upsample</code> layer, but also a transposed conv could be used. However, I <em>think</em> the former might be more likely.<br>
Do you see any mentions on the number of parameters? If the authors claim the <code>UpSampling</code> layers don’t use any trainable parameters, it should be an interpolation layer (i.e. <code>nn.Upsample</code>), on the other hand if these layers use parameters it would point towards a transposed conv.</p>