Increasing DCGAN convolutional layer filter size problem

username: asberman
cooked: <p>I’m trying to implement a DCGAN, with heavy inspiration from <a href="https://github.com/pytorch/examples/tree/master/dcgan" rel="nofollow noopener">DCGAN example from the Pytorch github</a> and a dataset of 64x64 images, but I’ve been unable to increase the filter sizes of the convolutional layers in the discriminator and generator. The implementation works with a max filter value of 512 (e.g. Conv2d(256,512,kernel_size, stride), but I have had no luck trying to increase the filter value to a max of 1024.</p>
<p>The above implementation’s discriminator is as follows (which I have used in my implementation):</p>
<pre><code class="lang-auto">nc = 3
ndf = 64
  self.main = nn.Sequential(
            # input is (nc) x 64 x 64
            nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf) x 32 x 32
            nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*2) x 16 x 16
            nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*4) x 8 x 8
            nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 8),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*8) x 4 x 4
            nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()
        )
</code></pre>
<p>I want to change the model such that the last layer is:<br>
<code>nn.Conv2d(ndf*16,1,4,1,0,bias=False)</code></p>
<p>so that the final layer has 1024 filters, as opposed to 512 (and similarly for the generator). However, when I implement this, changing only that and keeping the kernel size, channels (nc) etc. constant, I get</p>
<p><code>RuntimeError: Calculated input size: (2 x 2). Kernel size: (4 x 4). Kernel size can't greater than actual input size</code></p>
<p>How do I construct a discriminator with a final convolutional layer that has 1024 output filters (and the equivalent convolutional transpose layer in the generator with 1024 input filters)? I imagine I should be able to do this without changing the inputs (batch of 64 images that are 64px by 64px).</p>
<p>Any help would be greatly appreciated.</p>

username: ptrblck
cooked: <p>The first argument to <code>Conv2d</code> are the <code>in_channels</code>.<br>
From the docs:</p>
<blockquote>
<p>Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)</p>
</blockquote>
<p>The last layer in the discriminator has therefore <strong>one</strong> output channel and only one “pixel” (the shape before the last layer is <code>[batch_size, ndf*8, 4, 4]</code>, so that a <code>4x4</code> kernel returns only one value).</p>
<p>If you want to change the number of filters/channels try:</p>
<pre><code class="lang-auto">... 
nn.Conv2d(ndf * 4, ndf * 16, 4, 2, 1, bias=False),
nn.BatchNorm2d(ndf * 16),
nn.LeakyReLU(0.2, inplace=True),
# state size. (ndf*16) x 4 x 4
nn.Conv2d(ndf * 16, 1, 4, 1, 0, bias=False),
nn.Sigmoid()
</code></pre>

username: asberman
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> so much! I thought that out_channels ought to be 2 * n_channels (and couldn’t be 4*in_channels) so I hadn’t considered that. Does this mean in general out_channels need only be some scalar multiple of in_channels?</p>
<p>Changed my discriminator and generator as per your suggestions and it works!<br>
Really appreciate the quick, clear and super helpful reply.</p>
<p>Thanks again!</p>

username: ptrblck
cooked: <p>You are welcome!<br>
<code>out_channels</code> defines the number of kernels and as long as you use “vanilla” convolutions, i.e. without grouping, you can use any number you want.</p>
<p>Remember that each filter uses all input channels. Grouping changes this behavior.</p>

username: Marion2
cooked: <p>I tried to follow your advices,</p>
<p>I changed the Discriminator without any problem<br>
<img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji only-emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>class Discriminator(nn.Module):<br>
def <strong>init</strong>(self, ngpu):<br>
super(Discriminator, self).<strong>init</strong>()<br>
self.ngpu = ngpu<br>
self.main = nn.Sequential(<br>
# input is (nc) x 64 x 64<br>
nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),<br>
nn.LeakyReLU(0.2, inplace=True),<br>
# state size. (ndf) x 32 x 32<br>
nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ndf * 2),<br>
nn.LeakyReLU(0.2, inplace=True),<br>
# state size. (ndf<em>2) x 16 x 16<br>
nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ndf * 4),<br>
nn.LeakyReLU(0.2, inplace=True),<br>
# state size. (ndf</em>4) x 8 x 8<br>
nn.Conv2d(ndf * 4, ndf * 16, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ndf * 16),<br>
nn.LeakyReLU(0.2, inplace=True),<br>
# state size. (ndf*16) x 4 x 4<br>
nn.Conv2d(ndf * 16, 1, 4, 1, 0, bias=False),<br>
nn.Sigmoid()<br>
)</p>
<p>and I tried to change the Generator like that :</p>
<p>class Generator(nn.Module):<br>
def <strong>init</strong>(self, ngpu):<br>
super(Generator, self).<strong>init</strong>()<br>
self.ngpu = ngpu<br>
self.main = nn.Sequential(<br>
# input is Z, going into a convolution<br>
nn.ConvTranspose2d( nz, ngf * 8, 4, 1, 0, bias=False),<br>
nn.BatchNorm2d(ngf * 8),<br>
nn.ReLU(True),<br>
# state size. (ngf<em>8) x 4 x 4<br>
nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ngf * 4),<br>
nn.ReLU(True),<br>
# state size. (ngf</em>4) x 8 x 8<br>
nn.ConvTranspose2d( ngf * 4, ngf * 2, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ngf * 2),<br>
nn.ReLU(True),<br>
# state size. (ngf*2) x 16 x 16<br>
nn.ConvTranspose2d( ngf * 4, ngf * 16, 4, 2, 1, bias=False),<br>
nn.BatchNorm2d(ngf * 16),<br>
nn.ReLU(True),<br>
# state size. (ngf) x 32 x 32<br>
nn.ConvTranspose2d( ngf * 16, 1, 4, 1, 0, bias=False),<br>
nn.Tanh()<br>
# state size. (nc) x 64 x 64<br>
)</p>
<p>But when I run the training code I have this error :</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/50e019fc9ba38d0f7939022650eecfe61e35417e.png" data-download-href="https://discuss.pytorch.org/uploads/default/50e019fc9ba38d0f7939022650eecfe61e35417e" title="09"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/50e019fc9ba38d0f7939022650eecfe61e35417e_2_690x201.png" alt="09" data-base62-sha1="bxskDHnjpeZa2bVhILaoKmVchGu" width="690" height="201" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/50e019fc9ba38d0f7939022650eecfe61e35417e_2_690x201.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/50e019fc9ba38d0f7939022650eecfe61e35417e_2_1035x301.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/50e019fc9ba38d0f7939022650eecfe61e35417e_2_1380x402.png 2x" data-dominant-color="F8F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">09</span><span class="informations">2618×766 228 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, should I change the image_size from 64 to 1024 ?</p>
<p>I’d love to have some help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>It looks like the input channels of an <code>nn.ConvTranspose2d</code> layer are set to a wrong value.</p>
<pre><code class="lang-python">class Generator(nn.Module):
    def __init__(self, ngpu):
        super(Generator, self).__init__()
        self.ngpu = ngpu
        self.main = nn.Sequential(
        # input is Z, going into a convolution
        nn.ConvTranspose2d( nz, ngf * 8, 4, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 8),
        nn.ReLU(True),
        # state size. (ngf8) x 4 x 4
        nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True),
        # state size. (ngf4) x 8 x 8
        nn.ConvTranspose2d( ngf * 4, ngf * 2, 4, 2, 1, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True),
        # state size. (ngf*2) x 16 x 16
        nn.ConvTranspose2d( ngf * 2, ngf * 16, 4, 2, 1, bias=False), # !!! Change in_channels to ngf*2
        nn.BatchNorm2d(ngf * 16),
        nn.ReLU(True),
        # state size. (ngf) x 32 x 32
        nn.ConvTranspose2d( ngf * 16, 1, 4, 1, 0, bias=False),
        nn.Tanh()
        # state size. (nc) x 64 x 64
        )

    def forward(self, x):
        x = self.main(x)
</code></pre>
<p>See the comment where to change it.</p>

username: Marion2
cooked: <p>Thanks a lot for your answer !</p>
<p>I’ve done what you said but now I have this error when I start the training loop :</p>
<blockquote>
<p><strong>Starting Training Loop…</strong><br>
<strong>---------------------------------------------------------------------------</strong><br>
<strong>RuntimeError                              Traceback (most recent call last)</strong><br>
<strong> in ()</strong><br>
**     31         noise = torch.randn(b_size, nz, 1, 1, device=device)**<br>
**     32         # Generate fake image batch with G**<br>
<strong>—&gt; 33         fake = netG(noise)</strong><br>
**     34         label.fill_(fake_label)**<br>
**     35         # Classify all fake batch with D**</p>
<p><strong>6 frames</strong><br>
<strong>/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)</strong><br>
**   1695     return torch.batch_norm(**<br>
**   1696         input, weight, bias, running_mean, running_var,**<br>
<strong> → 1697         training, momentum, eps, torch.backends.cudnn.enabled</strong><br>
**   1698     )**<br>
**   1699 **</p>
<p><strong>RuntimeError: running_mean should contain 128 elements not 1024</strong></p>
</blockquote>
<p>Do you advise me to change some number here ?</p>
<blockquote>
<p><strong># Number of workers for dataloader</strong><br>
<strong>workers = 2</strong></p>
<p><strong># Batch size during training</strong><br>
<strong>batch_size = 128</strong></p>
<p><strong># Spatial size of training images. All images will be resized to this</strong><br>
<strong>#   size using a transformer.</strong><br>
<strong>image_size = 64</strong></p>
<p><strong># Number of channels in the training images. For color images this is 3</strong><br>
<strong>nc = 3</strong></p>
<p><strong># Size of z latent vector (i.e. size of generator input)</strong><br>
<strong>nz = 100</strong></p>
<p><strong># Size of feature maps in generator</strong><br>
<strong>ngf = 64</strong></p>
<p><strong># Size of feature maps in discriminator</strong><br>
<strong>ndf = 64</strong></p>
<p><strong># Number of training epochs</strong><br>
<strong>num_epochs = 5</strong></p>
<p><strong># Learning rate for optimizers</strong><br>
<strong>lr = 0.0002</strong></p>
<p><strong># Beta1 hyperparam for Adam optimizers</strong><br>
<strong>beta1 = 0.5</strong></p>
<p><strong># Number of GPUs available. Use 0 for CPU mode.</strong><br>
<strong>ngpu = 1</strong></p>
</blockquote>
<p>I tried to change “image_size” from 64 to 554 and 1024 but this didn’t worked. I also tried to change batch_size from 128 to 1024.</p>
<p>Thanks again for your help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>It seems some batchnorm layer in your generator does not use the same number of input channels as the output channels of the preceding conv layer.<br>
E.g.:</p>
<pre><code class="lang-python">...
nn.ConvTranspose2d( ngf * 2, ngf * 16, 4, 2, 1, bias=False),
nn.BatchNorm2d(ngf * 16),
...
</code></pre>
<p>Here you can see that the <code>nn.BatchNorm2d</code> takes <code>ngf*16</code> input channels, which corresponds to the same number of output channels of <code>nn.ConvTranspose2d</code>.<br>
Could you check these numbers?</p>

username: Marion2
cooked: <p>Thank you so much for your help, I changed that and now it works <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>