GAN => maping a random noise to an output space!

username: pytoB
cooked: <p>Hi Guys,<br>
I’ve some difficulties grasping the concept; that’s why I need your feedback!</p>
<p>I should map a random noise of shape (100, 1, 1) to an image tensor of shape (3, 64, 64).<br>
The generator to implement consists of 5 transposed convolutional layers:</p>
<h1><a name="p-472702-to-use-is-512-256-128-64-and-3-output-channels-for-these-layers-1" class="anchor" href="#p-472702-to-use-is-512-256-128-64-and-3-output-channels-for-these-layers-1"></a>To use is  512, 256, 128, 64, and 3 output channels for these layers.</h1>
<h1><a name="p-472702-use-a-kernel-size-of-4-2" class="anchor" href="#p-472702-use-a-kernel-size-of-4-2"></a>Use a kernel size of 4.</h1>
<h1><a name="p-472702-use-a-padding-of-0-in-the-first-transposed-convolution-and-a-padding-of-1-for-3" class="anchor" href="#p-472702-use-a-padding-of-0-in-the-first-transposed-convolution-and-a-padding-of-1-for-3"></a>Use a padding of 0 in the first transposed convolution and a padding of 1 for</h1>
<p>all subsequent transposed convolutions.</p>
<h1><a name="p-472702-do-not-use-a-bias-4" class="anchor" href="#p-472702-do-not-use-a-bias-4"></a>Do not use a bias.</h1>
<h1><a name="p-472702-use-a-stride-of-1-for-the-first-transposed-convolution-and-a-stride-of-2-for-all-subsequent-transposed-convolutions-5" class="anchor" href="#p-472702-use-a-stride-of-1-for-the-first-transposed-convolution-and-a-stride-of-2-for-all-subsequent-transposed-convolutions-5"></a>Use a stride of 1 for the first transposed convolution and a stride of 2 for all subsequent transposed convolutions.</h1>
<h1><a name="p-472702-apply-batch-normalization-after-each-transposed-convolutional-layer-except-for-the-last-one-6" class="anchor" href="#p-472702-apply-batch-normalization-after-each-transposed-convolutional-layer-except-for-the-last-one-6"></a>Apply batch normalization after each transposed convolutional layer except for the last one.</h1>
<h1><a name="p-472702-apply-a-relu-activation-function-after-each-batch-normalization-layer-use-the-torchnnrelu-function-for-this-7" class="anchor" href="#p-472702-apply-a-relu-activation-function-after-each-batch-normalization-layer-use-the-torchnnrelu-function-for-this-7"></a>Apply a ReLU activation function after each batch normalization layer. Use the <code>torch.nn.ReLU</code> function for this.</h1>
<h1><a name="p-472702-apply-a-hyperbolic-tangent-function-after-the-last-transposed-convolutional-layer-8" class="anchor" href="#p-472702-apply-a-hyperbolic-tangent-function-after-the-last-transposed-convolutional-layer-8"></a>Apply a hyperbolic tangent function after the last transposed convolutional layer.</h1>
<p>Return the output after this function.</p>
<p>My Implementation:</p>
<p>class Generator(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Generator, self).<strong>init</strong>()<br>
self.main = nn.Sequential(</p>
<pre><code>      #-----------------------------------------------
      #  1.Layer
      #-----------------------------------------------
      nn.ConvTranspose2d(in_channels=100,out_channels=512,
      kernel_size=(4,    4),stride=1,padding=0),
      nn.BatchNorm2d(512),
      nn.ReLU(inplace = True),
        #-----------------------------------------------
        #  2.Layer
        #-----------------------------------------------
     nn.ConvTranspose2d(in_channels=512,out_channels=256,
     kernel_size=(4, 4),stride=2, padding=1),
     nn.BatchNorm2d(256),
     nn.ReLU(inplace = True),
     #-----------------------------------------------
     #. 3.Layer
     #-----------------------------------------------
     nn.ConvTranspose2d(in_channels=256,out_channels=128,
     kernel_size=(4, 4),stride=2,padding=1),
     nn.BatchNorm2d(128),
     nn.ReLU(inplace = True),
     #-----------------------------------------------
     #  4.Layer
     #-----------------------------------------------
     nn.ConvTranspose2d(in_channels=128,out_channels=64,
     kernel_size=(4, 4),stride=2,padding=1),
     nn.BatchNorm2d(64),
     nn.ReLU(inplace = True),

   #-----------------------------------------------
  #  5.Layer
  #-----------------------------------------------
   nn.ConvTranspose2d(in_channels=64,out_channels=3*64*64,
   kernel_size=(4, 4),stride=2,padding=1),
   nn.Tanh()
</code></pre>
<p>)</p>
<p>def forward(self, data):<br>
return self.main(data)</p>
<p>I want to know if the implementation is correct. Thanks</p>

username: Dhia-naouali
cooked: <p>almost there, here is a commented version with the needed slight changes</p>
<pre data-code-wrap="python"><code class="lang-python">import torch 
from torch import nn

class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        

        # using a dict for all layers other than the first one, to reduce redundancy
        # it's the same as using `kernel_size=4, stride=2, padding=1, bias=False` in each layer
        sub_seq_kwargs = {
            "kernel_size": 4,
            "stride": 2,
            "padding": 1,
            "bias": False
        }

        self.main = nn.Sequential(
            # layer 1
            # by default pytorch include a bias for conv layers, so we'll need to set that to False
            nn.ConvTranspose2d(100, 512, kernel_size=4, stride=1, padding=0, bias=False), # =&gt; (bs, 512, 4, 4)
            nn.BatchNorm2d(512),
            nn.ReLU(inplace=True),

            # layer 2
            nn.ConvTranspose2d(512, 256, **sub_seq_kwargs), # =&gt; (bs, 256, 8, 8)
            nn.BatchNorm2d(256),
            nn.ReLU(inplace=True),

            # layer 3
            nn.ConvTranspose2d(256, 128, **sub_seq_kwargs), # =&gt; (bs, 128, 16, 16)
            nn.BatchNorm2d(128),
            nn.ReLU(inplace=True),

            # layer 4
            nn.ConvTranspose2d(128, 64, **sub_seq_kwargs), # =&gt; (bs, 64, 32, 32)
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),

            # layer 5
            # in_channels &amp; out_channels only affect the C dim of your tensor
            # so to get a (bs, 3, 64, 64) output we can use out_channels=3
            # spatial dims (height &amp; width) are augmented by the stride &amp; padding of the transposed conv layer
            nn.ConvTranspose2d(64, 3, **sub_seq_kwargs), # =&gt; (bs, 3, 64, 64)
            nn.Tanh()
        )

    def forward(self, data):
        return self.main(data)


g = Generator()
print(g(torch.randn(1, 100, 1, 1)).size())
</code></pre>

username: pytoB
cooked: <p>Many Thanks for the prompt Feedback!</p>