A question about Conditional GAN

username: Federico_Ottomano
cooked: <p>Hello, I’m trying to implement a simple conditional GAN and wondering if what I’ve done is correct.</p>
<p>As far as I’ve understood, a <em>conditional</em> GAN is based on a simple architectural modification of the <em>base</em> GAN where we concatenate a suitable target vector of properties, or labels (so we end up performing a sort of <em>semi-supervised</em> training).</p>
<p>Currently, my model made by generator and discriminator looks like this:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
from utils import data_generator, get_data_loaders
import pandas as pd


class Discriminator(nn.Module):
    def __init__(self, in_features):
        
        super().__init__()
        
        self.disc = nn.Sequential(
            nn.Linear(in_features, 128),
            nn.LeakyReLU(0.01),
            nn.Linear(128, 1),
            nn.Sigmoid(),
        )

    def forward(self, x, y):
        
        input_ = torch.cat([x,y],dim=1)
        
        return self.disc(input_)
    
class Generator(nn.Module):
    def __init__(self, z_dim, comp_dim):
        super().__init__()
        self.gen = nn.Sequential(
            nn.Linear(z_dim, 256),
            nn.LeakyReLU(0.01),
            nn.Linear(256, comp_dim),
            nn.Sigmoid(),  
        )

    def forward(self, x,y):
        input_ = torch.cat([x,y],dim=1)
        return self.gen(input_)
</code></pre>
<p>Do you think that this looks correct?</p>
<p>I’m wondering because I’ve seen people constructing a <code>nn.Embedding()</code> starting from the vector that we are trying to condition on, like in <a href="https://www.youtube.com/watch?v=Hp-jWm2SzR8&amp;t=399s" rel="noopener nofollow ugc">this tutorial</a> I’m following. I don’t really understand why is this the case. In my situation, for example, I have a vector of target properties that I wouldn’t convert into an embedding.</p>

username: Dazitu616
cooked: <p>If your <code>x</code> is just a <code>1d</code> vector and can be processed by <code>Linear</code>, simply concating it with label <code>y</code> is completely fine. But in the video you refer to, their input is an image, with height x width dimensions. So you cannot concat a 1d label to it (of course some people tile the 1d label to height x width, though I didn’t see much). Instead, they use learnable embeddings to convert the label to height x width shape tensors, and can be concated with input image <code>x</code></p>
<p>This is really a case-by-case problem. Sometimes people find out directly concating the 1d label is not optimal, while making it one-hot or learn an embedding for it is better, so people do that. This is not a correct or wrong problem but just better empirical findings</p>

username: Federico_Ottomano
cooked: <p>Many thanks for you answer. What I still don’t understand 100% is, in the case of images, why people do use a <em>learnable</em> embedding if we already know what the label is gonna be for our inputs? If I have correctly understood what conditional GAN is doing, it is considering concatenation of our to fake/true inputs to the <em>known</em> targets/labels, so I don’t quite get why is this something that gets learned using an embedding.</p>

username: Dazitu616
cooked: <p>Yes, there isn’t a strict answer. IMO, if the image features are very large (say 512 x 128 x 128), while the label is just a single number, then the network could have some optimization issues leveraging the label. Instead, if you learn embeddings to map label to say 128-dim, then it might be easier for the network to learn. But again, there is no theory here, just experience</p>