Can't optimize single parameter Tensor

patrickctrf: <p>I’m debugging my GAN training script because my generator does not learn anything. I’m using a Subset of my dataset with a single sample, so my generator should learn a constant output equal to that sample. But it doesn’t.</p>
<pre><code class="lang-auto"># Train Data
train_dataset = NsynthDatasetFourier(path="nsynth-train/", noise_length=noise_length)
train_dataset = Subset(train_dataset, [0, ])  # dummy dataset for testing script
train_dataloader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True, num_workers=1)
</code></pre>
<hr>
<p>Then I decided to create a <code>dummy_generator</code> that only outputs a <strong>Parameter Tensor</strong> saved as instance object. So it definitely should learn that tensor to be equal to the only sample um my dataset, but that tensor never changes.</p>
<pre><code class="lang-auto">class DummyGenerator(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__()

        self.dummy_tensor = nn.Parameter(torch.rand((1, 2, 1024, 128), requires_grad=True))

    def forward(self, x):
        return self.dummy_tensor
</code></pre>
<hr>
<p>Is my approach correct? Neither my original Generator or my DummyGenerator seems to learn, no matter how big is my learning rate. My discriminator seems to learn fine.</p>
<p><strong>If anybody can help me, I would be very grateful. My complete code is <a href="https://github.com/patrickctrf/projeto-ia376/tree/e3ptk/tests" rel="noopener nofollow ugc">here</a>.</strong></p>
<hr>
<p>P.S.: It is necessary to download <a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-train.jsonwav.tar.gz" rel="noopener nofollow ugc">Nsynth dataset</a> to run the complete code.</p>

ptrblck: <p>The use case is a bit strange, but should generally work as seen in this small example overfitting a static target:</p>
<pre><code class="lang-python">class DummyGenerator(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__()

        self.dummy_tensor = nn.Parameter(torch.rand((1, 2, 1024, 128), requires_grad=True))

    def forward(self):
        return self.dummy_tensor
    
model = DummyGenerator()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.MSELoss()

y = torch.randn(1, 2, 1024, 128)

for epoch in range(10000):
    optimizer.zero_grad()
    out = model()
    loss = criterion(out, y)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# ...
# epoch 9998, loss 1.416539715387577e-11
# epoch 9999, loss 1.4069788736859046e-11
</code></pre>
<p>Check if your parameter has a valid <code>.grad</code> attribute after the <code>backward</code> call and is indeed being updated.</p>

patrickctrf: <p>Thanks for your response. I tried a case similar to your example and changed my generator <strong>loss</strong> to minimize the difference from the <strong>target</strong>, instead of aiming discriminator’s output:</p>
<pre><code class="lang-auto">generator_discriminator_out = discriminator(generated_data)
generator_loss = criterion(generated_data, target)
# generator_loss = criterion(generator_discriminator_out, true_labels)
</code></pre>
<p>The generator successfully accomplishes this task and the discriminator’s output becomes 0.5 for every input (this is expected).</p>
<hr>
<p>But when I go back to the original loss for GANs (trying to be approved by the discriminator), the output from my generator doesn’t match the sample:</p>
<pre><code class="lang-auto">generator_discriminator_out = discriminator(generated_data)
# generator_loss = criterion(generated_data, target)
generator_loss = criterion(generator_discriminator_out, true_labels)
</code></pre>
<hr>
<p>This is an indicator that the discrimination is not learning the correct target, right?</p>
<p>Extra info: When I train with the <strong>whole dataset</strong> again, generator gets stuck with <strong>constant output</strong> for every input (and MSE loss goes to 1, the maximum value). I don’t know if it’s relevant information, but I don’t understand why my generator would stop in bad local minima like that.</p>