Optimizer parameters not updating

avalon1511: <p>My code performs a loss minimization from the output of a pretrained GAN. Apart from the GAN parameters I am also trying to optimize the latent variable z.</p>
<p>The code is set up as follows</p>
<pre><code class="lang-auto">with torch.no_grad():
   gan_out = G(z_initial)

z_initial.retain_grad = True
loss = MSEloss(gan_out, observed_image)

GAN_optim.zero_grad()
z_optim.zero_grad()
loss.backward()
GAN_optim.step()
z_optim.step()
</code></pre>
<p>The optimizers are defined as</p>
<pre><code class="lang-auto">GAN_optim = optim.RMSprop(G.parameters(), lr = 3e-5)
z_optim = optim.RMSprop([z_initial], lr = 1e-3)

# where z_initial is 

z_initial = torch.Variable(z_.clone().to(device), requires_grad = True)
</code></pre>
<p>After training, I am visualizing <code>z_initial</code> and the GAN weights. While the GAN weights are updating and changing <code>z_initial</code> is not.</p>
<p>The tensorboard visualization is done for every epoch as follows</p>
<pre><code class="lang-auto">writer.add_histogram('Z_prior', z_initial)
</code></pre>
<p>How can I make sure <code>z_initial</code> updates?</p>

thecho7: <p>How about using <code>nn.Parameter</code> instead of <code>Variable</code>?</p>
<p><code>z_initial = nn.Parameter(z_).to(device)</code></p>

avalon1511: <p>Tried that and had the same issue! Not sure what else I should do</p>

ptrblck: <p>Your code shouldn’t work at all, since you are wrapping the forward pass into a <code>no_grad</code> guard:</p>
<pre><code class="lang-python">G = nn.Linear(1, 1)
z_initial = torch.randn(1, 1)

with torch.no_grad():
   gan_out = G(z_initial)

z_initial.retain_grad = True
loss = nn.MSELoss()(gan_out, torch.rand_like(gan_out))
loss.backward()
# &gt; RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Could you explain the use case a bit and why you are disabling Autograd while trying to calculate the gradients?</p>

avalon1511: <p>Yeah I think you are right, I was considering that to be a problem. I require the generator to be frozen to get <code>gan_out</code>. That’s why I was disabling autograd.</p>
<p>Will using <code>G.no_grad()</code> work instead?</p>

ptrblck: <p>No, <code>no_grad()</code> is not a class method of <code>nn.Module</code>.<br>
Are you trying to update the input to <code>G</code> only while <code>G</code> is frozen?<br>
If so, this should work:</p>
<pre><code class="lang-python">G = nn.Linear(1, 1)
# freeze
for param in G.parameters():
    param.requires_grad = False

z_initial = torch.randn(1, 1, requires_grad=True)

gan_out = G(z_initial)

loss = nn.MSELoss()(gan_out, torch.rand_like(gan_out))
loss.backward()
print(z_initial.grad)
# &gt; tensor([[0.5429]])
print(G.weight.grad)
# &gt; None
</code></pre>

avalon1511: <p>Yeah my use case is specifically keeping G frozen and generating samples from it - it is a test time application.</p>
<p>I will try this method, thanks!</p>
<p>Another thing -</p>
<p>After this <code>loss.backward()</code> call, I need to modify <code>z_initial</code> to clamp it like so:</p>
<pre><code class="lang-auto">z_initial = torch.clamp(z_initial, -1., 1.)
</code></pre>
<p>I am not able to do this and I am getting an error saying the code is trying to traverse the graph twice after some variables have been freed. I understand the error since it makes sense.</p>
<p>However, once I call the optimizer for <code>z</code>, I need to clamp it. Is there any other way this can be done?</p>

ptrblck: <p>You might want to <code>clamp</code> the tensor inplace to avoid having to pass it to the optimizer repeatedly as seen here:</p>
<pre><code class="lang-python">G = nn.Linear(1, 1)
# freeze
for param in G.parameters():
    param.requires_grad = False

z_initial = torch.randn(1, 1, requires_grad=True)
optimizer = torch.optim.Adam([z_initial], lr=1e-3)

for epoch in range(10):
    optimizer.zero_grad()
    gan_out = G(z_initial)

    loss = nn.MSELoss()(gan_out, torch.rand_like(gan_out))
    loss.backward()
    print('epoch {}, loss {}, z_initial {}, .grad {}'.format(
        epoch, loss.item(), z_initial, z_initial.grad))
    optimizer.step()
    with torch.no_grad():
        torch.clamp_(z_initial, -1., 1.)
</code></pre>