Replace the input after forward but before backward?

username: roylight
cooked: <pre><code class="lang-auto">import torch
from torch.autograd import Variable
temp_a = torch.rand(1, 4)
a = torch.rand(1, 4)
b = a**2
c = b*2
d = c.mean()
a.copy_(temp_a)
d.backward()
</code></pre>
<p>Hi in this toy example, if I replace the original parameters or input of the model (e.g. <em>a</em>) with another set of the parameters (e.g. <em>temp_a</em>). Then I do backward. Is the calculated gradient w.r.t the original input <em>a</em> or the  <em>temp_a</em>.</p>
<p>This is a just a toy example that shows my intention. In my implementation,</p>
<ol>
<li>I cloned the model parameters at the beginning.</li>
<li>And then I do an intermediate step that would forward, backward, and update the model parameters with autograd.</li>
<li>I forward the model again and get a new output. And I tried to get the gradient of the current output w.r.t the original model parameters I cloned. So I copy the original model parameters back to the model just as I did in the toy example. And I will do the backward and get the gradient.</li>
<li>Finally, I will update the original model parameters with the gradient I get in the last step.</li>
</ol>
<p>Can this work as I expected? Thank you for your help.</p>

username: albanD
cooked: <p>Hi,</p>
<p>For your example, this will not work if you actually make a require gradients. It will show you that you can’t modify a inplace in a differentiable manner. And so, if you change it in a non-differentiable manner with torch.no_grad(),  it will compute the gradient wrt the original value of a (the one used in the forward).</p>
<p>For the nn.Module version, note that the optimizer step and model state dict loading are both done in a non-differentiable manner. So these ops will be ignored by the autograd.<br>
When you call backward, you will get the gradient wrt to the Parameter that was used in the forward.</p>

username: roylight
cooked: <p>Hi, thank you for your answer.</p>
<p>I didn’t use <code>load_dict</code> to copy parameters. Instead, I use:</p>
<pre><code class="lang-auto">def assign_model_parameters(model, temp_parameters):
    idx = 0
    assert len(temp_parameters) == len(list(model.parameters()))
    for param in model.parameters():
        param.data.copy_(temp_parameters[idx].data)
        idx += 1
</code></pre>
<p>And also, if I use <code>torch.autograd.grad</code> instead of <code>backward</code>,</p>
<pre><code class="lang-auto">        assign_model_parameters(model, initial_parameters)
        model_parameters = list(model.parameters())
        grads = torch.autograd.grad(second_loss, model_parameters, allow_unused=True)
</code></pre>
<p>where <code>second_loss</code> is the loss in the third step aforementioned. And I manually update the gradient,</p>
<pre><code class="lang-auto">def update_parameters(temp_parameters, loss, alpha, clip_grad=5.0):
    weights = temp_parameters
    grads = torch.autograd.grad(loss, weights, allow_unused=True)
    direct_clip_grad_norm_(grads, clip_grad)
    new_weights = []
    idx = 0
    for w, g in zip(weights, grads):
        if g is not None:
            #print (g)
            #print (idx)
            new_weights.append(w - alpha * g.detach())
        else:
            new_weights.append(w)
        idx += 1
    assert len(new_weights) == len(weights)
    return new_weights
</code></pre>
<p>Can it work as I expected?</p>

username: albanD
cooked: <p>Hi,</p>
<p>Using <code>.data</code> is the same as “doing it in a non-differentiable way”.<br>
Note that you should never use <code>.data</code> anymore and use <code>with torch.no_grad()</code> to do this as it is much safer (look for side effects of .data on this forum if you want to know more).</p>
<p><code>.grad()</code> or <code>.backward()</code> will give the same thing here.<br>
It will compute the gradient as if you did not changed the parameters (and because you use .data, will ingore the inplace change in the inplace checks and potentially compute these gradients wrong).</p>

username: roylight
cooked: <p>Thank you. May you give me some suggestion about how to achieve the effect I expected?</p>

username: albanD
cooked: <p>I can’t think of an easy way to do it I’m afraid. Have you looked at things like higher (<a href="https://github.com/facebookresearch/higher">https://github.com/facebookresearch/higher</a>) that might be doing what you want in a cleaner way no?</p>

username: roylight
cooked: <p>Hi thank you. I found a method in this question. <a href="https://discuss.pytorch.org/t/resolved-implementing-maml-in-pytorch/4053" class="inline-onebox">[resolved] Implementing MAML in PyTorch</a> They use the the modules such as <code>functional.linear</code> which has portable weights to achieve this effect. It is a little bit complicated because I have to manually modify the weight. But it seems to work.</p>