Change model parameters before back propagation

t170815518: <p>Hi, I am trying on a new incremental learning method, which requires the following step:<br>
The paper is here: <a href="https://core.ac.uk/download/pdf/196212537.pdf" rel="noopener nofollow ugc">ActiveLink: Deep Active Learning for Link Prediction in Knowledge Graphs</a></p>
<p>(There are i windows of data during each iteration i.)</p>
<ol>
<li>Temporarily update the model on the loss on Window i with the current model parameter.</li>
<li>Use this temporary parameter to accumulate the loss on Window 0-i.</li>
<li>The accumulated loss will be used to update the original parameter.</li>
</ol>
<p>To achieve this, my code is below:</p>
<pre><code class="lang-auto">        model.train()
        model.zero_grad()

        feed_dict = data_iterator.windows[i]

        previous_param = model.state_dict().copy()
        _temporary_update(model, feed_dict, inner_optimizer)

        def _closure():
            """ Uses the current parameters of the model on the current and previous windows within the range, and
            returns the total loss on these windows.
            """
            model.zero_grad()
            total_loss = 0

            for window_dict in data_iterator.iter_from_list(i, window_limit):
                loss = model.loss(window_dict)
                loss.backward()
                total_loss += loss.item()

            model.load_state_dict(previous_param)
            return total_loss

        optimizer.step(closure=_closure)
</code></pre>
<p>My question is about the <code>closure()</code> function. In Step 3, it is actually updating the model with <code>previous_param</code> with <code>total_loss</code>, which has been accumulated in the graph. I wonder if it is the correct way to do so? Will the optimizer update the model with <code>previous_param</code>?</p>
<p>Thank you in advance.</p>

ptrblck: <p>I think your approach might work and should be similar to this small code snippet, it I understand it correctly:</p>
<pre><code class="lang-python"># setup
model = nn.Linear(1, 1)
optimizer = torch.optim.SGD(model.parameters(), lr=1.)
sd = copy.deepcopy(model.state_dict())

# first update
out = model(torch.randn(1, 1))
out.mean().backward()
optimizer.step()

# use new gradients on old state_dict
optimizer.zero_grad()
out = model(torch.randn(1, 1))
out.mean().backward()
model.load_state_dict(sd)
optimizer.step()
</code></pre>
<p>I’m not familiar with your use case, but note that you are updating the “old” parameters with the gradients calculated using the forward pass (and loss) from the “new” parameters.<br>
Is this your use case indeed?</p>

t170815518: <p>Thank you for the reply, and sorry for the late reply.<br>
Yes, this is my use case where the “old” parameter is updated based on the gradients w.r.t another set of parameters.</p>
<p>I also checked other threads on the forum, where some people said the intermediary results won’t be changed with the model parameters. Is this the case?</p>

ptrblck: <p>Do you mean the intermediate activations, which were created during the forward pass?<br>
If so, then no they won’t be changed by just upgrading the parameters, since a new forward pass would be needed.</p>