Computational graph is only built once

zzzf: <p>I’m using PyTorch for auto-grad without any neural network involved.</p>
<p>I’m unable to provide a reproducible code otherwise it will be too many lines. From a high level, it looks like the following:</p>
<pre><code class="lang-auto">class MarkovModel(nn.module):
    def __init__(self):
        super(MarkovModel, self).__init__()
        self.potentials = nn.ParameterDict()

    def set_potantials(self, potentials):
        ...
</code></pre>
<pre><code class="lang-auto">class BeliefProb():
    def __init__(self):
        self.model = MarkovModel()
        self.belifs = ...
    def update_beliefs(self):
        ...
        self.beliefs = ...
        ...
    def inference(self):
        for i in range(SOME_CONSTANT):
            self.update_beliefs()
</code></pre>
<pre><code class="lang-auto">iters = ANOTHER_CONSTANT
model = MarkovModel()
potentials = ...
model.set_potentials(potentials)
bp = BeliefProb(model)
optimizer = Adam(bp.model.parameters(), lr=0.01)
target = ...
for i in range(iters):
    optimizer.zero_grad()
    bp.inference()
    loss = torch.abs(target - inference.beliefs).sum()
    loss.backward()
    optimizer.step()
</code></pre>
<p>If I set <code>iters = 1</code>, then it works fine. But if <code>iters&gt;1</code>, I got the complains that</p>
<p><code>RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.</code></p>
<p>It seems that calling <code>bp.inference()</code> can build the computational graph once, but when I called <code>bp.inference()</code> again, the computational graph will not be built for the second time.</p>

crcrpar: <p>I think avoiding multiple calls of <code>backward()</code> resolves.<br>
For example, initialize <code>loss</code> with <code>None</code> and accumulating <code>loss</code> calculated in each iteration and call <code>backward</code> after the for loop like</p>
<pre><code class="lang-python">loss = None
for _ range(iters):
    bp.inference()
    cur_loss = torch.abs(target - inference.beliefs).sum()
    if loss is None: loss = cur_loss
    else: loss += cur_loss
loss.backward()
</code></pre>
<p>Toy example is <a href="https://gist.github.com/crcrpar/6032340ba8f1f8a7f8e5260e2f537be1" class="inline-onebox" rel="noopener nofollow ugc">computational-graph-is-only-built-once.ipynb · GitHub</a></p>

zzzf: <p>Thanks for your reply. I think I forgot to add the <code>optimizer</code> in the code snippet. Please see the last block where I add the <code>optimizer.step()</code> during each iteration. Therefore, I can’t call loss.backward() after the loop.</p>

crcrpar: <p>The updated code snippet looks basically fine.</p>

albanD: <p>Hi,</p>
<p>What this error means is that some part of the graph is shared between iteration. This is most likely due to the fact that you pre-compute something outside of the for loop and re-use it at each iteration inside.<br>
If you want gradients flowing throw these computations, you should move them inside the for-loop.<br>
If you don’t want gradients flowing, you should <code>.detach()</code> the result of the pre-computations so that gradients won’t flow back there.</p>

zzzf: <p>Thanks for the reply! You are right, after I move pre-computation inside the loop, gradients propagate smoothly.</p>