GRU training problem with retain_graph

username: learner47
cooked: <p>I am aware that, while employing <code>loss.backward()</code> we need to specify <code>retain_graph=True</code> if there are multiple networks and multiple loss functions to optimize each network separately. But even with (or without) specifying this parameter I am getting errors. Following is an MWE to reproduce the issue (on PyTorch 1.6).</p>
<pre><code class="lang-auto">import torch
from torch import nn
from torch import optim
torch.autograd.set_detect_anomaly(True)


class GRU1(nn.Module):
    def __init__(self):
        super(GRU1, self).__init__()
        self.brnn = nn.GRU(input_size=2, bidirectional=True, num_layers=1, hidden_size=100)

    def forward(self, x):
        return self.brnn(x)


class GRU2(nn.Module):
    def __init__(self):
        super(GRU2, self).__init__()
        self.brnn = nn.GRU(input_size=200, bidirectional=True, num_layers=1, hidden_size=1)

    def forward(self, x):
        return self.brnn(x)


for i in range(100):
    gru1 = GRU1()
    gru2 = GRU2()
    gru1_opt = optim.Adam(gru1.parameters())
    gru2_opt = optim.Adam(gru2.parameters())
    gru1_opt.zero_grad()
    gru2_opt.zero_grad()
    criterion = nn.MSELoss()
    vector = torch.randn((15, 100, 2))
    gru1_output, _ = gru1(vector)  # (15, 100, 200)
    loss_gru1 = criterion(gru1_output, torch.randn((15, 100, 200)))
    loss_gru1.backward(retain_graph=True)
    gru1_opt.step()
    gru2_output, _ = gru2(gru1_output)  # (15, 100, 2)
    loss_gru2 = criterion(gru2_output, torch.randn((15, 100, 2)))
    loss_gru2.backward(retain_graph=True)
    gru2_opt.step()
    print(f"GRU1 loss: {loss_gru1.item()}, GRU2 loss: {loss_gru2.item()}")
</code></pre>
<p>With <code>retain_graph</code> set to <code>True</code> I get the error</p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 300]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
</code></pre>
<p>The error without the parameter is</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>
<p>which is expected.</p>
<p>Please point at what needs to be changed in the above code for it to begin training. Any help would be appreciated.</p>

username: albanD
cooked: <p>Hi,</p>
<p>The error</p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 300]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
</code></pre>
<p>Comes from the fact that when you run <code>    loss_gru2.backward()</code> (note that the retain_graph is not needed here), you actually backprop all the wait to <code>gru1</code>. But <code>gru1</code>'s paramters where modified inplace by <code>gre1_opt.step()</code>. Thus the error you’re seeing.</p>
<p>If you want the second part to only consider <code>gru2</code>, then you can detach the output from <code>gru1</code> to make them independent:</p>
<pre><code class="lang-auto">    gru2_output, _ = gru2(gru1_output.detach())  # (15, 100, 2)
</code></pre>
<p>This will prevent the backprop from going all the way to <code>gru1</code>.</p>
<p>Note that it will also remove the need from <code>retain_graph=True</code> in the first backward of <code>gru1</code> because now you don’t backprop through it again !</p>

username: learner47
cooked: <p><a class="mention" href="/u/alband">@albanD</a> hi, thanks for the response. However the use case I am looking at is to train the first network <code>GRU1</code> in the second part as well. How would that be achieved? I have added the updated code below:</p>
<pre><code class="lang-auto">import torch
from torch import nn
from torch import optim
torch.autograd.set_detect_anomaly(True)


class GRU1(nn.Module):
    def __init__(self):
        super(GRU1, self).__init__()
        self.brnn = nn.GRU(input_size=2, bidirectional=True, num_layers=1, hidden_size=100)

    def forward(self, x):
        return self.brnn(x)


class GRU2(nn.Module):
    def __init__(self):
        super(GRU2, self).__init__()
        self.brnn = nn.GRU(input_size=200, bidirectional=True, num_layers=1, hidden_size=1)

    def forward(self, x):
        return self.brnn(x)


gru1 = GRU1()
gru2 = GRU2()
gru1_opt = optim.Adam(gru1.parameters())
gru2_opt = optim.Adam(gru2.parameters())
criterion = nn.MSELoss()

for i in range(100):
    gru1_opt.zero_grad()
    gru2_opt.zero_grad()
    vector = torch.randn((15, 100, 2))
    gru1_output, _ = gru1(vector)  # (15, 100, 200)
    gru2_output, _ = gru2(gru1_output)  # (15, 100, 2)
    loss_gru1 = criterion(gru2_output, torch.randn((15, 100, 2)))
    loss_gru1.backward()
    gru1_opt.step()
    gru1_output, _ = gru1(gru2_output)
    gru2_output, _ = gru2(gru1_output)  # (15, 100, 2)
    loss_gru2 = criterion(gru2_output, torch.randn((15, 100, 2)))
    loss_gru2.backward()
    gru2_opt.step()
    print(f"GRU1 loss: {loss_gru1.item()}, GRU2 loss: {loss_gru2.item()}")
</code></pre>
<p>What should one do in such a case? Thanks!</p>

username: albanD
cooked: <p>Hi,</p>
<p>So you want the <code>gru2_opt</code> to use the gradient for both use of gru2?<br>
In that case, you will have to delay the <code>gru1_opt,step()</code> I’m afraid. Or recompute the forward after the gru1 net has been updated if that’s what you want.</p>

username: learner47
cooked: <p><a class="mention" href="/u/alband">@albanD</a> that is not what I was expecting, because <code>gru1_opt.step()</code> would help the netwrok predict better in the next time instant following <code>step()</code>. Is there any workaround? Also I got the following to work:</p>
<pre><code class="lang-auto">for i in range(100):
    gru1_opt.zero_grad()
    gru2_opt.zero_grad()
    vector = torch.randn((15, 100, 2))
    gru1_output, _ = gru1(vector)  # (15, 100, 200)
    gru2_output, _ = gru2(gru1_output)  # (15, 100, 2)
    loss_gru1 = criterion(gru2_output, torch.randn((15, 100, 2)))
    loss_gru1.backward(retain_graph=True)
    gru1_opt.step()
    gru1_output, _ = gru1(vector)
    gru2_output, _ = gru2(gru1_output)  # (15, 100, 2)
    loss_gru2 = criterion(gru2_output, torch.randn((15, 100, 2)))
    loss_gru2.backward(retain_graph=True)
    gru2_opt.step()
    print(f"GRU1 loss: {loss_gru1.item()}, GRU2 loss: {loss_gru2.item()}")
</code></pre>
<p>But I guess I will run into memory issues.</p>
<p>PS: In the first backprop I only need update the weights of <code>GRU1</code>. In the second backprop, I need to update the weights of <code>GRU2</code></p>
<p>In essence, I want to flow the gradients through <code>GRU2</code> and update the weights of <code>GRU1</code> in the first instance. In the second part I need to update the weights of <code>GRU2</code> only.</p>

username: albanD
cooked: <p>Yes this code sample will work as the second set of update is from <code>vector</code> which is not linked to the first set of forwards. That will work fine if it’s what you want.<br>
You can actually remove the <code>reain_graph=True</code> there are you don’t backprop multiple time <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<blockquote>
<p>But I guess I will run into memory issues.</p>
</blockquote>
<p>I don’t think so, why would you?</p>

username: learner47
cooked: <p><a class="mention" href="/u/alband">@albanD</a> because I would be retaining the graph every time, won’t I be having memory problem after some iterations?</p>

username: albanD
cooked: <p>Well you don’t need to retain the graph in this case I think.</p>
<p>But even if you do so, <code>retain_graph</code> only prevents it from being freed during the backward pass. But when nothing can access it anymore, it will be freed: it does not leak memory. It just delays when the memory is released.</p>