Residual connection with detach().clone() and required_gradients=True

username: ibro45
cooked: <p>I do know that residual/skip connections can be implemented by simply doing</p>
<pre><code class="lang-auto">out = someOperation(x)
residual = x
out += residual
return out
</code></pre>
<p>but I am wondering if we have the same outcome by doing it in the following way</p>
<pre><code class="lang-auto">out = someOperation(x)

residual = x.detach().clone()
residual.requires_grad = True

out += residual
return out
</code></pre>
<p>Now, I know you’re asking yourself why would I even go into this trouble if the first thing works well, but, unfortunately, it doesn’t in my case. I believe it is because I am creating a partially invertible architecture and that the error might be caused by the implementation of the library for invertible architectures, but I am not completely sure since I do not understand it fully yet.</p>
<p>If my implementation follows the first case, I get</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>
<p>The problem is not in performing <code>backwards()</code> separately for losses or anything such, so that is why I assume it may be caused by the invertible architectures library.</p>
<p>However, by implementing it as shown in the second case (stumbled upon it <a href="https://discuss.pytorch.org/t/why-is-error-trying-to-backward-through-the-graph-a-second-time-solved-by-detachment-of-variable/29031/2">here</a>), it does train, but I am worried if the gradients are kept the way they should be, which is why I’m looking for the answer here. Thanks!</p>

username: albanD
cooked: <p>Hi,</p>
<p>The <code>residual.requires_grad = True</code> is not really useful here as you won’t be able to access the computed grad.<br>
And the detach() will break the graph, that means that no gradient will flow back your residual connections. Not sure how that would impact the training, but you won’t have gradients anymore for sure.</p>
<blockquote>
<p>RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.</p>
</blockquote>
<p>This can happen for a few reasons You will need to share some code.<br>
Also does using <code>out = out + residual</code> solves your issue?</p>

username: ibro45
cooked: <p>Thanks for the quick response <a class="mention" href="/u/alband">@albanD</a></p>
<p>Sure, I’m training a GAN in which I’m using a 3D U-Net-like generator that’s made partially invertible using <a href="https://github.com/silvandeleemput/memcnn" rel="nofollow noopener">MemCNN</a>. The piece of code where I have this problem is</p>
<pre><code class="lang-auto">class DownTransition(nn.Module):
    def __init__(self, inChans, nConvs):
        super(DownTransition, self).__init__()
        outChans = 2*inChans
        self.down_conv_ab = self.build_down_conv(inChans, outChans)
        self.down_conv_ba = self.build_down_conv(inChans, outChans)
        self.core = nn.Sequential(*[RevBlock(outChans) for _ in range(nConvs)])
        self.relu = nn.PReLU(outChans)

    def build_down_conv(self, inChans, outChans):
        return nn.Sequential(nn.Conv3d(inChans, outChans, kernel_size=2, stride=2),
                             nn.BatchNorm3d(outChans),
                             nn.PReLU(outChans))

    def forward(self, x, inverse=False):
        if inverse:
            down_conv = self.down_conv_ba
            core = reversed(self.core)
        else:
            down_conv = self.down_conv_ab
            core = self.core

        down = down_conv(x)
        out = down  # problematic line(s)
        for block in core:
            out = block(out, inverse=inverse)
        
        out = out + down  # tried as suggested, didn't help
        return self.relu(out)
</code></pre>
<p>and the <code>RevBlock</code> class is:</p>
<pre><code class="lang-auto">class RevBlock(nn.Module):
    def __init__(self, nchan):
        super(RevBlock, self).__init__()
        
        invertible_module = memcnn.AdditiveCoupling(
            Fm=self.build_conv_block(nchan//2),
            Gm=self.build_conv_block(nchan//2)
        )

        self.rev_block = memcnn.InvertibleModuleWrapper(fn=invertible_module, 
                                                        keep_input=True, 
                                                        keep_input_inverse=True)

    def build_conv_block(self, nchan):
        block = nn.Sequential(nn.Conv3d(nchan, nchan, kernel_size=5, padding=2),
                              nn.BatchNorm3d(nchan),
                              nn.PReLU(nchan))
        return block

    def forward(self, x, inverse=False):
        if inverse:
            return self.rev_block.inverse(x)
        else:
            return self.rev_block(x)
</code></pre>
<p>Furthermore, you can see <a href="https://github.com/tychovdo/RevGAN/blob/2af25e6a8176eaab3d424db45fb6ee2cfc5dc9a3/models/unpaired_revgan3d_model.py#L152" rel="nofollow noopener">here how the forward and backward passes are implemented</a>. I’m using that framework and I haven’t done any changes in that part of the code.</p>
<p>Finally, the traceback I get is as follow</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 35, in &lt;module&gt;
    model.optimize_parameters()
  File "/rwthfs/rz/cluster/home/ft002207/ibRevGAN/models/unpaired_revgan3d_model.py", line 158, in optimize_parameters
    self.backward_G()
  File "/rwthfs/rz/cluster/home/ft002207/ibRevGAN/models/unpaired_revgan3d_model.py", line 150, in backward_G
    self.loss_G.backward()
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
  File "/rwthfs/rz/cluster/home/ft002207/ibRevGAN/memcnn/models/revop.py", line 38, in backward_hook
    temp_output.backward(gradient=grad_output)
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
  File "/rwthfs/rz/cluster/home/ft002207/ibRevGAN/memcnn/models/revop.py", line 38, in backward_hook
    temp_output.backward(gradient=grad_output)
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/ft002207/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

</code></pre>

username: albanD
cooked: <p>So the most likely problem is that you perform some operations in a differentiable manner outside of your training loop. And so these operations are part of the computational graph for multiple iterations.<br>
Could you give a small (30/40 lines) that I can use to reproduce this?</p>

username: ibro45
cooked: <p>Hi <a class="mention" href="/u/alband">@albanD</a>! Thanks for helping, here’s a minimal example with which you can reproduce the error, it is slightly longer though, 70 lines.</p>
<ul>
<li>PyTorch: 1.3.1</li>
<li>MemCNN: 1.2.1 (latest) -  <code>pip install memcnn</code>
</li>
</ul>
<p>Code:</p>
<pre><code class="lang-auto">import torch
from torch import nn, optim
import memcnn


class RevBlock(nn.Module):
    def __init__(self, nchan):
        super(RevBlock, self).__init__()

        invertible_module = memcnn.AdditiveCoupling(
            Fm=self.build_conv_block(nchan//2),
            Gm=self.build_conv_block(nchan//2)
        )

        self.rev_block = memcnn.InvertibleModuleWrapper(fn=invertible_module, 
                                                        keep_input=True, 
                                                        keep_input_inverse=True)

    def build_conv_block(self, nchan):
        return nn.Sequential(nn.Conv3d(nchan, nchan, kernel_size=5, padding=2),
                             nn.BatchNorm3d(nchan),
                             nn.PReLU(nchan))
        
    def forward(self, x, inverse=False):
        if inverse:
            return self.rev_block.inverse(x)
        else:
            return self.rev_block(x)


class DownTransition(nn.Module):
    def __init__(self, inChans, nConvs):
        super(DownTransition, self).__init__()
        outChans = 2*inChans
        self.down_conv_ab = self.build_down_conv(inChans, outChans)
        self.down_conv_ba = self.build_down_conv(inChans, outChans)
        self.core = nn.Sequential(*[RevBlock(outChans) for _ in range(nConvs)])
        self.relu = nn.PReLU(outChans)

    def build_down_conv(self, inChans, outChans):
        return nn.Sequential(nn.Conv3d(inChans, outChans, kernel_size=2, stride=2),
                             nn.BatchNorm3d(outChans),
                             nn.PReLU(outChans))

    def forward(self, x, inverse=False):
        if inverse:
            down_conv = self.down_conv_ba
            core = reversed(self.core)
        else:
            down_conv = self.down_conv_ab
            core = self.core

        down = down_conv(x)
        out = down
        for block in core:
            out = block(out, inverse=inverse)
        
        out = out + down
        return self.relu(out)


model = DownTransition(16, 2)
criterion = nn.L1Loss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

for i in range(10):
    optimizer.zero_grad()
    data, target = torch.rand((2,16,64,64,64)), torch.rand((2,32,32,32,32))
    out = model.forward(data)
    loss = criterion(out, target)
    loss.backward()
    optimizer.step()
</code></pre>
<p>Traceback:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "minimal.py", line 71, in &lt;module&gt;
    loss.backward()
  File "/home/bro/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/bro/anaconda3/envs/maastro/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>

username: ibro45
cooked: <p><a class="mention" href="/u/alband">@albanD</a> sorry for bothering you, I’m wondering if you had time to look into it? Thanks!</p>

username: albanD
cooked: <p>Hi,</p>
<p>I did check but I did not really had any useful update <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"><br>
In particular, I have no idea how memcnn works and the error comes from there as removing it fixes the issue.<br>
I guess you want to do something similar to what is explained in this post: <a href="https://discuss.pytorch.org/t/mysterious-trying-to-backward-through-the-graph-a-second-time-issue/71153/2" class="inline-onebox">Mysterious `trying to backward through the graph a second time' issue</a> to try and figure out where the problem comes from <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: ibro45
cooked: <p>Hi <a class="mention" href="/u/alband">@albanD</a>, I appreciate it, I did find out however that it works with PyTorch 1.1.0, so I guess I’ll go with that for the moment <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: albanD
cooked: <p>Well, you might want to be careful and double check that the computed gradients are correct for a simple case.<br>
We usually only introduce these errors at places where it used to do bad things <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>