Register_full_backward_hook returns None

conv8d: <p>The following is included in my model’s <code>__init__</code> function</p>
<pre><code class="lang-auto">self.grads = None
target_layer = self.model.features
target_layer.register_backward_hook(self.grad_hook)
</code></pre>
<p>And <code>grad_hook(...)</code> is defined as</p>
<pre><code class="lang-auto">def grad_hook(self, module, grad_input, grad_output):
        self.grads = grad_output[0].clone().detach()
</code></pre>
<p>This code works perfectly and I’m able to retrieve the gradients saved in <code>self.grads</code> after calling <code>.backward()</code>, but it prints the following warning:</p>
<pre><code class="lang-auto">UserWarning: 
Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. 
This hook will be missing some grad_input. 
Please use register_full_backward_hook to get the documented behavior.
warnings.warn("Using a non-full backward hook when the forward contains multiple autograd Nodes "
</code></pre>
<p>However, when I replace <code>register_backward_hook</code> by  <code>register_full_backward_hook</code> in <code>__init__</code>, the gradients are not saved, i.e., <code>self.grads</code> is <code>None</code></p>
<p>Why is this happening?</p>

ptrblck: <p>I cannot reproduce the issue with e.g. <code>AlexNet</code>:</p>
<pre><code class="lang-python">grads = 0
def grad_hook(module, grad_input, grad_output):
    global grads
    grads = grad_output[0].clone().detach()
    

model = models.alexnet()
model.features.register_backward_hook(grad_hook)

x = torch.randn(1, 3, 224, 224)
out = model(x)
out.mean().backward()
print(grads.abs().sum())
&gt; tensor(0.3220)

model = models.alexnet()
model.features.register_full_backward_hook(grad_hook)
grads = 0
out = model(x)
out.mean().backward()
print(grads.abs().sum())
&gt; tensor(0.3470)
</code></pre>
<p>Could you post the model definition, so that we could try to reproduce it?</p>

conv8d: <p>The following warning is printed when I use <code>register_full_backward_hook</code> and it’s not printed when using <code>register_backward_hook</code>.<br>
I’m using the pretrained SqueezeNet model from <code>torchvision</code> (<code>torchvision.models.squeezenet1_1(pretrained = True)</code>)</p>
<pre><code class="lang-auto">UserWarning: Output 0 of BackwardHookFunctionBackward is a view and is being modified inplace. 
This view was created inside a custom Function (or because an input was returned as-is) 
and the autograd logic to handle view+inplace would override the custom backward associated 
with the custom Function, leading to incorrect gradients. 
This behavior is deprecated and will be forbidden starting version 1.6. 
You can remove this warning by cloning the output of the custom Function. 
(Triggered internally at  /pytorch/torch/csrc/autograd/variable.cpp:547.)
 result = torch.relu_(input)
</code></pre>
<p><a href="https://stackoverflow.com/questions/67768535/getting-this-warning-output-0-of-backwardhookfunctionbackward-is-a-view-and-is" rel="noopener nofollow ugc">This</a> <code>stackoverflow</code> question seems to detail the same problem as well.</p>

ptrblck: <p>I still cannot reproduce this issue using my code snippet and by replacing <code>models.alexnet()</code> with <code>models.SqueezeNet()</code>. Both hooks return a valid grad value and the first approach will raise the expected deprecation warning.<br>
Were you able to run into the error using my code snippet?</p>
<p>Note that I’m using a source build with:</p>
<pre><code class="lang-python">torch.__version__
'1.9.0a0+git6d45d7a'

torchvision.__version__
'0.11.0a0+882e11d'
</code></pre>

conv8d: <p>Running your code also reproduces my issue, I’m using versions:</p>
<pre><code class="lang-auto">torch: 1.8.1+cu101
torchvision: 0.9.1+cu101
</code></pre>
<p><code>grad_hook</code> is not called when using <code>register_full_backward_hook</code>.</p>
<pre><code class="lang-auto">grads = None
def grad_hook(module, grad_input, grad_output):
    print("grad_hook called")
    global grads
    grads = grad_output[0].clone().detach()

# 1
model = models.alexnet()
model.features.register_backward_hook(grad_hook)
x = torch.randn(1, 3, 224, 224)
out = model(x)
out.mean().backward() # prints "grad_hook called"
print(grads.size()) # prints torch.Size([1, 256, 6, 6])
print(grads.abs().sum()) # prints tensor(0.3186)

# 2
model = models.alexnet()
model.features.register_full_backward_hook(grad_hook)
grads = 1234
x = torch.randn(1, 3, 224, 224)
out = model(x)
out.mean().backward()
print(grads) # prints 1234

</code></pre>

ptrblck: <p>In that case you might need to update to the latest nightly release or build PyTorch from source, as it seems that a newer version might have fixed this issue.</p>