RuntimeError: Module backward hook for grad_input is called before the grad_output one. This happens because the gradient in your nn.M odule flows to the Module's input without passing through the Module's output

username: GeoffNN
cooked: <p>I’m using a resnet18 from torchvision. When running the forward pass, I get the following warning:</p>
<p><code>UserWarning: Output 0 of BackwardHookFunctionBackward is a view and is being modified inplace. This view was created inside a custom Function (or because an input was returned as-is) and the autograd logic to handle view+inplace would override the custom backward associated with the custom Function, leading to incorrect gradients. This behavior is deprecated and will be forbidden starting version 1.6. You can remove this warning by cloning the output of the custom Function. </code></p>
<p>And when I run the backward pass, I get</p>
<p><code>Module backward hook for grad_input is called before the grad_output one. This happens because the gradient in your nn.Module flows to the Module's input without passing through the Module's output. Make sure that the output depends on the input and that the loss is computed based on the output.</code></p>
<p>The error seems to stem from <code>result = torch.relu_(input)</code><br>
This didn’t happen previously.</p>

username: prettycold
cooked: <p>Hi <a class="mention" href="/u/geoffnn">@GeoffNN</a> ,</p>
<p>I am facing a similar issue, were you able to re-solve this?</p>

username: morg
cooked: <p>Yep, I’m getting the same with resnet50 from torchvision</p>
<p>torch vesion: ‘1.8.1+cu102’<br>
torchvision version: ‘0.9.1+cu102’</p>

username: GeoffNN
cooked: <p>I was using the <code>wandb</code> library with <code>wandb.watch(model)</code>. This was causing the problem, so I removed this part.</p>

username: prettycold
cooked: <p>Thanks a lot <a class="mention" href="/u/geoffnn">@GeoffNN</a>. I too was using wandb.watch and on removing it now it goes through. You saved my day!</p>
<p>Just curious, how did you think of the “watch” to be the reason?</p>

username: GeoffNN
cooked: <p>Welcome! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p><code>wandb.watch</code> was logging the gradients, so had to call <code>backward()</code>, which seemed related to backpropagating twice in the graph.</p>

username: GeoffNN
cooked: <p>Were you also using <code>wandb</code>?</p>

username: prettycold
cooked: <p>Thanks <a class="mention" href="/u/geoffnn">@GeoffNN</a> <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"></p>

username: morg
cooked: <p>Yes I was actually! Downgrading to torchvision 0.8.2 actually fixed the issue.</p>
<p><code>pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 -f https://download.pytorch.org/whl/torch_stable.html </code></p>
<p>Think there is something in torchvision 0.9.0 that doesn’t play nice with wandb.watch? Just searched and found this <a href="https://github.com/wandb/client/issues/2157" rel="noopener nofollow ugc">issue on <code>wandb</code></a> too incase you hadn’t seen it</p>

username: prettycold
cooked: <p>I did try to downgrade but got CUDA Out of Memory error, so didn’t take that route. Somehow, didn’t come across this link though I was searching in all directions <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:"></p>