Inconsistency in Tensor.backward

bala: <p>Tensor.backward, as per documentation, frees the computation graph after the call to backward unless retain_graph is set to True. This is the exactly the behaviour I observed for the following example.</p>
<blockquote>
<p>x = torch.ones(1., requires_grad=True)<br>
y = torch.ones(1., requires_grad = True)<br>
w = x+y<br>
u=2*w<br>
u.backward()<br>
u.backward()</p>
</blockquote>
<p>2nd backward call results error which matches with what docs say.</p>
<p>But for the following example, I don’t find any error.</p>
<blockquote>
<p>x = torch.ones(1., requires_grad=True)<br>
y = torch.ones(1., requires_grad = True)<br>
w = x+y<br>
w.backward()<br>
w.backward()</p>
</blockquote>
<p>Kindly clarify. This post is similar to <a href="https://discuss.pytorch.org/t/inconsistent-behavior-for-running-backward-twice-without-retain-graph-true/39917">https://discuss.pytorch.org/t/inconsistent-behavior-for-running-backward-twice-without-retain-graph-true/39917</a></p>

ptrblck: <p>Similar to <a href="https://discuss.pytorch.org/t/runtimeerror-trying-to-backward-through-the-graph-a-second-time-but-the-buffers-have-already-been-freed-specify-retain-graph-true-when-calling-backward-the-first-time/6795/54">this question</a>:</p>
<aside class="quote no-group" data-username="albanD" data-post="54" data-topic="6795">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"><a href="https://discuss.pytorch.org/t/runtimeerror-trying-to-backward-through-the-graph-a-second-time-but-the-buffers-have-already-been-freed-specify-retain-graph-true-when-calling-backward-the-first-time/6795/54">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time</a></div>
<blockquote>
<p>This is an edge case, since the only op you do is an add and the add does not need any buffer, then there is no buffer that are missing when you do the second backward <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
</blockquote>
</aside>

bala: <p>Thanks for the response. I get it now. But I find another issue, with create_graph option of backward method. The docs say that setting it to True would build the graph of the derivative. But there is no clarity as to how it is built. I found some inconsitency in using this option as explained in the following example:</p>
<blockquote>
<p>x = torch.tensor(3., requires_grad = True)<br>
y = x**3<br>
print(f’{y.grad_fn}‘)<br>
y.backward(create_graph = True) # to facilitate computation of second order derivative of y<br>
print(f"y’: {x.grad}")</p>
<p>x.grad.data = torch.zeros_like(x)<br>
print(f’{x.grad.grad_fn}‘)<br>
x.grad.backward(create_graph = True)  # to facilitate computation of third order derivative of y<br>
print(f’y": {x.grad}’)</p>
<p>x.grad.data = torch.zeros_like(x)<br>
print(f’{x.grad.grad_fn}‘)<br>
x.grad.backward()<br>
print(f’y^3: {x.grad}’)</p>
</blockquote>
<p>I get the results as follows:</p>
<blockquote>
<p>&lt;PowBackward0 object at 0x7fb0d3503e10&gt;<br>
y’: 27.0<br>
&lt;CloneBackward object at 0x7fb0d3503dd8&gt;<br>
y": 18.0<br>
&lt;AddBackward0 object at 0x7fb0d3503908&gt;<br>
y^3: 24.0</p>
</blockquote>
<p>As can be seen, first and second derivatives (I have zeroed out the grad before computing second derivative)  are fine but third derivative is wrong. I thought by CloneBackward being the grad_fn, the graph of the derivative is simply the copy of the original graph with the node associated with the expression y=x**2  replaced by the node associated with the expression x.grad. But I don’t understand how AddBackward0 is the grad_fn after second call to backward with create_graph = True.</p>
<p>Seeking clarity on this.</p>