Autograd fails without giving any warning while doing matrix operations

snau: <p>I am trying to learn a parameter which is one element of a bigger matrix. The loss function does not directly use the learnable parameter, but a matrix having this parameter.</p>
<p>Simplified code snippet is given below to reproduce the error</p>
<pre><code class="lang-auto">import torch as t
import torch
from matplotlib import pyplot as plt

param = t.tensor([1], dtype=t.float, requires_grad=True)

a = torch.tensor([[param, 0],[0, 2]], dtype=t.float, requires_grad= True)
b = torch.tensor([[5], [6]], dtype=t.float, requires_grad = True)
c = torch.tensor([[50], [12]], dtype=t.float, requires_grad = True)

def calc_loss(A, B, C):
    X =  torch.sqrt(torch.mean((torch.matmul(A,B)-C)**2))
    return t.abs(X)

optimizer = t.optim.Adam([param], lr=1)
n = 10

values = t.zeros(n, dtype=t.float)
for i in range(n):
    optimizer.zero_grad()
    loss = calc_loss(a, b, c)
    print(f"gradient of the complex number are {param.grad}")
    print(f"calculated loss value is {loss}")
    loss.backward()
    optimizer.step()
    values[i] = param.detach()

# Plot the results
plt.plot(values, label='Learnable parameter')
plt.legend()
plt.xlabel('Iteration')
plt.ylabel('Parameter')
plt.show()

</code></pre>
<p>gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672<br>
gradient of the complex number are None<br>
calculated loss value is 31.819805145263672</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efa6668edeb006bea0e3e777834479a703042093.jpeg" alt="pytrochissue" data-base62-sha1="yc2CSDkVjRNFLWaZ7LwevP4UgTx" width="428" height="272"></p>
<p>The expected value for <code>param</code> is 10, but seems the gradient is always None. Somewhere the computational graph is brocken, but I am not able to track it.</p>

Lezcano: <p>Using a parameter to create a tensor just uses its value, does not propagate the gradient.<br>
Consider using advanced indexing, for example, by doing</p>
<pre><code class="lang-auto">a = torch.zeros(2, 2)
a[0, 0] = param
</code></pre>

snau: <p>If I use <code>a = torch.zeros(2, 2)</code> It gives other errors</p>
<p>But with<br>
<code>a = torch.tensor([[0, 0],[0, 2]], dtype=t.float, requires_grad= True)</code></p>
<p>It results in the following error. I tried param.clone as well, that also results in same error</p>
<pre><code class="lang-auto">RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.</code></pre>

snau: <p>Tried it with</p>
<pre><code class="lang-auto">a = torch.zeros(2, 2)
a[0,0] = param
a[1,1] = 2 
</code></pre>
<p>then I get the following error</p>
<p><code> RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.</code></p>
<p>I tried with retain_graph = True, but then the solution doesnt converge to the expected value of 10.</p>

snau: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> please help <img src="https://discuss.pytorch.org/images/emoji/apple/sweat.png?v=10" title=":sweat:" class="emoji" alt=":sweat:"></p>

KFrank: <p>Hi Rishi!</p>
<aside class="quote no-group quote-modified" data-username="snau" data-post="4" data-topic="134785" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/snau/48/43197_2.png" class="avatar"> snau:</div>
<blockquote>
<pre><code class="lang-auto">a = torch.zeros(2, 2)
a[0,0] = param
a[1,1] = 2 
</code></pre>
<p>then I get the following error<br>
…</p>
</blockquote>
</aside>
<p>You haven’t posted the complete code that leads to your error.  Your<br>
code fragment (which is Lezcano’s solution) should work for your use<br>
case.  Here is a complete example:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; param = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; a = torch.zeros (2, 2)
&gt;&gt;&gt; a[0, 0] = param
&gt;&gt;&gt; a[1, 1] = 2
&gt;&gt;&gt; a
tensor([[1., 0.],
        [0., 2.]], grad_fn=&lt;CopySlices&gt;)
&gt;&gt;&gt; param.grad
&gt;&gt;&gt; a.sum().backward()
&gt;&gt;&gt; param.grad
tensor([1.])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

snau: <p>Rest of the code remains the same, that is why I didn’t post it.  I tried your version and getting the same error.</p>
<pre><code class="lang-auto">import torch as t
import torch
from matplotlib import pyplot as plt

param = torch.ones (1, requires_grad = True)
a = torch.zeros (2, 2)
a[0, 0] = param
a[1, 1] = 2

b = torch.tensor([[5], [6]], dtype=t.float, requires_grad = True)
c = torch.tensor([[50], [12]], dtype=t.float, requires_grad = True)

def calc_loss(A, B, C):
    X =  torch.sqrt(torch.mean((torch.matmul(A,B)-C)**2))
    return t.abs(X)

optimizer = t.optim.Adam([param], lr=1)
n = 10

values = t.zeros(n, dtype=t.float)
for i in range(n):
    optimizer.zero_grad()
    loss = calc_loss(a, b, c)
    print(f"gradient of the complex number are {param.grad}")
    print(f"calculated loss value is {loss}")
    loss.backward()
    optimizer.step()
    values[i] = param.detach()

# Plot the results
plt.plot(values, label='Learnable parameter')
plt.legend()
plt.xlabel('Iteration')
plt.ylabel('Parameter')
plt.show()
</code></pre>
<p>Error message is</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.</code></pre>

KFrank: <p>Hi Rishi!</p>
<aside class="quote no-group" data-username="snau" data-post="7" data-topic="134785" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/snau/48/43197_2.png" class="avatar"> snau:</div>
<blockquote>
<p>I tried your version and getting the same error.</p>
</blockquote>
</aside>
<p>The problem is that you have to rebuild the “computation graph”<br>
before you call <code>.backward()</code> for a second time.  Both indexing<br>
into <code>a</code> and setting that “value” to <code>param</code> count as part of the<br>
computation graph.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.9.0
&gt;&gt;&gt;
&gt;&gt;&gt; param = torch.ones (1, requires_grad = True)
&gt;&gt;&gt;
&gt;&gt;&gt; a = torch.zeros (2, 2)
&gt;&gt;&gt; a[0,0] = param
&gt;&gt;&gt; a[1,1] = 2
&gt;&gt;&gt;
&gt;&gt;&gt; a.sum().backward()
&gt;&gt;&gt;
&gt;&gt;&gt; a[0,0] = param
&gt;&gt;&gt; a.sum().backward()   # fails
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;path_to_pytorch_install&gt;\torch\_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "&lt;path_to_pytorch_install&gt;\torch\autograd\__init__.py", line 149, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.
&gt;&gt;&gt;
&gt;&gt;&gt; b = torch.zeros (2, 2)
&gt;&gt;&gt; b[1,1] = 2
&gt;&gt;&gt;
&gt;&gt;&gt; a = b.clone()
&gt;&gt;&gt; a[0, 0] = param
&gt;&gt;&gt; a.sum().backward()
&gt;&gt;&gt;
&gt;&gt;&gt; a = b.clone()
&gt;&gt;&gt; a[0, 0] = param
&gt;&gt;&gt; a.sum().backward()   # works
</code></pre>
<p>Applying this in a tweaked version of your code:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch as t
&gt;&gt;&gt; import torch
&gt;&gt;&gt;
&gt;&gt;&gt; param = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; a = torch.zeros (2, 2)
&gt;&gt;&gt; # a[0, 0] = param   # we will do this with a_clone
... a[1, 1] = 2
&gt;&gt;&gt;
&gt;&gt;&gt; b = torch.tensor([[5], [6]], dtype=t.float, requires_grad = True)
&gt;&gt;&gt; c = torch.tensor([[50], [12]], dtype=t.float, requires_grad = True)
&gt;&gt;&gt;
&gt;&gt;&gt; def calc_loss(A, B, C):
...     X =  torch.sqrt(torch.mean((torch.matmul(A,B)-C)**2))
...     return t.abs(X)
...
&gt;&gt;&gt; optimizer = t.optim.Adam([param], lr=1)
&gt;&gt;&gt; n = 10
&gt;&gt;&gt;
&gt;&gt;&gt; values = t.zeros(n, dtype=t.float)
&gt;&gt;&gt; for i in range(n):
...     optimizer.zero_grad()
...     a_clone = a.clone()     # re-clone a in forward pass
...     a_clone[0, 0] = param   # re-"copy-slices" in forward pass
...     # loss = calc_loss(a, b, c)
...     loss = calc_loss (a_clone, b, c)  # use cloned copy in forward pass
...     print(f"calculated loss value is {loss}")
...     loss.backward()
...     print(f"gradient of the complex number are {param.grad}")   # move to after .backward()
...     optimizer.step()
...     values[i] = param.detach()
...
calculated loss value is 31.819805145263672
gradient of the complex number are tensor([-3.5355])
calculated loss value is 28.284271240234375
gradient of the complex number are tensor([-3.5355])
calculated loss value is 24.748737335205078
gradient of the complex number are tensor([-3.5355])
calculated loss value is 21.21320343017578
gradient of the complex number are tensor([-3.5355])
calculated loss value is 17.677669525146484
gradient of the complex number are tensor([-3.5355])
calculated loss value is 14.142135620117188
gradient of the complex number are tensor([-3.5355])
calculated loss value is 10.60660171508789
gradient of the complex number are tensor([-3.5355])
calculated loss value is 7.071067810058594
gradient of the complex number are tensor([-3.5355])
calculated loss value is 3.535533905029297
gradient of the complex number are tensor([-3.5355])
calculated loss value is 0.0
gradient of the complex number are tensor([nan])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

snau: <p>Thanks Frank ,</p>
<p>But it seems the code still have some issue, the gradient is constant over the epochs and becomes Nan afterwards.<br>
The output you posted also have same gradient for all the epochs<br>
<code>gradient of the number are tensor([-3.5355])</code></p>

snau: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a> <a class="mention" href="/u/lezcano">@Lezcano</a>,<br>
Do you think its worth trying to write my own gradient for this matrix assembly using <code> from torch.autograd import Function</code></p>

KFrank: <p>Hi Rishi!</p>
<aside class="quote no-group" data-username="snau" data-post="9" data-topic="134785" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/snau/48/43197_2.png" class="avatar"> snau:</div>
<blockquote>
<p>the gradient is constant over the epochs</p>
</blockquote>
</aside>
<p>Calculate analytically the derivative of <code>loss</code> with respect to <code>param</code>,<br>
specifically for the values of <code>a</code>, <code>b</code>, and <code>c</code> that you are using.  (Hint:<br>
You will discover that it is constant.)</p>
<aside class="quote no-group">
<blockquote>
<p>and becomes Nan afterwards.</p>
</blockquote>
</aside>
<p>In essence, you are having pytorch calculate <code>sqrt (x**2)</code>.<br>
<code>sqrt (x)</code> is singular at <code>x = 0</code>, and its derivative diverges (and<br>
pytorch – rightly – doesn’t replace <code>sqrt (x**2)</code> with <code>x</code>).</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; x = torch.tensor ([0.0], requires_grad = True)
&gt;&gt;&gt; torch.sqrt (x).backward()
&gt;&gt;&gt; x.grad
tensor([inf])
&gt;&gt;&gt; x = torch.tensor ([0.0], requires_grad = True)
&gt;&gt;&gt; torch.sqrt (x**2).backward()
&gt;&gt;&gt; x.grad
tensor([nan])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

snau: <p>Thanks Frank.</p>
<p>could work with this solution <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>