Question about using autograd function to compute derivative

arijitthegame: <p>Hi,<br>
Hope everyone is staying safe. I realize this may be a really stupid question. I am trying to use autograd.grad to compute derivative of some function I have defined. For simplicity, let us use a simple function f(x) = x^2. Here is my code:</p>
<p>import numpy as np<br>
import torch<br>
from torch.autograd import grad</p>
<p>x = np.array([1, 2, 3 , 4])<br>
y = x*x<br>
x = torch.from_numpy(x).float()<br>
y = torch.from_numpy(y).float()<br>
x.requires_grad_(True)<br>
y.requires_grad_(True)</p>
<p><span class="hashtag">#Now</span> I want to compute the derivative of y with respect to x<br>
dy = grad(y, x, grad_outputs = torch.ones_like(x),<br>
create_graph=True, retain_graph=True,<br>
only_inputs=True,<br>
allow_unused=True<br>
)[0]</p>
<p>But the output I get is (None,).<br>
I am very confused. Any help will be highly appreciated. Thanks a lot in advance.</p>

KFrank: <p>Hi Arijit!</p>
<aside class="quote no-group quote-modified" data-username="arijitthegame" data-post="1" data-topic="75465" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e495f1/48.png" class="avatar"> arijitthegame:</div>
<blockquote>
<p>I am trying to use autograd.grad to compute derivative of some function I have defined.<br>
…<br>
But the output I get is (None,).</p>
</blockquote>
</aside>
<p>The short answer is that you aren’t using pytorch to calculate your<br>
function.  (You are using python and numpy.)  So pytorch knows<br>
nothing about the relationship between <code>y</code> and <code>x</code>, and <em>can’t</em> calculate<br>
the derivative (gradient) for you.</p>
<aside class="quote no-group">
<blockquote>
<pre data-code-wrap="python"><code class="lang-python">import numpy as np
</code></pre>
</blockquote>
</aside>
<p>Clear your mind of numpy – work with pytorch tensors, instead.  Only<br>
<code>import numpy</code> if there is something you need to do that you <em>can’t</em> do<br>
without numpy.</p>
<aside class="quote no-group">
<blockquote>
<pre data-code-wrap="python"><code class="lang-python">import torch
from torch.autograd import grad

x = np.array([1, 2, 3 , 4])
y = x*x 
</code></pre>
</blockquote>
</aside>
<p><code>x</code> doesn’t have <code>requires_grad</code> set yet (e.g. <code>x.requires_grad_(True)</code>),<br>
so autograd won’t track the gradient of <code>y</code> with respect to <code>x</code>.  (Of course,<br>
<code>x</code> isn’t even a pytorch tensor yet.)</p>
<aside class="quote no-group">
<blockquote>
<pre data-code-wrap="python"><code class="lang-python">x.requires_grad_(True)
</code></pre>
</blockquote>
</aside>
<p>At this point, the cow is already out of the barn.  You’ve already<br>
calculated <code>x*x</code>, and there is no way for pytorch to go back and<br>
figure out what you did before you set <code>x.requires_grad_(True)</code>.</p>
<p>It would be well worth taking a look at pytorch’s <a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html" rel="noopener nofollow ugc">autograd tutorial</a>.</p>
<p>Good luck.</p>
<p>K. Frank</p>

arijitthegame: <p>Thank you so much. I figured it out.</p>