A problem about "autograd" and "special_bessel_j0"

RuiZhangJerry: <p>Hi, everyone. I plan to calculate the derivative of the Bessel function via the “autograd” function, the code is<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bf5539a9a27888798f75e3b372cca2ae5ddccc0e.png" data-download-href="https://discuss.pytorch.org/uploads/default/bf5539a9a27888798f75e3b372cca2ae5ddccc0e" title="Screenshot 2023-11-17 at 19.13.40"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf5539a9a27888798f75e3b372cca2ae5ddccc0e_2_690x291.png" alt="Screenshot 2023-11-17 at 19.13.40" data-base62-sha1="riBN2ZmH0a6aBYMoFeDn5v8cKyq" width="690" height="291" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf5539a9a27888798f75e3b372cca2ae5ddccc0e_2_690x291.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf5539a9a27888798f75e3b372cca2ae5ddccc0e_2_1035x436.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bf5539a9a27888798f75e3b372cca2ae5ddccc0e_2_1380x582.png 2x" data-dominant-color="373F48"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-11-17 at 19.13.40</span><span class="informations">1962×828 93.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but I received an error:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/1/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1" title="Screenshot 2023-11-17 at 19.14.58"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1_2_690x340.jpeg" alt="Screenshot 2023-11-17 at 19.14.58" data-base62-sha1="yrMHHMhMgSJorFbzey09VcI19MR" width="690" height="340" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1_2_690x340.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1_2_1035x510.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/1/f16dfb4c35eaca42c9dbff73896a6bc677e9bcf1_2_1380x680.jpeg 2x" data-dominant-color="33874B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-11-17 at 19.14.58</span><span class="informations">1920×947 263 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If I add the “requires_grad_(true)” to the “y”, just like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/1/c104798edf2b4a206e8f4878cf4ff45ce9277f94.png" data-download-href="https://discuss.pytorch.org/uploads/default/c104798edf2b4a206e8f4878cf4ff45ce9277f94" title="Screenshot 2023-11-17 at 19.20.41"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c104798edf2b4a206e8f4878cf4ff45ce9277f94_2_690x340.png" alt="Screenshot 2023-11-17 at 19.20.41" data-base62-sha1="rxvJP3EHEFOFvnd0nRGmtjUqpne" width="690" height="340" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c104798edf2b4a206e8f4878cf4ff45ce9277f94_2_690x340.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c104798edf2b4a206e8f4878cf4ff45ce9277f94_2_1035x510.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/1/c104798edf2b4a206e8f4878cf4ff45ce9277f94_2_1380x680.png 2x" data-dominant-color="353D46"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-11-17 at 19.20.41</span><span class="informations">2238×1104 216 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I also received an error:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/7/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef" title="Screenshot 2023-11-17 at 19.16.26"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef_2_690x340.jpeg" alt="Screenshot 2023-11-17 at 19.16.26" data-base62-sha1="cx7XRNUBF7ls53DzvoZkvWRWZ8b" width="690" height="340" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef_2_690x340.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef_2_1035x510.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d8c4e4898eeaa73dcf4a70c339c23453ad8cef_2_1380x680.jpeg 2x" data-dominant-color="32874B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-11-17 at 19.16.26</span><span class="informations">1920×947 260 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So I feel so confused. How can I calculate the derivatives normally? Thank you!</p>

KFrank: <p>Hi Rui!</p>
<aside class="quote no-group quote-modified" data-username="RuiZhangJerry" data-post="1" data-topic="192062" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ruizhangjerry/48/45150_2.png" class="avatar"> RuiZhangJerry:</div>
<blockquote>
<p>I plan to calculate the derivative of the Bessel function via the “autograd” function<br>
…<br>
but I received an error:</p>
</blockquote>
</aside>
<p>As near as I can tell, pytorch does not implement <code>backward()</code> for<br>
<code>torch.special.bessel_j0()</code> (nor <code>torch.special.bessel_y0()</code>),<br>
although I cannot find this explicitly documented anywhere.</p>
<p>It seems like a number of functions in <code>torch.special</code> don’t have<br>
<code>backward()</code> implemented.  The only related information I could find<br>
is this quote from the <a href="https://github.com/pytorch/pytorch/issues/50345" rel="noopener nofollow ugc">torch.special tracking issue:</a></p>
<blockquote>
<blockquote>
<p>Would a backward method to Function be needed for kv (both with respect to order nu and argument z)?</p>
</blockquote>
<p>Implementing support for autograd isn’t always required, but if the function is differentiable that’d be great!</p>
</blockquote>
<p>(So somebody was <em>talking</em> about implementing <code>backward()</code> for some of<br>
the bessel functions, implying that it hadn’t been done yet.)</p>
<aside class="quote no-group">
<blockquote>
<p>If I add the “requires_grad_(true)” to the “y”,<br>
…<br>
I also received an error:</p>
</blockquote>
</aside>
<p>In general, setting <code>requires_grad = True</code> to fix an error is a mistake.  In<br>
your case the result of <code>bessel_y0 (x)</code> doesn’t carry <code>requires_grad = True</code><br>
and is not connected by any computation graph to the operation,<br>
<code>bessel_y0 (x)</code>, that produced it.  Setting <code>requires_grad = True</code><br>
makes <code>y</code> a leaf of what could become a new computation graph but does<br>
not reconnect it to any existing graph.</p>
<aside class="quote no-group">
<blockquote>
<p>How can I calculate the derivatives normally?</p>
</blockquote>
</aside>
<p>The derivatives of <code>j0()</code> and <code>y0()</code> can be expressed in terms of <code>j1()</code><br>
and <code>y1()</code>, respectively, and these are implemented in <code>torch.special</code>.<br>
So if you just need the derivative, use <code>j1()</code> (or <code>y1()</code>) to evaluate it.</p>
<p>If, instead, you need to be able to use autograd  with, say, <code>j0()</code>, you<br>
could write a <a href="https://pytorch.org/docs/2.1/notes/extending.html#extending-torch-autograd" rel="noopener nofollow ugc">custom autograd function</a> for <code>j0()</code> that simply uses<br>
pytorch’s  <code>j0()</code> for its <code>forward()</code> method and implements its <code>backward()</code><br>
method by using pytorch’s <code>j1()</code> to compute <code>j0()</code>'s derivative.  (Doing so<br>
could be adequate for your use case, but could have numerical issues<br>
for some ranges of <code>x</code>.)</p>
<p>As an aside, please don’t post screenshots of textual information.   Doing<br>
so breaks accessibility, searchability, and copy-paste.</p>
<p>Best.</p>
<p>K. Frank</p>

RuiZhangJerry: <p>Hi，Frank！Thank you for your patient explanation and suggestions，Implementing the backward method seems a great way to solve the issue, I will give it a try!</p>
<p>Best.<br>
Rui</p>