Equivalent of scipy.special.hermite in PyTorch

AlphaBetaGamma96: <p>Hi All,</p>
<p>I was just wondering if there’s a way to do <code>scipy.special.hermite</code> within PyTorch natively (in order to utilize the GPU). My current approach to pass to the CPU and then move back to the GPU which <em>although</em> working, I’d prefer to utilize the GPU as much as possible! As it works can’t covert <code>cuda</code> to <code>numpy</code> error if you directly pass to <code>scipy.special.hermite</code>,</p>
<pre><code class="lang-auto">TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
</code></pre>
<p>An example code is shown below</p>
<pre><code class="lang-auto">import torch
from scipy.special import hermite

x=torch.randn(1000, device='cuda')

Hn_func = hermite(n=n, monic=False)
out = Hn_func(x.cpu()) 
out = out.to("cuda")
</code></pre>
<p>Thanks in advance!</p>

KFrank: <p>Hi Alpha!</p>
<aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="1" data-topic="158746" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>I was just wondering if there’s a way to do <code>scipy.special.hermite</code> within PyTorch natively (in order to utilize the GPU).</p>
</blockquote>
</aside>
<p>I’m not aware of pytorch support for Hermite polynomials.</p>
<p>Edit:</p>
<blockquote>
<p>Update:  I see support for Hermite polynomials in a recent nightly build,<br>
but it doesn’t appear to support autograd yet:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.13.0.dev20220727'
&gt;&gt;&gt; _ = torch.manual_seed (2022)
&gt;&gt;&gt; x = torch.randn (10, requires_grad = True)
&gt;&gt;&gt; out = torch.special.hermite_polynomial_h (x, 3)
&gt;&gt;&gt; out
tensor([ -2.2419,  -3.6784,  -2.6694,  -5.0147,   2.3845,   4.9064, -71.0367,
         15.3553,  45.3444,   2.9188])
&gt;&gt;&gt; print (out.grad_fn)
None
</code></pre>
</blockquote>
<p>A numerically-satisfactory old-school method to evaluate Hermite polynomials<br>
is to use their <a href="https://en.wikipedia.org/wiki/Hermite_polynomials#Recurrence_relation" rel="noopener nofollow ugc">recurrence relations.</a></p>
<p>In your example, <code>x</code> is a (cuda) tensor of shape <code>[1000]</code> for which you want<br>
to evaluate the Hermite polynomial of order <code>n</code>.</p>
<p>I would imagine writing a loop where you build up a list of the tensors<br>
<code>H_i (x)</code>, for <code>i in range (0, n + 1)</code>.  Autograd will be able to track the<br>
computation graph even though you are using a loop, so backpropagation<br>
will work.  I expect that you will get numerically satisfactory results and that<br>
autograd will work even for moderately large values of <code>n</code>.</p>
<p>You can double-check numerical stability by comparison with scipy or by<br>
repeating the computation in double-precision.</p>
<p>For large enough <code>n</code> it may end up being cheaper to use scipy on the cpu<br>
than to run the gpu loop.  If that is the case – and you need gradients – you<br>
might consider wrapping scipy’s <code>hermite()</code> in a custom autograd <code>Function</code><br>
and (presumably using scipy) implement the <code>Function</code>’s <code>backward()</code><br>
method.</p>
<p>Best.</p>
<p>K. Frank</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>!</p>
<p>Thanks for the detailed response! I had a feeling there might be a way to do it natively within PyTorch ops but it’s good to hear there’s a valid function in the nightly build!</p>
<p>I don’t need the gradient at the moment, but it’s something that might be useful in the future. I’m sure I can define it via a custom autograd Function if that’s needed! (or wait until its updated in a future PyTorch version)</p>
<p>Thanks!</p>