Modified Bessel Function of Order 0

username: Ishan_Gupta
cooked: <p>How to implement modified bessel functions in Pytorch? Any ideas/hints?</p>

username: tom
cooked: <p>For a CPU only version, you could use scipy.special</p>
<pre><code class="lang-auto">import scipy.special
# note, we do not differentiate w.r.t. nu
class ModifiedBesselFn(torch.autograd.Function):
    @staticmethod
    def forward(ctx, inp, nu):
        ctx._nu = nu
        ctx.save_for_backward(inp)
        return torch.from_numpy(scipy.special.iv(nu, inp.detach().numpy()))
    @staticmethod
    def backward(ctx, grad_out):
        inp, = ctx.saved_tensors
        nu = ctx._nu
        # formula is from Wikipedia
        return 0.5* grad_out *(ModifiedBesselFn.apply(inp, nu - 1.0)+ModifiedBesselFn.apply(inp, nu + 1.0)), None
modified_bessel = ModifiedBesselFn.apply
</code></pre>
<p>Then you can check that it works:</p>
<pre><code class="lang-auto">x = torch.randn(5, requires_grad=True, dtype=torch.float64)
torch.autograd.gradcheck(lambda x: modified_bessel(x, 0.0), (x,))
</code></pre>
<p>If you wanted to do this for cuda, too, you would have to write your own kernel (or open an issue at the PyTorch github).</p>
<p>The differentiation w.r.t. $nu$ seems more tricky, I don’t know whether it has a closed from solution.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Ishan_Gupta
cooked: <p>Thanks Tom,</p>
<p>This might be very naiive. But, I hope it is possible to just compute the backward pass for the variables related to Bessel in CPU and rest in GPU?</p>

username: fermat97
cooked: <p>Hi <a class="mention" href="/u/tom">@tom</a>, what do you mean by</p>
<blockquote>
<p>write your own kernel</p>
</blockquote>
<p>I need to compute the KL divergence of two <code>von Mises-Fisher</code> distributions for a variational loss function. Right now I do the computation with <code>scipy</code>. My GPU utility is very low (most of the time close to zero percent usage), I guessed that the reason is that the computation of the <code>scipy</code> is carried out on the CPU and then transferred to GPU. Therefore I would like to do all the computation on the GPU.</p>

username: tom
cooked: <p>You’d need to write a fuction for that yourself, but depending on what operations you need, you might be able to use the fuser, or write a cuda kernel. What is the formula for your calculation?</p>
<p>Best regards</p>
<p>Thomas</p>

username: fermat97
cooked: <p>Thank you <a class="mention" href="/u/tom">@tom</a> I have finally computed the KL term separately with <code>scipy</code> and then moved it to GPU, the gradient is not with respect to <code>kappa</code> so I think there shouldn’t be any problem. This is the KL-divergence term I would like to compute:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/7/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8" title="kld"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/7/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8_2_690x57.jpeg" alt="kld" data-base62-sha1="gVwquhHAQbuvzlCxshYgA4AnNxK" width="690" height="57" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/7/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8_2_690x57.jpeg, https://discuss.pytorch.org/uploads/default/original/2X/7/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/7/76a366bd1efaa6cf5f7fb71a6beb5e98a8b1c0f8.jpeg 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">kld</span><span class="informations">831×69 14.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><code>d</code> is the dimesntion of the latent space.</p>

username: Ja-Keoung_Koo
cooked: <p>How do you succeeded in using Bessel function without Scipy? I have a similar problem now.</p>

username: Founty
cooked: <p>have u finished pytorch GPU version of Modified Bessel functions?</p>