Integrating 6 variables in torch?

deepsana: <p>Hi there,</p>
<p>I am using pytorch to do variational energy minimization and I am trying to solve a definite integral:</p>
<p><code>\int_0^{2\pi} \int_0^{2\pi} \int_0^{\pi} \int_0^{\pi} \int_0^{L} \int_0^{L} f(x1, x2, theta1, theta2, phi1, phi2) dx1 dx2  dtheta1 dtheta2 dphi1 dphi2</code></p>
<p>The function is not simple to integrate by hand and I have only seen that <code>scipy.nquad</code> can handle something like this. But <code>scipy.nquad</code> doesn’t allow the use of <code>.backward()</code> and I couldn’t find anything after looking around.</p>
<p>Would anyone here be able to point me towards something that might be helpful?</p>
<p>Much appreciated! Thank you!!!</p>

KFrank: <p>Hi Deepsana!</p>
<aside class="quote no-group" data-username="deepsana" data-post="1" data-topic="187385" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/deepsana/48/62231_2.png" class="avatar"> deepsana:</div>
<blockquote>
<p>I am using pytorch to do variational energy minimization and I am trying to solve a definite integral:</p>
</blockquote>
</aside>
<p>I assume that your integrand depends on some parameters (with respect to<br>
which you are not integrating) and you wish to use autograd to compute the<br>
gradient of the value of the integral with respect to those parameters.</p>
<p>Numerical integration becomes increasingly difficult and expensive as the<br>
dimensionality of the integrand increases.   Six dimensions is already likely<br>
to be problematic.</p>
<p>I see that some of your variables are angles.  If your integrand becomes<br>
highly oscillatory (for some values of your parameters), you are likely to be<br>
in a world of hurt.</p>
<p>If you can integrate over any of the dimensions analytically – especially any<br>
oscillatory dimensions – you should do so, as it will be easier to integrate<br>
numerically over the remaining dimensions.</p>
<p>Pytorch does offer <a href="https://pytorch.org/docs/2.0/generated/torch.trapezoid.html" rel="noopener nofollow ugc">trapezoid(),</a> a tool for one-dimensional integration, but<br>
you’re unlikely to be successful using it in a nested fashion for a six-dimensional<br>
integral.</p>
<p>One approach would be to use a simple version of <a href="https://en.wikipedia.org/wiki/Monte_Carlo_integration" rel="noopener nofollow ugc">Monte Carlo integration</a><br>
where you randomly select a set of points from the six-dimensional region over<br>
which you are integrating and then average the integrand over those points.<br>
It might make sense to use the same fixed set of points as you optimize your<br>
parameters.</p>
<p>This can be expensive – lots of random points – but is straightforward and<br>
you will be able to backpropagate through the value of the integral without<br>
any problem (except expense).</p>
<p>If you know anything about the structure of your integrand, you can refine<br>
this approach with <em>importance sampling</em> where you cluster your random<br>
points more densely in regions where the integrand is large or is varying<br>
rapidly.</p>
<p>Last, torchquad (<a href="https://github.com/esa/torchquad" rel="noopener nofollow ugc">github</a> and <a href="https://joss.theoj.org/papers/10.21105/joss.03439" rel="noopener nofollow ugc">paper</a>) is supposedly a pytorch multidimensional<br>
integration package that supports backpropagation.  (I have not used it.)</p>
<p>If it works with autograd as advertised and they’ve done a good job implementing<br>
their chosen suite of integration methods, I would suggest that you start with<br>
<em>VEGAS Enhanced</em> – an adaptive importance / stratified-sampling Monte Carlo<br>
integration algorithm.</p>
<p>Good luck.</p>
<p>K. Frank</p>

deepsana: <p>Thank you very much for your recommendations!!! I will start exploring those.</p>

Saleh: <p>May I ask about your experience and how was the precision and speed of torchquad? I also have a same usecase where I need to optimize a loss function w.r.t the boundaries of integral.<br>
can you also comment about dimensionality? which method works better in higher dimensional settings?</p>
<p>Thank you in advance</p>