From Scipy spline to Pytorch

username: sebastienwood
cooked: <p>Hi !<br>
I’m working on a project that has the following pipeline:</p>
<ul>
<li>numerical integration of a given function</li>
<li>interpolation (using Scipy <code>rectBivariateSpline</code>)</li>
<li>inference: given a new tuple (x1, x2) use the interpolation to predict (y)</li>
</ul>
<p>Note the first two part can be done offline, and the spline function saved for future use (factoring the cost of numerical integration + spline fitting).<br>
I’m looking to integrate the inference part of this pipeline in a Pytorch model, with the support for running it on GPU and allowing learning by backpropagation to (x1, x2).</p>
<p>I’m not so sure the torch <code>interpolate</code> function is useful for that task. Would anyone happen to have an idea on how to tackle this issue ?<br>
Thanks</p>

username: KFrank
cooked: <p>Hi Sebastien!</p>
<aside class="quote no-group" data-username="sebastienwood" data-post="1" data-topic="151632" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sebastienwood/48/9787_2.png" class="avatar"> sebastienwood:</div>
<blockquote>
<p>Note the first two part can be done offline, and the spline function saved for future use</p>
</blockquote>
</aside>
<p>I am not aware of a built-in spline function being offered by pytorch.</p>
<p>If I understand your use case correctly, it is acceptable to <em>fit</em> the spline<br>
function once, in advance of using pytorch, but you want to be able to<br>
evaluate that pre-fit spline and backpropagate through it within the pytorch<br>
framework.</p>
<p>I see two approaches:</p>
<p>Fit the spline with scipy, and then extract the spline coefficients.  Implement<br>
the spline <em>evaluation</em> using (differentiable) pytorch tensor functions.  You<br>
will then get gpu support and autograd (backpropagation) “for free.”</p>
<p>Or:</p>
<p>Package your spline evaluation as a <a href="https://pytorch.org/docs/1.11/autograd.html#torch.autograd.Function" rel="noopener nofollow ugc">custom autograd Function</a>.  Use<br>
scipy to evaluate the spline in your <code>Function</code>’s <code>forward()</code> method.</p>
<p>Because you use scipy rather than pytorch for this evaluation, you won’t<br>
get autograd / backpropagation for free.  So you also have to implement<br>
your <code>Function</code>’s <code>backward()</code> method, used by autograd for its gradient<br>
computation.  I would suggest having scipy compute the spline’s derivative<br>
(together with its value) in <code>forward()</code> and save it in <code>ctx</code> for subsequent<br>
use in <code>backward()</code>.</p>
<p>Note, wrapping scipy’s “forward” / “backward” spline evaluation in a custom<br>
autorad <code>Function</code> won’t get you gpu support for free.  However, it’s possible<br>
that an optimized scipy cpu spline implementation could be adequate.</p>
<p>It shouldn’t be very hard to implement the spline evaluation in pytorch, but<br>
I’m lazy, so I would probably lean towards wrapping the scipy implementation<br>
in a <code>Function</code>.</p>
<p>Best.</p>
<p>K. Frank</p>