Erf and erfinv functions in PyTorch

username: Soniya
cooked: <p>I have a torch.FloatTensor say A of size u,v,x,y. I have to calculate its erf and erfinv. Is there any way to do this in PyTorch. Please help me.</p>

username: smth
cooked: <p>What is Erf (and what is it’s inverse)? Is this an abbreviation?</p>

username: Soniya
cooked: <p>Erf is error function also known as Gauss error function (<a href="https://en.wikipedia.org/wiki/Error_function" rel="nofollow noopener">https://en.wikipedia.org/wiki/Error_function</a>) and erfinv is its inverse.</p>

username: hadware
cooked: <p>I’m actually interested by this as well, could you give us some updates if you find a solution?</p>

username: AjayTalati
cooked: <pre><code class="lang-auto">import torch
from scipy import special
## reference - https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.special.erf.html

u = 1
v = 2
x = 3
y = 4

A = torch.randn(u,v,x,y)

special.erf( A.numpy() )
special.erfinv( A.numpy())
</code></pre>

username: Soniya
cooked: <p>Thanks Ajay! This is working fine. <img src="//discuss.pytorch.org/images/emoji/apple/blush.png?v=5" title=":blush:" class="emoji" alt=":blush:"></p>

username: cerisara
cooked: <p>Nope, it does not work, because I also need the error function but as an autograd function; I mean, I would like to insert this function in the computation graph and pytorch to automatically differentiate it !</p>
<p>If the function is not supported for now in pytorch, may I define my own custom autograd function ?</p>
<p>(actually the equivalent of TF function:<br>
<a href="https://www.tensorflow.org/versions/r0.11/api_docs/python/math_ops/basic_math_functions#erf" rel="nofollow noopener">https://www.tensorflow.org/versions/r0.11/api_docs/python/math_ops/basic_math_functions#erf</a> )</p>
<p>Thank you <span class="mention">@Soumith</span> and all devs !</p>

username: AjayTalati
cooked: <p>If you can do it with <code>torchtensors</code> then you can define your own custom autograd function.</p>
<p>It might be a little tedious, but we’ve all written our own custom functions and modules - that’s a lot of the fun <img src="//discuss.pytorch.org/images/emoji/apple/smile.png?v=5" title=":smile:" class="emoji" alt=":smile:"> .</p>
<p>It’s definitely worth doing, once you’ve done one, it’s a lot less scary - good luck <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: tom
cooked: <p>Hello <a class="mention" href="/u/cerisara">@cerisara</a>,</p>
<p>if you can live with an approximation, you could use the following (in the formula with the square root here: <a href="https://en.wikipedia.org/wiki/Error_function#Approximation_with_elementary_functions" rel="nofollow noopener">https://en.wikipedia.org/wiki/Error_function#Approximation_with_elementary_functions</a> )</p>
<pre><code class="lang-auto">import torch
a_for_erf = 8.0/(3.0*numpy.pi)*(numpy.pi-3.0)/(4.0-numpy.pi)
def erf_approx(x):
    return torch.sign(x)*torch.sqrt(1-torch.exp(-x*x*(4/numpy.pi+a_for_erf*x*x)/(1+a_for_erf*x*x)))
def erfinv_approx(x):
    b = -2/(numpy.pi*a_for_erf)-torch.log(1-x*x)/2
    return torch.sign(x)*torch.sqrt(b+torch.sqrt(b*b-torch.log(1-x*x)/a_for_erf))</code></pre>
<p>I must admit I don’t have a particular reason to use x*x for x**2 except that I copy-pasted it from a C version I typed up a couple of years ago.<br>
If you feed <code>Variable</code>s that should work as well.</p>
<p>To get an impression of how they look, you could plot it against the scipy.special functions like</p>
<pre><code class="lang-auto">from matplotlib import pyplot
%matplotlib inline
import scipy.special
x = numpy.linspace(-2,2,100)
pyplot.subplot(1,2,1)
pyplot.title('erf')
pyplot.plot(x,erf_approx(torch.from_numpy(x)).numpy(), label="approx")
pyplot.plot(x,scipy.special.erf(x),'--', label="scipy")
pyplot.legend()
pyplot.subplot(1,2,2)
y = scipy.special.erf(x)
pyplot.title('erfinv')
pyplot.plot(y,erfinv_approx(torch.from_numpy(y)).numpy(), label="approx")
pyplot.plot(y,scipy.special.erfinv(y),'--', label="scipy")
pyplot.legend()</code></pre>
<p>(the %matplotlib is for jupyter).</p>
<p>Best regards</p>
<p>Thomas</p>

username: AjayTalati
cooked: <p>Hi <a class="mention" href="/u/tom">@tom</a>,</p>
<p>nice work !!!</p>
<p>I found a very simple implementation of the sinkhorn-knopp matrix normalisation, we were looking at a while ago. It’s in MATLAB, but it’s very understandable,</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <a href="https://www.mathworks.com/matlabcentral/fileexchange/52930-sinkhorn-knopp-algorithm-for-matrix-normalisation?focused=5332757&amp;tab=function" target="_blank" rel="nofollow noopener">mathworks.com</a>
  </header>
  <article class="onebox-body">
    

<h3><a href="https://www.mathworks.com/matlabcentral/fileexchange/52930-sinkhorn-knopp-algorithm-for-matrix-normalisation?focused=5332757&amp;tab=function" target="_blank" rel="nofollow noopener">Sinkhorn-Knopp algorithm for matrix normalisation - File Exchange - MATLAB...</a></h3>

<p>Normalises a matrix so that the row and column sums are all unity</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Hopefully I’ll try to implement it in <code>torch</code> this weekend - that should be a lot of fun, and useful too.</p>
<p>All the best,</p>
<p>Aj</p>

username: cerisara
cooked: <p>Great !</p>
<p>Thank you, very useful !</p>

username: zhangxiangxiao
cooked: <p>One use case that I needed was to sample from a Gaussian VAE and do interpolation in uniform distributed variables. That is because u = erfinv(v) follows normal distribution if v follows uniform distribution via a technique called inverse sampling.</p>

username: ira
cooked: <p>These functions are now in master: <a href="https://github.com/pytorch/pytorch/pull/2799" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/2799</a>  <img src="//discuss.pytorch.org/images/emoji/apple/relaxed.png?v=5" title=":relaxed:" class="emoji" alt=":relaxed:"></p>

username: tom
cooked: <p>Awesome, thank you!</p>
<p>Best regards</p>
<p>Thomas</p>

username: MAsad
cooked: <p>Can you point me to the current code for erf and erfinv? I want to see how they and their gradients have been approximated.<br>
Many thanks!</p>