Fft performance

zkycaesar: <p>I’m comparing the performance of rfft in torch, numpy and scipy.<br>
The result is that, scipy is 1 time slower than torch, and numpy is 1time slower than scipy.<br>
That is strange. So I did another test. And here is the code:</p>
<pre><code class="lang-auto">    import torch
    import numpy as np
    import scipy
    data = np.random.random((10000000,)).astype(np.float32)
    print(data)
    print(data.dtype)
    lp_torch = torch.fft.rfftfreq(10000000, d=1./250000000) &lt; 1000000
    lp_scipy = scipy.fft.rfftfreq(10000000, d=1./250000000) &lt; 1000000
    lp_np = np.fft.rfftfreq(10000000, d=1./250000000) &lt; 1000000
    print('lp:')
    print(lp_torch)
    print(lp_scipy)
    print(lp_np)
    fft_torch = torch.fft.rfft(torch.tensor(data))
    fft_scipy = scipy.fft.rfft(data)
    fft_np = np.fft.rfft(data)
    print('fft:')
    print(fft_torch)
    print(fft_scipy)
    print(fft_np)
    print(fft_torch.dtype)
    print(fft_scipy.dtype)
    print(fft_np.dtype)
    pass_torch = torch.multiply(fft_torch, lp_torch)
    pass_scipy = scipy.multiply(fft_scipy, lp_scipy)
    pass_np = np.multiply(fft_np, lp_np)
    print('pass:')
    print(pass_torch)
    print(pass_scipy)
    print(pass_np)
    print(pass_torch.dtype)
    print(pass_scipy.dtype)
    print(pass_np.dtype)
    pass_torch = torch.fft.irfft(pass_torch)
    pass_scipy = scipy.fft.irfft(pass_scipy)
    pass_np = np.fft.irfft(pass_np)
    print('irfft:')
    print(pass_torch)
    print(pass_scipy)
    print(pass_np)
    print(pass_torch.dtype)
    print(pass_scipy.dtype)
    print(pass_np.dtype)
</code></pre>
<p>And the output:</p>
<pre><code class="lang-auto">[0.3066935  0.21171719 0.8174513  ... 0.88327384 0.20714097 0.36953172]
float32
lp:
tensor([ True,  True,  True,  ..., False, False, False])
[ True  True  True ... False False False]
[ True  True  True ... False False False]
fft:
tensor([ 5.0000e+06+0.0000j,  1.0524e+03-513.6312j,  3.9784e+02-411.0908j,
         ..., -3.0829e+02-346.9896j,  1.0980e+03+240.6152j,
        -9.4525e+02+0.0000j])
[ 5.0000470e+06  +0.j       1.0524075e+03-513.63116j
  3.9784344e+02-411.09076j ... -3.0828790e+02-346.98987j
  1.0979875e+03+240.6151j  -9.4500000e+02  +0.j     ]
[ 5.00004689e+06  +0.j          1.05240748e+03-513.63097737j
  3.97843491e+02-411.09077203j ... -3.08287811e+02-346.98967654j
  1.09798757e+03+240.61525509j -9.45161769e+02  +0.j        ]
torch.complex64
complex64
complex128
pass:
tensor([5.0000e+06+0.0000j, 1.0524e+03-513.6312j, 3.9784e+02-411.0908j,
         ..., 0.0000e+00-0.0000j, 0.0000e+00+0.0000j,
        -0.0000e+00+0.0000j])
[ 5.0000470e+06  +0.j       1.0524075e+03-513.63116j
  3.9784344e+02-411.09076j ...  0.0000000e+00  -0.j
  0.0000000e+00  +0.j      -0.0000000e+00  +0.j     ]
[ 5.00004689e+06  +0.j          1.05240748e+03-513.63097737j
  3.97843491e+02-411.09077203j ...  0.00000000e+00  -0.j
  0.00000000e+00  +0.j         -0.00000000e+00  +0.j        ]
torch.complex64
complex64
complex128
irfft:
tensor([0.4760, 0.4755, 0.4751,  ..., 0.4774, 0.4769, 0.4764])
[0.47597715 0.47552222 0.47507197 ... 0.4773674  0.4769001  0.47643661]
[0.47597718 0.47552222 0.47507193 ... 0.47736738 0.47690002 0.47643654]
torch.float32
float32
float64
</code></pre>
<p>I noticed that, the precision of their results are different. Numpy has the highest precision, and, although torch and scipy are all complex64, their results are still different.</p>
<p>So my question is: does torch do a float16 calculation during rfft?</p>
<p>I’m using torch 1.13.</p>

smth: <p>no, <code>torch</code> doesn’t do float16 calculations during <code>rfft</code> when your tensor is float32.<br>
the way they pad and stuff might be different between libraries, which might explain the differences.</p>