Torch vs SciPy FFT gives different results

Vladik: <p>I’m trying to write a full <code>torch</code> implementation for scikit-image’s <a href="https://scikit-image.org/docs/stable/api/skimage.registration.html#skimage.registration.phase_cross_correlation" rel="noopener nofollow ugc">phase_cross_correlation</a> algorithm.</p>
<p>It’s fast as hell, and it kinda works but performs worse (quality-wise)<br>
I was trying to debug the issue and was horrified to realize that <code>torch.fft.fftn</code> and <code>scipy.fftpack.fftn</code> sometimes don’t return the same results, I’ll elaborate with an example:</p>
<p>Testing on random arrays, I get good results:</p>
<pre><code class="lang-auto">import torch
import scipy.fftpack as s_fft,
import numpy as np

np.random.seed(42)
rnd = np.random.rand(5_000, 32, 32, 3)
t_rnd = torch.from_numpy(rnd)

np_fft = s_fft.fftn(rnd)
torch_fft = torch.fft.fftn(t_rnd)

torch.testing.assert_allclose(torch.from_numpy(np_fft), torch_fft)
</code></pre>
<blockquote>
<p>AssertionError: Tensor-likes are not equal!</p>
<p>Mismatched elements: 15182859 / 15360000 (98.8%)<br>
Greatest absolute difference: 6.906214515135596e-10 at index (1000, 0, 0, 0)<br>
Greatest relative difference: 1.234703274957512e-11 at index (0, 0, 16, 0)</p>
</blockquote>
<p>Ok, some small numeric error is to be expected so this is basically equal.<br>
But then when I test with some real image data (e.g. CIFAR) I get this:</p>
<pre><code class="lang-auto">from torchvision.datasets import CIFAR10

ds = CIFAR10('cifar')
np_cifar = ds.data[:5_000]
cifar = torch.from_numpy(np_cifar)

np_fft = s_fft.fftn(np_cifar)
torch_fft = torch.fft.fftn(cifar)

torch.testing.assert_allclose(torch.from_numpy(np_fft), torch_fft)
</code></pre>
<blockquote>
<p>Mismatched elements: 15359985 / 15360000 (100.0%)<br>
Greatest absolute difference: 48.80068015457514 at index (1000, 0, 0, 0)<br>
Greatest relative difference: 0.003518688442983903 at index (999, 20, 27, 1)</p>
</blockquote>
<p>On my actual data (which I cannot share) it was as bad as 0.4 relative difference.<br>
And after performing a few more operations (like multiplying by the conjugate and taking the inverse FFT) the results differ quite widely.</p>
<p>So, what the hell???</p>
<p><strong>Environment:</strong><br>
numpy 1.22.4<br>
scipy 1.8.1<br>
torch 1.13.1+cpu<br>
CPU is 10th gen Intel i7</p>

ptrblck: <p>Try to narrow down if the mismatches are created at specific locations (in the batch and/or in their spatial dimensions) as I would guess the largest mismatches might come from the borders.</p>

Vladik: <p>I was suspicious of the borders as well, but there was nothing I could do about it.<br>
For my data at work, I managed to mitigate the issue by subtracting the mean of each item in the batch.<br>
I don’t even know why I tried it or why it worked, but it did.<br>
This approach didn’t work for me in the CIFAR example though.</p>
<p>Do you think I should open a git issue about this?</p>

eqy: <p>I don’t think there is an actual bug here, just that you are implicitly mixing dtype conventions between numpy and PyTorch. With the following modified code snippet:</p>
<pre><code class="lang-auto">import torch
from torchvision.datasets import CIFAR10
import scipy.fftpack as s_fft
import numpy as np

ds = CIFAR10('cifar', download=True)
np_cifar = ds.data[:5_000]
cifar = torch.from_numpy(np_cifar)
print(np_cifar.dtype)
print(cifar.dtype)

np_fft = s_fft.fftn(np_cifar)
torch_fft = torch.fft.fftn(cifar.double())

torch.testing.assert_allclose(torch.from_numpy(np_fft), torch_fft)
</code></pre>
<p>I get</p>
<pre><code class="lang-auto">Mismatched elements: 15320160 / 15360000 (99.7%)
Greatest absolute difference: 1.2410069878908557e-07 at index (2000, 0, 0, 0)
Greatest relative difference: 1.1741440376456833e-11 at index (4001, 12, 5, 2)
</code></pre>