TST: reduce memory usage of fft test

tests should not use more than 500 MB of memory, this one uses more than
a GB. Also it didn't join its threads.