Torch.stft and torch.istft with window function

username: pepper8362
cooked: <p>Hi,</p>
<p>I tried something like this:</p>
<pre><code class="lang-auto">x = torch.randn(100000)
x_stft = torch.stft(x, n_fft=1024, hop_length=256, win_length=1024, window=torch.hann_window(1024), return_complex=True)
x_reconstruct = torch.istft(x_stft, n_fft=1024, hop_length=256, win_length=1024, window=torch.hann_window(1024), return_complex=False)
</code></pre>
<p>Since hanning window is applied twice, I would expect <code>x_reconstruct</code> to be <code>1.5*x</code> (<code>1.5</code> is the sum of <code>torch.hann_window(1024)**2</code> with <code>hop_length = win_length/4</code>). But instead I find <code>x_reconstruct</code> is equal to <code>x</code>.</p>
<p>Do <code>torch.stft</code> and <code>torch.istft</code> perform some internal processing to normalize the result when a window function is applied, rather than simply multiply the window function and the signal? If yes, any details on the whole procedure would be appreciated (e.g.  Is this happened in<code> torch.stft</code>, or <code>torch.istft</code>, or both? what is the detailed normalization process the two functions perform?).</p>
<p>Thank you!</p>

username: JuanFMontesinos
cooked: <p>Torch stft matches that of libros so you can use it as reference.<br>
I don’t understand where the 1.5 comes from. In fact if you check the docs.<br>
Wrt the inverse:<br>
<a href="https://librosa.org/doc/main/generated/librosa.istft.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://librosa.org/doc/main/generated/librosa.istft.html</a><br>
They just run an iterative minimization process for which they need to know the original window.</p>
<p>Also it wouldn’t be an inverse If doesn’t match the original signal.</p>

username: pepper8362
cooked: <aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="161741">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>I don’t understand where the 1.5 comes from.</p>
</blockquote>
</aside>
<p>Here I mean that the weight of window function accumulates duing fft and ifft, and eventually it scales signals by a factor (and if the hop length is chosen correctly, this factor can be a constant). A quick demo is as follows:</p>
<pre><code class="lang-auto">import numpy as np

#simulate two taps with 512-point window and 256-point hop
window_size = 512
hop_size = window_size // 4
n_hops = 4
n_fft = 512
dummy_signal = np.random.randn(window_size+hop_size*n_hops)
dummp_signal_taps = [dummy_signal[idx:idx+window_size] for idx in range(0, hop_size*n_hops + 1, hop_size)]
win = np.hanning(window_size)

#fft with window function
dummy_signal_taps_fft = [np.fft.rfft(s*win, n=n_fft) for s in dummp_signal_taps]

#ifft with window function
dummy_signal_taps_reconstruct = [np.fft.irfft(s_fft, n=n_fft) * win for s_fft in dummy_signal_taps_fft]

#reconstruct the original signal
dummy_signal_reconstruct = np.zeros(window_size+hop_size*n_hops)
for i,s in enumerate(dummy_signal_taps_reconstruct):
     dummy_signal_reconstruct[i*hop_size:i*hop_size+window_size] += s

#now check the reliable part of the reconstructed signal
dummy_signal_reliable = dummy_signal[(n_hops-1)*hop_size:-(n_hops-1)*hop_size]
dummy_signal_reconstruct_reliable = dummy_signal_reconstruct[(n_hops-1)*hop_size:-(n_hops-1)*hop_size]

print(dummy_signal_reconstruct_reliable / dummy_signal_reliable)
</code></pre>
<p>The printed results will show that, after fft and ifft with hanning window and a hop length of a quarter of the window size, the reconstructed signal is scaled by a factor of approximately 1.5.</p>
<aside class="quote no-group quote-modified" data-username="JuanFMontesinos" data-post="2" data-topic="161741">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>Torch stft matches that of libros …<br>
They just run an iterative minimization process for which they need to know the original window.<br>
Also it wouldn’t be an inverse If doesn’t match the original signal.</p>
</blockquote>
</aside>
<p>Thank you for the information! I will give it a look!</p>

username: JuanFMontesinos
cooked: <p>Hmmm in fact you are right:</p>
<pre><code class="lang-auto">    # Normalize by sum of squared window
    ifft_window_sum = window_sumsquare(
        window=window,
        n_frames=n_frames,
        win_length=win_length,
        n_fft=n_fft,
        hop_length=hop_length,
        dtype=dtype,
    )

    approx_nonzero_indices = ifft_window_sum &gt; util.tiny(ifft_window_sum)
    y[..., approx_nonzero_indices] /= ifft_window_sum[approx_nonzero_indices]
</code></pre>
<p>There is a normalization step which I think aims to compensate the windowing (as it’s only used when computing the stft but not the fft). Anyway should look into it in depth.</p>

username: pepper8362
cooked: <p>Thanks! I will have a look on it.</p>