Does torchaudio.transforms.spectrogram work correctly if n_fft > win_size

username: Mole_m7b5
cooked: <p>I’m using a custom loop to get the STFT of an audio signal, while doing some processing on a frame by frame basis. The number of resulting frames corresponds to the formula<br>
<code>number_of_samples = 1440000 + winsize</code> I pad either end of the signal with <code>winsize //2</code><br>
<code>number_of_frames = (number_of_samples)//(window_size - window_size//2)</code><br>
<code>%  = (1440000 + 1024)//(1024 - 512)</code><br>
<code>%  = 2814</code></p>
<p>During the loop to prevent aliasing I zero pad the end of both the window and input frame by length <code>winsize</code> but my hopsize still remains at 512. This extra zero padding shouldn’t affect the number of frames as it’s done on a frame by frame basis to the current time domain segment under analysis.</p>
<p>I then want to use <code>torchaudio.transforms.Melspectrogram</code> on the original audio signal - this would result in me having two STFT signals, one with linearly spaced frequency bins and one with mel spaced. However the number of frames outputted from the transform is not as expected depending on the value of <code>n_fft</code>. With the <code>n_fft  = winsize</code> and <code>center=True</code> it outputs 2816 frames and with <code>center=False</code> it outputs the expected 2814. However if <code>n_fft = 2048</code> and <code>winsize = 1024</code> it outputs 2812 frames. I can’t work out why <code>n_fft</code> would effect the number of total frames if frames are based on signal length, window size, and hopsize.</p>

username: nateanl
cooked: <p>Hi <a class="mention" href="/u/mole_m7b5">@Mole_m7b5</a>, thanks for posting the question. The number of frames is hard coded in <a href="https://github.com/pytorch/pytorch/blob/6285348f06f59b7ed31bd57254dc1b9ccfb4e0b3/aten/src/ATen/native/SpectralOps.cpp#L862" rel="noopener nofollow ugc">torch.stft</a></p>
<pre><code class="lang-auto">  int64_t n_frames = 1 + (len - n_fft) / hop_length;
  // time2col
  input = input.as_strided(
    {batch, n_frames, n_fft},
    {input.stride(0), hop_length * input.stride(1), input.stride(1)}
  );
  if (window_.defined()) {
    input = input.mul(window_);
  }
</code></pre>
<p>If you increase the value of <code>n_fft</code>, the number of frames will be decreased.</p>

username: Mole_m7b5
cooked: <p>Hi <a class="mention" href="/u/nateanl">@nateanl</a>, thanks for the response.</p>
<p>How does that interact when having a value <code>n_fft &gt; win_length</code>? Does it mean the FFT window actually extends outside of the segment covered by your (for example) Hann window?  Or is the <code>win_length</code> overridden to match <code>n_fft</code>?</p>

username: nateanl
cooked: <p>if <code>win_length</code> is shorter than <code>n_fft</code>, it will be zero padded on both sides to match the value of <code>n_fft</code>. When computing stft, only the samples multiplying the non-zero window values are used.</p>
<p>In other words, if your <code>win_length</code> and <code>hop_length</code> keep the same, and <code>n_fft</code> is increased, the frequency axis will have higher resolution (be up-sampled).</p>

username: Mole_m7b5
cooked: <p>That makes sense.</p>
<p>But if the <code>n_fft</code> value is matched by padding on a frame by frame basis after the windowed section has been grabbed, and only the original windowed section of the signal is transformed (but at a higher fft resolution due to the zero-padding), would it make more sense for the number of output frames to be hardcoded according to <code>win_length</code> and <code>hop_length</code>? Or is there a particular reason it’s hardcoded according to the n_fft?</p>

username: nateanl
cooked: <p>The design follows that of <a href="https://github.com/librosa/librosa/blob/eb603e7a91598d1e72d3cdeada0ade21a33f9c0c/librosa/core/spectrum.py#L240" rel="noopener nofollow ugc"><code>librosa stft</code></a>, which also uses <code>n_fft</code> to detect number of frames.</p>
<p>There’s also a discussion about <code>win_length</code> and <code>n_fft</code> you might be interested: <a href="https://github.com/librosa/librosa/issues/695" class="inline-onebox" rel="noopener nofollow ugc">Semantics of n_fft, window length, and frame length · Issue #695 · librosa/librosa · GitHub</a></p>

username: hossein
cooked: <p>Hi<br>
I can’t calculate the n =_frames in my case.<br>
The length of my sample is 90000 and n_fft = 1024, hop_length = 128. According to the formula, the resulting n_frame must be roughly = 696. But torch returns a matrix of n_frames = 704!</p>

username: nateanl
cooked: <p>Hi <a class="mention" href="/u/hossein">@hossein</a>, it is possible that num_frames returns 704, if you set <code>center=True</code> in <code>torch.stft</code>, and it is <code>True</code> by default. What’s your input arguments settings in <code>torch.stft</code>?</p>

username: hossein
cooked: <p>Actually my code is <code>T.Spectrogram(n_fft = 1024 , hop_length = 128)</code>. My soundwave has a shape of (1,90000). I can’t get 704!</p>

username: nateanl
cooked: <p>I see, Here is what I tried with T.spectrogram:</p>
<pre><code class="lang-python">import torchaudio
import torch
waveform = torch.rand(1, 90000)
transform = torchaudio.transforms.Spectrogram(n_fft=1024, hop_length=128)
spectrogram = transform(waveform)
print(spectrogram.shape)
</code></pre>
<p>The output is</p>
<pre><code class="lang-auto">torch.Size([1, 513, 704])
</code></pre>