Repository: speechbrain/speechbrain

Issue:
sig_length in ISTFT
About the ISTFT() in speechbrain.processing.features:

Because the length of different utterances is different, maybe we should move "sig_length" from class initialization to def forward(self, x)? 


Discussion:
Yes, lengths change in every batch. @williamaris, could you take a look into that?
Yes! I will as soon as I can! Do you think that sig_length should be a tensor of size [n_batch] and the user gives a length for each batch?
The input of the ISTF should be a tensor with the following format: (batch, time, channels). The batches are created by gathering N_batch sentences and padding them such as they all have the same length in the time axis. In our ASR experiment (see for instance the losses in nnet), we pass in the input of the forward method an optional tensor that contains the relative length of each sentence.
The tensor is like ([0.5 0.6 0.8 1.0]) where 1.0 is the longest sequence. The reason why we pass relative length is that in this way we can retrieve the correct number of padded steps even if our network changes the temporal resolution across its layers. It could be great to be aligned with that..
The parameter should be optional. When not specified, it should consider batch.shape[1] (time axis) as the length of each signal. What do you think?

So if I understood properly, the forward( ) method should look like "def forward(self, x, sig_length)" and "sig_length" should be optional. All I need to do is to move "sig_length" from the initialization to the forward( ) method.

Currently, the ISTFT works like this, (could you tell me if that part is ok with you?):

- If sig_length is specified, the signals resulting of the ISTFT will have the following dimension: "[batch, sig_length, channels]". To adjust the length, the signals are padded or cropped if needed.

- If sig_length is NOT specified, things can get a little bit tricky because the reconstructed signals don't necessarily have the same length as the initial signals. The length will depend on the time frames' length and the hop length. The length is estimated like this: estimated_length = (time_frame_length - 1) * hop_length + n_fft.
I think it is fine.  The point is that sig_length is changing every time we change from one batch to another. It is thus necessary to update it every time using the parameter sig_len given in input to the forward method. We can even make it a mandatory parameter if managing the case in which it is not specified is too tricky.

Resolving patch:
diff --git a/speechbrain/processing/features.py b/speechbrain/processing/features.py
--- a/speechbrain/processing/features.py
+++ b/speechbrain/processing/features.py
@@ -208,9 +208,6 @@ class ISTFT(torch.nn.Module):
     hop_length : float
         Length (in ms) of the hope of the sliding window used when computing
         the STFT.
-    sig_length : int
-        The length of the output signal in number of samples. If not specified
-        will be equal to: (time_step - 1) * hop_length + n_fft
     window_type : str
         Window function used to compute the STFT ('bartlett','blackman',
         'hamming', 'hann', default: hamming).
@@ -249,7 +246,6 @@ def __init__(
         sample_rate,
         win_length=25,
         hop_length=10,
-        sig_length=None,
         window_type="hamming",
         normalized_stft=False,
         center=True,
@@ -260,7 +256,6 @@ def __init__(
         self.sample_rate = sample_rate
         self.win_length = win_length
         self.hop_length = hop_length
-        self.sig_length = sig_length
         self.window_type = window_type
         self.normalized_stft = normalized_stft
         self.center = center
@@ -277,13 +272,17 @@ def __init__(
 
         self.window = self._create_window()
 
-    def forward(self, x):
+    def forward(self, x, sig_length=None):
         """ Returns the ISTFT generated from the input signal.
 
         Arguments
         ---------
         x : tensor
             A batch of audio signals in the frequency domain to transform.
+
+        sig_length : int
+            The length of the output signal in number of samples. If not
+            specified will be equal to: (time_step - 1) * hop_length + n_fft
         """
 
         or_shape = x.shape
@@ -356,28 +355,28 @@ def forward(self, x):
             estimated_length -= n_fft
 
         # Adjusting the size of the output signal if needed
-        if self.sig_length is not None:
+        if sig_length is not None:
 
-            if self.sig_length > estimated_length:
+            if sig_length > estimated_length:
 
                 if len(or_shape) == 5:
                     padding = torch.zeros(
                         (
                             or_shape[0],
                             or_shape[4],
-                            self.sig_length - estimated_length,
+                            sig_length - estimated_length,
                         )
                     )
 
                 else:
                     padding = torch.zeros(
-                        (or_shape[0], self.sig_length - estimated_length)
+                        (or_shape[0], sig_length - estimated_length)
                     )
 
                 istft = torch.cat((istft, padding), -1)
 
-            elif self.sig_length < estimated_length:
-                istft = istft[..., 0 : self.sig_length]
+            elif sig_length < estimated_length:
+                istft = istft[..., 0:sig_length]
 
         if len(or_shape) == 5:
             istft = istft.transpose(1, 2)


Test patch:
diff --git a/tests/unittests/test_features.py b/tests/unittests/test_features.py
--- a/tests/unittests/test_features.py
+++ b/tests/unittests/test_features.py
@@ -37,6 +37,6 @@ def test_istft():
 
     compute_stft = STFT(sample_rate=fs)
     compute_istft = ISTFT(sample_rate=fs)
-    out = compute_istft(compute_stft(inp))
+    out = compute_istft(compute_stft(inp), sig_length=16000)
 
     assert torch.sum(torch.abs(inp - out) < 1e-6) >= inp.numel() - 5