Repository: pytorch/audio

Issue:
iid_masks in Frequency and Time Masking
## 📚 Documentation of iid_masks Parameter in AxisMasking

Description of Problem
------
The Docstring and Documentation of Frequency and Axis Masking state the following for the 'iid_masks' Paramter:

> _iid_masks (bool): weather to apply the same mask to all
            the examples/channels in the batch. (Default: False)_

This explanation would let me conclude, that the default is using __different__ masks for all spectral representations in the batch.

However, the opposite is the case when using this on a 4D-Tensor (e.g. having a Batch dimension). Looking at the source code, the function _mask_along_axis_ is used. This function does not generate different masks as it also states in it's docstring.

The docstring of _iid_mask_along_axis_ __also__ states, that is is generating the same mask for all spectrograms. However, I am quite confident that this function is generating different masks for each spectrogram.

Concluding:
--------
Behaviour of the paramter is not as expected. _iid_mask_along_axis_ shoud be used instead of _mask_along_axis_.

Looking at the code I think at the moment it is not possible to create different masks for 3d tensors (e.g. without batch dimension).

Suggested Solution:
-------
* Fix Documentation that usage of Paramter is inversed
* Document that different masks only work for a tensor with batch dimension (not 100 % sure if I am right here)


Discussion:
Hi @Ektoplasmakugel 

Thanks for submitting the issue.

> The Docstring and Documentation of Frequency and Axis Masking state the following for the 'iid_masks' Paramter:
> 
> > _iid_masks (bool): weather to apply the same mask to all
> > the examples/channels in the batch. (Default: False)_
> 
> This explanation would let me conclude, that the default is using **different** masks for all spectral representations in the batch.
> 
> However, the opposite is the case when using this on a 4D-Tensor (e.g. having a Batch dimension). Looking at the source code, the function _mask_along_axis_ is used. This function does not generate different masks as it also states in it's docstring.

It indeed is confusing.

> The docstring of _iid_mask_along_axis_ **also** states, that is is generating the same mask for all spectrograms. However, I am quite confident that this function is generating different masks for each spectrogram.

Looking at the code, I think you are right and I suspect that the docstring for `mask_along_axis_iid` was copy-pasted from `mask_along_axis` but the part `All examples will have the same mask interval.` should have been removed.

> ## Concluding:
> Behaviour of the paramter is not as expected. _iid_mask_along_axis_ shoud be used instead of _mask_along_axis_.
> 
> Looking at the code I think at the moment it is not possible to create different masks for 3d tensors (e.g. without batch dimension).

I guess you talking about [here](https://github.com/pytorch/audio/blob/master/torchaudio/transforms.py#L801-L805), and I agree that this should be documented.
I think you know this but, for a workaround you can add batch dimension with `.unsqueeze(0)`.

> ## Suggested Solution:
> * Fix Documentation that usage of Paramter is inversed

I think changing the docsting to `whether to apply *different masks* to all the ...` will do. What do you think? If you have better suggestion, I am open to it. Also note that due to the concern for the backward compatibility, it's far easier to fix docstring rather than changing the implementation.

> * Document that different masks only work for a tensor with batch dimension (not 100 % sure if I am right here)

I think you are right. I will try to run the code just in case later.
Meanwhile would you like to make a pull request to change these?
I have not done this before, but give me some time and I will have a look and try to do that.

Resolving patch:
diff --git a/torchaudio/functional.py b/torchaudio/functional.py
--- a/torchaudio/functional.py
+++ b/torchaudio/functional.py
@@ -1442,7 +1442,6 @@ def mask_along_axis_iid(
     r"""
     Apply a mask along ``axis``. Mask will be applied from indices ``[v_0, v_0 + v)``, where
     ``v`` is sampled from ``uniform(0, mask_param)``, and ``v_0`` from ``uniform(0, max_v - v)``.
-    All examples will have the same mask interval.
 
     Args:
         specgrams (Tensor): Real spectrograms (batch, channel, freq, time)
diff --git a/torchaudio/transforms.py b/torchaudio/transforms.py
--- a/torchaudio/transforms.py
+++ b/torchaudio/transforms.py
@@ -779,6 +779,7 @@ class _AxisMasking(torch.nn.Module):
         mask_param (int): Maximum possible length of the mask.
         axis (int): What dimension the mask is applied on.
         iid_masks (bool): Applies iid masks to each of the examples in the batch dimension.
+            This option is applicable only when the input tensor is 4D.
     """
     __constants__ = ['mask_param', 'axis', 'iid_masks']
 
@@ -798,7 +799,6 @@ def forward(self, specgram: Tensor, mask_value: float = 0.) -> Tensor:
         Returns:
             Tensor: Masked spectrogram of dimensions (..., freq, time).
         """
-
         # if iid_masks flag marked and specgram has a batch dimension
         if self.iid_masks and specgram.dim() == 4:
             return F.mask_along_axis_iid(specgram, self.mask_param, mask_value, self.axis + 1)
@@ -812,10 +812,10 @@ class FrequencyMasking(_AxisMasking):
     Args:
         freq_mask_param (int): maximum possible length of the mask.
             Indices uniformly sampled from [0, freq_mask_param).
-        iid_masks (bool, optional): whether to apply the same mask to all
-            the examples/channels in the batch. (Default: ``False``)
+        iid_masks (bool, optional): whether to apply different masks to each
+            example/channel in the batch. (Default: ``False``)
+            This option is applicable only when the input tensor is 4D.
     """
-
     def __init__(self, freq_mask_param: int, iid_masks: bool = False) -> None:
         super(FrequencyMasking, self).__init__(freq_mask_param, 1, iid_masks)
 
@@ -826,10 +826,10 @@ class TimeMasking(_AxisMasking):
     Args:
         time_mask_param (int): maximum possible length of the mask.
             Indices uniformly sampled from [0, time_mask_param).
-        iid_masks (bool, optional): whether to apply the same mask to all
-            the examples/channels in the batch. (Default: ``False``)
+        iid_masks (bool, optional): whether to apply different masks to each
+            example/channel in the batch. (Default: ``False``)
+            This option is applicable only when the input tensor is 4D.
     """
-
     def __init__(self, time_mask_param: int, iid_masks: bool = False) -> None:
         super(TimeMasking, self).__init__(time_mask_param, 2, iid_masks)
 


Test patch: