Repository: pytorch/audio

Issue:
Using MelSpec when power=None
## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:

1. Use MelSpectrogram(..., power=None)

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
```
mel_spec = MelSpectrogram(
            sample_rate=self.feat_params["sample_rate"],
            n_fft=self.feat_params["n_window"],
            win_length=self.feat_params["n_window"],
            hop_length=self.feat_params["hop_length"],
            f_min=self.feat_params["f_min"],
            f_max=self.feat_params["f_max"],
            n_mels=self.feat_params["n_mels"],
            window_fn=torch.hamming_window,
            wkwargs={"periodic": False},
            power=None,
            )
mel_utt = mel_spec(utt_in_time)
```
<!-- A clear and concise description of what you expected to happen. -->

## Environment

 - What commands did you used to install torchaudio (conda/pip/build from source)?
 - If you are building from source, which commit is it?
 - What does `torchaudio.__version__` print? (If applicable)

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

 - PyTorch Version (e.g., 1.0): 1.8.1+cu111
 - Torchaudio: 0.8.1
 - OS (e.g., Linux): Linux
 - How you installed PyTorch (`conda`, `pip`, source): pip
 - Build command you used (if compiling from source): from pytorch.org
 - Python version: 3.85
 - CUDA/cuDNN version: 11.1
 - GPU models and configuration: A-100
 - Any other relevant information:

## Additional context
I actually wanted to implement time-stretching here. Think what I shall do is to use Spectrogram(..., power=0) to transform time-domain signal into frequency domain with complex numbers, and then use TimeStretch to do the time stretching. If I want to get the Mel_spec, I shall use MelScale to the obtained data from TimeStrecth right?
Anyways, just found there is a bug when am using Mel_Spec with power=0.



Discussion:
Hi @SaoYear 

To stretch mel spectrogram, you should be able to do `Spectrogram(power=None)` -> `TimeStretch` -> `.abs().power(power)` -> `MelScale`.

MelScale conversion, in my understanding, is simply weighted sum over different frequency bins, and it is implemented as matrix multiplication between spectrogram and frequency bin matrix. The reason why we encounter the error is frequency bin matrix is real-valued, while raw spectrogram is complex-valued and we recently switched to native complex type (`float.complex32`) for raw spectrogram.
I was aware of this error but I am not yet sure what is the right fix. Specifically, I am not sure if Mel scale conversion is defined on raw (complex valued) spectrogram. If you know a reference of scientific definition of Mel scale conversion, please let me know.
If we fix the mel scale conversion for raw spectrogram, (matrix multiplication of raw spectrogram and frequency bin matrix) in a way that frequency bin matrix is converted to complex type with 0 for imaginary value, I think the resulting modified spectrogram has a different norm than applying mel scale to a power spectrogram.
@carolineechen and I had a casual discussion and we are considering to change `MelSpectrogram`'s `power` annotation from `Optional[float]` to `float`.
What do you think?
Thanks! I have finished the time stretching part for Mel spec. 

As for the MFCC, I think a good reference might be SLP2 by J&M. 
https://github.com/rain1024/slp2-pdf/blob/master/chapter-wise-pdf/%5B09%5D%20Automatic%20Speech%20Recognition.pdf
Section 9.3. For my understanding, the Mel scale is designed for approaching human hearing, which is mostly related with energy only. So, it might be somehow weird to use complex spectrogram with Mel Scale. But... am not sure about that.

And I think changing optional float to float sounds like a great solution. It might reduce the confusion for some degree. 
@SaoYear 

Thanks for the link. Indeed the section 9.3.4 says `The result of FFT will be information about the amount of energy at each frequency band.`, so I think it's okay for us to only consider the case where `power` is not `None` for Mel scale.

@carolineechen Does this make sense to you?
@mthrok yup, I think it makes sense to only consider the case where `power` is not `None` for MelSpectrogram.

Resolving patch:
diff --git a/torchaudio/transforms.py b/torchaudio/transforms.py
--- a/torchaudio/transforms.py
+++ b/torchaudio/transforms.py
@@ -476,7 +476,7 @@ def __init__(self,
                  pad: int = 0,
                  n_mels: int = 128,
                  window_fn: Callable[..., Tensor] = torch.hann_window,
-                 power: Optional[float] = 2.,
+                 power: float = 2.,
                  normalized: bool = False,
                  wkwargs: Optional[dict] = None,
                  center: bool = True,


Test patch: