Repository: pytorch/audio

Issue:
Dithering constant
Why do `torchaudio.compliance.kaldi.fbank` and `torchaudio.compliance.kaldi.spectrogram` have so large `dither` default parameter (=1.0)? It very often just noises full output.

It's common to use dither around 0, e.g 0.00001 in QuartzNet, Jasper -- near to SOTA ASR models (https://github.com/NVIDIA/NeMo/blob/master/examples/asr/configs/quartznet15x5.yaml).

I want to notice that even in torchaudio tutorial we have dither = 0.0: https://pytorch.org/tutorials/beginner/audio_preprocessing_tutorial.html.

Also look at this issue and how it was resolved: https://github.com/pytorch/audio/issues/157



Discussion:
> Why do `torchaudio.compliance.kaldi.fbank` and `torchaudio.compliance.kaldi.spectrogram` have so large `dither` default parameter (=1.0)? It very often just noises full output.

Can you provide an example of code with noisy output using default value?

> It's common to use dither around 0, e.g 0.00001 in QuartzNet, Jasper -- near to SOTA ASR models (https://github.com/NVIDIA/NeMo/blob/master/examples/asr/configs/quartznet15x5.yaml).

Can you provide Kaldi's or other software's default value?

About example: as I already mentioned, @vincentqb, check this your issue https://github.com/pytorch/audio/issues/157
Thanks for pointing this out. We should make sure that`spectrogram`, `fbank`, and `mfcc` uses the same default. 

From #157, it does seem like a value of 1 is large. If `dither` is set to zero though, the user should specify the `energy_floor`. Thoughts on what could be a good default, and what other softwares do?
Addresses part of #263
Second this, the default right now makes the whole `torchaudio.compliace.kaldi` features totally unusable out-of-the-box. 
I spent one hour looking at possible bugs on labels only to find out that basically my model was fed noise because of the dither default value.
@popcornell I know that feel bro (the same problem I had had and after that I created this issue)
@popcornell @Oktai15 -- We're looking at what would be a good value to use. What would you say would be a reasonable nonzero value? What values do other packages use, and that you like?

@cpuhrsch -- I would have used 0. by default but the implementation says explicitly to specify `energy_floor` in that case, see [here](https://github.com/pytorch/audio/pull/119/files#r386478033). Do you have more context? Note that all the test where done with `dither=0.`: see [here](https://github.com/pytorch/audio/pull/119/files#r386496266) for the original testing, and [here](https://github.com/pytorch/audio/tree/master/test/assets/kaldi) where the second parameter (dither) in the file names for testing are all 0., e.g. `spec-XXX-0-...`.

In the absence of more information, I'd suggest `dither=1e-5`.

```python
import torchaudio
import matplotlib
%matplotlib inline
import matplotlib.pyplot as plt

filename = "steam-train-whistle-daniel_simon.mp3"
s, sr = torchaudio.load(filename)

spec0 = torchaudio.transforms.Spectrogram()(s)[0]
plt.imshow(spec0.log2().numpy(), cmap='gray')
plt.show()

spec1 = torchaudio.compliance.kaldi.spectrogram(s, dither=0.)
plt.imshow(spec1.t().numpy(), cmap='gray')
plt.show()

spec2 = torchaudio.compliance.kaldi.spectrogram(s, dither=1e-5)
plt.imshow(spec2.t().numpy(), cmap='gray')
plt.show()

# Mean absolute percent difference 
print(2*((spec1 - spec2).abs()/(spec1.abs() + spec2.abs())).mean())
# We see an average absolute percentage difference of 0.25%.
```
Based on [this discussion](https://github.com/pytorch/audio/pull/119/files#r386545216), we'll simply set `dither` to 0 and `energy_floor` to 1 by default. This also seems to behave very closely to a small value of dither, see below.
```
import torchaudio
import matplotlib
%matplotlib inline
import matplotlib.pyplot as plt

filename = "/Users/vincentqb/audio/test/assets/steam-train-whistle-daniel_simon.mp3"
s, sr = torchaudio.load(filename)

spec0 = torchaudio.transforms.Spectrogram()(s)[0]
plt.imshow(spec0.log2().numpy(), cmap='gray')
plt.show()

spec1 = torchaudio.compliance.kaldi.spectrogram(s, dither=0., energy_floor=1.)
plt.imshow(spec1.t().numpy(), cmap='gray')
plt.show()

spec2 = torchaudio.compliance.kaldi.spectrogram(s, dither=1e-6)
plt.imshow(spec2.t().numpy(), cmap='gray')
plt.show()

# Mean absolute percent difference 
print(2*((spec1 - spec2).abs()/(spec1.abs() + spec2.abs())).mean())
# We see an average absolute percentage difference of 0.16%.
```

Resolving patch:
diff --git a/torchaudio/compliance/kaldi.py b/torchaudio/compliance/kaldi.py
--- a/torchaudio/compliance/kaldi.py
+++ b/torchaudio/compliance/kaldi.py
@@ -194,7 +194,7 @@ def _subtract_column_mean(tensor, subtract_mean):
 
 
 def spectrogram(
-        waveform, blackman_coeff=0.42, channel=-1, dither=1.0, energy_floor=0.0,
+        waveform, blackman_coeff=0.42, channel=-1, dither=0.0, energy_floor=1.0,
         frame_length=25.0, frame_shift=10.0, min_duration=0.0,
         preemphasis_coefficient=0.97, raw_energy=True, remove_dc_offset=True,
         round_to_power_of_two=True, sample_frequency=16000.0, snip_edges=True,
@@ -207,10 +207,10 @@ def spectrogram(
         blackman_coeff (float): Constant coefficient for generalized Blackman window. (Default: ``0.42``)
         channel (int): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
         dither (float): Dithering constant (0.0 means no dither). If you turn this off, you should set
-            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``1.0``)
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
         energy_floor (float): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
             this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
-            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``0.0``)
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
         frame_length (float): Frame length in milliseconds (Default: ``25.0``)
         frame_shift (float): Frame shift in milliseconds (Default: ``10.0``)
         min_duration (float): Minimum duration of segments to process (in seconds). (Default: ``0.0``)
@@ -429,7 +429,7 @@ def get_mel_banks(num_bins, window_length_padded, sample_freq,
 
 
 def fbank(
-        waveform, blackman_coeff=0.42, channel=-1, dither=1.0, energy_floor=0.0,
+        waveform, blackman_coeff=0.42, channel=-1, dither=0.0, energy_floor=1.0,
         frame_length=25.0, frame_shift=10.0, high_freq=0.0, htk_compat=False, low_freq=20.0,
         min_duration=0.0, num_mel_bins=23, preemphasis_coefficient=0.97, raw_energy=True,
         remove_dc_offset=True, round_to_power_of_two=True, sample_frequency=16000.0,
@@ -443,10 +443,10 @@ def fbank(
         blackman_coeff (float): Constant coefficient for generalized Blackman window. (Default: ``0.42``)
         channel (int): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
         dither (float): Dithering constant (0.0 means no dither). If you turn this off, you should set
-            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``1.0``)
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
         energy_floor (float): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
             this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
-            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``0.0``)
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
         frame_length (float): Frame length in milliseconds (Default: ``25.0``)
         frame_shift (float): Frame shift in milliseconds (Default: ``10.0``)
         high_freq (float): High cutoff frequency for mel bins (if <= 0, offset from Nyquist) (Default: ``0.0``)
@@ -547,8 +547,8 @@ def _get_lifter_coeffs(num_ceps, cepstral_lifter):
 
 
 def mfcc(
-        waveform, blackman_coeff=0.42, cepstral_lifter=22.0, channel=-1, dither=1.0,
-        energy_floor=0.0, frame_length=25.0, frame_shift=10.0, high_freq=0.0, htk_compat=False,
+        waveform, blackman_coeff=0.42, cepstral_lifter=22.0, channel=-1, dither=0.0,
+        energy_floor=1.0, frame_length=25.0, frame_shift=10.0, high_freq=0.0, htk_compat=False,
         low_freq=20.0, num_ceps=13, min_duration=0.0, num_mel_bins=23, preemphasis_coefficient=0.97,
         raw_energy=True, remove_dc_offset=True, round_to_power_of_two=True,
         sample_frequency=16000.0, snip_edges=True, subtract_mean=False, use_energy=False,
@@ -562,10 +562,10 @@ def mfcc(
         cepstral_lifter (float): Constant that controls scaling of MFCCs (Default: ``22.0``)
         channel (int): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
         dither (float): Dithering constant (0.0 means no dither). If you turn this off, you should set
-            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``1.0``)
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
         energy_floor (float): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
             this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
-            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``0.0``)
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
         frame_length (float): Frame length in milliseconds (Default: ``25.0``)
         frame_shift (float): Frame shift in milliseconds (Default: ``10.0``)
         high_freq (float): High cutoff frequency for mel bins (if <= 0, offset from Nyquist) (Default: ``0.0``)


Test patch: