torchaudio 11.0 yields different results than torchaudio 12.1 when loading MP3
Opened by patrickvonplaten (closed)

## Describe the bug

When loading Common Voice with torchaudio 0.11.0 the results are different to 0.12.1 which leads to problems in transformers see: https://github.com/huggingface/transformers/pull/18749

## Steps to reproduce the bug

If you run the following code once with `torchaudio==0.11.0+cu102` and `torchaudio==0.12.1+cu102` you can see that the tensors differ. This is a pretty big breaking change and makes some integration tests fail in Transformers.

```python
#!/usr/bin/env python3
from datasets import load_dataset
import datasets
import numpy as np
import torch
import torchaudio
print("torch vesion", torch.__version__)
print("torchaudio vesion", torchaudio.__version__)

save_audio = True
load_audios = False

if save_audio:
    ds = load_dataset("common_voice", "en", split="train", streaming=True)
    ds = ds.cast_column("audio", datasets.Audio(sampling_rate=16_000))
    ds_iter = iter(ds)
    sample = next(ds_iter)

    np.save(f"audio_sample_{torch.__version__}", sample["audio"]["array"])
    print(sample["audio"]["array"])

if load_audios:
    array_torch_11 = np.load("/home/patrick/audio_sample_1.11.0+cu102.npy")
    print("Array 11 Shape", array_torch_11.shape)
    print("Array 11 abs sum", np.sum(np.abs(array_torch_11)))
    array_torch_12 = np.load("/home/patrick/audio_sample_1.12.1+cu102.npy")
    print("Array 12 Shape", array_torch_12.shape)
    print("Array 12 abs sum", np.sum(np.abs(array_torch_12)))
```

Having saved the tensors the print output yields:

```
torch vesion 1.12.1+cu102
torchaudio vesion 0.12.1+cu102
Array 11 Shape (122880,)
Array 11 abs sum 1396.4988
Array 12 Shape (123264,)
Array 12 abs sum 1396.5193
```

## Expected results
torchaudio 11.0 and 12.1 should yield same results.

## Actual results
See above.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.1.1.dev0
- Platform: Linux-5.18.10-76051810-generic-x86_64-with-glibc2.34
- Python version: 3.9.7
- PyArrow version: 6.0.1
- Pandas version: 1.4.2


Comments: Maybe we can just pass this along to torchaudio @lhoestq @albertvillanova ? It be great if you could investigate if the errors lies in datasets or in torchaudio.

torchaudio did a change in [0.12](https://github.com/pytorch/audio/releases/tag/v0.12.0) on MP3 decoding (which affects common voice):
> MP3 decoding is now handled by FFmpeg in sox_io backend. (https://github.com/pytorch/audio/pull/2419, https://github.com/pytorch/audio/pull/2428)
> - FFmpeg is now used as fallback in sox_io backend, and now MP3 decoding is handled by FFmpeg. To load MP3 audio with torchaudio.load, please install a compatible version of FFmpeg (Version 4 when using an official binary distribution).
> - Note that, whereas the previous MP3 decoding scheme pads the output audio, the new scheme does not. As a consequence, the new version returns shorter audio tensors.

Do we have a solution for this now? Should we just upgrade to `torchaudio 0.12.0` then? 

`datasets` supports `torchaudio` 0.12 if you have an environment that supports reading MP3 with `torchaudio`, i.e. if you have `ffmpeg>=4`

Closing as we no longer use `torchaudio` for decoding.