Repository: pytorch/audio

Issue:
soundfile save test failing
## 🐛 Bug

I've tried to build torchaudio from the source but one test is consistently failing (backend/soundfile/save_test.py::TestFileObject::test_fileobj_flac). This happens on both my Linux and OSX machine.

## To Reproduce

Steps to reproduce the behavior:

1. `git clone https://github.com/pytorch/audio`
2. `cd audio`
3. `git submodule update --init --recursive`
4. `conda create -n temp2 -c pytorch-nightly pytorch`
5. `conda activate temp2`
6. `pip install typing pytest scipy numpy parameterized pyyaml`
7. `NO_CUDA=1 BUILD_SOX=1 MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install` (for linux machine, it is build with `BUILD_SOX=1 python setup.py install`)
8. `export PATH="${PATH_TO_TORCHAUDIO}/audio/third_party/install/bin:${PATH}"`
9. `cd test/torchaudio_unittest`
10. `Pip install soundfile`
11. `pytest backend/soundfile/save_test.py` (more specifically `pytest backend/soundfile/save_test.py::TestFileObject::test_fileobj_flac`)

Here is the pytest message by running (`pytest backend/soundfile/save_test.py`).

```bash
============================================================================================================================ test session starts =============================================================================================================================
platform linux -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /data/home/yyyang/temp/audio
plugins: xdist-2.3.0, forked-1.3.0
collected 958 items

backend/soundfile/save_test.py ....................................................................................................................................................................................................................................... [ 24%]
...................................................................................................................................................................................................................................................................... [ 51%]
...................................................................................................................................................................................................................................................................... [ 78%]
.......................................................................................................................................................................................................F...                                                            [100%]
================================================================================================================================== FAILURES ==================================================================================================================================
______________________________________________________________________________________________________________________ TestFileObject.test_fileobj_flac ______________________________________________________________________________________________________________________

self = <torchaudio_unittest.backend.soundfile.save_test.TestFileObject testMethod=test_fileobj_flac>

    @skipIfFormatNotSupported("FLAC")
    @skipIfRocm
    def test_fileobj_flac(self):
        """Saving audio via file-like object works"""
>       self._test_fileobj('flac')

backend/soundfile/save_test.py:287:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
backend/soundfile/save_test.py:277: in _test_fileobj
    self.assertEqual(expected, found, atol=1e-4, rtol=1e-8)
/fsx/users/yyyang/conda/envs/torchaudio-upstream/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py:1407: in assertEqual
    self.assertEqual(x_, y_, atol=atol, rtol=rtol, msg=msg,
/fsx/users/yyyang/conda/envs/torchaudio-upstream/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py:1407: in assertEqual
    self.assertEqual(x_, y_, atol=atol, rtol=rtol, msg=msg,
/fsx/users/yyyang/conda/envs/torchaudio-upstream/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py:1419: in assertEqual
    super().assertTrue(result, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
E   AssertionError: False is not true : Scalars failed to compare as equal! Comparing -0.99993896484375 and 1.00006103515625 gives a difference of 2.0, but the allowed difference with rtol=1e-08 and atol=0.0001 is only 0.00010001000061035157!
============================================================================================================================== warnings summary ==============================================================================================================================
test/torchaudio_unittest/backend/soundfile/save_test.py: 10 warnings
  /data/home/yyyang/temp/audio/test/torchaudio_unittest/common_utils/wav_utils.py:77: WavFileWarning: Chunk (non-data) not understood, skipping it.
    sample_rate, data = scipy.io.wavfile.read(path)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================================================== short test summary info ===========================================================================================================================
FAILED backend/soundfile/save_test.py::TestFileObject::test_fileobj_flac - AssertionError: False is not true : Scalars failed to compare as equal! Comparing -0.99993896484375 and 1.00006103515625 gives a difference of 2.0, but the allowed difference with rtol=1e-08 a...
================================================================================================================= 1 failed, 957 passed, 10 warnings in 6.80s =================================================================================================================
```


## Expected behavior

The test should pass.

## Environment

 - What commands did you used to install torchaudio (conda/pip/build from source)?
 see above
 - If you are building from source, which commit is it?
 533c2d47f7c03370a26b9744f9ec2093857fc43a
 - What does `torchaudio.__version__` print? (If applicable)
```
>>> soundfile.__version__
'0.10.3'
>>> torchaudio.__version__
'0.10.0a0+533c2d4'
>>> scipy.__version__
'1.6.3'
```

OSX machine:
```
Collecting environment information...
PyTorch version: 1.10.0.dev20210618
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.4 (x86_64)
GCC version: Could not collect
Clang version: 12.0.5 (clang-1205.0.22.9)
CMake version: version 3.20.4
Libc version: N/A

Python version: 3.9 (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.10.0.dev20210618
[pip3] torchaudio==0.10.0a0+533c2d4
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.2.0           hecd8cb5_269
[conda] numpy                     1.20.3                   pypi_0    pypi
[conda] pytorch                   1.10.0.dev20210618         py3.9_0    pytorch-nightly
[conda] torchaudio                0.10.0a0+533c2d4          pypi_0    pypi
```

Linux machine:
```
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.19.6
Libc version: glibc-2.27

Python version: 3.9 (64-bit runtime)
Python platform: Linux-5.4.0-1041-aws-x86_64-with-glibc2.27
Is CUDA available: False
CUDA runtime version: 11.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.10.0.dev20210618
[pip3] torchaudio==0.10.0a0+533c2d4
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              hfd86e86_1
[conda] mkl                       2021.2.0           h06a4308_296
[conda] numpy                     1.20.3                   pypi_0    pypi
[conda] pytorch                   1.10.0.dev20210618 py3.9_cuda10.2_cudnn7.6.5_0    pytorch-nightly
[conda] torchaudio                0.10.0a0+533c2d4           dev_0    <develop>
```

## Additional context




Discussion:
What will happen if you change the following line to `return None`?

https://github.com/pytorch/audio/blob/533c2d47f7c03370a26b9744f9ec2093857fc43a/torchaudio/backend/soundfile_backend.py#L286
With this change, the test would pass in both linux and OSX.
Is this a bug or just a version issue?
It means that the reference data created in test is different from the current implementation of `torchaudi.save`.
First, we need to check which one is correct. If you have time to look into this, here is the step;

1. Get the signal generated during the test.
The audio data is saved in byte buffer at https://github.com/pytorch/audio/blob/533c2d47f7c03370a26b9744f9ec2093857fc43a/test/torchaudio_unittest/backend/soundfile/save_test.py#L273
So dump the content into a file.
Something like the following,
    ```python
    with open(self.get_temp_path(f'result.flac'), 'wb') as file_:
        file_.write(fileobj.read())
    ```
2. Run the test with `TORCHAUDIO_TEST_TEMP_DIR=<some_directory> pytest torchaudio_unittest.backend.soundfile.save_test.TestFileObject -k flac` so that intermediate files are saved after the test run.
3. Open the `test.flac` and `result.flac` with Audacity and check the signal shape. They are supposed to be linear.
Here is what I got from opening these two files with Audacity.
![Screen Shot 2021-06-21 at 8 28 14 AM](https://user-images.githubusercontent.com/4192725/122983147-878de580-d350-11eb-97d9-c2b1c96e850e.png)


I appears that soundfile is using "PCM_16" as default ([link](https://github.com/bastibe/python-soundfile/blob/744efb4b01abc72498a96b09115b42a4cabd85e4/soundfile.py#L124)) when None is passed to subtype, and the default for PCM_24 is used for torchaudio.
I think the fix is to change this line
https://github.com/pytorch/audio/blob/533c2d47f7c03370a26b9744f9ec2093857fc43a/test/torchaudio_unittest/backend/soundfile/save_test.py#L266
to something like
```
        subtype = None
        if ext == 'wav':
            subtype = 'FLOAT'
        elif ext == 'flac':
            subtype = 'PCM_24
```


Resolving patch:
diff --git a/torchaudio/backend/soundfile_backend.py b/torchaudio/backend/soundfile_backend.py
--- a/torchaudio/backend/soundfile_backend.py
+++ b/torchaudio/backend/soundfile_backend.py
@@ -283,7 +283,7 @@ def _get_subtype(
         if encoding:
             raise ValueError("flac does not support encoding.")
         if not bits_per_sample:
-            return "PCM_24"
+            return "PCM_16"
         if bits_per_sample > 24:
             raise ValueError("flac does not support bits_per_sample > 24.")
         return "PCM_S8" if bits_per_sample == 8 else f"PCM_{bits_per_sample}"
@@ -382,8 +382,8 @@ def save(
 
     ``"flac"``
         - 8-bit
-        - 16-bit
-        - 24-bit (default)
+        - 16-bit (default)
+        - 24-bit
 
     ``"ogg"``, ``"vorbis"``
         - Doesn't accept changing configuration.
@@ -416,6 +416,9 @@ def save(
 
     if bits_per_sample not in (None, 8, 16, 24, 32, 64):
         raise ValueError("Invalid bits_per_sample.")
+    if bits_per_sample == 24:
+        warnings.warn("Saving audio with 24 bits per sample might warp samples near -1. "
+                      "Using 16 bits per sample might be able to avoid this.")
     subtype = _get_subtype(src.dtype, ext, encoding, bits_per_sample)
 
     # sph is a extension used in TED-LIUM but soundfile does not recognize it as NIST format,


Test patch: