Repository: pytorch/audio

Issue:
libsox does not detect mp3 when extension is missing
## 🐛 Bug

Sox IO backend doesn't allow to load file with explicit filetype while doesn't offer auto detection.

## To Reproduce

Steps to reproduce the behavior:

```
>>> torchaudio.set_audio_backend("sox")
>>> torchaudio.load("/path/to/extensionless/file", filetype='mp3')
(tensor([[0., 0., 0.,  ..., 0., 0., 0.]]), 16000)
>>> torchaudio.set_audio_backend("sox_io")
>>> torchaudio.load("/path/to/extensionless/file", filetype='mp3')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: load() got an unexpected keyword argument 'filetype'
>>> torchaudio.load("/path/to/extensionless/file")
formats: can't determine type of file `/path/to/extensionless/file'
```

## Expected behavior

Expected an autodetect feature or allowance to pass an explicit format argument 

## Environment

torch/torchaudio build from source. master of v0.7.0 affected


Discussion:
Hi @orlando-labs

Thanks for the feedback. I will look into it.
Looks like this happens on specific formats, like `mp3`, but not on `wav` or `flac`.
Hi @orlando-labs 

Do you use other formats than `mp3` that I should be looking into?
Investigation Note:

On torchaudio side, error happens [here](https://github.com/pytorch/audio/blob/f3b9208f8ab970ce8bcf16b6137ade282053c4b1/torchaudio/csrc/sox_utils.cpp#L97), which was called [here](https://github.com/pytorch/audio/blob/f3b9208f8ab970ce8bcf16b6137ade282053c4b1/torchaudio/csrc/sox_effects.cpp#L103)

This happens because [`sox_open_read`](https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L608) had some internal error and returns null pointer.

The actual implementation of `sox_open_read` is found in [`open_read`](https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L460) and the `filetype` member is set at 

https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L559

but will be reset if an error happens.

https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L603

the error happens at

https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L539

This is because [`auto_detect_format`](https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L514) fails.

Interestingly, there is no mention to `mp3` in [`auto_detect_format`](https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.c#L43-L116). And MP3 detection sorely relies on file extension.

Note: In `formats.h`, MP3 is added to the list of supported formats.
    https://github.com/dmkrepo/libsox/blob/b9dd1a86e71bbd62221904e3e59dfaa9e5e72046/src/formats.h#L92-L93

Another way to fix is to add mp3 detection mechanism
- https://stackoverflow.com/a/11360513
- http://www.mp3-tech.org/programmer/frame_header.html
> Hi @orlando-labs
> 
> Do you use other formats than `mp3` that I should be looking into?

Hi, currently mp3 only
@orlando-labs 

I landed #1042, which adds header-based mp3 detection.
It will become available starting from tomorrow's nightly build (which happens at around 7:30 AM in Eastern Time)
I believe it works as long as the mp3 file is well-formatted, but can you try and tell if it works?
@vincentqb 

> (flagging as potentially BC-breaking if sox_io becomes the default in 0.8)

Oops I accidentally deleted the comment, but switching to sox_io *IS* BC-breaking so no need to flag something additionally.
@orlando-labs 

Did you have a chance to try out the nightly build?
I'll try it in a day
tried master torchaudio==0.8.0a0+3616961
and nightly torchaudio==0.8.0a0+cc04bbb
unfortunately, with no positive effect

```
Python 3.8.3 (default, Aug 31 2020, 16:03:14) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchaudio

>>> torchaudio.set_audio_backend("sox_io")

>>> torchaudio.load("/dataset/audio//50/pz/50pz2wwojr60uetqsyp40mmb48mi", format="mp3")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: load() got an unexpected keyword argument 'format'

>>> torchaudio.load("/dataset/audio//50/pz/50pz2wwojr60uetqsyp40mmb48mi")
formats: can't determine type of file `/bigdata/audio//50/pz/50pz2wwojr60uetqsyp40mmb48mi'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib64/python3.8/site-packages/torchaudio-0.8.0a0+3616961-py3.8-linux-x86_64.egg/torchaudio/backend/sox_io_backend.py", line 105, in load
    signal = torch.ops.torchaudio.sox_io_load_audio_file(
RuntimeError: Error loading audio file: failed to open file.

>>> torchaudio.set_audio_backend("sox")
/usr/local/lib64/python3.8/site-packages/torchaudio-0.8.0a0+3616961-py3.8-linux-x86_64.egg/torchaudio/backend/utils.py:59: UserWarning: "sox" backend is depredated and will be removed in 0.9.0. Please use "sox_io" backend.
  warnings.warn(

>>> torchaudio.load("/dataset/audio/50/pz/50pz2wwojr60uetqsyp40mmb48mi", filetype="mp3")
(tensor([[0.0000, 0.0000, 0.0000,  ..., 0.0018, 0.0062, 0.0067]]), 16000)
```

The file itself:

```
file /dataset/audio/50/pz/50pz2wwojr60uetqsyp40mmb48mi
/dataset/audio/50/pz/50pz2wwojr60uetqsyp40mmb48mi: Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v2,  24 kbps, 16 kHz, Monaural```
MP3 files I try were produced by default FFMPEG encoder
Can you provide the command used to generate/convert the file?
Sure, i can clearly reproduce the issue with just sine wave being generated with
```
ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -ar 16000 -f mp3 test_noext
...
torchaudio.load("test_noext")
formats: can't determine type of file `test_noext'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib64/python3.8/site-packages/torchaudio-0.8.0a0+3616961-py3.8-linux-x86_64.egg/torchaudio/backend/sox_io_backend.py", line 105, in load
    signal = torch.ops.torchaudio.sox_io_load_audio_file(
RuntimeError: Error loading audio file: failed to open file.
```
Thanks, confirmed the issue. I will come back with the solution.

Resolving patch:
diff --git a/torchaudio/backend/_soundfile_backend.py b/torchaudio/backend/_soundfile_backend.py
--- a/torchaudio/backend/_soundfile_backend.py
+++ b/torchaudio/backend/_soundfile_backend.py
@@ -45,6 +45,7 @@ def load(
     num_frames: int = -1,
     normalize: bool = True,
     channels_first: bool = True,
+    format: Optional[str] = None,
 ) -> Tuple[torch.Tensor, int]:
     """Load audio data from file.
 
@@ -99,6 +100,8 @@ def load(
         channels_first (bool):
             When True, the returned Tensor has dimension ``[channel, time]``.
             Otherwise, the returned Tensor's dimension is ``[time, channel]``.
+        format (str, optional):
+            Not used. PySoundFile does not accept format hint.
 
     Returns:
         torch.Tensor:
diff --git a/torchaudio/backend/sox_io_backend.py b/torchaudio/backend/sox_io_backend.py
--- a/torchaudio/backend/sox_io_backend.py
+++ b/torchaudio/backend/sox_io_backend.py
@@ -9,20 +9,27 @@
 
 
 @_mod_utils.requires_module('torchaudio._torchaudio')
-def info(filepath: str) -> AudioMetaData:
+def info(
+        filepath: str,
+        format: Optional[str] = None,
+) -> AudioMetaData:
     """Get signal information of an audio file.
 
     Args:
         filepath (str or pathlib.Path):
             Path to audio file. This function also handles ``pathlib.Path`` objects,
             but is annotated as ``str`` for TorchScript compatibility.
+        format (str, optional):
+            Override the format detection with the given format.
+            Providing the argument might help when libsox can not infer the format
+            from header or extension,
 
     Returns:
         AudioMetaData: Metadata of the given audio.
     """
     # Cast to str in case type is `pathlib.Path`
     filepath = str(filepath)
-    sinfo = torch.ops.torchaudio.sox_io_get_info(filepath)
+    sinfo = torch.ops.torchaudio.sox_io_get_info(filepath, format)
     return AudioMetaData(sinfo.get_sample_rate(), sinfo.get_num_frames(), sinfo.get_num_channels())
 
 
@@ -33,6 +40,7 @@ def load(
         num_frames: int = -1,
         normalize: bool = True,
         channels_first: bool = True,
+        format: Optional[str] = None,
 ) -> Tuple[torch.Tensor, int]:
     """Load audio data from file.
 
@@ -93,6 +101,10 @@ def load(
         channels_first (bool):
             When True, the returned Tensor has dimension ``[channel, time]``.
             Otherwise, the returned Tensor's dimension is ``[time, channel]``.
+        format (str, optional):
+            Override the format detection with the given format.
+            Providing the argument might help when libsox can not infer the format
+            from header or extension,
 
     Returns:
         torch.Tensor:
@@ -103,7 +115,7 @@ def load(
     # Cast to str in case type is `pathlib.Path`
     filepath = str(filepath)
     signal = torch.ops.torchaudio.sox_io_load_audio_file(
-        filepath, frame_offset, num_frames, normalize, channels_first)
+        filepath, frame_offset, num_frames, normalize, channels_first, format)
     return signal.get_tensor(), signal.get_sample_rate()
 
 
diff --git a/torchaudio/sox_effects/sox_effects.py b/torchaudio/sox_effects/sox_effects.py
--- a/torchaudio/sox_effects/sox_effects.py
+++ b/torchaudio/sox_effects/sox_effects.py
@@ -1,4 +1,4 @@
-from typing import List, Tuple
+from typing import List, Tuple, Optional
 
 import torch
 
@@ -157,6 +157,7 @@ def apply_effects_file(
         effects: List[List[str]],
         normalize: bool = True,
         channels_first: bool = True,
+        format: Optional[str] = None,
 ) -> Tuple[torch.Tensor, int]:
     """Apply sox effects to the audio file and load the resulting data as Tensor
 
@@ -180,6 +181,10 @@ def apply_effects_file(
             than integer WAV type.
         channels_first (bool): When True, the returned Tensor has dimension ``[channel, time]``.
             Otherwise, the returned Tensor's dimension is ``[time, channel]``.
+        format (str, optional):
+            Override the format detection with the given format.
+            Providing the argument might help when libsox can not infer the format
+            from header or extension,
 
     Returns:
         Tuple[torch.Tensor, int]: Resulting Tensor and sample rate.
@@ -249,5 +254,6 @@ def apply_effects_file(
     """
     # Get string representation of 'path' in case Path object is passed
     path = str(path)
-    signal = torch.ops.torchaudio.sox_effects_apply_effects_file(path, effects, normalize, channels_first)
+    signal = torch.ops.torchaudio.sox_effects_apply_effects_file(
+        path, effects, normalize, channels_first, format)
     return signal.get_tensor(), signal.get_sample_rate()


Test patch:
diff --git a/test/torchaudio_unittest/assets/mp3_without_ext b/test/torchaudio_unittest/assets/mp3_without_ext
new file mode 100644
Binary files /dev/null and b/test/torchaudio_unittest/assets/mp3_without_ext differ
diff --git a/test/torchaudio_unittest/sox_effect/sox_effect_test.py b/test/torchaudio_unittest/sox_effect/sox_effect_test.py
--- a/test/torchaudio_unittest/sox_effect/sox_effect_test.py
+++ b/test/torchaudio_unittest/sox_effect/sox_effect_test.py
@@ -8,6 +8,7 @@
     TempDirMixin,
     PytorchTestCase,
     skipIfNoExtension,
+    get_asset_path,
     get_sinusoid,
     get_wav_data,
     save_wav,
@@ -243,3 +244,21 @@ def test_vorbis(self, sample_rate, num_channels):
 
         assert sr == expected_sr
         self.assertEqual(found, expected)
+
+
+@skipIfNoExtension
+class TestApplyEffectFileWithoutExtension(PytorchTestCase):
+    def test_mp3(self):
+        """Providing format allows to read mp3 without extension
+
+        libsox does not check header for mp3
+
+        https://github.com/pytorch/audio/issues/1040
+
+        The file was generated with the following command
+            ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -ar 16000 -f mp3 test_noext
+        """
+        effects = [['band', '300', '10']]
+        path = get_asset_path("mp3_without_ext")
+        _, sr = sox_effects.apply_effects_file(path, effects, format="mp3")
+        assert sr == 16000
diff --git a/test/torchaudio_unittest/sox_io_backend/info_test.py b/test/torchaudio_unittest/sox_io_backend/info_test.py
--- a/test/torchaudio_unittest/sox_io_backend/info_test.py
+++ b/test/torchaudio_unittest/sox_io_backend/info_test.py
@@ -167,3 +167,20 @@ def test_opus(self, bitrate, num_channels, compression_level):
         assert info.sample_rate == 48000
         assert info.num_frames == 32768
         assert info.num_channels == num_channels
+
+
+@skipIfNoExtension
+class TestLoadWithoutExtension(PytorchTestCase):
+    def test_mp3(self):
+        """Providing `format` allows to read mp3 without extension
+
+        libsox does not check header for mp3
+
+        https://github.com/pytorch/audio/issues/1040
+
+        The file was generated with the following command
+            ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -ar 16000 -f mp3 test_noext
+        """
+        path = get_asset_path("mp3_without_ext")
+        sinfo = sox_io_backend.info(path, format="mp3")
+        assert sinfo.sample_rate == 16000
diff --git a/test/torchaudio_unittest/sox_io_backend/load_test.py b/test/torchaudio_unittest/sox_io_backend/load_test.py
--- a/test/torchaudio_unittest/sox_io_backend/load_test.py
+++ b/test/torchaudio_unittest/sox_io_backend/load_test.py
@@ -1,4 +1,3 @@
-import os
 import itertools
 
 from torchaudio.backend import sox_io_backend
@@ -355,26 +354,18 @@ def test_channels_first(self, channels_first):
         self.assertEqual(found, expected)
 
 
-@skipIfNoExec('sox')
 @skipIfNoExtension
-class TestLoadExtensionLess(TempDirMixin, PytorchTestCase):
-    """Given `format` parameter, `sox_io_backend.load` can load files without extension"""
-    original = None
-    path = None
+class TestLoadWithoutExtension(PytorchTestCase):
+    def test_mp3(self):
+        """Providing format allows to read mp3 without extension
 
-    def _make_file(self, format_):
-        sample_rate = 8000
-        path = self.get_temp_path(f'test.{format_}')
-        sox_utils.gen_audio_file(f'{path}', sample_rate, num_channels=2)
-        self.original = sox_io_backend.load(path)[0]
-        self.path = os.path.splitext(path)[0]
-        os.rename(path, self.path)
-
-    @parameterized.expand([
-        ('WAV', ), ('wav', ), ('MP3', ), ('mp3', ), ('FLAC', ), ('flac',),
-    ], name_func=name_func)
-    def test_format(self, format_):
-        """Providing format allows to read file without extension"""
-        self._make_file(format_)
-        found, _ = sox_io_backend.load(self.path)
-        self.assertEqual(found, self.original)
+        libsox does not check header for mp3
+
+        https://github.com/pytorch/audio/issues/1040
+
+        The file was generated with the following command
+            ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -ar 16000 -f mp3 test_noext
+        """
+        path = get_asset_path("mp3_without_ext")
+        _, sr = sox_io_backend.load(path, format="mp3")
+        assert sr == 16000