Repository: pytorch/audio

Issue #2356: Reading FLAC files from file-like objects is broken?
Created: 2022-04-28T10:07:05Z
🐛 Describe the bug
Calling torchaudio.load with a file-like object as argument to load FLAC-encoded files sometimes returns incomplete tensors. It seems to happen with non-trivial tensors after a certain length.
Minimal sample code:
Below I conduct the following test: I write a given input tensor on the filesystem using torchaudio.save in FLAC format, and then load it with torchaudio.load with a file-like object as argument. I repeat this with different tensors with different shape. The shape of the loaded tensor is written in comments; it does not always match the input tensor!
import torch
import torchaudio


fs = 16000
filename = 'example.flac'


def test(input):
    # write data
    torchaudio.save(filename, input, fs)

    # load data that was just saved using file object
    f = open(filename, 'rb')
    output, _ = torchaudio.load(f)

    # check the output shape
    print(int(input.shape == output.shape), output.shape)


torch.manual_seed(0)

# non-random vectors as int or float work fine
test(torch.zeros(1, 1600, dtype=torch.int))          # correct
test(torch.ones(1, 1600, dtype=torch.float))         # correct
test(10*torch.ones(2, 16000, dtype=torch.int))       # correct
test(-100*torch.ones(2, 160000, dtype=torch.float))  # correct

# short random vectors also work fine (0.1s @ 16kHz)
test(torch.randn(1, 1600))  # correct
test(torch.randn(2, 1600))  # correct
test(torch.rand(1, 1600))   # correct
test(torch.rand(2, 1600))   # correct

# longer random vectors produce wrong shapes! (1s @ 16kHz)
test(torch.randn(1, 16000))  # wrong, torch.Size([1, 4096]), raises warning "flac: decoder MD5 checksum mismatch."
test(torch.randn(2, 16000))  # wrong, torch.Size([2, 0])
test(torch.rand(1, 16000))   # wrong, torch.Size([1, 8192]), raises warning "flac: decoder MD5 checksum mismatch."
test(torch.rand(2, 16000))   # wrong, torch.Size([2, 0])

# maybe staying in [-1.0, 1.0] fixes it?
test(0.01*torch.randn(1, 16000))  # correct
test(0.01*torch.randn(2, 16000))  # wrong, torch.Size([2, 0])
test(0.01*torch.rand(1, 16000))   # correct
test(0.01*torch.rand(2, 16000))   # wrong, torch.Size([2, 0])

# what about longer waveforms? (10s @ 16kHz)
test(0.01*torch.randn(1, 160000))  # wrong, torch.Size([1, 24576]), raises warning "flac: decoder MD5 checksum mismatch."
test(0.01*torch.randn(2, 160000))  # wrong, torch.Size([2, 0])
test(0.01*torch.rand(1, 160000))   # wrong, torch.Size([1, 69632]), raises warning "flac: decoder MD5 checksum mismatch."
test(0.01*torch.rand(2, 160000))   # wrong, torch.Size([2, 0])
Results:

Using tensors full of zeros or ones, whether they are casted as int or float, always produce correct shapes
Using random tensors as short as 0.1s at 16kHz works too
Using longer random tensors of 1s or 10s at 16kHz produces wrong shapes!

1-channel tensors are loaded incomplete and raise a "flac: decoder MD5 checksum mismatch." warning
2-channel tensors are always returned empty!
Scaling down the single-channel random tensors fixed the issue for 1s-long tensors...
...but increasing the duration to 10s breaks it again!



Notes:

Writing/loading to WAV, OGG or VORBIS by changing the extension e.g. setting filename = example.wav always produces correct shapes
Writing/loading from filename instead of file-like object produces correct shapes
[Maybe a different bug] Writing/loading to MP3 "kind of" works but one needs to provide format="mp3" in torchaudio.load, as if the format was not inferred by the file header. I say "kind of" because the output shapes are actually exactly 704 samples longer than the input shapes! When omitting format="mp3", the following error is raised:

formats: can't determine type of file `'
Traceback (most recent call last):
  File "/home/phigon/dev/brever/temp2.py", line 24, in <module>
    test(torch.zeros(1, 1600, dtype=torch.int))                # correct
  File "/home/phigon/dev/brever/temp2.py", line 15, in test
    output, _ = torchaudio.load(f)
  File "/home/phigon/dev/brever/venv/lib64/python3.10/site-packages/torchaudio/backend/sox_io_backend.py", line 149, in load
    return torchaudio._torchaudio.load_audio_fileobj(
RuntimeError: Error loading audio file: failed to open file <in memory buffer>

Some more testing
Since in the code sample above I am writing the FLAC files with torchaudio, I thought maybe the issue comes from the writing and not the reading. But when trying to read from files created differently, problems arise too!
import subprocess

import numpy as np
import soundfile as sf
import torchaudio


fs = 16000
filename = 'example.flac'


np.random.seed(0)

# TEST 1: write with soundfile
x = np.random.randn(16000, 2)  # in soundfile shape is (samples, channels)
sf.write(filename, x, fs)
f = open(filename, "rb")
x, _ = torchaudio.load(f)
print(x.shape)
# good shape with libsndfile 1.0.25
# bad  shape with libsndfile 1.0.31, got torch.Size([2, 0])


# TEST 2: write with soundfile, different tensor
x = np.random.rand(160000, 1)
sf.write(filename, x, fs)
f = open(filename, "rb")
x, _ = torchaudio.load(f)
print(x.shape)
# bad  with libsndfile 1.0.25, got torch.Size([1, 40320])
# good with libsndfile 1.0.31


# TEST 3: write with ffmpeg
x = np.random.rand(16000, 1)
tmpfile = 'example.wav'
sf.write(tmpfile, x, fs)  # first write a temporary WAV file
subprocess.call([  # convert temporary file to FLAC with FFmpeg
    'ffmpeg',
    '-y',
    '-hide_banner',
    '-loglevel',
    'error',
    '-i',
    tmpfile,
    filename,
])
f = open(filename, "rb")
x, _ = torchaudio.load(f)
# raises RuntimeError, does not fix with format="flac"
Above I perform the following tests:

Write with soundfile a 1s-long 2-channel tensor of normally distributed samples

Output shape is correct with libsndfile 1.0.25, but wrong with libsndfile 1.0.31!


Write with soundfile a 10s-long 1-channel tensor of uniformly distributed samples

Output shape is correct with libsndfile 1.0.31, but wrong with libsndfile 1.0.25!


Write with FFmpeg

Can't even load, raises a RuntimeError:



formats: can't open input file `': FLAC ERROR whilst decoding metadata
Traceback (most recent call last):
  File "/home/phigon/dev/brever/temp3.py", line 47, in <module>
    x, _ = torchaudio.load(f)
  File "/home/phigon/dev/brever/venv/lib64/python3.10/site-packages/torchaudio/backend/sox_io_backend.py", line 149, in load
    return torchaudio._torchaudio.load_audio_fileobj(
RuntimeError: Error loading audio file: failed to open file <in memory buffer>

Here again, when calling torchaudio.load with the filename instead of the file-like object as argument, everything works fine.
FFmpeg version: 4.4.2
SoundFile version: 0.10.3.post1
Versions
Collecting environment information...
PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Fedora release 35 (Thirty Five) (x86_64)
GCC version: (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.34
Python version: 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)] (64-bit runtime)
Python platform: Linux-5.17.4-200.fc35.x86_64-x86_64-with-glibc2.34
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.11.0
[pip3] torchaudio==0.11.0
[pip3] torchvision==0.12.0
[conda] Could not collect

Duplicate issue #2948: Error thrown while reading FLAC files from file-like objects 
Created: 2023-01-03T08:06:30Z
🐛 Describe the bug
Error thrown while reading flac file like objects
While loading an audio file from flac file path seems to be working fine, When reading the same file as a file like object throws an error
minimal code to reproduce
import io
import torchaudio
import os

def get_binary_data(fpath: str) -> bytes:
    assert os.path.isfile(fpath)
    with open(fpath, "rb") as stream:
        binary_data = stream.read()
        return binary_data

torchaudio.set_audio_backend('sox_io')

audio, sample_rate = torchaudio.load("sample.flac")
print(len(audio.squeeze(0))/sample_rate)

audio = get_binary_data("sample.flac")
audio, sample_rate = torchaudio.load(io.BytesIO(audio))
print(len(audio.squeeze(0))/sample_rate)

expected behaviour
5.305854166666666
5.305854166666666

output
5.305854166666666
formats: can't open input file `': FLAC ERROR whilst decoding metadata
Output exceeds the [size limit](command:workbench.action.openSettings?[). Open the full output data [in a text editor](command:workbench.action.openLargeOutput?64e19f19-4197-4c6c-b456-82ccc0c176c4)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/home/local/ZOHOCORP/veera-pt4765/flac_issue/notebook.ipynb Cell 6' in <cell line: 2>()
      [1](vscode-notebook-cell:/home/local/ZOHOCORP/veera-pt4765/flac_issue/notebook.ipynb#ch0000004?line=0) audio = get_binary_data("sample.flac")
----> [2](vscode-notebook-cell:/home/local/ZOHOCORP/veera-pt4765/flac_issue/notebook.ipynb#ch0000004?line=1) audio, sample_rate = torchaudio.load(io.BytesIO(audio))
      [3](vscode-notebook-cell:/home/local/ZOHOCORP/veera-pt4765/flac_issue/notebook.ipynb#ch0000004?line=2) len(audio.squeeze(0))/sample_rate

File ~/anaconda3/envs/zspeech/lib/python3.8/site-packages/torchaudio/backend/sox_io_backend.py:220, in load(filepath, frame_offset, num_frames, normalize, channels_first, format)
    218         if ret is not None:
    219             return ret
--> 220         return _fallback_load_fileobj(filepath, frame_offset, num_frames, normalize, channels_first, format)
    221     filepath = os.fspath(filepath)
    222 ret = torch.ops.torchaudio.sox_io_load_audio_file(
    223     filepath, frame_offset, num_frames, normalize, channels_first, format
    224 )

File ~/anaconda3/envs/zspeech/lib/python3.8/site-packages/torchaudio/io/_compat.py:114, in load_audio_fileobj(src, frame_offset, num_frames, convert, channels_first, format)
    106 def load_audio_fileobj(
    107     src: str,
    108     frame_offset: int = 0,
   (...)
    112     format: Optional[str] = None,
    113 ) -> Tuple[torch.Tensor, int]:
--> 114     s = torchaudio._torchaudio_ffmpeg.StreamReaderFileObj(src, format, None, 4096)
    115     return _load_audio(s, frame_offset, num_frames, convert, channels_first)
...
frame #60: PyVectorcall_Call + 0x78 (0x4e8d08 in /home/local/ZOHOCORP/veera-pt4765/anaconda3/envs/zspeech/bin/python)
frame #61: _PyEval_EvalFrameDefault + 0x5bd8 (0x4ccbb8 in /home/local/ZOHOCORP/veera-pt4765/anaconda3/envs/zspeech/bin/python)
frame #62: _PyFunction_Vectorcall + 0x106 (0x4da136 in /home/local/ZOHOCORP/veera-pt4765/anaconda3/envs/zspeech/bin/python)
frame #63: _PyEval_EvalFrameDefault + 0xa3e (0x4c7a1e in /home/local/ZOHOCORP/veera-pt4765/anaconda3/envs/zspeech/bin/python)

Alternatives which worked
torchaudio.set_audio_backend('soundfile')

audio = get_binary_data("sample.flac")
audio, sample_rate = torchaudio.load(io.BytesIO(audio))
print(len(audio.squeeze(0))/sample_rate)

Resources to reproduce
notebook
raw audio used in notebook
flac audio with issue
Notes

The flac file then converted to wav and loaded the same as file like object. Worked

Versions
Collecting environment information...
PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.15 (default, Nov 24 2022, 15:19:38)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.14.0-1054-oem-x86_64-with-glibc2.17
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.961
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] pytorch-lightning==1.5.10
[pip3] torch==1.13.0
[pip3] torch-poly-lr-decay==0.0.1
[pip3] torchaudio==0.13.0
[pip3] torchmetrics==0.10.2
[conda] numpy                     1.23.4                   pypi_0    pypi
[conda] pytorch-lightning         1.5.10                   pypi_0    pypi
[conda] torch                     1.13.0                   pypi_0    pypi
[conda] torch-poly-lr-decay       0.0.1                    pypi_0    pypi
[conda] torchaudio                0.13.0                   pypi_0    pypi
[conda] torchmetrics              0.10.2                   pypi_0    pypi

Closing comment (2023-01-10T13:49:14Z): This issue is likely duplicate of #2356, and this is another limitation of libsox.
Can you try increasing the memory buffer size?
https://pytorch.org/audio/stable/generated/torchaudio.utils.sox_utils.html#set-buffer-size