audiofolder works on local env, but creates empty dataset in a remote one, what dependencies could I be missing/outdated
Opened by jcho19 (closed)

### Describe the bug

I'm using a custom audio dataset (400+ audio files) in the correct format for audiofolder. Although loading the dataset with audiofolder works in one local setup, it doesn't in a remote one (it just creates an empty dataset). I have both ffmpeg and libndfile installed on both computers, what could be missing/need to be updated in the one that doesn't work? On the remote env, libsndfile is 1.0.28 and ffmpeg is 4.2.1.

from datasets import load_dataset
ds = load_dataset("audiofolder", data_dir="...")


Here is the output (should be generating 400+ rows):
Downloading and preparing dataset audiofolder/default to ...
Downloading data files:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading data files: 0it [00:00, ?it/s]
Extracting data files: 0it [00:00, ?it/s]
Generating train split: 0 examples [00:00, ? examples/s]
Dataset audiofolder downloaded and prepared to ... Subsequent calls will reuse this data.
  0%|          | 0/1 [00:00<?, ?it/s]
DatasetDict({
    train: Dataset({
        features: ['audio', 'transcription'],
        num_rows: 1
    })
})






Here is my pip environment in the one that doesn't work (uses torch 1.11.a0 from shared env):
Package             Version
------------------- -------------------
aiofiles            22.1.0
aiohttp             3.8.3
aiosignal           1.3.1
altair              4.2.1
anyio               3.6.2
appdirs             1.4.4
argcomplete         2.0.0
argon2-cffi         20.1.0
astunparse          1.6.3
async-timeout       4.0.2
attrs               21.2.0
audioread           3.0.0
backcall            0.2.0
bleach              4.0.0
certifi             2021.10.8
cffi                1.14.6
charset-normalizer  2.0.12
click               8.1.3
contourpy           1.0.7
cycler              0.11.0
datasets            2.9.0
debugpy             1.4.1
decorator           5.0.9
defusedxml          0.7.1
dill                0.3.6
distlib             0.3.4
entrypoints         0.3
evaluate            0.4.0
expecttest          0.1.3
fastapi             0.89.1
ffmpy               0.3.0
filelock            3.6.0
fonttools           4.38.0
frozenlist          1.3.3
fsspec              2023.1.0
future              0.18.2
gradio              3.16.2
h11                 0.14.0
httpcore            0.16.3
httpx               0.23.3
huggingface-hub     0.12.0
idna                3.3
ipykernel           6.2.0
ipython             7.26.0
ipython-genutils    0.2.0
ipywidgets          7.6.3
jedi                0.18.0
Jinja2              3.0.1
jiwer               2.5.1
joblib              1.2.0
jsonschema          3.2.0
jupyter             1.0.0
jupyter-client      6.1.12
jupyter-console     6.4.0
jupyter-core        4.7.1
jupyterlab-pygments 0.1.2
jupyterlab-widgets  1.0.0
kiwisolver          1.4.4
Levenshtein         0.20.2
librosa             0.9.2
linkify-it-py       1.0.3
llvmlite            0.39.1
markdown-it-py      2.1.0
MarkupSafe          2.0.1
matplotlib          3.6.3
matplotlib-inline   0.1.2
mdit-py-plugins     0.3.3
mdurl               0.1.2
mistune             0.8.4
multidict           6.0.4
multiprocess        0.70.14
nbclient            0.5.4
nbconvert           6.1.0
nbformat            5.1.3
nest-asyncio        1.5.1
notebook            6.4.3
numba               0.56.4
numpy               1.20.3
orjson              3.8.5
packaging           21.0
pandas              1.5.3
pandocfilters       1.4.3
parso               0.8.2
pexpect             4.8.0
pickleshare         0.7.5
Pillow              9.4.0
pip                 22.3.1
pipx                1.1.0
platformdirs        2.5.2
pooch               1.6.0
prometheus-client   0.11.0
prompt-toolkit      3.0.19
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             10.0.1
pycparser           2.20
pycryptodome        3.16.0
pydantic            1.10.4
pydub               0.25.1
Pygments            2.10.0
pyparsing           2.4.7
pyrsistent          0.18.0
python-dateutil     2.8.2
python-multipart    0.0.5
pytz                2022.7.1
PyYAML              6.0
pyzmq               22.2.1
qtconsole           5.1.1
QtPy                1.10.0
rapidfuzz           2.13.7
regex               2022.10.31
requests            2.27.1
resampy             0.4.2
responses           0.18.0
rfc3986             1.5.0
scikit-learn        1.2.1
scipy               1.6.3
Send2Trash          1.8.0
setuptools          65.5.1
shiboken6           6.3.1
shiboken6-generator 6.3.1
six                 1.16.0
sniffio             1.3.0
soundfile           0.11.0
starlette           0.22.0
terminado           0.11.0
testpath            0.5.0
threadpoolctl       3.1.0
tokenizers          0.13.2
toolz               0.12.0
torch               1.11.0a0+gitunknown
tornado             6.1
tqdm                4.64.1
traitlets           5.0.5
transformers        4.27.0.dev0
types-dataclasses   0.6.4
typing_extensions   4.1.1
uc-micro-py         1.0.1
urllib3             1.26.9
userpath            1.8.0
uvicorn             0.20.0
virtualenv          20.14.1
wcwidth             0.2.5
webencodings        0.5.1
websockets          10.4
wheel               0.37.1
widgetsnbextension  3.5.1
xxhash              3.2.0
yarl                1.8.2

### Steps to reproduce the bug

Create a pip environment with the packages listed above (make sure ffmpeg and libsndfile is installed with same versions listed above).
Create a custom audio dataset and load it in with load_dataset("audiofolder", ...)



### Expected behavior

load_dataset should create a dataset with 400+ rows. 

### Environment info

- `datasets` version: 2.9.0
- Platform: Linux-3.10.0-1160.80.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.0
- PyArrow version: 10.0.1
- Pandas version: 1.5.3

Comments: