Can't import datasets: partially initialized module 'fsspec' has no attribute 'utils'
Opened by arunasank (closed)

## Describe the bug
Unable to import datasets

## Steps to reproduce the bug
```python
from datasets import Dataset, DatasetDict
```

## Expected results
The import works without errors

## Actual results
```
AttributeError                            Traceback (most recent call last)
<ipython-input-37-c8cfcbe62127> in <module>
     11 # from tqdm import tqdm
     12 # import torch
---> 13 from datasets import Dataset
     14 # from transformers import Trainer, TrainingArguments, AutoModel, AutoTokenizer, AutoModelForMaskedLM, DataCollatorForLanguageModeling
     15 # from sentence_transformers import SentenceTransformer

~/.local/lib/python3.8/site-packages/datasets/__init__.py in <module>
     31     )
     32 
---> 33 from .arrow_dataset import Dataset, concatenate_datasets
     34 from .arrow_reader import ArrowReader, ReadInstruction
     35 from .arrow_writer import ArrowWriter

~/.local/lib/python3.8/site-packages/datasets/arrow_dataset.py in <module>
     46 )
     47 
---> 48 import fsspec
     49 import numpy as np
     50 import pandas as pd

~/.local/lib/python3.8/site-packages/fsspec/__init__.py in <module>
     10 from . import _version, caching
     11 from .callbacks import Callback
---> 12 from .core import get_fs_token_paths, open, open_files, open_local
     13 from .exceptions import FSTimeoutError
     14 from .mapping import FSMap, get_mapper

~/.local/lib/python3.8/site-packages/fsspec/core.py in <module>
     16     caches,
     17 )
---> 18 from .compression import compr
     19 from .registry import filesystem, get_filesystem_class
     20 from .utils import (

~/.local/lib/python3.8/site-packages/fsspec/compression.py in <module>
     68 
     69 
---> 70 register_compression("zip", unzip, "zip")
     71 register_compression("bz2", BZ2File, "bz2")
     72 

~/.local/lib/python3.8/site-packages/fsspec/compression.py in register_compression(name, callback, extensions, force)
     44 
     45     for ext in extensions:
---> 46         if ext in fsspec.utils.compressions and not force:
     47             raise ValueError(
     48                 "Duplicate compression file extension: %s (%s)" % (ext, name)

AttributeError: partially initialized module 'fsspec' has no attribute 'utils' (most likely due to a circular import)
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.4
- Platform: Jupyter notebook
- Python version: 3.8.10
- PyArrow version: 7.0.0


Comments:
Update: `"python3 -c "from from datasets import Dataset, DatasetDict"` works, but not if I import without the `python3 -c`

Hi @arunasank, thanks for reporting.

It seems that this can be caused because you are using an old version of `fsspec`: the reason why it works if you run `python3` seems to be that `python3` runs in a Python virtual env (with an updated version of `fsspec`); whereas the error arises when you run the import from other Python virtual env (with an old version of `fsspec`).

In order to fix this, you should update `fsspec` from within the "problematic" Python virtual env:
```
pip install -U "fsspec[http]>=2021.05.0"

I'm closing this issue, @arunasank.

Feel free to re-open it if the problem persists. 

from lightgbm import LGBMModel,LGBMClassifier, plot_importance
after importing lib getting (partially initialized module 'fsspec' has no attribute 'utils' (most likely due to a circular import) error, can help me

@deepakmahtha I think you are not using `datasets`: this is the GitHub repository of Hugging Face Datasets.

If you are using `lightgbm`, you should report the issue to their repository instead.

Anyway, we have proposed a possible fix just in a comment above: to update fsspec.
https://github.com/huggingface/datasets/issues/3902#issuecomment-1066517824