[Pytorch 2.0] Cannot load `BERT` model `No module named 'torch._six'`
Author: philschmid
Pull request: no

### System Info

- `transformers` version: 4.27.1
- Platform: Linux-5.15.0-1031-aws-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.13.1
- PyTorch version (GPU?): 2.0.0+cu117 (True)

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduction

1. `!pip install "torch>=2.0" --extra-index-url https://download.pytorch.org/whl/cu117 --upgrade --quiet`
2. `!pip install "transformers==4.27.1" "datasets==2.9.0" "accelerate==0.17.1" "evaluate==0.4.0" tensorboard scikit-learn --upgrade --quiet`
3.  load model
```python
from transformers import AutoModelForSequenceClassification

# Model id to load the tokenizer
model_id = "bert-base-uncased"

model = AutoModelForSequenceClassification.from_pretrained(
    model_id, num_labels=2
)
```

### Expected behavior

Can load model, 

below is the error 

```bash
Traceback (most recent call last):
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1126, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/opt/conda/envs/pytorch/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py", line 42, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/modeling_utils.py", line 37, in <module>
    from .deepspeed import deepspeed_config, is_deepspeed_zero3_enabled
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/deepspeed.py", line 38, in <module>
    from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/accelerator.py", line 30, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/utils/__init__.py", line 105, in <module>
    from .launch import (
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/utils/launch.py", line 28, in <module>
    from ..utils.other import merge_dicts
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/accelerate/utils/other.py", line 28, in <module>
    from deepspeed import DeepSpeedEngine
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/__init__.py", line 15, in <module>
    from . import module_inject
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/module_inject/__init__.py", line 1, in <module>
    from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/module_inject/replace_module.py", line 801, in <module>
    from ..pipe import PipelineModule
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/pipe/__init__.py", line 1, in <module>
    from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/runtime/pipe/__init__.py", line 1, in <module>
    from .module import PipelineModule, LayerSpec, TiedLayerSpec
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/runtime/pipe/module.py", line 13, in <module>
    from .. import utils as ds_utils
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/deepspeed/runtime/utils.py", line 19, in <module>
    from torch._six import inf
ModuleNotFoundError: No module named 'torch._six'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 470, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 360, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 602, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 616, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 561, in getattribute_from_module
    if hasattr(module, attr):
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1116, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1128, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
No module named 'torch._six'
```

Comments:
Error was thrown since i had `deepspeed` installed which uses/used `torch._six` see: https://github.com/microsoft/DeepSpeed/pull/2863

Is DeepSpeed part of transformers 4.27.1? I get the same error message when using torch>=2.0 and transformers, but I am not installing deepspeed myself, at least not knowingly. How did you uninstall it or turn it off?

I am not using the Trainer or Accelerate directly. Instead I use:
```
... = AutoTokenizer.from_pretrained(tokenizer_variant,do_lower_case=True)
...
... = AutoModelForSequenceClassification.from_pretrained(variant, **config_params)

```


```
Successfully installed absl-py-1.4.0 altair-4.2.2 cachetools-5.3.0 entrypoints-0.4 filelock-3.10.0 google-auth-2.16.2 google-auth-oauthlib-0.4.6 grpcio-1.51.3 huggingface-hub-0.13.2 jsonlines-3.1.0 jsonschema-4.17.3 lit-15.0.7 markdown-3.4.1 mpmath-1.3.0 nltk-3.8.1 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 oauthlib-3.2.2 pyasn1-modules-0.2.8 pyrsistent-0.19.3 regex-2022.10.31 requests-oauthlib-1.3.1 sympy-1.11.1 tensorboard-2.12.0 tensorboard-data-server-0.7.0 tensorboard-plugin-wit-1.8.1 tokenizers-0.13.2 torch-2.0.0 transformers-4.27.1 triton-2.0.0
```

```
2023-03-17 18:37:15,836 - models.transformers - INFO - Loader AutoModel from pre-trained.
--
2023-03-17 18:37:16,156 - models.transformers - ERROR - Giving up on: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
No module named 'torch._six'
2023-03-17 18:37:16,156 - models.transformers - ERROR - Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
No module named 'torch._six'
Traceback (most recent call last):  File "/opt/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1126, in _get_module    return importlib.import_module("." + module_name, self.__name__)  File "/opt/conda/lib/python3.9/importlib/__init__.py", line 127, in import_module    return _bootstrap._gcd_import(name[level:], package, level)  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked  File "<frozen importlib._bootstrap_external>", line 850, in exec_module  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed  File "/opt/conda/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py", line 42, in <module>    from ...modeling_utils import PreTrainedModel  File "/opt/conda/lib/python3.9/site-packages/transformers/modeling_utils.py", line 83, in <module>    from accelerate import __version__ as accelerate_version  File "/opt/conda/lib/python3.9/site-packages/accelerate/__init__.py", line 7, in <module>    from .accelerator import Accelerator  File "/opt/conda/lib/python3.9/site-packages/accelerate/accelerator.py", line 29, in <module>    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state  File "/opt/conda/lib/python3.9/site-packages/accelerate/checkpointing.py", line 24, in <module>    from .utils import (  File "/opt/conda/lib/python3.9/site-packages/accelerate/utils/__init__.py", line 124, in <module>    from .other import (  File "/opt/conda/lib/python3.9/site-packages/accelerate/utils/other.py", line 27, in <module>    from deepspeed import DeepSpeedEngine  File "/opt/conda/lib/python3.9/site-packages/deepspeed/__init__.py", line 16, in <module>    from .runtime.engine import DeepSpeedEngine, DeepSpeedOptimizerCallable, DeepSpeedSchedulerCallable  File "/opt/conda/lib/python3.9/site-packages/deepspeed/runtime/engine.py", line 25, in <module>    from deepspeed.runtime.utils import see_memory_usage, get_ma_status, DummyOptim  File "/opt/conda/lib/python3.9/site-packages/deepspeed/runtime/utils.py", line 18, in <module>    from torch._six import inf
ModuleNotFoundError: No module named 'torch._six'

```

I just upgraded deepspeed to the 0.8.2 and it worked

```
pip install deepspeed --upgrade
```

I don't think I need deepspeed for what I wrote above, but adding deepspeed>=0.8.2 to requirements.txt works for me as well. Thanks @maloyan!

hey!
I have same error but I don't need to use deepspeed, any idea how to solve it?
thx!

for me this was because of apex, had to do `pip uninstall -y apex` a couple of times since I wasn't using it anyway

It worked for me.

> for me this was because of apex, had to do `pip uninstall -y apex` a couple of times since I wasn't using it anyway



> for me this was because of apex, had to do `pip uninstall -y apex` a couple of times since I wasn't using it anyway

Thank you so much, it worked for me

> for me this was because of apex, had to do `pip uninstall -y apex` a couple of times since I wasn't using it anyway

Thanks, but I thought we need apex for mix precision training?