Failed to import transformers.generation.utils
Author: Steaunk
Pull request: no

### System Info

Environment:
```
Python 3.10.12
torch==2.1.2
transformers==4.38.2
accelerate==0.27.2
vllm==0.3.3
```

My Code:
```
# from vllm import LLM
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m", device_map="auto")
# vllm = LLM(model="facebook/opt-125m")
```

I was trying to load the hf model and vllm model:
1. `pip install vllm`
2. `pip install accelerate` (since we have `device_map="auto"`)
3. run python code

---
Message:
```
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/LLM-hack/test.py", line 4, in <module>
    model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m", device_map="auto")
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 560, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 381, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 732, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 746, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 690, in getattribute_from_module
    if hasattr(module, attr):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1380, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1392, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.opt.modeling_opt because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
/usr/local/lib/python3.10/dist-packages/transformer_engine_extensions.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

My Code:
```
# from vllm import LLM
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m", device_map="auto")
# vllm = LLM(model="facebook/opt-125m")
```

1. `pip install vllm`
2. `pip install accelerate` (since we have `device_map="auto"`)
3. run python code

### Expected behavior

I am trying to load the hf model and vllm model, we should be able to load the both models.

Comments:
Hey ! Sorry I cannot reproduce this issue, seems incompatible installs / python versions

> Hey ! Sorry I cannot reproduce this issue, seems incompatible installs / python versions

Hi, I did this inside the docker `nvcr.io/nvidia/pytorch:24.01-py3` as well as `nvcr.io/nvidia/pytorch:23.10-py3`. The error is same.

@Steaunk  I'm also facing the same issue. Could you solve the issue?

> @Steaunk I'm also facing the same issue. Could you solve the issue?

I guess that's incompatible installs / python versions. So I change my docker image to `nvcr.io/nvidia/cuda:12.0.0-runtime-ubuntu22.04` and install python & torch from scratch. @NusratNB 

same question with 
Python 3.10.6
torch==2.1.2
transformers==4.39.1
accelerate==0.28.0
vllm=0.3.3

I also have the same question with:
Python 3.8.10
torch==2.1.0
transformers==  4.35.2 or 4.39.2
accelerate==0.28.0


> I guess that's incompatible installs / python versions. So I change my docker image to `nvcr.io/nvidia/cuda:12.0.0-runtime-ubuntu22.04` and install python & torch from scratch. @NusratNB


Hey @Steaunk, does this solve your issue? Did you install specific versions of python and torch or the latest? I'm facing a similar error when I use deepspeed with pytorch lightning.

> > I guess that's incompatible installs / python versions. So I change my docker image to `nvcr.io/nvidia/cuda:12.0.0-runtime-ubuntu22.04` and install python & torch from scratch. @NusratNB
> 
> Hey @Steaunk, does this solve your issue? Did you install specific versions of python and torch or the latest? I'm facing a similar error when I use deepspeed with pytorch lightning.

Yes, I solve this issue by changing the docker container and just install the latest one. Thus, I think there's something incompatible.

For me, I had to install torch separately with: pip3 install torch --index-url https://download.pytorch.org/whl/cpu
for this error to resolve.

@isabelayepes, can you paste the versions you're using here?

How to solve this problem? I have installed the latest transformers and torch2.5.1 python 3.12.
simply
```python
from transformers import (
    AutoTokenizer,
```
can trigger this.


> How to solve this problem? I have installed the latest transformers and torch2.5.1 python 3.12. simply
> 
> ```python
> from transformers import (
>     AutoTokenizer,
> ```
> 
> can trigger this.

@rangehow  Hi, have you solved it? I think there are some conflicts between pytorch and apex/te. I solved it by `pip uninstall apex`.

I love how everybody seems to say it's about incompatible software versions but not one actually shared what versions their working system has. There is absolutely no answer to this very commonly seen problem here, but the issue is closed.

> I love how everybody seems to say it's about incompatible software versions but not one actually shared what versions their working system has. There is absolutely no answer to this very commonly seen problem here, but the issue is closed.

I just had the same problem today with numpy and transformers conflicting, solved it now by uninstalling both of them and reinstalling versions 1.23.5 for numpy, 4.27.0 transformers

Unfortunately the newest version of vllm, version 0.8.5.post1, requires transformers>=4.51.1. So, installing transformers==4.27.0 created a new error.

Instead, I just reverted my version of numpy to 1.26.4 (although I believe 1.23.5 would work too) and kept the newest version of transformers. Here were my final versions:
```
numpy==1.26.4
transformers==4.51.3
vllm==0.8.5.post1
```