Failed to import transformers.generation.utils because of the following error
Author: GeLink9999
Pull request: no
Labels: name: bug

### System Info

when i input "transformers-cli env", its output is as below:
Traceback (most recent call last):
  File "/home/jovyan/.local/lib/python3.10/site-packages/transformer_engine/pytorch/__init__.py", line 22, in _load_library
    so_path = next(so_dir.glob(f"transformer_engine_torch.*.{extension}"))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jovyan/nas/envs/minicpm/bin/transformers-cli", line 33, in <module>
    sys.exit(load_entry_point('transformers==4.40.0', 'console_scripts', 'transformers-cli')())
  File "/home/jovyan/nas/envs/minicpm/bin/transformers-cli", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/commands/transformers_cli.py", line 25, in <module>
    from .run import RunCommand
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/commands/run.py", line 17, in <module>
    from ..pipelines import Pipeline, PipelineDataFormat, get_supported_tasks, pipeline
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/pipelines/__init__.py", line 47, in <module>
    from .audio_classification import AudioClassificationPipeline
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/pipelines/audio_classification.py", line 21, in <module>
    from .base import Pipeline, build_pipeline_init_args
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/pipelines/base.py", line 34, in <module>
    from ..modelcard import ModelCard
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/modelcard.py", line 48, in <module>
    from .training_args import ParallelMode
  File "/home/jovyan/nas/envs/minicpm/lib/python3.10/site-packages/transformers-4.40.0-py3.10.egg/transformers/training_args.py", line 73, in <module>
    from accelerate.state import AcceleratorState, PartialState
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/utils/__init__.py", line 181, in <module>
    from .bnb import has_4bit_bnb_layers, load_and_quantize_model
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/utils/bnb.py", line 29, in <module>
    from ..big_modeling import dispatch_model, init_empty_weights
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/big_modeling.py", line 24, in <module>
    from .hooks import (
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/hooks.py", line 30, in <module>
    from .utils.other import recursive_getattr
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/utils/other.py", line 36, in <module>
    from .transformer_engine import convert_model
  File "/home/jovyan/.local/lib/python3.10/site-packages/accelerate/utils/transformer_engine.py", line 21, in <module>
    import transformer_engine.pytorch as te
  File "/home/jovyan/.local/lib/python3.10/site-packages/transformer_engine/pytorch/__init__.py", line 34, in <module>
    _load_library()
  File "/home/jovyan/.local/lib/python3.10/site-packages/transformer_engine/pytorch/__init__.py", line 25, in _load_library
    so_path = next(so_dir.glob(f"transformer_engine_torch.*.{extension}"))
StopIteration

### Who can help?

when i run Minicpm model, it shows :
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
![image](https://github.com/user-attachments/assets/9618fa6c-d1fe-49be-8318-afec999feb2d)
 pip list
Package                           Version
--------------------------------- ------------
accelerate                        0.30.1
addict                            2.4.0
aiofiles                          23.2.1
aiohappyeyeballs                  2.3.7
aiohttp                           3.10.4
aiosignal                         1.3.1
annotated-types                   0.7.0
anyio                             4.4.0
async-timeout                     4.0.3
attrs                             24.2.0
autocommand                       2.2.2
backports.tarfile                 1.2.0
Bottleneck                        1.3.7
certifi                           2024.7.4
charset-normalizer                3.3.2
click                             8.1.7
cloudpickle                       3.0.0
cmake                             3.30.2
colorama                          0.4.6
contourpy                         1.2.1
cycler                            0.12.1
datasets                          2.21.0
decord                            0.6.0
dill                              0.3.8
diskcache                         5.6.3
distro                            1.9.0
editdistance                      0.6.2
einops                            0.7.0
et-xmlfile                        1.1.0
exceptiongroup                    1.2.2
fairscale                         0.4.0
fastapi                           0.112.1
ffmpy                             0.4.0
filelock                          3.15.4
fonttools                         4.53.1
frozenlist                        1.4.1
fsspec                            2024.6.1
git-lfs                           1.6
gradio                            4.41.0
gradio_client                     1.3.0
h11                               0.14.0
httpcore                          1.0.5
httptools                         0.6.1
httpx                             0.27.0
huggingface-hub                   0.24.5
idna                              3.7
importlib_metadata                8.2.0
importlib_resources               6.4.3
inflect                           7.3.1
interegular                       0.3.3
jaraco.context                    5.3.0
jaraco.functools                  4.0.1
jaraco.text                       3.12.1
Jinja2                            3.1.4
jiter                             0.5.0
joblib                            1.4.2
jsonlines                         4.0.0
jsonschema                        4.23.0
jsonschema-specifications         2023.12.1
kiwisolver                        1.4.5
lark                              1.2.2
llvmlite                          0.43.0
lm-format-enforcer                0.10.3
lxml                              5.3.0
markdown-it-py                    3.0.0
markdown2                         2.4.10
MarkupSafe                        2.1.5
matplotlib                        3.7.4
mdurl                             0.1.2
mkl-fft                           1.3.8
mkl-random                        1.2.4
mkl-service                       2.4.0
modelscope_studio                 0.4.0.9
more-itertools                    10.1.0
mpmath                            1.3.0
msgpack                           1.0.8
multidict                         6.0.5
multiprocess                      0.70.16
nest-asyncio                      1.6.0
networkx                          3.3
ninja                             1.11.1.1
nltk                              3.8.1
numba                             0.60.0
numexpr                           2.8.7
numpy                             1.24.4
nvidia-cublas-cu12                12.1.3.1
nvidia-cuda-cupti-cu12            12.1.105
nvidia-cuda-nvrtc-cu12            12.1.105
nvidia-cuda-runtime-cu12          12.1.105
nvidia-cudnn-cu12                 9.1.0.70
nvidia-cufft-cu12                 11.0.2.54
nvidia-curand-cu12                10.3.2.106
nvidia-cusolver-cu12              11.4.5.107
nvidia-cusparse-cu12              12.1.0.106
nvidia-ml-py                      12.560.30
nvidia-nccl-cu12                  2.20.5
nvidia-nvjitlink-cu12             12.6.20
nvidia-nvtx-cu12                  12.1.105
openai                            1.41.0
opencv-python-headless            4.5.5.64
openpyxl                          3.1.2
ordered-set                       4.1.0
orjson                            3.10.7
outlines                          0.0.46
packaging                         23.2
pandas                            2.2.2
peft                              0.12.0
Pillow                            10.1.0
pip                               24.2
platformdirs                      4.2.2
portalocker                       2.10.1
prometheus_client                 0.20.0
prometheus-fastapi-instrumentator 7.0.0
protobuf                          4.25.0
psutil                            6.0.0
py-cpuinfo                        9.0.0
pyairports                        2.1.1
pyarrow                           17.0.0
pycountry                         24.6.1
pydantic                          2.8.2
pydantic_core                     2.20.1
pydub                             0.25.1
Pygments                          2.18.0
pyparsing                         3.1.2
python-dateutil                   2.9.0.post0
python-dotenv                     1.0.1
python-multipart                  0.0.9
pytz                              2024.1
PyYAML                            6.0.2
pyzmq                             26.1.1
ray                               2.34.0
referencing                       0.35.1
regex                             2024.7.24
requests                          2.32.3
rich                              13.7.1
rpds-py                           0.20.0
ruff                              0.6.1
sacrebleu                         2.3.0
safetensors                       0.4.4
scikit-learn                      1.5.1
scipy                             1.12.0
seaborn                           0.13.0
semantic-version                  2.10.0
sentencepiece                     0.1.99
setuptools                        72.1.0
shellingham                       1.5.4
shortuuid                         1.0.11
six                               1.16.0
sniffio                           1.3.1
socksio                           1.0.0
starlette                         0.38.2
sympy                             1.13.2
tabulate                          0.9.0
threadpoolctl                     3.5.0
tiktoken                          0.7.0
timm                              0.9.10
tokenizers                        0.19.1
tomli                             2.0.1
tomlkit                           0.12.0
torch                             2.4.0
torchvision                       0.19.0
tqdm                              4.66.5
transformer-engine                1.9.0.post1
transformer_engine_cu12           1.9.0
transformers                      4.40.0
triton                            3.0.0
typeguard                         4.3.0
typer                             0.12.4
typing_extensions                 4.12.2
tzdata                            2023.3
urllib3                           2.2.2
uvicorn                           0.24.0.post1
uvloop                            0.20.0
vllm                              0.5.4
vllm-flash-attn                   2.6.1
watchfiles                        0.23.0
websockets                        12.0
wheel                             0.43.0
wrapt                             1.16.0
xformers                          0.0.27.post2
xxhash                            3.5.0
yarl                              1.9.4
zipp                              3.20.0

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1.download the model MiniCPM-V-2_6-int4:
https://huggingface.co/openbmb/MiniCPM-V-2_6-int4

2. run the python code:
import torch
from PIL import Image
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('../../checkpoints/MiniCPM-V-2_6-int4', trust_remote_code=True,
    attn_implementation='sdpa', torch_dtype=torch.bfloat16) 



### Expected behavior

it doesnot show any error

Comments:
Hi @GeLink9999, thanks for opening this issue! 

I looks like there's a clash happening with `transformer-engine` installed in your environment and the transformers library. Could you try uninstalling transformer-engine? 

Note - there is no need to download the model directly to load if working online. If this is the case, you can just pass in the relevant checkpoint and the model will be downloaded to the cache folder. This is recommended if you are not working offline as any changes in the upstream model repo will be reflected when you load the model again i.e. 

```py
import torch
from PIL import Image
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('openbmb/MiniCPM-V-2_6-int4', trust_remote_code=True,
attn_implementation='sdpa', torch_dtype=torch.bfloat16)
```

@amyeroberts thanks for the suggestions. I got the same error and uninstall transformer-engine works for me.

> Hi @GeLink9999, thanks for opening this issue!
> 
> I looks like there's a clash happening with `transformer-engine` installed in your environment and the transformers library. Could you try uninstalling transformer-engine?
> 
> Note - there is no need to download the model directly to load if working online. If this is the case, you can just pass in the relevant checkpoint and the model will be downloaded to the cache folder. This is recommended if you are not working offline as any changes in the upstream model repo will be reflected when you load the model again i.e.
> 
> ```python
> import torch
> from PIL import Image
> from transformers import AutoModel, AutoTokenizer
> model = AutoModel.from_pretrained('openbmb/MiniCPM-V-2_6-int4', trust_remote_code=True,
> attn_implementation='sdpa', torch_dtype=torch.bfloat16)
> ```

Thanks for you reply and kind remind, it works using pip uninstall transformer-engine.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.