[Version: 4.49.0] Qwen2.5-VL is not supported in vLLM because of transformers
Author: usun1997
Pull request: no
Labels: name: bug

### System Info

error: Model architectures Qwen2 5 VLForConditionalGeneration' failed to be inspected. Please check the logs for more details.

![Image](https://github.com/user-attachments/assets/e0d5f0cc-7c3a-4891-ba67-280db9127ece)

right now people say that they are using methods like:
1. pip install --upgrade git+https://github.com/huggingface/transformers.git@336dc69d63d56f232a183a3e7f52790429b871ef (https://github.com/vllm-project/vllm/issues/13285)
2. pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40 (https://github.com/vllm-project/vllm/issues/12932)
3. There is a breaking change in transformers dev. You need to update vLLM to latest dev and also redownload the HF model repo. (https://github.com/vllm-project/vllm/issues/12932)

I mean. This is not supposed to be like this. I can't connect to your github without a vpn, and with a vpn, I can't connect to my workspace. Could transformers team just fix the problem instead of letting people solve it by some weird method. Thanks!

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

use Xinference, download the newest vLLM so that you get transformers 4.49.0. You download Qwen2.5 VL and deploy in vLLM and get the error:

![Image](https://github.com/user-attachments/assets/d9c83523-2838-4dcd-a2ff-105d84d3eabe)

### Expected behavior

NO MORE ERROR DURING DEPLOYMENT OF QWEN2.5 VL 7B IN VLLM

Comments:
me too.

me too.

me too.

mee too

cc @zucchini-nlp, but it would help a lot if someone could post the logs/error to help us figure out what's going on here!

Hey all! The issue is being fixed on vllm side with https://github.com/vllm-project/vllm/pull/13592 afaik, the team will check compatibility with v4.49 release

cc @Isotr0py 

That's because the qwen2.5-vl implementation in vllm 0.7.2 (the latest release, not latest commit) is still trying import `Qwen2_5_VLImageProcessor` which has been removed in `transformers` 4.49.0 release.

vLLM team is planning to make a new release including the corresponding fix (https://github.com/vllm-project/vllm/pull/13286), perhaps today or tomorrow.

Hello! We have released `vllm==0.7.3` - Although https://github.com/vllm-project/vllm/pull/13602 didn't make it to the release, we have confirmed that our docker image and PyPI package both have `transformers==4.49.0` installed and should be compatible with it!

Still not working after upgrate to `vllm==0.7.3`

![Image](https://github.com/user-attachments/assets/d9b3f1fd-1b7a-4616-bcaa-3eac6490ee86)

> Still not working after upgrate to `vllm==0.7.3`
> 
> ![Image](https://github.com/user-attachments/assets/d9b3f1fd-1b7a-4616-bcaa-3eac6490ee86)

Can you try specifying the huggingface ID (e.g, `Qwen/Qwen2.5-VL-3B-Instruct`) directly?

> > Still not working after upgrate to `vllm==0.7.3`
> >
> 
> Can you try specifying the huggingface ID (e.g, `Qwen/Qwen2.5-VL-3B-Instruct`) directly?

Unfortunately, I can't.. my GPU cluster do not have access to huggingface.. but I've the latest version of the model stored in this path.. . 


Updates:

I managed to use the mentioned "weird" method to solve the case by using:

pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40

Do you know why it can be solved? Transformers 4.49.0dev version is helpful. Here, the version 4.49.0 dev can support Qwen2.5-VL-7B, but "pip install transformers --upgrade" to get 4.49.0 version does not help.

> Hey all! The issue is being fixed on vllm side with [vllm-project/vllm#13592](https://github.com/vllm-project/vllm/pull/13592) afaik, the team will check compatibility with v4.49 release
> 
> cc [@Isotr0py](https://github.com/Isotr0py)

GOOD NEWS

@usun1997 , hello , it still doesn't work, could u share your version?
my library version:
transformers                      4.49.0.dev0
transformers-stream-generator     0.0.5
triton                            3.1.0
trl                               0.15.1
typeguard                         4.4.1
typer                             0.15.1
typing_extensions                 4.12.2
tyro                              0.9.13
tzdata                            2025.1
uc-micro-py                       1.0.3
urllib3                           2.3.0
uvicorn                           0.34.0
uvloop                            0.21.0
virtualenv                        20.29.1
vllm                              0.7.3
watchfiles                        1.0.4
websockets                        14.2
Werkzeug                          3.1.3
wheel                             0.45.1
wrapt                             1.17.2
xattr                             1.1.4
xformers                          0.0.28.post3
xgrammar                          0.1.11
xxhash                            3.5.0
yarl                              1.18.3
zipp                              3.21.0
zstandard                         0.23.0

Check your config.json file, and change the processor name in preprocessor_config.json

"image_processor_type": "Qwen2VLImageProcessor",

This should work for most cases if you have latest transformers

> pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40

still not working for me.. Same error: ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details.

Has it been fixed? I still have the error.

@gouqi666 we meet again :) , have you found a fix for this issue?


 

> Has it been fixed? I still have the error.

Still not working for me...

haha,&nbsp; change&nbsp; Qwen2_5_VL to Qwen2_VL in the config.json



---Original---
From: "Kolesh ***@***.***&gt;
Date: Fri, Feb 28, 2025 00:04 AM
To: ***@***.***&gt;;
Cc: ***@***.******@***.***&gt;;
Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292)


   
@gouqi666 we meet again :) , have you found a fix for this issue?

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;
  koleshjr left a comment (huggingface/transformers#36292)
 
@gouqi666 we meet again :) , have you found a fix for this issue?
 
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;

> haha,&nbsp; change&nbsp; Qwen2_5_VL to Qwen2_VL in the config.json
> […](#)

I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b:

        <html>
        <body>
        <!--StartFragment-->
          | {
        -- | --
          | "min_pixels": 3136,
          | "max_pixels": 12845056,
          | "patch_size": 14,
          | "temporal_patch_size": 2,
          | "merge_size": 2,
          | "image_mean": [
          | 0.48145466,
          | 0.4578275,
          | 0.40821073
          | ],
          | "image_std": [
          | 0.26862954,
          | 0.26130258,
          | 0.27577711
          | ],
          | "image_processor_type": "Qwen2VLImageProcessor",
          | "processor_class": "Qwen2_5_VLProcessor"
          | }
        
        <!--EndFragment-->
        </body>
        </html>
        
but still I am facing this issue

have you updatef your local directory ？



---Original---
From: "Kolesh ***@***.***&gt;
Date: Fri, Feb 28, 2025 19:03 PM
To: ***@***.***&gt;;
Cc: ***@***.******@***.***&gt;;
Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292)


    
haha,&nbsp; change&nbsp; Qwen2_5_VL to Qwen2_VL in the config.json
 …
  
I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b:
     <html&gt;     <body&gt;     <!--StartFragment--&gt;     &nbsp; | {     -- | --     &nbsp; | "min_pixels": 3136,     &nbsp; | "max_pixels": 12845056,     &nbsp; | "patch_size": 14,     &nbsp; | "temporal_patch_size": 2,     &nbsp; | "merge_size": 2,     &nbsp; | "image_mean": [     &nbsp; | 0.48145466,     &nbsp; | 0.4578275,     &nbsp; | 0.40821073     &nbsp; | ],     &nbsp; | "image_std": [     &nbsp; | 0.26862954,     &nbsp; | 0.26130258,     &nbsp; | 0.27577711     &nbsp; | ],     &nbsp; | "image_processor_type": "Qwen2VLImageProcessor",     &nbsp; | "processor_class": "Qwen2_5_VLProcessor"     &nbsp; | }          <!--EndFragment--&gt;     </body&gt;     </html&gt;  
but still I am facing this issue

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;
  koleshjr left a comment (huggingface/transformers#36292)
  
haha,&nbsp; change&nbsp; Qwen2_5_VL to Qwen2_VL in the config.json
 …
  
I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b:
     <html&gt;     <body&gt;     <!--StartFragment--&gt;     &nbsp; | {     -- | --     &nbsp; | "min_pixels": 3136,     &nbsp; | "max_pixels": 12845056,     &nbsp; | "patch_size": 14,     &nbsp; | "temporal_patch_size": 2,     &nbsp; | "merge_size": 2,     &nbsp; | "image_mean": [     &nbsp; | 0.48145466,     &nbsp; | 0.4578275,     &nbsp; | 0.40821073     &nbsp; | ],     &nbsp; | "image_std": [     &nbsp; | 0.26862954,     &nbsp; | 0.26130258,     &nbsp; | 0.27577711     &nbsp; | ],     &nbsp; | "image_processor_type": "Qwen2VLImageProcessor",     &nbsp; | "processor_class": "Qwen2_5_VLProcessor"     &nbsp; | }          <!--EndFragment--&gt;     </body&gt;     </html&gt;  
but still I am facing this issue
 
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;

> have you updatef your local directory ？
> […](#)

Yes... same error.. even after updating the model localy.

change the transformer version&nbsp; to 4.49 and vllm to 0.7.3



---Original---
From: "Igor L ***@***.***&gt;
Date: Sat, Mar 1, 2025 05:05 AM
To: ***@***.***&gt;;
Cc: ***@***.******@***.***&gt;;
Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292)


    
have you updatef your local directory ？
 …
  
Yes... same error.. even after updating the model localy.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;
  igorpereirabr1 left a comment (huggingface/transformers#36292)
  
have you updatef your local directory ？
 …
  
Yes... same error.. even after updating the model localy.
 
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***&gt;

same error with vll==0.7.3 and transformers==4.49.0

Try using python 3.12. I had the same issue with 3.11 but now everything works fine.

> same error with vll==0.7.3 and transformers==4.49.0

I had solved this problem, It's my source code error，but  vllm send a puzzling message

> but vllm send a puzzling message

@philipwan what message is it? I can take it back to see if we can update on the vllm side so that we don't confuse users!

> > but vllm send a puzzling message
> 
> [@philipwan](https://github.com/philipwan) what message is it? I can take it back to see if we can update on the vllm side so that we don't confuse users!

"ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details."
These is  error message,  real reason for the error stored in vllm.log.  When I was a beginner, I mistakenly thought that all the error messages displayed on the screen were

I'm having the same problem with:

```
transformers 4.49.0.dev0
vllm 0.8.2
```
`python3.12.9`

downloaded the latest model from HF from today

`vllm serve /mnt/llms/models/Qwen2.5-VL-32B-Instruct --max-model-len 36000 --host 0.0.0.0 --port 5000 -tp 4`


```
Traceback (most recent call last):
  File "/home/ubuntuai/vllm/.venv/bin/vllm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 75, in main
    args.dispatch_function(args)
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 33, in cmd
    uvloop.run(run_server(args))
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 109, in run
    return __asyncio.run(
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
           ^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1016, in run_server
    async with build_async_engine_client(args) as engine_client:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 141, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 161, in build_async_engine_client_from_engine_args
    vllm_config = engine_args.create_engine_config(usage_context=usage_context)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1296, in create_engine_config
    model_config = self.create_model_config()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1141, in create_model_config
    return ModelConfig(
           ^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/config.py", line 399, in __init__
    self.multimodal_config = self._init_multimodal_config(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/config.py", line 467, in _init_multimodal_config
    if self.registry.is_multimodal_model(self.architectures):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 482, in is_multimodal_model
    model_cls, _ = self.inspect_model_cls(architectures)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 442, in inspect_model_cls
    return self._raise_for_unsupported(architectures)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 392, in _raise_for_unsupported
    raise ValueError(
ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details
```


> I'm having the same problem with:
> 
> ```
> transformers 4.49.0.dev0
> vllm 0.8.2
> ```
> 
> ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details

for me
> transformers 4.50.3
> vllm 0.8.2
>ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details

it works, but don't know the reason, damn

>from gevent import monkey
>monkey.patch_all()


I face the same error. For me, transformers  4.57.1 and vllm 0.7.3 wrok finally.