cannot import name 'Qwen2_5_VLImageProcessor' (on vLLM)
I'm trying to deploy the model on kubernetes using vllm.
I'm receiving an error when deploying, which seems related to the absence of references in code.cannot import name 'Qwen2_5_VLImageProcessor' from 'transformers.models.qwen2_5_vl' (/prod_venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/__init__.py).
I did all the updates suggested by documentation in the docker file. I even confronted locally and git and the code is the same.
RUN apt-get update && apt-get install -y git
RUN pip install vllm --upgrade
RUN pip install flashinfer-python -i https://flashinfer.ai/whl/cu124/torch2.5/
# Install the model libraries
RUN pip uninstall -y transformers
RUN pip install git+https://github.com/huggingface/transformers accelerate
Any suggestion about how to fix it?
transformers-cli env
transformersversion: 4.49.0.dev0- Platform: Linux-5.15.0-113-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.28.1
- Safetensors version: 0.5.2
- Accelerate version: 0.32.1
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?:
- Using GPU in script?:
- GPU type: NVIDIA A100-SXM4-40GB
Transformers dev has changed. Please update vLLM and also your local version of the HF Hub repo.
https://github.com/vllm-project/vllm/pull/12604#issuecomment-2661841629
@wulipc
I tried to rebuild the image this morning (later than the comment in the issue) receiving the same result.
New versions should be updated with pip install vllm --upgrade and pip install git+https://github.com/huggingface/transformers.
As you can see in the transformers-cli env command results, I'm using the latest transformers on Git and the latest Huggingface_hub version on pip.
Am I still missing something?
@wulipc I tried to rebuild the image this morning (later than the comment in the issue) receiving the same result.
New versions should be updated withpip install vllm --upgradeandpip install git+https://github.com/huggingface/transformers.As you can see in the
transformers-cli envcommand results, I'm using the latest transformers on Git and the latest Huggingface_hub version on pip.
Am I still missing something?
@cbrug Currently, the latest version of VLLM in PIP is 0.7.2, which does not include this fix. You need to compile and install the latest VLLM from the main branch. Or you can downgrade the transformer version, which is easier to do;
@cbrug Currently, the latest version of VLLM in PIP is 0.7.2, which does not include this fix. You need to compile and install the latest VLLM from the main branch. Or you can downgrade the transformer version, which is easier to do;
@wulipc
thanks for the feedback.
I tried to update vllm on docker file, but I'm afraid is too heavy.
It requested me to also install cuda toolkit. Whithout it there were errors about the missing parts.
When during the build it tries to install vllm, at a certain point (after at least 15-20 minutes on that step) it freezes and then fails (I use a VM to build the docker files, even with 300G of storage, 32 GB of RAM and 8 vCPU it fails).
There is another way to update?
RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
RUN dpkg -i cuda-keyring_1.1-1_all.deb
RUN apt-get update
RUN apt-get -y install cuda-toolkit-12-8
ENV CUDA_HOME /usr/local/cuda
ENV PATH "${CUDA_HOME}/bin:$PATH"
# RUN pip install git+https://github.com/vllm-project/vllm # No differences between this option or the one below
RUN git clone https://github.com/vllm-project/vllm.git
RUN cd vllm && pip install -e . # Step that blocks everything