llava-hf

/

llava-v1.6-mistral-7b-hf

like 231

Follow

Llava Hugging Face 109

TypeError Traceback (most recent call last)
in <cell line: 6>()
4 import requests
5
----> 6 processor = LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf")
7
8 model = LlavaNextForConditionalGeneration.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", torch_dtype=torch.float16, low_cpu_mem_usage=True)

1 frames
/usr/local/lib/python3.10/dist-packages/transformers/processing_utils.py in from_args_and_dict(cls, args, processor_dict, **kwargs)
391 del processor_dict["auto_map"]
392
--> 393 processor = cls(*args, **processor_dict)
394
395 # Update processor with kwargs if needed

TypeError: LlavaNextProcessor.init() got an unexpected keyword argument 'chat_template'

from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
import torch
from PIL import Image
import requests

processor = LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf")

model = LlavaNextForConditionalGeneration.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", torch_dtype=torch.float16, low_cpu_mem_usage=True) 
model.to("cuda:0")

# prepare image and text prompt, using the appropriate prompt template
url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
image = Image.open(requests.get(url, stream=True).raw)
prompt = "[INST] <image>\nWhat is shown in this image? [/INST]"

inputs = processor(prompt, image, return_tensors="pt").to("cuda:0")

# autoregressively complete prompt
output = model.generate(**inputs, max_new_tokens=100)

print(processor.decode(output[0], skip_special_tokens=True))

Update 1: transformers version 4.41.2