HuggingFaceM4

/

idefics2-8b

like 589

Follow

HuggingFaceM4 278

Having issues sending a locally-hosted image to TGI while hosting idefics2:

from huggingface_hub import InferenceClient

client = InferenceClient(model="http://localhost:8080")

# System prompt used in the playground for `idefics2-8b-chatty`
SYSTEM_PROMPT = "System: The following is a conversation between Idefics2, a highly knowledgeable and intelligent visual AI assistant created by Hugging Face, referred to as Assistant, and a human user called User. In the following interactions, User and Assistant will converse in natural language, and Assistant will do its best to answer User’s questions. Assistant has the ability to perceive images and reason about them, but it cannot generate images. Assistant was built to be respectful, polite and inclusive. It knows a lot, and always tells the truth. When prompted with an image, it does not make up facts.<end_of_utterance>\nAssistant: Hello, I'm Idefics2, Huggingface's latest multimodal assistant. How can I help you?<end_of_utterance>\n"
QUERY = "![](file:///test.jpg)Describe this image.<end_of_utterance>\nAssistant:"

generation_args = {
    "max_new_tokens": 512,
    "repetition_penalty": 1.1,
    "do_sample": False,
}
generated_text = client.text_generation(prompt=SYSTEM_PROMPT + QUERY, **generation_args)
generated_text

The code above produces the following error:

HTTPError                                 Traceback (most recent call last)
File ~/miniconda3/envs/finetuning/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py:304, in hf_raise_for_status(response, endpoint_name)
    303 try:
--> 304     response.raise_for_status()
    305 except HTTPError as e:

File ~/miniconda3/envs/finetuning/lib/python3.10/site-packages/requests/models.py:1024, in Response.raise_for_status(self)
   1023 if http_error_msg:
-> 1024     raise HTTPError(http_error_msg, response=self)

HTTPError: 422 Client Error: Unprocessable Entity for url: http://localhost:8080/

The above exception was the direct cause of the following exception:

HfHubHTTPError                            Traceback (most recent call last)
File ~/miniconda3/envs/finetuning/lib/python3.10/site-packages/huggingface_hub/inference/_client.py:2031, in InferenceClient.text_generation(self, prompt, details, stream, model, best_of, decoder_input_details, do_sample, frequency_penalty, grammar, max_new_tokens, repetition_penalty, return_full_text, seed, stop_sequences, temperature, top_k, top_n_tokens, top_p, truncate, typical_p, watermark)
   2030 try:
-> 2031     bytes_output = self.post(json=payload, model=model, task="text-generation", stream=stream)  # type: ignore
   2032 except HTTPError as e:

File ~/miniconda3/envs/finetuning/lib/python3.10/site-packages/huggingface_hub/inference/_client.py:273, in InferenceClient.post(self, json, data, model, task, stream)
    272 try:
--> 273     hf_raise_for_status(response)
    274     return response.iter_lines() if stream else response.content
...
--> 457     raise exception from http_error
    459 # Otherwise, fallback to default error
    460 raise http_error

ValidationError: Input validation error: invalid image content: ![](file:///test.jpg)

I have also tried using absolute paths, same end result.