Creating a handler.py file to support HF dedicated inference endpoints

#18

Hi all,

Made a quick handler.py file for people to use. It works for me pretty well. Feel free to test it out :)

Here is an example for when making a request to the endpoint:

import requests
import base64
import json

api_url = 'your API URL'
image_path = '' # Replace with the path to your image file
question = 'what is going on in this picture?'

with open(image_path, 'rb') as img:
encoded_image = base64.b64encode(img.read()).decode('utf-8')

payload = {
'inputs': {
'image': encoded_image,
'question': question
}
}

headers = {
"Accept" : "application/json",
"Authorization": "Bearer HUGGINGFACE_TOKEN",
"Content-Type": "application/json"
}

response = requests.post(api_url, headers=headers, data=json.dumps(payload))

print(response.json().get('body', {}))

Thanks! Is this the dedicated endpoints feature? https://huggingface.co/inference-endpoints/dedicated

Looks like it's pointing to your finetune btw

Yes, that's right! It's the dedicated endpoints feature for the HF platform. Also, sorry about that, I'll make sure to remove the fine-tuned model ;)

🫡

vikhyatk changed pull request status to merged

Sign up or log in to comment