Spaces:

ysharma

/

Explore_llamav2_with_TGI

like 1.2k

Running on CPU Upgrade

The person hosting this (ysharma) has an inference endpoint setuped. The hf_token is his huggingface token for that endpoint and api_url is the inference endpoint url.

hi @DhruvShek

Could I know if deploying an inference endpoint on-premise is free?

docker run -it -p 7860:7860 --platform=linux/amd64 \
    -e HF_TOKEN="YOUR_VALUE_HERE" \
    -e API_URL="YOUR_VALUE_HERE" \
    registry.hf.space/ysharma-explore-llamav2-with-tgi:latest python app.py

After I created this container, was there any server (endpoint) inside? Do I still need to deploy the server in other places individually?

Thank you

I did input the API_URL and the HF_TOKEN, but seem not working…. How I can troubleshot?

I did test with the cURL with the API_URL its working..

Hello @chieh
The inference endpoint has to be deployed on Hugging Face, and it's not free.
You will require to pay hourly pricings for inference endpoint.

When you create this container, there isn't any inference endpoint inside. You need to deploy it from Hugging Face.

I see!! Thanks for your information!