Is Bloom API inference point still working?
#204
by
nicholasKluge
- opened
I have been using the Bloom API inference for a while, but recently, my code just crashed (and it only crashes with Bloom, other models work just fine, like gpt-2 or distillgpt2). Here is my code to reproduce the error.
import json
import requests
API_KEY = <your_api_key_here>
def query(payload):
data = json.dumps(payload)
response = requests.request("POST",
"https://api-inference.huggingface.co/models/bigscience/bloom",
headers={
"Authorization": f"Bearer {os.environ.get('API_KEY')}"},
data=data)
return json.loads(response.content.decode("utf-8"))
response = query({"inputs": 'Help me hugging face community!',
"parameters": {"top_k": 10, "temperature": 0.3,
"max_new_tokens": 250,
"num_return_sequences": 1,
"repetition_penalty": 1.8,
'max_time': 30,
'do_sample': True,
'return_full_text': False},
"options": {'use_cache': False,
'wait_for_model': True}})
print(response[0]['generated_text'])
My output is an error:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What is wrong with my JSON request? It works fine with other models...
If you can help me, thank you in advance!
Hello! Try this:
import requests
API_URL = "https://api-inference.huggingface.co/models/bigscience/bloom"
headers = {"Authorization": "Bearer your_api_key_here"}
def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()
output = query({
"inputs": "The weather today is",
})
print(output)
Completely solved! Thank for your help ymoslem!
cakiki
changed discussion status to
closed