bigscience

/

bloom

like 4.55k

Are you using greedy algorithm? If yes it's expected that outputs are the same. Try using some sampling:

{"do_sample": true, ...}

Are you using greedy algorithm? If yes it's expected that outputs are the same. Try using some sampling:

{"do_sample": true, ...}

Im using python btw. Is this what it should look like? query({"inputs": ctx+promt, "do_sample": True})
Because it still gives me the same outputs for the same promts...

Where can I find a documentation for detailed parameters for this API @TimeRobber

I think you need to send {"input": blablabla, "parameters": {"do_sample": True}}

The documentation is available here: https://huggingface.co/docs/api-inference/detailed_parameters#text-generation-task . Though this is a custom inference setup and it doesn't provide all args right now. Not sure, but I believe this are the parameters we support. To be confirmed by @olivierdehaene

Yes, you need to provide the do_sample parameter as @TimeRobber explained.

This endpoint only supports:

• temperature
• topK
• topP
• do_sample
• max_new_tokens