bigscience

/

bloom

like 4.6k

Hi there!

You should switch to greedy sampling as opposed to nucleus sampling, as can be read in the notice in that screenshot you shared:

Switch to "greedy" for more accurate completion e.g. math/history/translations (but which may be repetitive/less inventive)

It's also usually advisable to use longer contexts.

How can we use "greedy" method when generating text using pipeline ?

@Neeraj17 You can pass parameters to the generate function as keyword arguments to your pipeline call. For greedy decoding, you would have to pass do_sample=False.

Bruh it has to have some context what to do... Just at least give it a few examples.

Yeah to be honest our deployment has been unstable due to us having a customised version in order to run fast. I think right now it's much stable.

The idea of providing a few examples on how to solve the task is to be clear on what you want the model to output.

To find why it did something wrong then just imagine you read the dataset and the task is to you... You didn't see a single file that starts with math... And the model is like "idk what do you want from me"

Btw in the first example it thought it was some stackoverflow code.