Why is `beam_search` not enabled in the samples?

#10
by floschne - opened

In the examples, when calling the generate function, beam search is disabled by providing do_sample=False. What is the reason behind this?

Llava Hugging Face org

Hi, passing do_sample = False disables top-k sampling, not beam search. See this blog post which explains all the details. Beam search can be enabled by passing the num_beams argument.

However, looking at the original code, it looks like they use sampling with a temperature of 0.2. cc @ybelkada

Llava Hugging Face org

Hi @floschne !
I second what niels said, you can do it manually for Llava as it should work out of the box! We just set it to False in order to get reproducible results in the demo

Sign up or log in to comment