Why is `beam_search` not enabled in the samples?
#10
by
floschne
- opened
In the examples, when calling the generate
function, beam search is disabled by providing do_sample=False
. What is the reason behind this?
Hi, passing do_sample
= False disables top-k sampling, not beam search. See this blog post which explains all the details. Beam search can be enabled by passing the num_beams
argument.
However, looking at the original code, it looks like they use sampling with a temperature of 0.2. cc @ybelkada