Before revision:  Although top-k sampling and nucleus sampling outperform beam search based decoding methods , they only focus on truncating the  "tail" of the distribution  and do not address the "head" part, which we show might contain tedious or even repetitive candidates with high probability that lead to repetition loops.
After revision:  Although top-k sampling and nucleus sampling outperform beam search based decoding methods , they only focus on truncating the unreliable "tail" of the distribution and do not address the "head" part, which we show might contain tedious or even repetitive candidates with high probability that lead to repetition loops.