Is there a way to batch decode?
#1
by
AdrSkapars
- opened
Batch encoding seems possible but batch decoding/ generating doesnt. You can batch generate with decode only models like gpt2 and llama but they dont have this perturbed_vector variable.