Tokenizer should use padding-side left.
#3
by
alexjc
- opened
Error when using batch_size > 1:
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.
I think the Tokenizer settings are incorrect, can be fixed in code but it's a hassle...
Should be fixed now.
blair-johnson
changed discussion status to
closed