Tokenizer __call__ does not truncate
#2
by
Muennighoff
- opened
Tokenizer call does not truncate when passing truncation=True & max_length
Tokenizer call does not truncate when passing truncation=True & max_length