would be great to have the example code for the attention mask and the pad token id

#2
by Tonic - opened

hi there, big fan of stable lm since the alpha :-)

this one seems great, but i'm getting warnings about the attention mask and the pad token id :-)

will dig in as always but it would be a quick fix on the example code.

congrats on the cool release ! πŸš€

hi there, big fan of stable lm since the alpha :-)

this one seems great, but i'm getting warnings about the attention mask and the pad token id :-)

will dig in as always but it would be a quick fix on the example code.

congrats on the cool release ! πŸš€

Adding pad_token_id=tokenizer.pad_token_id, fixes it:
tokens = model.generate(
inputs.to(model.device),
pad_token_id=tokenizer.pad_token_id,
max_new_tokens=1024,
temperature=0.8,
do_sample=True

(Tried with transformers version 4.35.2)

Sign up or log in to comment