Attention mask and pad token id warning

#28
by nottakumasato - opened

Hi,

I have been getting the below warning message when running inference. How can I fix it editing the example code in the docs?

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.

Hello

I don't know if you still need help, but if you do you can suppress the warnings by setting pad_token_id to eos_token_id in the generate method from the example code.
For example, you can update the example code with

generated_ids = model.generate(model_inputs, pad_token_id=tokenizer.eos_token_id, max_new_tokens=1000, do_sample=True).

Hope this helps:)

Sign up or log in to comment