Warning: The attention mask and the pad token id were not set..
Hi, when I infer the llama3-8b-instruct, there is an warning:
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
Although it doesn't impact the result, I still want to fix the warning. Any idea?
Thanks for answering.
when calling model.generate, setting pad_token_id=tokenizer.eos_token_id seems to remove the warning.
it seems to work fine still with that, although some explanation and reasoning wouldn't hurt. maybe i saw one when searching for the solution but i forget.. :)
model.generate(**encoded_input, pad_token_id=tokenizer.eos_token_id).just change this will be ok.
Hey there, i was also facing the same error and after tinkering with the tokenizer for a bit i found out that inputs = tokenizer.encode(prompt) returns just the input_ids but not the attention mask, Whereas inputs = tokenizer(prompt) returns both the input_ids and the attention mask.
So if you replace this code
inputs = tokenizer.encode(prompt)
output = model.generate(inputs)
With this
inputs = tokenizer(prompt)
output = model.generate(**inputs)
The error warning goes away. Hope that helps 😊.