mistralai

/

Mixtral-8x7B-Instruct-v0.1

like 3.82k

Has anyone faced the following issue while batch decoding -
I am using Mistral 8x7B v0.1 with 4bit quantization - I have initialized the tokenizer as
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast = False, padding_side = "left", add_eos_token = True, add_bos_token = True)
tokenizer.pad_token = tokenizer.eos_token
But, I still get the following warning message -
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.