HuggingFaceH4

/

mistral-7b-sft-beta

like 23

As explained here: https://github.com/huggingface/transformers/issues/23530 and here: https://github.com/huggingface/alignment-handbook/issues/127 when the model doesn't have a PAD token, the EOS one is repurposed within the alignment-handbook.

This is fine in terms of padding, but it leads to tokens in the chat template to be considered padding at train time, and therefore their label set to -100, and therefore not being backpropagated. Still, this model (and the DPO one) seem able to generate tokens at the end of each conversation turn.

Was there something done to address this before training it? Following the current recipes do not seem to address this issue.