What special token did you folks us for padding at the end? <fim_pad> seems to be used for fill in the middle.
#8
by
compressllama
- opened
I am using batch inference which needs padding at the end of some inputs due to different sequence lengths, I was going through the special tokens in the config file but i could not figure out the padding token
Following up, can you recommend a token i can use for padding?