Add Fill-in-Middle specific tokens
#26
by
tanaymehta
- opened
Request for adding FIM-specific tokens (prefix, middle and suffix) as discussed in the PR for adding FIM pipeline to transformers (https://github.com/huggingface/transformers/pull/27828#issuecomment-1970847921)
cc: @ArthurZ