custom_code / added_tokens.json
facebook-llama's picture
Upload 9 files
33c6fa0
raw
history blame contribute delete
223 Bytes
{
"<|endoftext|>": 50000,
"[SEP]": 50001,
"[CLS]": 50002,
"[MASK]": 50003,
"[UNUSED1]": 50004,
"[UNUSED2]": 50005,
"<|startofpiece|>": 50006,
"<|endofpiece|>": 50007,
"[sMASK]": 50008,
"[gMASK]": 50009
}