GerbilBlender-A-104m / added_tokens.json
crumb's picture
Upload tokenizer
f071f2c
{
"<causal>": 50260,
"<fitm_result>": 50259,
"<fitm_start>": 50257,
"<mlm_end>": 50263,
"<mlm_start>": 50261,
"<multiple_tok_mask>": 50258,
"<single_tok_mask>": 50262
}