evo-1-131k-base / tokenizer_config.json
Zymrael's picture
Create tokenizer_config.json
2c3da52 verified
raw
history blame
143 Bytes
{
"unk_token_id": 0,
"bos_token_id": 2,
"eos_token_id": 0,
"pad_token_id": 1,
"mask_token_id": 3,
"byte_level": true
}