trin_tokenizer_v3 / config.json
naclbit's picture
Added instruct tokens and config fix
c23f0b5
raw
history blame
249 Bytes
{
"attn_pdrop": 0.0,
"pad_token_id": 0,
"unk_token_id": 1,
"bos_token_id": 2,
"sep_token_id": 3,
"eos_token_id": 4,
"embd_pdrop": 0.0,
"transformers_version": "4.10.0.dev0",
"tokenizer_class": "T5Tokenizer",
"vocab_size": 52100
}