c2m / tokenizer_config.json
supermy's picture
Upload tokenizer
7a62b64
raw
history blame
126 Bytes
{
"add_special_token": false,
"lowercase": true,
"pad_token": "[PAD]",
"tokenizer_class": "PreTrainedTokenizerFast"
}