clip-huge-zh-75k-steps-bs4096 / tokenizer_config.json
lyua1225's picture
Upload tokenizer_config.json
24b1bc5
raw
history blame
284 Bytes
{
"cls_token": "[CLS]",
"mask_token": "[MASK]",
"unk_token": "[UNK]",
"tokenizer_class": "CLIPTokenizerRoberta",
"model_max_length": 77,
"use_fast": true,
"auto_map": {
"AutoTokenizer": [
"clip_tokenizer_roberta.CLIPTokenizerRoberta",
null
]
}
}