paddle-canine-s / tokenizer_config.json

update canine paddle model, commit from kevin

0ca3df3 almost 2 years ago

No virus

188 Bytes

{"bos_token": "", "pad_token": "\u0000", "eos_token": "", "cls_token": "", "sep_token": "", "mask_token": "", "model_max_length": 2048, "tokenizer_class": "CanineTokenizer"}