File size: 253 Bytes
6f2fb4b |
1 |
{"bos_token": "<bos>", "eos_token": "<eos>", "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "<pad>", "cls_token": "[CLS]", "mask_token": "[MASK]", "additional_special_tokens": ["<speaker1>", "<speaker2>", "<knowledge_sep>", "<knowledge_tag>"]} |