wilsontam's picture
initial commit
65e2b86
raw
history blame
253 Bytes
{"bos_token": "<bos>", "eos_token": "<eos>", "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "<pad>", "cls_token": "[CLS]", "mask_token": "[MASK]", "additional_special_tokens": ["<speaker1>", "<speaker2>", "<knowledge_sep>", "<knowledge_tag>"]}