sid1hant
/

tokenizer_for_python_code

tokenizer_for_python_code / special_tokens

Upload special_tokens

75134cf almost 4 years ago

90 Bytes

{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}