aliosm
/

ComVE-gpt2-medium

ComVE-gpt2-medium / tokenizer_config.json

system HF staff

Update tokenizer_config.json

a9bd994 almost 4 years ago

No virus

107 Bytes

{"max_len": 1024, "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}