GuaPeTe-2-tiny / tokenizer_config.json
system's picture
system HF staff
Update tokenizer_config.json
705ace6
raw history blame
No virus
107 Bytes
{"max_len": 1024, "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}