2chan_ruGPT3_small / tokenizer_config.json
TheBakerCat
Firs version of the 2chanruGPT-small model
d87ed6a
{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}