stas
/

tiny-wmt19-en-ru

tiny-wmt19-en-ru / tokenizer_config.json

HF staff

Update tokenizer_config.json

8e14c34 over 2 years ago

No virus

108 Bytes

{"langs": ["en", "ru"], "unk_token": "<unk>", "bos_token": "<s>", "sep_token": "</s>", "pad_token": "<pad>"}