ru-word-stress-transformer / tokenizer_config.json

configs

f363a3b about 2 years ago

210 Bytes

	{
	"bos_token": "[BOS]",
	"eos_token": "[EOS]",
	"pad_token": "[PAD]",
	"tokenizer_class": "CharTokenizer",
	"unk_token": "[UNK]",
	"auto_map": {
	"AutoTokenizer": "char_tokenizer.CharTokenizer"
	}
	}