voidful
/

bert-base-uncased-phone-bpe

bert-base-uncased-phone-bpe / tokenizer_config.json

add tokenizer

bf2d7a0 over 1 year ago

No virus

322 Bytes

	{
	"bos_token": "<s>",
	"do_lower_case": false,
	"eos_token": "</s>",
	"name_or_path": "./bert-base-uncased-phone-bpe",
	"pad_token": "[PAD]",
	"replace_word_delimiter_char": " ",
	"special_tokens_map_file": null,
	"tokenizer_class": "Wav2Vec2CTCTokenizer",
	"unk_token": "[UNK]",
	"word_delimiter_token": "\|"
	}