megagonlabs
/

transformers-ud-japanese-electra-base-discriminator

Inference Endpoints

Model card Files Files and versions Community

transformers-ud-japanese-electra-base-discriminator / tokenizer_config.json

hiroshi-matsuda-rit's picture

hiroshi-matsuda-rit

add model files

2b4c5ad almost 3 years ago

raw history blame

No virus

222 Bytes

	{
	"do_lower_case": false,
	"word_tokenizer_type": "sudachipy",
	"word_form_type": "surface",
	"subword_tokenizer_type": "wordpiece",
	"model_max_length": 512,
	"sudachipy_kwargs": {"split_mode":"A","dict_type":"core"}
	}