fasttext-jp-embedding / tokenizer_config.json
Taizo Kaneko
commit files to HF hub
76b4794
{
"auto_map": {
"AutoTokenizer": [
"fasttext_jp_tokenizer.FastTextJpTokenizer",
null
]
},
"tokenizer_class": "FastTextJpTokenizer"
}