Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
ZhankuiHe
/
multi_tokenizer
like
0
Model card
Files
Files and versions
Community
main
multi_tokenizer
/
tokenizer_config.json
ZhankuiHe
Upload tokenizer
007c018
verified
4 months ago
raw
history
blame
contribute
delete
No virus
124 Bytes
{
"tokenizers"
:
{
"nltk"
:
"NLTKTokenizer"
,
"entity"
:
"EntityTokenizer"
}
,
"tokenizer_key_for_decode"
:
"nltk"
}