temp-kgsf-rec / tokenizer_config.json
ZhankuiHe's picture
Upload tokenizer
95d813f verified
raw
history blame contribute delete
No virus
164 Bytes
{
"tokenizers": {
"item": "EntityTokenizer",
"entity": "KGSFBaseTokenizer",
"concept": "KGSFBaseTokenizer"
},
"tokenizer_key_for_decode": "item"
}