Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Qwen
/
Qwen-tokenizer
like
13
License:
tongyi-qianwen-license (other)
Model card
Files
Files and versions
Community
1
main
Qwen-tokenizer
/
added_tokens.json
JustinLin610
Upload 6 files
5909c82
verified
8 months ago
raw
Copy download link
history
blame
contribute
delete
No virus
80 Bytes
{
"<|endoftext|>"
:
151643
,
"<|im_end|>"
:
151645
,
"<|im_start|>"
:
151644
}