web2savar's picture
Upload tokenizer
6d8ec80 verified
raw
history blame contribute delete
No virus
131 Bytes
{
"[PAD]": 10,
"[UNK]": 9,
"|": 0,
"ç": 1,
"ö": 2,
"ü": 3,
"ğ": 4,
"ı": 5,
"ş": 6,
"̇": 7,
"…": 8
}