screeve-pos-lemmatizer / added_tokens.json
Nargizi's picture
add tokenizer
c1abbf6
{
"<POS>": 2010,
"A": 2005,
"Adv": 2006,
"Cj": 2000,
"Interj": 2008,
"N": 2004,
"Num": 2002,
"Other": 2007,
"Pp": 2009,
"Pron": 2003,
"V": 2001
}